diff --git a/CLAUDE.md b/CLAUDE.md index 3f8319e..c1de24c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -230,6 +230,7 @@ until it was dissolved on 2026-09-04. | [`docs/work/mvp/2026-09-04-open-items-dissolution.md`](docs/work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record, not a register.** The dissolved open-items register, moved here whole on 2026-09-04 with every open question in it decided. Nothing is appended to it; its item IDs stay reserved and still resolve, because they are cited from source. | no — archive | | `docs/first-release.md` | **Live.** Release-readiness: what the release path already does, and the blockers that must clear before a first publish. Was the `NFR-16` row of the dissolved deferral register. | yes | | `docs/deviations.md` | **Register.** The as-built audit of §10, and where a deviation found outside a phase lands. | yes | +| `docs/audit-67-decisions.md` | **Ledger.** Decisions taken during the audit #67 remediation run, and the release-machinery work it deferred. | yes | | `docs/assets/` | Vendored wordmark SVGs the root `README.md` renders. | yes | **Frozen means a maintenance tool refuses to write there**, not merely that you should not. The diff --git a/docs/README.md b/docs/README.md index 659726f..6b9507d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,6 +16,7 @@ is the index; the rule is that nothing in `docs/` is unowned, and nothing is wri | [the dissolved open-items register](./work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record.** Everything the register held when it was dissolved on 2026-09-04, with every open question in it decided. Item IDs stay reserved and still resolve | — | No — nothing is appended | | [`first-release.md`](./first-release.md) | Release readiness: what the release path already does, the confirmed mechanics, and the blockers that must clear before a first publish. Was the `NFR-16` row of `deferred-items.md`, which was dissolved on 2026-09-04 — its five still-live rows are archived under *Live deferrals* in [`work/mvp/2026-09-04-register-retirement-purge.md`](./work/mvp/2026-09-04-register-retirement-purge.md) | The maintainer | Yes — edited as blockers clear | | [`deviations.md`](./deviations.md) | The as-built audit of §10, and the landing point for a deviation found outside a phase | An audit or review | Yes — appends | +| [`audit-67-decisions.md`](./audit-67-decisions.md) | Decision ledger for the audit #67 remediation run: cross-task decisions, rejected alternatives, and the release-machinery work deferred from it | The remediation supervisor | Yes — appends | | [`assets/`](./assets/) | Vendored wordmark SVGs the root `README.md` renders | Copied from `dexpace/morphic` | Yes | ## Frozen means frozen diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md new file mode 100644 index 0000000..94c08f8 --- /dev/null +++ b/docs/audit-67-decisions.md @@ -0,0 +1,665 @@ +# Audit #67 remediation — decision ledger + +Supervisor-owned record for the remediation run of the 2026-09-04 audit +([umbrella #67](https://github.com/dexpace/nodejs-sdk/issues/67), subtasks #68–#82). One entry per +cross-task decision: which issue raised it, what was decided, the alternatives rejected, and which later +issues it constrains. Deferred work is listed at the end so the release pass can recover it. Umbrella +branch: `audit/remediation-67`, base `mvp`. Task branches: `audit/67/-`. + +A decision that departs from the spec text is also a dated row in +[`deviations.md`](./deviations.md) under "Deviations recorded outside a phase"; this file records the +*choice*, that file records the *deviation*. + +## Ground rules fixed before wave 1 + +### D0 — Where a remediation deviation is written (raised by #68, #69, #71, #72, #74, #75) +Several subtask issues say "record the reading in the Phase Nx ledger section". Those sections live in +`docs/work/mvp/`, which CLAUDE.md declares a dated record that is never retro-edited. **Decision:** every +deviation or reading this run records goes to `docs/deviations.md` under "Deviations recorded outside a +phase", dated, with `file:line` evidence and "Found by: audit #67 / #". Phase ledger sections are +not touched. *Rejected:* appending to phase ledgers (retro-edits a dated record; §10 is frozen so it is +not an option either). *Constrains:* every later subtask. + +### D1 — Release machinery is out of scope (raised by the run's own brief) +No changesets, no version bumps, no `docs/first-release.md` edits. Each PR lists what it skipped under +"Deferred — release machinery"; the consolidated list is at the end of this file. + +### D2 — Wave overlap adjustments +- Wave 1 (#68, #69) both edit the "Deviations recorded outside a phase" table in `docs/deviations.md` and + `packages/core/src/context/instrumentation.ts`. Kept concurrent: #68 edits existing rows/lines and the + `tracerFactory` TSDoc; #69 only *appends* rows at the end of the table and changes the single + `activeSpan` line. The supervisor resolves the adjacent-hunk conflict at merge. +- Wave 3 becomes #72, #74, #75 (all M3); #73 moves to wave 4 with #76, #77. Reason: #72 and #73 both + reshape `packages/core/src/retry/engine.ts` (`withTrail` vs. the per-attempt re-send of the template), + and #73's layering choice is easier to make on top of #72's landed trail shape. +- Wave 6 splits: #81 first, then #82. Both edit `undici-transport.ts` and add rows to + `packages/transport-conformance/src/run-suite.ts` + `fixtures.ts`; #82's "make undici match" clause + depends on #81's drop-set rewrite. + +## Decisions taken for #69 (M1) — the maintainer-decision items + +### D3 — CTX-15: fix, not ledger +`noopInstrumentationBundle.activeSpan` becomes `NOOP_SPAN`. One line plus a test. #80 lists the same item; +it is done here, #80 skips it. *Rejected:* ledger row (the fix is smaller than the row). + +**D3 outcome (2026-09-04, #69, PR #84).** The size estimate was wrong: importing `NOOP_SPAN` into +`context/instrumentation.ts` closed an import cycle with `observability/tracing.ts` (which imports +`InstrumentationBundle`), and `verify:import-cycles` counts type-only edges. Fixed as that gate prescribes: +`SpanContext`, `Span`, `Tracer`, `NOOP_SPAN`, `NOOP_TRACER` moved to a new leaf module +`packages/core/src/observability/span.ts`; `tracing.ts` re-exports them, so no import path and no API +report changed. The decision itself stands. *Constrains #80:* a `file:line` citation into `tracing.ts` for +those five names is now stale; CTX-15 is done, skip it. + +**Trap for every later subtask.** gts turns on `stripInternal`, and TypeScript tests it by substring-scanning +every leading comment of a declaration, line comments included. A module header that merely *mentions* the +`@internal` tag deletes the first exported declaration from the emitted `.d.ts` with no `tsc` diagnostic; +the failure surfaces one package later as an unresolved name in core's own `dist/`. Do not write that tag in +a file-level comment. + +**#68 outcome (2026-09-04, PR #83).** Round 1 corrected the TSDoc and guides; round 2 requested because +five `docs/deviations.md` anchors it found stale (items 2, 3, 4, 8, 11) were left unfixed as "outside the +partition" — they are the issue's own acceptance criterion. The false "nothing consumes either yet" claim at +`context/instrumentation.ts:8-13` was assigned to #68 in round 2. *Constrains #78:* item 17 now states the +cause-walk matches `IoError` and `TransportFailureError` only, and names #78 as the decider. *Constrains +#80:* the OBS-29 row is marked in progress with the 1:1 binding recorded as met at `pipeline/runtime.ts`; +the open part is caller-reachability of the operation span. *Constrains #71:* `auth.md`'s credential-shape +example is untouched and is #71's. + +### D4 — PIPE-37: ledger the gap, do not implement in M1 +No `PRE_REDIRECT` status-mapping pipeline step exists; `statusMappingStep` is a `ResponseStep`. Recorded +as a row in `deviations.md` naming the gap, the Phase 4→5 hand-off that dropped it, and the petstore +spike finding 2 as the same work. Implementing it is real pipeline work with public surface, outside a +docs-only milestone, and opening a tracking issue is a remote action this run is not authorised to take — +the maintainer opens it if wanted. *Constrains:* none of #70–#82 depends on it. + +### D5 — REDIR-3: keep the current-hop-method reading, pin it, ledger it +Spec text says "original request method". The port evaluates eligibility against the method of the +request being redirected at *this* hop. The two differ only after an opted-in 303 rewrote POST→GET and a +later 301/302 arrives: the port follows it (GET is in the default set), the literal reading would refuse +it. **Decision:** keep the port's reading — the rewritten GET is idempotent and body-less, the 303 rewrite +is opt-in, and refusing would make `allow303` half-useful — pin it with a test (`allow303: true`, POST, +303 then 301) and record the reading as a `deviations.md` row. *Rejected:* switching to the literal reading +(behaviour change inside a docs milestone; stricter without a safety gain). + +### D6 — PAGE-19: WHATWG relative resolution is the intended reading +`; rel=next` resolves against the page URL under WHATWG rules, so it is a followable relative +reference, not an unparseable one. A target that fails `new URL(target, base)` still ends the stream. +Pin with a test and ledger the reading. *Rejected:* adding an ad-hoc "looks unparseable" heuristic. + +### D7 — HTTP-46, IO-13, BODY-9, BODY-34, IO-38, transport `reasonPhrase`: ledger rows +All six are recorded as rows (evidence + reading). `reasonPhrase` sits beside §10 item 13; #82 reads it as +already done and does not re-ledger it. + +## Wave 1 — landed 2026-09-04 +PR #83 (#68) and PR #84 (#69) merged into the umbrella at `840f355`. One conflict (the OBS-29 row of +`deviations.md`): kept #68's "in progress, see #80" text and carried #69's moved `span.ts` citation. Merged +tree preflighted before the merge in a throwaway worktree (byte-identical result): all 20 steps passed. + +## Wave 2 — landed 2026-09-05 +PR #85 (#70) and PR #86 (#71) merged into the umbrella. One conflict (the import list of +`tests/conformance/xcut/security-by-default.conformance.test.ts`), unioned. Merged tree preflighted in a +throwaway worktree before the merge (byte-identical result): all 20 steps passed. Run paused here by the +maintainer; wave 3 (#72, #74, #75) not yet cut. + +## Decisions taken for wave 2 (M2) + +### D8 — #70: redact inside the error messages, and keep the raw URLs on the error properties +`SchemeDowngradeError` and `NonReplayableBodyError` build their messages from `redactUrl(url)`; `fromUrl` / +`toUrl` stay raw for program use. Reason: the message is what every logger, `cause` chain and consumer +`console.error` renders, so redacting at the source protects paths this SDK does not own, not only +`http.redirect.rejected`. If `emitRejected` can also carry the redacted URL fields the other redirect events +carry, add them — but the message fix is the required one. *Rejected as sole fix:* logging `error.name` plus +fields and dropping the message (leaves the raw message reachable through `cause` on the thrown error). +*Constrains:* none. + +### D9 — #71: credential classes, and "once guarded, always guarded" for the replay +- `BasicCredential` and `DigestCredential` become classes with `#password`, `toString` and the + `nodejs.util.inspect.custom` override, following whatever shape `auth/credential.ts` already uses for + `ApiKeyCredential` / `BearerToken` (class plus `createX()` factory if that is the pattern there). Public + shape change, free before the first version bump; `api:local` on core. +- Replay guard: if the original request required HTTPS (the step guarded it), `requireHttps` runs on the + replacement request unconditionally, regardless of which header names it carries. *Rejected:* building the + set of credential-carrying header names from configuration (misses a `challengeHook` that invents a + header). AUTH-8 names bearer, API-key and name-key only; the wider reading is a `deviations.md` row (D0). +- `docs/sdk-documentation/auth.md`'s credential-shape example is rewritten here (#68 left it). +*Constrains #74:* it edits `auth-step.ts` next wave on top of this; the guard site moves. + +**D8 outcome (2026-09-04, #70, PR #85).** Both messages built from `redactUrl()`; raw URLs stay on +`targetUrl` / `fromUrl` / `toUrl`; `http.redirect.rejected` gained `url.full` (redacted) like the sibling +events. A non-URL string handed to either public constructor now renders `[malformed url]` in the message +(OBS-15 totality read as the safe default; pinned). New fixture route `/redirect-secret-target` in +`tests/conformance/xcut/fixtures/server.ts` (307 to `/echo?access_token=`), reusable. +*Constrains #74:* build any URL-naming message from `redactUrl()` at the constructor. *Constrains #72, #78:* +`docs/sdk-documentation/errors.md` redirect section was edited here. + +**D9 outcome (2026-09-05, #71, PR #86).** `BasicCredential` / `DigestCredential` are classes in +`auth/credential.ts` with `#password`, read inside the package through an `@internal` `credentialPassword()` +friend hook; `DigestCredential` takes `algorithmPreference` as a third positional. No construction-time +validation on the classes — AUTH-14/AUTH-16 stay single-sourced in `basicHandler()` / `digestHandler()`, +which `authStep()` builds at construction, so a blank password still fails there. Replay guard keys on +`OutboundPlan.guarded` ("once guarded, always guarded"); `guardReplayScheme` now takes a `ReplayGuardInput` +bundle. Two `deviations.md` rows: AUTH-8 widened to every credential type; XCUT-16 guard deliberately wider +than its letter. `scripts/verify-consumer-types.mjs`'s fixture changed because no structural +`BasicCredential` shape exists any more. *Constrains #74:* `auth-step.ts` conflict surface is the +`./credential.js` import block, `buildHandlers`, `OutboundPlan`/`planOutbound`, `guardReplayScheme`, +`ChallengeDrive`; `credential.ts` now has a type-only import of `./digest.js`, so a new edge from +`digest.ts` back into `credential.ts` closes a cycle. A stubbed `ChallengingTransport` exists in +`security-by-default.conformance.test.ts` for clauses needing an `https://` hop. + +**Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is +inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. + +## Wave 3 — landed 2026-09-05 +PR #87 (#75), PR #89 (#74) and PR #88 (#72) merged into the umbrella at `2ea8b3e`, in that order. One +conflict (the `deviations.md` table tail: #75's `ASYNC-21` row and #74's `AUTH-22` row), unioned. Merged tree +preflighted in a throwaway worktree before the merge (byte-identical result): all 20 steps passed. The three +stale `retry/*` citations in `deviations.md` item 3 that #72 shifted were re-anchored on the umbrella after the +merge (`engine.ts:367`, `retry-step.ts:151`, `retry-dispatch.ts:55`). + +**Trap for every later subtask (git identity).** The #72 agent committed with +`git -c user.email="oaljarrah@dexpace.org"`, lifted from the harness's "user's email address" context line — +that address is the Claude login, and GitHub attributes it to a different account. Rewritten with +`--reset-author` and force-pushed before the merge; nothing on the umbrella carries it. Contract item 10 now +forbids any author override; the supervisor checks `git log --format=%ae` on a branch before merging it. + +## Decisions taken for wave 3 (M3) — pre-taken 2026-09-05, before dispatch + +### D10 — #72: the final typed error is surfaced as-is; the trail rides in a side table, read through `retryAttempts()` +The surfaced error of `retryStep` / `dispatchWithRetry` is the final attempt's own error, class untouched: +`instanceof TransportFailureError` holds for `maxAttempts` 1 and 3 alike, and an abort during backoff surfaces +the `CancellationError` that `abortToSdkError` built (`retry/engine.ts:385`), which `withTrail` at `:386` was +undoing. Earlier attempts' errors are reachable through a new `@public` accessor exported from core, +`retryAttempts(error: unknown): readonly unknown[]` — oldest first, the surfaced instance itself excluded +(RETRY-34's skip-self clause), `[]` for an error that carries no trail — backed by a module-private `WeakMap` +that the engine writes once per terminal failure. **Not a deviation, a correction:** RETRY-34 says the prior +failures are "attached to the surfaced exception as suppressed", which is Java's `addSuppressed` — the +surfaced exception stays what it is and grows a list. Wrapping it in `SuppressedError` made the surfaced +*type* a function of how many attempts ran, which is what XCUT-1's "assert the surfaced error is the +cancellation type" clause catches. No `deviations.md` row. `suppress()` stays for its RECOV-12 job. +*Rejected:* an own property (`attempts` / `errors` / `suppressed`) defined on the surfaced error — a foreign +error may be frozen or non-extensible, so `defineProperty` in the engine's failure path can itself throw; a +primitive thrown value cannot carry one at all; and `.suppressed` already means "the one secondary" on +`SuppressedErrorLike`. *Rejected:* a `RetryExhaustedError` wrapper (hides `CancellationError`, the row XCUT-1 +is about). *Rejected:* threading the trail through `cause` (`cause` is already the raw abort reason at +`:383`, and it means "why", not "before"). A primitive surfaced value is passed through unchanged with no +trail entry rather than wrapped. Update the `retryStep` TSDoc, `retry-dispatch.ts:45`'s `@throws` prose, +`docs/sdk-documentation/pipelines.md`'s retry section, the "suppressed trail" wording at +`docs/sdk-documentation/errors.md:188`, and `write-a-response-handler.md` so the RECOV-12 wrapper is documented +as the *only* place a `SuppressedError` is built. `api:local` on core. *Constrains #78:* the classify +cause-walk sees the typed error directly now, never through a `SuppressedError.error` hop. *Constrains #73:* +the trail accessor is the shape it layers on. + +### D11 — #74: parse every challenge header; emit `cnonce` for `-sess` regardless of `qop`; empty `realm`/`nonce` are unsatisfiable +- **Repeated `WWW-Authenticate` / `Proxy-Authenticate`.** `pickChallengeHeader` reads `headers.getAll(name)` + and parses each value with `parseChallenges`, concatenating the lists in wire order — parse-each rather + than comma-join, so a malformed later value cannot poison the parse of an earlier one. `rank` selects across + the concatenation. Conformance row in `packages/transport-conformance` (`run-suite.ts` + `fixtures.ts`): a + fixture route sending two `WWW-Authenticate` headers, asserting the *parsed challenge list* is identical + through both transports — `getAll` legitimately returns one comma-joined entry through fetch and two entries + through undici, and the list after parsing is the only thing the transport is answerable for. Fix the + `Set-Cookie`-only comment at `undici-transport.ts:334`; touch nothing else in that file (#81 owns it). +- **`-sess` without `qop`: emit `cnonce`.** RFC 7616 §3.4 says of `cnonce` "This parameter MUST be used by all + implementations", and §3.4.2 folds it into A1 for every `-sess` algorithm; a `-sess` response without it is + unverifiable by construction, which is what the port sends today (`digest.ts:337-343` hashes a cnonce the + header at `:386-387` omits). `nc` and `qop` stay conditional on a negotiated `qop`. AUTH-22's "emit + cnonce/nc/qop only when qop is negotiated" is RFC 2617's RFC 2069-compatibility form, which predates + `-sess`. Departure from AUTH-22's letter: one `deviations.md` row (D0). *Rejected:* declining the challenge — + it turns every `-sess`-without-`qop` server into a guaranteed 401 for no security gain, and the value is + already computed. `computeDigestResponse` vector for `MD5-sess` with no `qop`. +- **Empty `realm` or `nonce`** is unsatisfiable: `parseDigestChallenge` requires non-empty strings, the + challenge is declined and the next one tried (AUTH-25's "return no header when it cannot satisfy any"). No + row; AUTH-12's verbatim storage is unchanged, the check sits at selection. +- No new core exports. `api:local` on core only if a `@public` TSDoc changes. No changeset (D1). +*Constraints inherited:* D9 (the `auth-step.ts` surface #71 reshaped: `buildHandlers`, `OutboundPlan`, +`planOutbound`, `guardReplayScheme`, `ChallengeDrive`; `credential.ts` imports `./digest.js` type-only, so no +edge from `digest.ts` back into `credential.ts`); D8 (any URL-naming message is built from `redactUrl()`). + +### D12 — #75: keep the ownership transfer, and ledger it +`sseEvents$` / `typedSse$` keep passing `() => stream.close()` as `fromAsyncIterable`'s `release`. The +issue's "spec-faithful one-line change" is neither: (1) `SseStream` self-releases on **any** iterator +termination by SSE-30's own design — `#iterate`'s `finally` runs `#releaseQuietly()` when the runtime calls +`return()` (`packages/core/src/sse/stream.ts:136-138`), and `fromAsyncIterable` must call `iterator.return()` +exactly once (ASYNC-6), so the socket closes with or without the callback; a `for await` with `break` closes it +the same way. ASYNC-21's "MUST NOT close the caller-owned source" presumes a source whose iterator return does +not release, which this port's `SseStream` deliberately is not. (2) The release-*before*-`return()` ordering +is what settles an in-flight pull on unsubscribe (`packages/rx/src/from-async-iterable.ts:44-48`): an async +generator's `return()` queues behind a suspended `next()`, so dropping the callback would leave an unsubscribe +during a stalled read pending until the server sends a byte. Pagination passes no release because its pulls +are bounded HTTP exchanges; SSE's are not. Removing the callback would change only the failure channel and the +ordering, not whether the source closes — and it would reintroduce the hang. **Recorded as a deviation** from +ASYNC-21's non-closing clause: one `deviations.md` row (D0) naming `sse.ts:35,57-59`, `from-async-iterable.ts:103-108`, +the two reasons above, and the Phase 8b checklist gist at +`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67` that dropped the clause (a +dated record; not retro-edited). Tests in `packages/rx/src/sse.test.ts`: early unsubscribe, source error, and +end-of-source each close the underlying resource **exactly once** (count the resource's `close`, not the +facade's — `SseStream.close()` is idempotent by SSE-28, so the facade count proves nothing); plus the +unsubscribe-during-suspended-pull case settles. TSDoc on both functions states the transfer outright +("subscribing hands the stream to the adapter; do not call `close()` yourself, and do not iterate it +afterwards"), and `packages/rx/README.md` says the same beside its `for await` guidance. `api:local` on rx. +*Rejected:* dropping `release` (above). *Rejected:* a caller-facing `{ownership}` option (two behaviours to +document for a case with one correct answer). No changeset (D1). + + +**D10 outcome (2026-09-05, #72, PR #88).** `withTrail` became `attachTrail`; the trail lives in a new leaf +module `packages/core/src/retry/attempt-trail.ts` (`recordAttempts` internal, `retryAttempts` public, one +shared frozen empty list). An empty trail *deletes* a prior entry for a reused error instance; symbols are +excluded as weak keys rather than probed. Round 2 removed a public claim that `retryAttempts(e).length + 1` is +the send count — false on three reachable paths (the RETRY-32 gate, `stampAttempt` throwing, a non-abort +`Clock.sleep` rejection), and the `pipelines.md` example was unsound even narrowed to +`TransportFailureError`, because `abortToSdkError` synthesizes one for a timeout signal. Two engine cases pin +sends against trail length. `tests/node-conformance/retry.test.mjs` was outside the partition and edited anyway: +its case asserted the wrapper by name and was red the moment D10 landed. *Correction to D10's text:* +`classify.ts` never walked `.error`, and it ran per attempt before the terminal wrap, so the "hop" the +constraint on #78 described was not live; the end state it names is right. + +**D11 outcome (2026-09-05, #74, PR #89).** `pickChallengeHeader` returns every value (`getAll`), parsed per value +by a new `challengesOf`; a test with an unterminated quoted string in the first value discriminates parse-each +from comma-join. `cnonce` emitted for any `-sess` algorithm (row in `deviations.md`). Empty `realm`/`nonce` +decline (`!== ''`, not trimmed). **Deviation from D11's letter, accepted:** the conformance row cannot assert the +*parsed* list — `parseChallenges` is `@internal` and D11 forbade a new core export, and a real `authStep` drive is +blocked by AUTH-28 on the plain-`http` fixture — so `run-suite.ts`'s `challengeList()` splits the joined +`getAll` at `, Digest ` boundaries, scoped to the fixture's own two challenges and documented as not a parser. +The transport is answerable only for surfacing both values, which the row proves (red against a one-line +fixture). Found and fixed on the way: `run-suite.ts`'s header claimed TRANSPORT-10..14 were asserted elsewhere +while labelling rows with them; `node:http`'s `writeHead` rejects a `readonly string[]` header value, and +`gts lint` did not catch it — only `typecheck` did. *Constrains #82:* the new fixture route +`/repeated-challenge` and `registerInboundHeaderRows` exist; the `TRANSPORT-14` label is the closest MUST. + +**D12 outcome (2026-09-05, #75, PR #87).** No behaviour change. Ten `sse.test.ts` cases count the release the +*owned resource* sees (a structural `ReadableStream` double counting `reader.cancel()` and `body.cancel()` +separately) plus one in `tests/node-conformance/rx-bridge.test.mjs`. Measured counterfactual: deleting both +`release` arguments turns the two suspended-pull cases and two pre-existing idle-unsubscribe assertions red while +every exactly-once count stays green — so only the suspended-pull case discriminates the design; the counts pin +against a future double release, and the row says which is which. The row also closes `SSE-41`'s "documented +source ownership" clause, which Phase 8b marked done on documentation that named unsubscription only. New +`packages/rx/README.md` section "Who owns the stream"; `rx.api.md` regenerated byte-identical (prose only). + +### Wave 3 partition +| Task | Owns | Shared, append-only | +|---|---|---| +| #72 | `packages/core/src/retry/**`, the retry exports in `packages/core/src/index.ts`, `packages/core/etc/core.api.md` (retry names), `docs/sdk-documentation/pipelines.md` retry section, `errors.md:188`, `write-a-response-handler.md`, `tests/conformance/xcut/retry-safety.*` and `cancellation-and-timeout.*` | — | +| #74 | `packages/core/src/auth/{digest,auth-step,challenge}.ts` and tests, `undici-transport.ts:334` comment only, `packages/transport-conformance/src/{run-suite,fixtures}.ts`, `docs/sdk-documentation/auth.md` | `docs/deviations.md` (one row at table end) | +| #75 | `packages/rx/**` | `docs/deviations.md` (one row at table end) | +Known merge seams: the `deviations.md` table tail (#74 + #75), and `core.api.md` if #74 changes a `@public` +TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. + +## Wave 4 — landed 2026-09-05 +PR #90 (#73), PR #91 (#77) and PR #92 (#76) merged into the umbrella at `36c3d04`, in that order. One +conflict (the `deviations.md` table tail: #73's `RETRY-44` row against #76's `HTTP-35` and `HTTP-31` rows), +unioned. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all 20 +steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com` before merging. + +**Traps for every later subtask, added this wave.** +- `stripInternal` is wider than D3 recorded: **any** leading comment of a declaration that contains the + `@internal` substring — inside backticks, in ordinary prose — strips that declaration from the `.d.ts`. + #73 lost `dispatchWithRecovery` from `orchestrator.d.ts` by writing "Both are `@internal`" in its TSDoc. +- `bun run api:local` exits 0 on `ae-unresolved-link` *warnings*; `api:ci` (what `bun run api` and CI run) + exits 1 on them. A `{@link}` to an `@internal` name from a `@public` block therefore looks clean locally + and is red in CI. Backticked prose, as for the `.message` trap. +- `gts lint` and a green `bun test` do not type-check what `tsc` does: #74's `readonly string[]` header value + and #77's `Promise`-as-`BodyInit` both surfaced only on the preflight's `typecheck` step. +- `String.prototype.isWellFormed()` exists on the Node floor but not on `lib: ES2023`; use + `/\p{Surrogate}/u` (now in `http/rfc3986.ts`). + +## Decisions taken for wave 4 (#73 M3, #76 + #77 M4) — pre-taken 2026-09-05, before dispatch + +### D13 — #73: the request recovery chain runs once, above the retry loop +`dispatchWithRetry` applies `config.requestChain` **once** and hands the prepared request to `runWithRetry`; +each attempt then re-executes transport + response chain over `stampAttempt`'s fresh copy of that prepared +request. That is the layering `idempotencyKeyStep`'s `@public` TSDoc has claimed since `1f48926` ("runs ONCE per +call, upstream of retry"), and it is what RECOV-32's "one stable key across every retry" and RETRY-38's +"preserving any idempotency key" both presuppose — a key the engine re-sends from a pre-chain template is not +preserved, it is regenerated. **Reading of RETRY-44:** its "downstream chain" is whatever sits below the retry +loop; in the recovery stack that is transport + response chain, and its "upstream steps MUST NOT mutate the +shared in-flight request" clause is satisfied by construction, because upstream steps no longer run between +attempts at all. One `deviations.md` row records the reading (D0), because the port's own test named RETRY-44 +as the reason the request chain re-ran (`retry-dispatch.test.ts:67`, to be renamed). A request-chain failure +happens before the loop and is not retried: it never reached the wire, and it still passes through the response +chain once so RECOV-10/11's outcome handling is unchanged. *Rejected:* memoizing the key on the template +(`WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice +would replay the key and have the server drop a real second call. *Rejected:* re-running the chain over the +*prepared* request each attempt — the chain reads its own output, which is the mutation RETRY-44 forbids in +different clothes, and every step would have to be proven idempotent. Enumerate the shipped `RequestStep`s in +`recovery/` and state in the ledger outcome that none needs per-attempt re-execution; the attempt ordinal is +the engine's (RETRY-38). Tests: N attempts, one `generate()` call, the same header value on every wire send; +rename the RETRY-44 test to what it now proves. TSDoc on `dispatchWithRetry` and the orchestrator; no +public shape change (`dispatchWithRetry` is `@internal`). `api:local` if `idempotencyKeyStep`'s prose moves. +*Constrains #78:* `engine.ts` is edited here; wave 5 lands on top. + +### D14 — #76: reject at the setter, in the `DexpaceError` tree, and say so in `@throws` +- **Path params:** `Object.hasOwn(pathParams, name)`; a missing own property is `OperationAssemblyError` + (SEAM-27's "every placeholder MUST have a supplied value"). `{constructor}` with `{}` is the pin. +- **Dates:** `ifModifiedSince` / `ifUnmodifiedSince` throw `RequestConditionsValidationError` on + `Number.isNaN(date.getTime())`. +- **`timeoutMs`:** the setter rejects a non-integer and anything above `2**32 - 1` with + `RequestOptionsValidationError` — HTTP-35 puts the range check at the setter, and `AbortSignal.timeout()`'s + range is the only one a transport can honour. Rewrite the TSDoc paragraph that argues a fractional + millisecond is meaningful and flip the test that pins it. *Rejected:* rounding/clamping in `composeSignal` + (hides the caller's error where HTTP-35 says to surface it). Add `@throws` to `composeSignal` for whatever + it can still raise. +- **Lone surrogates:** `String.prototype.isWellFormed()` at the call site that supplied the value — + `QueryParams` builder `add`/`set` (name and value) and `substitutePathParams` — throwing the error class + that call site already throws for invalid input (`OperationAssemblyError` for path params; for query params + the class the builder uses today, or `UrlConstructionError` if it has none — no new error class). Then + prove by test that no `URIError` can escape `encode()`, `equals()` or `buildRequest`; if one still can, + report the path rather than adding a second mechanism. +- **`getAll`:** one shared frozen empty array in `Headers` and `QueryParams`; a test that the present-name + array is frozen too. +- **`Headers.equals`:** direct cases (name order, value order, subset, case). +- **`TeeSink`:** `Number.isInteger(tapLimit) || tapLimit === Number.POSITIVE_INFINITY`, else the error the + constructor already throws for a negative limit. +- `api:local` on core after the `@throws` edits. No changeset (D1). Do not touch `http/media-type.ts` (#77's). + +### D15 — #77: contract violation on an empty chunk, quote the boundary, gate the tap on `closed` +- **Empty chunks:** `#writeExactly` throws `SourceContractViolationError` on `value.length === 0` for a + positive request, same wording as `io/retention-window.ts:177-183`; a declared length of 0 stays a + legitimate empty write (BODY-10). Tests: an empty-only source, and an empty chunk between real chunks. +- **Boundary:** keep RFC 2046 `bchars` as the accepted grammar (HTTP-51 says reject what *violates* it, not + narrow it) and **quote** the `boundary=` parameter whenever it is not a pure `tchar` token, using + `http/media-type.ts`'s existing token/quoted-string rendering (export an internal helper from that file if + the class API does not reach it; #77 owns `media-type.ts` this wave). Round-trip test through + `Response.formData()` on Bun, and the same case in `tests/node-conformance/body-lifecycle.test.mjs`. + *Rejected:* narrowing `validateBoundary` to `tchar` (rejects boundaries RFC 2046 allows for a problem the + renderer owns). +- **Logging tap after `close()`:** `startDrain`, `snapshot` and `read` check `state.closed` first — + `snapshot()` returns the captured prefix without starting a drain, `read()` rejects with + `ClosedResourceError`, `error()` reports only a genuine drain failure. Tests: close-then-snapshot, + close-then-read, close-then-error. +- **Node conformance:** cases for `toReadableStream`, `toWritableStream`, `TeeSink`'s bridge, + `withRequestLogging` and `withResponseLogging` go into the **existing** topic files + (`io-byte-stream.test.mjs`, `body-lifecycle.test.mjs`), not one new file per bridge — the tree is + topic-named and flat, and its README lists members. Pull, cancel and lock behaviour is what to assert. +- `api:local` on core if a `@throws` changes. No changeset (D1). + + +**D13 outcome (2026-09-05, #73, PR #90).** `orchestrator.ts` split into `prepareRequest()` (request chain, +once, RECOV-2's throw-to-`Failure` applied) and `dispatchPrepared()` (transport + response chain + unwrap), +both `@internal`; `dispatchWithRecovery` is their composition, behaviour unchanged. `dispatchWithRetry` +applies the chain once and retries `dispatchPrepared` over `stampAttempt`'s copy; a request-chain failure +gets one trip through the response chain and no retry. `engine.ts` needed no functional edit (D13's +constraint on #78 was over-cautious). The shipped `RequestStep` list is exactly one — `idempotencyKeyStep`; +the issue's "client identity, auth stamps" are pipeline-stack `StepDescriptor`s, already re-driven per +attempt by `retryStep`'s `ctx.fork()`, which is RETRY-44 correctly applied to *that* stack. *Correction to +D13's text:* "one stable key across every retry" is `idempotency-key.ts`'s own TSDoc, not RECOV-32's, whose +letter is "invoked at most once per applicable request"; the row records it as a reading. + +**D14 outcome (2026-09-05, #76, PR #92).** All seven bullets landed test-first. Departures, accepted: +`/\p{Surrogate}/u` instead of `isWellFormed()` (not on `lib: ES2023`); `QueryParams.parse` substitutes +U+FFFD rather than throwing, because HTTP-31 is a MUST that `parse` never throws (row); `QueryParamsBuilder` +has no `set`, so `add` only; `tests/node-conformance/seams.test.mjs` edited outside the partition because +`AbortSignal.timeout()` is runtime-divergent (Bun accepts `1.5` and `2**32`; Node rejects both). Round 2 +fixed `http.md`'s now-false fractional-timeout sentence, the two error-class TSDocs, consolidated +`EMPTY_VALUE_LIST` into `http/builder.ts`, and appended the HTTP-35 and HTTP-31 rows. `core.api.md` +byte-identical (signatures unchanged). *Handed to #79:* the last `URIError` escape, `pagination/query-splice.ts` +with a server-supplied cursor. *Handed to #81/#82:* `defaultTimeoutMs` unvalidated on both transports. + +**D15 outcome (2026-09-05, #77, PR #91).** Empty chunk → `SourceContractViolationError` via +`assertNonEmptyChunk`, applied for `declared === 0` too (the only reading that also honours "never an +infinite spin"; recorded in a source comment, as IO-17 and BODY-25 did for the same qualifier — no row). +Boundary rendered through `MediaType.of(...).render()`, so nothing new is exported from `media-type.ts`. +Tap gated on `closed` with the checks ordered fits-cap → tail-consumed → closed, because a literal +closed-first gate breaks BODY-23's repeatable read. 22 Node-conformance cases across the two existing topic +files; **Bun's `Response.formData()` accepts `boundary=a,b`, Node's rejects it**, so the Bun round-trip rows +are regression guards and only the Node tree reproduces the bug — the clearest case yet for that tree +existing. The unknown-length `pipeTo` path still forwards empty chunks; out of HTTP-39's scope, noted. + +### Wave 4 partition +| Task | Owns | +|---|---| +| #73 | `packages/core/src/recovery/**`, `packages/core/src/retry/{retry-dispatch,engine}.ts` + tests, `tests/node-conformance/recovery-chain.test.mjs` and `retry.test.mjs` if a case belongs there, `docs/deviations.md` (one row at table end), `core.api.md` if `idempotencyKeyStep`'s TSDoc moves | +| #76 | `packages/core/src/http/{headers,query-params,request-options,request-conditions,rfc3986}.ts`, `packages/core/src/seams/{operation,transport}.ts`, `packages/core/src/io/tee-sink.ts` + their tests, `core.api.md` (`@throws` on those) | +| #77 | `packages/core/src/body/{stream-body,multipart-body,response-body-logging}.ts` + tests, `packages/core/src/http/media-type.ts` (helper export only), `tests/node-conformance/{io-byte-stream,body-lifecycle}.test.mjs` + that tree's README, `core.api.md` if a `@throws` changes | +Known merge seams: `core.api.md` (up to three), which the supervisor regenerates on the merged tree rather than +hand-merging. No two tasks share a source file. + +## Wave 5 — landed 2026-09-05 +PR #93 (#78), PR #95 (#80) and PR #94 (#79) merged into the umbrella at `4576658`, in that order. No conflicts: +#78's item 17 section edit and #80's OBS-29 row edit plus two appended rows sit in disjoint regions of +`deviations.md`. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all +20 steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com`. + +**Traps added this wave.** `gts --fix` deletes an `eslint-disable-next-line` whose next line is another +comment (the directive binds to the comment and becomes unused): prose first, directive last. And +`packages/core/src/io/index.ts` is a dead barrel nothing imports, whose file-level comment carries the +`@internal` substring and so ships a broken `dist/io/index.d.ts` today; harmless only because the modules it +re-exports emit `export {}`. Left for the release pass (deleting it is a design call). + +## Decisions taken for wave 5 (M4: #78, #79, #80) — pre-taken 2026-09-05, before dispatch + +### D16 — #78: `instanceof IoError` stays; it means "transport-layer failure", and item 17 says so +The classifier keeps `current instanceof IoError` (option b). The four flat leaves are SDK-internal contract +and lifecycle failures, deterministic on re-send: `SourceContractViolationError` and `ClosedResourceError` +are caller programming errors, `AllocationLimitError` is a cap the same request will hit again, and +`EndOfStreamError` is the exact-length-copy contract inside this package — a *wire* truncation is the +transport's to surface, as `TransportFailureError`, which is why `TRANSPORT-20` makes that class an `IoError` +and the leaves not. RETRY-2's "an I/O error" is read as that boundary. Five `classify.test.ts` cases pin one +answer per class; `docs/deviations.md` item 17's rationale paragraph and the anchor-correction block are +rewritten to state the rule (the row itself stays; #68 left the decision to #78), and `io/index.ts`'s +comment says what the cause-walk actually matches. *Rejected:* switching to `isIoError` and deciding per leaf +(only `EndOfStreamError` was ever a candidate, and it is the wrong layer to decide wire truncation). +- `delayOverride` returning a non-finite number (`NaN`, `±Infinity`) is treated exactly like one that throws + (RETRY-40): the computed schedule is used and the loop continues; log through the same path a throwing + override uses. A finite negative override keeps today's behaviour (inline, no wait — pinned already). +- `computeDelay`: `initialDelayMs === 0` short-circuits to `0` before the power is taken, so `0 * Infinity` + never happens; keep the `Math.min` saturation for the positive case. A test at the overflow attempt. +- No changeset (D1); `api:local` only if a `@public` TSDoc changes. *Constrains #80:* item 17 is a section + edit mid-file in `deviations.md`; #80 edits the OBS-29 row and appends — different regions. + +### D17 — #79: race the read against the signal; close on every paginator exit; a present `null` is a `DeserializationError` +- **Abort:** `deserializeFrom` and `serializeTo` race each pending `reader.read()` / `writer.write()` against + the signal (abort listener added once, removed in `finally`), then release the lock as today, so the + documented "an aborted call never leaves the caller's source locked" becomes true rather than narrowed. A + module-private helper inside `packages/codec-json` — core exports no abort-race utility and codec-json has + no dependencies. Mid-drain tests in `json-serde.test.ts` and in `tests/node-conformance/serde.test.mjs` + (Web Streams + `AbortSignal`: runtime-divergent, so the Node case is required, not optional). + *Rejected:* narrowing `write-a-serde.md` and `seams/serde.ts`'s promise to "checked between chunks". +- **Paginator:** the two invariants use the same close-then-rethrow discipline as `parseOrClose`; test with + `== null` so `items: null` fails the invariant, not the spread. The error class is the one `invariant()` + already throws; the message names the invariant. Tests: `parse → undefined`, `parse → {items: null}`, + resource `close` called exactly once (PAGE-27). +- **`tristate()`:** runtime `=== null` check after `inner.parse`, throwing `DeserializationError` (SERDE-14's + fourth state is a decode failure, not a programmer error). +- **SSE:** `#releaseWithInFlightError` no longer calls `onReleaseFailure`; the release failure rides as + `suppressed` only (SSE-30 scopes the hook to the clean-terminal path). Adjust the test that expected both. +- **Handed over from #76:** the one `URIError` still escaping core is `pagination/query-splice.ts` + (`spliceQueryParam` / `readQueryParam`, reached from `strategies.ts:44,79,85` with a server-supplied cursor; + repro `spliceQueryParam(new URL('https://h/?a=1'), 'cursor', '\uD800')`). Handle it here with the same + `hasLoneSurrogate` helper #76 added to `http/rfc3986.ts`, throwing the error class the strategies already use + for a malformed cursor; add the test. +- Two packages: `api:local` in `core` and `codec-json` if a `@public` TSDoc changes. No changesets (D1). + +### D18 — #80: `run`, not `enterWith`, in the runtime; a public `instrumentation` option; document the config wiring +- **Context leak:** `Runtime.send()` wraps its awaited body in `AsyncLocalStorage.run()` for both stores + (diagnostic fields, active span), so what the caller observes after `await send()` is what it had before. + The handle-based `pushDiagnosticFields` / `AsyncScopedStore.enter` stay for callers, with TSDoc stating + that the returned restore works only inside the continuation that called it; the runtime stops using them. + Tests: log *after* `await send()` and assert no `trace.id`; the second `send()` gets its own + `http.client.operation` span (OBS-29 — this is the leak that suppressed it). +- **Public path:** `PipelineBuilder`'s constructor gains an options bag `{instrumentation?, operationName?}` + (second parameter, optional, so no caller breaks), threaded to `createRuntime`'s `contextInit`; + `StandardResilienceOptions` gains the same two fields and forwards them. `operationName` reaches + `promoteToRequest` (CTX-16). Rewrite `tracerFactory`'s TSDoc and the `deviations.md` OBS-29 row to "met" + with the new `file:line`. `api:local` on core. The example in + `.changeset/2026-09-04-per-operation-span.md` is a dated release note: leave it, list it as deferred (D1). +- **Store hygiene:** `contextStore.install` moves inside the `try`, or the `try` opens before + `startOperationSpan`; `span.end()` runs once, behind an `ended` flag. Test: `store.size` equal before and + after a send whose `tracerFactory` throws. +- **Body-drain diagnostics:** both catches emit `http.instrumentation.bodyCaptureFailed` through the + existing `safeEmit` with `cause`, then return the empty capture as today (OBS-20). +- **`DEXPACE_LOG_LEVEL`:** the global configuration's default stays empty — defaulting it to + `defaultConfiguration()` would read the process environment at import time, which is a behaviour change + with no phase behind it. Instead: `docs/sdk-documentation/pipelines.md` (logging section) documents + `setGlobalConfiguration(defaultConfiguration())` as the wiring, and `LoggingStepSettings` gains an optional + `configKey` whose default is the current constant. Check whether an OBS-35 row already exists in + `deviations.md`; if not, append one recording that the default key is baked in (OBS-35's letter) and why + a required key was not chosen (every caller would have to name one to get any logging at all). +- **Smaller items:** `noopInstrumentationBundle.activeSpan` is DONE (D3, #69) — skip. `logging-step.ts`'s + per-request span is ended in a `finally`. `redactUrl(string)`'s WHATWG normalisation (host case, default + port, path) is documented in its TSDoc as inherent to parsing and left as is — re-rendering the original + authority by hand is a second URL renderer for no security gain. +- No changeset (D1). `tests/node-conformance/observability.test.mjs` gets the post-`send()` context case + (`AsyncLocalStorage` behaviour is Node's, and the Bun suite is the one that passed over the leak). + +**Carried to wave 6 (#81/#82):** `defaultTimeoutMs` is unvalidated on both transports (`fetch-transport.ts:73,205,215`, +`undici-transport.ts:95,401,419`) and is now the only path by which an out-of-range delay reaches +`AbortSignal.timeout()`; Node rejects `1.5`/`2**32`/`-1` with `RangeError`, Bun accepts the first two. Validate at the +transport factory the way `RequestOptionsBuilder.timeoutMs` now does (D14), with a conformance row. + + +**D16 outcome (2026-09-05, #78, PR #93).** Rule kept and written down in `classify.ts`, `io/index.ts` and item +17. Six classify cases (five leaves plus `TransportFailureError`, each asserting `isIoError`'s answer beside the +classifier's). Non-finite `delayOverride` is screened at the source and reported through the same +`http.retry.delayOverrideFailed` path a throw uses (`reportOverrideFailure`); finite negatives unchanged. +`computeDelay` short-circuits `initialDelayMs === 0`. Round 2 put the failure semantics on the `@public` +`RetryStepOptions.delayOverride` TSDoc. `core.api.md` byte-identical. + +**D17 outcome (2026-09-05, #79, PR #94).** `codec-json/src/abort-race.ts` races each pending read/write; both +Bun and Node abort cases hung to their deadlines before it. Paginator: `pageOrClose()` runs the PAGE-4 +invariants under the same `closeThenRethrow()` as `parseOrClose`; all four invariants (paginator + page) reject +`null` as well as `undefined`. `tristate()` rejects a present `undefined` too, not only `null` +(`NonNullable` excludes both). **Correction to D17's text:** the SSE double report never came from +`#releaseWithInFlightError` — it came from `#iterate`'s `finally` running `#releaseQuietly()` after the catch +had already released; fixed there. A second double report survives deliberately: `bindAbort`'s hook call plus +the `suppressed` copy when an iterator is parked in a read at abort time (the listener cannot know). Query-splice +surrogates → `UrlConstructionError` (the strategies used no class of their own). Round 2 aligned +`UrlConstructionError`'s TSDoc, both abort clauses in `seams/serde.ts`, and added rule 5 (PAGE-4) to +`write-a-paging-strategy.md`. + +**D18 outcome (2026-09-05, #80, PR #95).** `send()` runs under `runWithSnapshot` (diagnostic store) and +`runWithActiveSpan` (span), both `AsyncLocalStorage.run`; the handle forms stay with TSDoc on their reach. New +`@public` `PipelineOptions {instrumentation?, operationName?}` as `PipelineBuilder`'s optional second argument; +`StandardResilienceOptions extends PipelineOptions`; `seedFrom('flatten')` carries the options through an +`@internal` friend accessor (beyond D18's letter, so the documented derivation path does not drop the bundle). +`install` + span start inside one `try`; `endOnce` latch; LOGGING step's span in a `finally`. +`http.instrumentation.bodyCaptureFailed` at `verbose`; `LoggingStepSettings.configKey`; `pipelines.md` section +"Turning logging on from the environment". OBS-29 row closed, OBS-35 row appended. *Left for a later pass:* +`packages/core/README.md`'s Observability bullet does not yet point at `PipelineOptions.instrumentation`. + +### Wave 5 partition +| Task | Owns | +|---|---| +| #78 | `packages/core/src/retry/{classify,backoff,engine}.ts` + tests, `packages/core/src/io/index.ts` (comment), `docs/deviations.md` **item 17 section only**, `core.api.md` if a TSDoc changes | +| #79 | `packages/codec-json/**`, `packages/core/src/pagination/{paginator,page}.ts` + tests, the `tristate` schema module + test, `packages/core/src/sse/stream.ts` + tests, `docs/sdk-documentation/write-a-serde.md`, `tests/node-conformance/serde.test.mjs`, `codec-json.api.md` | +| #80 | `packages/core/src/observability/{diagnostic-context,logging-step,redaction}.ts`, `packages/core/src/pipeline/{runtime,builder}.ts`, `packages/core/src/auth/preset.ts`, `packages/core/src/context/instrumentation.ts` (TSDoc), their tests, `docs/sdk-documentation/pipelines.md` and the observability guide, `docs/deviations.md` **OBS-29 row edit + one appended row**, `tests/node-conformance/observability.test.mjs`, `core.api.md` | +Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint regions); `core.api.md` +(regenerated on the merged tree). + +## Wave 6a — landed 2026-09-05 +PR #96 (#81) merged into the umbrella at `808f6b0`. Single branch on the umbrella tip, so its own preflight (all 20 +steps passed, at `ca682c5`; round 2 changed a README only) is the merged tree's. Identity checked. + +## Wave 6b — landed 2026-09-05; the run is complete +PR #97 (#82) merged into the umbrella at `901b50e`. Single branch on the umbrella tip, so its own preflight (all +20 steps passed) is the merged tree's. Identity checked. All fifteen subtasks of #67 are merged; milestones 1–5 +are done. The seven `file:line` citations in `deviations.md` item 13 and the §10 `sideEffects` item that #81 +and #82 shifted were re-anchored on the umbrella after this merge. + +**D20 outcome (2026-09-05, #82, PR #97).** Three new `transport-shared` modules: `dispatch-classification.ts` +(allow-list of permanent recognitions — terminal argument codes, a cause-less `TypeError`, three known scheme +messages; `'bad port'` deliberately excluded because Node's `fetch` reports TRANSPORT-20's own dead-port probe +that way), `body-less.ts` (`body === null` for HEAD, 101/103/204/205/304, 2xx CONNECT; each adapter releases +the handle it declines — **Bun's `fetch` also returned a live stream for these, so both adapters needed it**), +`default-timeout.ts` (carried from #76; `TypeError` at both factories). `CONTROL_BYTE` now covers LF. +`ForkedSignal` always returns a live signal and gains `abort(reason)`, so the producer-failure branch can +cancel the native call; `producerFailure` classifies its own rejection as `TransportFailureError` so the shared +table cannot read it as permanent. **One `deviations.md` row beyond the task's letter, accepted:** the +classification departs from TRANSPORT-20's "*any* transport failure that produced no HTTP response", and +`write-a-transport.md` now prescribes the reading to third-party transports. The producer race is proven by an +instrumented-transport test, not a shared row (the suite cannot see the signal handed to the native call). +*Correction to D19's outcome:* `ftp://` on undici was already terminal via `UND_ERR_INVALID_ARG`. + +## Closing state (2026-09-05) +- Umbrella `audit/remediation-67`: fifteen task branches merged, PRs #83–#97 closed as merged, #67 checklist + fully ticked. Base `mvp` untouched at `1f48926`. +- Every merged wave preflighted on its exact tree (all 20 steps) before the merge; every task-branch commit + authored as the repository's configured identity. +- Left deliberately for the release pass: the "Deferred — release machinery" table below (fifteen changesets' + worth of notes), `docs/first-release.md`, the `ProxyType` union question, `packages/core/src/io/index.ts`'s + dead barrel, `packages/core/README.md`'s Observability pointer, the `bindAbort` second report in SSE, and the + unknown-length `pipeTo` path forwarding empty chunks. +- Not done by this run, on purpose: opening a PR from the umbrella into `mvp`, and any `gh issue create` (D4's + PIPE-37 tracking issue). Both are the maintainer's. + +## Decisions taken for wave 6 (M5: #81, then #82) — pre-taken 2026-09-05 + +### D19 — #81: degrade what undici cannot carry; one short-write detector; a typed refusal for SOCKS +- **Header rejections become logged drops (TRANSPORT-11/12).** `UNDICI_FORBIDDEN_HEADERS` gains `expect`, + `keep-alive` and `upgrade`; `connection` stays forwarded (§17's note) but a value other than `close` / + `keep-alive` is dropped and logged; every name is validated against RFC 9110 `token` in `toUndiciHeaders` + and a non-token name is dropped and logged, which is the degrade `fetch-transport.ts:146-152` already does + by `try`/`catch`. Check `FETCH_FORBIDDEN_HEADERS` against the same rows on Node's undici-backed `fetch` and + widen it only if a row proves a rejection. Conformance rows both transports run: `Expect: 100-continue`, + `Upgrade: websocket`, a non-token name — the send succeeds, the header is absent on the wire, the drop is + logged by name. +- **`fileBody` goes through `writeTo` (BODY-13).** The undici file branch stops handing `createReadStream` to + undici and takes the same `pumpBody` path the streamed case uses, so `transferred === count` runs on both + transports and a truncate-after-stat fails with `TransportFailureError` naming transferred-of-total. + *Rejected:* a byte-counting wrapper around the read stream (a second BODY-13 implementation, which is the + drift the shared pump exists to prevent). Conformance row: truncate a 1 MB file to 10 bytes after `stat`, + both transports reject identically. If the pump path changes the framing (`content-length` vs chunked), + say so in the PR and keep whatever the streamed case does today. +- **SOCKS:** `undiciTransport()` refuses `proxy.type !== 'http'` at the factory with the same `TypeError` + shape `toDispatchError` uses for a terminal misconfiguration (deliberately outside the `IoError` tree), + `@throws` documented. `ProxyType` keeps `socks4`/`socks5` — narrowing it is a public-shape decision for the + release pass (D1) — and one `deviations.md` row records "SOCKS is resolved by core and supported by neither + transport". `fetchTransport()` gets the same check if it accepts a proxy at all. +- Rows go in `run-suite.ts` + `fixtures.ts`; #82 lands on top of them (D2). `api:local` on `transport-undici` + (and `transport-fetch` if touched). No changesets (D1). +*Partition:* `packages/transport-undici/**`, `packages/transport-fetch/src/fetch-transport.ts` (forbidden set +only, if a row proves it), `packages/transport-conformance/src/**`, `docs/sdk-documentation/write-a-transport.md`, +`docs/deviations.md` (one appended row), the two transports' `api.md`. + + +**D19 outcome (2026-09-05, #81, PR #96).** `expect`/`keep-alive`/`upgrade` added to BOTH forbidden sets — a row +proved Node's undici-backed `fetch` rejects them with a *retryable* `TransportFailureError`, and Bun 1.3.14 +diverges a third way (forwards two, hangs on `upgrade`). undici gains a per-header RFC 9110 token guard and a +`connection`-value guard, all degrading to logged drops. **The undici file branch was deleted outright** rather +than rerouted, so `prepareBody` is the same two decisions on both transports; framing for a file ≤ 1,000,000 +bytes changed from chunked to `content-length`, matching fetch. SOCKS refused at the factory with a `TypeError`; +`ProxyType` keeps the union (narrowing it would breach `CFG-22`'s MUST for the model); one row. The streamed leg +of the truncate row lives in `tests/node-conformance/transport.test.mjs` because Bun's `Readable.fromWeb` leaks +the abort reason as an unhandled rejection. *Constrains #82:* new registrars `registerNativeRejectionRows`, +`registerFileBodyRows`, `registerProxyRefusalRows`, fixture `fileBodyFixture`, `TransportCapabilities.unsupportedProxy`; +the `fromWeb` leak sits in the producer-race code D20 edits; `ftp://` still reaches `dispatcher.request` unchecked. + +### D20 — #82 (after #81 merges): one classification table; `body === null` when there is none; abort the fork +- **Permanent errors on fetch:** the fetch transport classifies a `TypeError` from an unsupported scheme, an + invalid URL, a forbidden method or an invalid header exactly as the undici transport classifies its + `TERMINAL_ARGUMENT_CODES` — a bare `TypeError` with `{cause}`, outside the `IoError` tree — and the table + that decides it moves to `@dexpace/transport-shared` so the two cannot drift (the precedent is + `abort-mapping.ts`). `fetch failed` with a network `cause` stays `TransportFailureError`. Conformance row: + `ftp://` (or another scheme the runtime refuses) is non-retryable on both — assert `isIoError(e) === false`. +- **Body-less responses (204, 304, HEAD):** `Response.body` is `null` — the WHATWG shape, and what core's + model already types (`http/response.ts:18`); undici stops wrapping `result.body` unconditionally and + hands `null` for those three. Rows for 204, 304 and HEAD assert `body === null`, `contentLength`, and that + `reasonPhrase` is `undefined` or a string (the divergence is D7's row beside §10 item 13; the row does not + re-ledger it). *Rejected:* an empty stream on both (a consumer would have to read to learn there is nothing). +- **`CONTROL_BYTE`** becomes `/[\x00-\x08\x0A-\x1F\x7F]/u`, with a test in `transport-shared` for LF. +- **Producer-failure race:** the producer-failure branch aborts the forked signal before rethrowing, on both + transports, so a native call that resolves afterwards is cancelled rather than stranded with an unread body. + Verify with an instrumented transport whose native call resolves after the producer fails. +- `api:local` on `transport-shared`, `transport-fetch`, `transport-undici` as touched. No changesets (D1). +- **Carried from #76:** validate `defaultTimeoutMs` at both transport factories with the same integer + `1..2**32-1` rule `RequestOptionsBuilder.timeoutMs` uses (D14), typed error, `@throws`, a conformance row. +*Partition:* the three transport packages, `packages/transport-conformance/src/**`, +`docs/sdk-documentation/write-a-transport.md`, the three `api.md`s. + +## Deferred — release machinery (recoverable list) +| Issue / PR | Deferred item | +|---|---| +| #68 / PR #83 | patch notes for `@dexpace/core` and `@dexpace/codec-json`: shipped `.d.ts` prose changed for `Deserializer`, `jsonSerde()`, `InstrumentationBundle.tracerFactory`, `buildRequest`, `RequestConditions.applyTo` | +| #68 / PR #83 | `docs/first-release.md:117` claims `serde.ts:99,170` cite `H15` — `H15` appears nowhere in `serde.ts`; `:159` puts `deserializeFrom` at `:162` and `serializeTo` at `:96` (actual `:221`, `:104`). File suspended under D1 | +| #70 / PR #85 | patch changeset for `@dexpace/core`: redirect error messages now carry redacted URLs (and `[malformed url]` for unparseable input); `http.redirect.rejected` gained `url.full` | +| #71 / PR #86 | minor changeset for `@dexpace/core`: `BasicCredential`/`DigestCredential` become classes (breaking for object-literal callers); patch note for `authStep`'s `@throws PlaintextCredentialError` prose; `docs/first-release.md` untouched though this is its "free before the first bump" class | +| #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | +| #72 / PR #88 | minor changeset for `@dexpace/core`: retry surfaces the final typed error (was a `SuppressedError` wrapper); new `retryAttempts()` export | +| #74 / PR #89 | patch changeset for `@dexpace/core`: every `WWW-Authenticate`/`Proxy-Authenticate` value is read; `cnonce` emitted for `-sess`; empty `realm`/`nonce` declined | +| #75 / PR #87 | changeset for `@dexpace/rx` (issue says minor; only `.d.ts` prose moved, so patch is arguable): SSE ownership transfer documented | +| #73 / PR #90 | patch changeset for `@dexpace/core`: one idempotency key per logical request across retry attempts; `.changeset/2026-08-26-recovery-chain-primitives.md:14`'s "single `try`/`catch`" description of `dispatchWithRecovery` is now two functions (guarantee unchanged) | +| #76 / PR #92 | patch changeset for `@dexpace/core`: `timeoutMs` rejects non-integers and values above `2**32-1` (was accepted, threw at send); lone surrogates rejected at `QueryParamsBuilder.add` and path substitution; `getAll` frozen on every path | +| #77 / PR #91 | patch changeset for `@dexpace/core`: empty chunk in `streamBody` is a contract violation; multipart boundary quoted when not a token; logging tap safe after `close()`; `.d.ts` `@throws` prose changed | +| #78 / PR #93 | patch changeset for `@dexpace/core`: non-finite `delayOverride` ignored with a warning (was a `RangeError` after one send); `computeDelay` returns `0` not `NaN` for `initialDelayMs: 0` at overflow | +| #79 / PR #94 | patch changesets for `@dexpace/core` (paginator closes on malformed `PageInfo`; SSE release failure reported once; splice surrogate typed) and `@dexpace/codec-json` (abortable reads/writes; `tristate` rejects present null) | +| #80 / PR #95 | minor changeset for `@dexpace/core`: `PipelineOptions` (new public shape), `LoggingStepSettings.configKey`, context restored after `send()`; `.changeset/2026-09-04-per-operation-span.md`'s `createRuntime(...)` example is stale | +| #81 / PR #96 | patch changesets for `@dexpace/transport-undici` (header degrade, file body via `writeTo`, SOCKS refusal, file framing change) and `@dexpace/transport-fetch` (three names added to the drop set); `ProxyType` narrowing declined — keep the union | +| #82 / PR #97 | patch changesets for `@dexpace/transport-shared` (`CONTROL_BYTE`, classification table, body-less rule, `ForkedSignal.abort`, `requireValidDefaultTimeoutMs`), `@dexpace/transport-fetch` and `@dexpace/transport-undici` (permanent failures non-retryable; `body === null` for body-less responses; producer failure aborts the native call; `defaultTimeoutMs` validated) | diff --git a/docs/deviations.md b/docs/deviations.md index 7ffe3d6..0467a45 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -14,9 +14,16 @@ the claim. 2. **The collection point.** A deviation found outside a phase, by a review or a maintenance pass, with no phase ledger to write to and no permission to write to §10. It is appended under "Deviations recorded outside a phase" at the end of this file, dated, and folded into §10 the next time §10 - is deliberately amended. That section is empty today: the 2026-08-31 restructure swept the non-frozen tree - and found no unrecorded deviation, only three unrecorded *deferrals* (the dissolved register's U2) and six - mis-numbered register citations (the dissolved register's U6). + is deliberately amended. **That section is no longer empty, and the paragraph that once said it was is + this one.** The 2026-08-31 restructure swept the non-frozen tree and found no unrecorded deviation there, + only three unrecorded *deferrals* (the dissolved register's U2) and six mis-numbered register citations + (the dissolved register's U6) — a clean result, and one that held only because that sweep read the + *registers*. The 2026-09-02 register audit and the 2026-09-04 code audit + ([#67](https://github.com/dexpace/nodejs-sdk/issues/67)) both read something else and both filled the + section: the second went through the shipped **code** and found MUST-level narrowings and undecided + readings that lived only in a phase spec, only in a test comment, or nowhere at all. Read the table below + for what is in it — this paragraph deliberately does not count the rows, because a stated count is the + one thing in a collection point that goes wrong on the next append. **This file is the audit and the mutable collection point; §10 is the ledger and it is frozen.** (The cross-reference was added 2026-08-30, when nothing in the repo linked the two and the numbering they share had @@ -96,8 +103,9 @@ Items **7**, **11**, and the `NFR-12` half of **14** are absent from this list o **Verified.** `Transport.send()` is `Promise`-only, one method satisfying `SEAM-11` and `SEAM-16` at once (`packages/core/src/seams/transport.ts:49`). `ContextStore` holds a plain `Map` (`packages/core/src/context/store.ts:24`) with a fresh `Symbol()` per call -(`packages/core/src/context/context.ts:104,125,149`). `Next` is `(request?) => Promise` with no sync -twin (`packages/core/src/pipeline/step.ts:23`). `wrapCancellation` degenerates to `failure(error)` and says so +(`packages/core/src/context/context.ts:112`, defaulted per flavor at `:128` and `:149`). `Next` is +`(request?) => Promise` with no sync twin (`packages/core/src/pipeline/step.ts:23`). +`wrapCancellation` degenerates to `failure(error)` and says so (`packages/core/src/recovery/cancellation.ts:31`). `ASYNC-18` holds: SSE parses `retryMs` (`packages/core/src/sse/parser.ts:131`) but never acts on it — reconnection is caller-owned, so no adapter schedules a delay outside the retry engine. @@ -133,9 +141,9 @@ non-bridge clause survives and is enforced as an ordinary obligation on `send()` ## 3. Two retry stacks collapse into one, with the total-timeout budget explicitly opt-in -**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:354`) — with exactly two thin -callers: the pillar step (`packages/core/src/retry/retry-step.ts:137`) and the dispatch adapter -(`packages/core/src/retry/retry-dispatch.ts:53`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | +**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:367`) — with exactly two thin +callers: the pillar step (`packages/core/src/retry/retry-step.ts:151`) and the dispatch adapter +(`packages/core/src/retry/retry-dispatch.ts:85`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for `RETRY-28` (`packages/core/src/retry/settings.test.ts:20`). @@ -160,9 +168,9 @@ language-level ceiling. > **Corrected in the ledger 2026-08-29 — the text was wrong, the deviation is not.** Item 4's stated mitigation — "exporting only > concrete classes, never bare structural interfaces, from each package's public entry point" — is false as -> written. `packages/core/etc/core.api.md` exports **61 interfaces** against 58 classes, and at least one is -> a builder-built, validated, frozen type: `Configuration` is `export interface Configuration` -> (`packages/core/src/config/configuration.ts:72`) returned from `ConfigurationBuilder.build()`, and +> written. `packages/core/etc/core.api.md` exports interfaces and classes in comparable numbers, and at least +> one interface is a builder-built, validated, frozen type: `Configuration` is `export interface Configuration` +> (`packages/core/src/config/configuration.ts:100`) returned from `ConfigurationBuilder.build()`, and > `setGlobalConfiguration()` / `resolveProxyOptions()` accept any hand-rolled object of that shape. Most > other exported interfaces are seams (`Transport`, `Serde`, `Logger`) or options records, where structural > typing is the point. The *deviation* is real and uncorrectable; the *mitigation sentence* overstated what @@ -172,11 +180,16 @@ language-level ceiling. ## 5. Schema-as-witness replaces reflective generic-type capture -**Verified.** `Serde` is not generic in `T` (`packages/core/src/seams/serde.ts:182`); the witness is a -decode-time parameter, `deserialize(data, schema: Schema, typeName?)` -(`packages/core/src/seams/serde.ts:145`). The codec-configuration knobs are absent and documented as -absent — `packages/codec-json/src/json-serde.ts:236,242` explain that `SERDE-21`/`22` have no coercion -setting because there is no coercing codec, and `SERDE-23`'s unknown-field policy belongs to the schema. +**Verified.** `Serde` is not generic in `T` (`packages/core/src/seams/serde.ts:241`); the witness is a +decode-time parameter, `deserialize(data: Uint8Array, target: DecodeTarget): T` +(`packages/core/src/seams/serde.ts:194`, and `deserializeFrom` at `:221` takes the same target), +where `DecodeTarget` bundles the `Schema` witness with its optional diagnostic label +(`:122-124`). The witness moved into that object on 2026-09-04 — the SPI took it positionally as +`(data, schema, typeName?)` until then, which is the signature this row quoted; the deviation is +unchanged by the reshaping, since a schema *value* is still what stands in for a reflected type token. +The codec-configuration knobs are absent and documented as absent — +`packages/codec-json/src/json-serde.ts:244,250` explain that `SERDE-23`'s unknown-field policy belongs +to the schema and that `SERDE-21`/`22` have no coercion setting because there is no coercing codec. `packages/codec-json/src/conformance.test.ts:8` states outright that no code implements `SERDE-21` or `SERDE-22`. @@ -252,9 +265,9 @@ configuration file would be empty by construction. The structurally equivalent J 2026-08-30).** The `[Symbol.asyncDispose]` repair replaced three class members with a **module-scope `Object.defineProperty` statement** — a top-level side effect — in four files across three packages that all declare `"sideEffects": false` (`packages/core/src/sse/stream.ts:209`, -`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:314`, -`packages/transport-undici/src/undici-transport.ts:566`; the manifest field at `packages/core/package.json:20`, -`packages/transport-fetch/package.json:21`, `packages/transport-undici/package.json:21`). That field entitles a +`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:366`, +`packages/transport-undici/src/undici-transport.ts:674`; the manifest field at `packages/core/package.json:25`, +`packages/transport-fetch/package.json:26`, `packages/transport-undici/package.json:26`). That field entitles a bundler to drop a module whose exports go unused, and nothing forbids a future one from also dropping a top-level statement it judges inert — which would silently un-install disposal in the shipped artifact while every type still checks, the same failure shape this guard already exists for. `fixture-app.ts`'s @@ -274,10 +287,13 @@ bundle from 16,671 to 17,689 bytes against a 24 KiB budget (`packages/shrink-tes ## 12. The redirect/auth cross-origin marker is a real header, not a `WeakSet` **Verified.** `CROSS_ORIGIN_MARKER_HEADER` is set, cleared, and tested per hop -(`packages/core/src/redirect/cross-origin.ts:80,93,104,118`). The auth step reads it before deciding whether -to react to a challenge, not just whether to stamp (`packages/core/src/auth/auth-step.ts:387-390`). An -independent `POST_AUTH` backstop strips it even in a pipeline with no auth step -(`packages/core/src/redirect/strip-marker-step.ts`), satisfying `REDIR-11(c)`'s porter caveat. +(`packages/core/src/redirect/cross-origin.ts:80,93,104,118`). The auth step reads it once on the outbound +pass (`packages/core/src/auth/auth-step.ts:395`) and gates both branches on the answer — preemptive stamping +at `:401`, and whether to react to a challenge at all at `:786`, not merely whether to stamp. The answer +rides across the dispatch on `OutboundPlan.crossOrigin` (`:375`, and `:370-372` for why), because the marker +itself is cleared from the request before it reaches the wire. An independent `POST_AUTH` backstop strips it +even in a pipeline with no auth step (`packages/core/src/redirect/strip-marker-step.ts`), satisfying +`REDIR-11(c)`'s porter caveat. **Why it cannot be corrected.** The reference's in-process marker was tried and withdrawn during Phase 5b's own drafting: retry's attempt-stamping sits between redirect and auth and produces a **fresh `Request` @@ -294,9 +310,9 @@ readings; reversing either would widen an attack surface for a caller convenienc ## 13. Transport adapters have platform-shaped gaps the reference does not **Verified.** `Protocol.HTTP_1_1` is hardcoded in both adapters -(`packages/transport-fetch/src/fetch-transport.ts:178`, `packages/transport-undici/src/undici-transport.ts:346`). +(`packages/transport-fetch/src/fetch-transport.ts:201`, `packages/transport-undici/src/undici-transport.ts:429`). `transport-fetch` documents having no `proxy` option at all -(`packages/transport-fetch/src/fetch-transport.ts:62-65`). The proxy `challengeHandler` is surfaced with a +(`packages/transport-fetch/src/fetch-transport.ts:77-83`). The proxy `challengeHandler` is surfaced with a warning rather than dispatched (`packages/transport-undici/src/challenge-handler.ts:27,50`). **Why it cannot be corrected.** Four of the five clauses are closed by the platform, not by choice: @@ -379,7 +395,7 @@ produce that evidence; it unblocks at first release and not before. ## 15. A server-issued ETag containing obs-text does not round-trip **Verified.** `RequestConditions.applyTo` writes every entity tag through the outbound `Headers` builder's -`set` (`packages/core/src/http/request-conditions.ts:129-142`), which enforces `HTTP-18`'s HTAB + printable +`set` (`packages/core/src/http/request-conditions.ts:133-146`), which enforces `HTTP-18`'s HTAB + printable ASCII 0x20–0x7E rule (`packages/core/src/http/ascii-validation.ts:16`). The inbound path is separately laxer and permits obs-text, exactly as `HTTP-19` requires (`packages/core/src/http/ascii-validation.ts:41`, `packages/core/src/http/headers.ts:246,262`). @@ -412,21 +428,50 @@ single-subscription behavior is forced by HTTP itself, since a consumed response **Verified.** `IoError extends DexpaceError` (`packages/core/src/io/errors.ts:13`); the four I/O leaves — `EndOfStreamError`, `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` — each -extend `DexpaceError` **directly** (lines 29, 49, 65, 80) and are grouped by the `isIoError` predicate -(line 102) rather than by a middle tier. `TransportFailureError extends IoError` (line 126) is the single +extend `DexpaceError` **directly** (lines 29, 51, 67, 83) and are grouped by the `isIoError` predicate +(line 108) rather than by a middle tier. `TransportFailureError extends IoError` (line 132) is the single three-level branch. **Why it cannot be corrected.** `TRANSPORT-20` requires `TransportFailureError` to *be* an `IoError` — the -subtyping is the requirement, not an artifact of modelling. It is also load-bearing: `classify.ts`'s -cause-walk returns retryable for any `IoError`, so the `extends` is what makes a no-response transport -failure retryable with zero edits to the retry layer. A flat sibling would have to be enumerated by hand in -the retry classifier, and again for every transport added later — trading one level of depth for an -open-ended maintenance obligation that the styleguide's own rule exists to prevent. Held at exactly three; -a fourth level is not sanctioned by this entry. +subtyping is the requirement, not an artifact of modelling. It is also load-bearing, and **what it bears is +a boundary, not a category**: `classify.ts`'s cause-walk tests `current instanceof IoError` +(`packages/core/src/retry/classify.ts:90`), and because the tree is flat that branch matches exactly two of +the six classes in `io/errors.ts` — `IoError` itself and `TransportFailureError`. It does **not** match the +four leaves `isIoError` groups. That is the intended reading, decided by audit #67 / #78 and stated here +because it is this deviation's own consequence: the branch means "the wire failed". A send that produced no +response is `RETRY-4`'s unconditionally-retryable condition, `TRANSPORT-20` names `TransportFailureError` as +what carries it, and the `extends` is what routes it to the retry layer with no edit there. A flat sibling +would have to be enumerated by hand in the retry classifier, and again for every transport added later — +trading one level of depth for an open-ended maintenance obligation that the styleguide's own rule exists +to prevent. Held at exactly three; a fourth level is not sanctioned by this entry. + +**What the other four leaves are, and why they stay outside the branch.** `EndOfStreamError`, +`SourceContractViolationError`, `ClosedResourceError` and `AllocationLimitError` are this package's own +contract and lifecycle failures, and every one of them is deterministic on re-send: a closed resource +(`IO-42`) and a source that returned zero bytes for a positive read (`IO-17`) are caller programming errors, +an allocation cap (`IO-9`) is a limit the same request hits again, and `EndOfStreamError` is the +exact-length-copy contract inside `io/` — a *wire* truncation is the transport's to report, as a +`TransportFailureError`, which is the layer that can tell one from a complete short body. `RETRY-2`'s "an +I/O error" is read as that boundary. Widening the branch to `isIoError` would retry all four; only +`EndOfStreamError` was ever a candidate, and `io/` is the wrong layer to decide whether a stream ended early +because the wire broke. Six cases in `packages/core/src/retry/classify.test.ts` pin one answer per class, +each asserting both what `isIoError` says and what the classifier says, so the disagreement is recorded as a +decision rather than an oversight — and re-parenting any leaf under `IoError` turns four of them red. + +> **Anchor correction 2026-09-04 (audit #67 / #68), rule supplied 2026-09-05 (audit #67 / #78).** The line +> numbers in this section were stale on 2026-09-04 and were re-derived then; `classify.ts`'s branch moved +> from `:73` to `:90` on 2026-09-05 when the paragraph above it was written. The substantive point this +> section used to make — "the cause-walk returns retryable for any `IoError`" — read as covering all five +> classes `isIoError` names, and it never did. #68 recorded the gap and left the answer to #78; #78 chose to +> keep `instanceof IoError` and to say what it means, which is the two paragraphs above. Nothing in the +> deviation itself (the three-level branch, and why it stays) ever turned on that answer. ## Deviations recorded outside a phase -Four rows as of 2026-09-02, all from the `docs/work/mvp/2026-09-04-open-items-dissolution.md` register audit (that file's Section V). +Every row here was found by a pass over shipped code rather than produced by a phase. The +`docs/work/mvp/2026-09-04-open-items-dissolution.md` register audit (that file's Section V) opened the +section on 2026-09-02; later reviews and audits append to it. Deliberately uncounted — a stated total is a +number that goes wrong on the next append, which is that file's U10. A row here has no owning phase — it was found by a review, an audit, or a maintenance pass over shipped code. It is recorded on the day it is found rather than waiting for `docs/sdk-design-nodejs/10-…`, which is in a @@ -437,9 +482,28 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended |---|---|---|---|---| | **`HTTP-11`'s range classifications are on `Status` only, not mirrored onto `Response`.** The spec places them on both: `docs/product-spec/04-core-http-domain-model.md:23` reads "Status MUST classify by range … **and a response MUST expose these derived from its status**", and appendix C (`appendix-c-consolidated-normative-requirement-index.md:47`) restates it as "Response MUST expose these same classifications derived from its status." The port ships them once, on `Status`, reachable as `response.status.isSuccess`. Six delegating getters on `Response` would duplicate surface that cannot drift, since there would be one implementation behind both — but the letter of the requirement does name the response, so the reading is recorded rather than assumed | the dissolved register's A3, register audit | 2026-09-02 | `packages/core/src/http/status.ts` carries all six; `packages/core/src/http/response.ts` carries `status` and no classification of its own | not yet in §10 | | **`REDIR-20`'s "fully override" is read as scoped to code/method eligibility, not to the safety mechanics that follow it.** A configured redirect predicate replaces the built-in follow decision; it does **not** bypass userinfo stripping, credential hygiene, the downgrade guard, body replayability, or loop/cap detection. Those are stated as unconditional MUSTs elsewhere in the same chapter and are not "should this kind of redirect be followed" policy — a caller predicate opting to follow a 307 with a single-use body still cannot make that body re-sendable. Genuinely ambiguous wording, decided one way and now recorded as decided | the dissolved register's G4, register audit | 2026-09-02 | `packages/core/src/redirect/decide.ts` consults `settings.predicate` at the eligibility gate and runs every later guard unconditionally; pinned by "the predicate does NOT bypass the safety mechanics" in `decide.test.ts`. Phase 9's conformance sweep was to re-confirm this and closed without doing so | not yet in §10 | -| **Span-shaped tracing carries `OBS-29`'s ordered lifecycle, but not its 1:1 tracer-to-operation binding.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names: a span is started once (`observability/logging-step.ts:427`) and ended once on exactly one of two paths — `span.end()` for success (`:398`), or `span.recordException(error)` then `span.end()` for failure (`:414-415`) — mutually exclusive, once each, in order. **The 1:1 binding is NOT met**, and this row says so plainly rather than implying the requirement is covered: `PIPE-2` fixes the `LOGGING` pillar step *inside* the `RETRY` and `REDIRECT` pipelines, so a span is opened per transmission attempt and per redirect hop, not once per logical operation (the dissolved register's L4 records the same). The per-attempt and retries-exhausted events therefore have no operation-scoped tracer to attach to. Appendix C's own entry (`appendix-c-consolidated-normative-requirement-index.md:509`) anticipates the gap: "This is a documented emission contract; pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced." The wiring stays open as the dissolved register's V2 | the dissolved register's L1/V2, register audit | 2026-09-02 | `packages/core/src/observability/tracing.ts:40-42` (`Tracer`, one method); `observability/logging-step.ts:398,414-415,427` (the lifecycle); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | -| **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` ships fifteen and `context/` three, so the rule is applied where it earns its place | not yet in §10 | +| **`OBS-29` is carried by spans rather than by the named tracer callbacks; both halves are now met, and the operation span is reachable from the public API.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`, and the requirement is discharged under that vocabulary. **Ordering:** `Runtime.send` opens one span before the drive and ends it exactly once, behind an `ended` latch that a throwing `end()` cannot get past (`pipeline/runtime.ts:78-96`), and the per-attempt spans the LOGGING pillar step opens follow the same shape from a single `finally` (`observability/logging-step.ts:469`, then `:457,494`). **1:1 binding:** the span is opened outside every pillar, so a retry's second attempt and a redirect's second hop stay inside the first call's span (`pipeline/runtime.ts:57-67,280-282`); before 2026-09-05 a leaked `enterWith` made the *previous* call's ended span read as active and suppressed the next one, so only the first operation per async context got a span at all — the binding was stated and not delivered. **Caller reachability**, which this row carried as the open half from 2026-09-02 to 2026-09-05, is closed: `PipelineOptions` (`pipeline/builder.ts:32-50`) is `@public`, is the second constructor argument of `PipelineBuilder` (`:71`) and is extended by `StandardResilienceOptions` (`auth/preset.ts:23,115-118`), so `createInstrumentationBundle`'s result has somewhere public to go. What remains a deviation is only the **vocabulary**: a consumer sees `startSpan`/`end`/`recordException`, not `operationStarted`/`operationSucceeded`/`operationFailed`, and appendix C's own note that "pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced" (`appendix-c-consolidated-normative-requirement-index.md:509`) is now out of date for this port | the dissolved register's L1/V2, register audit; anchors re-derived by audit #67 / #68; finished by audit #67 / #80 | 2026-09-02, re-anchored 2026-09-04, closed 2026-09-05 | `packages/core/src/observability/span.ts:54-56` (`Tracer`, one method — declared in `tracing.ts` until 2026-09-04, when audit #67 / #69 moved the inert tracing declarations into their own module to break an import cycle; `tracing.ts` re-exports them, so the public path is unchanged); `packages/core/src/pipeline/runtime.ts:57-67,78-96,206-233,280-282` (the operation span, its single `end()`, and the `run`-scoped stores that keep the binding true across calls); `packages/core/src/pipeline/builder.ts:32-50,71,293` and `packages/core/src/auth/preset.ts:23,115-118` (the public route to a bundle); `observability/logging-step.ts:457,469,494` (the per-attempt span); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | +| **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` and `context/` both carry them, so the rule is applied where it earns its place. Counted qualitatively on purpose — the two figures this cell used to state were wrong by the time anyone read them; re-derive with `grep -rn 'invariant(' packages/core/src/ --include='*.ts' \| grep -v '\.test\.'` | not yet in §10 | | **`PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path, and the port implements `REDIR-22`.** Two MUSTs name the same trigger and prescribe opposite dispositions. `docs/product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `docs/product-spec/10-redirect-handling.md:22` (`REDIR-22`): "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." The port closes, then throws, on three grounds: `REDIR-6` independently fixes the control flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response never *returned* cannot be "returned unclosed"; specific governs general, since §10 of the spec owns the redirect step's lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return unclosed as it requires. **The erratum this needs is proposed below and is not applied here**, because `docs/product-spec/` is frozen and correcting a normative sentence is the specification owner's act, not a maintenance one | the dissolved register's G1, Phase 5b design's Deviation Ledger | 2026-09-04 | `packages/core/src/redirect/redirect-step.ts` closes before throwing, with the reasoning asserted inline in `redirect-step.test.ts`; nothing in the code waits on the erratum | not yet in §10 | +| **`PIPE-37`'s outermost pre-redirect status-mapping step was never built, and until this row nothing recorded that.** `PIPE-37` (MUST) requires a step whose correctness depends on the single terminal response — status-to-typed-error mapping is its own worked example — to occupy the outermost pre-redirect slot, so it runs outside both the redirect and the retry loop. The port ships the *mapping*, but as `statusMappingStep`, a `ResponseStep` on the response-recovery chain (RECOV-15/RECOV-16), not as a pipeline `Step` carrying `stage: 'PRE_REDIRECT'`. The slot itself exists and is installable — `config/clientIdentityStep` occupies it today — so this is a wiring gap, not a missing mechanism. It has an owner in writing and the owner never took it: Phase 4's checklist marked the row ⏳ and said "the obligation lands on whichever phase wires 4b's `statusMappingStep` into a real pipeline — **Phase 5**", and Phase 5 shipped without it, with no deferral carrying the hand-off forward. That is why a code audit found it and no checklist did. **Ledgered rather than implemented.** Installing a `PRE_REDIRECT` mapping step is public pipeline surface with a real behaviour change behind it, and the petstore spike arrives at the same work from the other side: its finding 2 wants a declarative `StatusErrorMap` applied *at* the `toHttpError` call site rather than wrapped around it, so a mapped error class can see a decoded payload instead of raw bytes. Whoever does one does both | audit #67 / #69 | 2026-09-04 | `packages/core/src/recovery/status-mapping.ts:26` (`statusMappingStep(response: Response): Promise` — a `ResponseStep`, with no `StepDescriptor` and no stage); `packages/core/src/pipeline/stage.ts:39` (`PRE_REDIRECT` heads `STAGE_ORDER`) and `packages/core/src/config/client-identity-step.ts:124` (the one shipped step that occupies it); `docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md:141` (the dropped hand-off); `docs/product-spec/08-execution-pipelines.md:26` (PIPE-37 shares that line with PIPE-25/36/38); `examples/petstore/FINDINGS.md:96-97` | not yet in §10 | +| **`REDIR-3`'s eligibility test reads the CURRENT hop's method, where the spec says the ORIGINAL request's.** `isEligibleByCode` is handed `currentRequest.method`, the method of the request being redirected at this hop. The two readings agree on every chain but one: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 or 302 arriving on that rewritten hop is then followed under the default {GET, HEAD} set, where the literal reading would refuse it because the request that started the chain was a POST. **Kept, deliberately.** The rewritten GET is idempotent and carries no body — REDIR-5 dropped it — so the literal reading buys no wire safety here, only a refusal; and refusing would make `allow303` half-useful, opting into the rewrite but not into anything the rewritten request can reach. Every other guard on the hop is unaffected, since eligibility is step 3 of eight and the userinfo strip, downgrade guard, replayability gate, loop detection and hop cap all run after it regardless. Switching to the literal reading stays a narrow, mechanical change — thread the seed request's method through `RedirectContext` — if a later reading of §10 wants it | audit #67 / #69 | 2026-09-04 | `packages/core/src/redirect/decide.ts:241` passes `currentRequest.method` into `packages/core/src/redirect/codes.ts:69`'s `eligibility.allowedMethods.has(method)`; `docs/product-spec/10-redirect-handling.md:8` is the "ORIGINAL request method" wording. Pinned by "a 303-rewritten GET makes a following 301 eligible under the default method set" in `packages/core/src/redirect/decide.test.ts`, which goes red the moment the reference point moves | not yet in §10 | +| **`PAGE-19`'s own conformance fixture `; rel=next` does not end the stream on this platform — it is followed.** The requirement's normative sentence is "a target that cannot resolve into a valid URL MUST be treated as end-of-stream", and its illustrative conformance note offers `` as an instance. Under WHATWG `URL` — the resolver `strategies.ts` uses, and the only RFC 3986 resolver available without a runtime dependency (SEAM-1) — a supplied base makes that string a perfectly ordinary relative *path* reference: it resolves to `/repo/not%20a%20url`. It resolves, so the port follows it. **The normative half is satisfied exactly as written**: a target that genuinely fails `new URL(target, base)` returns the page with no next request and throws nothing. Only the fixture disagrees, and it disagrees because it was written against a resolver that rejects a space. **Rejected:** an ad-hoc "looks unparseable" heuristic in front of the resolver, which would have to guess at strings RFC 3986 defines, and would make the followable-relative-reference clause of the same requirement wrong instead | audit #67 / #69 | 2026-09-04 | `packages/core/src/pagination/strategies.ts:115-120` (the `new URL(target, response.request.url)` resolve, with the `catch` returning `pageInfo(items)`); `docs/product-spec/12-pagination.md:52` is the conformance note. Both halves pinned in `packages/core/src/pagination/strategies.test.ts` — "the spec's `` fixture is a RELATIVE reference, so it is followed" and "an unresolvable target ends the stream rather than throwing" | not yet in §10 | +| **`Request.equals` compares the body by reference identity, where `HTTP-46` says by value.** `HTTP-46` requires equality to compare "method, headers, and body by value", with only the URL singled out for textual comparison. The port compares method, `url.href` and headers by value as required, and the body with `===`. **Why it stays.** `Body` is a lifecycle object, not a value: a `StreamBody` is single-use (BODY-9) and reading its bytes to compare them consumes it, which would make `equals` destructive — an equality operator that empties its operands is worse than one that under-reports. The variants that *could* compare cheaply (`ByteArrayBody`, `StringBody`) would give a comparison whose cost and semantics depend on which variant a caller happened to build, which is the drift `HTTP-1`'s value-model rules exist to prevent. Reference identity is sound in the safe direction — it never reports two different bodies equal — and it is what `Request.equals`'s own TSDoc has always said it does. Archived as blocked in the dissolution record and in neither §10 nor the register; recorded here so it is somewhere a reader will look | audit #67 / #69 | 2026-09-04 | `packages/core/src/http/request.ts:137` (`this.#body === other.#body`, beside the by-value method, `url.href` and `headers.equals` comparisons on 134-136); `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:82` | not yet in §10 | +| **`IO-13`'s "symmetric write-side encodings" ship for UTF-8 and ISO-8859-1 only; every other charset throws on write.** The read side stays fully general through `TextDecoder`. The write side does not, because `TextEncoder` is UTF-8-only — there is no `TextEncoder('iso-8859-1')` — and SEAM-1 forbids adding an encoding library to `@dexpace/core`. ISO-8859-1 is therefore hand-rolled as the direct code-point-to-byte map, which is also what lets the decode side round-trip it: WHATWG maps the label `iso-8859-1` onto windows-1252, so delegating would break the symmetry `IO-13` is about. Any other label raises `IoError` naming the charset rather than silently re-encoding as UTF-8 and corrupting the bytes on the wire. `IO-13`'s own conformance note names ISO-8859-1 as the non-UTF-8 case, so the two shipped encodings are the two the requirement exercises. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** — it never reached §10 or this file, which is the gap this row closes | audit #67 / #69 | 2026-09-04 | `packages/core/src/io/text-codec.ts:32-51` (`encodeText`, and the `unsupported write charset` throw at :36-39); `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:406` | not yet in §10 | +| **`BODY-9`'s mark/reset replay path for a stream-backed body is not built: `StreamBody` is always single-use.** `BODY-9` is a SHOULD, and it is conditional — replayable "when and only when the stream supports mark/reset". Node's `ReadableStream` has no generic mark/reset to support, so the condition is never met and the SHOULD's own fallback ("otherwise it MUST be single-use") is the branch that applies. `StreamBody.replayable` is a hardcoded `false`, which every consumer already reads: `decide.ts` fails a non-303 redirect carrying one (REDIR-6), and the retry pillar refuses to re-send it. A caller who wants replay materializes first or uses `byteArrayBody`. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/stream-body.ts:24` (`readonly replayable = false`); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:441` | not yet in §10 | +| **`BODY-34`'s one shared preview cap covers the two logging tees only, not `toHttpError`'s error-body capture.** Read literally as "every in-memory capture in the package", `BODY-34` would put the request-side tee, the response-side drain and `toHttpError`'s error buffer behind one configurable value. They cannot share one: `HTTP-52` **fixes** the error-body cap at 1 MiB, and a spec-fixed constant cannot also be the configurable shared setting. The two capture sites `BODY-34` actually names — the request-side tee-capture and the response-side drain-on-first-access — do share one cap, so the requirement's own enumeration is satisfied; only the wider reading is not. `http-status-error.ts` states the split at the constant rather than leaving it to be inferred. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/http-status-error.ts:17-19` (`ERROR_BODY_CAP_BYTES`, with the "Deliberately NOT BODY-34's shared preview cap" note); `packages/core/src/body/response-body-logging.ts:47` and `packages/core/src/body/response-body-logging.ts:105-115` (the shared `cap` and the bounded drain that honours it); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:447` | not yet in §10 | +| **`IO-38`'s cross-thread close visibility has no subject on this platform and is recorded as not applicable, not as satisfied.** The requirement presupposes that a source or buffer instance can reach a second thread, so that closing it there invalidates a slice being read here. None can. Class instances are not structured-cloneable at all — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: it holds a `ReadableStreamDefaultReader`, which is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject there either. This row exists because "not applicable" and "done" look identical in a checklist and are not the same claim. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** | audit #67 / #69 | 2026-09-04 | `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:401`; `packages/core/src/io/buffered-source.ts:47` (the reader it holds); no `worker_threads`, `postMessage` or `structuredClone` call exists anywhere in `packages/core/src/` | not yet in §10 | +| **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:203` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:425-441` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | +| **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | +| **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | +| **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 | +| **`AUTH-22`'s "emit cnonce/nc/qop only when qop is negotiated" is not applied to `cnonce` for a `-sess` algorithm.** A `-sess` HA1 is `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so the client nonce is an *input to the hash* for `MD5-sess` and `SHA-256-sess` whatever `qop` the challenge offered. The port implemented AUTH-22 to the letter: it drew a fresh cnonce, folded it into HA1, and then omitted it from the header whenever `qop` was absent — a response no server can verify, because it has no way to reconstruct HA1. AUTH-30 bounds the re-challenge replay to one 401, so every such exchange simply failed. **`cnonce` is now emitted for any `-sess` algorithm; `nc` and `qop` stay conditional exactly as AUTH-22 says**, because RFC 2069's response input is `H(HA1:nonce:HA2)` and carries no nonce count, so emitting one would advertise a count the response was not computed over. RFC 7616 §3.4 states the wider rule outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's clause is RFC 2617's RFC 2069-compatibility form, written before `-sess` existed, and the requirement's own AUTH-15 mandates both `-sess` algorithms, so the two sentences cannot both be followed. *Rejected:* declining a `-sess`-without-`qop` challenge instead, which turns every such server into a guaranteed 401 for no security gain, when the value the server needs has already been computed | audit #67 / #74 | 2026-09-05 | `packages/core/src/auth/digest.ts:345-350` (the `-sess` HA1 that consumes the cnonce) against `:405-408` (`buildHeaderValue`, where the `else if` now emits it); `docs/product-spec/11-authentication.md:18` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:357` are AUTH-22's wording. Pinned by the `digestHandler -sess without qop (AUTH-17/AUTH-22)` block in `packages/core/src/auth/digest.test.ts` — one row asserting the header carries `cnonce` and neither `nc` nor `qop`, one recomputing the response from the header's OWN cnonce so a value drawn twice would fail — and by the `MD5-sess, no qop` vector in the same file | not yet in §10 | +| **`RETRY-44`'s "downstream chain" is read as everything BELOW the retry point, which in the recovery stack excludes the request chain.** The requirement has two clauses: each attempt re-executes the downstream chain with fresh per-attempt state, and "upstream steps MUST NOT mutate the shared in-flight request between attempts". The port originally read the first clause as covering the *whole* recovery chain and re-ran `RequestRecoveryChain.apply()` per attempt, with a test that said so by name. That makes `packages/core/src/recovery/idempotency-key.ts` generate a fresh key on every attempt, so three attempts of one logical request reach the server as three unrelated writes — the precise failure `RECOV-32` exists to prevent, and the opposite of what that step's own `@public` TSDoc promises. **The chain is now applied once, above the loop; each attempt re-executes transport plus response chain over `stampAttempt`'s fresh copy of the prepared request.** Under this reading both clauses hold and the second holds *by construction*: upstream steps cannot mutate the in-flight request between attempts because they no longer run between attempts. The pillar stack is untouched — there "downstream" is the forked continuation (`ctx.fork()`), and `retryStep` still re-drives it per attempt. *Rejected:* memoizing the key on the template (a `WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice would replay the key and have the server drop a genuine second call. *Rejected:* re-running the chain over the *prepared* request each attempt — the chain would read its own output, which is clause two's mutation in different clothes, and every shipped and caller-written step would have to be proven idempotent. One consequence recorded rather than assumed: the re-send gate (`RETRY-5`/`RECOV-18`) now judges the prepared request rather than the caller's, which is what a retry would actually re-send | audit #67 / #73 | 2026-09-05 | `packages/core/src/retry/retry-dispatch.ts:83-88` (the chain applied once, then `runWithRetry`) against `:27-33` (the per-attempt half); `packages/core/src/recovery/orchestrator.ts:62` (`prepareRequest`) and `:117` (`dispatchPrepared`); `packages/core/src/retry/engine.ts:243` is the gate that now reads the prepared request. `RETRY-44`'s wording is `docs/product-spec/09-retry-and-resilience.md:35` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:306`. Pinned by `packages/core/src/retry/retry-dispatch.test.ts:126` (chain applied once), `:169` (one `generate()`, one key on three sends), `:201` (the `RETRY-38` ordinal varies while the key does not) and `:227` (a request-chain throw is not retried and meets the recovery phase exactly once) | not yet in §10 | +| **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | +| **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | +| **`OBS-35`'s "MUST NOT bake in a default config key name" is satisfied by making the key configurable, not by removing the default.** `OBS-35` (SHOULD) asks for a tolerant, layered log-level resolution and adds one MUST: no baked-in default key name. The port ships `CFG_KEY_LOG_LEVEL` (`DEXPACE_LOG_LEVEL`) as `CFG-14`'s well-known key and, until 2026-09-05, read it unconditionally. It is now `LoggingStepSettings.configKey`'s default: a caller names their own key and the resolution is otherwise identical. **Why the default stays.** A required key would mean no caller gets ambient granularity without naming one first, which trades a MUST about *naming* for a worse default experience, and `CFG-14` — which this port also implements — exists precisely to standardise the name. The layered resolution itself is `CFG-1`'s (override → environment → normalised property → default) and is tolerant as the requirement asks. **A second, quieter half:** the process-wide configuration slot starts empty (`CFG-13`), so no key of any name resolves until a host calls `setGlobalConfiguration(defaultConfiguration())`. That is deliberate — defaulting the slot to a configuration that reads `process.env` would make an import-time environment read the SDK's default behaviour — and it is now documented as the required wiring rather than left to be discovered | audit #67 / #80 | 2026-09-05 | `packages/core/src/observability/logging-step.ts:64-79,94-104` (the setting and the resolution); `packages/core/src/config/configuration.ts:311,354-358,368` (`CFG_KEY_LOG_LEVEL`, `defaultConfiguration`, the empty default slot); `docs/sdk-documentation/pipelines.md` "Turning logging on from the environment"; `docs/product-spec/15-instrumentation-and-observability.md:66` (the requirement) | not yet in §10 | +| **`CFG-22`'s SOCKS proxy types are resolved by the configuration layer and supported by neither shipped transport; the refusal is at the transport factory, and `ProxyType` keeps them.** `CFG-22` (MUST) requires the proxy model to carry "the proxy protocol type (HTTP, SOCKS4, SOCKS5)", and the port implements it in full: `ProxyType` is `'http' \| 'socks4' \| 'socks5'`, and `resolveProxyOptions` maps `ALL_PROXY`/`HTTPS_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto it. Nothing can then send over one. `@dexpace/transport-undici` builds undici's `ProxyAgent`, which is an HTTP `CONNECT` tunnel reading its `uri` as a URL, and `@dexpace/transport-fetch` ships no `proxy` option at all because Node's bare global `fetch` exposes no proxy hook outside undici internals. So a configuration that resolves cleanly has no transport that can honour it. **What changed on 2026-09-05 (audit #67 / #81).** Until then the discovery was `new ProxyAgent({uri: 'socks5://…'})` throwing undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` out of a public factory — untyped, undocumented, and outside the SDK's error vocabulary. `undiciTransport()` now refuses `proxy.type !== 'http'` at construction with a `TypeError` naming the type, before any dispatcher is allocated, deliberately outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable (RETRY-2). That is `TRANSPORT-30`'s "make the limitation discoverable rather than silently misbehaving" applied at the earliest point that can. **Why `ProxyType` still admits `socks4`/`socks5`.** Narrowing a `@public` union is a breaking change and therefore a release-pass decision, which this run is not taking (the run's release machinery is suspended); and `CFG-22`'s MUST is about the *model*, which would then no longer satisfy it. The honest state is a configuration layer that is complete and a transport layer that is not, which is what this row records. A future transport — a `node:net` SOCKS dialer, or a `ProxyAgent` replacement — closes the gap without a model change. | audit #67 / #81 | 2026-09-05 | `packages/core/src/config/proxy.ts:34` (`ProxyType`), `:372-380` (the scheme map); `packages/transport-undici/src/undici-transport.ts:147,160-167,201` (the supported type, the refusal, and where it runs); `packages/transport-fetch/src/fetch-transport.ts:79-82` (no `proxy` option, and why); `docs/product-spec/16-configuration.md:42` (`CFG-22`); `docs/product-spec/17-transport-adapter-conformance-contract.md:48` (`TRANSPORT-30`) | not yet in §10 | +| **`TRANSPORT-20`'s "any transport failure that produced no HTTP response" is read as an exchange that failed, not as a request the native client refused to make.** `TRANSPORT-20` (MUST) names four instances — connection refused, DNS/TLS failure, peer reset, connect/read timeout — and requires the retryable `TransportFailureError`. A scheme the client will not speak (`ftp://`), a forbidden method (`CONNECT`), a method that is not a token, a body on a GET: all of them also produce no HTTP response, so the literal reading makes them retryable too. The port refuses that. `retry/classify.ts:90` is an allow-list over `instanceof IoError`, so retryable would mean the caller's entire retry budget spent re-proving a URL that cannot change between attempts, and the requirement's own enumeration is four ways an *exchange* fails, not four ways an argument is rejected. Such a refusal surfaces as a bare `TypeError` carrying the native error as `cause`, outside the `IoError` tree, which is the same class both transports already raise for a misconfiguration caught at construction. **What changed on 2026-09-05 (audit #67 / #82).** The reading is not new — `@dexpace/transport-undici` has applied it to undici's `UND_ERR_INVALID_ARG` / `UND_ERR_NOT_SUPPORTED` since Phase 8a — but it was recorded only in that phase's checklist, and `@dexpace/transport-fetch` did the opposite for the identical condition: every native rejection became `TransportFailureError`. The decision now lives in one table in `@dexpace/transport-shared` that both adapters call, so the two cannot answer differently again, and `docs/sdk-documentation/write-a-transport.md`'s rule 4 tells a third transport to use it. **The MUST is still the default.** The table is an allow-list of three positive recognitions and everything else falls through to retryable; `'bad port'` is excluded by name, because port 1 is on WHATWG's blocked list and so `TRANSPORT-20`'s own dead-port conformance probe arrives with that reason on Node's `fetch`. | audit #67 / #82 | 2026-09-05 | `packages/transport-shared/src/dispatch-classification.ts:17,40,81,123` (the two tables, the predicate and the mapping, with the `'bad port'` exclusion documented at `:35`); `packages/transport-fetch/src/fetch-transport.ts:340` and `packages/transport-undici/src/undici-transport.ts:308,317` (the two call sites); `packages/core/src/retry/classify.ts:89-90` (the allow-list this is answerable to); `docs/product-spec/17-transport-adapter-conformance-contract.md:38` (the requirement); `docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md:57` (where the reading was recorded before this row). Pinned by `packages/transport-conformance/src/run-suite.ts:387` on both adapters and by `tests/node-conformance/transport.test.mjs:351`, whose runtimes report the same refusal in entirely different shapes | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) diff --git a/docs/sdk-documentation/auth.md b/docs/sdk-documentation/auth.md index 8f17c2b..8873c4e 100644 --- a/docs/sdk-documentation/auth.md +++ b/docs/sdk-documentation/auth.md @@ -16,8 +16,12 @@ interface AuthTiers { The most specific tier that is set wins (`AUTH-4`). `perCall` comes from `RequestOptions.newBuilder().auth(descriptor)`, which is how one call opts out of, or into, something -different from the client default. Only the *selection* is tiered; whether a selected requirement can -be satisfied at all is `AUTH-6`, and failing it is an `AuthResolutionError`. +different from the client default. **`operation` comes from `RequestOptions.operationAuth`**, set with +`RequestOptions.newBuilder().operationAuth(descriptor)` — both per-call slots ride on the same +`RequestOptions`, and the AUTH pillar step reads them into the two tiers +(`packages/core/src/auth/auth-step.ts:252,756`; `packages/core/src/auth/resolve.ts:19` names the mapping). `client` is the +configured default. Only the *selection* is tiered; whether a selected requirement can be satisfied at +all is `AUTH-6`, and failing it is an `AuthResolutionError`. An `AuthDescriptor` is a list of `AuthRequirement`s, each a scheme plus optional scopes and parameters. Both are built by factory, never as an object literal: @@ -39,8 +43,8 @@ The factories validate and freeze (`AUTH-3`). A descriptor containing a `NO_AUTH ```typescript interface AuthCredentialSet { readonly bearer?: {provider: TokenProvider; marginMs?: number}; - readonly basic?: {username: string; password: string}; - readonly digest?: {username: string; password: string; algorithmPreference?: DigestAlgorithm[]}; + readonly basic?: BasicCredential; + readonly digest?: DigestCredential; readonly apiKey?: {credential: ApiKeyCredential | NameKeyCredential; headerName?: string; prefix?: string}; } ``` @@ -49,18 +53,37 @@ The five schemes are `OAUTH2`, `API_KEY`, `BASIC`, `DIGEST` and `NO_AUTH`. A req scheme; the credential set supplies the material for it. A requirement with no matching credential is an `AuthResolutionError` at send time, not a silent unauthenticated request. -**`ApiKeyCredential`, `NameKeyCredential` and `BearerToken` are nominal, not structural.** Each carries -a `#private` field, so no caller-side object literal is assignable to them and the validation in each -factory cannot be routed around. They also override `toString()` and Node's inspect symbol, so a -credential cannot leak into a log line or a stack trace by accident. +**Every credential type is nominal, not structural.** `ApiKeyCredential`, `NameKeyCredential`, +`BearerToken`, `BasicCredential` and `DigestCredential` each carry a `#private` field, so no +caller-side object literal is assignable to them and the validation behind each cannot be routed +around. All five override `toString()` and Node's inspect symbol, so a credential cannot leak into a +log line or a stack trace by accident. ```typescript -import {ApiKeyCredential, createBearerToken} from '@dexpace/core'; +import { + ApiKeyCredential, + BasicCredential, + createBearerToken, + DigestCredential, +} from '@dexpace/core'; + +const credentials = { + basic: new BasicCredential('alice', 'super-secret'), + digest: new DigestCredential('bob', 'super-secret', ['SHA-256', 'MD5']), + apiKey: {credential: new ApiKeyCredential('super-secret'), headerName: 'X-Api-Key'}, +}; String(new ApiKeyCredential('super-secret')); // 'ApiKeyCredential{key=***}' String(createBearerToken('t', 1)); // 'BearerToken{token=***, expiresAt=1}' — the expiry survives +String(credentials.basic); // 'BasicCredential{username=alice, password=***}' ``` +There is no `password` property to read back, by design: `BasicCredential` and `DigestCredential` +shipped as plain `{username, password}` records until 2026-09-04, and a plain property is reachable +through `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect` — so +`util.inspect(credentials)` printed the password in clear beside `ApiKeyCredential{key=***}`. The +username and the Digest algorithm preference stay visible; `AUTH-8` permits non-secret fields to. + The inspect symbol matters as much as `toString`: `console.log(credential)` and `util.inspect` do not route an object argument through `toString`, so both are overridden (`AUTH-8`). @@ -102,6 +125,14 @@ request is dispatched (`AUTH-28`). There is no option to disable it. `NO_AUTH` n is why an auth-less `standardResilience()` installs a `NO_AUTH`-only step rather than no step at all — the pillar slot stays filled and the behaviour stays uniform. +**Once guarded, always guarded.** A `challengeHook` may return any request it likes, including one +whose URL has been downgraded to `http://`. When the outbound pass ran the guard, the replay is +guarded too — unconditionally, without inspecting a single header name, because +`ApiKeyCredentialConfig.headerName` lets this step stamp a header no fixed list would contain. On a +`NO_AUTH` hop, which is never guarded outbound, a replacement carrying `Authorization` or +`Proxy-Authorization` still trips the guard. A genuinely credential-free re-issue over `http://` is +what `XCUT-16` explicitly permits, and it still proceeds. + ## Challenges ```typescript @@ -130,6 +161,23 @@ when the deferral register was dissolved on 2026-09-04). **Basic and Digest never stamp preemptively.** They react to a challenge. That is an interpretation of `§11` rather than a stated requirement, and it is ledgered as one. +**Every challenge the response offered is considered, not just the first.** A server may send +`WWW-Authenticate` (or `Proxy-Authenticate`) once with several comma-separated challenges, or once per +challenge — RFC 9110 §5.3 permits both, and RFC 7616 §3.3 recommends the repeated form for Digest +algorithm discovery. Which of the two shapes reaches the step is partly the transport's accident: +`@dexpace/transport-fetch` comma-joins repeated values because WHATWG `Headers` does, +`@dexpace/transport-undici` keeps them apart. The step reads *every* value and parses each on its own, +so the offer is the same list either way, and `@dexpace/transport-conformance` carries a row asserting +that. A challenge the SDK cannot answer — an unsupported algorithm, an `auth-int`-only `qop`, a realm +this client cannot echo back, an empty `realm` or `nonce` — is declined and the next one is tried; a +`401` offering nothing answerable surfaces unchanged and unclosed. + +**Digest `-sess` sends `cnonce` whether or not `qop` was negotiated.** `MD5-sess` and `SHA-256-sess` +fold the client nonce into HA1 (RFC 7616 §3.4.2), so a server handed a `-sess` response without one +cannot verify it. `nc` and `qop` stay conditional on a negotiated `qop`. `AUTH-22`'s literal wording +says all three are conditional; the departure is a row in +[`deviations.md`](../deviations.md). + ## Redirects and credentials Credentials are attached at an origin and must not follow a request to a different one. The redirect diff --git a/docs/sdk-documentation/errors.md b/docs/sdk-documentation/errors.md index 3be0a84..fd62950 100644 --- a/docs/sdk-documentation/errors.md +++ b/docs/sdk-documentation/errors.md @@ -112,6 +112,11 @@ export async function call(): Promise { the two apart at the point of abort — which is exactly how a transport decides which of the two errors to raise. +**Neither narrowing changes when a retry pillar is installed.** What `retryStep` throws once it gives +up is the final attempt's own error, unwrapped, so the two `instanceof` checks above read the same at +`maxAttempts: 1` and at `maxAttempts: 3`. The earlier attempts ride beside it and are read with +`retryAttempts(error)` — see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars). + ## Narrowing helpers Three predicates exist for the cases where `instanceof` on a union is tedious: @@ -162,12 +167,30 @@ Eight classes that a `@throws` tag named but no package exported were promoted t | `CursorAlreadyAdvancedError` | a step reusing an already-invoked `next()`/`fork()` continuation | | `EndOfStreamError` | a `BufferedSource` read that required more bytes than the source delivered | +**The two redirect errors carry redacted messages and raw properties, and the split is deliberate.** +`SchemeDowngradeError.message` and `NonReplayableBodyError.message` name their URLs in the redacted +form `OBS-11`/`OBS-12` define — userinfo as `***:***@`, every non-allow-listed query value as `***` — +because a message is rendered by every logger, every `cause` chain and every consumer `console.error`, +and `http.redirect.rejected` hands the error straight to `LogEvent.cause()`. The unredacted URLs stay +on `fromUrl` / `toUrl` / `targetUrl`, which is where program code reads them: + +```ts +try { + await client.send(request); +} catch (error) { + if (error instanceof SchemeDowngradeError) { + console.error(error.message); // https://***:***@api.example.com/v1?token=*** + retargetTo(error.toUrl); // the real URL, for code rather than for a log + } +} +``` + Two more joined them on the same date, both from `XCUT-8`'s "never fabricate a successful exception": | Error | Raised by | |---|---| | `HttpStatusValidationError` | `new HttpStatusError(status, …)` when `status` is not an integer in 400–599. The constructor validated nothing before, so a consumer could build an `HttpStatusError` claiming a `200`. `toHttpError` is the total form — it returns `null` instead of throwing | -| `RetryDiscardedResponseError` | the retry engine's suppressed trail, for a response it discarded whose status is outside 400–599. Reachable only if you widen `RetrySettings.retryableStatuses` to include a non-error code; the trail previously said `HttpStatusError` for it, which claimed an HTTP failure that had not happened | +| `RetryDiscardedResponseError` | the retry engine's prior-attempt trail (`retryAttempts()`), for a response it discarded whose status is outside 400–599. Reachable only if you widen `RetrySettings.retryableStatuses` to include a non-error code; the trail previously said `HttpStatusError` for it, which claimed an HTTP failure that had not happened | All of them descend from `DexpaceError`, so the broad catch works too. diff --git a/docs/sdk-documentation/http.md b/docs/sdk-documentation/http.md index b570319..4ef7dc6 100644 --- a/docs/sdk-documentation/http.md +++ b/docs/sdk-documentation/http.md @@ -110,8 +110,9 @@ const options = RequestOptions.newBuilder() `RequestOptions.EMPTY` is the shared no-op instance. A step reads it as `ctx.options`, and a transport receives it as `send()`'s second argument. Both range checks are the **full** range, not only the lower bound: `maxRetries` rejects anything that is not a non-negative integer, and -`timeoutMs` rejects zero, negatives, `Infinity` and `NaN` alike (`HTTP-35`). A fractional -`timeoutMs` is accepted — a timeout is a duration, not a count. +`timeoutMs` rejects zero, negatives, `Infinity`, `NaN`, a fractional value and anything above +`2**32 - 1` (`HTTP-35`) — the range is `AbortSignal.timeout()`'s, the only one a transport can +honor. `auth` on the builder is the **per-call** auth tier, the highest-precedence one; see [`auth.md`](./auth.md). diff --git a/docs/sdk-documentation/pipelines.md b/docs/sdk-documentation/pipelines.md index 1336c5e..93fb6e7 100644 --- a/docs/sdk-documentation/pipelines.md +++ b/docs/sdk-documentation/pipelines.md @@ -144,6 +144,58 @@ All three carry the same `InstrumentationBundle`, so trace and span identity sur `AsyncLocalStorage`-based, which is why `@dexpace/rx` installs no RxJS scheduler — adding `observeOn`/`subscribeOn` downstream makes reinstating the context the caller's job. +**What a call leaves behind is nothing.** `send()` scopes both async-scoped stores with +`AsyncLocalStorage.run`, so the moment it settles — resolved or rejected — the active span and the +diagnostic fields are what they were before it. That is a guarantee about *your* context, not only the +pipeline's: an application log emitted after `await client.send(...)` carries no `trace.id` from it. +Inside the call the usual rules apply, and a step that pushes fields with the handle form +(`activateSpanForCorrelation`) should still close its scope in the same continuation that opened it. + +## Instrumenting a pipeline + +Both constructors take an options bag that the built pipeline carries into every call: + +```typescript +import { + PipelineBuilder, + createInstrumentationBundle, + standardResilience, + type PipelineOptions, + type Runtime, + type Tracer, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +declare const otelTracer: Tracer; // whatever your tracing library hands you + +const instrumented: PipelineOptions = { + instrumentation: createInstrumentationBundle(() => otelTracer), + operationName: 'GetUser', +}; + +// `StandardResilienceOptions` extends `PipelineOptions`, so the preset takes the same two fields +// beside its per-pillar ones. +export const preset: Runtime = standardResilience(fetchTransport(), instrumented); +export const handBuilt: Runtime = new PipelineBuilder( + fetchTransport(), + instrumented, +).build(); +``` + +- **`instrumentation`** is the bundle every context of the call carries, and its `tracerFactory` is + what opens spans. `send()` asks it for `'http.client.operation'` once per call and opens **one** + span there (`OBS-29`), outside every pillar — a retry attempt and a redirect hop stay inside it. The + LOGGING pillar asks again per transmission, and those spans are its children. Omitted, the pipeline + carries the no-op bundle (`CTX-15`) and opens no span at all. +- **`operationName`** is `CTX-16`'s advisory label. It is carried unchanged across every promotion, + readable from a custom step as `ctx.context.operationName`, and used to name the tracer the LOGGING + pillar asks for. It never influences the request, the dispatch decision, or the store key. + +Both are per-pipeline, not per-call: build a second pipeline for a second operation name. +`PipelineBuilder.seedFrom(runtime, 'flatten')` carries them over, since the flattened builder replaces +the runtime it seeded from; `'nest'` does not need to, because the seeded runtime is still there, +driving its own contexts as the terminal transport. + ## The four shipped pillars | Pillar | Factory | Key settings | @@ -151,10 +203,52 @@ All three carry the same `InstrumentationBundle`, so trace and span identity sur | Retry | `retryStep(options?)` | `maxAttempts`, `retryableStatuses`, `totalTimeoutMs`, `attemptHeaderName`, backoff (`initialDelayMs`, `multiplier`, `maxDelayMs`, `jitter`, `fixedDelayMs`), injectable `clock`/`random` | | Redirect | `redirectStep(overrides?)` | `maxHops`, `allowedMethods`, `allow303`, `allowSchemeDowngrade`, `locationHeader`, `predicate` | | Auth | `authStep(settings)` | `credentials`, `tiers`, `challengeHook`, `bearerMarginMs` — see [`auth.md`](./auth.md) | -| Logging | `loggingStep(settings?)` | `granularity`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | - -Retry and redirect are worth two notes each, because both surprise people: - +| Logging | `loggingStep(settings?)` | `granularity`, `configKey`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | + +Retry and redirect are worth a few notes each, because both surprise people: + +- **What retry throws is the last attempt's own error.** The class you catch does not depend on how + many attempts ran: a refused connection is a `TransportFailureError` whether `maxAttempts` was 1 or + 3, and an abort that lands during a backoff wait is a `CancellationError` (`XCUT-1`). The earlier + attempts are not thrown away — `retryAttempts(caught)` returns them, oldest first, with the error + you passed in excluded from its own trail (`RETRY-34`): + + ```typescript + import { + retryAttempts, + TransportFailureError, + type Request, + type Runtime, + } from '@dexpace/core'; + + declare const runtime: Runtime; + declare const request: Request; + + export async function send(): Promise { + try { + await runtime.send(request); + } catch (error) { + if (error instanceof TransportFailureError) { + for (const prior of retryAttempts(error)) { + console.error('an earlier attempt failed:', prior); + } + } + throw error; + } + } + ``` + + One entry per attempt that failed *before* the one you caught — which is not an attempt count, so + resist writing `length + 1`. The surfaced error is an attempt's own only when it came from one, and + sometimes it did not: a cancellation or timeout the engine observes between attempts is synthesized + at that gate, and so is a failure from stamping the attempt header, which runs before the request + goes out. On those paths the trail already covers every send. Narrowing the catch does not help — + a timeout signal is mapped to `TransportFailureError`, the same class a real send failure raises. + + The trail is a side table keyed by the error, not a property on it, so nothing is added to an + object you may not own; an error that never went through a retry loop answers with an empty list. + Until 2026-09-05 the pillar wrapped its terminal failure in a `SuppressedError` instead, which made + the surfaced class a function of the attempt budget. - **Retry pacing honours the server.** `Retry-After`, `X-RateLimit-Reset` and friends are parsed in a fixed precedence and win over computed backoff. Every computed delta is clamped to a 365-day ceiling that `RETRY-18` mandates — so a server that sends `X-RateLimit-Reset` in milliseconds @@ -169,6 +263,47 @@ Retry and redirect are worth two notes each, because both surprise people: - **A non-replayable body ends a redirect.** `PIPE-40` and `REDIR-22` disagree about what should happen; this port closes the response and throws (`docs/work/mvp/2026-09-04-open-items-dissolution.md` G1). +### Turning logging on from the environment + +`loggingStep()` with no `granularity` resolves one from layered configuration (`OBS-35`), and the +default is `'none'`. What it reads is the process-wide configuration slot — **which starts empty**. +`CFG-13` makes that slot last-write-wins with no ambient default, so nothing in this SDK reads +`process.env` until you say so, and `DEXPACE_LOG_LEVEL=headers` on its own changes nothing: + +```typescript +import { + defaultConfiguration, + loggingStep, + setGlobalConfiguration, + standardResilience, + type Runtime, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +// Once, at start-up. `defaultConfiguration()` is the production wiring: its environment seam reads +// the live `process.env` on every lookup, and its property seam finds nothing (Node has no ambient +// key/value store distinct from the environment). +setGlobalConfiguration(defaultConfiguration()); + +// Now `DEXPACE_LOG_LEVEL=headers` reaches the step. `configKey` renames the variable it reads — +// `OBS-35` says an SDK must not bake a key name in, and `DEXPACE_LOG_LEVEL` is only the default. +export const client: Runtime = standardResilience(fetchTransport(), { + logging: {configKey: 'ACME_HTTP_LOG_LEVEL'}, +}); +export const step = loggingStep({configKey: 'ACME_HTTP_LOG_LEVEL'}); +``` + +Parsing is tolerant either way: `' Headers '` and `'HEADERS'` both resolve to `'headers'`, and an +absent, empty or unrecognised value falls back to `'none'`. An explicit `granularity` in the settings +wins over both, and is the right choice for a library that does not want to read a host application's +environment at all. + +Body-preview capture is best-effort and says so when it fails: a request body that cannot be probed +or a response body that errors mid-drain emits `http.instrumentation.bodyCaptureFailed`, carrying the +direction and the cause, and the request still completes (`OBS-20`). Like every +`http.instrumentation.*` diagnostic it emits at `verbose`, so a logger that filters that level will +not show it. + ## Testing a pipeline Nothing here needs a socket. A `Transport` is two methods, so the test double is a literal: diff --git a/docs/sdk-documentation/write-a-paging-strategy.md b/docs/sdk-documentation/write-a-paging-strategy.md index d2bc52c..5913cbf 100644 --- a/docs/sdk-documentation/write-a-paging-strategy.md +++ b/docs/sdk-documentation/write-a-paging-strategy.md @@ -13,9 +13,18 @@ interface PageInfo { } ``` -Given the page that just arrived and the request template the walk started from, produce this page's -items and the request that fetches the next one. `undefined` for `nextRequest` is how a walk ends — -there is no separate "done" flag to keep consistent with it. +Given the page that just arrived and the request that fetched it, produce this page's items and the +request that fetches the next one. `undefined` for `nextRequest` is how a walk ends — there is no +separate "done" flag to keep consistent with it. + +**`template` is not the request the walk started from.** The glossary calls it "the original request +template", but the engine passes the request it sent for *this* page and then makes your `nextRequest` +the following hop's template — it advances with the walk +(`packages/core/src/pagination/paginator.ts:165,213`, and the contract on +`PaginationStrategy.parse` at `strategy.ts:10-15`). Read the parameter as "the request to derive the +next one from". Where you specifically want the URL the response actually came from — after a redirect +or a step's rewrite — use `response.request.url`, which is what `pageNumberStrategy` reads its current +page number from. ## Three ship already @@ -86,7 +95,7 @@ for await (const page of paginator.pages()) { /* page by page */ } independent walks, not two views of one. That is also why `@dexpace/rx`'s `pageItems$`/`pages$` are cold and repeatable while its SSE observables are not. -## The four rules +## The five rules **1. Take everything you need from the response before your promise settles** (`PAGE-5`). The response you are handed is live and single-use; the engine may close it the moment `parse` resolves. @@ -101,8 +110,10 @@ protects — single use, nothing retained — is what the async signature preser [`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md), where the dissolved deferral register's rows went — precisely so an async signature does not later read as an oversight. Every shipped strategy's `extract` above is `async` for the same reason. -**2. Build `nextRequest` from the template, not from the response.** The template carries the headers, -auth tier and options the walk was started with. A next request built from scratch loses all of them. +**2. Build `nextRequest` from the template, not from scratch.** The template carries the headers, auth +tier and options the walk was started with, and it is the previous hop's request rather than page one's, +so deriving from it accumulates the walk's state instead of re-deriving it. A next request built from +scratch loses all of that. ```typescript const next = template @@ -124,6 +135,20 @@ which is an erratum recorded in `docs/knowledge/notes/pagination.md` and `docs/w `maxPages` on `PaginatorInit` is the backstop, not the design. Loop detection is not the paginator's job. +**5. Always return a well-formed `PageInfo`** (`PAGE-4`). `items` must be an array — an empty one is +fine and is a perfectly valid non-terminal page — and `nextRequest === undefined` is the **single, +exclusive** end-of-stream signal. A `PageInfo` that is itself `null` or `undefined`, or whose `items` +is either, is a programmer error and the engine treats it as one: it closes the response and throws +an assertion naming the invariant you broke. It does **not** end the walk quietly, because "the +strategy forgot to `return`" and "the server ran out of pages" must not look the same from the +outside. Use `pageInfo(items, next?)` and this cannot happen; the check exists because `parse` +crosses a seam, where an `any`-typed decode or a trusted server field can produce a shape the types +say is impossible. + +Terminating and failing are different acts. To *end* the walk, return `pageInfo(items)` with no next +request. To *fail* it, throw — the engine closes the response and your error reaches the consumer +unwrapped (`PAGE-13`, `PAGE-28`). + `PaginationError` is reserved for engine misuse and precondition violations — not for "the server returned a page I did not understand", which is your `extract`'s error to raise. diff --git a/docs/sdk-documentation/write-a-response-handler.md b/docs/sdk-documentation/write-a-response-handler.md index ed46630..7611147 100644 --- a/docs/sdk-documentation/write-a-response-handler.md +++ b/docs/sdk-documentation/write-a-response-handler.md @@ -41,9 +41,14 @@ DOM global under a `lib` that includes `"DOM"`, and the two are unrelated types `Response` has `status: Status`, `close()` and `request`, and no `json()`. The mistake typechecks until you try to pass one. -`decodeSuccessResponse` delegates to `decodeResponse` on a 2xx and to `toHttpError` otherwise, so a -`4xx` becomes an `HttpStatusError` carrying the status, the headers and a bounded body preview -(`SERDE-28`). +`decodeSuccessResponse` delegates to `decodeResponse` on a 2xx and to `toHttpError` on a **4xx or 5xx**, +so a `404` becomes an `HttpStatusError` carrying `status`, a replayable `body()` and a non-consuming +`preview()` — a bounded in-memory copy at the 1 MiB `HTTP-52`/`BODY-30` cap, taken before the response +was closed. There is no headers accessor on it; read anything else you need off the response before you +hand it over. Any *other* non-2xx — a `1xx`, or an unfollowed `3xx` such as a `304` — is neither decoded +nor mapped: the response is closed and a `DeserializationError` is raised whose message leads with the +status code and which carries `ETag` and `Location` as fields, so conditional and redirect context +survives the close (`SERDE-28`). ## What the shipped handlers guarantee, and what you must reproduce @@ -77,6 +82,18 @@ release failure. **`instanceof SuppressedError` is not a valid test**: the class project's declared Node floor and a structurally identical stand-in is built there instead. Test the shape, or read `.error` unconditionally. +**A failed release is the only thing in this SDK that builds one.** Every site that constructs the +pairing — the response chain, the redirect and auth and retry pillars, the serde handlers, the SSE +stream, the paginator — is doing this one job: a `close()` that threw while an error was already in +flight. It is a genuine two-value shape, which is why it fits. + +The retry pillar used to build one for a *second* reason, folding N prior attempt errors into a +nested chain of pairs so that what you caught after three attempts was a wrapper rather than the +failure. It no longer does: it surfaces the last attempt's error unwrapped and records the earlier +ones beside it, read back with `retryAttempts()` (see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars)). +So if you catch a `'SuppressedError'` from this SDK, `.suppressed` is a teardown failure, never an +earlier attempt. + **3. Only payload failures are re-typed.** `SERDE-12`: a malformed body or a shape mismatch becomes a `DeserializationError` with the original chained; a genuine stream failure propagates untouched, because re-wrapping it would tell a caller their payload was malformed when their socket dropped. diff --git a/docs/sdk-documentation/write-a-serde.md b/docs/sdk-documentation/write-a-serde.md index daea858..5893836 100644 --- a/docs/sdk-documentation/write-a-serde.md +++ b/docs/sdk-documentation/write-a-serde.md @@ -61,6 +61,33 @@ import { const TEXT = new TextEncoder(); +/** + * Settle when `operation` settles, or as soon as `signal` aborts — whichever comes first, with the + * caller's own `reason` surfaced verbatim. `throwIfAborted()` alone cannot interrupt a `read()` or + * `write()` that never resolves, which is the case that leaves a caller's stream locked forever. + */ +const raceAbort = async ( + operation: Promise, + signal: AbortSignal | undefined, +): Promise => { + if (signal === undefined) return operation; + signal.throwIfAborted(); + let onAbort = (): void => undefined; + const aborted = new Promise((_resolve, reject) => { + onAbort = (): void => { + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + try { + // The loser of the race keeps `Promise.race`'s own handler, so a `read()` that rejects after + // the lock is released never becomes an unhandled rejection. + return await Promise.race([operation, aborted]); + } finally { + signal.removeEventListener('abort', onAbort); + } +}; + export function csvSerde(): Serde { const encode = (value: unknown): string => { if (!Array.isArray(value)) { @@ -106,7 +133,8 @@ export function csvSerde(): Serde { options?.signal?.throwIfAborted(); // before the lock: an aborted call leaves the sink free const writer = sink.getWriter(); // TypeError if contended — a programmer error, not re-typed try { - await writer.write(TEXT.encode(encode(value))); + // Raced, not just checked: a slow sink parks this write, and the abort must reach it. + await raceAbort(writer.write(TEXT.encode(encode(value))), options?.signal); } finally { writer.releaseLock(); // never close or abort: the caller owns the sink (SERDE-3) } @@ -115,14 +143,15 @@ export function csvSerde(): Serde { deserializer: { deserialize: (data, target) => decode(new TextDecoder().decode(data), target), async deserializeFrom(source, target, options) { - options?.signal?.throwIfAborted(); // before the lock, and again after every read below + options?.signal?.throwIfAborted(); // before the lock: an aborted call never takes one const reader = source.getReader(); const chunks: Uint8Array[] = []; try { for (;;) { - const {done, value} = await reader.read(); + // Raced, not checked between chunks: a source that stalls mid-body parks the loop + // inside `read()`, where a between-chunks check never runs again. + const {done, value} = await raceAbort(reader.read(), options?.signal); if (done) break; - options?.signal?.throwIfAborted(); chunks.push(value); } } finally { @@ -141,7 +170,7 @@ export function csvSerde(): Serde { } ``` -Six rules, all visible above: +Seven rules, all visible above: 1. **Raise `SerializationError` / `DeserializationError`, never a raw error** — with one stated exception: `serializeInto`'s out-of-range or does-not-fit case is a plain `RangeError` with no @@ -160,6 +189,17 @@ Six rules, all visible above: target, on **every** entry point (`SERDE-13`), never return a `null` that detonates at a later field access. The fallback label is the literal `'the target type'`; each codec repeats it, because `SEAM-1` leaves core with no exported constant to share. +7. **An abort must race the pending operation, not sit between two of them** (`SERDE-3`). The seam + promises that "an aborted call never leaves the caller's source locked", and a + `throwIfAborted()` between chunks cannot keep it: a source that stalls mid-body parks the drain + inside `read()`, so the call never settles and the lock is never released. Race each pending + `read()`/`write()` against the signal, remove the listener in a `finally`, then release the lock + as usual. Releasing a reader with a read still outstanding is legal on every supported runtime + and does unlock the stream — the outstanding read rejects, differently per runtime + (`AbortError` on Bun 1.3.14, `TypeError: Invalid state: Releasing reader` on Node 20.3 and 26, + measured 2026-09-05), which is why the caller must see the signal's `reason` instead. + `@dexpace/codec-json` holds **one** listener for the whole drive rather than one per chunk; the + example above takes the simpler per-operation form. `mediaType` is the default `Content-Type` — `serdeBody(value, serde)` reads it, and a caller may override per body. diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index dbf2648..27c2576 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -44,7 +44,7 @@ export function echoTransport(): Transport { Note `setInbound`, not `set`: values a server sent are accepted leniently. Using the strict setter on a real server's headers means a response with an obs-text byte in it becomes unreadable. -## Nine rules a real transport must follow +## Thirteen rules a real transport must follow The full contract is `docs/product-spec/17-transport-adapter-conformance-contract.md`, thirty `TRANSPORT-N` clauses. These are the ones that are easy to get wrong. @@ -61,24 +61,81 @@ client, so forwarding a caller's copy corrupts framing. `Connection` is in the d `fetch`-class transport and not for an undici-class one — §17 says so explicitly. Log the **name**, never the value, and dedupe per name by default. -**3. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the +**3. Whatever your native client refuses, drop that header — never the request** (`TRANSPORT-12`). +This is the half that is easy to miss, because the refusal happens somewhere you are not looking. +`@dexpace/core` admits every printable ASCII byte in a header name (`http/ascii-validation.ts`), so +`X Custom` is a model-valid name no HTTP client on this platform will carry. Both shipped clients +also refuse `Expect`, `Keep-Alive` and `Upgrade` outright, and undici refuses `Connection` with any +value but `close`/`keep-alive`. Find out *where* your client decides: WHATWG `Headers.append` throws +at construction, which a `try`/`catch` degrades for free; undici validates inside `dispatch`, so +that transport has to ask the question itself before handing the array over. Getting this wrong +does not look like a bug in your transport — it looks like a retryable network failure that burns +the caller's whole retry budget re-proving a permanent misconfiguration. The shared suite has a row +per name. + +**4. Classify a native rejection with the shared table, not by hand** (`TRANSPORT-20`, `RETRY-2`). +There are two kinds, and they are not the same kind of thing. An *exchange* that failed — connection +refused, DNS, TLS, peer reset, read timeout — is the retryable `TransportFailureError`, which +`TRANSPORT-20` makes a MUST. A *request* the client refused before dispatching — an unsupported +scheme, a forbidden method, an argument its own validation rejects — can never succeed on a retry, +and `retry/classify.ts` is an allow-list over `IoError`, so reporting it as anything outside that +tree makes it non-retryable for free. Both shipped transports report it as a bare `TypeError` with +the native error as `cause`, matching the `TypeError` they already raise for a misconfiguration +caught at construction. + +Telling the two apart is runtime-specific enough that you should not: call +`toDispatchFailure(error, fallbackMessage)` from `@dexpace/transport-shared`. Node's global `fetch` +reports an unsupported scheme as `TypeError: fetch failed` with an `unknown scheme` *cause* — the +same top-level shape as a DNS failure — while Bun 1.3.14 reports it as +`TypeError [ERR_INVALID_ARG_VALUE]` with no cause, and undici's dispatcher as +`UND_ERR_INVALID_ARG`. The two shipped transports disagreed about `ftp://` until audit #67 / #82 for +exactly that reason. The default is retryable, so a shape the table does not recognize keeps +`TRANSPORT-20`'s MUST. + +**5. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the retryable `TransportFailureError`; a caller abort is the terminal `CancellationError`. A raw `DOMException` must never surface. `isTimeoutSignal(signal)` is how you tell them apart. -**4. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie -a response body's lifetime to the signal they were given, so dispatch over a **fork** of the signal -and detach it at delivery. Get this wrong and a caller who aborts a moment after `send()` resolves -finds the body they already own torn out from under them. - -**5. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do +**6. Dispatch over a fork of the caller's signal — and keep the fork even when there is no signal** +(`SEAM-16`, `TRANSPORT-9`). Both native clients tie a response body's lifetime to the signal they +were given, so a caller who aborts a moment after `send()` resolves would find the body they already +own torn out from under them. Fork the signal, forward the caller's abort through it, and detach at +delivery. + +The fork runs the other way too, and that half is easy to miss. When a streaming request-body +producer fails while the native call is still pending, your `send()` rejects and nothing is left +awaiting that call: a response arriving afterwards is dropped with its body neither read nor +released, which is the leak `TRANSPORT-9` names. Abort the fork before you rethrow. That is why +`forkSignal()` hands back a live signal even when the caller supplied none and no timeout was +composed — a send with no signal at all is precisely the case where nothing could cancel it. Read +whether the *caller* aborted before you pull the fork yourself, or every producer failure surfaces +as a `CancellationError`; and let `detach()` latch the abort, so the second direction cannot become +the `SEAM-16` violation the first one exists to prevent. + +**7. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do not close it. -**6. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is +**8. A response that can carry no body must report `body === null`** (`TRANSPORT-24`, +`TRANSPORT-25`). The WHATWG null-body statuses — `101`, `103`, `204`, `205`, `304` — plus every +`HEAD` response and a 2xx `CONNECT`. Do not forward whatever your native client produced: three of +the four combinations the two shipped adapters meet disagree. undici's dispatcher always hands back +a `BodyReadable`; Node's `fetch` returns `null`; Bun 1.3.14's `fetch` returns a live +`ReadableStream`. `hasNoResponseBody(method, status)` in `@dexpace/transport-shared` is the rule, so +that a consumer can branch on `null` instead of reading to discover there is nothing there. + +Whatever handle you then decline to expose is yours to release — `cancel()` it, `dump()` it — before +you return. `Response.close()` is a no-op on a null body, so nobody else will, and an undrained +`BodyReadable` holds a pooled connection open until the dispatcher times it out. + +A `Content-Length` on a body-less response is not a lie to correct: on a `HEAD` it describes the +body a `GET` would have returned, and it must survive verbatim. Only the body is absent. + +**9. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is never touched by your `close()`. One you constructed is yours to close. Make that decision once, at construction, and make supplying both a caller-owned client *and* an option that would build one a construction-time `TypeError` rather than a silent win for one of them. -**7. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). +**10. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). No unbounded await — a graceful drain would stall teardown for as long as one in-flight send against a slow peer takes. Destroying is the sanctioned choice; in-flight sends then reject with `CancellationError`, and so does a `send()` issued after `close()`, because it cannot succeed over a @@ -86,11 +143,36 @@ dispatcher that no longer exists and so is not a retryable failure. Declare your (`SEAM-15`) either way: `@dexpace/transport-fetch`'s `close()` is a documented no-op over a runtime global it does not own, and `send()` keeps working after it. -**8. Recognize a file body structurally** (`TRANSPORT-28`). `body.kind === 'file'` widens the body to -`FileBodyDescriptor` — `path`, `start`, `count` — and lets you dispatch straight off the file. Never -`instanceof` against `@dexpace/body-file`: a transport must not depend on it. - -**9. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. +**11. Recognize a file body structurally, and still write it through `writeTo`** +(`TRANSPORT-28`, `BODY-13`). `body.kind === 'file'` widens the body to `FileBodyDescriptor` — +`path`, `start`, `count`. Never `instanceof` against `@dexpace/body-file`: a transport must not +depend on it. + +Reading `path` yourself is the trap. It is a shorter path to the wire, and it skips the +descriptor's own `writeTo`, which is where `BODY-13`'s `transferred === count` check lives — the +only thing that can notice a file truncated between `stat` and `send`, because `Content-Length` is +dropped outbound (rule 2) so the framing cannot. `@dexpace/transport-undici` did exactly this until +2026-09-05 and uploaded short files with a 200. Unless your client has a genuine kernel `sendfile` +path, treat a file body as an ordinary `Body` and let `writeTo` produce the bytes; `TRANSPORT-28`'s +zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the +wire — are the descriptor's to keep, not yours. + +**12. Refuse at construction what you cannot honour** (`TRANSPORT-30`, `HTTP-35`). `ProxyType` +admits `socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand +you a proxy your client cannot build. Reject it in the factory with a typed error that names the +type, before you allocate anything — not on the first send, where it arrives as whatever the native +client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a +misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. + +A transport-wide default timeout is the same shape of decision. It ends up in +`AbortSignal.timeout()`, whose range is an integer in `1 .. 2**32 - 1`, and nothing downstream will +check it for you: `RequestOptions.timeoutMs` is validated at its setter, so an unchecked +`defaultTimeoutMs` is the last path by which `1.5` or `2**32` reaches a deadline — where Node throws +a `RangeError` on the first send and Bun 1.3.14 quietly accepts it. Call +`requireValidDefaultTimeoutMs(value)` from `@dexpace/transport-shared` first thing in your +constructor, before anything is allocated. + +**13. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. ## Prove it @@ -105,11 +187,18 @@ runTransportConformanceSuite('my-transport', () => myTransport(), { supportsInternalCancel: false, // TRANSPORT-8: a cancel path distinct from a caller abort supportsProxy: false, // TRANSPORT-30 dropsConnectionHeader: true, // TRANSPORT-11: is `Connection` in your drop set? + // HTTP-35, required: the rows hand this values `AbortSignal.timeout()` refuses and expect your + // factory to refuse them too, rather than deferring the failure to the first send. + buildWithDefaultTimeoutMs: value => myTransport({defaultTimeoutMs: value}), + // TRANSPORT-30, optional: a proxy type your configuration can express and your client cannot + // honour. Omit it and the row asserts `supportsProxy` is false, rather than skipping. + // unsupportedProxy: {type: 'socks5', build: () => myTransport({proxy: socks5Proxy})}, }); ``` -The three capability flags are the only clauses §17 scopes to a subset of transports; everything else -runs unconditionally. The suite starts its own fixture server, and a second one on a separate origin +Those capability entries are the clauses §17 scopes to a subset of transports, plus the one builder +the suite needs to construct a deliberately misconfigured transport; everything else runs +unconditionally. The suite starts its own fixture server, and a second one on a separate origin for the rows that deliberately leave a connection unusable — a client that reuses a poisoned connection otherwise fails thirty rows downstream, which is a debugging problem of a different order. @@ -120,15 +209,18 @@ The package is `private` and its `exports` name `./src/index.ts`, so it resolves `@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are `@internal` and it is not a package to install directly, but reading it is the fastest way to see -what a correct implementation of rules 2, 3, 4 and 6 looks like: +what a correct implementation of rules 2, 3, 4, 5, 6, 8, 9 and 12 looks like: | Module | Concern | |---|---| -| `header-mapping.ts` | The outbound drop-and-degrade pass and the lenient inbound copy | +| `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | | `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | -| `abort-mapping.ts` | The single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | +| `default-timeout.ts` | Rule 12: the range check a transport-wide default timeout has to pass | +| `dispatch-classification.ts` | Rule 4: the one table deciding permanent-versus-retryable for a native rejection | +| `body-less.ts` | Rule 8: which method/status pairs can carry no response body at all | +| `abort-mapping.ts` | Rule 5's single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | | `body-pump.ts` | Turning a `Body` into a request stream the transport owns, plus idempotent teardown for an abandoned producer | -| `signal-fork.ts` | Rule 4's fork-and-detach | +| `signal-fork.ts` | Rule 6's fork-and-detach | ## Package it diff --git a/packages/codec-json/src/abort-race.ts b/packages/codec-json/src/abort-race.ts new file mode 100644 index 0000000..357158b --- /dev/null +++ b/packages/codec-json/src/abort-race.ts @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/abort-race.ts + +/** + * One abort listener held for the length of a whole stream drive, plus the race that lets it settle + * an operation that is already *pending*. + * + * @internal + */ +export interface AbortRace { + /** + * Settle with `operation`, or reject with the signal's `reason` the moment it aborts — whichever + * happens first. + * + * Also rejects before `operation` is even consulted when the signal is already aborted, which is + * the between-chunks check the loop used to make for itself. + */ + race(operation: Promise): Promise; + + /** Drop the abort listener. Call from the `finally` that releases the stream lock. */ + release(): void; +} + +/** The no-signal case: no listener to install, no race to run, no allocation per chunk. */ +const UNRACED: AbortRace = Object.freeze({ + race: (operation: Promise): Promise => operation, + release: (): void => undefined, +}); + +/** + * Bind `signal` to a single listener that can interrupt any number of pending operations + * (SERDE-3, audit #67 / #79). + * + * `throwIfAborted()` between chunks is not enough on its own: a `reader.read()` that never resolves + * is never raced against anything, so the drain parks inside it, the call never settles, and + * `source.locked` stays `true` for the rest of the process — the opposite of the seam's promise that + * "an aborted call never leaves the caller's source locked". Racing the pending operation is what + * makes that promise true rather than aspirational. + * + * The signal's `reason` is surfaced verbatim, never re-typed: a caller aborting with its own error + * gets that error back, and a bare `abort()` gets the platform's `AbortError` `DOMException`, which + * is exactly what `throwIfAborted()` would have thrown. + * + * One listener per call, not one per chunk — a 10 000-chunk body would otherwise register and remove + * 10 000 listeners on a signal the caller may hold for the life of a request. + * + * @param signal - the caller's signal, or `undefined` when the call took none. + * @returns a race bound to `signal`, whose `release()` removes the listener. + * @internal + */ +export function abortRace(signal: AbortSignal | undefined): AbortRace { + if (signal === undefined) return UNRACED; + + let onAbort = (): void => undefined; + const aborted = new Promise((_resolve, reject) => { + onAbort = (): void => { + // The seam documents that a caller sees its own abort `reason` verbatim, and a caller may + // abort with any value at all — `controller.abort('gone')` is legal. Re-typing it here would + // break that contract, and it is also exactly what `throwIfAborted()` throws. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- see above; re-enable if the seam ever narrows `reason` to an Error + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + // An abort can land while nothing is racing this promise — between two reads, or after the last + // one and before `release()`. That rejection would be an unhandled one, which takes the process + // down under Node's default policy (`docs/knowledge/harvested/cancellation-and-timeouts.md:26`). + // A no-op handler marks it handled without stopping `Promise.race` below from seeing it. + void aborted.catch(() => undefined); + + return Object.freeze({ + async race(operation: Promise): Promise { + signal.throwIfAborted(); + // A pending `operation` that rejects after losing the race is still settled through + // `Promise.race`'s own handler, so it never becomes an unhandled rejection either — measured + // on Bun 1.3.14 and Node 20.3/26, where releasing a reader with a read outstanding rejects + // that read (`AbortError` on Bun, `TypeError` on Node). + return Promise.race([operation, aborted]); + }, + release(): void { + signal.removeEventListener('abort', onAbort); + }, + }); +} diff --git a/packages/codec-json/src/json-serde.test.ts b/packages/codec-json/src/json-serde.test.ts index 2de27f1..db85038 100644 --- a/packages/codec-json/src/json-serde.test.ts +++ b/packages/codec-json/src/json-serde.test.ts @@ -32,6 +32,37 @@ async function rejection(promise: Promise): Promise { } } +/** How long a raced abort is given to settle a parked read or write before the case fails. */ +const SETTLE_MS = 250; + +/** + * Fails with a named error instead of letting the runner time out, so a regression reads as "the + * abort never settled the call" rather than as a five-second stall with no diagnosis. + * + * `Promise.race` keeps a handler on `promise`, so a later rejection of the losing side is never an + * unhandled one. + */ +async function settleWithin(promise: Promise, ms: number): Promise { + let timer: ReturnType | undefined; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** One macrotask, which is long enough for a drain loop to reach its second, parked read. */ +function untilParked(): Promise { + return new Promise(resolve => { + setTimeout(resolve, 5); + }); +} + test('declares application/json as its wire media type', () => { expect(jsonSerde().mediaType).toBe('application/json'); }); @@ -673,3 +704,140 @@ describe('the options argument stays optional on both stream methods', () => { ).toEqual({a: 1}); }); }); + +// Module-scope, not describe-local: the pending-abort suite is split across sibling describes to +// stay inside `max-lines-per-function`, and both halves need these. +const PENDING_ABORT_SERDE = jsonSerde(); +const passthroughSchema: Schema = {parse: (i: unknown) => i}; + +/** + * Hands over one chunk and then never produces another, so the drain parks *inside* + * `reader.read()` — the state a between-chunks signal check structurally cannot observe. + */ +function stallingSource( + first: string, + onCancel?: () => void, +): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(first)); + }, + pull() { + return new Promise(() => undefined); + }, + cancel() { + onCancel?.(); + }, + }); +} + +describe('an abort that lands while a READ is pending (audit #67 / #79)', () => { + test('deserializeFrom settles with the caller reason and unlocks the source (SERDE-3)', async () => { + let cancelled = false; + const source = stallingSource('{"a":', () => { + cancelled = true; + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + // The whole point of the fix: the caller gets its stream back, still usable. + expect(source.locked).toBe(false); + expect(cancelled).toBe(false); + }); + + test('an abort with no reason surfaces the platform AbortError the seam documents', async () => { + const source = stallingSource('{"a":'); + const controller = new AbortController(); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(); + + expect(await settleWithin(settled, SETTLE_MS)).toMatchObject({ + name: 'AbortError', + }); + expect(source.locked).toBe(false); + }); +}); + +describe('an abort that lands while a WRITE is pending (audit #67 / #79)', () => { + test('serializeTo settles with the caller reason and unlocks the sink (SERDE-3)', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write() { + return new Promise(() => undefined); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + + const settled = rejection( + PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + expect(sink.locked).toBe(false); + expect(closed).toBe(false); + expect(aborted).toBe(false); + }); + + test('a signal that never fires leaves both directions unchanged', async () => { + const controller = new AbortController(); + const source = new ReadableStream({ + start(streamController) { + streamController.enqueue(new TextEncoder().encode('{"a":')); + streamController.enqueue(new TextEncoder().encode('1}')); + streamController.close(); + }, + }); + const written: string[] = []; + const sink = new WritableStream({ + write(chunk) { + written.push(new TextDecoder().decode(chunk)); + }, + }); + + expect( + await PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ).toEqual({a: 1}); + await PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }); + + expect(written.join('')).toBe('{"a":1}'); + expect(source.locked).toBe(false); + expect(sink.locked).toBe(false); + }); +}); diff --git a/packages/codec-json/src/json-serde.ts b/packages/codec-json/src/json-serde.ts index 140390d..d33cf67 100644 --- a/packages/codec-json/src/json-serde.ts +++ b/packages/codec-json/src/json-serde.ts @@ -8,6 +8,7 @@ import { type Serde, type Serializer, } from '@dexpace/core'; +import {abortRace} from './abort-race.js'; import { degradeTopLevelTristate, tristateReplacer, @@ -128,14 +129,22 @@ function makeSerializer(wiring: TristateWiring): Serializer { // Encoded before the lock is taken: a failed encode then leaves the caller's sink untouched // and still usable, rather than locked-and-released around a write that never happened. const bytes = encodeToBytes(value, wiring); - // Checked after the encode and before the lock, so an aborted call leaves the sink neither - // locked nor closed (SERDE-3). One write follows, so there is no loop to check inside. - options?.signal?.throwIfAborted(); + // Checked after the encode and before the lock, so an aborted call never takes the lock at + // all (SERDE-3). + const signal = options?.signal; + signal?.throwIfAborted(); const writer = sink.getWriter(); + // After `getWriter()`, so a contended sink leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); try { - await writer.write(bytes); + // Raced, not merely checked before: one write is still one operation that can park + // indefinitely against a slow sink, and the abort has to reach it (audit #67 / #79). + await race.race(writer.write(bytes)); } finally { - // Release the lock, never close: the sink is caller-owned (SERDE-3). + race.release(); + // Release the lock, never close: the sink is caller-owned (SERDE-3). A write still + // outstanding at this point stays outstanding — aborting it is the owner's call, not ours. writer.releaseLock(); } }, @@ -209,24 +218,31 @@ function makeDeserializer(): Deserializer { // A streaming TextDecoder keeps multi-byte characters intact across chunk boundaries; decoding // each chunk independently would corrupt any character split across two reads. // Checked before the lock so an aborted call leaves the source neither locked nor cancelled - // (SERDE-3), and again after every read so a long drain stops promptly. + // (SERDE-3), and raced against every read so a stalled one stops too. const signal = options?.signal; signal?.throwIfAborted(); const decoder = new TextDecoder('utf-8'); const reader = source.getReader(); + // After `getReader()`, so a contended source leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); let text = ''; try { for (;;) { - // Serial by necessity: each read depends on the previous one advancing the cursor. - const {done, value} = await reader.read(); + // Serial by necessity: each read depends on the previous one advancing the cursor. Raced + // rather than checked between reads: a source that stalls mid-body parks the loop inside + // `read()`, where a between-chunks check never runs again (audit #67 / #79). + const {done, value} = await race.race(reader.read()); if (done) break; - signal?.throwIfAborted(); text += decoder.decode(value, {stream: true}); } text += decoder.decode(); } finally { + race.release(); // Release the lock, never cancel: the source is caller-owned (SERDE-3). A stream failure // surfaces from `read()` and propagates unwrapped (SERDE-12) — it is not caught here. + // Releasing with a read outstanding is legal on every supported runtime and unlocks the + // stream; the outstanding read rejects, and `Promise.race` above still owns that rejection. reader.releaseLock(); } return decodeText(text, target); @@ -264,12 +280,14 @@ function makeDeserializer(): Deserializer { * `JSON.stringify(v, tristateReplacer)` gets the nested and array-element behaviour but not the * top-level degradation — see {@link tristateReplacer}. * - * **A top-level wire `null` never decodes (SERDE-13).** Every decode target is treated as non-null: - * a schema value carries no nullability this codec could read, so the rejection is unconditional and - * runs *before* the schema. A `200` whose entire body is the literal `null` therefore raises a - * `DeserializationError`, and `tristate(inner)` is a field combinator for use inside - * `tristateObject`, not a top-level decode target. Deliberate: checking after the schema would let a - * permissive schema such as `{parse: (i) => i}` return that `null` as a non-null `T`. + * **A top-level wire `null` decodes only into a target that admits one (SERDE-13).** A schema value + * carries no nullability this codec could read, so the rejection is unconditional *by default* and + * runs *before* the schema: a `200` whose entire body is the literal `null` raises a + * `DeserializationError`. Setting `admitsNull: true` on the `DecodeTarget` is the caller stating what + * the schema value cannot — that `T` includes `null` — and skips the check, which is the one case + * where `tristate(inner)` serves as a top-level target rather than a field combinator for use inside + * `tristateObject`. Checking after the schema instead would let a permissive schema such as + * `{parse: (i) => i}` return that `null` as a non-null `T`. * * @param options - opt out of the Tristate wiring; everything else is fixed by the format. * @returns a frozen, stateless bundle safe to share across concurrent operations (SERDE-29). diff --git a/packages/codec-json/src/tristate-schema.test.ts b/packages/codec-json/src/tristate-schema.test.ts index c6c248c..2c68193 100644 --- a/packages/codec-json/src/tristate-schema.test.ts +++ b/packages/codec-json/src/tristate-schema.test.ts @@ -1,10 +1,16 @@ // SPDX-License-Identifier: MIT // packages/codec-json/src/tristate-schema.test.ts -// Exercises: SERDE-16 (missing → Absent, explicit null → Null, value → Present with element type preserved), -// SERDE-17 (a missing key resolves to Absent via the combinator's own default, not a JSON.parse reviver), -// SERDE-29 (both combinators return frozen schemas, so a shared schema cannot acquire state). +// Exercises: SERDE-14 (three states and only three — a Present can never carry null), SERDE-16 (missing → +// Absent, explicit null → Null, value → Present with element type preserved), SERDE-17 (a missing key resolves +// to Absent via the combinator's own default, not a JSON.parse reviver), SERDE-29 (both combinators return +// frozen schemas, so a shared schema cannot acquire state). import {expect, test} from 'bun:test'; -import {valueOrNull, type Schema, type Tristate} from '@dexpace/core'; +import { + DeserializationError, + valueOrNull, + type Schema, + type Tristate, +} from '@dexpace/core'; import {expectTypeOf} from 'expect-type'; import {MISSING, tristate, tristateObject} from './tristate-schema.js'; @@ -219,3 +225,38 @@ test('both combinators return frozen schemas, like the bundle itself', () => { expect(Object.isFrozen(tristate(identity))).toBe(true); expect(Object.isFrozen(tristateObject({a: identity}))).toBe(true); }); + +// SERDE-14 has three states. `present(null)` is a fourth, and the type system alone cannot keep it +// out: `present` takes `NonNullable`, but `inner.parse`'s declared `T` is unconstrained, so the +// cast that satisfies the compiler is exactly where a normalizing schema slips through +// (audit #67 / #79). +const nullifying: Schema = {parse: () => null}; +const erasing: Schema = {parse: () => undefined}; + +test('an inner schema that normalizes a value to null is a decode failure (SERDE-14)', () => { + expect(() => tristate(nullifying).parse('a value')).toThrow( + DeserializationError, + ); + expect(() => tristate(nullifying).parse('a value')).toThrow( + /present Tristate cannot carry null/, + ); +}); + +test('an inner schema that normalizes a value to undefined is rejected the same way', () => { + expect(() => tristate(erasing).parse('a value')).toThrow( + DeserializationError, + ); +}); + +test('the wire null and missing-key paths still decode ahead of that check (SERDE-16)', () => { + // Neither reaches `inner.parse`, so a normalizing inner schema cannot turn a legitimate Null or + // Absent into a failure. + expect(tristate(nullifying).parse(null).kind).toBe('null'); + expect(tristate(nullifying).parse(MISSING).kind).toBe('absent'); +}); + +test('a nullifying field schema fails the whole tristateObject decode (SERDE-14)', () => { + expect(() => tristateObject({age: nullifying}).parse({age: 30})).toThrow( + DeserializationError, + ); +}); diff --git a/packages/codec-json/src/tristate-schema.ts b/packages/codec-json/src/tristate-schema.ts index d201233..1e473b1 100644 --- a/packages/codec-json/src/tristate-schema.ts +++ b/packages/codec-json/src/tristate-schema.ts @@ -2,6 +2,7 @@ // packages/codec-json/src/tristate-schema.ts import { absent, + DeserializationError, nullValue, present, type Schema, @@ -31,6 +32,8 @@ export const MISSING: unique symbol = Symbol('@dexpace/codec-json.missing'); * @param inner - the schema for the value a Present carries. * @returns a schema producing `Tristate`: a missing-key sentinel or `undefined` yields Absent, a wire * `null` yields Null, anything else runs through `inner` and yields Present. + * @throws DeserializationError when `inner` resolves a present value to `null` or `undefined`. SERDE-14 + * has three states; a Present carrying nothing would be a fourth, and the wire said the key was there. * @public */ export function tristate(inner: Schema): Schema> { @@ -40,9 +43,23 @@ export function tristate(inner: Schema): Schema> { parse(input: unknown): Tristate { if (input === MISSING || input === undefined) return absent(); if (input === null) return nullValue(); - // `as NonNullable`: the null and undefined branches returned above, so the value cannot be - // nullish — a fact the compiler cannot derive through `inner.parse`'s unconstrained `T`. - return present(inner.parse(input) as NonNullable); + const value = inner.parse(input); + // The type system alone cannot hold SERDE-14's third state to non-null values: `present` takes + // `NonNullable`, but `T` here is whatever the caller's schema declares, so a schema that + // NORMALIZES to null — a Zod `.transform()`, a "" → null cleanup — type-checks and produces + // `{kind: 'present', value: null}`, the fourth state the union exists to forbid. Checked at + // run time, and reported as a decode failure rather than an assertion: the input came off the + // wire, and the pairing of that input with that schema is what has no Tristate (audit #67 / + // #79). `undefined` is rejected with it — `NonNullable` excludes both, and a Present of + // `undefined` is Absent wearing the wrong label. + if (value === null || value === undefined) { + throw new DeserializationError( + 'a present Tristate cannot carry null or undefined; the inner schema resolved a wire value to one (SERDE-14)', + ); + } + // `as NonNullable`: the nullish cases have all returned or thrown above, a fact the + // compiler cannot derive through `inner.parse`'s unconstrained `T`. + return present(value); }, }); } @@ -62,6 +79,8 @@ export function tristate(inner: Schema): Schema> { * @param shape - a schema per Tristate-decoded field, keyed by wire name. * @returns a schema producing an object whose named keys are `Tristate`-wrapped. * @throws TypeError when the value being parsed is not a non-null object, or is an array. + * @throws DeserializationError when a field's own schema resolves a present wire value to `null` or + * `undefined`, which is {@link tristate}'s check applied per field (SERDE-14). * @public */ export function tristateObject>>( diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index 4809b58..cfcf87d 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -105,8 +105,10 @@ export interface BackoffSettings { } // @public -export interface BasicCredential { - readonly password: string; +export class BasicCredential { + [INSPECT](): string; + constructor(username: string, password: string); + toString(): string; readonly username: string; } @@ -345,9 +347,11 @@ export class DexpaceError extends Error { export type DigestAlgorithm = 'MD5' | 'MD5-sess' | 'SHA-256' | 'SHA-256-sess'; // @public -export interface DigestCredential { - readonly algorithmPreference?: readonly DigestAlgorithm[] | undefined; - readonly password: string; +export class DigestCredential { + [INSPECT](): string; + constructor(username: string, password: string, algorithmPreference?: readonly DigestAlgorithm[]); + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + toString(): string; readonly username: string; } @@ -664,6 +668,7 @@ export function loggingStep(settings?: LoggingStepSettings): StepDescriptor; // @public export interface LoggingStepSettings { readonly clock?: Clock | undefined; + readonly configKey?: string | undefined; readonly droppedHeaderPolicy?: DroppedHeaderPolicy | undefined; readonly granularity?: LoggingGranularity | undefined; readonly logger?: Logger | undefined; @@ -910,7 +915,7 @@ export class PillarCollisionError extends DexpaceError { // @public export class PipelineBuilder { - constructor(transport: Transport); + constructor(transport: Transport, options?: PipelineOptions); append(descriptor: StepDescriptor): this; appendAll(descriptors: readonly StepDescriptor[]): this; build(): Runtime; @@ -924,6 +929,12 @@ export class PipelineBuilder { static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder; } +// @public +export interface PipelineOptions { + readonly instrumentation?: InstrumentationBundle | undefined; + readonly operationName?: string | undefined; +} + // @public export class PlaintextCredentialError extends DexpaceError { constructor(stepName: string, scheme: string); @@ -1172,6 +1183,9 @@ export type ResponseStep = (response: Response_2) => Promise; // @public export const RETRYABLE_STATUSES: ReadonlySet; +// @public +export function retryAttempts(error: unknown): readonly unknown[]; + // @public export class RetryDiscardedResponseError extends DexpaceError { constructor(status: number, options?: ErrorOptions); @@ -1373,7 +1387,7 @@ export const STAGE_ORDER: readonly Stage[]; export function standardResilience(transport: Transport, options?: StandardResilienceOptions): Runtime; // @public -export interface StandardResilienceOptions { +export interface StandardResilienceOptions extends PipelineOptions { readonly auth?: AuthStepSettings | undefined; readonly logging?: LoggingStepSettings | undefined; readonly redirect?: Partial | undefined; diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts index 6c428d3..f8e266e 100644 --- a/packages/core/src/auth/auth-step.test.ts +++ b/packages/core/src/auth/auth-step.test.ts @@ -4,10 +4,14 @@ // lands before any token fetch or header write, and it applies only on the credential-attaching path -- // a marker-suppressed cross-origin re-issue may proceed credential-free over any scheme), // AUTH-27 (exactly one AUTH-stage descriptor, pinned to the pillar), AUTH-28 (HTTPS guard, -// NO_AUTH exempt, re-applied on the replay path), AUTH-29 (the cross-origin marker skips the guard and +// NO_AUTH exempt, re-applied on the replay path -- unconditionally once the outbound pass guarded the +// hop, whatever header names the replacement carries, and additionally on an Authorization or +// Proxy-Authorization the hook adds to an unguarded NO_AUTH hop), AUTH-29 (the cross-origin marker skips the guard and // stamping, is cleared from the outbound headers, and suppresses the challenge reaction too -- so the // credential cannot re-enter via the 401), AUTH-25 (a 407 is answered from Proxy-Authenticate into -// Proxy-Authorization), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives +// Proxy-Authorization), AUTH-12/AUTH-13 (EVERY value of the matching challenge header is parsed, each +// value on its own, so a repeated WWW-Authenticate/Proxy-Authenticate offers its later challenges too +// and a malformed earlier value cannot swallow them), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives // exactly once through a fresh fork()), AUTH-31 (a non-replayable replacement body surfaces the // original challenge unchanged and unclosed), AUTH-32 (a throwing hook closes the challenge response // before propagating), AUTH-33 (no matching challenge header, or a hook yielding nothing -> unchanged), @@ -41,7 +45,12 @@ import { availableSchemesOf, type AuthCredentialSet, } from './auth-step.js'; -import {createBearerToken, ApiKeyCredential} from './credential.js'; +import { + createBearerToken, + ApiKeyCredential, + BasicCredential, + DigestCredential, +} from './credential.js'; import {createAuthDescriptor} from './descriptor.js'; import {AuthResolutionError, PlaintextCredentialError} from './errors.js'; import {createAuthRequirement} from './requirement.js'; @@ -120,6 +129,27 @@ function challengeResponse( return {response, cancelCount: base.cancelCount}; } +/** + * A challenge response carrying the same challenge header SEVERAL times — the wire shape RFC 7616 + * §3.3 recommends for algorithm discovery, and the one `@dexpace/transport-undici` hands over as + * separate entries rather than one comma-joined value. + * + * `addInbound` in a loop, never `setInbound`: `set` REPLACES, so building the fixture with it would + * quietly assert against a single-valued header and the row would pass against the very bug it exists + * to catch. + */ +function repeatedChallengeResponse( + status: number, + headerName: string, + headerValues: readonly string[], +): {response: Response; cancelCount: () => number} { + const base = countingResponse(status); + const builder = base.response.headers.newBuilder(); + for (const value of headerValues) builder.addInbound(headerName, value); + const response = base.response.newBuilder().headers(builder.build()).build(); + return {response, cancelCount: base.cancelCount}; +} + /** A one-shot request body: `StreamBody.replayable` is `false` (AUTH-31's gate). */ function oneShotPost(url = 'https://example.com/a'): Request { const stream = new ReadableStream({ @@ -236,8 +266,8 @@ describe('availableSchemesOf (AUTH-5)', () => { test('maps each configured credential to its scheme', () => { const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, - digest: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p'), bearer: {provider: () => Promise.resolve(createBearerToken('t'))}, apiKey: {credential: new ApiKeyCredential('k')}, }; @@ -379,7 +409,7 @@ describe('authStep: the HTTPS guard and tier resolution (AUTH-6/AUTH-28)', () => test('BASIC/DIGEST never stamp preemptively -- the outbound request carries no Authorization', async () => { const transport = new FakeTransport([countingResponse(200).response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -437,7 +467,7 @@ describe('authStep: the cross-origin marker (AUTH-29)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -464,7 +494,7 @@ describe('authStep: challenge detection (AUTH-25/AUTH-33)', () => { success.response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -492,7 +522,7 @@ describe('authStep: challenge detection (AUTH-25/AUTH-33)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -508,7 +538,7 @@ describe('authStep: challenge detection, negative cases (AUTH-33)', () => { const the401 = countingResponse(401); const transport = new FakeTransport([the401.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -541,7 +571,7 @@ describe('authStep: the challenge replay (AUTH-30/AUTH-31)', () => { success.response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -566,7 +596,7 @@ describe('authStep: the challenge replay (AUTH-30/AUTH-31)', () => { ); const transport = new FakeTransport([first.response, second.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -590,7 +620,7 @@ describe('authStep: answering a Digest challenge (AUTH-15..AUTH-22)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - digest: {username: 'u', password: 'p'}, + digest: new DigestCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('DIGEST')}); @@ -605,6 +635,112 @@ describe('authStep: answering a Digest challenge (AUTH-15..AUTH-22)', () => { }); }); +describe('authStep: repeated challenge headers (AUTH-12/AUTH-16/AUTH-25)', () => { + test('a later WWW-Authenticate entry is answered when the first is unsupported', async () => { + // RFC 7616 §3.3's algorithm-discovery shape: one header per algorithm, strongest first. Reading + // only `headers.get(...)` saw the SHA-512-256 line, found nothing satisfiable, and surfaced the + // 401 — while the identical pair comma-joined into ONE value authenticated (audit #67 / #74). + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce="n", algorithm=SHA-512-256', + 'Digest realm="r", nonce="n", algorithm=SHA-256, qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + const value = transport.calls[1]?.request.headers.get('Authorization'); + expect(value).toContain('algorithm=SHA-256'); + expect(value).toContain('qop=auth'); + }); + + test('a repeated Proxy-Authenticate is read the same way (AUTH-25)', async () => { + const challenged = repeatedChallengeResponse(407, 'Proxy-Authenticate', [ + 'Negotiate abc123', + 'Basic realm="p"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Proxy-Authorization') + ?.startsWith('Basic '), + ).toBe(true); + }); +}); + +describe('authStep: one offered challenge declined, the next answered (AUTH-13/AUTH-16)', () => { + test('a malformed FIRST entry cannot swallow a satisfiable later one (AUTH-13)', async () => { + // The row that fixes the parse strategy rather than only the read. Comma-joining the two values + // before parsing lets the unterminated quoted string in the first run on into the second — the + // scanner closes it at the `"` of `realm="r"`, and the whole satisfiable challenge disappears + // into a realm value. Parsing each value on its own bounds the damage at the value that carries + // it, which is what AUTH-13's "recovers to the next top-level comma" is reaching for. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="unterminated', + 'Digest realm="r", nonce="n", qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="r"', + ); + }); + + test('a challenge with an empty nonce is declined and the next one answered', async () => { + // Both halves of the same 401: a truncated first challenge that parses to `nonce: ''`, and a + // well-formed second. Declining is only useful if the step then keeps looking, which is what + // AUTH-25's "return no header when it cannot satisfy ANY offered challenge" requires -- the + // quantifier is over the whole offer, not over the first entry. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce=', + 'Digest realm="second", nonce="n", qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="second"', + ); + }); +}); + describe('authStep: the replayability gate (AUTH-31)', () => { test('an unsatisfiable challenge leaves the response unchanged (AUTH-25/AUTH-33)', async () => { const challenged = challengeResponse( @@ -617,7 +753,7 @@ describe('authStep: the replayability gate (AUTH-31)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -641,7 +777,7 @@ describe('authStep: the replayability gate (AUTH-31)', () => { ); const transport = new FakeTransport([challenged.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -749,7 +885,7 @@ describe('authStep: hook override and non-reactive schemes (AUTH-30)', () => { ]); let hookInvoked = false; const descriptor = authStep({ - credentials: {basic: {username: 'u', password: 'p'}}, + credentials: {basic: new BasicCredential('u', 'p')}, tiers: tiersFor('BASIC'), challengeHook: (_response, request) => { hookInvoked = true; @@ -1005,7 +1141,87 @@ describe('authStep: the replay HTTPS guard (AUTH-28)', () => { expect(transport.sendCount).toBe(1); // the replacement never reached the wire expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw, not leaked }); +}); + +describe('authStep: once guarded, the replay stays guarded (AUTH-28)', () => { + test('a replacement carrying a NON-standard credential header over plaintext is refused (AUTH-28)', async () => { + // The reported hole (audit #67 / #71): the guard tested two header NAMES, and + // `ApiKeyCredentialConfig.headerName` lets this very step stamp any header it is told to. A hook + // that downgrades the URL and answers with `X-Api-Key` therefore went out in clear text, with no + // `PlaintextCredentialError`. The rule is now "the outbound pass guarded this hop, so the replay + // is guarded too", which does not depend on reading header names at all. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }, + tiers: tiersFor('API_KEY'), // the outbound pass DID guard: the seed is https + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .url('http://example.com/a') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); // the replacement never reached the wire + expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw + }); + + test('a header-free replacement over plaintext is refused too, once the hop was guarded (AUTH-28)', async () => { + // The other half of the same rule, and the reason it is stated as "once guarded, always guarded" + // rather than as a wider header list: a hook is free to invent a credential carrier this step has + // never heard of, so no enumeration of names can be complete. The scheme that made the outbound + // pass credentialed is what the replay inherits. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: {credential: new ApiKeyCredential('secret')}, + }, + tiers: tiersFor('API_KEY'), + challengeHook: (_response, request) => + Promise.resolve( + Request.newBuilder() + .url('http://example.com/b') + .method(request.method) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); + }); +}); +describe('authStep: an unguarded NO_AUTH hop (AUTH-28/AUTH-29)', () => { test('a credential-free replacement over plaintext is NOT blocked by the replay guard (AUTH-28/AUTH-29)', async () => { const challenged = challengeResponse( 401, @@ -1137,7 +1353,7 @@ describe('authStep: answering an unrecognized scheme through challengeHook', () countingResponse(200).response, ]); const descriptor = authStep({ - credentials: {basic: {username: 'u', password: 'p'}}, + credentials: {basic: new BasicCredential('u', 'p')}, tiers: tiersFor('BASIC'), challengeHook: (_response, request) => Promise.resolve( @@ -1459,7 +1675,7 @@ describe('authStep: a challenge this client cannot echo (AUTH-21/AUTH-22)', () = ); const transport = new FakeTransport([challenge.response]); const descriptor = authStep({ - credentials: {digest: {username: 'u', password: 'p'}}, + credentials: {digest: new DigestCredential('u', 'p')}, tiers: tiersFor('DIGEST'), }); diff --git a/packages/core/src/auth/auth-step.ts b/packages/core/src/auth/auth-step.ts index 540a9d3..042eadf 100644 --- a/packages/core/src/auth/auth-step.ts +++ b/packages/core/src/auth/auth-step.ts @@ -15,48 +15,21 @@ import {BearerTokenCache} from './bearer-cache.js'; import {parseChallenges} from './challenge.js'; import type {Challenge, ChallengeHandler} from './challenge.js'; import {composingHandler, type ComposingHandler} from './composing-handler.js'; -import type { - ApiKeyCredential, - NameKeyCredential, - TokenProvider, +import { + credentialPassword, + type ApiKeyCredential, + type BasicCredential, + type DigestCredential, + type NameKeyCredential, + type TokenProvider, } from './credential.js'; import type {AuthDescriptor} from './descriptor.js'; import {digestHandler} from './digest.js'; -import type {DigestAlgorithm} from './digest.js'; import {PlaintextCredentialError} from './errors.js'; import {resolveAuthRequirement, type AuthTiers} from './resolve.js'; import type {AuthScheme} from './scheme.js'; import {stampStaticKey} from './static-key.js'; -/** - * Username and password for the `BASIC` scheme. - * - * @public - */ -export interface BasicCredential { - /** The user id (AUTH-14: non-empty; whitespace permitted). */ - readonly username: string; - /** The password (AUTH-14: non-empty; whitespace permitted). */ - readonly password: string; -} - -/** - * Username, password, and algorithm preference for the `DIGEST` scheme. - * - * @public - */ -export interface DigestCredential { - /** The user id. Must not be blank. */ - readonly username: string; - /** The password. Must not be blank. */ - readonly password: string; - /** - * Preferred-first order, and also the acceptable set (AUTH-16). Omitted means strongest-first over - * all four supported algorithms. - */ - readonly algorithmPreference?: readonly DigestAlgorithm[] | undefined; -} - /** * The token source and refresh margin for the `OAUTH2` scheme. * @@ -145,16 +118,24 @@ function buildHandlers( credentials: AuthCredentialSet, ): readonly ChallengeHandler[] { const handlers: ChallengeHandler[] = []; + // `credentialPassword()`, not a public `.password` property: AUTH-8's secret stays off the + // published surface and this function is the one sanctioned reader, exactly as `stampStaticKey` + // is for `credentialKey()`. if (credentials.digest !== undefined) { handlers.push( - digestHandler(credentials.digest.username, credentials.digest.password, { - algorithmPreference: credentials.digest.algorithmPreference, - }), + digestHandler( + credentials.digest.username, + credentialPassword(credentials.digest), + {algorithmPreference: credentials.digest.algorithmPreference}, + ), ); } if (credentials.basic !== undefined) { handlers.push( - basicHandler(credentials.basic.username, credentials.basic.password), + basicHandler( + credentials.basic.username, + credentialPassword(credentials.basic), + ), ); } return handlers; @@ -373,6 +354,12 @@ async function preemptiveStamp( */ interface OutboundPlan { readonly crossOrigin: boolean; + /** + * Whether the outbound pass ran {@link requireHttps} on this hop -- i.e. whether this is one of + * AUTH-28's "paths where a credential will be attached". Carried past the dispatch because the + * replay inherits it: see {@link guardReplayScheme}. + */ + readonly guarded: boolean; readonly outbound: Request; } @@ -398,13 +385,19 @@ async function planOutbound( .headers(clearCrossOriginMarker(seedRequest.headers)) .build(); - if (crossOrigin) return {crossOrigin, outbound: cleared}; - if (context.scheme !== 'NO_AUTH') requireHttps(cleared.url, context.scheme); - return {crossOrigin, outbound: await preemptiveStamp(cleared, context)}; + if (crossOrigin) return {crossOrigin, guarded: false, outbound: cleared}; + const guarded = context.scheme !== 'NO_AUTH'; + if (guarded) requireHttps(cleared.url, context.scheme); + return { + crossOrigin, + guarded, + outbound: await preemptiveStamp(cleared, context), + }; } interface ChallengeSelection { - readonly value: string; + /** Every value the matching challenge header carried, in wire order. */ + readonly values: readonly string[]; readonly isProxy: boolean; } @@ -412,16 +405,37 @@ interface ChallengeSelection { * AUTH-25: a 401 is answered from `WWW-Authenticate`, a 407 from `Proxy-Authenticate`. Reading only * the header that matches the status keeps the pairing honest — a 401 carrying a stray * `Proxy-Authenticate` must not produce a `Proxy-Authorization`, and vice versa. + * + * `getAll`, not `get`. RFC 9110 §5.3 lets a server send a list-valued field either comma-joined into + * one line or repeated across several, and RFC 7616 §3.3 recommends the repeated form for Digest + * algorithm discovery — one challenge per algorithm, strongest first. `get()` returns the FIRST value + * only, so a 401 offering `algorithm=SHA-512-256` (unsupported here) on line one and + * `algorithm=SHA-256, qop="auth"` on line two ended with no `Authorization` at all, while the same two + * challenges comma-joined authenticated (audit #67 / #74). Which shape reaches this step is the + * transport's accident, not the server's intent: `@dexpace/transport-fetch` joins repeated values, + * `@dexpace/transport-undici` keeps them apart, and both are legal. */ function pickChallengeHeader( response: Response, ): ChallengeSelection | undefined { - if (response.status.code === 401) { - const www = response.headers.get('WWW-Authenticate'); - return www === undefined ? undefined : {value: www, isProxy: false}; - } - const proxy = response.headers.get('Proxy-Authenticate'); - return proxy === undefined ? undefined : {value: proxy, isProxy: true}; + const isProxy = response.status.code !== 401; + const values = response.headers.getAll( + isProxy ? 'Proxy-Authenticate' : 'WWW-Authenticate', + ); + return values.length === 0 ? undefined : {values, isProxy}; +} + +/** + * Every challenge the selected header offered, in wire order (AUTH-12). + * + * Each value is parsed on its OWN, and the lists are concatenated — never joined into one string + * first. `parseChallenges` is total (AUTH-13), but its recovery is bounded by the string it is handed: + * an unterminated quoted string terminates at end-of-input, so joining lets a malformed earlier value + * swallow a satisfiable later one whole. Parsing per value keeps the blast radius of a broken header + * line inside that line. + */ +function challengesOf(selection: ChallengeSelection): readonly Challenge[] { + return selection.values.flatMap(value => parseChallenges(value)); } interface DefaultHookContext { @@ -449,7 +463,7 @@ async function oauth2ChallengeHook( const rejected = request.headers.get(headerName); // AUTH-36: no Authorization on the rejected request -> surface the challenge unchanged. if (rejected === undefined) return undefined; - const challenges: readonly Challenge[] = parseChallenges(selection.value); + const challenges: readonly Challenge[] = challengesOf(selection); if (!challenges.some(challenge => challenge.scheme === 'bearer')) { return undefined; // AUTH-36: the response advertises no Bearer challenge } @@ -495,7 +509,7 @@ async function basicDigestChallengeHook( selection: ChallengeSelection, context: DefaultHookContext, ): Promise { - const challenges = parseChallenges(selection.value); + const challenges = challengesOf(selection); const url = request.url; // HTTP-5: a fresh URL per access, so read it once. const requestTarget = `${url.pathname}${url.search}`; const value = await context.composing.stamp(challenges, { @@ -537,24 +551,45 @@ async function defaultChallengeHook( } } +/** What {@link guardReplayScheme} needs. Bundled to stay inside `max-params`. */ +interface ReplayGuardInput { + readonly replacement: Request; + readonly response: Response; + readonly scheme: AuthScheme; + /** {@link OutboundPlan.guarded} for this hop. */ + readonly outboundGuarded: boolean; +} + /** * AUTH-28 on the REPLAY path. The outbound guard is not sufficient here: it is skipped entirely for * `NO_AUTH`, and nothing constrains a caller-supplied hook to preserve the request URL. A replay - * carrying a credential header is by definition "a path where a credential will be attached", and - * AUTH-28 says ANY such path. + * carrying a credential is by definition "a path where a credential will be attached", and AUTH-28 + * says ANY such path. + * + * **Once guarded, always guarded.** When the outbound pass ran the guard, so does the replay -- + * unconditionally, without inspecting a single header name. The rule used to be "the replacement + * carries `Authorization` or `Proxy-Authorization`", and that missed the case this step creates + * itself: `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, so a hook + * answering a 401 with `X-Api-Key: SECRET` over a downgraded `http://` URL went out in clear text + * with no `PlaintextCredentialError` (audit #67 / #71). Deriving the credential-carrying names from + * configuration instead was considered and rejected: a `challengeHook` may invent a carrier this step + * has never been told about, so no enumeration can be complete, whereas "this hop is credentialed" + * is a fact the outbound pass already decided. + * + * The header test survives as a SECOND arm rather than being replaced, because it still reaches + * somewhere the first cannot: a `NO_AUTH` hop is never guarded outbound, and a hook that answers its + * challenge with an `Authorization` header is attaching a credential all the same. * * The challenge response is closed before the throw, for the same reason AUTH-32 closes it on a hook * throw: this is past the point where the caller still owns it, so propagating unclosed leaks the body. */ -async function guardReplayScheme( - replacement: Request, - response: Response, - scheme: AuthScheme, -): Promise { - const carriesCredential = +async function guardReplayScheme(input: ReplayGuardInput): Promise { + const {replacement, response, scheme, outboundGuarded} = input; + const attachesCredential = + outboundGuarded || replacement.headers.has('Authorization') || replacement.headers.has('Proxy-Authorization'); - if (!carriesCredential) return; + if (!attachesCredential) return; try { requireHttps(replacement.url, scheme); } catch (error) { @@ -598,6 +633,8 @@ async function runHook( interface ChallengeDrive { readonly response: Response; readonly outbound: Request; + /** {@link OutboundPlan.guarded}, carried in for {@link guardReplayScheme}. */ + readonly outboundGuarded: boolean; readonly fork: () => (request?: Request) => Promise; readonly settings: AuthStepSettings; readonly hookContext: DefaultHookContext; @@ -613,7 +650,8 @@ interface ChallengeDrive { * failure. */ async function handleChallenge(drive: ChallengeDrive): Promise { - const {response, outbound, fork, settings, hookContext} = drive; + const {response, outbound, outboundGuarded, fork, settings, hookContext} = + drive; const selection = pickChallengeHeader(response); // AUTH-33: no matching challenge header -> unchanged, and the hook is never consulted. @@ -660,7 +698,13 @@ async function handleChallenge(drive: ChallengeDrive): Promise { // a second wire send on a request nobody is waiting for is the one thing that must not happen. if (isAborted(hookContext.signal)) return response; - await guardReplayScheme(replacement, response, hookContext.scheme); // AUTH-28 + // AUTH-28 + await guardReplayScheme({ + replacement, + response, + scheme: hookContext.scheme, + outboundGuarded, + }); await response.close(); // AUTH-30: the original is closed before the replacement is driven. // AUTH-30: exactly once, through a FRESH chain copy, with no further challenge handling on it. @@ -697,6 +741,8 @@ async function handleChallenge(drive: ChallengeDrive): Promise { * @returns the descriptor to install in a pipeline's AUTH slot. * @throws PlaintextCredentialError — as a rejected promise — when the resolved scheme would attach a * credential over a non-HTTPS URL (AUTH-28), on the outbound pass and again on a challenge replay. + * A replay whose hop was guarded outbound is guarded again whatever URL and headers the hook chose, + * so a hook that downgrades the scheme fails here rather than on the wire. * Recover by fixing the endpoint's scheme; retrying will not help. * @throws AuthResolutionError — as a rejected promise — when the selected tier lists no scheme with a * matching configured credential (AUTH-6; AUTH-4 governs only WHICH tier is selected), or when the @@ -769,7 +815,7 @@ export function authStep(settings: AuthStepSettings): StepDescriptor { signal, }; - const {crossOrigin, outbound} = await planOutbound( + const {crossOrigin, guarded, outbound} = await planOutbound( seedRequest, stampContext, ); @@ -788,6 +834,7 @@ export function authStep(settings: AuthStepSettings): StepDescriptor { return handleChallenge({ response, outbound, + outboundGuarded: guarded, fork, settings, hookContext: {...stampContext, composing}, diff --git a/packages/core/src/auth/credential.test.ts b/packages/core/src/auth/credential.test.ts index 23f76c9..f15189e 100644 --- a/packages/core/src/auth/credential.test.ts +++ b/packages/core/src/auth/credential.test.ts @@ -1,20 +1,33 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/credential.test.ts // Exercises: AUTH-8 (BearerToken is value-equal; ApiKeyCredential/NameKeyCredential are -// reference-equal via bare `===`, no equals() override; ALL THREE redact their secret in every -// string/diagnostic form and none is reachable through JSON.stringify/Object.keys), AUTH-9 (blank +// reference-equal via bare `===`, no equals() override; ALL FIVE credential types -- bearer, API key, +// name key, Basic and Digest -- redact their secret in every string/diagnostic form and none is +// reachable through JSON.stringify/Object.keys, including when a whole AuthCredentialSet is handed to +// `util.inspect`), AUTH-9 (blank // rejected as a programmer error, and every type is nominal so the validation cannot be routed // around), AUTH-10 (expiry math: undefined never locally expires; expired iff nowMs + marginMs > -// expiresAt), AUTH-26 (`credentialKey()` is the sole read path for a static key's secret). +// expiresAt), AUTH-26 (`credentialKey()` is the sole read path for a static key's secret), +// AUTH-14/AUTH-16 (the Basic and Digest credentials carry the username and the algorithm preference +// as non-secret fields, which AUTH-8 permits to stay visible). import {describe, expect, test} from 'bun:test'; +// The ONE `node:` import in this package's tests, and it is the point of the AuthCredentialSet rows +// below: the reported leak was `util.inspect(credentials)` printing `password: 'hunter2'`, so the row +// that proves it closed has to drive the real `util.inspect`, not the hook it happens to call. +import {inspect} from 'node:util'; import {InvariantViolation} from '../invariant.js'; +import type {AuthCredentialSet} from './auth-step.js'; +import type {DigestAlgorithm} from './digest.js'; import { ApiKeyCredential, + BasicCredential, BearerToken, + DigestCredential, NameKeyCredential, bearerTokensEqual, createBearerToken, credentialKey, + credentialPassword, isBearerTokenExpired, } from './credential.js'; @@ -23,7 +36,12 @@ import { const INSPECT = Symbol.for('nodejs.util.inspect.custom'); function inspectOf( - value: ApiKeyCredential | BearerToken | NameKeyCredential, + value: + | ApiKeyCredential + | BasicCredential + | BearerToken + | DigestCredential + | NameKeyCredential, ): string { const hooks = value as unknown as Record string) | undefined>; const hook = hooks[INSPECT]; @@ -191,3 +209,138 @@ describe('NameKeyCredential (AUTH-8)', () => { expect(serialized).not.toContain('super-secret'); }); }); + +describe('BasicCredential (AUTH-8/AUTH-14)', () => { + test('two instances with identical fields are NOT equal -- reference identity only', () => { + expect( + new BasicCredential('u', 'p') === new BasicCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep the username', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credential.toString()).toContain('alice'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(JSON.stringify(credential)).toContain('alice'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual(['username']); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + // No public `password` property: that is exactly what the structural interface this class + // replaced put on the published surface (AUTH-8). + expect('password' in credential).toBe(false); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new BasicCredential('u', 'p'))).toBe(true); + }); +}); + +describe('DigestCredential (AUTH-8/AUTH-16)', () => { + test('two instances with identical fields are NOT equal', () => { + expect( + new DigestCredential('u', 'p') === new DigestCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep username and preference', () => { + const credential = new DigestCredential('bob', 'super-secret', ['SHA-256']); + expect(credential.toString()).toContain('bob'); + expect(credential.toString()).toContain('SHA-256'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(JSON.stringify(credential)).toContain('bob'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual([ + 'username', + 'algorithmPreference', + ]); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + expect('password' in credential).toBe(false); + }); + + test('an omitted algorithmPreference stays undefined -- the handler owns the default', () => { + // `digestHandler` applies AUTH-16's strongest-first default. Materializing it here would put a + // second copy of that list on the public surface, free to drift from the one that is used. + expect( + new DigestCredential('bob', 'p').algorithmPreference, + ).toBeUndefined(); + }); + + test('the algorithm preference is copied and frozen, not aliased (HTTP-3)', () => { + const supplied: DigestAlgorithm[] = ['SHA-256', 'MD5']; + const credential = new DigestCredential('bob', 'p', supplied); + supplied.push('MD5-sess'); + + expect(credential.algorithmPreference).toEqual(['SHA-256', 'MD5']); + expect(Object.isFrozen(credential.algorithmPreference)).toBe(true); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new DigestCredential('u', 'p'))).toBe(true); + }); +}); + +describe('a whole AuthCredentialSet is diagnostic-safe (AUTH-8)', () => { + /** Every scheme's material at once -- the shape a caller hands `authStep()`. */ + function everyCredential(): AuthCredentialSet { + return { + basic: new BasicCredential('alice', 'basic-hunter2'), + digest: new DigestCredential('bob', 'digest-hunter2', ['SHA-256']), + apiKey: { + credential: new NameKeyCredential('x-api-key', 'api-key-hunter2'), + headerName: 'X-Api-Key', + }, + bearer: { + provider: () => + Promise.resolve(createBearerToken('bearer-hunter2', 1000)), + }, + }; + } + + test('util.inspect prints no secret from any of the four schemes', () => { + const rendered = inspect(everyCredential(), {depth: null}); + + expect(rendered).not.toContain('basic-hunter2'); + expect(rendered).not.toContain('digest-hunter2'); + expect(rendered).not.toContain('api-key-hunter2'); + // The non-secret fields AUTH-8 permits to stay visible are still there, which is what makes the + // rendering worth printing at all. + expect(rendered).toContain('alice'); + expect(rendered).toContain('bob'); + expect(rendered).toContain('x-api-key'); + }); + + test('util.inspect honours the inspect hook on a BearerToken too', () => { + // `inspectOf` proves the hook exists; this proves `util.inspect` actually calls it. + expect(inspect(createBearerToken('bearer-hunter2', 1000))).not.toContain( + 'bearer-hunter2', + ); + }); + + test('JSON.stringify serializes no secret from any of the four schemes', () => { + const serialized = JSON.stringify(everyCredential()); + + expect(serialized).not.toContain('basic-hunter2'); + expect(serialized).not.toContain('digest-hunter2'); + expect(serialized).not.toContain('api-key-hunter2'); + }); +}); diff --git a/packages/core/src/auth/credential.ts b/packages/core/src/auth/credential.ts index 1fb9403..93d95df 100644 --- a/packages/core/src/auth/credential.ts +++ b/packages/core/src/auth/credential.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/credential.ts import {invariant} from '../invariant.js'; +import type {DigestAlgorithm} from './digest.js'; // No `as unique symbol` cast: TypeScript rejects `unique symbol` in a type assertion (TS1335). A // `const` initialized directly by a `Symbol.for()` call already gets the `unique symbol` type, which @@ -283,6 +284,173 @@ export class NameKeyCredential { } } +/** + * The friend-class hooks for the two password credentials' secrets, and the reason those two are + * classes at all. `auth-step.ts`'s `buildHandlers` -- the ONLY sanctioned reader -- reaches them + * through {@link credentialPassword}. + * + * They shipped as structural interfaces with a public `readonly password: string` until 2026-09-04, + * which put a live password on the published `.d.ts` and, worse, on the object graph: `util.inspect` + * of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and + * `JSON.stringify` serialized it. AUTH-8's redaction clause names bearer, API-key and name-key + * explicitly; reading it as covering EVERY credential type is a deliberate widening, recorded in + * `docs/deviations.md` (found by audit #67 / #71). + */ +let readBasicPassword: (credential: BasicCredential) => string; +let readDigestPassword: (credential: DigestCredential) => string; + +/** + * The in-package read hook for a Basic or Digest credential's password. + * + * Exported (still internal-only, absent from the package barrel) because the one caller -- + * `buildHandlers` in `auth-step.ts` -- lives in another module and TypeScript has no friend-class + * visibility to express that with. The same shape {@link credentialKey} uses for the static keys. + * + * @param credential - the credential whose password is about to reach a handler. + * @returns the raw password. + * + * @internal + */ +export function credentialPassword( + credential: BasicCredential | DigestCredential, +): string { + return credential instanceof BasicCredential + ? readBasicPassword(credential) + : readDigestPassword(credential); +} + +/** + * Username and password for the `BASIC` scheme (AUTH-8, AUTH-14). + * + * `#password`, not `readonly password`, for the runtime-privacy reason {@link ApiKeyCredential} + * states at length: `private` is erased and a plain property is reachable through + * `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect`. The + * redacted `toString`/inspect pair below is what those paths get instead. `username` is non-secret, + * which AUTH-8 explicitly permits to stay visible. + * + * Reference equality, like the two key credentials and for the same reason: there is deliberately no + * `equals` member anywhere in this module. + * + * **Validation is not repeated here.** AUTH-14's rule -- non-empty, whitespace permitted, which is + * deliberately laxer than `.trim().length > 0` -- lives in `basicHandler()`, and `authStep()` builds + * a handler for every configured credential at construction, so a blank password still fails + * synchronously from that factory. Restating the rule here would put a second copy of it one edit + * away from disagreeing with the copy that is actually applied to the wire. + * + * @public + */ +export class BasicCredential { + /** The user id (AUTH-14: non-empty; whitespace permitted). Non-secret, so visible. */ + readonly username: string; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + */ + constructor(username: string, password: string) { + this.username = username; + this.#password = password; + Object.freeze(this); + } + + static { + readBasicPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username survives, the password does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + return `BasicCredential{username=${this.username}, password=***}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`, which do not route object arguments through + * `toString`. See {@link ApiKeyCredential} for why both hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * Username, password, and algorithm preference for the `DIGEST` scheme (AUTH-8, AUTH-16). + * + * `#password` for the same runtime-privacy reason as {@link BasicCredential}. `username` and + * `algorithmPreference` are non-secret and stay visible. + * + * **Validation is not repeated here**, for the reason {@link BasicCredential} states: AUTH-16's + * acceptable-set rule and the blank/header-safety checks live in `digestHandler()`, which + * `authStep()` builds at construction. + * + * @public + */ +export class DigestCredential { + /** The user id. Non-secret, so visible. */ + readonly username: string; + /** + * Preferred-first order, and also the acceptable set (AUTH-16). `undefined` means strongest-first + * over all four supported algorithms -- the default is applied by `digestHandler()`, not + * materialized here, so there is only ever one copy of that list. + */ + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + * @param algorithmPreference - preferred-first acceptable algorithms; copied and frozen, so a + * caller mutating the array afterwards cannot change what this credential accepts (HTTP-3's + * no-aliasing rule, applied to the one collection this type holds). + */ + constructor( + username: string, + password: string, + algorithmPreference?: readonly DigestAlgorithm[], + ) { + this.username = username; + this.#password = password; + this.algorithmPreference = + algorithmPreference === undefined + ? undefined + : Object.freeze([...algorithmPreference]); + Object.freeze(this); + } + + static { + readDigestPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username and the algorithm preference survive, the password + * does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + const preference = + this.algorithmPreference === undefined + ? 'default' + : this.algorithmPreference.join('|'); + return `DigestCredential{username=${this.username}, password=***, algorithmPreference=${preference}}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`. See {@link ApiKeyCredential} for why both + * hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + /** * AUTH-11: an async token source. A plain function type, no class. * diff --git a/packages/core/src/auth/digest.test.ts b/packages/core/src/auth/digest.test.ts index d887af9..93b474e 100644 --- a/packages/core/src/auth/digest.test.ts +++ b/packages/core/src/auth/digest.test.ts @@ -5,11 +5,14 @@ // (the client nonce is drawn from a CSPRNG with >= 128 bits of entropy, never a non-cryptographic RNG), // AUTH-15 (exactly {MD5, MD5-sess, SHA-256, SHA-256-sess}, qop=auth or absent, declines // auth-int and unsupported algorithms), AUTH-16 (satisfiability: scheme/realm/nonce/qop/algorithm, -// and configured-preference order over wire order), AUTH-17 (HA1/HA2/response per RFC 7616/2069, +// and configured-preference order over wire order -- realm and nonce must carry a VALUE, so a +// truncated `nonce=` is declined rather than echoed back as `nonce=""`), AUTH-17 (HA1/HA2/response per RFC 7616/2069, // verified against independently-computed vectors), AUTH-18/AUTH-19 (nonce count: starts at 1, // increments only on nonce reuse, 8 lower-case hex digits, bounded and drained to the cap), // AUTH-20 (client nonce from crypto.getRandomValues, >=128 bits), AUTH-21 (UTF-8 vs ISO-8859-1 by -// charset), AUTH-22 (quoting, and cnonce/nc/qop emitted only when qop negotiated), AUTH-25 +// charset), AUTH-22 (quoting, and nc/qop emitted only when qop negotiated -- with cnonce emitted for +// every -sess algorithm whatever qop says, a recorded departure from AUTH-22's letter: RFC 7616 3.4.2 +// folds cnonce into a -sess HA1, so a server cannot verify a -sess response that omits it), AUTH-25 // (Authorization vs Proxy-Authorization is the CALLER's job -- stamp() returns only the value). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; @@ -91,6 +94,20 @@ describe('computeDigestResponse (verified against RFC 2617/7616 vectors)', () => ).toBe('8e3825c57e897f5a0dec6c2d4e5059d0'); }); + test('MD5-sess, no qop -- cnonce still folds into HA1 (RFC 7616 3.4.2)', async () => { + // The vector the `-sess`-without-`qop` fix is pinned on. HA1 is + // H(H(user:realm:pass):nonce:cnonce) for every `-sess` algorithm, whatever `qop` the server + // offered, and the response is then RFC 2069's H(HA1:nonce:HA2) because no qop was negotiated. + // Recomputed independently from the same RFC 2617 3.5 inputs the vectors above use. + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5-sess', + hasQopAuth: false, + }), + ).toBe('4726bc10c33fa6cb357eb27807b1cce8'); + }); + test('SHA-256, qop=auth', async () => { expect( await computeDigestResponse({ @@ -242,6 +259,33 @@ describe('digestHandler: credential validation (AUTH-9/AUTH-22)', () => { }); }); +describe('digestHandler: a realm or nonce with no value (AUTH-16)', () => { + test('canHandle declines an EMPTY realm or nonce, not only an absent one (AUTH-16)', () => { + // A truncated `WWW-Authenticate: Digest realm="r", nonce=` parses to `nonce: ''` -- AUTH-12 stores + // values verbatim after unquoting and AUTH-13 keeps what it parsed before the malformed tail, both + // correctly. The `=== undefined` test downstream then read that as present, and the client sent + // `nonce=""` back: a response computed over an empty nonce, which no server can have issued + // (audit #67 / #74). AUTH-16's "carries realm and nonce" is read as carrying a VALUE. + const handler = digestHandler('u', 'p'); + expect(handler.canHandle(digestChallenge({realm: REALM, nonce: ''}))).toBe( + false, + ); + expect(handler.canHandle(digestChallenge({realm: '', nonce: NONCE}))).toBe( + false, + ); + }); + + test('rank is worst-possible for an empty realm or nonce, so a sibling wins', () => { + // The declining half is only useful if the composer can still get past it: `rank` and `canHandle` + // must agree, or a challenge that cannot be stamped would still sort first (AUTH-25's "return no + // header when it cannot satisfy any" applies per challenge, not per response). + const handler = digestHandler('u', 'p'); + expect(handler.rank?.(digestChallenge({realm: REALM, nonce: ''}))).toBe( + Number.MAX_SAFE_INTEGER, + ); + }); +}); + describe('digestHandler: challenge selection (AUTH-15/AUTH-16)', () => { test('canHandle accepts a well-formed Digest challenge', () => { const handler = digestHandler('u', 'p'); @@ -421,9 +465,55 @@ describe('digestHandler opaque and qop emission (AUTH-22)', () => { }); }); +describe('digestHandler -sess without qop (AUTH-17/AUTH-22)', () => { + test('stamp() emits cnonce -- and only cnonce -- for a -sess algorithm with no qop', async () => { + // A `-sess` HA1 is H(H(u:r:p):nonce:cnonce). Hashing a fresh random cnonce and then leaving it + // off the wire made every such response unverifiable by construction, and AUTH-30 bounds the + // replay to one 401, so the request simply failed (audit #67 / #74). RFC 7616 3.4: "cnonce: This + // parameter MUST be used by all implementations". `nc` and `qop` stay conditional on a negotiated + // qop -- a `deviations.md` row records the departure from AUTH-22's letter. + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + expect(value).toContain('cnonce="'); + expect(value).not.toContain('qop='); + expect(value).not.toContain('nc='); + }); + + test('the emitted cnonce is the one the response was computed with (-sess, no qop)', async () => { + // The row that makes the header VERIFIABLE rather than merely populated: a cnonce emitted from a + // second draw would satisfy the assertion above and still be worthless to the server. Recomputing + // HA1 from the header's own cnonce has to reproduce the header's own response. + const handler = digestHandler('Mufasa', 'Circle Of Life'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + const cnonce = /cnonce="(?[0-9a-f]+)"/u.exec(value)?.groups?.value; + const response = /response="(?[0-9a-f]+)"/u.exec(value)?.groups + ?.value; + expect(cnonce).toBeDefined(); + expect(response).toBe( + await computeDigestResponse({ + ...BASE, + isUtf8: false, // no charset on the challenge -- AUTH-21's ISO-8859-1 branch + algorithm: 'MD5-sess', + hasQopAuth: false, + cnonce: cnonce ?? '', + nc: '', + }), + ); + }); +}); + describe('digestHandler nonce-count sequencing (AUTH-18)', () => { - test('stamp() omits cnonce/nc/qop when the challenge negotiated no qop (AUTH-22)', async () => { + test('stamp() omits cnonce/nc/qop for a NON-sess algorithm with no qop (AUTH-22)', async () => { const handler = digestHandler('u', 'p'); + // No `algorithm` parameter, so AUTH-16's default applies: plain MD5, whose HA1 does not involve + // the client nonce at all. AUTH-22's "emit cnonce/nc/qop only when qop is negotiated" is exactly + // right here, and this is the branch it was written for. const challenge = digestChallenge({realm: REALM, nonce: NONCE}); const value = await handler.stamp(challenge, REQUEST_CONTEXT); expect(value).not.toContain('qop='); diff --git a/packages/core/src/auth/digest.ts b/packages/core/src/auth/digest.ts index ba4862d..fdfaa2a 100644 --- a/packages/core/src/auth/digest.ts +++ b/packages/core/src/auth/digest.ts @@ -233,10 +233,17 @@ function isHeaderSafeEcho(info: { } /** - * AUTH-16: satisfiable if and only if the scheme is `digest`, `realm` and `nonce` are both present, - * `qop` is absent or contains `auth`, the algorithm (defaulting to `MD5`) is in the caller's - * configured preference list, and every field AUTH-22 echoes back is header-safe + * AUTH-16: satisfiable if and only if the scheme is `digest`, `realm` and `nonce` both carry a + * non-empty value, `qop` is absent or contains `auth`, the algorithm (defaulting to `MD5`) is in the + * caller's configured preference list, and every field AUTH-22 echoes back is header-safe * ({@link isHeaderSafeEcho}). + * + * "Carries realm and nonce" is read as carrying a VALUE, not merely a key. A truncated header — + * `Digest realm="r", nonce=` — parses to `nonce: ''`, which is AUTH-12 and AUTH-13 both behaving + * exactly as specified: values are stored verbatim after unquoting, and what was parsed before a + * malformed tail is kept. Testing `=== undefined` here accepted that and sent `nonce=""` back, a + * response computed over a nonce no server can have issued. The challenge is declined instead, so the + * next one is tried and a 401 offering nothing else surfaces unchanged (AUTH-25, AUTH-33). */ function parseDigestChallenge( challenge: Challenge, @@ -245,7 +252,8 @@ function parseDigestChallenge( if (challenge.scheme !== 'digest') return undefined; const realm = challenge.params.get('realm'); const nonce = challenge.params.get('nonce'); - if (realm === undefined || nonce === undefined) return undefined; + if (realm === undefined || realm === '') return undefined; + if (nonce === undefined || nonce === '') return undefined; const qopRaw = challenge.params.get('qop'); const hasQop = qopRaw !== undefined; @@ -368,8 +376,19 @@ interface HeaderValueParams { /** * AUTH-22: quotes `username`/`realm`/`nonce`/`uri`/`response`/`cnonce`/`opaque`; leaves - * `qop`/`nc`/`algorithm` unquoted, with the full algorithm spelling; emits `cnonce`/`nc`/`qop` only - * when `qop` was actually negotiated. + * `qop`/`nc`/`algorithm` unquoted, with the full algorithm spelling; emits `nc`/`qop` only when `qop` + * was actually negotiated. + * + * **`cnonce` is emitted for every `-sess` algorithm, negotiated `qop` or not** — a deliberate + * departure from AUTH-22's letter, recorded in `docs/deviations.md`. A `-sess` HA1 is + * `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so a server handed a `-sess` response with + * no `cnonce` cannot recompute HA1 and cannot verify anything: the request is unanswerable by + * construction, and AUTH-30 bounds the replay to one 401, so it simply fails. RFC 7616 §3.4 states it + * outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's wording is RFC + * 2617's RFC 2069-compatibility form, which predates `-sess` entirely. + * + * `nc` stays conditional and stays out: RFC 2069's response input is `H(HA1:nonce:HA2)`, with no nonce + * count in it, so emitting one would advertise a count the response was not computed over. */ function buildHeaderValue(params: HeaderValueParams): string { const {username, info, uri, response, cnonce, nc} = params; @@ -385,6 +404,8 @@ function buildHeaderValue(params: HeaderValueParams): string { if (info.opaque !== undefined) parts.push(`opaque=${quote(info.opaque)}`); if (info.hasQopAuth) parts.push('qop=auth', `nc=${nc}`, `cnonce=${quote(cnonce)}`); + else if (info.algorithm.endsWith('-sess')) + parts.push(`cnonce=${quote(cnonce)}`); return `Digest ${parts.join(', ')}`; } diff --git a/packages/core/src/auth/preset.test.ts b/packages/core/src/auth/preset.test.ts index 1c1a614..733207b 100644 --- a/packages/core/src/auth/preset.test.ts +++ b/packages/core/src/auth/preset.test.ts @@ -3,10 +3,16 @@ // Exercises: PIPE-24 ("installs into empty pillar slots only" -- true by construction, since the preset // always starts from a fresh PipelineBuilder), PIPE-39 (installs exactly the pillars that exist), and // jointly with 5b: PIPE-2's "auth executes per redirect hop, not once for the whole call" plus -// AUTH-29's marker-CONSUMPTION side (5b produced the marker and routed consumption here). +// AUTH-29's marker-CONSUMPTION side (5b produced the marker and routed consumption here), OBS-29 + +// CTX-16 (the preset forwards an instrumentation bundle and an operation name to the built pipeline). import {describe, expect, test} from 'bun:test'; import {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; import {PipelineBuilder} from '../pipeline/builder.js'; import type {StepDescriptor} from '../pipeline/step.js'; import {CROSS_ORIGIN_MARKER_HEADER} from '../redirect/cross-origin.js'; @@ -216,3 +222,67 @@ describe('standardResilience logging options (Phase 7b)', () => { expect(events).toEqual(['http.request', 'http.response']); }); }); + +describe('standardResilience instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle opens one operation span per send, and the name reaches the step', async () => { + const spanNames: string[] = []; + const factoryNames: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + const tracer: Tracer = { + startSpan(name: string): Span { + spanNames.push(name); + return span; + }, + }; + let seen: string | undefined; + const probe: StepDescriptor = { + type: Symbol('operation-name-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen = + 'operationName' in ctx.context + ? ctx.context.operationName + : undefined; + return ctx.next(request); + }, + }; + + const runtime = PipelineBuilder.seedFrom( + standardResilience( + new FakeTransport([ + countingResponse(200).response, + countingResponse(200).response, + ]), + { + instrumentation: createInstrumentationBundle(operationName => { + factoryNames.push(operationName); + return tracer; + }), + operationName: 'GetUser', + }, + ), + 'flatten', + ) + .append(probe) + .build(); + + await runtime.send(aRequest()); + await runtime.send(aRequest()); + + // `Runtime.send()` asks the factory for the operation's own tracer once per call (OBS-29's 1:1 + // binding), and the LOGGING pillar asks for one labelled with CTX-16's operation name per attempt. + expect( + factoryNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(factoryNames.filter(name => name === 'GetUser')).toHaveLength(2); + expect( + spanNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(seen).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/auth/preset.ts b/packages/core/src/auth/preset.ts index 965ca4a..6317375 100644 --- a/packages/core/src/auth/preset.ts +++ b/packages/core/src/auth/preset.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/preset.ts -import {PipelineBuilder} from '../pipeline/builder.js'; +import {PipelineBuilder, type PipelineOptions} from '../pipeline/builder.js'; import type {Runtime} from '../pipeline/runtime.js'; import type {RedirectSettings} from '../redirect/settings.js'; import {withRedirect} from '../redirect/strip-marker-step.js'; @@ -20,7 +20,7 @@ import { * * @public */ -export interface StandardResilienceOptions { +export interface StandardResilienceOptions extends PipelineOptions { /** Retry settings and injected seams; omitted yields 5a's spec defaults. */ readonly retry?: RetryStepOptions | undefined; /** Redirect policy overrides; omitted yields 5b's spec defaults. */ @@ -62,6 +62,12 @@ function noAuthSettings(): AuthStepSettings { * `LOGGING` installs {@link loggingStep} to emit telemetry and metrics around dispatches. `SERDE` remains * reserved with no shipped behavior anywhere in this roadmap's current scope. * + * The two inherited {@link PipelineOptions} fields are pipeline-wide rather than per-pillar: + * `instrumentation` is the bundle every context of every call carries, and the source of the one + * `http.client.operation` span each `send()` opens (`OBS-29`); `operationName` is `CTX-16`'s advisory + * label. Omitting `instrumentation` leaves the no-op bundle in place, which opens no span at all — it + * is the only way to switch tracing on for a preset-built pipeline. + * * This function only assembles the pipeline. The failures below surface from the returned runtime's * `send()`, and are documented here because this factory is where a caller chooses the auth * configuration that determines whether they can occur at all. @@ -104,7 +110,12 @@ export function standardResilience( transport: Transport, options: StandardResilienceOptions = {}, ): Runtime { - const builder = new PipelineBuilder(transport); + // The two `PipelineOptions` fields are pipeline-wide rather than per-pillar, so they go to the + // builder rather than into a step's settings; everything below installs one pillar each. + const builder = new PipelineBuilder(transport, { + instrumentation: options.instrumentation, + operationName: options.operationName, + }); return withRedirect(builder, options.redirect) .append(retryStep(options.retry)) .append(authStep(options.auth ?? noAuthSettings())) diff --git a/packages/core/src/body/multipart-body.test.ts b/packages/core/src/body/multipart-body.test.ts index 5b03d39..d616008 100644 --- a/packages/core/src/body/multipart-body.test.ts +++ b/packages/core/src/body/multipart-body.test.ts @@ -1,8 +1,9 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/multipart-body.test.ts // Exercises: BODY-2 (composite replayability, unknown-length collapse), HTTP-51 (shared framing routine, -// boundary generation/validation, header quoting, and a part media type that cannot break the framing), -// HTTP-26 (a media type is header-safe), RECOV-12 (a close failure never masks the primary failure) +// boundary generation/validation, header quoting, a boundary parameter rendered so an RFC 9110 parser +// can read it, and a part media type that cannot break the framing), HTTP-26 (a media type is +// header-safe), RECOV-12 (a close failure never masks the primary failure) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {MediaTypeParseError} from '../http/errors.js'; @@ -66,9 +67,11 @@ function collectingSink(): { }; } -async function drain(body: { +// `Uint8Array`, not the bare alias: `BodyInit` excludes a view over a `SharedArrayBuffer`, +// so the default `ArrayBufferLike` parameter is not assignable to the platform `Response` below. +async function drainBytes(body: { writeTo: (sink: WritableStream) => Promise; -}): Promise { +}): Promise> { const chunks: Uint8Array[] = []; await body.writeTo(new WritableStream({write: c => void chunks.push(c)})); const total = chunks.reduce((s, c) => s + c.length, 0); @@ -78,7 +81,13 @@ async function drain(body: { out.set(c, offset); offset += c.length; } - return new TextDecoder().decode(out); + return out; +} + +async function drain(body: { + writeTo: (sink: WritableStream) => Promise; +}): Promise { + return new TextDecoder().decode(await drainBytes(body)); } describe('MultipartBody replayability and length (BODY-2)', () => { @@ -186,6 +195,47 @@ describe('MultipartBody boundary generation and validation (HTTP-51)', () => { }); }); +describe('the rendered boundary parameter is a parseable one (HTTP-51)', () => { + // RFC 2046 `bchars` and RFC 9110 `tchar` are different sets: ' ', ',', ':', '=', '?', '/', '(' and + // ')' are legal in a boundary and illegal bare in a header parameter value. `validateBoundary` admits + // the first grammar and the renderer owes the second, so a boundary the constructor accepts must come + // back out quoted rather than bare. + test('a boundary that is not a bare token is quoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'a,b').mediaType, + ).toBe('multipart/form-data; boundary="a,b"'); + }); + + test('a boundary that IS a bare token is left unquoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1').mediaType, + ).toBe('multipart/form-data; boundary=plain-1'); + // The generated default stays byte-identical: it is drawn from ALPHA/DIGIT only. + expect( + multipartBody([{name: 'a', body: stringBody('x')}]).mediaType, + ).toMatch(/^multipart\/form-data; boundary=dexpace-[A-Za-z0-9]{32}$/); + }); + + test.each([['a,b'], ['bound ary'], ['a:b'], ['a=b'], ['a?b'], ['(a)/b']])( + 'the header a peer receives round-trips through a real parameter parser: %p', + async boundary => { + // The runtime's own multipart parser, standing in for the peer. Bun's happens to tolerate the + // unquoted form, so this is the regression guard and NOT the reproducer: Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`, which is why the + // same case is also in `tests/node-conformance/body-lifecycle.test.mjs`. Two independent + // parsers disagreeing about our own Content-Type is exactly what that tree is for. + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const response = new globalThis.Response(await drainBytes(body), { + headers: {'content-type': body.mediaType}, + }); + expect((await response.formData()).get('field')).toBe('value'); + }, + ); +}); + describe('MultipartBodyBuilder (HTTP-2, HTTP-3)', () => { test('static newBuilder and instance newBuilder pre-populates parts and boundary', async () => { const original = MultipartBody.newBuilder() diff --git a/packages/core/src/body/multipart-body.ts b/packages/core/src/body/multipart-body.ts index 6df3020..e1affd3 100644 --- a/packages/core/src/body/multipart-body.ts +++ b/packages/core/src/body/multipart-body.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/multipart-body.ts import type {Builder} from '../http/builder.js'; +import {MediaType} from '../http/media-type.js'; import {EndOfStreamError} from '../io/errors.js'; import {invariant} from '../invariant.js'; import type {Body} from './body.js'; @@ -77,6 +78,31 @@ function renderPartHeader(part: MultipartPart, boundary: string): Uint8Array { return new TextEncoder().encode(header); } +/** + * HTTP-51: the `Content-Type` a peer actually parses. + * + * RFC 2046 `bchars` and RFC 9110 `tchar` are different sets. `BOUNDARY_PATTERN` above admits ' ', ',', + * ':', '=', '?', '/', '(' and ')', none of which is a `tchar`, so interpolating the boundary bare + * produces a parameter value that stops at the first offending byte -- `boundary=a,b` reads as + * `boundary=a` plus a junk parameter, and the peer then never finds a delimiter. Node's own FormData + * parser rejects such a body outright with `TypeError: Failed to parse body as FormData`. + * + * Rendered through {@link MediaType} rather than a second quoting routine here: it is the module that + * owns HTTP-25's token-or-quoted-string decision, and `parse(render(x)) === x` is its guarantee. A + * boundary that IS a bare token still renders bare, so the generated default is byte-identical to what + * this class emitted before. + * + * Narrowing `validateBoundary` to `tchar` instead was rejected: HTTP-51 asks that a boundary VIOLATING + * the RFC 2046 grammar be refused, not that a conforming one be. The defect is in the rendering. + */ +function renderMediaType(boundary: string): string { + return MediaType.of( + 'multipart', + 'form-data', + new Map([['boundary', boundary]]), + ).render(); +} + function trailerBytes(boundary: string): Uint8Array { return new TextEncoder().encode(`--${boundary}--\r\n`); } @@ -144,7 +170,10 @@ function nonClosingSink( export class MultipartBody implements Body { /** Discriminates this variant within the {@link Body} union. */ readonly kind = 'multipart' as const; - /** `multipart/form-data` carrying the boundary this instance frames its parts with. */ + /** + * `multipart/form-data` carrying the boundary this instance frames its parts with, with the + * `boundary` parameter quoted whenever it is not a bare RFC 9110 token (HTTP-51). + */ readonly mediaType: string; /** The total framed byte count, or -1 when any part's own length is unknown (BODY-2). */ readonly contentLength: number; @@ -157,7 +186,7 @@ export class MultipartBody implements Body { if (boundary !== undefined) validateBoundary(boundary); this.#boundary = boundary ?? generateBoundary(); this.#parts = [...parts]; - this.mediaType = `multipart/form-data; boundary=${this.#boundary}`; + this.mediaType = renderMediaType(this.#boundary); this.replayable = this.#parts.every(part => part.body.replayable); this.contentLength = computeContentLength(this.#parts, this.#boundary); invariant( diff --git a/packages/core/src/body/request-body-logging.ts b/packages/core/src/body/request-body-logging.ts index 28779a8..c52165d 100644 --- a/packages/core/src/body/request-body-logging.ts +++ b/packages/core/src/body/request-body-logging.ts @@ -73,6 +73,10 @@ function tappedSink(state: TapState): WritableStream { * Mirrors up to tapCapBytes of each writeTo call into an internal tap while forwarding the full, * untruncated payload to the primary sink (BODY-17). The tap clears at the start of every write so a * retry against a replayable delegate does not accumulate stale bytes (BODY-18). No handle onto the tap's + * buffer escapes: `snapshot()` returns a fresh, independent copy of the current contents (BODY-19, + * `../io/byte-queue.ts:96`), so a caller holding one cannot observe or disturb a later write. Each + * `materialize()` wraps its own `ByteQueue` for the same reason (BODY-21). + * * @internal Consumed by the LOGGING pillar step (OBS-36). */ export function withRequestLogging( diff --git a/packages/core/src/body/response-body-logging.test.ts b/packages/core/src/body/response-body-logging.test.ts index d118273..9fe6b83 100644 --- a/packages/core/src/body/response-body-logging.test.ts +++ b/packages/core/src/body/response-body-logging.test.ts @@ -4,11 +4,16 @@ // reads), BODY-24 (exceeds-cap: prefix+tail once, second read fails), BODY-26 (drain failure cached, // partial bytes retained, error() does not drain), BODY-27 (close-once shared guard), BODY-28 (captured // buffer survives close), BODY-29 (reported length), BODY-32 (negative cap rejected), BODY-25 (a -// zero-length delegate chunk is a stream-contract violation, never end-of-stream) +// zero-length delegate chunk is a stream-contract violation, never end-of-stream), BODY-27/BODY-28 again +// (close() ends the drain rather than poisoning the wrapper: snapshot still serves the captured prefix, +// read() reports IO-42's state error, error() reports only a genuine drain failure) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {InvariantViolation} from '../invariant.js'; -import {SourceContractViolationError} from '../io/errors.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; import {withResponseLogging} from './response-body-logging.js'; function readableOf(...chunks: number[][]): ReadableStream { @@ -282,6 +287,87 @@ describe('the tail path enforces the same chunk contract (BODY-25)', () => { }); }); +describe('the tap is inert after close(), not poisoned by it (BODY-27, BODY-28)', () => { + // closeDelegate releases the reader. Every entry point that used to start a drain unconditionally then + // read from a detached reader, so `snapshot()` cached a raw `TypeError: Invalid state: The reader is + // not attached to a stream` as the wrapper's failure -- and `error()` reported that forever, over a + // capture that never failed. BODY-28 says the captured bytes survive close; they cannot survive it + // behind a fabricated error. + + test('close-then-snapshot returns the captured prefix and starts no drain', async () => { + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + await logged.read(); // exceeds-cap: the prefix is captured, the delegate stays live + await logged.close(); + + expect([...logged.snapshot()]).toEqual([1, 2]); + await new Promise(resolve => setTimeout(resolve, 0)); // a drain started here would settle by now + expect(logged.error()).toBeNull(); + }); + + test('close-before-any-read leaves snapshot empty and error() null', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + expect([...logged.snapshot()]).toEqual([]); + await new Promise(resolve => setTimeout(resolve, 0)); + expect(logged.error()).toBeNull(); + }); + + test('close-then-read rejects with ClosedResourceError, not a detached-reader TypeError', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + const error = await rejection(logged.read()); + expect(error).toBeInstanceOf(ClosedResourceError); + expect(error.message).toBe('LoggedResponseBody is closed'); + }); + + test('close-then-read in the exceeds-cap regime rejects too -- there is no live tail left', async () => { + // The drain stopped at the cap and nobody took the tail, so the captured prefix is NOT the whole + // body. Serving it would hand the consumer a silently truncated response; the delegate that held + // the rest is gone. + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + logged.snapshot(); // starts the drain without taking the tail + await new Promise(resolve => setTimeout(resolve, 0)); + expect([...logged.snapshot()]).toEqual([1, 2]); + await logged.close(); + + expect(await rejection(logged.read())).toBeInstanceOf(ClosedResourceError); + }); + + test('close-then-error reports only a genuine drain failure (BODY-26)', async () => { + const boom = new Error('upstream reset'); + const failing = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + }, + pull(controller) { + controller.error(boom); + }, + }); + const logged = withResponseLogging(failing, 100); + expect(await rejection(logged.read())).toBe(boom); + // `cancel()` on an errored stream rejects with that stream's own stored error, which the + // 'a non-TypeError from cancel() propagates' case above already pins. Not what this test is about. + await logged.close().catch(() => undefined); + + // The real failure is not displaced by a close-induced one, and snapshot still shows the partial + // capture BODY-26 asked to be retained. + expect(logged.error()).toBe(boom); + expect([...logged.snapshot()]).toEqual([1, 2]); + }); + + test('the fits-cap regime still serves repeatable reads after its own close (BODY-23, BODY-28)', async () => { + // The drain itself closes the delegate on this path, so "closed" must NOT mean "unreadable". + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + await logged.close(); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); + expect(logged.error()).toBeNull(); + }); +}); + describe('snapshot is a drain trigger (BODY-22)', () => { test('calling snapshot starts the drain, without a read()', async () => { const logged = withResponseLogging(readableOf([1, 2, 3]), 100); diff --git a/packages/core/src/body/response-body-logging.ts b/packages/core/src/body/response-body-logging.ts index a5ebc0a..0e23e01 100644 --- a/packages/core/src/body/response-body-logging.ts +++ b/packages/core/src/body/response-body-logging.ts @@ -2,7 +2,10 @@ // packages/core/src/body/response-body-logging.ts import {invariant} from '../invariant.js'; import {ByteQueue} from '../io/byte-queue.js'; -import {SourceContractViolationError} from '../io/errors.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; import {MAX_BYTE_ARRAY_LENGTH} from '../io/limits.js'; import {ConsumedBodyError} from './errors.js'; @@ -20,11 +23,19 @@ export interface LoggedResponseBody { * (BODY-22). Fits-cap regime: every call, including calls after the first, returns a fresh * non-consuming view over the captured bytes (BODY-23). Exceeds-cap regime: exactly one call is * allowed; a second throws (BODY-24). If the drain failed, every call re-throws the cached error. + * After `close()` in any regime but fits-cap, throws `ClosedResourceError`: the delegate is gone and + * the captured prefix is not the whole body (BODY-27, BODY-28). */ read(): Promise>; - /** Non-consuming; reflects whatever has been captured so far, even after a failed drain (BODY-26). */ + /** + * Non-consuming; reflects whatever has been captured so far, even after a failed drain (BODY-26) and + * after `close()` (BODY-28), which it never restarts a drain past. + */ snapshot(): Uint8Array; - /** The cached drain failure, or null. MUST NOT trigger a drain (BODY-26). */ + /** + * The cached drain failure, or null. MUST NOT trigger a drain (BODY-26), and reports only a genuine + * upstream failure -- never one manufactured by reading past `close()`. + */ error(): Error | null; /** Captured size iff fully captured within the cap, else the delegate's declared length (BODY-29). */ readonly contentLength: number; @@ -129,8 +140,16 @@ async function drainOnce(state: DrainState): Promise { * The detached `.catch` matters: a snapshot-triggered drain has no awaiter, so without it a drain failure * becomes an unhandled rejection. Attaching a handler to a *copy* leaves the stored promise rejected, so * `read()` still re-throws the cached failure on every call (BODY-26). + * + * BODY-28: after `close()` there is nothing left to drain -- `closeDelegate` released the reader, so + * starting one here reads from a detached reader, raises a raw `TypeError: Invalid state`, and + * `drainOnce`'s catch caches it as this wrapper's `failure`. `error()` would then report a fabricated + * upstream failure forever, over a capture that never failed, and the captured bytes BODY-28 promises + * survive close would be reachable only past that lie. A drain already in flight is left alone: on the + * fits-cap path the drain closes the delegate itself, and its own promise is what `read()` awaits. */ function startDrain(state: DrainState): Promise { + if (state.closed && state.started === undefined) return Promise.resolve(); state.started ??= drainOnce(state); void state.started.catch(() => undefined); return state.started; @@ -226,10 +245,17 @@ export function withResponseLogging( return { async read(): Promise> { await startDrain(state); // a cached failure re-throws here on every call (BODY-26) + // Ordered deliberately. `fits` first: on that path the drain closed the delegate itself, and + // BODY-23 still requires every later read to be a fresh non-consuming view -- "closed" there does + // not mean "unreadable" (BODY-28). if (state.regime === 'fits') return capturedStream(state); if (state.tailConsumed) { throw new ConsumedBodyError('logged-response'); } + // Anything else with the delegate gone: there is no live tail to continue from, and the captured + // prefix is not the whole body, so serving it would hand the consumer a silently truncated + // response. IO-42's state error, not the raw `TypeError` a detached reader throws. + if (state.closed) throw new ClosedResourceError('LoggedResponseBody'); state.tailConsumed = true; return tailStream(state); }, @@ -238,6 +264,7 @@ export function withResponseLogging( // so it starts the drain and returns what has been captured so far rather than awaiting it; a // later read() awaits the very same in-flight promise, so the delegate is still read exactly once. // (BODY-26's "snapshot returns the partial bytes without throwing" is why it cannot await here.) + // After close() `startDrain` is a no-op, so this is the post-mortem accessor BODY-28 asks for. void startDrain(state); return state.captured.snapshot(); }, diff --git a/packages/core/src/body/stream-body.test.ts b/packages/core/src/body/stream-body.test.ts index 3530875..b966275 100644 --- a/packages/core/src/body/stream-body.test.ts +++ b/packages/core/src/body/stream-body.test.ts @@ -6,11 +6,13 @@ // delivered-of-declared, and an overrunning stream is stopped BEFORE the extra bytes reach the sink), // IO-3 (a contentLength below the -1 sentinel is rejected), HTTP-26/HTTP-51 (a media type is // header-safe), RECOV-12 (a close failure never masks the primary write failure), HTTP-1 (frozen at -// construction so the declared length cannot be desynced from the written bytes) +// construction so the declared length cannot be desynced from the written bytes), HTTP-39/BODY-10 again +// (a zero-length delivery during an exact-length copy is a source-contract violation, never a no-op and +// never spun on, and no empty chunk reaches the sink) import {describe, expect, test} from 'bun:test'; import {MediaTypeParseError} from '../http/errors.js'; import {InvariantViolation} from '../invariant.js'; -import {EndOfStreamError} from '../io/errors.js'; +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; import {ConsumedBodyError} from './errors.js'; import {streamBody} from './stream-body.js'; @@ -49,6 +51,16 @@ function collectingSink(): { }; } +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves instead. */ +async function rejection(promise: Promise): Promise { + try { + await promise; + } catch (error: unknown) { + return error; + } + throw new Error('expected a rejection, but the promise resolved'); +} + describe('caller stream ownership (BODY-8)', () => { test('a sink failure does not cancel the caller stream on the unknown-length path', async () => { // `pipeTo`'s default (`preventCancel: false`) cancels the SOURCE when the destination errors, @@ -230,6 +242,85 @@ describe('a mis-framed body never reaches the wire (HTTP-39/BODY-10)', () => { }); }); +describe('a zero-length delivery is a source-contract violation (HTTP-39/BODY-10)', () => { + /** + * The auditor's probe, bounded so an unguarded run terminates instead of hanging the suite: a source + * that delivers nothing but empty chunks, then finally ends. Unbounded is the real-world shape, and + * the pull count below is what proves the copy did not spin on it. + */ + function emptyOnly(limit: number): { + stream: ReadableStream; + pulls: () => number; + } { + let pulls = 0; + const stream = new ReadableStream({ + pull(controller) { + pulls += 1; + if (pulls > limit) { + controller.close(); + return; + } + controller.enqueue(new Uint8Array(0)); + }, + }); + return {stream, pulls: () => pulls}; + } + + test('an empty-only source raises on the first empty chunk rather than spinning on it', async () => { + const {stream, pulls} = emptyOnly(1000); + const {state, sink} = probeSink(); + const error = await rejection( + streamBody(stream, undefined, 3).writeTo(sink), + ); + + // Not EndOfStreamError after a thousand futile reads: an unbounded source of empty chunks never + // ends, so nothing downstream can ever diagnose it, and every one of those chunks reaches the sink. + expect(error).toBeInstanceOf(SourceContractViolationError); + // Two, not one: the default queuing strategy reads one chunk ahead, so the source is pulled again + // the moment our read drains its queue. What matters is that it is not `limit`. + expect(pulls()).toBeLessThanOrEqual(2); + expect(state.written).toEqual([]); + expect(state.aborted).toBe(true); // a mis-framed body is never signalled as a clean close + }); + + test('an empty chunk between real chunks is raised, and never reaches the sink', async () => { + const emptyBetween = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths: number[] = []; + const sink = new WritableStream({ + write: chunk => void chunkLengths.push(chunk.length), + }); + + expect( + await rejection(streamBody(emptyBetween, undefined, 3).writeTo(sink)), + ).toBeInstanceOf(SourceContractViolationError); + // `io/buffered-sink.ts` writeString: a zero-length chunk is HTTP/1.1 chunked encoding's TERMINATING + // chunk, so forwarding one ends the request body early on the wire. + expect(chunkLengths).toEqual([2]); + }); + + test('a declared length of 0 over a source that just closes stays a legitimate empty write (BODY-10)', async () => { + const {state, sink} = probeSink(); + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo(sink); + expect(state.written).toEqual([]); + expect(state.closed).toBe(true); + }); +}); + describe('StreamBody media type and failure propagation', () => { test('rejects a media type carrying CR/LF (HTTP-26/HTTP-51)', () => { expect(() => diff --git a/packages/core/src/body/stream-body.ts b/packages/core/src/body/stream-body.ts index c0ba250..093e0a7 100644 --- a/packages/core/src/body/stream-body.ts +++ b/packages/core/src/body/stream-body.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/stream-body.ts -import {EndOfStreamError} from '../io/errors.js'; +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; import {invariant} from '../invariant.js'; import type {Body} from './body.js'; import {ConsumedBodyError} from './errors.js'; @@ -8,6 +8,33 @@ import {freezeBody} from './freeze-body.js'; import {assertHeaderSafeMediaType} from './media-type-safety.js'; import {withBodyWriter} from './write-body.js'; +/** + * HTTP-39/BODY-10: a zero-length delivery during an exact-length copy is a source-contract violation, + * never end-of-stream and never something to spin on -- `{done: true}` is the only end signal. + * + * Two failure modes, and neither one is diagnosable anywhere else. A source that only ever yields empty + * chunks never ends, so the delivered-of-declared check below is never reached; and every empty chunk + * that gets past here reaches the transport sink, where to an HTTP/1.1 chunked-encoding transport a + * zero-length chunk is the TERMINATING chunk (`io/buffered-sink.ts`'s `writeString`) -- so tolerating + * one ends the request body early on the wire while the copy still believes it is mid-body. + * + * The wording and the error type are `io/retention-window.ts`'s deliberately: `read()` carries no + * requested count, so the requirement's "for a positive requested count" has no literal analog, and a + * request body reaches both this copy and `BufferedSource` -- a divergence would make the same upstream + * fail or succeed depending only on which wrapper it passed through. `body/response-body-logging.ts` + * makes the same call for BODY-25 on the response side. + * + * A declared length of 0 is still a legitimate empty write (BODY-10): that is a source that signals + * `{done: true}` immediately, which never reaches this check. + */ +function assertNonEmptyChunk(value: Uint8Array): void { + if (value.length === 0) { + throw new SourceContractViolationError( + 'source delivered 0 bytes without signalling end of stream', + ); + } +} + /** * A single-use body backed by a caller-supplied stream. * @@ -50,6 +77,8 @@ export class StreamBody implements Body { * @throws {@link ConsumedBodyError} on a second call -- this body is single-use (BODY-3). * @throws EndOfStreamError when a declared `contentLength` disagrees with the bytes the stream * actually yields, in either direction (HTTP-39/BODY-10). + * @throws SourceContractViolationError when the stream delivers a zero-length chunk without + * signalling end of stream during a declared-length write (HTTP-39/BODY-10). */ async writeTo(sink: WritableStream): Promise { if (this.#consumed) throw new ConsumedBodyError('stream'); @@ -80,6 +109,7 @@ export class StreamBody implements Body { // Serial by necessity: each read depends on the previous one advancing the cursor. const {done, value} = await reader.read(); if (done) break; + assertNonEmptyChunk(value); // HTTP-39/BODY-10 // Checked BEFORE the write, not after the loop: once a transport has stamped the declared // Content-Length, an overrun byte sits on the socket where the peer reads it as the start of // the next message, and a thrown error cannot recall bytes already written (HTTP-39/BODY-10). @@ -109,6 +139,8 @@ export class StreamBody implements Body { * @throws ConsumedBodyError from `writeTo` when the body has already been written once (BODY-3). * @throws EndOfStreamError from `writeTo` when the stream yields a byte count other than the declared * `contentLength` (HTTP-39/BODY-10). + * @throws SourceContractViolationError from `writeTo` when the stream delivers a zero-length chunk + * without signalling end of stream during a declared-length write (HTTP-39/BODY-10). * @public */ export function streamBody( diff --git a/packages/core/src/context/instrumentation.test.ts b/packages/core/src/context/instrumentation.test.ts index acd5f3f..d647d4d 100644 --- a/packages/core/src/context/instrumentation.test.ts +++ b/packages/core/src/context/instrumentation.test.ts @@ -3,6 +3,7 @@ // Exercises: CTX-14 (bundle shape), CTX-15 (no-op default: invalid sentinels, isValid/isRemote false, // no-op span/tracer factory), CTX-20 (tracer factory safe to invoke concurrently, emits nothing) import {describe, expect, test} from 'bun:test'; +import {NOOP_SPAN} from '../observability/span.js'; import {noopInstrumentationBundle} from './instrumentation.js'; describe('noopInstrumentationBundle (CTX-15)', () => { @@ -26,11 +27,14 @@ describe('noopInstrumentationBundle (CTX-15)', () => { expect(noopInstrumentationBundle.isRemote).toBe(false); }); - // CTX-15 says "a no-op span". With `activeSpan` typed `unknown` until a real tracing adapter lands - // (Phase 7), there is no Span shape to build a no-op instance of, so absence is the encoding. Logged as a - // partial deviation in the design's Deviation Ledger -- revisit when the adapter defines Span. - test('has no active span', () => { - expect(noopInstrumentationBundle.activeSpan).toBeUndefined(); + // CTX-15 says "a no-op span", and it means an object: the requirement lists the no-op span beside the + // no-op tracer factory, and `createInstrumentationBundle` has always used `NOOP_SPAN` for the ENABLED + // bundle (`observability/tracing.ts`). Phase 4a shipped `undefined` here because no `Span` type existed + // yet and recorded the gap as a partial deviation; `Span`/`NOOP_SPAN` landed in Phase 7b, so the reason + // expired and the two bundles no longer disagree. Identity, not shape: a caller narrowing `unknown` may + // compare against the exported singleton. + test('carries the no-op span singleton, not undefined', () => { + expect(noopInstrumentationBundle.activeSpan).toBe(NOOP_SPAN); }); test('tracerFactory emits nothing and is safe to invoke repeatedly (CTX-20)', () => { diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts index 4bd4d15..5eb7b08 100644 --- a/packages/core/src/context/instrumentation.ts +++ b/packages/core/src/context/instrumentation.ts @@ -1,16 +1,23 @@ // SPDX-License-Identifier: MIT // packages/core/src/context/instrumentation.ts +import {NOOP_SPAN} from '../observability/span.js'; /** * Correlation/instrumentation bundle every execution context carries (CTX-14), reachable from a * custom step as `ctx.context.instrumentation`. * - * **Two members are provisional.** `activeSpan` and `tracerFactory` are typed `unknown` rather than a - * Span/Tracer interface because nothing in the package consumes either yet, and the real tracing - * adapter — Phase 7a — owns their eventual shape. They are published as `unknown` deliberately, not - * accidentally: narrowing them later to a concrete Span/Tracer type is a widening of what a caller - * may pass and a narrowing of what they receive, so code that reads either today should treat it as - * opaque and re-check when 7a lands. Every other member below is stable. + * **Two members are still typed `unknown`, and the reason this note used to give is no longer true.** + * `activeSpan` and `tracerFactory` are `unknown` rather than a `Span`/`Tracer`. The original reason — + * "nothing in the package consumes either yet, pending Phase 7a" — expired when Phase 7a landed + * (`bd37a08`) and shipped `Span` and `Tracer` in `observability/tracing.ts`. That phase did not narrow + * these two: `tracerFactory` is consumed, by `pipeline/runtime.ts:60-65` and + * `observability/logging-step.ts:305-312`, both of which reach a `Tracer` through a cast; + * `createInstrumentationBundle` fills `activeSpan` (`observability/tracing.ts:191`), and nothing in + * this package reads it back. + * + * They stay `unknown` because narrowing a published member is a breaking change rather than a + * maintenance edit — it widens what a caller may pass and narrows what they receive — so it belongs + * to a deliberate version bump. Treat either as opaque until then. Every other member below is stable. * * @public */ @@ -30,19 +37,38 @@ export interface InstrumentationBundle { /** Whether the trace context was propagated in from a caller rather than started locally. */ readonly isRemote: boolean; /** - * The span this call runs inside, or `undefined` when tracing is disabled. + * The span this call runs inside. Never absent: the disabled-tracing default carries the inert + * `NOOP_SPAN` singleton, which is what CTX-15's "a no-op span" asks for. * * PROVISIONAL: typed `unknown` pending Phase 7a's tracing adapter — see this interface's own note. */ readonly activeSpan: unknown; /** - * Starts a child span for `operationName`. A no-op returning `undefined` when tracing is disabled. + * Returns the tracer to open `operationName`'s span from — a **tracer**, not a started span. Every + * consumer in this package narrows the result and calls `startSpan()` on it itself + * (`packages/core/src/pipeline/runtime.ts:60-65`, `observability/logging-step.ts:305-312`), and + * `createInstrumentationBundle` supplies a `(operationName: string) => Tracer` + * (`observability/tracing.ts:180,191`). A no-op returning `undefined` when tracing is disabled; + * both consumers substitute `NOOP_TRACER` for that `undefined`. + * + * **It is asked twice per call, for two different scopes.** `Runtime.send()` asks for + * `'http.client.operation'`'s tracer once per call and opens `OBS-29`'s one-per-operation span from + * it, outside every pillar, so a retry attempt and a redirect hop stay inside it. The LOGGING pillar + * step asks again per transmission, under `CTX-16`'s operation name when the pipeline was built with + * one and `'http.client.request'` otherwise, and its spans are children of the first. Returning one + * shared tracer for both is fine; `OBS-29`'s 1:1 clause is about the operation *span*, which + * `send()` opens exactly once whatever this returns. + * + * A consumer reaches this by building a bundle with `createInstrumentationBundle(tracerFactory)` and + * passing it as `PipelineOptions.instrumentation` — to `new PipelineBuilder(transport, options)` or + * to `standardResilience(transport, options)`. Before 2026-09-05 there was no public route: every + * pipeline a consumer could build carried the no-op bundle (audit #67 / #80). * - * PROVISIONAL: the return type is `unknown` pending Phase 7a's tracing adapter — see this - * interface's own note. + * PROVISIONAL: the return type is `unknown`, and narrowing it is a version-bump decision — see this + * interface's own note for why Phase 7a landing did not settle it. * - * @param operationName - the operation to name the child span after. - * @returns the started span, or `undefined` when tracing is disabled. + * @param operationName - the operation whose span the returned tracer will be asked to start. + * @returns the tracer for that operation, or `undefined` when tracing is disabled. */ readonly tracerFactory: (operationName: string) => unknown; } @@ -62,6 +88,6 @@ export const noopInstrumentationBundle: InstrumentationBundle = Object.freeze({ traceIdEncoding: 'none', isValid: false, isRemote: false, - activeSpan: undefined, + activeSpan: NOOP_SPAN, tracerFactory: () => undefined, }); diff --git a/packages/core/src/http/builder.ts b/packages/core/src/http/builder.ts index 5411659..c4bc1e4 100644 --- a/packages/core/src/http/builder.ts +++ b/packages/core/src/http/builder.ts @@ -20,6 +20,22 @@ export interface Builder { build(): T; } +/** + * The one frozen empty list every multi-value accessor returns for an absent name. + * + * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: + * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and the + * TSDoc on `Headers.getAll` and `QueryParams.getAll` promises a frozen list on every path. Both + * returned a fresh `[]` on a miss, which was neither (audit #67 / #76). Sharing one instance is + * safe precisely because it is frozen — there is no state in it to alias, and no caller can add + * any. + * + * Lives here rather than in either model because both need it and the two models deliberately + * import nothing from each other; this module is already the shared construction helper they both + * import from. + */ +export const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); + /** * Returns `value` when present, throwing a field-named error when it is `null` or `undefined`. * diff --git a/packages/core/src/http/errors.ts b/packages/core/src/http/errors.ts index 9a3d055..bc8e01a 100644 --- a/packages/core/src/http/errors.ts +++ b/packages/core/src/http/errors.ts @@ -127,16 +127,43 @@ export class MediaTypeParseError extends DexpaceError {} export class ProtocolParseError extends DexpaceError {} /** - * Thrown when a request URL is malformed or not absolute; the message carries the offending input - * and the underlying parse failure is chained as `cause` (HTTP-47). + * Thrown when a URL cannot be constructed from what a caller supplied. + * + * Three cases, all of them "this input has no URL form": + * + * - A request URL that is malformed or not absolute (HTTP-47). The message carries the offending + * input and the underlying parse failure is chained as `cause`. + * - A base URL handed to `buildRequest()` that is malformed, not absolute, or carries a fragment + * (SEAM-27). + * - A query-parameter name or value carrying an unpaired surrogate. Such a string has no UTF-8 + * form, so RFC 3986 percent-encoding is undefined for it and `QueryParams.encode()` could only + * fail; `QueryParamsBuilder.add` rejects it at the call that supplied it instead. `cause` is not + * set on this path — nothing was caught, the input was inspected (HTTP-29, audit #67 / #76). + * `QueryParams.parse` does NOT throw it: HTTP-31 makes parsing lenient, so it substitutes U+FFFD. + * Pagination's query splice — `spliceQueryParam` and `readQueryParam`, behind `cursorStrategy` + * and `pageNumberStrategy` — rejects the same input for the same reason, through the same + * predicate (PAGE-22, audit #67 / #79). That one is reachable without any caller mistake, since + * the cursor is server-supplied; its message names the parameter and never echoes the value. * * @public */ export class UrlConstructionError extends DexpaceError {} /** - * Thrown when a per-call operational override is out of range — a non-null timeout that is zero or - * negative, or a negative max-retries (HTTP-35). + * Thrown when a per-call operational override is out of range (HTTP-35). + * + * The ranges checked are the FULL ranges, not the lower bounds the requirement's own wording names. + * HTTP-35's point is that an out-of-range override is a loud error at the call site that supplied + * it, never a value reinterpreted downstream, and a value that only *some* consumer refuses is the + * same failure moved one seam away: + * + * - `timeoutMs` must be an integer in `1 .. 2**32 - 1` — the range `AbortSignal.timeout()` accepts, + * which is the only one a transport can honour. Zero, negatives, `Infinity`, `NaN`, a fractional + * millisecond and anything above the ceiling are all rejected. Zero is rejected rather than + * reinterpreted: it means "no timeout" in one transport and is an error in another. + * - `maxRetries` must be a non-negative integer. `0` is accepted and means "disable retries for + * this call", distinct from `undefined`; `Infinity` and `NaN` are rejected because they make a + * retry driver's ceiling test permanently false and its loop unbounded. * * @public */ diff --git a/packages/core/src/http/headers.test.ts b/packages/core/src/http/headers.test.ts index 762158a..618e6bb 100644 --- a/packages/core/src/http/headers.test.ts +++ b/packages/core/src/http/headers.test.ts @@ -9,7 +9,9 @@ // XCUT-15's ingested-collection clause (a builder defensively copies what it is handed, so mutating that // collection after build() cannot alter the built model, and a derived builder never aliases its source), // HTTP-17 (outbound name validation + trim), HTTP-18 (outbound value validation), HTTP-19 (inbound leniency), -// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop) +// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop), +// HTTP-5 again (getAll returns a FROZEN list on both the present-name and the absent-name path), +// HTTP-13 once more (Headers.equals asserted directly, not only through Request.equals) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {Headers, HeaderName} from './headers.js'; @@ -251,3 +253,119 @@ describe('HeaderName.lowerCased (HTTP-21)', () => { expect(name.raw).toBe('Content-Type'); }); }); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const headers = Headers.newBuilder() + .add('X-Tag', 'a') + .add('X-Tag', 'b') + .build(); + + test('the present-name list is frozen', () => { + // Asserted directly for the first time by audit #67 / #76. `build()` freezes each value list, + // and `getAll` returns that same reference rather than a copy, so the freeze is the whole of + // HTTP-5's "cannot mutate the model through the returned value" on this accessor — if the + // freeze were ever dropped, nothing else would notice. + const values = headers.getAll('X-Tag'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('c')).toThrow(TypeError); + expect(headers.getAll('X-Tag')).toEqual(['a', 'b']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + // It was a fresh `[]` — unfrozen, against the TSDoc's promise of a frozen list, and a fresh + // allocation on every miss. + const first = headers.getAll('nope'); + const second = headers.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); + +// Reached only through `Request.equals` until audit #67 / #76. Each case below is a way the +// comparison could be wrong that a Request-level test would not isolate. +function buildHeaders(pairs: readonly (readonly [string, string])[]): Headers { + const builder = Headers.newBuilder(); + for (const [name, value] of pairs) builder.add(name, value); + return builder.build(); +} + +describe('Headers.equals directly (HTTP-13): names and casing', () => { + test('is reflexive and true for an identical construction', () => { + const a = buildHeaders([['X-A', '1']]); + expect(a.equals(a)).toBe(true); + expect(a.equals(buildHeaders([['X-A', '1']]))).toBe(true); + }); + + test('name casing does not participate — HTTP-13 folds names', () => { + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['x-a', '1']])), + ).toBe(true); + }); + + test('value casing DOES participate — only names are folded', () => { + expect( + buildHeaders([['X-A', 'v']]).equals(buildHeaders([['X-A', 'V']])), + ).toBe(false); + }); + + test('the order of distinct NAMES does not matter', () => { + const ab = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + const ba = buildHeaders([ + ['X-B', '2'], + ['X-A', '1'], + ]); + expect(ab.equals(ba)).toBe(true); + expect(ba.equals(ab)).toBe(true); + }); +}); + +describe('Headers.equals directly (HTTP-13): values, order and subsets', () => { + test('the order of VALUES under one name does matter (HTTP-14)', () => { + const ab = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'b'], + ]); + const ba = buildHeaders([ + ['X-T', 'b'], + ['X-T', 'a'], + ]); + expect(ab.equals(ba)).toBe(false); + }); + + test('a strict subset is not equal, in either direction', () => { + const one = buildHeaders([['X-A', '1']]); + const two = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('same name count, disjoint names, is not equal', () => { + // The length pre-check passes here, so this is the case that proves the per-name lookup runs. + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['X-B', '1']])), + ).toBe(false); + }); + + test('same names with different value COUNTS is not equal', () => { + const one = buildHeaders([['X-T', 'a']]); + const two = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'a'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('two empty instances are equal', () => { + expect( + Headers.newBuilder().build().equals(Headers.newBuilder().build()), + ).toBe(true); + }); +}); diff --git a/packages/core/src/http/headers.ts b/packages/core/src/http/headers.ts index 7cf0350..f452d7e 100644 --- a/packages/core/src/http/headers.ts +++ b/packages/core/src/http/headers.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/headers.ts -import type {Builder} from './builder.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; import {HeaderValidationError} from './errors.js'; import { hasForbiddenNameByte, @@ -130,10 +130,14 @@ export class Headers { * Returns every value stored under `name`, in insertion order. * * @param name - the header name, as a string or a {@link HeaderName}. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string | HeaderName): readonly string[] { - return this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? []; + return ( + this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? + EMPTY_VALUE_LIST + ); } /** diff --git a/packages/core/src/http/query-params.test.ts b/packages/core/src/http/query-params.test.ts index 2865794..fb51ebf 100644 --- a/packages/core/src/http/query-params.test.ts +++ b/packages/core/src/http/query-params.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.test.ts // Exercises: HTTP-28 (case-sensitive, multi-value, value-less param), HTTP-29/32 (RFC 3986 encoding), -// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse) +// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse), +// HTTP-5 (getAll returns a frozen list on every path, present name or absent) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -9,6 +10,22 @@ import { decodeQueryComponent, encodeQueryComponent, } from './query-params.js'; +import {UrlConstructionError} from './errors.js'; + +/** + * Strings that mix ordinary text with UNPAIRED surrogate code units. `fc.string()` alone never + * produces one — its default unit is printable ASCII — so the URIError path it is here to cover + * would go ungenerated. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); describe('case-sensitive names and multi-value (HTTP-28)', () => { test('page and Page are distinct names', () => { @@ -149,3 +166,96 @@ test('the component decoder treats a literal + as data, not a space (HTTP-29, PA test('the decoder falls back to raw text on malformed percent-encoding (HTTP-31)', () => { expect(decodeQueryComponent('a%zzb')).toBe('a%zzb'); }); + +describe('lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string carrying an unpaired surrogate. Before audit #67 / #76 that escaped the + // `DexpaceError` tree entirely, and it escaped from `encode()` and `equals()` — accessors whose + // TSDoc documents no throw at all — rather than from the `add()` that accepted the value. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate value', 'c', LONE_HIGH], + ['a lone low surrogate value', 'c', LONE_LOW], + ['a lone surrogate name', LONE_HIGH, 'v'], + ['a lone surrogate inside a longer value', 'c', `ok${LONE_HIGH}ok`], + ])('add() rejects %s with UrlConstructionError', (_label, name, value) => { + expect(() => QueryParams.newBuilder().add(name, value)).toThrow( + UrlConstructionError, + ); + }); + + test('a well-formed surrogate PAIR is accepted and encoded as UTF-8', () => { + // U+1F600, one code point spelled with two code units. Rejecting this too would make the check + // "no astral characters", which HTTP-29 does not say. + expect( + QueryParams.newBuilder().add('emoji', '\u{1F600}').build().encode(), + ).toBe('emoji=%F0%9F%98%80'); + }); + + test('parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing', () => { + // HTTP-31 is MUST-level about `parse` never throwing, so the strict `add()` path cannot be the + // one `parse` uses for a value it did not choose. Replacement matches what the platform's own + // query serializer does with the same input: `new URL('https://x/?a=\uD800').search` is + // `?a=%EF%BF%BD` (measured 2026-09-05). This is `Headers`' outbound/inbound split, applied to + // the query model. + const parsed = QueryParams.parse(`a=${LONE_HIGH}`); + expect(parsed.get('a')).toBe('�'); + expect(parsed.encode()).toBe('a=%EF%BF%BD'); + }); + + test('parse() sanitizes the NAME as well as the value', () => { + const parsed = QueryParams.parse(`${LONE_HIGH}=v`); + expect(parsed.has('�')).toBe(true); + expect(parsed.encode()).toBe('%EF%BF%BD=v'); + }); + + test('no URIError escapes encode() or equals(), whatever the builder admitted (property)', () => { + fc.assert( + fc.property(surrogateBearingString, surrogateBearingString, (n, v) => { + let params: QueryParams; + try { + params = QueryParams.newBuilder().add(n, v).build(); + } catch (e: unknown) { + expect(e).toBeInstanceOf(UrlConstructionError); + return; + } + expect(() => params.encode()).not.toThrow(); + expect(() => + params.equals(QueryParams.newBuilder().build()), + ).not.toThrow(); + }), + {numRuns: 500}, + ); + }); + + test('no anything escapes parse(), whatever it is handed (property, HTTP-31)', () => { + fc.assert( + fc.property(surrogateBearingString, raw => { + const parsed = QueryParams.parse(raw); + expect(() => parsed.encode()).not.toThrow(); + }), + {numRuns: 500}, + ); + }); +}); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const params = QueryParams.newBuilder().add('x', '1').add('x', '2').build(); + + test('the present-name list is frozen', () => { + const values = params.getAll('x'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('3')).toThrow(TypeError); + expect(params.getAll('x')).toEqual(['1', '2']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + const first = params.getAll('nope'); + const second = params.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); diff --git a/packages/core/src/http/query-params.ts b/packages/core/src/http/query-params.ts index a990680..0166b9b 100644 --- a/packages/core/src/http/query-params.ts +++ b/packages/core/src/http/query-params.ts @@ -1,7 +1,12 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.ts -import type {Builder} from './builder.js'; -import {encodeRfc3986Component} from './rfc3986.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; +import {UrlConstructionError} from './errors.js'; +import { + encodeRfc3986Component, + hasLoneSurrogate, + toWellFormed, +} from './rfc3986.js'; /** * @internal @@ -28,6 +33,22 @@ export function decodeQueryComponent(value: string): string { } } +/** + * The strict half of the surrogate rule, applied by `add()`. + * + * `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + * on an unpaired surrogate — outside the `DexpaceError` tree, and out of `encode()` or `equals()` + * rather than out of the call that supplied the value. Rejecting here puts the failure at the call + * site, which is the same place HTTP-35 and HTTP-17/18 put theirs (audit #67 / #76). + */ +function requireWellFormed(kind: 'name' | 'value', text: string): void { + if (hasLoneSurrogate(text)) { + throw new UrlConstructionError( + `query parameter ${kind} contains an unpaired surrogate and cannot be percent-encoded`, + ); + } +} + let createQueryParams: ( valuesByName: ReadonlyMap, insertionOrder: readonly string[], @@ -43,7 +64,9 @@ let createQueryParams: ( * Encoding and parsing are deliberately asymmetric and kept as separate operations. * {@link QueryParams.encode} is strict RFC 3986 percent-encoding — not * `application/x-www-form-urlencoded`, so a space is `%20` and never `+` (HTTP-29/32). - * {@link QueryParams.parse} is lenient and never throws (HTTP-31). + * {@link QueryParams.parse} is lenient and never throws (HTTP-31). The asymmetry extends to + * unpaired surrogates: {@link QueryParamsBuilder.add} rejects one, while + * {@link QueryParams.parse} substitutes U+FFFD for it. * * @example * ```ts @@ -99,8 +122,10 @@ export class QueryParams { * (HTTP-31). * * A `null`, `undefined`, or blank input yields empty parameters; a leading `?` is tolerated; a - * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; and - * malformed percent-encoding falls back to the raw text rather than failing. + * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; + * malformed percent-encoding falls back to the raw text rather than failing; and an unpaired + * surrogate is replaced with U+FFFD rather than rejected the way + * {@link QueryParamsBuilder.add} rejects one, so the result is always encodable. * * @param raw - the query string, with or without its leading `?`. * @returns the parsed, frozen parameters. @@ -116,9 +141,12 @@ export class QueryParams { const eqIndex = segment.indexOf('='); const rawName = eqIndex === -1 ? segment : segment.slice(0, eqIndex); const rawValue = eqIndex === -1 ? '' : segment.slice(eqIndex + 1); + // HTTP-31 is MUST-level that parsing never throws, so the strict `add()` path above cannot be + // the one `parse` uses on text it did not choose — exactly the split `Headers` draws between + // its outbound (`add`) and inbound (`addInbound`) methods for HTTP-18 against HTTP-19. builder.add( - decodeQueryComponent(rawName), - decodeQueryComponent(rawValue), + toWellFormed(decodeQueryComponent(rawName)), + toWellFormed(decodeQueryComponent(rawValue)), ); } return builder.build(); @@ -138,10 +166,11 @@ export class QueryParams { * Returns every value stored under `name`, in insertion order. * * @param name - the parameter name. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string): readonly string[] { - return this.#valuesByName.get(name) ?? []; + return this.#valuesByName.get(name) ?? EMPTY_VALUE_LIST; } /** @@ -201,9 +230,15 @@ export class QueryParamsBuilder implements Builder { * @param value - the value; `null` records a value-less parameter as a single empty string * (HTTP-28). * @returns this builder, for chaining. + * @throws {@link UrlConstructionError} when the name or the value carries an unpaired surrogate. + * Such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined for it and + * {@link QueryParams.encode} could only fail — it is rejected here, at the call that supplied it. + * A well-formed surrogate pair is ordinary text and is accepted (HTTP-29). */ add(name: string, value: string | null): this { const actualValue = value ?? ''; + requireWellFormed('name', name); + requireWellFormed('value', actualValue); if (!this.#valuesByName.has(name)) { this.#insertionOrder.push(name); this.#valuesByName.set(name, []); diff --git a/packages/core/src/http/request-conditions.test.ts b/packages/core/src/http/request-conditions.test.ts index 4382f64..b89c3bb 100644 --- a/packages/core/src/http/request-conditions.test.ts +++ b/packages/core/src/http/request-conditions.test.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request-conditions.test.ts -// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag exclusivity) +// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag +// exclusivity, and an invalid Date rejected at the setter rather than emitted as `Invalid Date`) import {describe, expect, test} from 'bun:test'; import { RequestConditions, @@ -134,3 +135,53 @@ describe('ifUnmodifiedSince (HTTP-50)', () => { ]); }); }); + +describe('an invalid Date is rejected at the setter (HTTP-50)', () => { + // `toRfc1123` is `date.toUTCString()`, which renders the literal string `Invalid Date` for a NaN + // time value. `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar + // waves it through and it reaches the wire as `If-Modified-Since: Invalid Date` — a header no + // server can evaluate, produced from a caller mistake made several frames earlier. HTTP-50's + // "emit RFC 1123 dates" is not satisfiable from a NaN instant, so the setter is where it fails. + // Measured on the pre-fix tree, audit #67 / #76. + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifModifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifUnmodifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test('the message names the setter, so the caller knows which field to fix', () => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(new Date('nope')), + ).toThrow(/If-Modified-Since/); + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(new Date('nope')), + ).toThrow(/If-Unmodified-Since/); + }); + + test('no invalid instant can reach applyTo, so no header renders "Invalid Date"', () => { + const builder = RequestConditions.newBuilder(); + expect(() => builder.ifModifiedSince(new Date('nope'))).toThrow( + RequestConditionsValidationError, + ); + const headers = builder.build().applyTo(Headers.newBuilder().build()); + expect(headers.has('If-Modified-Since')).toBe(false); + }); +}); diff --git a/packages/core/src/http/request-conditions.ts b/packages/core/src/http/request-conditions.ts index 2af7ec9..db1d564 100644 --- a/packages/core/src/http/request-conditions.ts +++ b/packages/core/src/http/request-conditions.ts @@ -30,6 +30,25 @@ function toRfc1123(date: Date): string { return date.toUTCString(); } +/** + * Copies `date` after rejecting a NaN time value. + * + * `toUTCString()` is total — it renders the literal string `Invalid Date` rather than throwing — + * and `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar accepts it + * and it reaches the wire as `If-Modified-Since: Invalid Date`. HTTP-50 requires an RFC 1123 date, + * which a NaN instant cannot produce, so the failure belongs at the setter that was handed the bad + * `Date` rather than several frames downstream (audit #67 / #76). + */ +function copyValidInstant(date: Date, headerName: string): Date { + const time = date.getTime(); + if (Number.isNaN(time)) { + throw new RequestConditionsValidationError( + `${headerName}: date must be a valid instant, got an invalid Date`, + ); + } + return new Date(time); +} + // eslint-disable-next-line max-params -- private, builder-internal plumbing; field count fixed by HTTP-50 let createRequestConditions: ( ifMatch: readonly ETag[], @@ -45,7 +64,9 @@ let createRequestConditions: ( * Multiple entity-tags emit as one comma-separated header; dates emit in RFC 1123 form. * {@link RequestConditions.applyTo} uses `set`, never `add`, so applying the same conditions twice * cannot duplicate a header. The any-tag (`*`) is mutually exclusive with concrete entity-tags, and - * repeated `*` collapses to one — enforced when the tag is added, not at emission. + * repeated `*` collapses to one — enforced when the tag is added, not at emission. An invalid + * `Date` is likewise rejected by the setter that was handed it, so no instance can emit the literal + * header value `Invalid Date`. * * @example * ```ts @@ -118,8 +139,12 @@ export class RequestConditions { * untouched. * * Emission goes through the strict outbound header path, which rejects obs-text. An ETag whose - * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here; reconciling that - * against HTTP-18 is left to a later phase rather than guessed at now. + * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here, so a server-issued + * one does not round-trip. That is decided, not open: HTTP-18 is MUST-level and reinforced by + * XCUT-18's splitting defense, HTTP-48's permission is SHOULD-level, and no relaxed emit path is + * added — see item 15 of + * `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` and its + * as-built audit in `docs/deviations.md`. * * @param headers - the headers to derive from; not modified. * @returns a new {@link Headers} carrying the preconditions. @@ -200,9 +225,12 @@ export class RequestConditionsBuilder implements Builder { * @param date - the instant; copied, not aliased, so a caller mutating its own `Date` after * `build()` cannot change what {@link RequestConditions.applyTo} emits. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value. An + * invalid `Date` renders as the literal `Invalid Date`, which the outbound header grammar accepts + * and no server can evaluate, so it is rejected here rather than emitted (HTTP-50). */ ifModifiedSince(date: Date): this { - this.#ifModifiedSince = new Date(date.getTime()); + this.#ifModifiedSince = copyValidInstant(date, 'If-Modified-Since'); return this; } @@ -212,9 +240,11 @@ export class RequestConditionsBuilder implements Builder { * @param date - the instant; copied, not aliased, exactly as in * {@link RequestConditionsBuilder.ifModifiedSince}. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value, for the + * reason given on {@link RequestConditionsBuilder.ifModifiedSince} (HTTP-50). */ ifUnmodifiedSince(date: Date): this { - this.#ifUnmodifiedSince = new Date(date.getTime()); + this.#ifUnmodifiedSince = copyValidInstant(date, 'If-Unmodified-Since'); return this; } diff --git a/packages/core/src/http/request-options.test.ts b/packages/core/src/http/request-options.test.ts index 7bec354..bcfc28f 100644 --- a/packages/core/src/http/request-options.test.ts +++ b/packages/core/src/http/request-options.test.ts @@ -3,6 +3,7 @@ // Exercises: HTTP-34 (EMPTY sentinel, defensive tag copy), HTTP-35 (timeout/maxRetries validation), // AUTH-4 (the per-call auth descriptor tier, added in Phase 5c) import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; import {createAuthDescriptor} from '../auth/descriptor.js'; import {createAuthRequirement} from '../auth/requirement.js'; import {RequestOptions} from './request-options.js'; @@ -74,6 +75,31 @@ describe('operation auth descriptor (AUTH-4, docs/work/mvp/2026-09-04-open-items }); }); +const timeoutCandidate = fc.oneof( + fc.double({noNaN: false}), + fc.integer({min: -10, max: 10}), + fc.constantFrom(2 ** 32 - 1, 2 ** 32, Number.MAX_SAFE_INTEGER), +); + +/** + * Either the builder refuses `candidate` with the typed error, or it admits a value inside + * `AbortSignal.timeout()`'s range. There is no third outcome — that is the whole HTTP-35 claim. + */ +function expectAdmittedTimeoutInRange(candidate: number): void { + let accepted: number | undefined; + try { + accepted = RequestOptions.newBuilder() + .timeoutMs(candidate) + .build().timeoutMs; + } catch (e: unknown) { + expect(e).toBeInstanceOf(RequestOptionsValidationError); + return; + } + expect(Number.isInteger(accepted)).toBe(true); + expect(accepted).toBeGreaterThanOrEqual(1); + expect(accepted).toBeLessThanOrEqual(2 ** 32 - 1); +} + describe('timeout validation (HTTP-35)', () => { test('rejects zero or negative timeout', () => { expect(() => RequestOptions.newBuilder().timeoutMs(0)).toThrow( @@ -93,9 +119,47 @@ describe('timeout validation (HTTP-35)', () => { ); }); - test('accepts a fractional millisecond timeout, which a deadline can honor', () => { - expect(RequestOptions.newBuilder().timeoutMs(1.5).build().timeoutMs).toBe( - 1.5, + // Flipped by audit #67 / #76. The old case pinned `timeoutMs(1.5)` as accepted "because a deadline + // can honor a fractional millisecond". Nothing downstream can: the only consumer is + // `composeSignal`, which hands the value to `AbortSignal.timeout()`, and that throws + // `RangeError: The value of "delay" is out of range. It must be an integer.` — inside the + // transport, one seam away from the setter that accepted it. HTTP-35 puts the range check at the + // setter, so the range checked is `AbortSignal.timeout()`'s, the only one a transport can honor. + test('rejects a fractional timeout, which no transport deadline can honor', () => { + expect(() => RequestOptions.newBuilder().timeoutMs(1.5)).toThrow( + RequestOptionsValidationError, + ); + }); + + test("rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1", () => { + expect(() => RequestOptions.newBuilder().timeoutMs(2 ** 32)).toThrow( + RequestOptionsValidationError, + ); + expect(() => + RequestOptions.newBuilder().timeoutMs(Number.MAX_SAFE_INTEGER), + ).toThrow(RequestOptionsValidationError); + }); + + test('accepts the ceiling itself, so the boundary is inclusive', () => { + expect( + RequestOptions.newBuilder() + .timeoutMs(2 ** 32 - 1) + .build().timeoutMs, + ).toBe(2 ** 32 - 1); + }); + + // The invariant, stated runtime-independently: a value this setter accepts is inside + // `AbortSignal.timeout()`'s documented range, and anything else fails here. It is asserted as a + // property rather than against `AbortSignal.timeout()` itself because the two runtimes disagree — + // Bun accepts `1.5` and `2**32` where Node raises `RangeError` — which is precisely why the range + // is checked in the model instead of left to whichever runtime the caller happens to be on. + // `tests/node-conformance/seams.test.mjs` closes the Node half. + test('every accepted timeout is an integer in 1..2**32 - 1 (property)', () => { + fc.assert( + fc.property(timeoutCandidate, candidate => { + expectAdmittedTimeoutInRange(candidate); + }), + {numRuns: 500}, ); }); diff --git a/packages/core/src/http/request-options.ts b/packages/core/src/http/request-options.ts index 77e7e7c..b0258dd 100644 --- a/packages/core/src/http/request-options.ts +++ b/packages/core/src/http/request-options.ts @@ -4,6 +4,13 @@ import type {AuthDescriptor} from '../auth/descriptor.js'; import type {Builder} from './builder.js'; import {RequestOptionsValidationError} from './errors.js'; +/** + * The largest timeout `AbortSignal.timeout()` accepts, and therefore the largest one this model + * will hold: the platform rejects anything above it with `RangeError: The value of "delay" is out + * of range. It must be >= 0 && <= 4294967295.` (HTTP-35, audit #67 / #76). + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + // eslint-disable-next-line max-params -- private, builder-internal plumbing; one parameter per HTTP-34 field let createRequestOptions: ( timeoutMs: number | undefined, @@ -92,7 +99,11 @@ export class RequestOptions { .operationAuth(this.#operationAuth); } - /** The per-call timeout in milliseconds, or `undefined` to use the configured default. */ + /** + * The per-call timeout in milliseconds, or `undefined` to use the configured default. Always an + * integer in `1 .. 2**32 - 1` when defined — {@link RequestOptionsBuilder.timeoutMs} admits + * nothing else, so a transport can pass it to `AbortSignal.timeout()` unchecked. + */ get timeoutMs(): number | undefined { return this.#timeoutMs; } @@ -172,20 +183,31 @@ export class RequestOptionsBuilder implements Builder { * The range check is the FULL range, not merely its lower bound. `Infinity` and `NaN` are as out * of range as `-1`: a non-finite deadline is one no clock can compare against, so it degrades to * "no deadline" silently rather than failing at the call site that supplied it, which is exactly - * what HTTP-35 exists to prevent. Not required to be integral, unlike `maxRetries` -- a timeout is - * a duration and a fractional millisecond is meaningful. + * what HTTP-35 exists to prevent. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range a transport can honor. `composeSignal` is the one consumer of this value and it + * hands it straight to `AbortSignal.timeout()`, which throws a `RangeError` on a fractional + * millisecond and on anything above `4294967295`. A timeout this setter accepted and a transport + * then refused is the failure HTTP-35 exists to move to the call site, so integrality is checked + * here and not treated as an implementation detail of one transport (audit #67 / #76 flipped the + * earlier reading, which accepted `1.5` on the argument that a duration may be fractional; no + * consumer of this field can express one). * * @param value - the timeout in milliseconds, or `undefined` for no override. Zero is rejected * rather than reinterpreted: it means "no timeout" in one transport and is an error in another * (HTTP-35). * @returns this builder, for chaining. - * @throws {@link RequestOptionsValidationError} when a defined value is zero, negative, or not - * finite. + * @throws {@link RequestOptionsValidationError} when a defined value is zero, negative, not + * finite, not an integer, or greater than `2**32 - 1`. */ timeoutMs(value: number | undefined): this { - if (value !== undefined && !(Number.isFinite(value) && value > 0)) { + if ( + value !== undefined && + !(Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { throw new RequestOptionsValidationError( - `timeout must be a finite positive duration, got ${String(value)}`, + `timeout must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, got ${String(value)}`, ); } this.#timeoutMs = value; diff --git a/packages/core/src/http/rfc3986.ts b/packages/core/src/http/rfc3986.ts index a2d6db5..c96955a 100644 --- a/packages/core/src/http/rfc3986.ts +++ b/packages/core/src/http/rfc3986.ts @@ -1,13 +1,66 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/rfc3986.ts +/** + * Matches an UNPAIRED surrogate code unit, and only an unpaired one. + * + * In a `u`-mode pattern the engine works in code points, so a well-formed surrogate pair is one + * non-surrogate code point and does not match, while a lone high or low unit stays a surrogate code + * point and does. Equivalent to `!String.prototype.isWellFormed()`, which is ES2024 and so outside + * this repo's declared `lib` (`tsconfig.base.json` pins `ES2023`) even though the + * `engines.node >= 20.3` runtime has it — raising `lib` for one predicate is a wider change than + * the predicate is worth. + * + * Two patterns, one rule: `.test()` must not carry `lastIndex` between calls, and `.replace()` must + * be global. Neither is exported; the two functions below are, so no caller can pick the wrong one. + */ +const LONE_SURROGATE = /\p{Surrogate}/u; +const LONE_SURROGATE_GLOBAL = /\p{Surrogate}/gu; + +/** Unicode's replacement character, what a lenient repair substitutes for an unpaired surrogate. */ +const REPLACEMENT_CHARACTER = '\uFFFD'; + +/** + * Whether `value` carries an unpaired surrogate, and so has no UTF-8 form and cannot be + * percent-encoded. The strict half of the rule: a call site that was HANDED such a string rejects + * it (audit #67 / #76). + * + * @param value - the string to inspect. + * @returns `true` when at least one surrogate code unit is unpaired. + */ +export function hasLoneSurrogate(value: string): boolean { + return LONE_SURROGATE.test(value); +} + +/** + * `value` with every unpaired surrogate replaced by U+FFFD. The lenient half, for a call site that + * MUST NOT throw — `QueryParams.parse` under HTTP-31. Matches what the platform's own query + * serializer does with the same input: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` + * (measured 2026-09-05). + * + * @param value - the string to repair. + * @returns `value` with unpaired surrogates replaced; the same string when there are none. + */ +export function toWellFormed(value: string): string { + return value.replace(LONE_SURROGATE_GLOBAL, REPLACEMENT_CHARACTER); +} + /** * Percent-encodes a single URL component per RFC 3986, patching `encodeURIComponent`'s divergence: * `encodeURIComponent` leaves `! * ' ( )` unescaped, but none of them are in RFC 3986's unreserved * set (HTTP-29). * - * @param value - the raw component value. + * Deliberately NOT total, and deliberately not guarded here. `encodeURIComponent` throws + * `URIError: URI malformed` on a string carrying an unpaired surrogate, because such a string has + * no UTF-8 form. Every caller in this package rejects or repairs that input BEFORE reaching here — + * `QueryParamsBuilder.add`, `QueryParams.parse` and `substitutePathParams` each do, and each throws + * the error class its own call site already throws — so a second, silent guard inside the encoder + * would only move the failure back off the call site (audit #67 / #76). + * + * @param value - the raw component value; must not carry an unpaired surrogate. * @returns the percent-encoded component. + * @throws A platform `URIError` when `value` carries an unpaired surrogate. Callers validate first; + * see the note above. */ export function encodeRfc3986Component(value: string): string { return encodeURIComponent(value).replace( diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index fc7ae83..49eadf2 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -113,6 +113,11 @@ export { ReservedStageError, } from './pipeline/errors.js'; export {retryStep} from './retry/retry-step.js'; +// RETRY-34's read side. The retry pillar surfaces the FINAL attempt's own error, so `instanceof` +// against it does not depend on how many attempts ran; this is how the earlier ones are reached. +// Exported alongside `retryStep` because the two are one contract: nothing else in the barrel can +// tell a caller that the error they caught is the third of three. +export {retryAttempts} from './retry/attempt-trail.js'; // The trail entry for a response the engine discarded whose status is outside 400-599 — reachable // only through a caller-widened `retryableStatuses`. export {RetryDiscardedResponseError} from './retry/errors.js'; @@ -152,6 +157,10 @@ export type { RequestContext, } from './context/context.js'; export type {InstrumentationBundle} from './context/instrumentation.js'; +// `PipelineOptions` is what a caller passes to `new PipelineBuilder(transport, options)` and, by +// extension, to `standardResilience`; it is the only public route to `OBS-29`'s per-operation span +// and `CTX-16`'s operation name, so it is exported beside the bundle it carries. +export type {PipelineOptions} from './pipeline/builder.js'; export type {BackoffSettings} from './retry/backoff.js'; export type {RetrySettings} from './retry/settings.js'; export type {RetryStepOptions} from './retry/retry-step.js'; @@ -165,20 +174,20 @@ export type { ApiKeyCredentialConfig, AuthCredentialSet, AuthStepSettings, - BasicCredential, BearerCredential, ChallengeHook, - DigestCredential, } from './auth/auth-step.js'; export type {AuthTiers} from './auth/resolve.js'; export type {AuthScheme} from './auth/scheme.js'; export type {DigestAlgorithm} from './auth/digest.js'; // Factories, not bare interfaces: AUTH-3 validates and freezes inside `createAuthDescriptor`, and -// `ApiKeyCredential`/`NameKeyCredential`/`BearerToken` are NOMINAL -- they carry a `#` field, so no -// caller-side object literal is assignable and the AUTH-9 validation in each factory cannot be routed -// around. Without these, API_KEY and OAUTH2 auth are unreachable from outside the package. -// `BearerToken` is a VALUE export, not a type-only one: it is a class, and `TokenProvider` returns it. +// every credential type is NOMINAL -- each carries a `#` field, so no caller-side object literal is +// assignable and the AUTH-9 validation in each factory cannot be routed around. Without these, +// API_KEY, OAUTH2, BASIC and DIGEST auth are unreachable from outside the package. +// All five are VALUE exports, not type-only ones: they are classes, `TokenProvider` returns a +// `BearerToken`, and `BasicCredential`/`DigestCredential` became classes on 2026-09-04 so AUTH-8's +// redaction covers their passwords too (audit #67 / #71). export type {AuthDescriptor} from './auth/descriptor.js'; export {createAuthDescriptor} from './auth/descriptor.js'; export type {AuthRequirement} from './auth/requirement.js'; @@ -189,7 +198,9 @@ export { export type {TokenProvider} from './auth/credential.js'; export { ApiKeyCredential, + BasicCredential, BearerToken, + DigestCredential, NameKeyCredential, bearerTokensEqual, createBearerToken, diff --git a/packages/core/src/io/index.ts b/packages/core/src/io/index.ts index c943288..1358a5a 100644 --- a/packages/core/src/io/index.ts +++ b/packages/core/src/io/index.ts @@ -7,13 +7,26 @@ // api-extractor surface so a later phase can promote them deliberately (styleguide 10.3), or not at // all: 3b shaped BODY-1's write-to-sink around the platform's WritableStream rather than BufferedSink. // -// The ERROR leaves are a different case, and this comment claimed otherwise until 2026-09-02. Phase -// 8a promoted `IoError` and `TransportFailureError` to the public barrel (`index.ts:34`) because -// TRANSPORT-20 makes the subtyping a requirement and `retry/classify.ts`'s cause-walk is -// load-bearing on it, and Phase 9's U9 pass promoted `EndOfStreamError` — it was the subject of four -// `@throws` tags on public symbols with no class a caller could catch. `isIoError`, -// `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` remain internal -// (docs/work/mvp/2026-09-04-open-items-dissolution.md H8). +// The ERROR leaves are a different case, and this comment has twice claimed otherwise. Phase 8a +// promoted `IoError` and `TransportFailureError` to the public barrel because TRANSPORT-20 makes the +// subtyping a requirement and `retry/classify.ts`'s cause-walk is load-bearing on it, and Phase 9's +// U9 pass promoted `EndOfStreamError` — it was the subject of four `@throws` tags on public symbols +// with no class a caller could catch. `1f48926` finished the set: `isIoError`, +// `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` are exported as +// well, so every error symbol re-exported below is also on `packages/core/src/index.ts:39-48` and in +// `packages/core/etc/core.api.md`. Nothing in this file's error block is internal any more +// (docs/work/mvp/2026-09-04-open-items-dissolution.md H8, whose remaining sub-item was the category +// catch `isIoError` now provides). +// +// "Load-bearing on it" is narrower than it sounds, and the difference is a decision rather than an +// accident. The cause-walk at `../retry/classify.ts:90` tests `instanceof IoError`, so it matches +// `IoError` and `TransportFailureError` — and NOT the four leaves below, which extend `DexpaceError` +// directly and are grouped only by `isIoError`. That branch means "the wire failed": a send that +// produced no response is retryable (RETRY-4, TRANSPORT-20), while a violated source contract, a +// closed resource, an allocation cap and a short exact-length copy are this package's own failures +// and repeat identically on the next attempt. Audit #67 / #78 decided it; `docs/deviations.md` +// item 17 carries the rationale and `../retry/classify.test.ts` pins one answer per class. Do not +// re-parent a leaf under `IoError` to tidy the tree — that silently makes it retryable. export {BufferedSink} from './buffered-sink.js'; export {BufferedSource} from './buffered-source.js'; export {ByteQueue, copyBytes} from './byte-queue.js'; diff --git a/packages/core/src/io/tee-sink.test.ts b/packages/core/src/io/tee-sink.test.ts index 32560d8..0ac8c76 100644 --- a/packages/core/src/io/tee-sink.test.ts +++ b/packages/core/src/io/tee-sink.test.ts @@ -7,7 +7,8 @@ // IO-42 (write after close rejects with the source intact), // IO-13 (the tap mirrors the primary's exact encoded bytes, and refuses a label identically), // IO-16 (the tee's own writable bridge still feeds the tap), -// IO-3 (a negative count is an argument error, rejected before any transfer) +// IO-3 (a negative count is an argument error, rejected before any transfer; and a non-integral +// tapLimit is one too, rejected at the constructor rather than at the first write) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {BufferedSink} from './buffered-sink.js'; @@ -15,6 +16,7 @@ import {BufferedSource} from './buffered-source.js'; import {ByteQueue} from './byte-queue.js'; import {ClosedResourceError} from './errors.js'; import {TeeSink} from './tee-sink.js'; +import {InvariantViolation} from '../invariant.js'; import {writeAll} from './pump.js'; import { collectingWritableStream, @@ -300,4 +302,45 @@ describe('argument validation (IO-3)', () => { 'InvariantViolation', ); }); + + // The constructor checked `tapLimit >= 0` only, so a fractional cap was accepted and the fault + // surfaced at the FIRST write instead: `#mirror` computes `room = tapLimit - tap.size`, hands it + // to `ByteQueue.copyTo`, and `assertCount` rejects it as `count must be a non-negative integer, + // got 2.5` — a message about the wrong parameter, at the wrong call, on a path where the primary + // write has not run yet. A byte count is integral for the same reason `count` is (IO-3, IO-26). + // Measured on the pre-fix tree, audit #67 / #76. + test.each([2.5, 0.5, -0.5, Number.NaN, Number.NEGATIVE_INFINITY])( + 'a tapLimit of %p is rejected at the constructor', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).toThrow(InvariantViolation); + }, + ); + + test('the message names tapLimit, not count', () => { + const {stream} = collectingWritableStream(); + expect(() => new TeeSink(BufferedSink.overStream(stream), 2.5)).toThrow( + /tapLimit/, + ); + }); + + test.each([0, 1, 4096, Number.POSITIVE_INFINITY])( + 'a tapLimit of %p is still accepted', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).not.toThrow(); + }, + ); + + test('the unbounded default stays Infinity, which is not an integer', () => { + // `Number.isInteger(Infinity)` is false, so the check has to admit it explicitly — it is the + // documented default and the cap `#mirror` reads as "no cap" (IO-26). + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect(tee.snapshot().length).toBe(0); + }); }); diff --git a/packages/core/src/io/tee-sink.ts b/packages/core/src/io/tee-sink.ts index dc6e560..83b1d15 100644 --- a/packages/core/src/io/tee-sink.ts +++ b/packages/core/src/io/tee-sink.ts @@ -31,9 +31,18 @@ export class TeeSink implements Sink { readonly #tapLimit: number; constructor(primary: Sink, tapLimit: number = Number.POSITIVE_INFINITY) { + // Integrality, not merely `>= 0`. A tap cap is a byte count, so IO-3's rule for `count` applies + // to it — and a fractional cap was not merely odd, it was deferred: `#mirror` computes + // `room = tapLimit - tap.size` and hands it to `ByteQueue.copyTo`, where `assertCount` rejects + // it as "count must be a non-negative integer, got 2.5". That names the wrong parameter, fires + // at the first write rather than at the construction that supplied it, and does so before the + // primary write on a path that has already taken bytes from the caller. `Infinity` is admitted + // explicitly: it is not an integer, and it is the documented unbounded default (IO-26). + // Audit #67 / #76. invariant( - tapLimit >= 0, - `tapLimit must be non-negative, got ${String(tapLimit)}`, + (Number.isInteger(tapLimit) && tapLimit >= 0) || + tapLimit === Number.POSITIVE_INFINITY, + `tapLimit must be a non-negative integer or Infinity, got ${String(tapLimit)}`, ); this.#primary = primary; this.#tapLimit = tapLimit; diff --git a/packages/core/src/observability/diagnostic-context.test.ts b/packages/core/src/observability/diagnostic-context.test.ts index 7ec1c79..ba80ff5 100644 --- a/packages/core/src/observability/diagnostic-context.test.ts +++ b/packages/core/src/observability/diagnostic-context.test.ts @@ -1,7 +1,9 @@ // SPDX-License-Identifier: MIT // packages/core/src/observability/diagnostic-context.test.ts // Exercises: OBS-10 (default allow-list {trace.id, span.id}, null allow-list folds all, null values skipped), -// OBS-24 (immutable snapshot bridge: capture, reinstall, restore including on throw). +// OBS-24 (immutable snapshot bridge: capture, reinstall, restore including on throw), OBS-22/OBS-23's +// scoping mechanics (AsyncScopedStore.run unwinds across an await and over an enter() its callback made, +// where the enter() handle alone does not). import {describe, expect, test} from 'bun:test'; import { captureDiagnosticSnapshot, @@ -156,6 +158,38 @@ describe('pushDiagnosticFields', () => { }); describe('createAsyncScopedStore', () => { + test('run installs the value for the callback and restores across an await', async () => { + const store = createAsyncScopedStore(); + expect(store.get()).toBeUndefined(); + + await store.run('scoped', async () => { + expect(store.get()).toBe('scoped'); + await Promise.resolve(); + expect(store.get()).toBe('scoped'); + }); + + expect(store.get()).toBeUndefined(); + }); + + test('run unwinds an enter() its callback left open, and restores on a throw', () => { + const store = createAsyncScopedStore(); + store.run('outer', () => { + // The handle form, deliberately never closed -- what the LOGGING pillar's correlation scope + // effectively does once its close() lands in a later continuation. + store.enter('inner'); + expect(store.get()).toBe('inner'); + }); + expect(store.get()).toBeUndefined(); + + expect(() => { + store.run('outer', () => { + store.enter('inner'); + throw new Error('boom'); + }); + }).toThrow('boom'); + expect(store.get()).toBeUndefined(); + }); + test('enter installs value and returned restore function resets prior value', () => { const store = createAsyncScopedStore(); expect(store.get()).toBeUndefined(); diff --git a/packages/core/src/observability/diagnostic-context.ts b/packages/core/src/observability/diagnostic-context.ts index 9cfa324..4c0ecf5 100644 --- a/packages/core/src/observability/diagnostic-context.ts +++ b/packages/core/src/observability/diagnostic-context.ts @@ -77,6 +77,15 @@ export function runWithSnapshot( * The scope-handle form of `withDiagnosticFields`, for callers that cannot express their scope as a single * callback -- OBS-23's span-correlation scope is one: a pipeline step pushes before `await next(...)` and * restores after, with the two halves in different statements. Returns the restore function. + * + * **The restore reaches only the continuation that called it.** `enterWith` installs the store on the + * *current* async resource and every resource created from it; the returned function does the same with + * the previous store. Call it after an `await` and it runs on a different resource, so the caller that + * pushed keeps the pushed fields for the rest of its own continuation -- which is a leak when that caller + * is a library entry point and the continuation is the application's. Use `withDiagnosticFields` whenever + * the scope CAN be written as one callback; that form is `AsyncLocalStorage.run`, which restores on exit + * by construction. `Runtime.send` used this handle until 2026-09-05 and leaked `trace.id`/`span.id` into + * every subsequent application log (audit #67 / #80). */ export function pushDiagnosticFields( fields: Readonly>, @@ -105,14 +114,37 @@ export function pushDiagnosticFields( */ export interface AsyncScopedStore { get(): T | undefined; - /** Installs `value` for the rest of this async context; the returned function restores the prior value. */ + /** + * Installs `value` for the rest of this async context; the returned function restores the prior value. + * + * Carries `pushDiagnosticFields`' caveat verbatim: the restore is an `enterWith` of its own, so it takes + * effect only on the async resource that runs it. A handle closed after an `await` leaves `value` + * installed on the resource that entered it. Prefer {@link AsyncScopedStore.run} for any scope that can + * be written as one callback. + */ enter(value: T): () => void; + /** + * Runs `fn` with `value` installed, restoring whatever was installed before when `fn` returns -- + * including on a throw, and including for anything `fn` itself entered with the handle form. This is + * `AsyncLocalStorage.run`, so the restore is structural rather than a call a later continuation has to + * remember to make, and it is what a library entry point must use if the caller's context is to survive + * the call. + * + * `fn`'s return value is passed through untouched: an `async` callback hands back its promise, and the + * store is restored when the callback's synchronous prefix returns, not when the promise settles. That + * is the intended scoping -- everything the promise chain does inherits the store from the resource it + * was created on. + */ + run(value: T, fn: () => R): R; } export function createAsyncScopedStore(): AsyncScopedStore { const scoped = new AsyncLocalStorage(); return { get: () => scoped.getStore(), + run(value: T, fn: () => R): R { + return scoped.run(value, fn); + }, enter(value: T): () => void { const previous = scoped.getStore(); scoped.enterWith(value); diff --git a/packages/core/src/observability/logging-step.test.ts b/packages/core/src/observability/logging-step.test.ts index 82894e6..ba7e12d 100644 --- a/packages/core/src/observability/logging-step.test.ts +++ b/packages/core/src/observability/logging-step.test.ts @@ -6,7 +6,11 @@ // OBS-34 (granularity gates log events, not span/metrics), OBS-35 (level resolves from // Configuration, tolerant/case-insensitive), OBS-39 (stable http.request/http.response event names/keys, // url.full always redacted), OBS-20 (a throwing Logger is caught and re-surfaced as http.instrumentation.*; -// a throwing tracer/meter propagates, NOT caught), OBS-36, OBS-37, OBS-38 (body previews). +// a throwing tracer/meter propagates, NOT caught), OBS-36, OBS-37, OBS-38 (body previews), OBS-20's +// body-drain half (a failed request-side probe or response-side drain emits +// http.instrumentation.bodyCaptureFailed and the request still completes), OBS-35's "MUST NOT bake in a +// default config key name" (the configKey setting), OBS-21 (the per-request span ends exactly once, +// even when end() itself throws). import {afterEach, describe, expect, test} from 'bun:test'; import {Headers} from '../http/headers.js'; import {Protocol} from '../http/protocol.js'; @@ -21,6 +25,7 @@ import { ConfigurationBuilder, setGlobalConfiguration, } from '../config/configuration.js'; +import type {Body} from '../body/body.js'; import {stringBody} from '../body/simple-bodies.js'; import type {Logger, LogEvent} from './logger.js'; import type {Meter} from './metrics.js'; @@ -616,3 +621,160 @@ describe('request body preview and validation (OBS-36, OBS-38)', () => { expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); }); }); + +/** A declared-length response whose body errors on the first read -- a drain that cannot finish. */ +function failingResponse(error: Error): Response { + const failingStream = new ReadableStream({ + start(controller) { + controller.error(error); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers( + Headers.newBuilder() + .set('content-type', 'text/plain') + .set('content-length', '100') + .build(), + ) + .body(failingStream) + .build(); +} + +describe('body-drain diagnostics (OBS-20)', () => { + test('a failing response drain emits http.instrumentation.bodyCaptureFailed with the cause', async () => { + const {logger, events} = spyLogger(); + const broke = new Error('stream broke'); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([failingResponse(broke)]), + ); + + expect(response.status.code).toBe(200); // OBS-20: the request still completes + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('response'); + expect(diagnostic?.cause).toBe(broke); + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); + }); + + test('a failing request-body probe emits the same diagnostic and still sends the request', async () => { + const {logger, events} = spyLogger(); + const gone = new Error('ENOENT: the file went away'); + // What `fileBody()` over a deleted file looks like to the step: replayable, so the tap probes + // it, and the probe is the thing that fails. + const brokenBody: Body = { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: 3, + replayable: true, + writeTo: () => Promise.reject(gone), + }; + const request = Request.newBuilder() + .url('https://example.com/upload') + .method('POST') + .body(brokenBody) + .build(); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([countingResponse(200).response]), + request, + ); + + expect(response.status.code).toBe(200); + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('request'); + expect(diagnostic?.cause).toBe(gone); + // The empty capture still ships, as it did before: the tap saw no bytes, and the preview field is + // what the tap holds. What changed is that the empty string is no longer the ONLY evidence -- the + // diagnostic above says which direction failed and why. + const requestEvent = events.find(e => e.event === 'http.request'); + expect(requestEvent?.['http.request.body.preview']).toBe(''); + }); +}); + +describe('the config key the ambient granularity is read from (OBS-35)', () => { + test('configKey names the key, so the baked-in default is not the only one', async () => { + setGlobalConfiguration( + new ConfigurationBuilder() + .put('ACME_SDK_LOG_LEVEL', 'headers') + .put(CFG_KEY_LOG_LEVEL, 'none') + .build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({logger, configKey: 'ACME_SDK_LOG_LEVEL'}), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events.map(e => e.event)).toEqual(['http.request', 'http.response']); + }); + + test('an explicit granularity still wins over the configured key (OBS-34)', async () => { + setGlobalConfiguration( + new ConfigurationBuilder().put('ACME_SDK_LOG_LEVEL', 'headers').build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({ + logger, + configKey: 'ACME_SDK_LOG_LEVEL', + granularity: 'none', + }), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events).toHaveLength(0); + }); +}); + +describe('the per-request span ends exactly once (OBS-21, OBS-29)', () => { + test('an end() that throws on the success path is not called a second time', async () => { + const {logger} = spyLogger(); + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span = { + isRecording: true, + setAttribute: () => span, + recordException: (error: unknown) => { + exceptions.push(error); + return span; + }, + end: (): void => { + ends += 1; + throw endFailed; + }, + }; + + let caught: unknown; + try { + await send( + loggingStep({ + logger, + granularity: 'headers', + tracerFactory: () => ({startSpan: () => span}), + }), + new FakeTransport([countingResponse(200).response]), + ); + } catch (error) { + caught = error; + } + + expect(caught).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([]); + }); +}); diff --git a/packages/core/src/observability/logging-step.ts b/packages/core/src/observability/logging-step.ts index 11f60d7..dfd5cd2 100644 --- a/packages/core/src/observability/logging-step.ts +++ b/packages/core/src/observability/logging-step.ts @@ -59,8 +59,24 @@ export interface LoggingStepSettings { * Default: 'info' (failures always emit at 'error'). */ readonly severity?: LogLevel | undefined; - /** Granularity of logging (default: resolved from Configuration via CFG_KEY_LOG_LEVEL, fallback 'none'). */ + /** Granularity of logging (default: resolved from Configuration via {@link LoggingStepSettings.configKey}, fallback 'none'). */ readonly granularity?: LoggingGranularity | undefined; + /** + * The configuration key the ambient granularity is read from when `granularity` is omitted (OBS-35). + * + * OBS-35 says the SDK MUST NOT bake in a default key name, and `CFG_KEY_LOG_LEVEL` + * (`DEXPACE_LOG_LEVEL`) is exactly that — CFG-14's well-known key, kept as the default because a + * required key would mean no caller gets ambient logging without naming one first. Set this to read + * a host application's own key instead; the resolution is the same layered, tolerant one either way. + * + * Nothing installs a configuration that reads the process environment by default: the global slot + * starts empty (CFG-13), so `setGlobalConfiguration(defaultConfiguration())` is the wiring that + * makes any environment variable — this one included — reachable. See + * `docs/sdk-documentation/pipelines.md`, "Turning logging on from the environment". + * + * @defaultValue {@link CFG_KEY_LOG_LEVEL} + */ + readonly configKey?: string | undefined; /** Byte limit for request/response body previews (default: 8192). */ readonly previewSizeBytes?: number | undefined; /** Optional custom tracer factory. */ @@ -82,7 +98,7 @@ function resolveGranularity(settings: LoggingStepSettings): LoggingGranularity { return 'none'; } const raw = getGlobalConfiguration() - .getString(CFG_KEY_LOG_LEVEL, 'none') + .getString(settings.configKey ?? CFG_KEY_LOG_LEVEL, 'none') ?.trim() .toLowerCase(); if (raw === 'headers' || raw === 'body') return raw; @@ -162,6 +178,32 @@ function safeEmit(logger: Logger, build: () => void): void { } } +/** + * OBS-20's body-drain clause. A capture failure is contained -- the request completes and the caller's + * body is untouched -- but containment is not silence: the failure re-surfaces as a best-effort + * `http.instrumentation.*` diagnostic, through the same {@link safeEmit} every other emission uses, so a + * secondary failure while reporting it is swallowed in turn. + * + * `verbose`, the level its sibling `http.instrumentation.logFailure` already emits at: nothing about the + * request changed, and what was lost is a diagnostic preview. Before 2026-09-05 both catches returned an + * empty capture and emitted nothing at all, so a `fileBody()` over a deleted file logged + * `"http.request.body.preview": ""` with no trace of why (audit #67 / #80). + */ +function emitBodyCaptureFailure( + logger: Logger, + direction: 'request' | 'response', + error: unknown, +): void { + safeEmit(logger, () => { + logger + .atLevel('verbose') + .event('http.instrumentation.bodyCaptureFailed') + .field('http.message.direction', direction) + .cause(error) + .emit(); + }); +} + /** * Stable identity symbol for the LOGGING pillar step. * @@ -273,12 +315,13 @@ function resolveTracer( /** Captures response body preview safely when content-length is declared (OBS-36, OBS-37). */ async function captureResponseBody( response: Response, - previewSizeBytes: number, + context: EmitContext, ): Promise<{ readonly response: Response; readonly preview: string | undefined; readonly size: number | undefined; }> { + const previewSizeBytes = context.previewSizeBytes; try { const hasContentLength = response.headers.has('content-length'); if (response.body === null || !hasContentLength) { @@ -306,33 +349,34 @@ async function captureResponseBody( ); const size = snap.length > 0 ? snap.length : undefined; return {response: captured, preview, size}; - } catch { - // OBS-20: body-drain failure must never fail the request + } catch (error) { + // OBS-20: a body-drain failure must never fail the request -- and must not vanish either. + emitBodyCaptureFailure(context.logger, 'response', error); return {response, preview: undefined, size: undefined}; } } async function prepareRequestBody( request: Request, - granularity: LoggingGranularity, - previewSizeBytes: number, + context: EmitContext, ): Promise<{ readonly outbound: Request; readonly preview: string | undefined; readonly size: number | undefined; }> { - if (granularity !== 'body' || request.body === undefined) { + if (context.granularity !== 'body' || request.body === undefined) { return {outbound: request, preview: undefined, size: undefined}; } - const logged = withRequestLogging(request.body, previewSizeBytes); + const logged = withRequestLogging(request.body, context.previewSizeBytes); if (logged.replayable) { const probeSink = new WritableStream({ write: () => undefined, }); try { await logged.writeTo(probeSink); - } catch { - // probe error is ignored per OBS-20 + } catch (error) { + // OBS-20: the probe is diagnostic-only, so its failure is contained -- and reported. + emitBodyCaptureFailure(context.logger, 'request', error); } } const snap = logged.snapshot(); @@ -368,7 +412,7 @@ interface PipelineExecutionArgs { async function executePipeline(args: PipelineExecutionArgs): Promise { const {ctx, plan, outbound, startedAt, span} = args; - const {emitContext, instruments, previewSizeBytes} = plan; + const {emitContext, instruments} = plan; try { const response = await ctx.next(outbound); const { @@ -376,7 +420,7 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { preview, size, } = emitContext.granularity === 'body' - ? await captureResponseBody(response, previewSizeBytes) + ? await captureResponseBody(response, emitContext) : {response, preview: undefined, size: undefined}; const elapsedMs = instruments.clock.monotonic() - startedAt; @@ -395,7 +439,6 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { size, }); - span.end(); return captured; } catch (caught) { const error = toError(caught); @@ -412,7 +455,6 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { emitFailureEvent(emitContext, {error, elapsedMs}); span.recordException(error); - span.end(); throw caught; } } @@ -422,7 +464,7 @@ async function handleRequestExecution( ctx: StepContext, plan: ExecutionPlan, ): Promise { - const {settings, emitContext, instruments, previewSizeBytes} = plan; + const {settings, emitContext, instruments} = plan; const tracer = resolveTracer(settings, ctx); const span = tracer.startSpan('http.client.request'); const scope = activateSpanForCorrelation(span); @@ -431,8 +473,7 @@ async function handleRequestExecution( try { const {outbound, preview, size} = await prepareRequestBody( request, - emitContext.granularity, - previewSizeBytes, + emitContext, ); emitRequestEvent(emitContext, outbound, {preview, size}); return await executePipeline({ @@ -443,7 +484,17 @@ async function handleRequestExecution( span, }); } finally { - scope.close(); + // ONE exit for `end()`, and it is here rather than on each path inside `executePipeline`: an + // `end()` that threw on the success path used to land in that function's own `catch`, which + // recorded the exception and called `end()` a second time on a span the tracer had already + // closed (OBS-21's idempotent-end clause is the tracer's promise, not this step's licence). + // Nested rather than sequential so a throwing `end()` -- which OBS-20 deliberately does not + // catch, because OBS-30 makes it the SPI's promise not to -- still cannot leak the scope. + try { + span.end(); + } finally { + scope.close(); + } } } diff --git a/packages/core/src/observability/redaction.test.ts b/packages/core/src/observability/redaction.test.ts index 6374a3c..8ce88ff 100644 --- a/packages/core/src/observability/redaction.test.ts +++ b/packages/core/src/observability/redaction.test.ts @@ -102,6 +102,15 @@ describe('redactUrl: delimiters and total safety (OBS-14..15)', () => { expect(redactUrl('not a url at all ###')).toBe('[malformed url]'); }); + test('the output is re-rendered from the parsed URL, so WHATWG normalisation shows (OBS-14)', () => { + // Pinned, not fixed: the result is assembled from `URL`'s components, so host case, a default + // port and an empty path normalise on the way through. Documented on `redactUrl` as inherent to + // parsing, and left as is by audit #67 / #80 -- re-rendering the caller's authority by hand would + // be a second URL renderer for no gain in what OBS-11..15 asks for. + expect(redactUrl('https://EXAMPLE.com:443')).toBe('https://example.com/'); + expect(redactUrl('http://Example.COM:80/p')).toBe('http://example.com/p'); + }); + test('property: never throws for any string', () => { fc.assert( fc.property(fc.string(), value => { diff --git a/packages/core/src/observability/redaction.ts b/packages/core/src/observability/redaction.ts index 5d8a137..b398840 100644 --- a/packages/core/src/observability/redaction.ts +++ b/packages/core/src/observability/redaction.ts @@ -52,6 +52,16 @@ function hasHashDelimiter(input: URL | string): boolean { /** * Redacts sensitive components from a URL according to spec rules (OBS-11..15). * + * **The result is a re-rendered URL, not the caller's string with holes in it.** Every input goes + * through WHATWG `URL`, and the output is assembled from its parsed components, so the normalisations + * parsing performs come with it: the host is lower-cased, a default port for the scheme + * (`https://h:443/`) is dropped, a missing path becomes `/`, and percent-encoding is canonicalised. + * A log line therefore need not match the request line byte for byte. That is inherent to parsing and + * is left as is deliberately (audit #67 / #80): re-rendering the original authority by hand would mean + * a second URL renderer in this package, maintained against WHATWG, for no gain in what OBS-11..15 + * actually asks for — that userinfo, non-allow-listed query values and fragment values do not reach a + * log. Compare identity elsewhere; this is for humans and log pipelines. + * * @param input - the URL or string to redact. * @param queryAllowList - set of allowed query parameter names (default: \{api-version\}). * @returns the redacted URL string, or '[malformed url]' if parsing fails. diff --git a/packages/core/src/observability/span.ts b/packages/core/src/observability/span.ts new file mode 100644 index 0000000..9b45f33 --- /dev/null +++ b/packages/core/src/observability/span.ts @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/span.ts + +// The inert half of the tracing surface: the span/tracer shapes (OBS-21, OBS-23) and the two no-op +// singletons (OBS-25) that need nothing else in the package to exist. +// +// This module exists to break an import cycle, not to introduce a layer. `tracing.ts` needs +// `InstrumentationBundle` from `context/instrumentation.ts` to type `createInstrumentationBundle`, and +// `context/instrumentation.ts` needs `NOOP_SPAN` for CTX-15's disabled-tracing default. Holding both in +// `tracing.ts` closes a cycle that `bun run verify:import-cycles` rejects -- type-only edges count there, +// deliberately -- and that gate's own message prescribes this fix: move the shared declaration into a +// module both sides can import. So this file imports nothing, and `tracing.ts` re-exports every name +// below, which is why no existing import path changed. +// +// Line comments, and deliberately none of them writes out the internal-marker JSDoc tag. Two traps stack +// at the top of a module: a leading `/** */` block binds to the first declaration below it rather than to +// the file, and gts turns on `stripInternal`, whose test is a plain substring scan of EVERY leading comment +// range of a declaration -- line comments included, doc block or not. So a module header that merely +// MENTIONS that tag deletes `SpanContext` from the emitted `.d.ts` while `tsc` stays silent; the failure +// surfaces one package later, as an unresolved name inside core's own `dist/`. Measured twice on the way +// to this wording. If this note is ever reworded, rebuild and check `dist/observability/span.d.ts` still +// declares all four names. + +/** + * Contextual metadata identifying a trace and span in distributed tracing (OBS-23, OBS-26). + * + * @public + */ +export interface SpanContext { + readonly traceId: string; + readonly spanId: string; + readonly traceFlags?: number | undefined; + readonly traceState?: string | undefined; +} + +/** + * A structural subset of `@opentelemetry/api`'s own `Span` shape (OBS-21, OBS-23). + * + * @public + */ +export interface Span { + readonly isRecording: boolean; + setAttribute(key: string, value: unknown): this; + recordException(error: unknown): this; + end(): void; + spanContext?(): SpanContext | undefined; +} + +/** + * Tracing facade interface for creating spans. + * + * @public + */ +export interface Tracer { + startSpan(name: string): Span; +} + +/** + * Inert no-op {@link Span} singleton (OBS-21, OBS-25). + * + * @public + */ +export const NOOP_SPAN: Span = Object.freeze({ + isRecording: false, + setAttribute(): Span { + return NOOP_SPAN; + }, + recordException(): Span { + return NOOP_SPAN; + }, + end(): void { + return; + }, + spanContext(): SpanContext { + return {traceId: '0'.repeat(32), spanId: '0'.repeat(16)}; + }, +}); + +/** + * Inert no-op {@link Tracer} singleton (OBS-25). + * + * @public + */ +export const NOOP_TRACER: Tracer = Object.freeze({ + startSpan(): Span { + return NOOP_SPAN; + }, +}); diff --git a/packages/core/src/observability/tracing.ts b/packages/core/src/observability/tracing.ts index 408198e..c98d692 100644 --- a/packages/core/src/observability/tracing.ts +++ b/packages/core/src/observability/tracing.ts @@ -5,73 +5,16 @@ import { createAsyncScopedStore, pushDiagnosticFields, } from './diagnostic-context.js'; +import {NOOP_SPAN, NOOP_TRACER} from './span.js'; +import type {Span, Tracer} from './span.js'; import type {InstrumentationBundle} from '../context/instrumentation.js'; -/** - * Contextual metadata identifying a trace and span in distributed tracing (OBS-23, OBS-26). - * - * @public - */ -export interface SpanContext { - readonly traceId: string; - readonly spanId: string; - readonly traceFlags?: number | undefined; - readonly traceState?: string | undefined; -} - -/** - * A structural subset of `@opentelemetry/api`'s own `Span` shape (OBS-21, OBS-23). - * - * @public - */ -export interface Span { - readonly isRecording: boolean; - setAttribute(key: string, value: unknown): this; - recordException(error: unknown): this; - end(): void; - spanContext?(): SpanContext | undefined; -} - -/** - * Tracing facade interface for creating spans. - * - * @public - */ -export interface Tracer { - startSpan(name: string): Span; -} - -/** - * Inert no-op {@link Span} singleton (OBS-21, OBS-25). - * - * @public - */ -export const NOOP_SPAN: Span = Object.freeze({ - isRecording: false, - setAttribute(): Span { - return NOOP_SPAN; - }, - recordException(): Span { - return NOOP_SPAN; - }, - end(): void { - return; - }, - spanContext(): SpanContext { - return {traceId: '0'.repeat(32), spanId: '0'.repeat(16)}; - }, -}); - -/** - * Inert no-op {@link Tracer} singleton (OBS-25). - * - * @public - */ -export const NOOP_TRACER: Tracer = Object.freeze({ - startSpan(): Span { - return NOOP_SPAN; - }, -}); +// The span/tracer shapes and the two inert singletons live in `span.js` so that +// `context/instrumentation.ts` can reach `NOOP_SPAN` for CTX-15's disabled-tracing default without +// closing an import cycle with this module's `InstrumentationBundle` type edge. They are re-exported +// verbatim here, which is this module's published surface and every existing import path. +export type {Span, SpanContext, Tracer} from './span.js'; +export {NOOP_SPAN, NOOP_TRACER} from './span.js'; function randomHex(byteLength: number): string { const bytes = new Uint8Array(byteLength); @@ -141,17 +84,42 @@ export function getActiveSpan(): Span { * @public */ export function activateSpan(span: Span): Scope { + requireSpan(span, 'activateSpan'); + + const restore = spanStorage.enter(span); + return {close: restore}; +} + +function requireSpan(span: Span, caller: string): void { invariant( (span as unknown) !== null && (span as unknown) !== undefined, - 'activateSpan: span is required', + `${caller}: span is required`, ); invariant( typeof span.end === 'function', - 'activateSpan: span must implement end()', + `${caller}: span must implement end()`, ); +} - const restore = spanStorage.enter(span); - return {close: restore}; +/** + * The callback form of {@link activateSpan}, for a scope that can be written as one function: `span` is + * active for the whole of `fn`, and whatever was active before is active again the moment `fn` returns. + * + * `activateSpan`'s handle cannot make that promise across an `await`. Its `close()` is an `enterWith` on + * whichever async resource happens to run it, so a scope opened before an `await` and closed after one + * leaves the span installed on the resource that opened it -- the caller's, when the opener is + * `Runtime.send`. This form is `AsyncLocalStorage.run`, which unwinds structurally instead. The handle + * stays because OBS-22 specifies one; this is what the runtime uses. + * + * @param span - the span to activate for the extent of `fn`. + * @param fn - the work to run with `span` active. Its result is passed through untouched. + * @returns whatever `fn` returned. + * + * @internal + */ +export function runWithActiveSpan(span: Span, fn: () => T): T { + requireSpan(span, 'runWithActiveSpan'); + return spanStorage.run(span, fn); } /** Extracts trace.id and span.id from OpenTelemetry-compatible spanContext() if present. */ diff --git a/packages/core/src/pagination/lifecycle.test.ts b/packages/core/src/pagination/lifecycle.test.ts index 5ea37ab..9a5cb27 100644 --- a/packages/core/src/pagination/lifecycle.test.ts +++ b/packages/core/src/pagination/lifecycle.test.ts @@ -1,9 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/lifecycle.test.ts -// Exercises: PAGE-11 (close BEFORE yielding items — the assertion appendix B does not make), PAGE-12 -// (close-on-abandon), PAGE-13 (parse failure closes inline, close error suppressed), PAGE-14 (single-use page -// view), PAGE-15 (close errors surface), PAGE-27 (exactly once on every path), PAGE-32 (consumer throw keeps -// consumer error primary, discarding return-phase close error). +// Exercises: PAGE-4 (a malformed parse result closes the response and names the invariant), PAGE-11 (close +// BEFORE yielding items — the assertion appendix B does not make), PAGE-12 (close-on-abandon), PAGE-13 (parse +// failure closes inline, close error suppressed), PAGE-14 (single-use page view), PAGE-15 (close errors +// surface), PAGE-27 (exactly once on every path), PAGE-32 (consumer throw keeps consumer error primary, +// discarding return-phase close error). import {expect, test} from 'bun:test'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {IoError} from '../io/errors.js'; @@ -451,3 +452,78 @@ test.each([ } }, ); + +// A strategy is caller code, and `parse`'s declared return type does not survive the seam: an +// `any`-typed JSON decode, a forgotten `return`, or a server field the caller trusted all land here +// as a shape the engine's own types say cannot exist (PAGE-4). The casts below ARE the test — they +// reproduce the four values that reach `#walk` in practice. +function malformedStrategy(result: unknown): PaginationStrategy { + return {parse: () => Promise.resolve(result as PageInfo)}; +} + +test.each([ + ['undefined', undefined, /never null or undefined/], + ['null', null, /never null or undefined/], + ['{items: null}', {items: null, nextRequest: undefined}, /PageInfo\.items/], + ['{items: undefined}', {nextRequest: undefined}, /PageInfo\.items/], +])( + 'a strategy that returns %s closes the response exactly once and names the invariant (PAGE-4, PAGE-27)', + async (_name, result, message) => { + const closed: number[] = []; + const transport = transportOf(1, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(result), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + // Before the fix `items: null` reached the spread in `Page`'s constructor and surfaced as a + // bare `TypeError` from array iteration, which names nothing a caller can act on. + expect((caught as Error).message).toMatch(message); + expect(closed).toEqual([0]); + }, +); + +test('a close failure while rejecting a malformed PageInfo is suppressed, not masking (PAGE-4, PAGE-13)', async () => { + const closeFailure = new IoError('close failed'); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(undefined), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect((suppressed.error as Error).message).toMatch( + /never null or undefined/, + ); + expect(suppressed.suppressed).toBe(closeFailure); +}); diff --git a/packages/core/src/pagination/page.test.ts b/packages/core/src/pagination/page.test.ts index 9bf8487..3c4901a 100644 --- a/packages/core/src/pagination/page.test.ts +++ b/packages/core/src/pagination/page.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/page.test.ts -// Exercises: PAGE-2 (items and metadata survive close; items never null), PAGE-3 (one owned response, closed -// exactly once), PAGE-4 (PageInfo shape, undefined next-request is the end signal). +// Exercises: PAGE-2 (items and metadata survive close; items never null, at construction too), PAGE-3 (one +// owned response, never null, closed exactly once), PAGE-4 (PageInfo shape, undefined next-request is the end +// signal). import {expect, test} from 'bun:test'; import {Page, pageInfo} from './page.js'; @@ -125,3 +126,25 @@ test('no "undefined" prototype key survives the guarded install (PAGE-12)', () = 'undefined', ); }); + +// `Page` is `@public`, so these guards are reachable from consumer code, not only from the walk — +// and until audit #67 / #79 their messages said "never null" while the check tested `!== undefined`. +// A `null` therefore reached the item copy and surfaced as a bare `TypeError` from spread. +test.each([ + ['null items', null, /items must never be null/], + ['undefined items', undefined, /items must never be null/], +])('a Page rejects %s at construction (PAGE-2)', (_name, items, message) => { + const {response} = fakeResponse(); + expect(() => + makePage(response, items as unknown as readonly number[]), + ).toThrow(message); +}); + +test('a Page rejects a null response at construction (PAGE-3)', () => { + expect(() => + makePage( + null as unknown as ConstructorParameters>[0], + [1], + ), + ).toThrow(/must own a response/); +}); diff --git a/packages/core/src/pagination/page.ts b/packages/core/src/pagination/page.ts index 9553812..0391257 100644 --- a/packages/core/src/pagination/page.ts +++ b/packages/core/src/pagination/page.ts @@ -67,12 +67,15 @@ export class Page { readonly #response: Response; constructor(response: Response, items: readonly T[]) { + // `!== null` as well as `!== undefined`: both messages have always said "null", and testing only + // for `undefined` let a `null` through to the item copy below, where it surfaced as a bare + // `TypeError` from spread — outside the error tree and naming neither field (audit #67 / #79). invariant( - (response as unknown) !== undefined, + (response as unknown) !== undefined && (response as unknown) !== null, 'a Page must own a response (PAGE-3)', ); invariant( - (items as unknown) !== undefined, + (items as unknown) !== undefined && (items as unknown) !== null, 'a Page’s items must never be null (PAGE-2)', ); diff --git a/packages/core/src/pagination/paginator.ts b/packages/core/src/pagination/paginator.ts index e331316..2a41f91 100644 --- a/packages/core/src/pagination/paginator.ts +++ b/packages/core/src/pagination/paginator.ts @@ -196,20 +196,7 @@ export class Paginator { response, request, ); - - // PAGE-4: parse must always return a well-formed result and must never signal termination through a - // side channel. A strategy that returns nothing is a programmer error, so it crashes at the fault - // rather than silently ending the walk as if the server had run out of pages. - invariant( - (info as unknown) !== undefined, - 'PaginationStrategy.parse must return a PageInfo, never undefined', - ); - invariant( - (info.items as unknown) !== undefined, - 'PageInfo.items must never be null or absent (PAGE-2)', - ); - - held = new Page(response, info.items); + held = await pageOrClose(response, info); request = info.nextRequest; yield held; } @@ -236,17 +223,67 @@ async function parseOrClose( try { return await strategy.parse(response, template); } catch (parseError: unknown) { - try { - await response.close(); - } catch (closeError: unknown) { - throw suppress( - parseError, - closeError, - 'pagination parse failed and releasing the response also failed', - ); - } - throw parseError; + return closeThenRethrow(response, parseError, 'pagination parse failed'); + } +} + +/** + * PAGE-4: `parse` must always return a well-formed result, and must never signal termination through a side + * channel. A strategy that returns nothing is a programmer error, so the walk crashes at the fault rather than + * silently ending as if the server had run out of pages. + * + * PAGE-27: and it crashes *after* releasing the response. `parse` returning a malformed value is the one exit + * from this loop the `finally` in `#walk` cannot cover — `held` is still `undefined` there, because assigning it + * is precisely what failed — so, like PAGE-13's parse rejection, the release happens inline (audit #67 / #79). + * + * Both checks reject `null` as well as `undefined`, which is what their messages have always claimed. Testing + * only for `undefined` let `{items: null}` through to `Page`'s constructor, where the item copy surfaced as a + * bare `TypeError` from spread — naming nothing a caller could act on, and leaking the response on the way. + * + * Not async: the only asynchrony here is the close, and only on the failure path. + */ +function pageOrClose( + response: Response, + info: PageInfo, +): Promise> { + try { + invariant( + (info as unknown) !== undefined && (info as unknown) !== null, + 'PaginationStrategy.parse must return a PageInfo, never null or undefined', + ); + invariant( + (info.items as unknown) !== undefined && (info.items as unknown) !== null, + 'PageInfo.items must never be null or absent (PAGE-2)', + ); + return Promise.resolve(new Page(response, info.items)); + } catch (buildError: unknown) { + return closeThenRethrow( + response, + buildError, + 'the pagination strategy returned a malformed PageInfo', + ); + } +} + +/** + * Release `response`, then rethrow `primary`. Shared by the two inline-close paths so they cannot drift: a close + * failure is attached as suppressed and never masks the failure that got here first (PAGE-13, PAGE-15). + */ +async function closeThenRethrow( + response: Response, + primary: unknown, + context: string, +): Promise { + try { + await response.close(); + } catch (closeError: unknown) { + throw suppress( + primary, + closeError, + `${context} and releasing the response also failed`, + ); } + throw primary; } /** PAGE-26: on an already-settled cancellation path, a close error is swallowed — nothing is left to report to. */ diff --git a/packages/core/src/pagination/query-splice.property.test.ts b/packages/core/src/pagination/query-splice.property.test.ts index f6c6761..56a51af 100644 --- a/packages/core/src/pagination/query-splice.property.test.ts +++ b/packages/core/src/pagination/query-splice.property.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.property.test.ts -import {test} from 'bun:test'; +import {expect, test} from 'bun:test'; import fc from 'fast-check'; +import {UrlConstructionError} from '../http/errors.js'; import {readQueryParam, spliceQueryParam} from './query-splice.js'; /** @@ -51,3 +52,40 @@ test('write-then-read is the identity for any value (PAGE-22)', () => { }), ); }); + +/** + * Strings that mix ordinary query text with UNPAIRED surrogate code units — the same generator + * `http/query-params.test.ts` uses, for the same reason: `fc.string()`'s default unit is printable + * ASCII, so the `URIError` path would otherwise go ungenerated. That is exactly why the identity + * property above never caught it. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); + +test('no URIError escapes the splice or the read, whatever a server sent (PAGE-22)', () => { + fc.assert( + fc.property( + surrogateBearingString, + surrogateBearingString, + (name, value) => { + const url = new URL('https://h/p?a=1'); + try { + const out = spliceQueryParam(url, name, value); + expect(readQueryParam(out, name)).toBe(value); + } catch (e: unknown) { + // The one sanctioned failure: inside the error tree, from the call that was handed the + // value. A `URIError` here means the guard was bypassed. + expect(e).toBeInstanceOf(UrlConstructionError); + } + }, + ), + {numRuns: 500}, + ); +}); diff --git a/packages/core/src/pagination/query-splice.test.ts b/packages/core/src/pagination/query-splice.test.ts index dc1ca1c..ca546a7 100644 --- a/packages/core/src/pagination/query-splice.test.ts +++ b/packages/core/src/pagination/query-splice.test.ts @@ -1,9 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.test.ts // Exercises: PAGE-21 (verbatim splice, untargeted params byte-for-byte), PAGE-22 (RFC 3986 component encoding, -// literal + is data), PAGE-23 (replace-first / append / remove, order preserved), PAGE-24 (non-query components -// preserved exactly). -import {expect, test} from 'bun:test'; +// literal + is data; a component with no UTF-8 form is rejected as UrlConstructionError), PAGE-23 +// (replace-first / append / remove, order preserved), PAGE-24 (non-query components preserved exactly). +import {describe, expect, test} from 'bun:test'; +import {UrlConstructionError} from '../http/errors.js'; import {readQueryParam, spliceQueryParam} from './query-splice.js'; const at = (href: string): URL => new URL(href); @@ -139,3 +140,57 @@ test('stray empty segments are skipped, matching HTTP-31 query parsing', () => { query(spliceQueryParam(at('https://h/p?a=1&&b=2&page=1'), 'page', '2')), ).toBe('a=1&b=2&page=2'); }); + +describe('a cursor with no UTF-8 form is rejected here, not inside encodeURIComponent (PAGE-22)', () => { + // The splice shares `HTTP-29`'s component encoder, and `encodeURIComponent` throws a bare + // `URIError: URI malformed` on a string carrying an unpaired surrogate. A cursor is SERVER + // -supplied — `{"next":"\ud800"}` is well-formed JSON — so this is reachable without any caller + // mistake, and until audit #67 / #79 it escaped the `DexpaceError` tree entirely. #76 closed the + // same hole at `QueryParamsBuilder.add` and left this one named. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate', LONE_HIGH], + ['a lone low surrogate', LONE_LOW], + ['a lone surrogate inside a longer cursor', `ok${LONE_HIGH}ok`], + ])('spliceQueryParam rejects %s as a value', (_label, value) => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', value), + ).toThrow(UrlConstructionError); + }); + + test('the message names the parameter and never echoes the value', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).toThrow(/value of query parameter "cursor"/); + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).not.toThrow(/secret/); + }); + + test('spliceQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, '2'), + ).toThrow(UrlConstructionError); + }); + + test('readQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => readQueryParam(at('https://h/p?a=1'), LONE_HIGH)).toThrow( + UrlConstructionError, + ); + }); + + test('removing a parameter still validates the name', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, undefined), + ).toThrow(UrlConstructionError); + }); + + test('a well-formed surrogate PAIR is ordinary text and splices normally', () => { + // Rejecting this too would make the rule "no astral characters", which PAGE-22 does not say. + const out = spliceQueryParam(at('https://h/p?a=1'), 'cursor', '\u{1F600}'); + expect(query(out)).toBe('a=1&cursor=%F0%9F%98%80'); + expect(readQueryParam(out, 'cursor')).toBe('\u{1F600}'); + }); +}); diff --git a/packages/core/src/pagination/query-splice.ts b/packages/core/src/pagination/query-splice.ts index ba38d77..754cc6a 100644 --- a/packages/core/src/pagination/query-splice.ts +++ b/packages/core/src/pagination/query-splice.ts @@ -1,9 +1,40 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.ts +import {UrlConstructionError} from '../http/errors.js'; import { decodeQueryComponent, encodeQueryComponent, } from '../http/query-params.js'; +import {hasLoneSurrogate} from '../http/rfc3986.js'; + +/** + * `encodeQueryComponent` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` on a string + * carrying an unpaired surrogate — such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined + * for it (PAGE-22, HTTP-29). + * + * Reachable here without any caller mistake, which is what separates this call site from the others #76 closed: + * a cursor is SERVER-supplied, and `{"next":"\ud800"}` is well-formed JSON that `JSON.parse` hands back + * verbatim. The failure was a bare `URIError` from inside a strategy's `parse`, outside the `DexpaceError` tree + * and naming neither the parameter nor the page it came from (audit #67 / #79). + * + * The same class `QueryParamsBuilder.add` throws for the same input, since it is the same defect in the same + * encoder; `hasLoneSurrogate` is the single-sourced predicate, so the two cannot drift. The value itself is + * never echoed — a cursor is opaque server state and can carry a session token. + */ +function requireEncodable( + what: 'name' | 'value', + parameterName: string, + text: string, +): void { + if (!hasLoneSurrogate(text)) return; + const subject = + what === 'name' + ? 'a query parameter name' + : `the value of query parameter "${parameterName}"`; + throw new UrlConstructionError( + `${subject} contains an unpaired surrogate and cannot be percent-encoded`, + ); +} /** * Rewrite one query parameter, splicing the raw query string rather than re-rendering it (PAGE-21–PAGE-24). @@ -19,6 +50,9 @@ import { * Passing `undefined` removes the parameter. Setting replaces the first occurrence in place and drops later * duplicates — the single-value convention paging parameters follow. Everything else is copied byte-for-byte. * + * @throws UrlConstructionError when `name` or `value` carries an unpaired surrogate, and so has no + * percent-encoded form. + * * @internal */ export function spliceQueryParam( @@ -26,6 +60,8 @@ export function spliceQueryParam( name: string, value: string | undefined, ): URL { + requireEncodable('name', name, name); + if (value !== undefined) requireEncodable('value', name, value); const encodedName = encodeQueryComponent(name); const segments = splitQuery(url.search); @@ -60,9 +96,12 @@ export function spliceQueryParam( * A literal `+` reads back as `+`, `%20` as a space, a value-less flag as the empty string, and an absent name * as `undefined`. First match wins. * + * @throws UrlConstructionError when `name` carries an unpaired surrogate, and so has no percent-encoded form. + * * @internal */ export function readQueryParam(url: URL, name: string): string | undefined { + requireEncodable('name', name, name); const encodedName = encodeQueryComponent(name); for (const segment of splitQuery(url.search)) { if (nameOf(segment) !== encodedName) continue; diff --git a/packages/core/src/pagination/strategies.test.ts b/packages/core/src/pagination/strategies.test.ts index 264602c..8eebdf1 100644 --- a/packages/core/src/pagination/strategies.test.ts +++ b/packages/core/src/pagination/strategies.test.ts @@ -3,8 +3,12 @@ // Exercises: PAGE-16 (cursor: single body read, null OR empty ends, configurable parameter), PAGE-17 // (page-number: empty items ends, start-page fallback on absent/empty/garbage, configurable name and start), // PAGE-18/19/20 (link header: rel=next, RFC 3986 reference resolution, query-only reference preserves the path, -// unresolvable target ends the stream without throwing). +// unresolvable target ends the stream without throwing, and the spec's own `` conformance fixture +// resolving as a relative reference instead -- recorded as a deliberate reading in docs/deviations.md under +// "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69)), PAGE-22 (a server-supplied cursor with +// no UTF-8 form fails inside the error tree). import {expect, test} from 'bun:test'; +import {DexpaceError, UrlConstructionError} from '../http/errors.js'; import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import { @@ -208,6 +212,33 @@ test('an unresolvable target ends the stream rather than throwing (PAGE-19)', as expect(info.items).toEqual(['a']); }); +test("the spec's `` fixture is a RELATIVE reference, so it is followed (PAGE-19)", async () => { + // PAGE-19's conformance note (`docs/product-spec/12-pagination.md:52`) gives `; rel=next` as an + // example of "stream ends, no exception". Under WHATWG URL — the resolver `strategies.ts` uses, and the + // only one available without a runtime dependency (SEAM-1) — a base makes that string a perfectly valid + // relative path reference: it resolves to `/repo/not%20a%20url`. The requirement's own normative sentence + // is "a target that CANNOT RESOLVE into a valid URL", and this one resolves, so the port follows it. Only + // the illustrative fixture disagrees; the test below keeps the end-of-stream half honest with a target that + // genuinely fails to resolve. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase" — + // rejected alternative: an ad-hoc "looks unparseable" heuristic in front of the resolver. + const strategy = linkHeaderStrategy({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({ + url: 'https://api.test/repo/issues?page=1', + headers: {link: ['; rel="next"']}, + }), + template('https://api.test/repo/issues?page=1'), + ); + expect( + () => new URL('not a url', 'https://api.test/repo/issues'), + ).not.toThrow(); + expect(info.nextRequest?.url.href).toBe( + 'https://api.test/repo/not%20a%20url', + ); +}); + test('the fixture above really is unparseable — the guard is not vacuous (PAGE-19)', () => { expect(() => new URL('http://[', 'https://api.test/items')).toThrow(); // And the near-miss that does NOT throw, pinned so nobody "simplifies" the fixture back to it later. @@ -265,3 +296,40 @@ test('one strategy instance is safe across two concurrent walks (PAGE-5)', async expect(first.nextRequest?.url.search).toBe('?page=2'); expect(second.nextRequest?.url.search).toBe('?page=10'); }); + +// ---- a server-supplied component with no UTF-8 form (PAGE-22, audit #67 / #79) ---- + +test('a cursor carrying an unpaired surrogate fails as UrlConstructionError, not URIError', async () => { + // `{"next":"\ud800"}` is well-formed JSON, so `extract` can hand one back without the caller + // having done anything wrong. Before the fix this surfaced as a bare `URIError: URI malformed` + // from inside `encodeURIComponent`, outside the `DexpaceError` tree. + const strategy = cursorStrategy({ + extract: () => Promise.resolve({items: ['a'], cursor: 'next\uD800'}), + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); + expect(caught).toBeInstanceOf(DexpaceError); +}); + +test('a page-number parameter name carrying an unpaired surrogate fails the same way', async () => { + const strategy = pageNumberStrategy({ + extract: () => Promise.resolve(['a']), + parameterName: 'p\uD800', + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); +}); diff --git a/packages/core/src/pagination/strategies.ts b/packages/core/src/pagination/strategies.ts index 165ce12..84bf409 100644 --- a/packages/core/src/pagination/strategies.ts +++ b/packages/core/src/pagination/strategies.ts @@ -23,6 +23,10 @@ function withUrl(template: Request, url: URL): Request { * A `null` **or empty** cursor ends the stream — both, because a server returning `""` for "no more pages" is * common enough that treating it as a real cursor produces an infinite walk. * + * @throws UrlConstructionError from `parse` when the cursor the server sent, or `parameterName`, carries an + * unpaired surrogate: such a string has no UTF-8 form and so no percent-encoded form either (PAGE-22). The + * engine closes the response on that path like any other parse failure (PAGE-13). + * * @public */ export function cursorStrategy(init: { @@ -54,9 +58,16 @@ export function cursorStrategy(init: { * An empty items list ends the stream **before** any arithmetic runs — defensive against servers that keep * returning an empty page past the end instead of signalling termination, which would otherwise walk forever. * - * The current page comes from the *executed* request's query, not the template's, because the template never - * changes across the walk. An absent, empty, or non-numeric value falls back to `startPage`; `startPage: 0` - * supports 0-based servers. + * The current page comes from the *executed* request's query (`response.request.url`), not the template's. + * The template is not a fixed page-1 request -- it advances with the walk, since this function returns the + * next one as `nextRequest` and the engine makes that the following hop's template + * (`paginator.ts:165,213`; the contract is on `PaginationStrategy.parse` in `strategy.ts:10-15`). It is the + * *pre-flight* request for this hop, so it is the response's own request that reflects a redirect or any + * rewrite a step applied on the way out, and that is the page number worth incrementing. An absent, empty, + * or non-numeric value falls back to `startPage`; `startPage: 0` supports 0-based servers. + * + * @throws UrlConstructionError from `parse` when `parameterName` carries an unpaired surrogate, which has no + * percent-encoded form (PAGE-22). * * @public */ diff --git a/packages/core/src/pipeline/builder.test.ts b/packages/core/src/pipeline/builder.test.ts index ad22b1a..9e5b306 100644 --- a/packages/core/src/pipeline/builder.test.ts +++ b/packages/core/src/pipeline/builder.test.ts @@ -8,13 +8,20 @@ // scratch), PIPE-23 (a colliding reload leaves prior content untouched, and a same-type pillar repeat inside // one batch seats only one step), PIPE-25 (flatten order), PIPE-38 (appendAll preserves batch order; // prependAll reverses it), PIPE-1/PIPE-2 (a built pipeline, driven: entry in STAGE_ORDER, exit reversed), -// PIPE-35 (seedFrom's explicit, non-defaulted flatten-vs-nest modes) +// PIPE-35 (seedFrom's explicit, non-defaulted flatten-vs-nest modes), OBS-29 + CTX-16 (the public +// instrumentation options bag: the supplied bundle opens the operation span, the operation name reaches +// the request context, and flatten seeding carries both) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {Protocol} from '../http/protocol.js'; import {Request} from '../http/request.js'; import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; import type {Transport} from '../seams/transport.js'; import {PipelineBuilder} from './builder.js'; import { @@ -545,3 +552,98 @@ describe('PipelineBuilder.seedFrom nest mode (PIPE-35)', () => { expect(runtime.transport).toBe(seeded); }); }); + +/** Records the name of every span it is asked to open, so a test can count operations. */ +function countingTracer(): {tracer: Tracer; names: string[]} { + const names: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + return { + names, + tracer: { + startSpan(name: string): Span { + names.push(name); + return span; + }, + }, + }; +} + +/** Captures what the drive's `RequestContext` says, from inside the pipeline. */ +function contextProbe(seen: { + operationName?: string | undefined; +}): StepDescriptor { + return { + type: Symbol('context-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen.operationName = + 'operationName' in ctx.context ? ctx.context.operationName : undefined; + return ctx.next(request); + }, + }; +} + +describe('PipelineBuilder instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle is what opens the per-operation span', async () => { + const {tracer, names} = countingTracer(); + const runtime = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append(descriptor('probe', 'PRE_SERDE')) + .build(); + + await runtime.send(aRequest('https://example.com')); + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation', 'http.client.operation']); + }); + + test('operationName reaches the request context every step reads (CTX-16)', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport(), { + operationName: 'GetUser', + }) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBe('GetUser'); + }); + + test('no options bag means the no-op bundle and no operation name', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport()) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBeUndefined(); + }); + + test('flatten seeding carries the seed runtime’s options (PIPE-35)', async () => { + const {tracer, names} = countingTracer(); + const seen: {operationName?: string | undefined} = {}; + const seeded = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + operationName: 'GetUser', + }) + .append(descriptor('seeded', 'LOGGING')) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'flatten') + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation']); + expect(seen.operationName).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/pipeline/builder.ts b/packages/core/src/pipeline/builder.ts index 43d0573..83b383e 100644 --- a/packages/core/src/pipeline/builder.ts +++ b/packages/core/src/pipeline/builder.ts @@ -1,5 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/pipeline/builder.ts +import type {InstrumentationBundle} from '../context/instrumentation.js'; import {invariant} from '../invariant.js'; import type {Transport} from '../seams/transport.js'; import { @@ -8,7 +9,7 @@ import { PillarCollisionError, ReservedStageError, } from './errors.js'; -import {createRuntime, type Runtime} from './runtime.js'; +import {createRuntime, pipelineOptionsOf, type Runtime} from './runtime.js'; import {PILLAR_STAGES, STAGE_ORDER, type Stage} from './stage.js'; import type {StepDescriptor} from './step.js'; @@ -17,6 +18,37 @@ interface AnchorLocation { readonly index: number; } +/** + * What a pipeline carries into every call it drives, as opposed to what a single call carries + * (`RequestOptions`) or what one step is configured with (a pillar's own settings). + * + * Both fields are per-pipeline by construction: `CTX-4` gives every `send()` its own context key, and + * the bundle is shared by reference across that call's three promotions (`CTX-2`/`CTX-3`), so a client + * that needs two operation names builds two pipelines — cheap, since `seedFrom(runtime, 'flatten')` + * derives the second from the first and carries these options with it. + * + * @public + */ +export interface PipelineOptions { + /** + * The correlation bundle every context of every call carries (`CTX-14`), and — through its + * `tracerFactory` — the source of the one `http.client.operation` span `Runtime.send()` opens per + * logical operation (`OBS-29`). Build one with `createInstrumentationBundle(tracerFactory)`. + * + * @defaultValue the disabled-tracing no-op bundle (`CTX-15`), which opens no span at all + */ + readonly instrumentation?: InstrumentationBundle | undefined; + /** + * The advisory operation label (`CTX-16`) — a schema-defined operation id such as `'GetUser'`. + * Carried unchanged from the request context through every promotion, exposed to the tracing seam + * (the LOGGING pillar step names its per-attempt span with it), and never an input to the request, + * the dispatch decision or the store key. + * + * @defaultValue `undefined` — a raw request that belongs to no named operation + */ + readonly operationName?: string | undefined; +} + /** * Assembles a stage-based pipeline via surgical edits (PIPE-7, PIPE-18..PIPE-24), flattening into an * immutable Runtime at build() time (PIPE-25). Mutable while being built; the produced Runtime is frozen. @@ -26,9 +58,19 @@ interface AnchorLocation { export class PipelineBuilder { readonly #buckets = new Map(); readonly #transport: Transport; + readonly #options: PipelineOptions; - constructor(transport: Transport) { + /** + * @param transport - the terminal transport the built pipeline dispatches to. Never closed by the + * pipeline (PIPE-27). + * @param options - what the built pipeline carries into every call: the instrumentation bundle and + * the advisory operation name. Optional, and optional in the second position deliberately — this + * is the only public way to reach `OBS-29`'s per-operation span and `CTX-16`'s operation name, + * and adding it must not break the one-argument construction every existing caller writes. + */ + constructor(transport: Transport, options: PipelineOptions = {}) { this.#transport = transport; + this.#options = options; } /** @@ -225,7 +267,15 @@ export class PipelineBuilder { */ static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder { if (mode === 'flatten') { - return new PipelineBuilder(runtime.transport).appendAll(runtime.steps); + // Flatten produces the pipeline that REPLACES `runtime`, so it inherits `runtime`'s + // `PipelineOptions` along with its steps: dropping them would silently un-trace a client that + // seeded from a traced preset, and there is no other way for the derived builder to recover + // them. `nest` needs no such carry -- `runtime` is still there, as the terminal transport, + // driving its own contexts with its own bundle. + return new PipelineBuilder( + runtime.transport, + pipelineOptionsOf(runtime), + ).appendAll(runtime.steps); } return new PipelineBuilder(runtime); } @@ -238,7 +288,9 @@ export class PipelineBuilder { const bucket = this.#buckets.get(stage); if (bucket !== undefined) flattened.push(...bucket); } - return createRuntime(flattened, this.#transport); // Runtime copies and freezes -- PIPE-10/PIPE-25. + // Runtime copies and freezes -- PIPE-10/PIPE-25. `PipelineOptions` is structurally the public + // half of `ContextInit`; the `key` half stays in-package, because CTX-4 wants one key per call. + return createRuntime(flattened, this.#transport, this.#options); } #rejectReservedStage(stage: Stage, operation: string): void { diff --git a/packages/core/src/pipeline/runtime.test.ts b/packages/core/src/pipeline/runtime.test.ts index 3db8ef9..6844ab4 100644 --- a/packages/core/src/pipeline/runtime.test.ts +++ b/packages/core/src/pipeline/runtime.test.ts @@ -8,7 +8,10 @@ // with one send() method, and nests inside another pipeline with the caller's options intact), PIPE-27 // (close() never touches the wrapped transport), CTX-17's positive half (the first store entry is installed // by the first promotion), CTX-1/2/3/6 (exchangeSource pins the call key and instrumentation when it -// rebuilds) +// rebuilds), OBS-22/OBS-23 (the caller's active span and diagnostic fields are what they were once +// send() settles, either way), OBS-29 (one operation span per send, ended exactly once even when end() +// throws, and a second send gets its own), CTX-11 (a throwing tracerFactory leaks no store entry), +// CTX-16 (the pipeline's operation name reaches the request context) import {describe, expect, test} from 'bun:test'; import { createRequestContext, @@ -22,7 +25,13 @@ import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; import {invariant} from '../invariant.js'; import { + getDiagnosticContext, + pushDiagnosticFields, +} from '../observability/diagnostic-context.js'; +import { + NOOP_SPAN, createInstrumentationBundle, + getActiveSpan, type Span, type Tracer, } from '../observability/tracing.js'; @@ -460,3 +469,124 @@ describe('the per-operation span: 1:1 with a logical operation (OBS-29)', () => expect(response.status.code).toBe(200); }); }); + +describe('async-context hygiene across send() (OBS-22, OBS-23, OBS-29)', () => { + test('the caller observes no active span after `await send()` resolves', async () => { + const {tracer} = recordingTracer(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + + await runtimeWith(tracer, new RecordingTransport(aResponse(200))).send( + aRequest('https://example.com'), + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('the caller observes no active span after `await send()` rejects', async () => { + const {tracer} = recordingTracer(); + const failing: StepDescriptor = { + type: Symbol('failing'), + stage: 'PRE_REDIRECT', + fn: () => Promise.reject(new Error('boom')), + }; + + await runtimeWith(tracer, new RecordingTransport(aResponse(200)), [failing]) + .send(aRequest('https://example.com')) + .then( + () => undefined, + () => undefined, + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('diagnostic fields a step pushed do not outlive the call (OBS-23)', async () => { + const pushing: StepDescriptor = { + type: Symbol('pushing'), + stage: 'PRE_REDIRECT', + fn: (request, ctx) => { + // What `activateSpanForCorrelation` does inside the LOGGING pillar step: a handle-based push + // whose restore runs in a later continuation and therefore never reaches this caller. + pushDiagnosticFields({'trace.id': 't-leaked', 'span.id': 's-leaked'}); + return ctx.next(request); + }, + }; + + await runtimeWith( + recordingTracer().tracer, + new RecordingTransport(aResponse(200)), + [pushing], + ).send(aRequest('https://example.com')); + + expect(getDiagnosticContext(null)).toEqual({}); + }); + + test('a second send() on the same runtime opens its own operation span (OBS-29)', async () => { + const {tracer, spans} = recordingTracer(); + const runtime = runtimeWith(tracer, new RecordingTransport(aResponse(200))); + + await runtime.send(aRequest('https://example.com/one')); + await runtime.send(aRequest('https://example.com/two')); + + expect(spans.length).toBe(2); + expect(spans.map(span => span.ended)).toEqual([1, 1]); + }); +}); + +describe('store and span hygiene on a failing tracer (CTX-11, OBS-29)', () => { + test('a throwing tracerFactory leaves the context store the size it found it', async () => { + const boom = new Error('tracer down'); + const runtime = createRuntime( + [passthroughStep()], + new RecordingTransport(aResponse(200)), + { + instrumentation: createInstrumentationBundle(() => { + throw boom; + }), + }, + ); + + const before = contextStore.size; + const thrown = await runtime.send(aRequest('https://example.com')).then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(boom); + expect(contextStore.size).toBe(before); + }); + + test('an end() that throws on the success path is not called a second time', async () => { + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span: Span = { + isRecording: true, + setAttribute(): Span { + return span; + }, + recordException(error: unknown): Span { + exceptions.push(error); + return span; + }, + end(): void { + ends += 1; + throw endFailed; + }, + }; + + const thrown = await runtimeWith( + {startSpan: () => span}, + new RecordingTransport(aResponse(200)), + ) + .send(aRequest('https://example.com')) + .then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([endFailed]); + }); +}); diff --git a/packages/core/src/pipeline/runtime.ts b/packages/core/src/pipeline/runtime.ts index 03c3ad6..fb87486 100644 --- a/packages/core/src/pipeline/runtime.ts +++ b/packages/core/src/pipeline/runtime.ts @@ -11,9 +11,13 @@ import { } from '../context/context.js'; import {contextStore} from '../context/store.js'; import { - activateSpan, + captureDiagnosticSnapshot, + runWithSnapshot, +} from '../observability/diagnostic-context.js'; +import { getActiveSpan, NOOP_TRACER, + runWithActiveSpan, type Span, type Tracer, } from '../observability/tracing.js'; @@ -24,8 +28,12 @@ import type {Transport} from '../seams/transport.js'; import {Cursor} from './cursor.js'; import type {StepDescriptor} from './step.js'; -/** What `Runtime.send()` passes to `createDispatchContext`; `operationName` is not a dispatch-stage concept. */ -type RuntimeContextInit = Omit; +/** + * What a built pipeline carries into every drive. `createDispatchContext` takes the `instrumentation` + * and `key` halves -- `operationName` is not a dispatch-stage concept (CTX-16 introduces it at the + * request stage) -- and `send()` hands the name to `promoteToRequest` itself, one promotion later. + */ +type RuntimeContextInit = ContextInit; /** The advisory span name, matching what the LOGGING pillar step uses for its per-attempt spans. */ const OPERATION_SPAN_NAME = 'http.client.operation'; @@ -58,6 +66,36 @@ function startOperationSpan(context: RequestContext): Span | undefined { return span.isRecording ? span : undefined; } +/** + * Runs `drive` as the body of `span` and ends that span EXACTLY once, whichever way it finishes + * (OBS-29: `operationSucceeded` and `operationFailed` are mutually exclusive and happen once each). + * + * The `ended` latch is not belt-and-braces. `end()` is caller-supplied through `tracerFactory`, and + * OBS-20 deliberately does not wrap tracer calls -- so a throwing `end()` on the success path lands + * in the `catch` below, which is obliged to `recordException` the failure it now has to surface. + * Without the latch that path called `end()` a second time on a span the tracer already closed. + */ +async function driveWithSpan( + span: Span, + drive: () => Promise, +): Promise { + let ended = false; + const endOnce = (): void => { + if (ended) return; + ended = true; + span.end(); + }; + try { + const response = await drive(); + endOnce(); + return response; + } catch (error: unknown) { + span.recordException(error); + endOnce(); + throw error; + } +} + /** * The request context to promote from once the drive finishes: the original, unless a step substituted the * outbound request (PIPE-14), in which case an off-chain rebuild around the request that was actually sent, @@ -98,6 +136,14 @@ let create: ( contextInit: RuntimeContextInit, ) => Runtime; +/** + * The read half of the same friend-class hook: `PipelineBuilder.seedFrom(runtime, 'flatten')` builds + * the pipeline that replaces `runtime`, so it has to recover the options `runtime` was built with, + * and `#contextInit` is private. Surfaced as {@link pipelineOptionsOf} rather than as a getter, + * because a getter on this `@public` class would publish `ContextInit`'s in-package `key` slot. + */ +let readContextInit: (runtime: Runtime) => RuntimeContextInit; + /** * The built, immutable pipeline (PIPE-10, PIPE-25). Implements `Transport` itself (PIPE-26) -- Phase 2's * `Transport` SPI has one method (`send`), so there is no second `sendAsync` entry point to delegate through. @@ -134,6 +180,7 @@ export class Runtime implements Transport { static { create = (steps, transport, contextInit) => new Runtime(steps, transport, contextInit); + readContextInit = runtime => runtime.#contextInit; } /** @@ -155,6 +202,12 @@ export class Runtime implements Transport { * tracer and no span is already active — `OBS-29`'s "one tracer instance per logical operation". * It is the parent of whatever per-attempt spans the LOGGING pillar step opens inside the RETRY * and REDIRECT pipelines, which `PIPE-2` fixes there and which are therefore per *transmission*. + * Supply the tracer through `PipelineOptions.instrumentation`. + * + * @remarks The caller's async context is restored when this settles, resolved or rejected: the + * active span and the diagnostic fields (`trace.id`, `span.id`) are what they were before the + * call, so an application log emitted after `await send()` carries nothing from it. Both stores + * are scoped with `AsyncLocalStorage.run`, which also unwinds a scope a step left open. */ async send( request: Request, @@ -165,15 +218,45 @@ export class Runtime implements Transport { // PIPE-9: an empty pipeline dispatches directly to the terminal transport, no cursor allocated. return this.#transport.send(request, options, signal); } + // Every async-scoped store this call touches is RE-RUN around the drive rather than entered in + // place. `runWithSnapshot(captureDiagnosticSnapshot())` re-enters the caller's OWN diagnostic + // store under `AsyncLocalStorage.run`, which changes nothing a step can observe and everything + // about what survives the call: a `pushDiagnosticFields` below -- the LOGGING pillar's OBS-23 + // correlation scope is the shipped one -- now unwinds when `send()` returns. `#drive` does the + // same for the span slot with `runWithActiveSpan`. + // + // Until 2026-09-05 both slots were `enterWith` plus a restore closure called from a `finally`. + // `enterWith` installs on the async resource running it, and that resource is the CALLER's -- + // `send`'s synchronous prefix runs there -- while the `finally` runs on a resource created by + // the first `await` inside. So the restore reached nothing the caller could see: after + // `await send()` the ended operation span was still "active", suppressing the next call's span + // (OBS-29's 1:1 binding), and this call's `trace.id`/`span.id` rode into every subsequent + // application log through any core `Logger` (audit #67 / #80). + return runWithSnapshot(captureDiagnosticSnapshot(), () => + this.#drive(request, options, signal), + ); + } + + /** + * One drive, inside the re-entered stores `send()` established. Split out so `send()` is the + * scoping statement and nothing else: the whole body has to sit inside the `run` callback for the + * unwind to cover it, and a body that long inline reads as if the callback were optional. + */ + async #drive( + request: Request, + options: RequestOptions | undefined, + signal: AbortSignal | undefined, + ): Promise { const dispatchContext = createDispatchContext(this.#contextInit); - const requestContext = promoteToRequest(dispatchContext, request); - contextStore.install(requestContext); // CTX-17's positive half: the first store entry, at the first promotion. + // CTX-16: the operation name this pipeline was built with enters at the request stage and is + // carried unchanged by every promotion after it. + const requestContext = promoteToRequest( + dispatchContext, + request, + this.#contextInit.operationName, + ); let currentContext: ExecutionContext = requestContext; // tracks the latest install for the finally below. - // OBS-29's 1:1 binding. Started before the drive and outside every pillar, so a retry's second - // attempt and a redirect's second hop are the same operation as the first. - const span = startOperationSpan(requestContext); - const scope = span === undefined ? undefined : activateSpan(span); - try { + const drive = async (): Promise => { const cursor = new Cursor({ steps: this.#steps, transport: this.#transport, @@ -190,16 +273,20 @@ export class Runtime implements Transport { ); contextStore.install(exchangeContext); // install-or-replace under the same key (CTX-8). currentContext = exchangeContext; - span?.end(); return response; - } catch (error: unknown) { - // OBS-29: `operationFailed` and `operationSucceeded` are mutually exclusive and happen once - // each. `end()` is reached from exactly one of these two paths, never both. - span?.recordException(error); - span?.end(); - throw error; + }; + // CTX-11/CTX-17: the install and everything that can throw after it are inside ONE try, so the + // `finally` evicts on every path. `startOperationSpan` calls a caller-supplied `tracerFactory`, + // which OBS-30 says must not throw and nothing enforces; installed outside the try, one throwing + // factory left an entry in the process-wide store per failed send. + try { + contextStore.install(requestContext); // CTX-17's positive half: the first store entry, at the first promotion. + // OBS-29's 1:1 binding. Started before the drive and outside every pillar, so a retry's second + // attempt and a redirect's second hop are the same operation as the first. + const span = startOperationSpan(requestContext); + if (span === undefined) return await drive(); + return await runWithActiveSpan(span, () => driveWithSpan(span, drive)); } finally { - scope?.close(); contextStore.close(currentContext); // always the most recently installed context for this call. } } @@ -253,9 +340,11 @@ export class Runtime implements Transport { * * @param steps - the flattened, stage-ordered step array. Copied and frozen. * @param transport - the terminal transport. Never closed by the pipeline (PIPE-27). - * @param contextInit - what each drive's dispatch context is built from: the `instrumentation` - * bundle whose `tracerFactory` supplies `OBS-29`'s per-operation span, and an optional `key` - * pinning two contexts to one store slot (CTX-5). Defaults to the no-op bundle and a fresh key. + * @param contextInit - what each drive's context chain is built from: the `instrumentation` bundle + * whose `tracerFactory` supplies `OBS-29`'s per-operation span, the advisory `operationName` + * every promotion carries (CTX-16), and an optional `key` pinning two contexts to one store slot + * (CTX-5). Defaults to the no-op bundle, no operation name, and a fresh key. `PipelineBuilder`'s + * second constructor argument is the public way to supply the first two. * @returns the built, immutable runtime. * * @internal @@ -267,3 +356,18 @@ export function createRuntime( ): Runtime { return create(steps, transport, contextInit); } + +/** + * What `runtime` was built to carry into every call — the instrumentation bundle and the advisory + * operation name. The `key` slot of `ContextInit` rides along in the returned object when the + * in-package caller pinned one; `PipelineBuilder` never does. + * + * @param runtime - the built pipeline to read. + * @returns its context init, by reference. Not a copy: `createDispatchContext` and + * `promoteToRequest` only read it, and the object came from a caller that already owns it. + * + * @internal + */ +export function pipelineOptionsOf(runtime: Runtime): RuntimeContextInit { + return readContextInit(runtime); +} diff --git a/packages/core/src/recovery/idempotency-key.ts b/packages/core/src/recovery/idempotency-key.ts index 4f81df0..e2b967f 100644 --- a/packages/core/src/recovery/idempotency-key.ts +++ b/packages/core/src/recovery/idempotency-key.ts @@ -30,9 +30,17 @@ export interface IdempotencyKeyOptions { /** * A `RequestStep` that stamps an idempotency key on write requests (RECOV-32). * - * Runs ONCE per call, upstream of retry -- not per attempt. `retry/attempt-stamp.ts` is its sibling: - * that one writes the attempt ordinal on each per-attempt copy and preserves whatever this wrote - * (RETRY-38), so the server sees one stable key across every retry of the same logical request. + * Runs ONCE per logical request, upstream of retry -- not per attempt. `retry/attempt-stamp.ts` is + * its sibling: that one writes the attempt ordinal on each per-attempt copy and preserves whatever + * this wrote (RETRY-38), so the server sees one stable key across every retry of the same logical + * request. + * + * **That is a property of the composition, and the SDK's own retry adapter is what supplies it**: + * `retry/retry-dispatch.ts` applies the `RequestRecoveryChain` once, above the retry loop, and each + * attempt re-sends a copy of the request it produced. On its own a step can only promise RECOV-32's + * letter -- `generate()` is invoked at most once per *application* to an applicable request -- so a + * caller who re-applies their own chain per attempt will get a fresh key per attempt. Install the + * chain once and let the retry layer sit below it. * * @param options - the key strategy plus the header name, method set, and existing-key policy. * @returns the request step to install in a `RequestRecoveryChain`. diff --git a/packages/core/src/recovery/orchestrator.ts b/packages/core/src/recovery/orchestrator.ts index 3f3e9c8..16e9645 100644 --- a/packages/core/src/recovery/orchestrator.ts +++ b/packages/core/src/recovery/orchestrator.ts @@ -29,47 +29,98 @@ export interface DispatchConfig { } /** - * The unified recovery-chain orchestrator (RECOV-2, RECOV-10, RECOV-11). + * {@link DispatchConfig} without the request chain: everything the phases BELOW that chain need. * - * One `try`/`catch` wraps both the request chain's `apply()` and the transport invocation, so every - * throwable from either is caught and converted into a Failure before the response chain runs — a - * before-request throw cannot skip after-error handling. That conversion goes through - * {@link wrapCancellation} (RECOV-11), this orchestrator's catch being its only call site; RECOV-2's - * guarantee rests on that helper never throwing, since this catch clause is the last place a - * throwable could escape without meeting the recovery hooks. + * A retry loop drives {@link dispatchPrepared} through this, having already run the request chain + * once above itself. Naming it as a subset rather than duplicating the fields keeps the two shapes + * from drifting when `DispatchConfig` grows. * - * The final unwrap returns the response on a Success, or rethrows the Failure's throwable - * **unchanged** — no wrapping, no substitution (RECOV-10). Surfacing a typed exception is a - * recovery step's own responsibility, never this function's. + * @internal + */ +export type PreparedDispatchConfig = Omit; + +/** + * The request chain, run ONCE per logical request, with RECOV-2's conversion already applied: a + * throwing step becomes a Failure here rather than propagating, so no caller has to catch it. That + * conversion goes through {@link wrapCancellation} (RECOV-11); RECOV-2's guarantee rests on that + * helper never throwing, since this `catch` is the last place a request-chain throwable could + * escape without meeting the recovery hooks. * - * @param request - the request to prepare and send. - * @param config - transport, chains, and the per-call options and signal. - * @returns the response the terminal outcome carries. - * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an - * `Error`. + * Split out of {@link dispatchWithRecovery} on 2026-09-05 so `retry/retry-dispatch.ts` can run this + * half once for a whole logical request and repeat only the half below it per attempt. Before that + * split every retry attempt re-ran the request chain, and `recovery/idempotency-key.ts` generated a + * fresh key on each one — three attempts of one logical request reached the server as three + * distinct idempotency keys, which is exactly what RECOV-32's key is bought to prevent. * - * @public + * @param request - the request to prepare. + * @param requestChain - the ordered request steps (RECOV-3). + * @returns a Success carrying the prepared request, or a Failure carrying whatever a step threw. + * Never throws, for any input. + * + * @internal */ -export async function dispatchWithRecovery( +export async function prepareRequest( request: Request, - config: DispatchConfig, -): Promise { - let outcome: Outcome; + requestChain: RequestRecoveryChain, +): Promise> { try { - const preparedRequest = await config.requestChain.apply(request); - outcome = success( + return success(await requestChain.apply(request)); + } catch (error) { + return wrapCancellation(error); + } +} + +/** + * The transport hop, with RECOV-2's conversion applied to whatever it throws. + * + * A `prepared` that is already a Failure short-circuits it: the transport is not called, and the + * failure is handed on for the response chain to see. Both branches widen `Outcome` and + * `Outcome` to `Outcome` without a cast, because the failure variant does not + * mention the type parameter — and neither branch throws, which is what keeps RECOV-2 absolute. + */ +async function sendPrepared( + prepared: Outcome, + config: PreparedDispatchConfig, +): Promise> { + if (prepared.kind === 'failure') return prepared; + try { + return success( await config.transport.send( - preparedRequest, + prepared.value, config.options, config.signal, ), ); } catch (error) { - // RECOV-11: `Outcome` widens to `Outcome` without a cast, and never throws, - // which is what keeps RECOV-2 absolute. - outcome = wrapCancellation(error); + return wrapCancellation(error); } - const finalOutcome = await config.responseChain.apply(outcome); +} + +/** + * Everything below the request chain — the transport hop, the response chain, and RECOV-10's + * terminal unwrap. This is the part a retry loop repeats, once per wire send (RETRY-44's + * "downstream chain"). + * + * It takes {@link prepareRequest}'s outcome rather than a bare `Request` because a request-chain + * failure still owes RECOV-2 a trip through the response and recovery chains before it surfaces. + * On that input the transport is not called at all, which is the whole difference between the two + * variants. + * + * @param prepared - {@link prepareRequest}'s result for this logical request. + * @param config - transport, response chain, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @internal + */ +export async function dispatchPrepared( + prepared: Outcome, + config: PreparedDispatchConfig, +): Promise { + const finalOutcome = await config.responseChain.apply( + await sendPrepared(prepared, config), + ); return fold( finalOutcome, response => response, @@ -78,3 +129,39 @@ export async function dispatchWithRecovery( }, ); } + +/** + * The unified recovery-chain orchestrator (RECOV-2, RECOV-10, RECOV-11). + * + * The two halves it composes are named: `prepareRequest` runs the request chain, and + * `dispatchPrepared` runs the transport hop and the response chain. Neither is exported from the + * package, so both are backticked rather than `{@link}`ed — api-extractor cannot resolve a + * reference out of the published surface into one, and the unresolved link is an error, not a + * warning to live with. Every throwable from either half is caught and converted into a Failure + * before the response chain runs — a before-request throw cannot skip after-error handling. + * + * The final unwrap returns the response on a Success, or rethrows the Failure's throwable + * **unchanged** — no wrapping, no substitution (RECOV-10). Surfacing a typed exception is a + * recovery step's own responsibility, never this function's. + * + * **One dispatch is one wire send.** Nothing here retries; a caller that wants retries composes the + * two halves itself so that the request chain runs once and only the second half repeats + * (`retry/retry-dispatch.ts`). + * + * @param request - the request to prepare and send. + * @param config - transport, chains, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @public + */ +export async function dispatchWithRecovery( + request: Request, + config: DispatchConfig, +): Promise { + return dispatchPrepared( + await prepareRequest(request, config.requestChain), + config, + ); +} diff --git a/packages/core/src/recovery/request-chain.ts b/packages/core/src/recovery/request-chain.ts index 7876e87..3426126 100644 --- a/packages/core/src/recovery/request-chain.ts +++ b/packages/core/src/recovery/request-chain.ts @@ -13,8 +13,13 @@ export type RequestStep = (request: Request) => Promise; /** * A sequential left-to-right fold over request steps (RECOV-3): the output of step N is the input * of step N+1, an empty chain returns its input unchanged, and a throwing step aborts the remainder - * and propagates — `dispatchWithRecovery` (`orchestrator.ts`) converts that propagation into a - * `Failure` per RECOV-2, which is the only reason propagating here is safe. + * and propagates — `prepareRequest` (`orchestrator.ts`) converts that propagation into a `Failure` + * per RECOV-2, which is the only reason propagating here is safe. Every entry point runs the chain + * through that one helper, the retry adapter included. + * + * **Apply the chain once per logical request, not once per wire send.** Steps here are the ones + * whose output must stay stable across a retry — `idempotencyKeyStep` above all — and the SDK's own + * retry adapter sits below this chain for that reason (`retry/retry-dispatch.ts`). * * Safe under concurrent `apply()` calls (RECOV-14): after construction the instance holds nothing * but its step array, and every piece of per-call state lives in `apply()`'s locals. A later phase diff --git a/packages/core/src/redirect/decide.test.ts b/packages/core/src/redirect/decide.test.ts index 14a84e9..1a90f60 100644 --- a/packages/core/src/redirect/decide.test.ts +++ b/packages/core/src/redirect/decide.test.ts @@ -15,7 +15,9 @@ // replayability gate; 303 exempt), REDIR-7 (Authorization always stripped), REDIR-9/REDIR-10 (Cookie and // Proxy-Authorization stripped only cross-origin), REDIR-11 (the marker set only on a cross-origin hop), // REDIR-5 (the 303 GET rebuild drops the body and every Content-* header), REDIR-3/REDIR-4 (a followed -// method-preserving redirect keeps the original method). +// method-preserving redirect keeps the original method), and REDIR-3's eligibility reference point -- the +// CURRENT hop's method rather than the spec's literal "original request method", recorded as a deliberate +// reading in docs/deviations.md under "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import type {Body} from '../body/body.js'; @@ -753,6 +755,51 @@ describe('header construction -- the 303 rebuild and method preservation', () => }); }); +describe('REDIR-3 measures eligibility against the CURRENT hop, not the original request', () => { + // The spec says "the ORIGINAL request method" (`docs/product-spec/10-redirect-handling.md:8`); the port + // reads `currentRequest.method` (`decide.ts:241` into `codes.ts:69`). The two readings agree on every + // chain except one, and this is it: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 arriving on + // that rewritten hop is followed under the DEFAULT {GET, HEAD} set, where the literal reading would refuse + // it because the request that started the chain was a POST. Kept deliberately -- the rewritten GET is + // idempotent and carries no body, so the literal reading buys no safety, and refusing the hop would make + // `allow303` half-useful. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase". + test('a 303-rewritten GET makes a following 301 eligible under the default method set (REDIR-3/REDIR-5)', () => { + const original = aRequest({ + method: 'POST', + url: 'https://example.com/a', + body: stringBody('x'), + }); + const first = decide( + aResponse(303, 'https://example.com/b'), + contextFor(original), + redirectSettings({allow303: true}), + ); + expect(first.kind).toBe('follow'); + if (first.kind !== 'follow') return; + expect(first.nextRequest.method).toBe('GET'); + + // Nothing about the second hop is opted into: the default set is {GET, HEAD} and excludes the method + // the chain started with, which is what makes the two readings disagree here rather than coincide. + const settings = redirectSettings(); + expect(settings.allowedMethods.has('POST')).toBe(false); + expect(original.method).toBe('POST'); + + const second = decide( + aResponse(301, 'https://example.com/c'), + contextFor(first.nextRequest, { + visited: new Set([original.url.href, first.nextRequest.url.href]), + redirectsFollowed: 1, + }), + settings, + ); + expect(second.kind).toBe('follow'); + if (second.kind === 'follow') { + expect(second.nextRequest.url.href).toBe('https://example.com/c'); + expect(second.nextRequest.method).toBe('GET'); + } + }); +}); + describe('loop detection survives URL normalization', () => { // `visited` keys on `URL.href`, which WHATWG normalizes -- so a server cannot spin the loop past the // cap by varying only the case of the scheme/host or by writing the scheme's default port out. Both diff --git a/packages/core/src/redirect/errors.test.ts b/packages/core/src/redirect/errors.test.ts index b9442b6..67f6ec1 100644 --- a/packages/core/src/redirect/errors.test.ts +++ b/packages/core/src/redirect/errors.test.ts @@ -4,6 +4,11 @@ // corrupting or truncating the re-send), REDIR-15 (an HTTPS->HTTP hop is rejected with a clear error by // default). Both are operational failures a caller can legitimately hit mid-redirect, so both are typed // error leaves rather than `invariant()` programmer-error assertions. +// Also: OBS-11 (userinfo is always redacted to `***:***@`), OBS-12 (query values are `***` unless +// allow-listed), OBS-15 (redaction is total -- an unparseable input yields the sentinel, never a throw) +// and XCUT-19(a)/(b) as they apply to the ERROR MESSAGE rather than to a log field. The message is what +// every logger, `cause` chain and consumer `console.error` renders, so it is redacted at construction; +// `targetUrl` / `fromUrl` / `toUrl` stay raw for program use. import {describe, expect, test} from 'bun:test'; import {DexpaceError} from '../http/errors.js'; import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; @@ -31,6 +36,36 @@ describe('NonReplayableBodyError', () => { new NonReplayableBodyError('https://example.com/next', {cause}).cause, ).toBe(cause); }); + + test('redacts userinfo and non-allow-listed query values in the message', () => { + const error = new NonReplayableBodyError( + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET&api-version=2', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + // OBS-12's default allow-list is exactly {api-version}, and the message inherits it whole. + expect(error.message).toContain('api-version=2'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + }); + + test('keeps the RAW target on the field even when the message is redacted', () => { + const raw = + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET'; + expect(new NonReplayableBodyError(raw).targetUrl).toBe(raw); + }); + + test('degrades an unparseable target to the sentinel rather than throwing (OBS-15)', () => { + // A target that never parsed cannot be redacted, and OBS-15 makes that total: the sentinel, not + // the raw string, because "unparseable" is not the same as "carries no secret". + const error = new NonReplayableBodyError('not a url?token=SUPERSECRET'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.targetUrl).toBe('not a url?token=SUPERSECRET'); + }); }); describe('SchemeDowngradeError', () => { @@ -59,4 +94,36 @@ describe('SchemeDowngradeError', () => { const error = new SchemeDowngradeError('https://a', 'http://b', {cause}); expect(error.cause).toBe(cause); }); + + test('redacts userinfo and non-allow-listed query values on BOTH sides of the message', () => { + const error = new SchemeDowngradeError( + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET', + 'http://example.com/next?code=ALSOSECRET', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + expect(error.message).toContain('code=***'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.message).not.toContain('ALSOSECRET'); + }); + + test('keeps BOTH raw URLs on the fields even when the message is redacted', () => { + const from = 'https://alice:hunter2@example.com/start?access_token=SECRET'; + const to = 'http://example.com/next?code=ALSOSECRET'; + const error = new SchemeDowngradeError(from, to); + + expect(error.fromUrl).toBe(from); + expect(error.toUrl).toBe(to); + }); + + test('degrades an unparseable side to the sentinel rather than throwing (OBS-15)', () => { + const error = new SchemeDowngradeError('::::', 'http://example.com/next'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).toContain('http://example.com/next'); + expect(error.fromUrl).toBe('::::'); + }); }); diff --git a/packages/core/src/redirect/errors.ts b/packages/core/src/redirect/errors.ts index 16ec10b..b893f3a 100644 --- a/packages/core/src/redirect/errors.ts +++ b/packages/core/src/redirect/errors.ts @@ -1,6 +1,18 @@ // SPDX-License-Identifier: MIT // packages/core/src/redirect/errors.ts import {DexpaceError} from '../http/errors.js'; +import {redactUrl} from '../observability/redaction.js'; + +// Both messages below interpolate `redactUrl(...)`, never the raw URL -- XCUT-19(a)/(b) and +// OBS-11/OBS-12 applied to the MESSAGE rather than to a log field. An error message is a public API: +// it travels into every logger, every `cause` chain, and every consumer's own `console.error`, and +// this SDK owns none of those. `http.redirect.rejected` in particular hands the decision error to +// `LogEvent.cause()`, which renders it as `name: message` (`observability/logger.ts`), so a raw +// `from`/`to` URL here put userinfo and query-string tokens into the log record in clear text however +// carefully the surrounding fields were redacted. Redacting at construction is the only placement that +// also covers the paths this SDK does not own. OBS-15 makes the call safe from a constructor: an input +// that will not parse yields `[malformed url]` rather than throwing. The raw value stays on the +// error's own property, which is what program code reads. /** * REDIR-6: a method-preserving redirect (301/302/307/308) re-sends the original body, so the body must @@ -8,15 +20,23 @@ import {DexpaceError} from '../http/errors.js'; * already-consumed single-use body -- this one is a fail-fast gate evaluated BEFORE any write is * attempted, and names replayability specifically as the requirement demands. * + * @remarks + * The `message` names the target in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link NonReplayableBodyError.targetUrl} for the raw URL. + * * @public */ export class NonReplayableBodyError extends DexpaceError { /** - * The redirect target that would have received the re-send. + * The redirect target that would have received the re-send, **raw and unredacted**. * * Carried as a field, not only interpolated into the message, per * `docs/knowledge/harvested/error-handling.md` -- so it survives serialization and reaches a structured log * without anyone parsing the message back apart. Phase 7b's rejection event reads it directly. + * + * This is the raw URL; the `message` carries the redacted form. Program code that needs the real + * target reads this property; anything that renders to a human or to a log backend reads the + * message. */ readonly targetUrl: string; @@ -26,7 +46,7 @@ export class NonReplayableBodyError extends DexpaceError { */ constructor(targetUrl: string, options?: ErrorOptions) { super( - `cannot follow redirect to '${targetUrl}': request body is not replayable`, + `cannot follow redirect to '${redactUrl(targetUrl)}': request body is not replayable`, options, ); this.targetUrl = targetUrl; @@ -37,12 +57,23 @@ export class NonReplayableBodyError extends DexpaceError { * REDIR-15: an HTTPS-to-HTTP hop, rejected unless `RedirectSettings.allowSchemeDowngrade` is set. * Evaluated per hop transition, so an HTTPS-to-HTTP-to-HTTPS chain flags only the hop that downgraded. * + * @remarks + * The `message` names both URLs in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link SchemeDowngradeError.fromUrl} and + * {@link SchemeDowngradeError.toUrl} for the raw URLs. + * * @public */ export class SchemeDowngradeError extends DexpaceError { - /** The current hop's request URL -- the HTTPS side of the rejected transition. */ + /** + * The current hop's request URL -- the HTTPS side of the rejected transition, **raw and + * unredacted**. The `message` carries the redacted form. + */ readonly fromUrl: string; - /** The resolved redirect target -- the HTTP side of the rejected transition. */ + /** + * The resolved redirect target -- the HTTP side of the rejected transition, **raw and unredacted**. + * The `message` carries the redacted form. + */ readonly toUrl: string; /** @@ -52,7 +83,7 @@ export class SchemeDowngradeError extends DexpaceError { */ constructor(fromUrl: string, toUrl: string, options?: ErrorOptions) { super( - `redirect from '${fromUrl}' to '${toUrl}' would downgrade HTTPS to HTTP`, + `redirect from '${redactUrl(fromUrl)}' to '${redactUrl(toUrl)}' would downgrade HTTPS to HTTP`, options, ); this.fromUrl = fromUrl; diff --git a/packages/core/src/redirect/redirect-step.test.ts b/packages/core/src/redirect/redirect-step.test.ts index 3608ed9..9fb52cd 100644 --- a/packages/core/src/redirect/redirect-step.test.ts +++ b/packages/core/src/redirect/redirect-step.test.ts @@ -9,6 +9,11 @@ // closes the current response and propagates SchemeDowngradeError), and the cancellation check (an // an abort DURING a hop returns the current response open rather than issuing a further hop, while a // signal already aborted at entry is refused by the cursor before the step runs -- see V15). +// +// Also: REDIR-28 / XCUT-19 / OBS-11 / OBS-12 for the `http.redirect.rejected` record specifically. The +// other three redirect events already route their URL fields through `redactUrl()`; the rejection +// event carried the decision error's MESSAGE, which interpolated the raw `from`/`to` URLs, so a +// userinfo password or a query-string token reached the log record in clear text. import {describe, expect, test} from 'bun:test'; import { createRequestContext, @@ -28,6 +33,7 @@ import type {SuppressedErrorLike} from '../suppress.js'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; import {REDIRECT_STEP_TYPE, redirectStep} from './redirect-step.js'; +import type {RedirectSettings} from './settings.js'; const SEED = Request.newBuilder().url('https://example.com/start').build(); const CANCEL_FAILURE = new Error('cancel exploded'); @@ -587,3 +593,140 @@ describe('Phase 7b retrofit: redirect rejection logging', () => { } }); }); + +// --- REDIR-28 / XCUT-19: the rejection record --------------------------------------------------- + +/** A seed carrying both things OBS-11 and OBS-12 name: userinfo, and a non-allow-listed query value. */ +const SECRET_SEED = + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET'; +/** What `redactUrl()` makes of {@link SECRET_SEED}; the shape every URL field on this path must have. */ +const REDACTED_SEED = 'https://***:***@example.com/start?access_token=***'; + +/** + * Drives one redirect step to settlement over `transport`, capturing every record it emits, and + * restores the shipped no-op logger before returning. Rejections are swallowed: each caller here is + * asserting on the LOG, and the thrown error's own redaction is `errors.test.ts`'s row. + */ +async function recordsFromRun( + seed: Request, + transport: FakeTransport, + overrides?: Partial, +): Promise[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await rejectionOf( + new Cursor({ + steps: [redirectStep(overrides)], + transport, + request: seed, + context: aRequestContext(seed), + }).advance(), + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + +/** The one `http.redirect.rejected` record, rendered field-by-field as a logger backend sees it. */ +function rejection( + events: Map[], +): Map | undefined { + const rejected = events.filter( + e => e.get('event') === 'http.redirect.rejected', + ); + expect(rejected).toHaveLength(1); + return rejected[0]; +} + +/** Asserts no field of `record` carries any of `secrets` in clear text -- XCUT-19's actual claim. */ +function expectNoSecret( + record: Map | undefined, + secrets: readonly string[], +): void { + for (const field of record?.values() ?? []) { + for (const secret of secrets) { + expect(String(field)).not.toContain(secret); + } + } +} + +const SECRETS = ['alice', 'hunter2', 'SUPERSECRET', 'ALSOSECRET'] as const; + +describe('REDIR-28/XCUT-19: the rejection record never carries a raw URL', () => { + test('redacts the downgrade rejection cause and its url.full field', async () => { + const hop = countingResponse(301); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([ + withLocation(hop.response, 'http://example.com/next?code=ALSOSECRET'), + ]), + ); + + const record = rejection(records); + // The logger renders a cause as `name: message` (observability/logger.ts), so the MESSAGE is the + // log field. The 8192-byte field cap truncates nothing here, and would not have helped if it did. + expect(String(record?.get('cause'))).toStartWith('SchemeDowngradeError: '); + expect(String(record?.get('cause'))).toContain('***:***@'); + expect(String(record?.get('cause'))).toContain('access_token=***'); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('redacts the non-replayable-body rejection cause and its url.full field', async () => { + const oneShot = streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); + const hop = countingResponse(307); + const records = await recordsFromRun( + Request.newBuilder() + .method('POST') + .url(SECRET_SEED) + .body(oneShot) + .build(), + new FakeTransport([ + withLocation(hop.response, 'https://example.com/next?code=ALSOSECRET'), + ]), + {allowedMethods: new Set(['GET', 'HEAD', 'POST'])}, + ); + + const record = rejection(records); + expect(String(record?.get('cause'))).toStartWith( + 'NonReplayableBodyError: ', + ); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('carries url.full on a rejection with no error, matching the other events', async () => { + // The `return-current` rejections -- loop detected, hop cap, malformed Location -- emit the same + // event with no cause. They still name the hop, and it is still redacted. The malformed-Location + // one is the sharpest case: REDIR-28 lets the sibling `malformedLocation` event log the header + // RAW, so this record is the only redacted URL anywhere on that path. + const hop = countingResponse(302); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([withLocation(hop.response, 'javascript:alert(1)')]), + ); + + const record = rejection(records); + expect(record?.has('cause')).toBe(false); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); +}); diff --git a/packages/core/src/redirect/redirect-step.ts b/packages/core/src/redirect/redirect-step.ts index e3951d3..c324d3f 100644 --- a/packages/core/src/redirect/redirect-step.ts +++ b/packages/core/src/redirect/redirect-step.ts @@ -88,11 +88,22 @@ function emitStopReason(stop: { } } -function emitRejected(error?: unknown): void { +/** + * REDIR-28's rejection event. Carries `url.full` -- the hop the rejection is about -- through + * `redactUrl()`, the same field and the same policy `http.redirect.hop`, `loopDetected` and + * `malformedLocation` already use; before that this record named no URL at all, and the only URL a + * reader could recover was the raw one interpolated into the cause's message. + * + * The cause itself is safe to attach because `redirect/errors.ts` now builds both message texts from + * `redactUrl()` output. That is deliberately fixed at the ERROR rather than dropped here: `cause` is + * also what a caller's own `console.error` renders, and this step cannot redact that one. + */ +function emitRejected(request: Request, error?: unknown): void { try { const event = getGlobalLogger() .atLevel('warning') - .event('http.redirect.rejected'); + .event('http.redirect.rejected') + .field('url.full', redactUrl(request.url)); if (error !== undefined) { event.cause(error); } @@ -216,7 +227,7 @@ export function redirectStep( if (decision.kind === 'return-current') { if (response.status.isRedirect) { - emitRejected(); + emitRejected(request); emitStopReason({ reason: decision.reason, request, @@ -227,7 +238,7 @@ export function redirectStep( } if (decision.kind === 'fail') { const releaseError = await releaseQuietly(response); - emitRejected(decision.error); + emitRejected(request, decision.error); throw withReleaseFailure(decision.error, releaseError); } if (signal?.aborted === true) return response; diff --git a/packages/core/src/retry/attempt-trail.test.ts b/packages/core/src/retry/attempt-trail.test.ts new file mode 100644 index 0000000..b5835be --- /dev/null +++ b/packages/core/src/retry/attempt-trail.test.ts @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.test.ts +// Exercises: RETRY-34 (the prior-attempt trail rides alongside the surfaced error rather than +// replacing it; the surfaced instance is skipped; a run with no priors leaves no trail behind), +// XCUT-1 (recording a trail never changes the surfaced value's class). +import {describe, expect, test} from 'bun:test'; +import {IoError} from '../io/errors.js'; +import {CancellationError} from '../seams/transport.js'; +import {recordAttempts, retryAttempts} from './attempt-trail.js'; + +describe('retryAttempts -- the read side', () => { + test('returns an empty list for an error that never went through the engine', () => { + expect(retryAttempts(new IoError('never retried'))).toEqual([]); + }); + + test('returns an empty list for a primitive, which cannot carry a trail at all', () => { + // RETRY-34's trail is keyed by identity, so a string, a number or a symbol throw passes + // through the engine unchanged and unannotated rather than being wrapped to make room. + expect(retryAttempts('a bare string throw')).toEqual([]); + expect(retryAttempts(42)).toEqual([]); + expect(retryAttempts(Symbol('thrown'))).toEqual([]); + expect(retryAttempts(null)).toEqual([]); + expect(retryAttempts(undefined)).toEqual([]); + }); + + test('returns the recorded attempts oldest first', () => { + const first = new IoError('first'); + const second = new IoError('second'); + const surfaced = new IoError('third'); + + recordAttempts(surfaced, [first, second]); + + expect(retryAttempts(surfaced)).toEqual([first, second]); + }); + + test('hands back a frozen list, so one caller cannot edit a trail another caller holds', () => { + const surfaced = new IoError('surfaced'); + recordAttempts(surfaced, [new IoError('prior')]); + + const attempts = retryAttempts(surfaced); + + expect(Object.isFrozen(attempts)).toBe(true); + }); + + test('does not read a trail off the cause chain -- only off the instance itself', () => { + const inner = new IoError('inner'); + recordAttempts(inner, [new IoError('prior')]); + const outer = new IoError('outer', {cause: inner}); + + expect(retryAttempts(outer)).toEqual([]); + }); +}); + +describe('recordAttempts -- the write side', () => { + test('leaves the class and identity of the error untouched (XCUT-1)', () => { + const surfaced = new CancellationError('operation cancelled'); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(surfaced).toBeInstanceOf(CancellationError); + expect(surfaced.name).toBe('CancellationError'); + }); + + test('adds no own property, so a JSON or structured-clone round trip is unchanged', () => { + const surfaced = new IoError('surfaced'); + const before = Object.getOwnPropertyNames(surfaced).sort(); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(Object.getOwnPropertyNames(surfaced).sort()).toEqual(before); + }); + + test('copies the trail, so a later push by the engine cannot mutate a published list', () => { + const surfaced = new IoError('surfaced'); + const trail: unknown[] = [new IoError('prior')]; + + recordAttempts(surfaced, trail); + trail.push(new IoError('added afterwards')); + + expect(retryAttempts(surfaced)).toHaveLength(1); + }); +}); + +describe('recordAttempts -- the values it can and cannot key on', () => { + test('is a no-op on a frozen error rather than throwing', () => { + // The reason the trail is a side table and not an own property: a foreign error may be frozen + // or non-extensible, and `defineProperty` inside the failure path of the engine would then replace + // the failure the caller cares about with a TypeError. + const surfaced = Object.freeze(new IoError('frozen by its author')); + const prior = new IoError('prior'); + + expect(() => { + recordAttempts(surfaced, [prior]); + }).not.toThrow(); + expect(retryAttempts(surfaced)).toEqual([prior]); + }); + + test('is a no-op for a primitive surfaced value', () => { + expect(() => { + recordAttempts('a bare string throw', [new IoError('prior')]); + }).not.toThrow(); + expect(retryAttempts('a bare string throw')).toEqual([]); + }); + + test('records against a thrown function, which is an object for these purposes', () => { + const surfaced = (): void => undefined; + const prior = new IoError('prior'); + + recordAttempts(surfaced, [prior]); + + expect(retryAttempts(surfaced)).toEqual([prior]); + }); +}); + +describe('recordAttempts -- a reused error instance', () => { + test('an empty trail clears any entry a previous run left on a reused instance', () => { + // Error singletons are ordinary in fakes and in transports that reuse one instance. The RETRY-34 + // clause "on eventual success the prior trail MUST be discarded" is worth nothing if the NEXT run + // to surface that same instance still reports the old one. + const reused = new IoError('reused across runs'); + recordAttempts(reused, [new IoError('from the first run')]); + + recordAttempts(reused, []); + + expect(retryAttempts(reused)).toEqual([]); + }); + + test('the latest recording wins for a reused instance', () => { + const reused = new IoError('reused across runs'); + const older = new IoError('from the first run'); + const newer = new IoError('from the second run'); + + recordAttempts(reused, [older]); + recordAttempts(reused, [newer]); + + expect(retryAttempts(reused)).toEqual([newer]); + }); +}); diff --git a/packages/core/src/retry/attempt-trail.ts b/packages/core/src/retry/attempt-trail.ts new file mode 100644 index 0000000..b526913 --- /dev/null +++ b/packages/core/src/retry/attempt-trail.ts @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.ts + +/** + * RETRY-34's prior-attempt trail, held in a side table keyed by the surfaced throwable rather than + * written onto it. + * + * A `WeakMap` because the key is the error a caller is about to catch: the entry has to disappear + * when that error does, and a `Map` here would pin every failed request's error graph -- including + * whatever the buffered `HttpStatusError` bodies hold -- for the life of the process. + */ +const attemptTrails = new WeakMap(); + +/** One shared frozen empty list, so the common "no trail" answer allocates nothing. */ +const NO_ATTEMPTS: readonly unknown[] = Object.freeze([]); + +/** + * The subset of throwables a `WeakMap` can key on. Objects and functions qualify; primitives do + * not, and a registered symbol throws when used as a weak key, so symbols are excluded outright + * rather than probed. + */ +function trailKey(value: unknown): object | undefined { + if (typeof value === 'function') return value; + return typeof value === 'object' && value !== null ? value : undefined; +} + +/** + * Records the errors of the earlier attempts against the error the retry engine is about to + * surface (`RETRY-34`). + * + * Written into a side table instead of onto the error for three reasons the engine cannot rule out + * about a throwable it did not construct: it may be frozen or otherwise non-extensible, so a + * `defineProperty` in the failure path would itself throw and replace the failure the caller cares + * about; it may be a primitive, which can carry nothing at all; and `suppressed` already means + * "the one secondary" on `SuppressedErrorLike`, so reusing the name would collide with + * `RECOV-12`'s pairing. + * + * `attempts` MUST already have the surfaced instance filtered out -- `RETRY-34`'s skip-self clause, + * applied by the engine's `attachTrail` (`engine.ts`), which is the one caller. An empty + * `attempts` DELETES any entry a previous run left, so a transport that reuses a single error + * instance across calls reports the trail of the run that just surfaced it rather than a stale one. + * + * The list is copied and frozen, so the engine's own mutable `trail` array cannot be observed + * growing after the fact. + * + * @param error - the throwable the engine is surfacing; a primitive is silently ignored. + * @param attempts - the earlier attempts' errors, oldest first, surfaced instance excluded. + * + * @internal + */ +export function recordAttempts( + error: unknown, + attempts: readonly unknown[], +): void { + const key = trailKey(error); + if (key === undefined) return; + if (attempts.length === 0) { + attemptTrails.delete(key); + return; + } + attemptTrails.set(key, Object.freeze([...attempts])); +} + +/** + * The errors of the attempts that came before the one you caught. + * + * The retry pillar surfaces the **final** attempt's own error, unwrapped: `instanceof` against it + * answers the same for one attempt as for ten, and a cancellation that ended a backoff wait arrives + * as `CancellationError` rather than as something carrying one. The earlier attempts are not + * discarded — they are recorded here, one entry per attempt that failed BEFORE the error you caught. + * A worked example is in `docs/sdk-documentation/pipelines.md`. + * + * **That is not an attempt count, and `length + 1` is not one either.** The arithmetic holds only + * when the surfaced error is itself an attempt's, and on three reachable paths it is not: a + * cancellation or timeout the engine observes at its `RETRY-32` gate is synthesized there rather + * than raised by a send; a failure from stamping the attempt header is raised before the request + * goes out; and a `Clock.sleep` that rejects for something other than an abort fails after the + * attempt it followed is already in the trail. On each of those the trail already accounts for every + * send, so adding one overstates it. Narrowing the catch does not rescue the sum — `abortToSdkError` + * yields `TransportFailureError` for a timeout signal, so even that class can reach you without a + * send behind it. + * + * Oldest first, and the error you passed in is never a member of its own trail (`RETRY-34`'s + * skip-self clause, which matters because a transport may reuse one error instance across + * attempts). A run that succeeded, a failure that was never retried, and any error this SDK did not + * surface from a retry loop all answer with an empty list — this never throws and never returns + * `undefined`. + * + * The result is frozen, and it is read by identity: an error reached through another error's + * `cause` has its own trail or none, never its wrapper's. + * + * @param error - the throwable a retrying pipeline surfaced; any value, not necessarily an `Error`. + * @returns the earlier attempts' errors, oldest first, or an empty list when there are none. + * + * @public + */ +export function retryAttempts(error: unknown): readonly unknown[] { + const key = trailKey(error); + if (key === undefined) return NO_ATTEMPTS; + return attemptTrails.get(key) ?? NO_ATTEMPTS; +} diff --git a/packages/core/src/retry/backoff.test.ts b/packages/core/src/retry/backoff.test.ts index 836ca61..ed872eb 100644 --- a/packages/core/src/retry/backoff.test.ts +++ b/packages/core/src/retry/backoff.test.ts @@ -2,7 +2,8 @@ // packages/core/src/retry/backoff.test.ts // Exercises: RETRY-9 (initialDelay * multiplier^(attempt-1), 1-indexed, capped), RETRY-10 (symmetric // jitter bounds, midpoint, j=0 identity, negative floors to zero), RETRY-11 (attempt < 1 rejected, -// overflow saturates), RETRY-43 (fixed delay disables backoff AND jitter). +// overflow saturates -- INCLUDING at a zero initial delay, where `0 * Infinity` used to give NaN; +// audit #67 / #78), RETRY-43 (fixed delay disables backoff AND jitter). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {computeDelay, type BackoffSettings} from './backoff.js'; @@ -41,6 +42,69 @@ describe('exponential schedule', () => { }); }); +/** + * RETRY-11's "saturating rather than throwing" has one hole, and it is the zero base. Every other + * accepted setting overflows into `Math.min`'s cap; `0 * Infinity` overflows into `NaN`, which + * `Math.min` propagates. Audit #67 / #78. + */ +describe('a zero initial delay (RETRY-11)', () => { + test('stays zero where the power overflows', () => { + // Downstream, NaN is worse than a large number: `overshootsBudget` reads false for it, the + // engine's `delayMs <= 0` guard reads false for it, and it lands in `Clock.sleep` as a + // RangeError that replaces the failure being retried. `retrySettings()` accepts both settings + // below (`initialDelayMs >= 0`, finite `multiplier >= 1`), so RETRY-11 covers them. + const hugeMultiplier: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + }; + expect(computeDelay(3, hugeMultiplier, never)).toBe(0); + + const manyAttempts: BackoffSettings = {...SETTINGS, initialDelayMs: 0}; + // 2 ** 1099 is Infinity: the first attempt at which the doubling schedule overflows a double. + expect(computeDelay(1100, manyAttempts, never)).toBe(0); + }); + + test('stays zero under jitter too (RETRY-10)', () => { + const jitteredZero: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + jitter: 1, + }; + expect(computeDelay(4, jitteredZero, () => 0)).toBe(0); + expect(computeDelay(4, jitteredZero, () => 1)).toBe(0); + }); + + test('property: every accepted schedule is finite and non-negative (RETRY-11)', () => { + // The ranges are exactly what `retrySettings()` admits, so a passing property means no + // configuration a caller can build reaches the engine as a non-finite delay. `initialDelayMs` + // is drawn through an explicit `constant(0)` arm: the failing region needs a zero base AND an + // overflowing power together, and 100 runs of an unbiased double never produced the pair. + const accepted = fc.record({ + initialDelayMs: fc.oneof( + fc.constant(0), + fc.double({min: 0, max: 1e9, noNaN: true}), + ), + multiplier: fc.double({min: 1, max: 1e300, noNaN: true}), + maxDelayMs: fc.double({min: 0, max: 1e9, noNaN: true}), + jitter: fc.double({min: 0, max: 1, noNaN: true}), + }); + + fc.assert( + fc.property( + fc.integer({min: 1, max: 5000}), + accepted, + (attempt, settings) => { + const delay = computeDelay(attempt, settings, never); + expect(Number.isFinite(delay)).toBe(true); + expect(delay).toBeGreaterThanOrEqual(0); + }, + ), + ); + }); +}); + describe('symmetric jitter', () => { const jittered: BackoffSettings = {...SETTINGS, jitter: 0.2}; diff --git a/packages/core/src/retry/backoff.ts b/packages/core/src/retry/backoff.ts index 63f8700..b505258 100644 --- a/packages/core/src/retry/backoff.ts +++ b/packages/core/src/retry/backoff.ts @@ -49,6 +49,17 @@ function applyJitter( * Overflow-safe by construction (RETRY-11): a large attempt makes `**` return `Infinity`, which * `Math.min` absorbs into the cap. It saturates; it never throws. * + * Except at a zero base, where the saturation does not hold and the guard below is what supplies it. + * `0 * Infinity` is `NaN`, and `Math.min` propagates `NaN` rather than clamping it -- so + * `initialDelayMs: 0` with any multiplier above 1 produced a `NaN` delay at the attempt where the + * power overflows (`multiplier: 2` reaches it at attempt 1100; `multiplier: 1e200` at attempt 3). + * `retrySettings()` accepts both configurations. Downstream, `NaN` is worse than a large number: it + * fails every comparison, so the engine's budget check, its overshoot check and its `delayMs <= 0` + * short-circuit all read false and it arrives at `Clock.sleep`, which rejects with a `RangeError` + * that replaces the failure being retried. Short-circuiting the zero base before the power is taken + * is exact rather than a repair: the schedule's value there is `0` at every attempt, and jitter + * around `0` is `0` for any sample (audit #67 / #78). + * * `random` is injected so jitter is assertable rather than statistical -- the same determinism seam * CFG-15 wants for the clock. * @@ -71,6 +82,8 @@ export function computeDelay( `retry attempt must be 1-indexed and >= 1, got ${String(attempt)}`, ); if (settings.fixedDelayMs !== undefined) return settings.fixedDelayMs; + // Before the power, not after: `0 * Infinity` is the one product `Math.min` cannot absorb. + if (settings.initialDelayMs === 0) return 0; const growth = settings.initialDelayMs * settings.multiplier ** (attempt - 1); return applyJitter( Math.min(growth, settings.maxDelayMs), diff --git a/packages/core/src/retry/classify.test.ts b/packages/core/src/retry/classify.test.ts index 521fd5a..03ee970 100644 --- a/packages/core/src/retry/classify.test.ts +++ b/packages/core/src/retry/classify.test.ts @@ -1,10 +1,12 @@ // SPDX-License-Identifier: MIT // packages/core/src/retry/classify.test.ts // Exercises: RETRY-1 (single-sourced status set, 501/505 excluded), RETRY-2 (iterative -// identity-tracking cause walk, cycle-safe), RETRY-3 (retryability derived from status, not a stored -// flag), RETRY-4 (transport failures always retryable), RETRY-5/6/7 (re-sendability), RETRY-8 (both -// axes required), RETRY-23/24 (cancellation vs timeout), RETRY-25 (allow-list makes the fatal -// exclusion vacuous), RETRY-37 (configured set is authoritative -- widens AND narrows), +// identity-tracking cause walk, cycle-safe; and the I/O boundary the walk tests -- one case per +// error class in `io/errors.ts`, see the block below), RETRY-3 (retryability derived from status, +// not a stored flag), RETRY-4 (transport failures always retryable), RETRY-5/6/7 (re-sendability), +// RETRY-8 (both axes required), RETRY-23/24 (cancellation vs timeout), RETRY-25 (allow-list makes +// the fatal exclusion vacuous), RETRY-37 (configured set is authoritative -- widens AND narrows), +// TRANSPORT-20 (a no-response send surfaces as a retryable I/O subtype), // XCUT-5 (the baked retryability flag comes from ONE shared status classifier covering 408/429/all // 5xx except 501 and 505 -- asserted below. This port has no separately-cached boolean field: the // classifier is a pure function of HttpStatusError.status, which never changes post-construction @@ -16,7 +18,15 @@ import {stringBody} from '../body/simple-bodies.js'; import {streamBody} from '../body/stream-body.js'; import type {Body} from '../body/body.js'; import {Request} from '../http/request.js'; -import {IoError} from '../io/errors.js'; +import { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, + TransportFailureError, +} from '../io/errors.js'; import {CancellationError} from '../seams/transport.js'; import { RETRYABLE_STATUSES, @@ -119,6 +129,75 @@ describe('isRetryableFailure', () => { }); }); +/** + * One case per class in `io/errors.ts`, pinning the boundary RETRY-2's "an I/O error" is read as. + * + * `isIoError` accepts all six classes the file declares; `classify.ts`'s walk tests + * `instanceof IoError`, which two of them satisfy. That gap was undecided until audit #67 / #78 + * decided it (`docs/deviations.md` item 17): the branch means "the wire failed", so `IoError` and + * `TransportFailureError` retry and the four flat leaves do not. Each leaf case asserts BOTH halves + * -- that `isIoError` accepts the value, and what the classifier answers for it -- because the two + * disagreeing is the decision, and a test that only asserted the classifier would read as an + * oversight rather than a choice. + * + * These are the guard on re-parenting: moving any leaf back under `IoError`, or switching the branch + * to `isIoError`, turns four of them red instead of quietly making a deterministic failure retryable. + * Measured 2026-09-05 by making that one-line change: exactly these four fail. + */ +describe('the I/O boundary the cause-walk tests (RETRY-2/RETRY-4, TRANSPORT-20)', () => { + test('IoError itself is retryable', () => { + const error = new IoError('connection refused'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('TransportFailureError is retryable (TRANSPORT-20)', () => { + // The one class TRANSPORT-20 requires to BE an IoError. A send that produced no response is the + // canonical retryable condition (RETRY-4), and the `extends` is what carries it here. + const error = new TransportFailureError('ECONNREFUSED'); + expect(error).toBeInstanceOf(IoError); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('EndOfStreamError is NOT retryable, buried in a cause chain either', () => { + // The exact-length-copy contract inside io/, not a wire truncation: a short copy repeats on the + // next attempt. A truncated response is the transport's to report, as TransportFailureError. + const error = new EndOfStreamError(3, 8); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + // Asserted through a wrapper too: the walk is what would rescue it if the branch widened, so the + // shallow case alone would not catch a change made one hop up. + expect( + isRetryableFailure( + new Error('read failed', {cause: error}), + RETRYABLE_STATUSES, + ), + ).toBe(false); + }); + + test('SourceContractViolationError is NOT retryable', () => { + // A foreign source that returned zero bytes for a positive read (IO-17) is a programming error + // in the source, deterministic on re-send. + const error = new SourceContractViolationError('source returned 0 bytes'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('ClosedResourceError is NOT retryable', () => { + // Using a closed resource (IO-42) is a caller lifecycle error; the resource stays closed. + const error = new ClosedResourceError('response body'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('AllocationLimitError is NOT retryable', () => { + // A cap the same request hits again (IO-9); retrying spends the budget to fail identically. + const error = new AllocationLimitError(2 ** 32, 2 ** 31 - 1); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); +}); + describe('isRetryableFailure -- cancellation, timeouts, and the allow-list', () => { test('a user abort is never retryable (RETRY-23)', () => { const controller = new AbortController(); diff --git a/packages/core/src/retry/classify.ts b/packages/core/src/retry/classify.ts index 7073395..f911e70 100644 --- a/packages/core/src/retry/classify.ts +++ b/packages/core/src/retry/classify.ts @@ -53,6 +53,22 @@ function causeOf(value: unknown): unknown { * surfaces as an `IoError` subclass and is therefore retryable unconditionally at this level * (RETRY-4). * + * **The I/O branch tests `instanceof IoError`, not `isIoError`, and the difference is the rule.** + * `io/errors.ts` groups six classes under `isIoError`, but only two of them descend from `IoError`: + * `IoError` itself, and `TransportFailureError` -- the class TRANSPORT-20 requires a send that + * produced no response to surface, and the reason that `extends` is a requirement rather than a + * modelling choice (`docs/deviations.md` item 17). Those two mean "the wire failed", and RETRY-2's + * "an I/O error" is read as exactly that boundary. The other four -- `EndOfStreamError`, + * `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` -- extend + * `DexpaceError` directly and are deliberately outside it: they are this package's own contract and + * lifecycle failures and are deterministic on re-send. A closed resource or a violated source + * contract is a caller programming error, an allocation cap is a limit the same request hits again, + * and `EndOfStreamError` is the exact-length-copy contract inside `io/` -- a *wire* truncation is the + * transport's to report, as a `TransportFailureError`. Widening this branch to `isIoError` would + * retry all four. Decided by audit #67 / #78; one case per class in `classify.test.ts` pins the + * answer, so a later re-parenting of any leaf under `IoError` changes a test rather than passing + * silently. + * * @param error - whatever was thrown; any value, not necessarily an `Error`. * @param statuses - the CONFIGURED set, authoritative on its own -- it both widens and narrows * relative to `RETRYABLE_STATUSES`, and the built-in classifier is not AND-ed in (RETRY-37). @@ -70,6 +86,7 @@ export function isRetryableFailure( seen.add(current); // RETRY-3: derived from the carried status at classification time, never a stored per-subclass flag. if (current instanceof HttpStatusError) return statuses.has(current.status); + // Deliberately `instanceof IoError`, never `isIoError` -- see the boundary paragraph above. if (current instanceof IoError) return true; if (isTimeoutAbort(current)) return true; current = causeOf(current); diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts index cbb1919..b1fdd20 100644 --- a/packages/core/src/retry/engine.test.ts +++ b/packages/core/src/retry/engine.test.ts @@ -3,10 +3,14 @@ // Exercises: RETRY-7/8 (both axes gate), RETRY-20 (a hint replaces the schedule, unjittered), RETRY-22 // (a pacing failure never masks the upstream failure), RETRY-26/31 (cancellable wait, zero delay // inline), RETRY-27/RECOV-20 (total-timeout budget with per-attempt shrinking), RETRY-32 (no attempts -// after cancellation), RETRY-34 (suppressed trail on failure, discarded on success, skip-self), +// after cancellation), RETRY-34 (the prior-attempt trail rides BESIDE the surfaced error, discarded +// on success, skip-self), XCUT-1 (the surfaced type does not depend on how many attempts ran -- the +// final attempt's own error is what the engine hands back, cancellation included), // RETRY-35/RECOV-16 (body released before the wait, bounded buffering), RETRY-36/RECOV-19 (503,503,200 // terminates on the 200; a surviving response is returned LIVE), RETRY-39/40 (delay precedence; a -// throwing override is non-fatal), RETRY-42/RECOV-28 (per-call state). +// throwing override is non-fatal -- and a non-finite RETURN from one is the same case, falling back +// to the schedule and logging through the same event; audit #67 / #78), RETRY-42/RECOV-28 (per-call +// state). import {describe, expect, test} from 'bun:test'; import {HttpStatusError} from '../body/http-status-error.js'; import type {Clock} from '../config/clock.js'; @@ -15,10 +19,12 @@ import {Protocol} from '../http/protocol.js'; import {Request} from '../http/request.js'; import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; -import {IoError} from '../io/errors.js'; +import {IoError, TransportFailureError} from '../io/errors.js'; import {failure, success, type Outcome} from '../recovery/outcome.js'; +import {CancellationError} from '../seams/transport.js'; import type {SuppressedErrorLike} from '../suppress.js'; import {countingResponse} from '../testing/fake-transport.js'; +import {retryAttempts} from './attempt-trail.js'; import {runWithRetry, type RetryConfig, type RetryDispatch} from './engine.js'; import {retrySettings, type RetrySettings} from './settings.js'; @@ -29,9 +35,13 @@ const BARE_POST = Request.newBuilder() .build(); /** - * The suppressed pair is asserted on SHAPE, never `instanceof SuppressedError`: the native class is - * absent on this package's Node floor (>=20.3), where `suppress()` returns a structural stand-in and - * an `instanceof` assertion would silently assert nothing. + * The one remaining pairing this engine can build is RECOV-12's -- a release failure riding along + * with the primary it must not mask. The retry TRAIL is no longer a `SuppressedError` chain, so the + * only assertions below that use this predicate are the release ones. + * + * Asserted on SHAPE, never `instanceof SuppressedError`: the native class is absent on this + * package's Node floor (>=20.3), where `suppress()` returns a structural stand-in and an + * `instanceof` assertion would silently assert nothing. */ function isSuppressedShape(value: unknown): value is SuppressedErrorLike { return ( @@ -87,6 +97,66 @@ function scriptedDispatch( return Object.assign(dispatch, {calls}); } +/** A clock that records every duration it is asked to sleep for, and returns immediately. */ +function recordingClock(slept: number[]): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => { + slept.push(durationMs); + return Promise.resolve(); + }, + }; +} + +/** + * `defaultClock`'s own precondition, modelled: `Clock.sleep` rejects a non-finite duration with a + * `RangeError` (`config/clock.ts:148-157`). A fake that slept for any duration at all would hide the + * bug; this guard is what surfaces it. + */ +function guardingClock(): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => + Number.isFinite(durationMs) + ? Promise.resolve() + : Promise.reject( + new RangeError( + `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, + ), + ), + }; +} + +/** One retryable failure, then a 200: the shortest script that drives exactly one delay decision. */ +function oneFailureThenSuccess(): RetryDispatch { + return scriptedDispatch([ + failure(new IoError('first')), + success(countingResponse(200).response), + ]); +} + +/** Installs a capturing global logger for the duration of `body` and returns what it emitted. */ +async function captureLogEvents( + body: () => Promise, +): Promise[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await body(); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + describe('eligibility (RETRY-7/8)', () => { test('a non-retryable failure is surfaced after exactly one attempt', async () => { const dispatch = scriptedDispatch([failure(new TypeError('bad'))]); @@ -220,6 +290,94 @@ describe('delay resolution (RETRY-39/40)', () => { expect(outcome.kind).toBe('success'); expect(dispatch.calls).toHaveLength(2); }); + + test('a finite override is honored unchanged, fractional and huge alike (RETRY-39)', async () => { + // The finiteness guard below screens `NaN` and the two infinities and nothing else. A fractional + // or very large delay is still a delay, and RETRY-39 gives the caller precedence over the + // schedule -- 5000 ms of `fixedDelayMs` here, which neither run waits. + const slept: number[] = []; + const clock = recordingClock(slept); + + for (const override of [0.5, Number.MAX_SAFE_INTEGER]) { + await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('reset'))]), + { + settings: retrySettings({maxAttempts: 2, fixedDelayMs: 5000}), + clock, + random: () => 0.5, + delayOverride: () => override, + }, + ); + } + + expect(slept).toEqual([0.5, Number.MAX_SAFE_INTEGER]); + }); +}); + +/** + * RETRY-40 makes a bad override non-fatal. A throw was handled; a non-finite RETURN was not, and it + * is the worse of the two, because `NaN` fails every comparison downstream instead of failing loudly + * at the override. Audit #67 / #78 reads the two as one case: drop the value, use the computed + * schedule, keep going. + */ +describe('a non-finite delayOverride is the throwing case (RETRY-40)', () => { + test('every non-finite value falls back to the computed schedule', async () => { + // Asserted on the delays the clock was ASKED for. Three sends alone would also pass against a + // fake that quietly slept for `NaN`, which is most of them. + for (const bad of [ + Number.NaN, + Number.POSITIVE_INFINITY, + Number.NEGATIVE_INFINITY, + ]) { + const slept: number[] = []; + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('connection refused')), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({ + maxAttempts: 3, + initialDelayMs: 200, + multiplier: 2, + jitter: 0, + }), + clock: recordingClock(slept), + random: () => 0.5, + delayOverride: () => bad, + }); + + expect(dispatch.calls).toHaveLength(3); + expect(slept).toEqual([200, 400]); + expect(outcome.kind).toBe('failure'); + } + }); + + test('it never reaches Clock.sleep as a duration, so the real failure survives', async () => { + // The reported symptom, against a clock that guards its input the way `defaultClock` does: the + // rejection was folded into the terminal failure by RETRY-33's catch-all, so `() => NaN` with + // `maxAttempts: 3` gave ONE send and surfaced a `RangeError` about `durationMs` in place of the + // transport failure being retried -- with the real error demoted to the trail. + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('first')), + failure(new TransportFailureError('second')), + failure(new TransportFailureError('third')), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + clock: guardingClock(), + random: () => 0.5, + delayOverride: () => Number.NaN, + }); + + expect(dispatch.calls).toHaveLength(3); + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect((outcome.error as Error).message).toBe('third'); + expect(retryAttempts(outcome.error)).toHaveLength(2); + }); }); describe('server pacing hints (RETRY-20/22)', () => { @@ -364,12 +522,11 @@ describe('cancellation (RETRY-26/32)', () => { }); }); -describe('suppressed trail (RETRY-34)', () => { - test('prior attempt failures ride along as suppressed on the surfaced error', async () => { - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - failure(new IoError('second')), - ]); +describe('the prior-attempt trail (RETRY-34)', () => { + test('the FINAL attempt error is surfaced, with the priors reachable beside it', async () => { + const first = new IoError('first'); + const second = new IoError('second'); + const dispatch = scriptedDispatch([failure(first), failure(second)]); const outcome = await runWithRetry( GET, @@ -379,12 +536,45 @@ describe('suppressed trail (RETRY-34)', () => { expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); + // Not a wrapper: the surfaced value IS attempt 2's own error, so a caller's `instanceof` reads + // the same here as it does after a single attempt. + expect(outcome.error).toBe(second); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); + + test('the surfaced TYPE does not depend on how many attempts ran (XCUT-1)', async () => { + const one = await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('refused'))]), + configOf({maxAttempts: 1, fixedDelayMs: 0}), + ); + // Distinct instances, because `scriptedDispatch` repeats its last entry and RETRY-34's + // skip-self guard would otherwise empty the trail this row needs to be non-empty. + const three = await runWithRetry( + GET, + scriptedDispatch([ + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(one.kind).toBe('failure'); + expect(three.kind).toBe('failure'); + if (one.kind !== 'failure' || three.kind !== 'failure') return; + // Until 2026-09-05 the second of these was a `SuppressedError` and this row was false: the + // surfaced CLASS was a function of the attempt budget, which is what XCUT-1's conformance + // clause ("assert the surfaced error is the cancellation type") catches on the abort path. + expect(one.error).toBeInstanceOf(TransportFailureError); + expect(three.error).toBeInstanceOf(TransportFailureError); + expect(retryAttempts(three.error)).toHaveLength(2); }); test('the trail is discarded entirely on eventual success', async () => { + const first = new IoError('first'); const dispatch = scriptedDispatch([ - failure(new IoError('first')), + failure(first), success(countingResponse(200).response), ]); @@ -395,10 +585,12 @@ describe('suppressed trail (RETRY-34)', () => { ); expect(outcome.kind).toBe('success'); + // Nothing was recorded anywhere: the discarded failure carries no trail of its own either. + expect(retryAttempts(first)).toEqual([]); }); }); -describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', () => { +describe('the trail -- skip-self and single-attempt shapes (RETRY-34)', () => { test('a reused instance never suppresses itself (RETRY-34 skip-self)', async () => { const reused = new IoError('same instance every time'); const dispatch = scriptedDispatch([failure(reused)]); @@ -410,6 +602,8 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( ); expect(outcome).toEqual(failure(reused)); + // Skip-self leaves nothing at all for a transport that reuses one instance across attempts. + expect(retryAttempts(reused)).toEqual([]); }); test('a single failed attempt surfaces its error unwrapped', async () => { @@ -419,6 +613,7 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( expect(await runWithRetry(GET, dispatch, configOf())).toEqual( failure(only), ); + expect(retryAttempts(only)).toEqual([]); }); test('a discarded 503 becomes a buffered HttpStatusError in the trail (RECOV-16)', async () => { @@ -438,10 +633,8 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(IoError); - expect(outcome.error.suppressed).toBeInstanceOf(HttpStatusError); + expect(outcome.error).toBeInstanceOf(IoError); + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); }); }); @@ -469,13 +662,10 @@ describe('a discarded response OUTSIDE the 400-599 band (V14/N2, XCUT-8)', () => expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.suppressed).toBeInstanceOf( - RetryDiscardedResponseError, - ); - expect(outcome.error.suppressed).not.toBeInstanceOf(HttpStatusError); - expect((outcome.error.suppressed as {status: number}).status).toBe(200); + const [discarded] = retryAttempts(outcome.error); + expect(discarded).toBeInstanceOf(RetryDiscardedResponseError); + expect(discarded).not.toBeInstanceOf(HttpStatusError); + expect((discarded as {status: number}).status).toBe(200); }); test('a discarded 404 still becomes an HttpStatusError -- the band is unchanged', async () => { @@ -496,9 +686,7 @@ describe('a discarded response OUTSIDE the 400-599 band (V14/N2, XCUT-8)', () => expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.suppressed).toBeInstanceOf(HttpStatusError); + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); }); }); @@ -599,9 +787,7 @@ describe('the inter-attempt wait -- degenerate and hostile delays', () => { // instead of silently becoming an extra attempt. expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(RangeError); + expect(outcome.error).toBeInstanceOf(RangeError); expect(dispatch.calls).toHaveLength(1); }); }); @@ -663,6 +849,90 @@ describe('cancellation while an attempt is in flight (RETRY-32)', () => { }); }); +describe('a cancelled backoff surfaces the cancellation TYPE (XCUT-1)', () => { + /** Aborts from inside attempt 1, so the loop reaches its RETRY-32 check with a one-entry trail. */ + function abortAfterOneAttempt( + controller: AbortController, + first: unknown, + ): RetryDispatch & {sends: number} { + const dispatch = (): Promise> => { + dispatch.sends += 1; + controller.abort(); + return Promise.resolve(failure(first)); + }; + dispatch.sends = 0; + return dispatch; + } + + test('the surfaced error is CancellationError, with the prior attempt beside it', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // `abortToSdkError` maps the abort to this type one line before the trail is attached; until + // 2026-09-05 the trail wrapper undid the mapping immediately, and a cancelled backoff ALWAYS + // has a non-empty trail -- so `instanceof CancellationError` was false for every one of them. + expect(outcome.error).toBeInstanceOf(CancellationError); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); + + test('the trail already covers every send, so length is NOT one less than the count', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // The surfaced error is SYNTHESIZED at the RETRY-32 gate, not raised by a send, so it is not an + // attempt's error and the trail already accounts for all of them. `length + 1` would say two + // sends where one happened -- which is why no TSDoc here offers that arithmetic. + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); +}); + +describe('the RETRY-32 gate can synthesize a TransportFailureError too', () => { + test('a TIMEOUT signal takes the same synthesized path, as TransportFailureError', async () => { + // `abortToSdkError` branches on `isTimeoutSignal`, so the engine's own RETRY-32 gate can + // synthesize a `TransportFailureError` too. Narrowing a catch to that class therefore does NOT + // guarantee the caught error came from a send. + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch: RetryDispatch & {sends: number} = Object.assign( + (): Promise> => { + dispatch.sends += 1; + controller.abort(new DOMException('timed out', 'TimeoutError')); + return Promise.resolve(failure(first)); + }, + {sends: 0}, + ); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect(outcome.error).not.toBeInstanceOf(CancellationError); + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); +}); + describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { test('a throw from inside the attempt is folded into a failure outcome, trail intact', async () => { const calls: number[] = []; @@ -681,16 +951,14 @@ describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { expect(calls).toEqual([1, 2]); expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(RangeError); + expect(outcome.error).toBeInstanceOf(RangeError); // RETRY-34: attempt 1's failure would have been lost had the throw escaped as a rejection. - expect((outcome.error.suppressed as Error).message).toBe('first'); + expect((retryAttempts(outcome.error)[0] as Error).message).toBe('first'); }); }); -describe('the suppressed trail with more than two entries (RETRY-34)', () => { - test('three distinct attempt failures fold into a nested chain', async () => { +describe('the trail with more than two entries (RETRY-34)', () => { + test('three distinct attempt failures list flat, oldest first', async () => { const dispatch = scriptedDispatch([ failure(new IoError('first')), failure(new IoError('second')), @@ -705,15 +973,12 @@ describe('the suppressed trail with more than two entries (RETRY-34)', () => { expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect((outcome.error.error as Error).message).toBe('third'); - // The two priors folded into a nested pair, oldest innermost. - const folded = outcome.error.suppressed; - expect(isSuppressedShape(folded)).toBe(true); - if (!isSuppressedShape(folded)) return; - expect((folded.error as Error).message).toBe('second'); - expect((folded.suppressed as Error).message).toBe('first'); + expect((outcome.error as Error).message).toBe('third'); + // A flat list in wire order. The `SuppressedError` pair is binary, so N priors used to fold + // into a nested chain a caller had to walk; nothing about RETRY-34 asked for that shape. + expect( + retryAttempts(outcome.error).map(entry => (entry as Error).message), + ).toEqual(['first', 'second']); }); }); @@ -876,70 +1141,66 @@ describe('per-call state (RETRY-42, RECOV-28)', () => { describe('Phase 7b retrofit: structured retry logging', () => { test('emits attemptFailed per retry and exhausted when attempts run out', async () => { - const {createLogger, setGlobalLogger, NOOP_LOGGER} = - await import('../observability/logger.js'); - const events: Map[] = []; - const testLogger = createLogger((_level, fields) => { - events.push(new Map(fields)); + const events = await captureLogEvents(async () => { + await runWithRetry( + GET, + scriptedDispatch([ + failure(new IoError('first')), + failure(new IoError('second')), + failure(new IoError('third')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); }); - setGlobalLogger(testLogger); - - try { - const config = configOf({maxAttempts: 3, fixedDelayMs: 0}); - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - failure(new IoError('second')), - failure(new IoError('third')), - ]); - - await runWithRetry(GET, dispatch, config); - const failedEvents = events.filter( - e => e.get('event') === 'http.retry.attemptFailed', - ); - expect(failedEvents).toHaveLength(2); - expect(failedEvents[0]?.get('attempt')).toBe(1); - expect(failedEvents[1]?.get('attempt')).toBe(2); + const failedEvents = events.filter( + e => e.get('event') === 'http.retry.attemptFailed', + ); + expect(failedEvents).toHaveLength(2); + expect(failedEvents[0]?.get('attempt')).toBe(1); + expect(failedEvents[1]?.get('attempt')).toBe(2); - const exhaustedEvents = events.filter( - e => e.get('event') === 'http.retry.exhausted', - ); - expect(exhaustedEvents).toHaveLength(1); - expect(exhaustedEvents[0]?.get('attempts')).toBe(3); - } finally { - setGlobalLogger(NOOP_LOGGER); - } + const exhaustedEvents = events.filter( + e => e.get('event') === 'http.retry.exhausted', + ); + expect(exhaustedEvents).toHaveLength(1); + expect(exhaustedEvents[0]?.get('attempts')).toBe(3); }); test('emits delayOverrideFailed when delayOverride throws', async () => { - const {createLogger, setGlobalLogger, NOOP_LOGGER} = - await import('../observability/logger.js'); - const events: Map[] = []; - const testLogger = createLogger((_level, fields) => { - events.push(new Map(fields)); - }); - setGlobalLogger(testLogger); - - try { - const config: RetryConfig = { + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { ...configOf({maxAttempts: 2, fixedDelayMs: 0}), delayOverride: () => { throw new Error('bad override'); }, - }; - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - success(countingResponse(200).response), - ]); + }); + }); + + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe('Error: bad override'); + }); - await runWithRetry(GET, dispatch, config); + test('emits delayOverrideFailed when delayOverride returns a non-finite delay', async () => { + // "Treated exactly like one that throws" (RETRY-40) is a claim about the diagnostic too: an + // override dropped in silence is a schedule the operator cannot explain from the configuration. + // Same event, same level, same emit path -- only the cause differs, and it names the value. + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { + ...configOf({maxAttempts: 2, fixedDelayMs: 0}), + delayOverride: () => Number.NaN, + }); + }); - const overrideFailed = events.filter( - e => e.get('event') === 'http.retry.delayOverrideFailed', - ); - expect(overrideFailed).toHaveLength(1); - } finally { - setGlobalLogger(NOOP_LOGGER); - } + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe( + 'delayOverride returned a non-finite delay: NaN', + ); }); }); diff --git a/packages/core/src/retry/engine.ts b/packages/core/src/retry/engine.ts index 0403d76..056a6f3 100644 --- a/packages/core/src/retry/engine.ts +++ b/packages/core/src/retry/engine.ts @@ -8,7 +8,7 @@ import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import {failure, type Outcome} from '../recovery/outcome.js'; import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; -import {suppress} from '../suppress.js'; +import {recordAttempts} from './attempt-trail.js'; import {stampAttempt} from './attempt-stamp.js'; import {computeDelay} from './backoff.js'; import {RetryDiscardedResponseError} from './errors.js'; @@ -48,7 +48,7 @@ export interface RetryConfig { readonly clock: Clock; /** Injectable randomness -- jitter and the X-RateLimit-Reset spread both draw from it. */ readonly random: () => number; - /** Highest-precedence delay source (RETRY-39). A throw is non-fatal (RETRY-40). */ + /** Highest-precedence delay source (RETRY-39). A throw, or a non-finite result, is non-fatal (RETRY-40). */ readonly delayOverride?: ((attempt: number) => number | undefined) | undefined; } @@ -102,25 +102,61 @@ function overshootsBudget(delayMs: number, state: LoopState): boolean { } /** - * RETRY-40: a throwing user override is ignored, never fatal. Emits http.retry.delayOverrideFailed at warning level. + * RETRY-40's diagnostic half, shared by both ways an override can fail. An ignored override is a + * schedule the operator cannot explain from the configuration alone, so neither way is silent. + */ +function reportOverrideFailure(cause: unknown): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('http.retry.delayOverrideFailed') + .cause(cause) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request or retry loop + } +} + +/** + * RETRY-40: a misbehaving user override is ignored, never fatal. Emits + * http.retry.delayOverrideFailed at warning level. + * + * TWO ways to misbehave, one answer. A throw was always handled here. A non-finite RETURN was not, + * and it was the more damaging of the two: `NaN` and the infinities pass every guard downstream -- + * `overshootsBudget` and `budgetExhausted` compare false, {@link waitFor}'s `delayMs <= 0` + * short-circuit compares false -- and arrive at `Clock.sleep`, which rejects a non-finite duration + * with a `RangeError`. RETRY-33's catch-all then folds that rejection into the terminal failure, so + * a `delayOverride` returning `NaN` under `maxAttempts: 3` produced ONE send and surfaced a + * `RangeError` about `durationMs`, with the transport failure it was retrying demoted to the trail. + * Audit #67 / #78 reads the two as one case: drop the value, use the computed schedule, keep going. + * + * The screen is finiteness alone. A finite negative keeps its existing behaviour -- {@link waitFor} + * continues inline without a timer (RETRY-31), which is the same answer the budget clamp already + * produces -- and a fractional or very large delay is a delay RETRY-39 gives the caller precedence + * for. + * + * The check sits OUTSIDE the `try` on purpose: a logger that throws while reporting a non-finite + * result must not be re-reported as an override that threw. */ function callerOverride(state: LoopState): number | undefined { const {delayOverride} = state.config; if (delayOverride === undefined) return undefined; + let delayMs: number | undefined; try { - return delayOverride(state.attempt); + delayMs = delayOverride(state.attempt); } catch (error) { - try { - getGlobalLogger() - .atLevel('warning') - .event('http.retry.delayOverrideFailed') - .cause(error) - .emit(); - } catch { - // OBS-20: logger failure must never fail the request or retry loop - } + reportOverrideFailure(error); + return undefined; + } + if (delayMs !== undefined && !Number.isFinite(delayMs)) { + // A string cause, not a synthesized Error: nothing threw, and the value that was rejected is + // the whole diagnostic. + reportOverrideFailure( + `delayOverride returned a non-finite delay: ${String(delayMs)}`, + ); return undefined; } + return delayMs; } /** RETRY-39: caller override -> server pacing hint -> fixed delay -> exponential backoff. */ @@ -247,27 +283,35 @@ async function decideRetry( } /** - * RETRY-34: prior failures ride along as `suppressed` on the surfaced error; the surfaced instance - * itself is skipped, so a reused throwable cannot suppress itself. On success the trail is discarded - * whole. + * RETRY-34: prior failures ride ALONGSIDE the surfaced error, recorded in `attempt-trail.ts`'s side + * table and read back through the public `retryAttempts()`. The surfaced instance itself is skipped, + * so a reused throwable never appears in its own trail. On success the trail is discarded whole -- + * nothing is written, and the outcome is returned untouched. + * + * **The outcome's error is returned unchanged, class and identity intact.** Until 2026-09-05 this + * function wrapped it in a `SuppressedError` pair instead, which made the surfaced TYPE a function of + * how many attempts ran: one attempt surfaced `TransportFailureError`, three surfaced a wrapper with + * the `TransportFailureError` at `.error`. XCUT-1's conformance clause -- "assert the surfaced error + * is the cancellation type" -- is the row that catches it, because a cancellation during backoff + * ALWAYS has a non-empty trail: `abortToSdkError` maps the abort to `CancellationError` below, and + * the wrapper undid that mapping on the very next line. RETRY-34 asks for the prior failures to be + * "attached to the surfaced exception", which is the JVM's `addSuppressed` -- the exception stays + * what it is and grows a list -- not for the exception to be replaced by a container. * - * The suppressed pair is a binary shape, so N entries fold into a nested chain. Built through Phase - * 4b's `suppress()` helper rather than `new SuppressedError(...)`: the native class reached Node only - * in 24.0.0 and this package's floor is `>=20.3`, so the direct form neither type-checks nor runs - * there. Argument order is controlled explicitly -- native `using` disposal builds the pair the other - * way round, making the *later* error primary. + * `suppress()` keeps its RECOV-12 job elsewhere in this file: `withReleaseFailure` pairs a release + * failure with the primary it must not mask. That is a genuine two-value pairing; an N-entry attempt + * history folded into a binary shape was never one. */ -function withTrail( +function attachTrail( outcome: Outcome, trail: readonly unknown[], ): Outcome { if (outcome.kind === 'success') return outcome; - const prior = trail.filter(entry => entry !== outcome.error); - if (prior.length === 0) return outcome; - const folded = prior.reduce((accumulated, entry) => - suppress(entry, accumulated, 'earlier retry attempt failed'), + recordAttempts( + outcome.error, + trail.filter(entry => entry !== outcome.error), ); - return failure(suppress(outcome.error, folded, 'retry attempts exhausted')); + return outcome; } /** @@ -284,7 +328,9 @@ function withTrail( * A non-positive delay short-circuits before `sleep` is reached: it continues inline with no timer * (RETRY-31), which is reachable after RETRY-17's past-instant hint and after the budget clamp, and * it is also what keeps a caller `delayOverride` returning a negative number out of `sleep`'s - * negative-duration rejection (RETRY-40 makes a bad override non-fatal). + * negative-duration rejection (RETRY-40 makes a bad override non-fatal). It does NOT catch a + * non-finite one -- `NaN <= 0` is false -- which is why {@link callerOverride} screens those at the + * source rather than here. * * Cancellation RESOLVES here rather than propagating: RETRY-26 wants the loop's next iteration to * observe the signal and stop through its own RETRY-32 path, so the abort rejection is the one @@ -346,12 +392,16 @@ function maybeEmitExhausted( * iterative, so N retries build no continuation chain and no stack growth. RETRY-33's "every * terminal path returns an Outcome" is honored literally -- an attempt that throws is folded into a * failure outcome carrying the trail, rather than left to surface as a bare rejected promise that - * would drop RETRY-34's suppressed attempts on the floor. + * would drop RETRY-34's prior attempts on the floor. * - * @param request - the captured template every attempt re-sends. + * @param request - the captured template every attempt re-sends. Whatever the caller captured is + * already final: the pillar adapter passes the request arriving at the RETRY stage, and the + * recovery adapter passes the output of a request chain it applied ONCE, above this loop + * (RECOV-32 -- one idempotency key per logical request). This loop only ever copies it. * @param dispatch - performs one attempt and reports its outcome without throwing. * @param config - settings, clock, randomness, signal, and the optional delay override. - * @returns the terminal outcome, with RETRY-34's suppressed trail attached on failure. + * @returns the terminal outcome. On failure the error is the FINAL attempt's own, unwrapped, with + * RETRY-34's prior-attempt trail recorded beside it for `retryAttempts()`. * * @internal */ @@ -383,7 +433,7 @@ export async function runWithRetry( // and silently missed a cancelled backoff. The raw reason is kept as `.cause`. if (config.signal?.aborted === true) { const cancellation = abortToSdkError(config.signal, config.signal.reason); - return withTrail(failure(cancellation), trail); + return attachTrail(failure(cancellation), trail); } try { @@ -396,7 +446,7 @@ export async function runWithRetry( const decision = await runAttempt(dispatch, state); if (decision.kind === 'stop') { maybeEmitExhausted(decision.outcome, trail.length, state); - return withTrail(decision.outcome, trail); + return attachTrail(decision.outcome, trail); } trail.push(decision.error); @@ -418,7 +468,7 @@ export async function runWithRetry( // `stampAttempt`'s header build, `toHttpError`'s body drain, and a misbehaving injected // clock's `sleep` -- and letting any of them escape would discard the whole suppressed trail // RETRY-34 requires the surfaced failure to carry. - return withTrail(failure(error), trail); + return attachTrail(failure(error), trail); } } } diff --git a/packages/core/src/retry/errors.ts b/packages/core/src/retry/errors.ts index ed0ba07..bfaeccc 100644 --- a/packages/core/src/retry/errors.ts +++ b/packages/core/src/retry/errors.ts @@ -7,8 +7,9 @@ import {DexpaceError} from '../http/errors.js'; * * Reachable only when a caller widens `RetrySettings.retryableStatuses` to include a non-error code: * the engine then retries a 2xx or 3xx, and every response it discards still owes `RETRY-34` an - * entry in the suppressed trail. `toHttpError` correctly returns `null` for such a status - * (`BODY-31` hands a non-error response back intact), so there is nothing for it to build. + * entry in the prior-attempt trail `retryAttempts()` reads. `toHttpError` correctly returns `null` + * for such a status (`BODY-31` hands a non-error response back intact), so there is nothing for it + * to build. * * Until 2026-09-02 the engine fabricated `new HttpStatusError(200, …)` here — precisely the * "successful exception" `XCUT-8` forbids, constructed by core itself, and contradicting diff --git a/packages/core/src/retry/retry-dispatch.test.ts b/packages/core/src/retry/retry-dispatch.test.ts index 4d82b0b..de83837 100644 --- a/packages/core/src/retry/retry-dispatch.test.ts +++ b/packages/core/src/retry/retry-dispatch.test.ts @@ -1,42 +1,101 @@ // SPDX-License-Identifier: MIT // packages/core/src/retry/retry-dispatch.test.ts -// Exercises: RECOV-17..20 (the recovery stack's retry lands here), RETRY-44 (each attempt re-runs the -// WHOLE recovery chain -- request chain, transport, response chain), RETRY-13/14/RECOV-30 (both entry -// points share one engine, so the schedule cannot drift). +// Exercises: RECOV-17..20 (the recovery stack's retry lands here), RECOV-32 (one idempotency key per +// LOGICAL request -- the strategy runs once and every wire send carries its result), RETRY-38 (the +// per-attempt ordinal is stamped on a fresh copy and preserves that key), RETRY-44 (fresh per-attempt +// state below the retry point; upstream steps do not run between attempts), RECOV-2 (a request-chain +// throw still meets the response and recovery hooks), RETRY-13/14/RECOV-30 (both entry points share +// one engine, so the schedule cannot drift). import {describe, expect, test} from 'bun:test'; +import {stringBody} from '../body/simple-bodies.js'; import type {Clock} from '../config/clock.js'; import {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; import {IoError} from '../io/errors.js'; +import {idempotencyKeyStep} from '../recovery/idempotency-key.js'; +import {failure, success, type Outcome} from '../recovery/outcome.js'; import {RequestRecoveryChain} from '../recovery/request-chain.js'; -import {ResponseRecoveryChain} from '../recovery/response-chain.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, +} from '../recovery/response-chain.js'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {dispatchWithRetry, type RetryDispatchConfig} from './retry-dispatch.js'; -import {retrySettings} from './settings.js'; +import {retrySettings, type RetrySettings} from './settings.js'; const GET = Request.newBuilder().url('https://example.com').build(); +/** A POST the re-send gate lets through: RETRY-5 wants a body, and a replayable one. */ +function replayablePost(): Request { + return Request.newBuilder() + .method('POST') + .url('https://example.com') + .body(stringBody('payload')) + .build(); +} + const zeroClock: Clock = { now: () => 0, monotonic: () => 0, sleep: () => Promise.resolve(), }; +/** The third parameter is an object because `max-params` errors at four. */ +interface ConfigExtras { + readonly settings?: Partial; + readonly recoverySteps?: readonly RecoveryStep[]; +} + function configOf( transport: FakeTransport, requestSteps = new RequestRecoveryChain([]), + extras: ConfigExtras = {}, ): RetryDispatchConfig { return { transport, requestChain: requestSteps, - responseChain: new ResponseRecoveryChain([], []), + responseChain: new ResponseRecoveryChain([], extras.recoverySteps ?? []), retry: { - settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + settings: retrySettings({ + maxAttempts: 3, + fixedDelayMs: 0, + ...extras.settings, + }), clock: zeroClock, random: () => 0.5, }, }; } +/** + * A one-step request chain whose key strategy is COUNTED and whose keys are DISTINCT. Both matter: + * a strategy returning one constant value would pass every assertion below even if it were called + * once per attempt, which is the bug these cases exist to catch. + */ +function countingKeyChain(): { + chain: RequestRecoveryChain; + generated: () => number; +} { + let generated = 0; + const chain = new RequestRecoveryChain([ + idempotencyKeyStep({ + generate: () => { + generated += 1; + return `key-${String(generated)}`; + }, + }), + ]); + return {chain, generated: () => generated}; +} + +/** The named header seen by each wire send, in order. */ +function headerSent( + transport: FakeTransport, + name = 'Idempotency-Key', +): (string | undefined)[] { + return transport.calls.map(call => call.request.headers.get(name)); +} + /** * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this * runner's type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper @@ -64,7 +123,7 @@ describe('dispatchWithRetry', () => { expect(transport.sendCount).toBe(2); }); - test('re-runs the request recovery chain on every attempt (RETRY-44)', async () => { + test('applies the request recovery chain ONCE per logical request, not per attempt (RETRY-44)', async () => { let applications = 0; const chain = new RequestRecoveryChain([ request => { @@ -79,7 +138,8 @@ describe('dispatchWithRetry', () => { await dispatchWithRetry(GET, configOf(transport, chain)); - expect(applications).toBe(2); + expect(transport.sendCount).toBe(2); + expect(applications).toBe(1); }); test('rethrows the terminal failure unchanged in shape', async () => { @@ -104,3 +164,97 @@ describe('dispatchWithRetry', () => { expect(transport.sendCount).toBe(1); }); }); + +describe('dispatchWithRetry and the idempotency key (RECOV-32)', () => { + test('generates ONE key for three attempts and sends it on all three', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + const response = await dispatchWithRetry( + replayablePost(), + configOf(transport, chain), + ); + + expect(response.status.code).toBe(200); + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the run that exhausts the budget sends the same key on every attempt too', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([new IoError('reset')]); + + await rejectionOf( + dispatchWithRetry(replayablePost(), configOf(transport, chain)), + ); + + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the attempt ordinal varies per send while the key does not (RETRY-38)', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + await dispatchWithRetry( + replayablePost(), + configOf(transport, chain, {settings: {attemptHeaderName: 'X-Attempt'}}), + ); + + // The ordinal is the ENGINE's, written per attempt on `stampAttempt`'s fresh copy; the key is + // the request chain's, written once above the loop. Both survive on every send. + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + expect(headerSent(transport, 'X-Attempt')).toEqual(['1', '2', '3']); + }); +}); + +describe('dispatchWithRetry and a failing request chain (RECOV-2)', () => { + const boom = new IoError('request step failed'); + const throwingChain = (): RequestRecoveryChain => + new RequestRecoveryChain([() => Promise.reject(boom)]); + + test('does not retry it, never reaches the transport, and runs the recovery phase once', async () => { + const seen: Outcome[] = []; + const recovery: RecoveryStep = outcome => { + seen.push(outcome); + return Promise.resolve(outcome); + }; + const transport = new FakeTransport([countingResponse(200).response]); + + const thrown = await rejectionOf( + dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ), + ); + + expect(thrown).toBe(boom); + expect(transport.sendCount).toBe(0); + expect(seen).toEqual([failure(boom)]); + }); + + test('a recovery step may still convert that failure into a success', async () => { + const substitute = countingResponse(204).response; + const recovery: RecoveryStep = () => Promise.resolve(success(substitute)); + const transport = new FakeTransport([countingResponse(200).response]); + + const response = await dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ); + + expect(response).toBe(substitute); + expect(transport.sendCount).toBe(0); + }); +}); diff --git a/packages/core/src/retry/retry-dispatch.ts b/packages/core/src/retry/retry-dispatch.ts index b87b29e..f641b44 100644 --- a/packages/core/src/retry/retry-dispatch.ts +++ b/packages/core/src/retry/retry-dispatch.ts @@ -3,7 +3,8 @@ import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import { - dispatchWithRecovery, + dispatchPrepared, + prepareRequest, type DispatchConfig, } from '../recovery/orchestrator.js'; import {failure, fold, success} from '../recovery/outcome.js'; @@ -18,10 +19,15 @@ export interface RetryDispatchConfig extends DispatchConfig { readonly retry: RetryConfig; } +/** + * One attempt: the transport hop and the response chain over the ALREADY-prepared request the + * engine handed back from `stampAttempt`. The request chain is deliberately not in here — see + * {@link dispatchWithRetry}. + */ function attemptVia(config: RetryDispatchConfig): RetryDispatch { return async request => { try { - return success(await dispatchWithRecovery(request, config)); + return success(await dispatchPrepared(success(request), config)); } catch (error) { return failure(error); } @@ -32,9 +38,31 @@ function attemptVia(config: RetryDispatchConfig): RetryDispatch { * The recovery-chain entry point for retry (RECOV-17..RECOV-20). * * NOT a `RecoveryStep` -- a recovery step receives an outcome and has no way to re-dispatch. This - * wraps 4b's orchestrator instead, mirroring its `(request, config)` shape, so each attempt re-runs - * the ENTIRE recovery chain: request chain, transport, response chain. That is the recovery-side - * mirror of what `ctx.fork()` does for the pillar step (RETRY-44). + * composes 4b's orchestrator halves instead, mirroring its `(request, config)` shape. + * + * **The request chain runs ONCE, above the loop; each attempt repeats only what is below it** -- + * the transport hop and the response chain, over `stampAttempt`'s fresh copy of the one prepared + * request. That is the layering `recovery/idempotency-key.ts` documents and RECOV-32 needs: its + * `generate()` is invoked once per logical request, so all N attempts reach the server under one + * key, and RETRY-38's per-attempt ordinal is written on the copy without disturbing it. Until + * 2026-09-05 this function re-ran the whole recovery chain per attempt and three attempts produced + * three different keys, defeating the header's entire purpose (audit #67, issue #73). + * + * RETRY-44 is satisfied, not traded away. Its "downstream chain" is whatever sits below the retry + * point, which here is transport plus response chain, and that is re-executed with fresh + * per-attempt state every time. Its second clause -- upstream steps MUST NOT mutate the shared + * in-flight request between attempts -- holds by construction now, because upstream steps no longer + * run between attempts at all. + * + * A request-chain failure is NOT retried: it never reached the wire, so RETRY-5's re-send gate has + * nothing to judge and re-running the step that just threw would only throw again. It still passes + * through the response and recovery chains exactly once, so RECOV-2's "no throwable bypasses the + * recovery hooks" and RECOV-10's unwrap are unchanged. + * + * One consequence worth naming: the engine's re-send gate (RETRY-5/RECOV-18, `isResendable`) now + * reads the PREPARED request rather than the caller's, so a request step that swaps in a + * non-replayable body makes the call non-retryable -- which is the honest answer, since the + * prepared request is what a retry would have to re-send. * * Shares `runWithRetry` with the pillar adapter, which is what makes RETRY-13/RETRY-14/RECOV-30's * "the two stacks must not drift" structural rather than a discipline. @@ -42,7 +70,9 @@ function attemptVia(config: RetryDispatchConfig): RetryDispatch { * @param request - the request to prepare, send, and possibly re-send. * @param config - the recovery chains, transport, and retry policy. * @returns the response of the terminal successful attempt. - * @throws Whatever the terminal Failure carries, with RETRY-34's suppressed trail attached. + * @throws Whatever the FINAL attempt failed with, unwrapped -- the same class a single-attempt run + * would have thrown. RETRY-34's earlier attempts are recorded beside it and read back through + * `retryAttempts()`. A request-chain throwable surfaces the same way, with no trail. * * @internal */ @@ -50,7 +80,13 @@ export async function dispatchWithRetry( request: Request, config: RetryDispatchConfig, ): Promise { - const outcome = await runWithRetry(request, attemptVia(config), config.retry); + const prepared = await prepareRequest(request, config.requestChain); + if (prepared.kind === 'failure') return dispatchPrepared(prepared, config); + const outcome = await runWithRetry( + prepared.value, + attemptVia(config), + config.retry, + ); return fold( outcome, response => response, diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts index 09b5534..2680959 100644 --- a/packages/core/src/retry/retry-step.ts +++ b/packages/core/src/retry/retry-step.ts @@ -44,6 +44,11 @@ export interface RetryStepOptions { * RETRY-39's caller override: returns the delay in milliseconds to use for `attempt`, or * `undefined` to fall through to the configured schedule for that attempt. * + * A throw, or a non-finite result, is ignored: the configured schedule is used for that attempt + * and `http.retry.delayOverrideFailed` is logged at warning level (RETRY-40). Neither aborts the + * retry loop. A finite negative is honored as a delay and continues inline without a timer + * (RETRY-31). + * * @defaultValue absent, so every attempt uses the configured schedule */ readonly delayOverride?: @@ -65,12 +70,17 @@ function attemptVia(fork: () => Next): RetryDispatch { * RETRY-41/HTTP-35: the per-call `RequestOptions.maxRetries` override wins over the configured budget * when present. The option counts retries; `maxAttempts` counts total sends, hence the `+ 1`. * - * The value IS revalidated here. `RequestOptionsBuilder.maxRetries` rejects only a negative value, - * which is strictly weaker than the `Number.isFinite(...) && >= 1` guard `retrySettings()` applies - * to the configured budget -- it admits `Infinity`, `NaN`, and fractions. Any of the first two - * reaching `maxAttempts` makes the engine's `attempt >= maxAttempts` gate permanently false and the - * retry loop unbounded, so the per-call route must not be the one path into the engine that skips - * the check the configured route enforces. + * The value IS revalidated here, and the two guards now agree. `RequestOptionsBuilder.maxRetries` + * rejects a negative, fractional or non-finite value at the call site that supplied it + * (`../http/request-options.ts:212-219`, pinned by `request-options.test.ts`'s + * `maxRetries validation (HTTP-35)` block), so this `invariant` is the engine asserting its own + * precondition rather than the only thing enforcing it -- it should be unreachable, and tripping it + * means the builder's guard was weakened. That was not true when this comment was first written: the + * builder then rejected only a negative value, which was strictly weaker than the + * `Number.isFinite(...) && >= 1` guard `retrySettings()` applies to the configured budget. The + * assertion stays either way, because `Infinity` or `NaN` reaching `maxAttempts` makes the engine's + * `attempt >= maxAttempts` gate permanently false and the retry loop unbounded, and the per-call + * route must not be the one path into the engine that skips the check the configured route enforces. * * The derived object is frozen: a spread of a frozen source is NOT itself frozen, and RETRY-42 * requires every policy component to be immutable after construction, not merely typed `readonly`. @@ -108,6 +118,15 @@ function configFrom( * `ctx.fork` is asserted rather than checked -- RETRY is in `PILLAR_STAGES`, so its absence means the * descriptor was installed somewhere it cannot be, which is a programmer error. * + * **What it throws when it gives up is the FINAL attempt's own error, unwrapped.** The class you + * catch does not depend on how many attempts ran: a transport failure surfaces as + * `TransportFailureError` whether `maxAttempts` was 1 or 3, and an abort that ended a backoff wait + * surfaces as `CancellationError` (`XCUT-1`). The earlier attempts' errors are not lost -- read them + * with `retryAttempts(caught)`, oldest first: one entry per attempt that failed BEFORE the error you + * caught, which is not the same as an attempt count (`RETRY-34`, and see `retryAttempts` for why the + * difference bites). A response the loop discards is always closed first; the response that ENDS the + * loop is returned live and unread, and closing it is yours. + * * @param options - settings overrides and the injected clock, randomness, and delay override. * @returns the descriptor to install in a pipeline's RETRY slot. * diff --git a/packages/core/src/seams/operation.test.ts b/packages/core/src/seams/operation.test.ts index 0217b72..5e588a9 100644 --- a/packages/core/src/seams/operation.test.ts +++ b/packages/core/src/seams/operation.test.ts @@ -1,7 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/operation.test.ts // Exercises: SEAM-26 (the four projections default to empty), SEAM-27 (buildRequest's encoding and base-URL -// composition rules), reusing HTTP-29's encodeRfc3986Component for path-segment encoding. +// composition rules, and that a placeholder is satisfied only by an OWN property of pathParams), +// HTTP-7 (a body projected onto a body-forbidding method fails assembly), reusing +// HTTP-29's encodeRfc3986Component for path-segment encoding — including the unpaired-surrogate input it +// cannot encode, which is rejected here rather than allowed to escape as a bare URIError. import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -9,7 +12,10 @@ import { OperationAssemblyError, type OperationDescriptor, } from './operation.js'; -import {UrlConstructionError} from '../http/errors.js'; +import { + RequestBodyNotAllowedError, + UrlConstructionError, +} from '../http/errors.js'; import {QueryParams} from '../http/query-params.js'; import {Headers} from '../http/headers.js'; @@ -113,6 +119,20 @@ describe('operation headers and body projections are threaded through', () => { expect(request.headers.get('x-trace')).toBe('abc'); expect(request.body).toBe(body); }); + + // HTTP-7: assembly ends at `Request.Builder.build()`, so the builder's method/body legality check is + // buildRequest's. Pinned because the throw reaches a caller through `buildRequest` and was absent from + // its `@throws` list until audit #67 / #68 added it -- an undocumented, unpinned throw path is exactly + // the kind that a later refactor swallows. + test('a body on a body-forbidding method throws RequestBodyNotAllowedError', () => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/pets', + body: stringBody('Fido'), + }), + ).toThrow(RequestBodyNotAllowedError); + }); }); describe('SEAM-27: dot-segment path-param values are rejected, not silently normalized away', () => { @@ -140,6 +160,136 @@ describe('SEAM-27: dot-segment path-param values are rejected, not silently norm }); }); +describe('SEAM-27: a placeholder is satisfied only by an OWN property of pathParams', () => { + // `pathParams?.[name]` reached the whole prototype chain, so `{constructor}` against `{}` resolved to + // `Object`'s own constructor, stringified, and shipped + // `/users/function%20Object%28%29%20%7B%20%5Bnative%20code%5D%20%7D` instead of failing assembly. Every + // placeholder MUST have a *supplied* value (SEAM-27); a name the caller never supplied is a missing value + // whatever `Object.prototype` happens to carry. Measured on the pre-fix tree, audit #67 / #76. + test.each([ + 'constructor', + 'toString', + 'hasOwnProperty', + 'valueOf', + '__proto__', + ])( + 'a {%s} placeholder against empty pathParams throws OperationAssemblyError', + name => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: `/users/{${name}}`, + pathParams: {}, + }), + ).toThrow(OperationAssemblyError); + }, + ); + + test('the error names the placeholder, not the inherited member it resolved to', () => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {}, + }), + ).toThrow(/missing value for path parameter "constructor"/); + }); + + test('an own property named like a prototype member is still honored', () => { + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {constructor: 'me'}, + }); + expect(request.url.pathname).toBe('/users/me'); + }); + + test('a null-prototype pathParams object still resolves its own keys', () => { + const pathParams = Object.assign(Object.create(null) as object, { + id: 'x', + }) as Record; + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{id}', + pathParams, + }); + expect(request.url.pathname).toBe('/users/x'); + }); +}); + +describe('SEAM-27: an unpaired surrogate in a path-param value is rejected', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string with no UTF-8 form. Before audit #67 / #76 that escaped `buildRequest` outside the + // `DexpaceError` tree and outside its `@throws` list. It is now `OperationAssemblyError`, the + // class this call site already throws for a path-param value it cannot use. + test.each([ + ['a lone high surrogate', '\uD800'], + ['a lone low surrogate', '\uDFFF'], + ['a lone surrogate inside a longer value', 'ok\uD800ok'], + ])('%s throws OperationAssemblyError', (_label, value) => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }), + ).toThrow(OperationAssemblyError); + }); + + test('a well-formed surrogate pair is ordinary text and is encoded', () => { + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: '\u{1F600}'}, + }); + expect(request.url.pathname).toBe('/things/%F0%9F%98%80'); + }); + + test('the query projection cannot carry one either, from a builder or from parse()', () => { + // `composeQuery` calls `operationQuery.encode()`, the second `encodeRfc3986Component` path into + // `buildRequest`. Both ways of obtaining a `QueryParams` are now closed: the builder rejects an + // unpaired surrogate, `parse` replaces it. + expect(() => QueryParams.newBuilder().add('c', '\uD800')).toThrow( + /unpaired surrogate/, + ); + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things', + query: QueryParams.parse('c=\uD800'), + }); + expect(request.url.search).toBe('?c=%EF%BF%BD'); + }); + + test('no URIError escapes buildRequest, whatever the descriptor carries (property)', () => { + fc.assert( + fc.property( + fc.string({ + unit: fc.oneof( + fc.constantFrom('a', '/', '.', '%', ' ', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + minLength: 1, + maxLength: 6, + }), + value => { + try { + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }); + } catch (e: unknown) { + expect(e).toBeInstanceOf(OperationAssemblyError); + } + }, + ), + {numRuns: 500}, + ); + }); +}); + describe('a path-param value containing / is encoded, not split (property)', () => { test('holds for arbitrary generated path-param values', () => { fc.assert( diff --git a/packages/core/src/seams/operation.ts b/packages/core/src/seams/operation.ts index 260c36e..0683691 100644 --- a/packages/core/src/seams/operation.ts +++ b/packages/core/src/seams/operation.ts @@ -6,13 +6,14 @@ import type {Headers} from '../http/headers.js'; import type {QueryParams} from '../http/query-params.js'; import type {Method} from '../http/method.js'; import {UrlConstructionError, DexpaceError} from '../http/errors.js'; -import {encodeRfc3986Component} from '../http/rfc3986.js'; +import {encodeRfc3986Component, hasLoneSurrogate} from '../http/rfc3986.js'; /** * Thrown when `buildRequest()` cannot assemble a request from its descriptor: a `{name}` - * placeholder in `pathTemplate` has no value in `pathParams`, or a supplied value is a dot segment - * (`.`/`..`) that the WHATWG URL parser would normalize into a path rewrite instead of keeping as - * one literal segment. + * placeholder in `pathTemplate` has no OWN value in `pathParams` — an inherited member such as + * `constructor` does not satisfy one — or a supplied value is a dot segment (`.`/`..`) that the + * WHATWG URL parser would normalize into a path rewrite instead of keeping as one literal segment, + * or a supplied value carries an unpaired surrogate and so has no percent-encoded form. * * @public */ @@ -50,9 +51,10 @@ export interface OperationDescriptor { readonly pathTemplate: string; /** - * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have a value here; - * each value is percent-encoded as a single path segment, so a value containing `/` cannot inject - * an extra segment (SEAM-27). Defaults to empty. + * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have an OWN property + * here — a name reachable only through the prototype chain, such as `constructor`, is treated as + * absent — and each value is percent-encoded as a single path segment, so a value containing `/` + * cannot inject an extra segment (SEAM-27). Defaults to empty. */ readonly pathParams?: Readonly> | undefined; @@ -103,7 +105,16 @@ function substitutePathParams( pathParams: Readonly> | undefined, ): string { return template.replace(PATH_PARAM_RE, (_match, name: string) => { - const value = pathParams?.[name]; + // `Object.hasOwn`, not `pathParams?.[name]`: the indexed read walks the prototype chain, so + // `{constructor}` against `{}` resolved to `Object.prototype.constructor`, stringified through + // `encodeRfc3986Component`, and shipped a native-code source text as a path segment instead of + // failing assembly. SEAM-27 requires every placeholder to have a *supplied* value, and a name + // the caller never supplied is missing whatever `Object.prototype` happens to carry + // (audit #67 / #76). + const value = + pathParams !== undefined && Object.hasOwn(pathParams, name) + ? pathParams[name] + : undefined; if (value === undefined) { throw new OperationAssemblyError( `missing value for path parameter "${name}"`, @@ -120,6 +131,17 @@ function substitutePathParams( name, ); } + // A string carrying an unpaired surrogate has no UTF-8 form, so `encodeRfc3986Component` — i.e. + // `encodeURIComponent` — throws a bare `URIError: URI malformed`. That escaped `buildRequest` + // outside the `DexpaceError` tree and outside its documented `@throws`. Rejected here with the + // class this call site already throws, rather than guarded inside the encoder, so the failure + // names the parameter (audit #67 / #76). + if (hasLoneSurrogate(value)) { + throw new OperationAssemblyError( + `path parameter "${name}" contains an unpaired surrogate and cannot be percent-encoded`, + name, + ); + } return encodeRfc3986Component(value); }); } @@ -158,10 +180,15 @@ function composeQuery( * @param baseUrl - the absolute base URL to project the operation onto. * @param operation - the operation to assemble into a request. * @returns the assembled request. - * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no value in `pathParams`, - * or a supplied value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. + * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no own value in + * `pathParams` (a name inherited from `Object.prototype` does not count as supplied), or a supplied + * value is a dot segment (`.`/`..`), or a supplied value carries an unpaired surrogate — fix the + * descriptor; no request was assembled. * @throws {@link UrlConstructionError} when `baseUrl` is malformed, non-absolute, or carries a * fragment — supply a clean absolute base URL. + * @throws {@link RequestBodyNotAllowedError} when the descriptor pairs a body with GET, HEAD, TRACE + * or CONNECT (HTTP-7). Assembly ends at `Request.Builder.build()`, so that builder's validation is + * this function's validation. * * @public */ diff --git a/packages/core/src/seams/serde.ts b/packages/core/src/seams/serde.ts index bb0c404..adced9b 100644 --- a/packages/core/src/seams/serde.ts +++ b/packages/core/src/seams/serde.ts @@ -94,8 +94,10 @@ export interface Serializer { * because the caller owns it (SERDE-3). * * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named - * `'AbortError'` when none was given. Checked before the writer lock is taken, so an aborted call - * never leaves the caller's sink locked and never closes it (SERDE-3). + * `'AbortError'` when none was given. Checked before the writer lock is taken, and then raced + * against each pending write, so an aborted call never leaves the caller's sink locked and never + * closes it (SERDE-3). A write parked against a slow sink is the case the pre-check cannot cover; + * the write itself is left outstanding, because aborting it would be taking ownership. * * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the * project-wide test for whether an API owes one. Buffered-bytes APIs — `serialize`, @@ -162,13 +164,15 @@ export interface DecodeTarget { * streams bytes straight into {@link Deserializer.deserializeFrom} and never holds a parsed value to * inspect, and core owning a parser would violate SEAM-1. * - * **Every decode target is treated as non-null.** An implementation sees a schema *value*, which - * carries no nullability it could read, so the check above cannot be conditional — it rejects a - * top-level wire `null` unconditionally. A legitimately nullable top-level target is therefore - * outside this contract: a `200` whose whole body is the literal `null` does not decode, and - * `tristate(inner)` is a *field* combinator rather than a top-level decode target. This is - * deliberate — the alternative lets a permissive schema such as `{parse: (i) => i}` launder a wire - * `null` into a non-null `T`, which is the heap pollution SERDE-5 and SERDE-13 exist to prevent. + * **A decode target is treated as non-null unless the caller says otherwise.** An implementation + * sees a schema *value*, which carries no nullability it could read, so the check above cannot be + * derived from the schema — it rejects a top-level wire `null` unconditionally *by default*. The one + * way to admit one is {@link DecodeTarget.admitsNull}, the caller stating what the schema value + * cannot: that `T` includes `null`. With it set the check is skipped, the `null` reaches the schema, + * which is free to reject it, and `tristate(inner)` can serve as a top-level target rather than only + * a *field* combinator. Off by default, and deliberately so — the alternative lets a permissive + * schema such as `{parse: (i) => i}` launder a wire `null` into a non-null `T`, which is the heap + * pollution SERDE-5 and SERDE-13 exist to prevent. * * **One spelling, both layers.** Every decode entry point takes the schema and its diagnostic label * bundled as a {@link DecodeTarget}: the SPI here, and `decodeResponse` / `decodeSuccessResponse` @@ -208,8 +212,11 @@ export interface Deserializer { * not re-typed, because a contended source is a programmer error rather than a decode failure. * * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named - * `'AbortError'` when none was given. Checked before the reader lock is taken and between reads, - * so an aborted call never leaves the caller's source locked and never cancels it (SERDE-3). + * `'AbortError'` when none was given. Checked before the reader lock is taken, and then raced + * against each pending read, so an aborted call never leaves the caller's source locked and never + * cancels it (SERDE-3). Racing is the load-bearing half: a source that stalls mid-body parks the + * drain inside a read that a between-reads check can never reach again, and an implementation + * that only checks between reads leaves that call unsettled and that source locked forever. * * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the * project-wide test for whether an API owes one. The abort reaches the drain loop; the CPU-bound diff --git a/packages/core/src/seams/transport.test.ts b/packages/core/src/seams/transport.test.ts index 81a78ea..d43d497 100644 --- a/packages/core/src/seams/transport.test.ts +++ b/packages/core/src/seams/transport.test.ts @@ -3,12 +3,16 @@ // Exercises: SEAM-18's residual (composeSignal is the per-call-options-threading helper's cancellation half), // XCUT-2 (timeout vs. caller-cancellation told apart by signal.reason.name, not a message string). // No stub Transport is constructed — neither composeSignal nor isTimeoutSignal takes or returns one. +// Also HTTP-35 (composeSignal's documented RangeError is AbortSignal.timeout()'s own, and no value +// RequestOptionsBuilder accepts can produce it). import {describe, expect, test} from 'bun:test'; import { composeSignal, isTimeoutSignal, CancellationError, } from './transport.js'; +import {RequestOptions} from '../http/request-options.js'; +import {RequestOptionsValidationError} from '../http/errors.js'; describe('composeSignal', () => { test('returns undefined when neither input is supplied', () => { @@ -55,3 +59,29 @@ describe('isTimeoutSignal', () => { expect(isTimeoutSignal(new AbortController().signal)).toBe(false); }); }); + +describe('composeSignal timeout range (HTTP-35)', () => { + // `composeSignal` hands `timeoutMs` straight to `AbortSignal.timeout()`, and what that does with + // an out-of-range delay is a RUNTIME decision, measured 2026-09-05: Node raises + // `RangeError: The value of "delay" is out of range` for `1.5`, for `2**32` and for `-1`; Bun + // accepts `1.5` and `2**32` and raises a `TypeError` for `-1`. So the only claim assertable on + // both is the one below — that no value `RequestOptionsBuilder` accepts can reach that fork at + // all. `tests/node-conformance/seams.test.mjs` asserts the Node half, where the throw is real. + // This is why audit #67 / #76 put the range check in the model rather than clamping here. + test('every timeout RequestOptionsBuilder accepts composes without throwing', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + expect(() => composeSignal(undefined, accepted)).not.toThrow(); + } + }); + + test('a timeout the builder rejects never reaches composeSignal', () => { + for (const value of [1.5, 2 ** 32, -1, 0]) { + expect(() => RequestOptions.newBuilder().timeoutMs(value)).toThrow( + RequestOptionsValidationError, + ); + } + }); +}); diff --git a/packages/core/src/seams/transport.ts b/packages/core/src/seams/transport.ts index 8cd62fc..8388603 100644 --- a/packages/core/src/seams/transport.ts +++ b/packages/core/src/seams/transport.ts @@ -76,8 +76,20 @@ export interface Transport { * logic. * * @param userSignal - an optional caller-supplied abort signal. - * @param timeoutMs - an optional timeout, in milliseconds. + * @param timeoutMs - an optional timeout, in milliseconds. Must be an integer in + * `1 .. 2**32 - 1` — the range Node's `AbortSignal.timeout()` accepts. A value taken from + * {@link RequestOptions.timeoutMs} always is, because {@link RequestOptionsBuilder.timeoutMs} + * rejects everything else at the call site (HTTP-35). A transport's own `defaultTimeoutMs` + * construction option is NOT validated by this package and is the one remaining way an + * out-of-range value reaches here. * @returns the composed signal, the sole supplied signal, or `undefined` when neither is supplied. + * @throws Whatever the host runtime's `AbortSignal.timeout()` raises for an out-of-range delay, + * unwrapped. The runtimes disagree, measured 2026-09-05: Node raises `RangeError` for a fractional + * value, for anything above `4294967295`, and for a negative one; Bun accepts the first two and + * raises `TypeError` for the third. Not wrapped in a `DexpaceError` and not clamped here — it is a + * programming error in whatever supplied the value, and the divergence is exactly why the range + * lives on {@link RequestOptionsBuilder.timeoutMs}, which rejects every such value identically on + * both runtimes (HTTP-35, audit #67 / #76). * * @public */ diff --git a/packages/core/src/sse/stream.test.ts b/packages/core/src/sse/stream.test.ts index 7f3b866..1c361a4 100644 --- a/packages/core/src/sse/stream.test.ts +++ b/packages/core/src/sse/stream.test.ts @@ -169,6 +169,47 @@ test('a mid-stream read failure releases before propagating, with the close erro expect(closeCount).toBe(1); }); +test('a release failure during an in-flight error is reported exactly once (SSE-29, SSE-30)', async () => { + const readFailure = new IoError('socket reset'); + const closeFailure = new IoError('close failed too'); + const reported: unknown[] = []; + let closeCount = 0; + const web = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.error(readFailure); + }, + }); + const stream = new SseStream( + new SseParser(BufferedSource.overStream(web)), + { + close(): Promise { + closeCount += 1; + return Promise.reject(closeFailure); + }, + }, + {onReleaseFailure: e => reported.push(e)}, + ); + + let caught: unknown; + try { + for await (const event of stream) { + void event; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.error).toBe(readFailure); + expect(suppressed.suppressed).toBe(closeFailure); + // SSE-30 scopes the hook to the automatic CLEAN terminal path. With an error already in flight the + // release failure is on the thrown error, and calling the hook as well makes one failure arrive + // twice — once in whatever logs `onReleaseFailure`, once in whatever logs the caught error. + expect(reported).toEqual([]); + expect(closeCount).toBe(1); +}); + test('sseStreamFrom binds lifecycle to the response body (SSE-32)', async () => { let responseClosed = 0; const response = { diff --git a/packages/core/src/sse/stream.ts b/packages/core/src/sse/stream.ts index 980f4ad..919827e 100644 --- a/packages/core/src/sse/stream.ts +++ b/packages/core/src/sse/stream.ts @@ -28,6 +28,11 @@ export interface SseStreamOptions { * Called when a release fails on a clean automatic terminal path, where SSE-30 requires the failure to be * reported out-of-band and swallowed rather than thrown (throwing would discard events already delivered). * + * Called on **that** path only, which is SSE-30's own scope: "no error in flight". A release that fails + * while an error is already propagating is attached to that error as `suppressed` instead, and a release + * that fails during an explicit `close()` rejects that call — reporting either one here as well would + * deliver a single failure twice. + * * Defaults to a no-op. Phase 7 wires a real `Logger` in here without reshaping this class — the same * "mechanism now, wiring later" split Phase 3b used for its logging tees. */ @@ -121,6 +126,9 @@ export class SseStream implements AsyncIterable { } async *#iterate(): AsyncGenerator { + // Whether the catch below already released and already accounted for a release failure. A local, + // not a field: it is read exactly once, by the `finally` of this one generator activation. + let releasedWithError = false; try { for (;;) { // A close observed between pulls ends iteration cleanly, without reading from a torn-down resource. @@ -132,10 +140,17 @@ export class SseStream implements AsyncIterable { } catch (e: unknown) { // SSE-29: release BEFORE the error propagates, and attach a release failure as suppressed rather than // letting it mask the real cause. + releasedWithError = true; await this.#releaseWithInFlightError(e); } finally { // Covers clean end-of-stream and early `break` (the runtime calls `.return()`, which runs this block). - await this.#releaseQuietly(); + // + // Skipped after the catch, which has already released. Running it there awaited the same rejected + // `#closing` promise and handed the close failure to `onReleaseFailure` as well — so one failure was + // reported twice, once out-of-band and once as `suppressed` on the error the consumer catches. SSE-30 + // scopes the hook to the automatic CLEAN terminal path, where there is nothing to throw to; with an + // error in flight there is (audit #67 / #79). + if (!releasedWithError) await this.#releaseQuietly(); } } @@ -176,7 +191,10 @@ export class SseStream implements AsyncIterable { } } - /** SSE-29 / SSE-36: an error is already in flight, so it stays primary and the close error is suppressed. */ + /** + * SSE-29 / SSE-36: an error is already in flight, so it stays primary and the close error is suppressed — + * and NOT also handed to `onReleaseFailure`, which is the clean-terminal path's channel. + */ async #releaseWithInFlightError(primary: unknown): Promise { this.#closed = true; const releasePromise = (this.#closing ??= this.#resource.close()); diff --git a/packages/rx/README.md b/packages/rx/README.md index 2ef018e..fd35cb4 100644 --- a/packages/rx/README.md +++ b/packages/rx/README.md @@ -45,7 +45,44 @@ own guard and surfaces its error through the `Observable`'s error channel. call (`PAGE-8`), so each subscription drives an independent fetch sequence. Both release their source on `unsubscribe()`, including while idle — an SSE stream waiting on the next event is -closed immediately rather than when the server next sends something. +closed immediately rather than when the server next sends something. Which of the two *owns* that source +differs, and that is the next section. + +## Who owns the stream + +`sseEvents$`/`typedSse$` **take ownership of the `SseStream` you hand them.** Subscribing takes the stream's one +iterator (`SSE-26`), and the adapter releases the stream on every termination — `unsubscribe()`, end-of-source +and a source error alike. Do not call `close()` on it yourself, and do not iterate it afterwards; `unsubscribe()` +is how you stop early. + +That covers stopping while the stream is **idle**, which is where a live event stream spends almost all of its +time. The adapter runs the release *ahead of* the iterator's `return()` on purpose: an async generator's +`return()` queues behind a suspended `next()`, so on its own it cannot settle a read that is waiting on a server +which will never send again. + +It is also what a plain `for await` over the same stream already does — `SseStream` releases its resource when +its iterator returns, so `break`ing out of the loop closes the response body too. The reactive form transfers +ownership for the same reason the loop does, not as an extra: + +```typescript +import {sseStreamFrom, type Response} from '@dexpace/core'; + +declare const response: Response; + +// No `close()` here either: leaving the loop releases the response body. +for await (const event of sseStreamFrom(response)) { + if (event.event === 'done') break; +} +``` + +This is a deliberate departure from `ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" +clause, and it is recorded as one — see the `ASYNC-21` row of [`docs/deviations.md`](../../docs/deviations.md). + +`pageItems$`/`pages$` transfer nothing, and attach no release callback. The `Paginator`'s own walk already owns +each page's response — `items()` closes a page before yielding any of its items (`PAGE-11`), and `pages()` +closes the held page from the generator's `finally` (`PAGE-12`), which the iterator's `return()` drives. That is +enough there because a paginator's pulls are bounded HTTP exchanges; an SSE pull is a wait on a server that may +never answer, which is the difference the callback exists for. ## Notes diff --git a/packages/rx/src/sse.test.ts b/packages/rx/src/sse.test.ts index 03e4a71..08425b4 100644 --- a/packages/rx/src/sse.test.ts +++ b/packages/rx/src/sse.test.ts @@ -2,13 +2,20 @@ // packages/rx/src/sse.test.ts // // Exercises: SSE-41 (reactive adapter), SSE-26 (single-pass: second subscription fails loudly), -// SSE-33-36 (typed adapter mapping over reactive stream), ASYNC-21, ASYNC-6. +// SSE-33-36 (typed adapter mapping over reactive stream), ASYNC-21, ASYNC-6, SSE-28 (idempotent release), +// SSE-30 (quiet automatic release). +// +// The `resource ownership` blocks pin the deliberate departure from ASYNC-21's "MUST NOT close the +// caller-owned source on any termination" clause -- see the ASYNC-21 row of `docs/deviations.md`. They count +// the release the OWNED RESOURCE sees, never `SseStream.close()` calls: `close()` memoizes its release +// promise (SSE-28), so a facade-level count reads "once" no matter how many paths call it. import {describe, expect, test} from 'bun:test'; import {firstValueFrom, toArray} from 'rxjs'; import { Protocol, Request, Response, + SseLineTooLongError, SseStream, SseStreamError, sseStreamFrom, @@ -76,6 +83,134 @@ function makeUnclosedSseStream(text: string, onCancel?: () => void): SseStream { return sseStreamFrom(response); } +/** + * The two halves of the ONE resource `sseStreamFrom` hands the facade -- `closingBoth(source, response)` in + * core's `sse/stream.ts`. `socket` is the byte stream's own teardown, which the platform invokes at most once + * per stream and not at all once a producer has already ended it, so it corroborates rather than carries the + * count. + */ +interface ReleaseCounts { + /** `BufferedSource.close()` -> `RetentionWindow.close()` -> `reader.cancel()`. */ + source: number; + /** `Response.close()` -> `body.cancel()`. */ + response: number; + /** The `ReadableStream`'s own `cancel` hook. */ + socket: number; +} + +interface CountingSseStream { + readonly stream: SseStream; + readonly releases: ReleaseCounts; + /** Settles the first time the byte stream itself is torn down. */ + readonly socketTornDown: Promise; +} + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both levels + * `sseStreamFrom` uses. + * + * A structural double rather than a subclass, because `ResponseBuilder.body()` stores what it is handed and + * runs no `instanceof` check. A platform `ReadableStream` cannot do this job on its own: its underlying + * `cancel` hook is invoked at most once by specification and never at all after the producer closed the + * controller, so a second release would collapse into the first and read as clean. + */ +function countingBody( + bytes: ReadableStream, + releases: ReleaseCounts, +): ReadableStream { + const reader = (): ReadableStreamDefaultReader => { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => { + real.releaseLock(); + }, + cancel: async (reason?: unknown) => { + releases.source += 1; + return real.cancel(reason); + }, + } as unknown as ReadableStreamDefaultReader; + }; + return { + get locked(): boolean { + return bytes.locked; + }, + getReader: reader, + cancel: async (reason?: unknown) => { + releases.response += 1; + return bytes.cancel(reason); + }, + } as unknown as ReadableStream; +} + +/** + * An `SseStream` over a body that reports every release it is asked for. + * + * `ended: false` leaves the producer's controller open, which is the idle state a live event stream sits in + * between events -- the reader stays suspended in a pull and only a cancel can settle it. + */ +function makeCountingSseStream( + text: string, + options: {readonly ended: boolean; readonly maxLineBytes?: number}, +): CountingSseStream { + const releases: ReleaseCounts = {source: 0, response: 0, socket: 0}; + let tornDown = (): void => undefined; + const socketTornDown = new Promise(resolve => { + tornDown = resolve; + }); + const bytes = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (options.ended) controller.close(); + }, + cancel() { + releases.socket += 1; + tornDown(); + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, releases)) + .build(); + return { + stream: sseStreamFrom(response, {maxLineBytes: options.maxLineBytes}), + releases, + socketTornDown, + }; +} + +/** Lets the adapter's teardown, which is driven off the microtask queue, run to completion. */ +const settle = async (): Promise => { + await new Promise(resolve => setTimeout(resolve, 20)); +}; + +/** + * Awaits `promise`, failing with a line number rather than hanging when it never settles. + * + * The suspended-pull case regresses as a *hang*, and a bare `await` would surface that as a bare runner + * timeout naming no assertion. + */ +async function within(ms: number, promise: Promise): Promise { + let timer: ReturnType | undefined; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`the teardown did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + describe('sseEvents$', () => { test('emits every parsed SseEvent in order and completes at end-of-stream', async () => { const stream = makeSseStreamFixture('data: one\n\ndata: two\n\n'); @@ -203,3 +338,144 @@ describe('typedSse$', () => { expect(cancelCalled).toBe(true); }); }); + +describe('sseEvents$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\n', + {ended: true}, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + expect(events).toHaveLength(2); + await settle(); + // `socket: 0` is not a miss: the producer ended the byte stream, so there is nothing left for the + // platform to tear down. The two halves the facade owns are still released, once each. + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a source error releases each half exactly once and surfaces the error unwrapped', async () => { + const {stream, releases} = makeCountingSseStream( + `data: ${'x'.repeat(64)}\n\n`, + {ended: false, maxLineBytes: 16}, + ); + + const failure = await rejection(firstValueFrom(sseEvents$(stream))); + + // SSE-29 releases before the error propagates; the adapter's own release then finds it already done. + expect(failure).toBeInstanceOf(SseLineTooLongError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('early unsubscribe releases each half exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\ndata: three\n\n', + {ended: false}, + ); + + const subscription = sseEvents$(stream).subscribe({ + next() { + subscription.unsubscribe(); + }, + }); + + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: one\n\n', + {ended: false}, + ); + const received: string[] = []; + const subscription = sseEvents$(stream).subscribe({ + next(event) { + const item = event.data[0]; + if (item !== undefined) received.push(item); + }, + }); + + await settle(); + expect(received).toEqual(['one']); + expect(releases.socket).toBe(0); + + // The server will never send another byte, so the reader is parked in a pull. Only the release running + // AHEAD of `iterator.return()` settles it -- a `return()` on an async generator queues behind the + // in-flight `next()`. Drop the release from `sseEvents$` and this never resolves. + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); + +describe('typedSse$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream('data: 1\n\ndata: 2\n\n', { + ended: true, + }); + + const values = await firstValueFrom( + typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).pipe(toArray()), + ); + + expect(values).toEqual([1, 2]); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a throwing mapper releases each half exactly once (SSE-36)', async () => { + const {stream, releases} = makeCountingSseStream('data: one\n\n', { + ended: false, + }); + + // Three release paths converge here: `runMapper`'s explicit `close()`, the adapter's `release`, and the + // mapping generator's `return()` unwinding into the facade's own quiet release. + const failure = await rejection( + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + expect(failure).toBeInstanceOf(TypeError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown through the mapping generator', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: 100\n\n', + {ended: false}, + ); + const subscription = typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).subscribe({ + next() { + // ignore + }, + }); + + await settle(); + expect(releases.socket).toBe(0); + + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); diff --git a/packages/rx/src/sse.ts b/packages/rx/src/sse.ts index e4a343f..c7686e0 100644 --- a/packages/rx/src/sse.ts +++ b/packages/rx/src/sse.ts @@ -12,13 +12,24 @@ import {fromAsyncIterable} from './from-async-iterable.js'; /** * Bridges an {@link @dexpace/core#SseStream} to an RxJS `Observable` (SSE-41, ASYNC-21). * + * **Subscribing transfers ownership of `stream` to the returned `Observable`.** The adapter releases it on + * every termination path -- unsubscription, end-of-source and a source error alike (ASYNC-6). Do not call + * `stream.close()` yourself, and do not iterate `stream` after passing it here: its iterator may be taken at + * most once (SSE-26) and this function takes it, so the subscription's lifetime is the stream's lifetime. + * Closing it out from under a live subscription corrupts nothing -- `close()` is idempotent (SSE-28) -- but it + * ends the stream from the wrong end; `unsubscribe()` is the supported way. + * + * That transfer is a deliberate departure from ASYNC-21's "MUST NOT close the caller-owned source on any + * termination" clause, recorded in this project's deviation register. It is what releases the response body + * when a subscriber unsubscribes from an *idle* stream -- the state a long-lived event stream sits in almost + * all of the time -- because an async generator's `return()` queues behind the suspended pull it would + * otherwise have to interrupt, and so cannot settle one on its own. + * * Single-subscription: `SseStream` wraps an already-open, single-use HTTP response body (BODY-14) and is itself * single-pass (SSE-26) -- obtaining an iterator succeeds at most once and a second attempt fails loudly. * Subscribing to the returned `Observable` a second time reaches `SseStream`'s own guard and surfaces an error * through the `Observable`'s error channel. * - * Unsubscribing closes the stream, releasing the response body even when no event is in flight (ASYNC-6). - * * Diagnostic context propagates on its own (ASYNC-8–ASYNC-11): every pull runs inside the continuation chain * that called `subscribe()`, which is exactly what Node's `AsyncLocalStorage` tracks. That holds only for the * `Observable` returned here -- a caller who pipes it through an RxJS scheduler operator (`observeOn`, @@ -38,8 +49,13 @@ export function sseEvents$(stream: SseStream): Observable { /** * Bridges an {@link @dexpace/core#SseStream} to a typed RxJS `Observable` via an {@link @dexpace/core#SseMapper} (SSE-41, ASYNC-21, SSE-33–SSE-36). * - * Shares every note on {@link sseEvents$}: single-subscription, release-on-unsubscribe, and automatic - * diagnostic-context propagation through the unscheduled path. + * **Subscribing transfers ownership of `stream` to the returned `Observable`,** exactly as in + * {@link sseEvents$} and for the same reasons: the adapter releases the stream on every termination path, so + * do not call `stream.close()` yourself and do not iterate `stream` after passing it here. The mapper sits + * between the stream and the subscriber; it does not change who owns the stream. + * + * Shares every other note on {@link sseEvents$} too: single-subscription, and automatic diagnostic-context + * propagation through the unscheduled path. * * A throwing mapper reaches the `Observable`'s error channel unwrapped (ASYNC-13), after `typedSseStream` has * released the stream (SSE-36). diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts index cf8fc78..c4a1bf7 100644 --- a/packages/transport-conformance/src/fixtures.ts +++ b/packages/transport-conformance/src/fixtures.ts @@ -1,11 +1,13 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/fixtures.ts +import {createReadStream} from 'node:fs'; import { createServer, type IncomingMessage, type Server, type ServerResponse, } from 'node:http'; +import type {FileBodyDescriptor} from '@dexpace/core'; /** A running fixture server, addressable by URL and shut down through {@link TestServer.close}. */ export interface TestServer { @@ -21,11 +23,77 @@ const SLOW_RESPONSE_MS = 5_000; const DRIP_INTERVAL_MS = 50; const DRIP_CHUNKS = 20; +/** + * `/repeated-challenge`'s two `WWW-Authenticate` lines, in wire order. Exported so the suite asserts + * against what the server actually sent rather than against a second copy that can drift from it. + * + * The first algorithm is deliberately one no SDK handler supports, so a transport that keeps only the + * first value produces a challenge nothing can answer — the shape audit #67 / #74 found. + */ +export const REPEATED_CHALLENGES: readonly string[] = [ + 'Digest realm="conformance", nonce="n1", algorithm=SHA-512-256', + 'Digest realm="conformance", nonce="n1", algorithm=SHA-256, qop="auth"', +]; + +/** + * `/fixed-length`'s payload. Its length is what a HEAD response advertises and does not deliver, so + * it is exported: the row asserts the header survived the body-less decision rather than asserting + * a number written twice. + */ +export const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + +/** `/not-modified`'s validator, the one header a 304 exists to carry. */ +export const NOT_MODIFIED_ETAG = '"conformance-v1"'; + +/** + * The three fixtures whose responses can carry no body at all, in their own function because the + * main switch is at the 70-line lint cap -- and because they are one topic (TRANSPORT-24/25). + * `req` is not needed: `node:http` suppresses the body of a HEAD response by itself. + * + * @param pathname - the requested path. + * @param res - the response to write. + * @returns `true` when this function answered, `false` to fall through to {@link route}. + */ +function routeBodyless(pathname: string, res: ServerResponse): boolean { + switch (pathname) { + case '/no-content': + // TRANSPORT-24 with the WHATWG null-body rule: a 204 has no body and no framing to describe + // one. Node's `node:http` sends no `Content-Length` here at all; Bun 1.3.14's sends `0`. The + // row therefore asserts the header is absent-or-zero, never a positive length -- what a + // transport is answerable for is the body SHAPE, which is the same on both. + res.writeHead(204); + res.end(); + return true; + case '/not-modified': + // Deliberately WITHOUT a `Content-Length`, though RFC 9110 15.4.5 permits a 304 to carry the + // one a 200 would have had. undici 6.28.0 believes it: a 304 declaring 17 bytes leaves the + // dispatcher waiting for a body that cannot come, and the exchange dies with + // `UND_ERR_SOCKET: other side closed` (measured 2026-09-05, Node and Bun alike). That is + // undici's bug to have, not this suite's to provoke -- the row is about the ETag surviving. + res.writeHead(304, {etag: NOT_MODIFIED_ETAG}); + res.end(); + return true; + case '/fixed-length': + // The HEAD row's target. The declared length describes the body a GET would return, so the + // header promises bytes the HEAD response will not deliver: a transport that framed a stream + // from it hands the caller a read that never completes. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return true; + default: + return false; + } +} + function route( pathname: string, req: IncomingMessage, res: ServerResponse, ): void { + if (routeBodyless(pathname, res)) return; switch (pathname) { case '/echo-headers': res.writeHead(200, {'content-type': 'application/json'}); @@ -101,6 +169,21 @@ function route( res.end('done'); }, SLOW_RESPONSE_MS).unref(); return; + case '/repeated-challenge': + // AUTH-12/AUTH-25: the same challenge header sent TWICE, which RFC 9110 5.3 permits for any + // list-valued field and RFC 7616 3.3 recommends for Digest algorithm discovery -- one challenge + // per algorithm, strongest first. The two transports legitimately surface it differently: + // WHATWG `Headers` comma-joins every name but `Set-Cookie`, so `@dexpace/transport-fetch` + // delivers one entry, while undici arrays any repeated header and `@dexpace/transport-undici` + // keeps two. Neither may LOSE one, which is what the row asserts. + // + // An array value in `writeHead`, not two `setHeader` calls: `setHeader` on the same name + // replaces, which would make the fixture single-valued and the row vacuous. Spread into a + // MUTABLE copy -- `OutgoingHttpHeader` is `string | string[]`, so a `readonly string[]` does + // not satisfy it, and handing the exported constant itself to `node:http` would alias it. + res.writeHead(401, {'www-authenticate': [...REPEATED_CHALLENGES]}); + res.end(); + return; case '/redirect': res.writeHead(302, {location: '/echo-headers'}); res.end(); @@ -141,3 +224,77 @@ export function startFixtureServer(): Promise { }); }); } + +/** What {@link fileBodyFixture} needs beyond the path; mirrors `fileBody()`'s own option bag. */ +export interface FileBodyFixtureOptions { + /** The byte offset the descriptor declares; defaults to 0. */ + readonly start?: number; + /** The byte count the descriptor declares, captured as `fileBody()` captures it from `stat`. */ + readonly count: number; + /** Incremented on every `writeTo` call, so a row can assert the transport used it. */ + readonly writes?: {count: number}; +} + +/** + * A `kind: 'file'` request body over a real path, carrying BODY-13's `transferred === count` check + * itself — the shape `@dexpace/body-file`'s `fileBody()` produces, minus the construction-time + * validation no row here needs. + * + * **Deliberately a stand-in, not the real factory.** `@dexpace/transport-conformance` is `private`, + * resolves unbuilt, and depends on `@dexpace/core` alone; taking `@dexpace/body-file` would put a + * ninth entry in the root `build:deps` chain for one row. A real `fileBody()` crossing a real + * transport already has a home — `tests/node-conformance/transport.test.mjs`, which is the only + * layer that can host it. What a *transport* is answerable for is narrower, and is exactly what this + * exercises: TRANSPORT-28's structural recognition on `kind` alone, and that the declared length is + * honoured by calling the descriptor's own `writeTo` rather than by reading `path` behind its back. + * + * @param path - the file to stream; read fresh on every `writeTo`, as BODY-11 requires. + * @param options - the declared range, and an optional write counter. + * @returns a frozen descriptor a transport must recognise structurally. + */ +export function fileBodyFixture( + path: string, + options: FileBodyFixtureOptions, +): FileBodyDescriptor { + const start = options.start ?? 0; + const {count} = options; + return Object.freeze({ + kind: 'file' as const, + mediaType: 'application/octet-stream', + contentLength: count, + replayable: true, + path, + start, + count, + async writeTo(sink: WritableStream): Promise { + if (options.writes !== undefined) options.writes.count += 1; + const writer = sink.getWriter(); + if (count === 0) { + writer.releaseLock(); + return; + } + const stream = createReadStream(path, {start, end: start + count - 1}); + let transferred = 0; + try { + for await (const chunk of stream) { + const bytes = chunk as Buffer; + await writer.write(new Uint8Array(bytes)); + transferred += bytes.byteLength; + } + if (transferred !== count) { + // BODY-13's exact sentence, and its exact wording in `@dexpace/body-file`: the error names + // transferred-of-total, so a row can assert the numbers rather than only the class. + throw new Error( + `short write: transferred ${String(transferred)} of ${String(count)} bytes`, + ); + } + } catch (error) { + await writer.abort(error); + throw error; + } finally { + stream.destroy(); + writer.releaseLock(); + } + }, + }); +} diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 316110a..1e28538 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -1,15 +1,36 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot -// drift. Exercises: TRANSPORT-1..9, TRANSPORT-15..17, TRANSPORT-20..27, TRANSPORT-29, SEAM-12, -// SEAM-16, SEAM-30, NFR-15. TRANSPORT-10..14 are asserted at their source in @dexpace/transport-shared; -// TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's full flow is -// transport-undici's challenge-handler.test.ts. +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..29, BODY-13, +// RETRY-2 (a permanent misconfiguration is outside the retryable IoError tree), +// HTTP-35 (a transport-wide default timeout outside AbortSignal.timeout()'s range is refused at the +// factory), +// SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is +// answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one +// outbound header pass both adapters call -- is asserted at its source in +// @dexpace/transport-shared; the rows here cover what each adapter decides for itself, which since +// audit #67 / #81 includes TRANSPORT-11/12's per-header degrade, because the two adapters had four +// different answers for the same model-valid header and only a shared row could say so. +// TRANSPORT-18's collapse is a Deviation Ledger row, as is TRANSPORT-28's zero-copy SHOULD, whose two +// MUSTs the file-body rows do assert; TRANSPORT-30's +// full proxy-challenge flow is transport-undici's challenge-handler.test.ts, and only its +// unsupported-type refusal is a row here. TRANSPORT-22 is NOT driven from here -- +// forcing an adaptation throw needs a per-transport hook into the native response, so each adapter +// asserts it against its own (transport-fetch's fetch-transport.test.ts:123, transport-undici's +// undici-transport.test.ts:615). TRANSPORT-9's producer-failure race is not driven from here +// either, and for the same reason: proving that a native call still pending when the producer fails +// is CANCELLED needs the signal the adapter handed it, which only an instrumented native client can +// show -- a `FetchLike` in transport-fetch's suite and a bring-your-own `Dispatcher` in +// transport-undici's, each with a call that resolves after the producer has already lost the race. +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; import { getBuildInfo, getGlobalLogger, Headers, + isIoError, Request, RequestOptions, setGlobalLogger, @@ -17,7 +38,29 @@ import { type Logger, type Transport, } from '@dexpace/core'; -import {startFixtureServer, type TestServer} from './fixtures.js'; +import { + fileBodyFixture, + FIXED_LENGTH_BODY, + NOT_MODIFIED_ETAG, + REPEATED_CHALLENGES, + startFixtureServer, + type TestServer, +} from './fixtures.js'; + +/** + * A proxy configuration the transport under test cannot honour, so TRANSPORT-30's + * "discoverable rather than silently misbehaving" clause has something to be asserted against. + * + * Supplied by the adapter because only the adapter knows which of `ProxyType`'s values its native + * client refuses — `@dexpace/core` resolves `socks4`/`socks5` from `ALL_PROXY` (CFG-22) and neither + * shipped transport can carry either. + */ +export interface UnsupportedProxy { + /** The `ProxyOptions.type` the native client cannot honour; the refusal must name it. */ + readonly type: string; + /** Builds a transport configured with that type. Expected to throw rather than return one. */ + build(): Transport; +} /** * The clauses `docs/product-spec/17-transport-adapter-conformance-contract.md` scopes to only one @@ -33,6 +76,25 @@ export interface TransportCapabilities { readonly supportsProxy: boolean; /** TRANSPORT-11: whether `Connection` is in this transport's outbound drop set. */ readonly dropsConnectionHeader: boolean; + /** + * TRANSPORT-30: a proxy type this transport's configuration can express and its native client + * cannot honour. Omit it when the transport takes no proxy at all, or honours every type it can + * be handed — the row then asserts that omission is the truth rather than skipping. + */ + readonly unsupportedProxy?: UnsupportedProxy; + /** + * HTTP-35: builds a transport whose transport-wide default timeout is `value`. + * + * Required, not a capability flag, because §17 assumes every transport has one — TRANSPORT-5 is + * written as "a per-call override … overriding the configured default for that call only". The + * rows hand it values `AbortSignal.timeout()` cannot take and expect the factory to refuse them, + * because a default nobody checked is the last path by which such a value reaches a deadline + * (`RequestOptions.timeoutMs` has been checked at its setter since audit #67 / #76). + * + * Typed `number` on purpose: `0`, `-1`, `1.5`, `2**32` and `NaN` are all legitimately `number`, + * so the row needs no cast to express what it is testing. + */ + buildWithDefaultTimeoutMs(value: number): Transport; } /** What every row below needs: a transport factory, the live fixture origin, and the capability flags. */ @@ -310,8 +372,43 @@ function registerProducerRows(ctx: SuiteContext): void { }); } +/** + * A scheme every runtime under test refuses and `@dexpace/core` accepts. + * + * `Request` validates a URL by handing it to WHATWG `URL`, which parses `ftp:` perfectly well and + * gives it a real origin — so an `ftp://` request reaches the native client, which refuses it. + * `foo://` would be refused too, but its `origin` is the string `"null"`, which changes what undici + * is even asked; `ftp:` keeps the two adapters answering the same question. + */ +const UNSUPPORTED_SCHEME_URL = 'ftp://example.com/anything'; + +function registerPermanentFailureRows(ctx: SuiteContext): void { + describe('TRANSPORT-20, RETRY-2: a permanent misconfiguration is not a retryable failure', () => { + test('an unsupported URL scheme fails outside the IoError tree', async () => { + // The two adapters answered this oppositely until audit #67 / #82. undici's dispatcher + // rejects `ftp://` with `UND_ERR_INVALID_ARG`, which `transport-undici` already mapped to a + // TypeError; `fetch` rejects with a TypeError whose shape depends on the runtime -- Node's + // undici-backed one says `fetch failed` with an `unknown scheme` cause, Bun 1.3.14 says + // `protocol must be http:, https: or s3:` with `code: ERR_INVALID_ARG_VALUE` -- and + // `transport-fetch` classified all of it as the RETRYABLE TransportFailureError. + // + // `classify.ts` returns true for every IoError, so that verdict spends the caller's entire + // retry budget re-proving a URL no retry can fix. `isIoError(e) === false` is the assertion + // because it is exactly what the retry engine asks. + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(UNSUPPORTED_SCHEME_URL) + .build(); + const error = await rejection(transport.send(request)); + expect(isIoError(error)).toBe(false); + expect(error).toBeInstanceOf(TypeError); + }); + }); + }); +} + function registerFailureRows(ctx: SuiteContext): void { - describe('TRANSPORT-4/5/6/20/22: failure classification and socket release', () => { + describe('TRANSPORT-4/5/6/20: failure classification and per-call timeouts', () => { test('a dead port surfaces the retryable TransportFailureError', async () => { await withTransport(ctx.makeTransport, async transport => { const request = Request.newBuilder().url('http://127.0.0.1:1').build(); @@ -528,6 +625,319 @@ function registerHeaderRows(ctx: SuiteContext): void { }); } +/** + * Headers every model layer in this SDK accepts and at least one shipped native client refuses + * outright, paired with the value that provokes the refusal. + * + * `expect`, `keep-alive` and `upgrade` are `undici`'s three unconditional rejections + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`), and Node's global `fetch` is undici-backed, so both adapters + * meet them. `X Custom` is the non-token name: `@dexpace/core` admits any printable ASCII byte in + * a header name (`http/ascii-validation.ts:29-33`), while both native layers require RFC 9110 + * `token`. + * + * TRANSPORT-12 is what makes these one table rather than four transport-specific quirks: whatever + * the native client refuses, the transport drops *that header only* and still dispatches. + */ +const NATIVE_REJECTED_HEADERS: readonly (readonly [string, string])[] = [ + ['Expect', '100-continue'], + ['Keep-Alive', 'timeout=5'], + ['Upgrade', 'websocket'], + ['X Custom', 'model-valid, non-token'], +]; + +function registerNativeRejectionRows(ctx: SuiteContext): void { + describe('TRANSPORT-11/12/13: a header the native client refuses degrades to a logged drop', () => { + for (const [name, value] of NATIVE_REJECTED_HEADERS) { + test(`${name} is dropped and logged, and the rest of the request still dispatches`, async () => { + let echoed: Record = {}; + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers( + Headers.newBuilder() + .set(name, value) + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + echoed = await readEchoedHeaders(transport, request); + }); + }); + // The send resolved at all, which is TRANSPORT-12's "the resulting native exception MUST NOT + // escape the send contract"; the two header assertions are its "the bad header is absent, + // the normal header present". + expect(echoed[name.toLowerCase()]).toBeUndefined(); + expect(echoed['x-pass-through']).toBe('survives'); + // TRANSPORT-13: the drop is discoverable by name, never silent. + expect(dropped).toContain(name.toLowerCase()); + }); + } + + test('a Connection value the native client cannot carry is dropped on both transports', async () => { + // `Connection` is the one name whose drop set legitimately differs (TRANSPORT-11's own note, + // and `registerDropSetRows` below), so this row asserts the intersection: `upgrade` is neither + // `close` nor `keep-alive`, undici rejects it outright (`request.js:400-404`), and the WHATWG + // layer forbids the name entirely. Both must drop it, whichever reason applies. + // + // Asserted through the log rather than the echo for the same reason `registerDropSetRows` + // does: both clients set a `Connection` header of their own, so the wire cannot tell a + // forwarded caller header from the client's. + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers(Headers.newBuilder().set('Connection', 'upgrade').build()) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(200); + await response.close(); + }); + }); + expect(dropped).toContain('connection'); + }); + }); +} + +/** + * The declared length of the truncate-after-stat file body, deliberately **below** both shipped + * adapters' 1,000,000-byte materialize bound, so this row drives the buffered path. + * + * The streamed path is asserted in `tests/node-conformance/transport.test.mjs` instead, and that is + * not a preference. Bun 1.3.14's `Readable.fromWeb` leaks the abort reason as two or three unhandled + * rejections when the web readable behind it is aborted mid-pull, which is exactly what a producer + * failure on the streamed path does; `bun:test` then fails whichever row happens to be running. + * Isolated to sixteen lines with no SDK code in them, and clean under `node --test` on both + * transports (measured 2026-09-05, audit #67 / #81). Raising this constant past 1,000,000 will make + * the row red for that reason and no other. + */ +const TRUNCATED_FILE_BYTES = 64; + +/** How far a truncate-after-stat cuts the file back; small enough that no read can be a full one. */ +const TRUNCATED_TO_BYTES = 10; + +/** The intact file-body fixture's size, and the byte range the ranged row asks for inside it. */ +const INTACT_FILE_BYTES = 64; +const INTACT_RANGE = {start: 10, count: 20} as const; + +/** Distinguishable bytes, so a misaligned send fails on content and not merely on length. */ +function fileFixtureBytes(size: number): Uint8Array { + const bytes = new Uint8Array(size); + for (let index = 0; index < size; index += 1) { + bytes[index] = 33 + ((index * 7) % 94); + } + return bytes; +} + +/** Writes `size` fixture bytes to a fresh temporary file, runs `body`, and removes the directory. */ +async function withFixtureFile( + size: number, + body: (path: string) => Promise, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dexpace-conformance-file-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, fileFixtureBytes(size)); + return await body(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + +/** + * Every message on `error` and its `cause` chain, joined. + * + * BODY-13's transferred-of-total text surfaces at a different depth per path: the streamed path + * rethrows the producer's own failure as the message, while the buffered path wraps it as a cause + * under a fixed "request body could not be written". Both satisfy the requirement; asserting on the + * chain is what lets one row cover both without pinning either transport's wrapper wording. + */ +function messageChain(error: unknown): string { + const parts: string[] = []; + let current: unknown = error; + for (let depth = 0; depth < 8 && current instanceof Error; depth += 1) { + parts.push(current.message); + current = current.cause; + } + return parts.join(' <- '); +} + +function registerFileBodyRows(ctx: SuiteContext): void { + describe('TRANSPORT-28, BODY-13: a file body is dispatched through its own writeTo', () => { + test('an intact ranged file body puts exactly its declared bytes on the wire', async () => { + await withFixtureFile(INTACT_FILE_BYTES, async path => { + const writes = {count: 0}; + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-body')) + .body(fileBodyFixture(path, {...INTACT_RANGE, writes})) + .build(), + ); + // TRANSPORT-28's "assert exactly that byte range reaches the wire". + expect([...(await response.bytes())]).toEqual([ + ...fileFixtureBytes(INTACT_FILE_BYTES).slice( + INTACT_RANGE.start, + INTACT_RANGE.start + INTACT_RANGE.count, + ), + ]); + }); + // TRANSPORT-17's counterpart for a replayable body: a transport that reads `path` itself + // rather than calling `writeTo` would put the same bytes on the wire and leave this at 0. + expect(writes.count).toBe(1); + }); + }); + + test('a file body truncated after its length was captured fails the send', async () => { + await withFixtureFile(TRUNCATED_FILE_BYTES, async path => { + // The descriptor is built first, exactly as `fileBody()` captures `count` from `stat`, and + // the file shrinks underneath it afterwards. `content-length` is dropped outbound, so the + // wire cannot detect this either — BODY-13's check inside `writeTo` is the only thing that + // can, and a transport that hands the path to its native client never runs it + // (audit #67 / #81, where undici POSTed the ten surviving bytes and resolved 200). + const body = fileBodyFixture(path, {count: TRUNCATED_FILE_BYTES}); + await truncate(path, TRUNCATED_TO_BYTES); + const error = await withTransport(ctx.makeTransport, transport => + rejection( + transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.isolatedUrl('/echo-body')) + .body(body) + .build(), + ), + ), + ); + expect(error).toMatchObject({name: 'TransportFailureError'}); + expect(messageChain(error)).toContain( + `transferred ${String(TRUNCATED_TO_BYTES)} of ${String(TRUNCATED_FILE_BYTES)} bytes`, + ); + }); + }); + }); +} + +/** + * The fixture's challenge list, recovered from however this transport chose to split it. + * + * Scoped to `/repeated-challenge` on purpose, and deliberately NOT a general RFC 7235 parser: the two + * fixture challenges are `Digest`-schemed and carry no comma inside a quoted value, so a split at each + * `, Digest ` boundary recovers exactly what the server wrote. `@dexpace/core`'s real parser is + * `@internal` and unreachable from here — the core-side rows for it live in + * `packages/core/src/auth/auth-step.test.ts`. + */ +function challengeList(values: readonly string[]): readonly string[] { + return values.join(', ').split(/,\s+(?=Digest )/u); +} + +function registerBodylessRows(ctx: SuiteContext): void { + describe('TRANSPORT-24/25/27: a response that can carry no body reports none', () => { + // `body === null` is the WHATWG shape and the one `@dexpace/core` already types + // (`http/response.ts:18`); it is also the only shape a consumer can branch on without reading. + // Three of the four native combinations disagreed until audit #67 / #82 -- undici's dispatcher + // always hands back a `BodyReadable`, Node's `fetch` returns `null`, Bun 1.3.14's `fetch` + // returns a live `ReadableStream` -- so each adapter decides for itself now and these rows are + // what say so. The alternative, an empty stream on both, makes a consumer read to learn there + // is nothing there. + // + // `reasonPhrase` is `undefined`-or-string on purpose: `fetch` surfaces `statusText` and undici's + // `ResponseData` has no such field, a divergence recorded beside §10 item 13. Asserting the + // union is what keeps this row about the body shape. + test('a 204 carries a null body and no positive length', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/no-content')).build(), + ); + expect(response.status.code).toBe(204); + expect(response.body).toBeNull(); + // Absent on Node's `node:http`, `'0'` on Bun 1.3.14's -- what a transport is answerable for + // is that it never invented a length for a body that does not exist. + expect([undefined, '0']).toContain( + response.headers.get('content-length'), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + // Idempotent and non-blocking over a response the transport already released. + await response.close(); + await response.close(); + }); + }); + + test('a 304 carries a null body and still delivers its validator', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/not-modified')).build(), + ); + expect(response.status.code).toBe(304); + expect(response.body).toBeNull(); + // A 304 exists to carry validators; dropping the body must not drop them. + expect(response.headers.get('etag')).toBe(NOT_MODIFIED_ETAG); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('a HEAD carries a null body and keeps the length it advertises', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(ctx.url('/fixed-length')) + .build(), + ); + expect(response.status.code).toBe(200); + expect(response.body).toBeNull(); + // The header describes the body a GET would have returned and must survive verbatim -- + // this is the one body-less case where a length is meaningful (TRANSPORT-27). + expect(response.headers.get('content-length')).toBe( + String(FIXED_LENGTH_BODY.length), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('the same resource over GET does carry its body', async () => { + // The twin of the three rows above: nulling a body-less response must not null an ordinary + // one, and `/fixed-length` is the same route the HEAD row just read nothing from. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/fixed-length')).build(), + ); + expect(response.body).toBeInstanceOf(ReadableStream); + expect(await response.text()).toBe(FIXED_LENGTH_BODY); + }); + }); + }); +} + +function registerInboundHeaderRows(ctx: SuiteContext): void { + describe('TRANSPORT-14, AUTH-12/AUTH-25: a repeated inbound header keeps every value', () => { + test('two WWW-Authenticate lines reach the pipeline as the same challenge list', async () => { + // The two transports split this differently and both are right: WHATWG `Headers` comma-joins + // every name but `Set-Cookie`, so `@dexpace/transport-fetch` yields ONE `getAll` entry, while + // undici arrays any repeated header and `@dexpace/transport-undici` yields TWO. RFC 9110 5.3 + // makes the two wire shapes equivalent, so the entry count is not what either adapter is + // answerable for — the challenge list after the split is, and it must be identical. + // + // What this row guards is the loss: before audit #67 / #74 the auth step read + // `headers.get(...)`, saw only the first line, and left a 401 offering an answerable SHA-256 + // challenge unanswered through undici while the identical offer authenticated through fetch. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/repeated-challenge')).build(), + ); + expect(response.status.code).toBe(401); + expect( + challengeList(response.headers.getAll('WWW-Authenticate')), + ).toEqual([...REPEATED_CHALLENGES]); + await response.close(); + }); + }); + }); +} + function registerDropSetRows(ctx: SuiteContext): void { describe('TRANSPORT-11/13: the transport-specific drop set', () => { test('the Connection header follows this transport’s documented drop set', async () => { @@ -554,6 +964,91 @@ function registerDropSetRows(ctx: SuiteContext): void { }); } +function registerProxyRefusalRows(ctx: SuiteContext): void { + describe('TRANSPORT-30: a proxy the native client cannot honour is refused, not attempted', () => { + test('an unsupported proxy type fails at construction with a typed, naming error', () => { + const unsupported = ctx.capabilities.unsupportedProxy; + if (unsupported === undefined) { + // The row still runs here rather than being skipped, and asserts the only thing left to + // assert: that there is genuinely no proxy surface to mis-set. `@dexpace/transport-fetch` + // is this leg — it ships no `proxy` option at all, deliberately (design doc §6), so + // TRANSPORT-30 has no subject on it and a silent skip would look the same as a gap. + expect(ctx.capabilities.supportsProxy).toBe(false); + return; + } + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first send, + // where it arrives as whatever the native client raises — the shape TRANSPORT-30 rules out. + void unsupported.build(); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // Not an `IoError`: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can + // fix is non-retryable for free (RETRY-2). An undici `InvalidArgumentError` escaping raw + // would fail this too, being neither a `TypeError` nor documented. + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the type that was refused, not merely that something was. + expect((thrown as Error).message).toContain(unsupported.type); + }); + }); +} + +/** + * Defaults `AbortSignal.timeout()` refuses. `1.5` and `2**32` are the two Bun 1.3.14 accepts and + * Node rejects with a `RangeError`, which is what made an unvalidated default a per-runtime + * behaviour rather than a per-caller error. + */ +const UNHONOURABLE_TIMEOUTS: readonly number[] = [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, +]; + +function registerDefaultTimeoutRows(ctx: SuiteContext): void { + describe('HTTP-35, TRANSPORT-5: an unhonourable default timeout is refused at construction', () => { + for (const value of UNHONOURABLE_TIMEOUTS) { + test(`a default of ${String(value)} fails the factory, not the first send`, () => { + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first + // send, where it arrives as a raw `RangeError` out of `AbortSignal.timeout()` on Node -- + // or, on Bun, as no failure at all and a deadline nobody asked for. + void ctx.capabilities.buildWithDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // The same shape every other construction-time refusal in these transports has, and + // outside the IoError tree for the same reason (RETRY-2). + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the value that was refused. + expect((thrown as Error).message).toContain(String(value)); + }); + } + + test('a default inside the range builds a transport that still sends', async () => { + // The twin: narrowing the accepted range must not reject a legitimate default. 30s is the + // shape a caller actually configures, and the send proves the value reached `composeSignal` + // without tripping it. + await withTransport( + () => ctx.capabilities.buildWithDefaultTimeoutMs(30_000), + async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/echo-headers')).build(), + ); + expect(response.status.code).toBe(200); + await response.close(); + }, + ); + }); + }); +} + function registerScopedRows(ctx: SuiteContext): void { if (ctx.capabilities.supportsInternalCancel) { describe('TRANSPORT-8: an internal cancel is told apart from a timeout', () => { @@ -635,10 +1130,17 @@ export function runTransportConformanceSuite( registerBodyRows(ctx); registerProducerRows(ctx); registerFailureRows(ctx); + registerPermanentFailureRows(ctx); registerCancellationRows(ctx); registerLifecycleRows(ctx); registerHeaderRows(ctx); + registerNativeRejectionRows(ctx); + registerFileBodyRows(ctx); + registerBodylessRows(ctx); + registerInboundHeaderRows(ctx); registerDropSetRows(ctx); + registerProxyRefusalRows(ctx); + registerDefaultTimeoutRows(ctx); registerScopedRows(ctx); }); } diff --git a/packages/transport-fetch/README.md b/packages/transport-fetch/README.md index 3ef2b5b..60891f4 100644 --- a/packages/transport-fetch/README.md +++ b/packages/transport-fetch/README.md @@ -50,13 +50,37 @@ records the decision and the four reasons the floor does not move instead. - Redirects are **never** followed (`redirect: 'manual'`). The SDK pipeline is the redirect authority (`TRANSPORT-1`/`TRANSPORT-2`). -- `Content-Length`, `Host`, `Transfer-Encoding`, and `Connection` are dropped outbound — the client - computes its own framing — and every drop is logged by name (never by value) through the global - logger, deduped per name by default (`TRANSPORT-11`/`TRANSPORT-13`). +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped outbound because the client computes + its own framing; `Connection`, `Expect`, `Keep-Alive` and `Upgrade` because the layer underneath + refuses them. WHATWG names all four forbidden request headers, but the implementations do not + enforce that list and disagree about what happens instead: on Node the global `fetch` is + undici-backed, so an undropped `Expect`/`Keep-Alive`/`Upgrade` reaches undici's own validation and + fails the send with the **retryable** `TransportFailureError` — a permanent misconfiguration + spending the caller's whole retry budget — while Bun 1.3.14 forwards the first two to the wire and + hangs on the third until something else times the call out. Dropping the name is the one behaviour + `TRANSPORT-12` asks for, + and it matches `@dexpace/transport-undici` (measured 2026-09-05; audit #67 / #81). +- A header name the WHATWG `Headers` layer rejects — `@dexpace/core` admits every printable ASCII + byte in a name, so `X Custom` is model-valid and unsendable — degrades to the same drop, never a + failed send (`TRANSPORT-12`). +- Every drop is logged by name (never by value) through the global logger, deduped per name by + default (`TRANSPORT-11`/`TRANSPORT-13`). - An abort that fires **after** `send()` resolved does not close the delivered body: the caller owns it (`SEAM-16`). Cancellation stays live for the whole in-flight window. - A timeout surfaces as the retryable `TransportFailureError`; a caller abort as the terminal `CancellationError` (`TRANSPORT-3`/`TRANSPORT-4`). A raw `DOMException` is never surfaced. +- A request `fetch` refused to make — an unsupported scheme such as `ftp://`, a forbidden method, an + argument its own validation rejects — is a bare `TypeError` outside the `IoError` tree, so + `retry/classify.ts`'s allow-list makes it non-retryable (`RETRY-2`). A failed *exchange* stays the + retryable `TransportFailureError` (`TRANSPORT-20`). The table that tells them apart is + `@dexpace/transport-shared`'s, shared with `@dexpace/transport-undici`, because the runtimes report + the same refusal in three different shapes (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. Node's `fetch` says so itself; Bun + 1.3.14's returns a live `ReadableStream` for all three, which this transport cancels and replaces + with `null` so the shape is the SDK's rather than the runtime's. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `fetchTransport()`, not a + failure on the first send (`HTTP-35`). ## Conformance diff --git a/packages/transport-fetch/src/fetch-transport.conformance.test.ts b/packages/transport-fetch/src/fetch-transport.conformance.test.ts index 4ab7aa1..260693b 100644 --- a/packages/transport-fetch/src/fetch-transport.conformance.test.ts +++ b/packages/transport-fetch/src/fetch-transport.conformance.test.ts @@ -11,4 +11,6 @@ runTransportConformanceSuite('fetchTransport', () => fetchTransport(), { supportsProxy: false, // TRANSPORT-11: `Connection` is a WHATWG forbidden request header, so fetch drops it either way. dropsConnectionHeader: true, + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => fetchTransport({defaultTimeoutMs: value}), }); diff --git a/packages/transport-fetch/src/fetch-transport.test.ts b/packages/transport-fetch/src/fetch-transport.test.ts index 8eea329..6b3c036 100644 --- a/packages/transport-fetch/src/fetch-transport.test.ts +++ b/packages/transport-fetch/src/fetch-transport.test.ts @@ -5,15 +5,20 @@ // its close owns nothing to release), // TRANSPORT-2 (no retrying/redirecting dispatcher is ever composed), TRANSPORT-15/16 // (close is a documented no-op), TRANSPORT-17/19 (single-use body written once, abandoned producer -// unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), TRANSPORT-30 +// unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), +// TRANSPORT-20 with RETRY-2 (a permanent misconfiguration is classified outside the IoError tree, +// a failed exchange inside it), TRANSPORT-9 (a producer that loses the race cancels the native call +// it raced, so no response is stranded), TRANSPORT-30 // (no proxy option exists at all), SEAM-30 (no producer is left running for its rejection to reach // Node's default unhandledRejection policy) import {describe, expect, test} from 'bun:test'; import { byteArrayBody, Headers, + isIoError, Request, streamBody, + TransportFailureError, type Body, } from '@dexpace/core'; import {fetchTransport} from './fetch-transport.js'; @@ -237,6 +242,64 @@ describe('fetchTransport request-body failures', () => { }); }); +describe('fetchTransport failure classification (TRANSPORT-20, RETRY-2)', () => { + /** Every shape a runtime's `fetch` uses to say "these arguments can never work". */ + const permanent: readonly (readonly [string, Error])[] = [ + // Node's undici-backed `fetch`, thrown out of the `Request` constructor: no cause, because no + // dispatch was ever attempted. + [ + 'a forbidden method', + new TypeError("'CONNECT' HTTP method is unsupported."), + ], + [ + 'a non-token method', + new TypeError("'BAD METHOD' is not a valid HTTP method."), + ], + // The same runtime's scheme refusal, which it can only report as a network error. + [ + 'an unsupported scheme', + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + ], + // Bun 1.3.14's shape for the same scheme refusal: a code, and no cause at all. + [ + "Bun's coded scheme refusal", + Object.assign(new TypeError('protocol must be http:, https: or s3:'), { + code: 'ERR_INVALID_ARG_VALUE', + }), + ], + ]; + + for (const [what, cause] of permanent) { + test(`${what} is terminal, outside the IoError tree`, async () => { + // A permanent misconfiguration classified as TransportFailureError is an IoError, and + // `classify.ts` returns true for every IoError -- so the caller's whole retry budget goes on + // re-proving it. The undici twin has refused its own equivalents since Phase 8a; this + // transport refused none of them until audit #67 / #82. + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TypeError); + expect(isIoError(error)).toBe(false); + expect((error as Error).cause).toBe(cause); + }); + } + + test('a network failure reported the same way stays retryable', async () => { + // The twin of the rows above: `fetch failed` is also how every genuine connect/DNS/TLS failure + // arrives, so the cause is the only discriminator and narrowing must not swallow this. + const cause = new TypeError('fetch failed', { + cause: Object.assign(new Error('getaddrinfo ENOTFOUND h.invalid'), { + code: 'ENOTFOUND', + }), + }); + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://h.invalid/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(isIoError(error)).toBe(true); + }); +}); + describe('fetchTransport lifecycle', () => { test('TRANSPORT-15/16: close is a no-op and send still works afterwards (SEAM-15)', async () => { const recorder = recordingFetch(); @@ -283,14 +346,86 @@ describe('fetchTransport lifecycle', () => { expect(recorder.calls.length).toBe(0); }); - test('defaultTimeoutMs applies when the call supplies no timeout of its own', async () => { - const recorder = recordingFetch(); + test('defaultTimeoutMs bounds a call that supplies no timeout of its own', async () => { + // Asserted through the outcome, not through "a signal was handed over": every send dispatches + // with a forked signal since audit #67 / #82, so the presence of one no longer discriminates. + // The double honours its signal the way a real `fetch` does, which is what makes the composed + // deadline observable. const transport = fetchTransport({ - fetch: recorder.fetch, - defaultTimeoutMs: 5_000, + fetch: (_input, init) => + new Promise((_resolve, reject) => { + init.signal?.addEventListener('abort', () => { + reject(init.signal?.reason as Error); + }); + }), + defaultTimeoutMs: 20, }); const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); - await (await transport.send(request)).close(); - expect(recorder.calls[0]?.signal).toBeInstanceOf(AbortSignal); + // TRANSPORT-4: a timeout is the retryable failure, never a cancellation, and the fork carries + // the source's `TimeoutError` reason through for `isTimeoutSignal` to read. + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + message: 'request timed out', + }); + }); +}); + +describe('fetchTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + test('a producer that loses the race takes the pending fetch down with it', async () => { + // The send rejects the moment `writeTo` fails, while `fetch` is still pending -- and until + // audit #67 / #82 nothing then cancelled it. With no caller signal and no timeout the transport + // dispatched with NO signal at all, so a response arriving afterwards was dropped with its body + // neither read nor cancelled: a connection held for as long as the pool would keep it. + let dispatched: AbortSignal | undefined; + let settled = false; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return new Promise(resolve => { + setTimeout(() => { + settled = true; + resolve(new globalThis.Response('late', {status: 200})); + }, 30); + }); + }, + }); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + // The producer's own classification, not the native table's: `writeTo` failing is a transport + // failure, and the send must not be waiting on `fetch` to say so. + expect(error).toBeInstanceOf(TransportFailureError); + expect(settled).toBe(false); + expect(dispatched?.aborted).toBe(true); + expect((dispatched?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + // The twin: `abort` is latched by `detach`, so the fork's second direction cannot become the + // very violation its first direction exists to prevent. + let dispatched: AbortSignal | undefined; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return Promise.resolve(new globalThis.Response('ok', {status: 200})); + }, + }); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const response = await transport.send(request); + expect(await response.text()).toBe('ok'); + expect(dispatched?.aborted).toBe(false); }); }); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index e76236d..784c33b 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -16,11 +16,14 @@ import { createDropLogger, degradeInboundHeaders, forkSignal, + hasNoResponseBody, isMaterializable, mapOutboundHeaders, materializeBody, producerFailure, pumpBody, + requireValidDefaultTimeoutMs, + toDispatchFailure, type ForkedSignal, type HeaderDropLogging, } from '@dexpace/transport-shared'; @@ -38,15 +41,29 @@ import { const REDIRECT_MODE = 'manual' as const; /** - * TRANSPORT-11's outbound drop set for this transport. `connection` is in it because WHATWG `fetch` - * treats it as a forbidden request header and would strip it silently — dropping it here makes the - * removal observable through the drop log instead. + * TRANSPORT-11's outbound drop set for this transport. + * + * `connection` is in it because WHATWG `fetch` treats it as a forbidden request header and would + * strip it silently — dropping it here makes the removal observable through the drop log instead. + * + * `expect`, `keep-alive` and `upgrade` are in it because the *implementations* do not honour the + * WHATWG forbidden-header list at all. Node's global `fetch` is undici-backed and undici's `Headers` + * deliberately does not implement forbidden names in a non-browser environment, so the three reach + * `lib/core/request.js:398,409` and reject the dispatch; `fetch()` then rejects with a bare + * `TypeError: fetch failed`, which this transport can only classify as the **retryable** + * `TransportFailureError` — a permanent misconfiguration that spends the caller's whole retry budget + * re-proving itself. Bun 1.3.14 diverges again: it forwards `expect` and `keep-alive` to the wire + * and hangs indefinitely on `upgrade`. Measured on both, 2026-09-05, audit #67 / #81. Neither + * outcome is TRANSPORT-12's, and a name in the drop set is the one behaviour that is. */ const FETCH_FORBIDDEN_HEADERS = [ 'content-length', 'host', 'transfer-encoding', 'connection', + 'expect', + 'keep-alive', + 'upgrade', ] as const; /** @@ -69,7 +86,13 @@ const MAX_MATERIALIZED_BODY_BYTES = 1_000_000; export interface FetchTransportOptions { /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ readonly headerDropLogging?: HeaderDropLogging; - /** A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. */ + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link fetchTransport} rather than by the first send (HTTP-35). + */ readonly defaultTimeoutMs?: number; /** A custom `fetch` implementation; defaults to `globalThis.fetch`. */ readonly fetch?: FetchLike; @@ -179,7 +202,16 @@ function adaptResponse( .status(Status.of(fetchResponse.status)) .reasonPhrase(fetchResponse.statusText || undefined) .headers(headers) - .body(fetchResponse.body) + // Decided here, not inherited from the runtime. Node's `fetch` returns `null` for 204, 304 + // and HEAD as the spec requires, and Bun 1.3.14's returns a live `ReadableStream` for all + // three (measured 2026-09-05) -- so forwarding `fetchResponse.body` made the SHAPE of a + // body-less response a property of the runtime rather than of this SDK. `#exchange` releases + // whatever handle this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, fetchResponse.status) + ? null + : fetchResponse.body, + ) .build() ); } @@ -198,6 +230,9 @@ class FetchTransport implements Transport { readonly #defaultTimeoutMs: number | undefined; constructor(options: FetchTransportOptions) { + // Before anything else: a default no `AbortSignal.timeout()` can take is a caller error, and + // HTTP-35 puts it where it was supplied rather than at the first send (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); this.#logDrops = createDropLogger( options.headerDropLogging ?? 'first-per-name', ); @@ -246,7 +281,13 @@ class FetchTransport implements Transport { try { // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. - return adaptResponse(request, fetchResponse, this.#logDrops); + const response = adaptResponse(request, fetchResponse, this.#logDrops); + if (response.body === null && fetchResponse.body !== null) { + // A runtime handed a body for a response that cannot have one. Nothing references it any + // more, so releasing it is this transport's, not the caller's (TRANSPORT-25, SEAM-30). + await fetchResponse.body.cancel().catch(() => undefined); + } + return response; } catch (error) { await fetchResponse.body?.cancel().catch(() => undefined); // TRANSPORT-19: nothing is delivered on this path either, so the producer is owed its teardown @@ -270,7 +311,7 @@ class FetchTransport implements Transport { }; if (prepared.init !== undefined) init.body = prepared.init; if (prepared.duplex !== undefined) init.duplex = prepared.duplex; - if (signal !== undefined) init.signal = signal; + init.signal = signal; try { // Raced, not sequenced: a producer failure must surface even while `fetch` is still pending, @@ -280,12 +321,23 @@ class FetchTransport implements Transport { producerFailure(prepared.done), ]); } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const abortedByCaller = signal.aborted; await prepared.abandon(error); - if (signal?.aborted) throw abortToSdkError(signal, error); - throw new TransportFailureError( - error instanceof Error ? error.message : 'fetch failed', - {cause: error}, - ); + if (abortedByCaller) throw abortToSdkError(signal, error); + // TRANSPORT-9: when the producer lost the race, `fetch` is still pending. Nothing awaits it + // any more, so a response that arrives later would be dropped with its body neither read nor + // cancelled -- a leaked connection for as long as the pool keeps it. Pulling the fork takes + // the native call down instead. On the path where `fetch` itself rejected there is nothing + // left to cancel and this is inert (audit #67 / #82). + plan.fork.abort(error); + // TRANSPORT-20 versus RETRY-2, decided by the table in `@dexpace/transport-shared` rather + // than here: until audit #67 / #82 every native rejection became `TransportFailureError`, + // which `classify.ts` reports retryable for being an `IoError`, so an `ftp://` URL or a + // `CONNECT` method spent the caller's whole retry budget re-proving a permanent + // misconfiguration. The undici twin already refused those; the two must not disagree. + throw toDispatchFailure(error, 'fetch failed'); } } @@ -335,6 +387,9 @@ if (typeof Symbol.asyncDispose === 'symbol') { * * @param options - optional transport settings. * @returns a transport ready to send; release it with `close()`. + * @throws `TypeError` when `defaultTimeoutMs` is not an integer number of milliseconds in + * `1 .. 2**32 - 1` — `AbortSignal.timeout()`'s range, and therefore the only one a per-call + * deadline can be built from. * * @public */ diff --git a/packages/transport-shared/README.md b/packages/transport-shared/README.md index dc04786..91242ee 100644 --- a/packages/transport-shared/README.md +++ b/packages/transport-shared/README.md @@ -18,5 +18,8 @@ other, not merely of the rest of the tree. | `header-mapping.ts` | `TRANSPORT-10`/`TRANSPORT-12`'s outbound drop-and-degrade pass and `TRANSPORT-14`'s lenient inbound copy, which preserves obs-text values rather than rejecting them | | `drop-log.ts` | `TRANSPORT-13`'s bounded, case-insensitive, drain-to-cap dedup of already-logged drop names. Names only — never values | | `abort-mapping.ts` | The single mapping from an aborted signal to a canonical SDK error: `TransportFailureError` on timeout, `CancellationError` otherwise. A raw `DOMException` is never surfaced | -| `body-pump.ts` | Turning a `Body` into a request stream the transport owns the closing of, plus `TRANSPORT-19`'s idempotent teardown for an abandoned producer | -| `signal-fork.ts` | `SEAM-16`'s abort-after-delivery rule: both native clients tie a response body's lifetime to the signal they were given, so the transport dispatches over a fork it detaches at delivery | +| `dispatch-classification.ts` | The single mapping from a *native rejection* to one: the retryable `TransportFailureError` `TRANSPORT-20` requires for a failed exchange, and a bare `TypeError` outside the `IoError` tree for a request the client refused to make. An allow-list, so an unrecognised rejection stays retryable | +| `body-less.ts` | Which method/status pairs can carry no response body at all, so `Response.body` is `null` for a 204, a 304 or a HEAD on every runtime rather than on whichever ones agree with the spec | +| `default-timeout.ts` | `HTTP-35`'s range check for a transport-wide default timeout: an integer in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s and therefore every transport's | +| `body-pump.ts` | Turning a `Body` into a request stream the transport owns the closing of, plus `TRANSPORT-19`'s idempotent teardown for an abandoned producer, and the classification of a producer's own failure | +| `signal-fork.ts` | `SEAM-16`'s abort-after-delivery rule: both native clients tie a response body's lifetime to the signal they were given, so the transport dispatches over a fork it detaches at delivery — and `TRANSPORT-9`'s other direction, a handle the transport pulls to cancel a native call it has abandoned | diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md index 2ee005e..4ac3d3d 100644 --- a/packages/transport-shared/etc/transport-shared.api.md +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -4,9 +4,10 @@ ```ts -import type { Body as Body_2 } from '@dexpace/core'; +import { Body as Body_2 } from '@dexpace/core'; import { DexpaceError } from '@dexpace/core'; import { Headers as Headers_2 } from '@dexpace/core'; +import type { Method } from '@dexpace/core'; // Warning: (ae-internal-missing-underscore) The name "abortToSdkError" should be prefixed with an underscore because the declaration is marked as @internal // @@ -39,8 +40,9 @@ export function degradeInboundHeaders(raw: Iterable): // // @internal export interface ForkedSignal { + abort(reason: unknown): void; detach(): void; - readonly signal: AbortSignal | undefined; + readonly signal: AbortSignal; } // Warning: (ae-internal-missing-underscore) The name "forkSignal" should be prefixed with an underscore because the declaration is marked as @internal @@ -48,6 +50,11 @@ export interface ForkedSignal { // @internal export function forkSignal(source: AbortSignal | undefined): ForkedSignal; +// Warning: (ae-internal-missing-underscore) The name "hasNoResponseBody" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function hasNoResponseBody(method: Method, status: number): boolean; + // Warning: (ae-internal-missing-underscore) The name "HeaderDropLogging" should be prefixed with an underscore because the declaration is marked as @internal // // @internal @@ -58,6 +65,11 @@ export type HeaderDropLogging = 'all' | 'first-per-name' | 'quiet'; // @internal export function isMaterializable(body: Body_2, maxBytes: number): boolean; +// Warning: (ae-internal-missing-underscore) The name "isPermanentDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function isPermanentDispatchFailure(error: unknown): boolean; + // Warning: (ae-internal-missing-underscore) The name "mapOutboundHeaders" should be prefixed with an underscore because the declaration is marked as @internal // // @internal @@ -88,6 +100,16 @@ export function producerFailure(done: Promise | undefined): Promise // @internal export function pumpBody(body: Body_2): BodyPump; +// Warning: (ae-internal-missing-underscore) The name "requireValidDefaultTimeoutMs" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function requireValidDefaultTimeoutMs(value: number | undefined): void; + +// Warning: (ae-internal-missing-underscore) The name "toDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function toDispatchFailure(error: unknown, fallbackMessage: string): Error; + // (No @packageDocumentation comment for this package) ``` diff --git a/packages/transport-shared/src/body-less.test.ts b/packages/transport-shared/src/body-less.test.ts new file mode 100644 index 0000000..a7ff678 --- /dev/null +++ b/packages/transport-shared/src/body-less.test.ts @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.test.ts +// Exercises: TRANSPORT-24 (every status is surfaced faithfully, including the ones that carry no +// body), TRANSPORT-25 (a response whose body a transport declines to expose still has its native +// handle released), TRANSPORT-27 (an absent length is the unknown-length case, not a failure) +import {describe, expect, test} from 'bun:test'; +import type {Method} from '@dexpace/core'; +import {hasNoResponseBody} from './body-less.js'; + +describe('hasNoResponseBody', () => { + test('the WHATWG null-body statuses carry none, whatever the method', () => { + for (const status of [101, 103, 204, 205, 304]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + true, + ]); + expect([status, hasNoResponseBody('POST', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('an ordinary status carries one', () => { + for (const status of [200, 201, 206, 302, 400, 404, 500, 520]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + false, + ]); + } + }); + + test('HEAD never carries one, whatever the status', () => { + // The Content-Length of a HEAD response describes the body a GET would have returned, so a + // transport that framed a stream from it would hand the caller a read that never completes. + for (const status of [200, 206, 404, 500]) { + expect([status, hasNoResponseBody('HEAD', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('a 2xx CONNECT is a tunnel, a failed CONNECT is an ordinary error response', () => { + expect(hasNoResponseBody('CONNECT', 200)).toBe(true); + expect(hasNoResponseBody('CONNECT', 299)).toBe(true); + expect(hasNoResponseBody('CONNECT', 407)).toBe(false); + expect(hasNoResponseBody('CONNECT', 502)).toBe(false); + }); + + test('every other method the model admits is decided by the status alone', () => { + const methods: readonly Method[] = [ + 'GET', + 'POST', + 'PUT', + 'DELETE', + 'OPTIONS', + 'TRACE', + 'PATCH', + ]; + for (const method of methods) { + expect([method, hasNoResponseBody(method, 204)]).toEqual([method, true]); + expect([method, hasNoResponseBody(method, 200)]).toEqual([method, false]); + } + }); +}); diff --git a/packages/transport-shared/src/body-less.ts b/packages/transport-shared/src/body-less.ts new file mode 100644 index 0000000..6cbdb5a --- /dev/null +++ b/packages/transport-shared/src/body-less.ts @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.ts +import type {Method} from '@dexpace/core'; + +/** + * The statuses that can never carry a body, whatever the request was: WHATWG fetch's null-body + * status set, which is RFC 9110's own list of body-less statuses (`101`, `103`, `204`, `205`, `304`). + * A `Content-Length` on one of them describes the body a `200` would have had and frames nothing. + */ +const NULL_BODY_STATUSES: ReadonlySet = new Set([ + 101, 103, 204, 205, 304, +]); + +/** The lower and upper bounds of the 2xx range, inside which a `CONNECT` response is body-less. */ +const OK_MIN = 200; +const OK_MAX = 299; + +/** + * Whether the response to `method` with `status` can carry a body at all. + * + * The WHATWG rule, and what `@dexpace/core`'s model already types: `Response.body` is + * `ReadableStream | null` (`http/response.ts:18`), and `null` is what a consumer can + * branch on without reading. Both shipped adapters apply this rather than forwarding whatever their + * native client happened to produce, because three of the four combinations disagreed until audit + * #67 / #82: + * + * - undici's dispatcher always hands back a `BodyReadable`, so `@dexpace/transport-undici` wrapped + * an empty stream for 204, 304 and HEAD alike; + * - Node's global `fetch` returns `null` for all three, per the spec; + * - Bun 1.3.14's `fetch` returns a live `ReadableStream` for all three (measured 2026-09-05), so + * `@dexpace/transport-fetch` inherited the runtime's answer rather than the contract's. + * + * A transport that decides here instead reports the same shape on every runtime, which is what a + * conformance row can assert. Whatever native handle it then declines to expose is its own to + * release — an undrained `BodyReadable` holds the pooled connection open (TRANSPORT-25, SEAM-30). + * + * @param method - the request method; always a canonical uppercase token (HTTP-9). + * @param status - the response status code as the server sent it. + * @returns `true` when the adapted response must carry `body === null`. + * + * @internal + */ +export function hasNoResponseBody(method: Method, status: number): boolean { + if (method === 'HEAD') return true; + // A 2xx CONNECT switches the connection to a tunnel; anything after the blank line is tunnelled + // bytes, not a body. A non-2xx CONNECT is an ordinary error response and may carry one. + if (method === 'CONNECT') return status >= OK_MIN && status <= OK_MAX; + return NULL_BODY_STATUSES.has(status); +} diff --git a/packages/transport-shared/src/body-pump.ts b/packages/transport-shared/src/body-pump.ts index a7e7f6d..f657484 100644 --- a/packages/transport-shared/src/body-pump.ts +++ b/packages/transport-shared/src/body-pump.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/body-pump.ts -import type {Body} from '@dexpace/core'; +import {TransportFailureError, type Body} from '@dexpace/core'; /** * A streaming request body in flight: the stream to hand the native client, the producer's own @@ -81,9 +81,17 @@ export function pumpBody(body: Body): BodyPump { * unhandled one. Without it that late rejection reaches Node's default `unhandledRejection` policy * and takes the process down — the exact hazard SEAM-30 names, arriving from the request side. * + * The rejection is classified **here**, as the retryable `TransportFailureError` both transports + * already reported for it, and not left raw for the caller's catch to guess at. A body that could + * not be written is a failure of this layer, and the catch that receives it is the one that also + * receives the *native* client's rejections — which since audit #67 / #82 go through a table that + * can call a bare `TypeError` a permanent misconfiguration. A producer that happened to throw one + * would have been read as the wire refusing the request. Classifying at the source is what + * `prepareBody`'s buffered branch already does with the same failure. + * * @param done - the producer settlement from {@link pumpBody}, or `undefined` when the body was not * streamed. - * @returns a promise that rejects with the producer's failure and never resolves. + * @returns a promise that rejects with the producer's failure, wrapped, and never resolves. * * @internal */ @@ -91,9 +99,18 @@ export function producerFailure( done: Promise | undefined, ): Promise { if (done === undefined) return new Promise(() => undefined); - // `then` with no rejection handler: a producer *success* says nothing about the response, so the - // derived promise only ever carries the failure onward. - return done.then(() => new Promise(() => undefined)); + return done.then( + // A producer *success* says nothing about the response, so only the failure is carried onward. + () => new Promise(() => undefined), + (cause: unknown) => { + throw new TransportFailureError( + cause instanceof Error + ? cause.message + : 'request body could not be written', + {cause}, + ); + }, + ); } /** diff --git a/packages/transport-shared/src/default-timeout.test.ts b/packages/transport-shared/src/default-timeout.test.ts new file mode 100644 index 0000000..a4637ef --- /dev/null +++ b/packages/transport-shared/src/default-timeout.test.ts @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.test.ts +// Exercises: HTTP-35 (a timeout outside the range a transport can honour is refused where it was +// supplied, not where it is used), TRANSPORT-5 (a per-call override replaces a transport default, +// so the default is a real configuration value and answerable for its own range) +import {describe, expect, test} from 'bun:test'; +import {isIoError} from '@dexpace/core'; +import {requireValidDefaultTimeoutMs} from './default-timeout.js'; + +describe('requireValidDefaultTimeoutMs', () => { + test('accepts undefined and every integer in the honourable range', () => { + for (const value of [undefined, 1, 50, 30_000, 2 ** 32 - 1]) { + expect(() => { + requireValidDefaultTimeoutMs(value); + }).not.toThrow(); + } + }); + + test('refuses everything AbortSignal.timeout() cannot take', () => { + // The full range, not merely its lower bound. `1.5` and `2**32` are the two Bun 1.3.14 accepts + // and Node rejects with a `RangeError`, which is the divergence that made an unvalidated + // default a per-runtime behaviour rather than a per-caller error. + for (const value of [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, + ]) { + let thrown: unknown; + try { + requireValidDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect([value, thrown instanceof TypeError]).toEqual([value, true]); + // Outside the `IoError` tree, like every other construction-time refusal these transports + // raise: nothing retries a factory, and one class for all of them is easier to catch. + expect([value, isIoError(thrown)]).toEqual([value, false]); + // "Discoverable": the message names the value that was refused, not merely that one was. + expect((thrown as Error).message).toContain(String(value)); + } + }); + + test('the message names the range as well as the value', () => { + expect(() => { + requireValidDefaultTimeoutMs(0); + }).toThrow('1..4294967295'); + }); +}); diff --git a/packages/transport-shared/src/default-timeout.ts b/packages/transport-shared/src/default-timeout.ts new file mode 100644 index 0000000..afa6907 --- /dev/null +++ b/packages/transport-shared/src/default-timeout.ts @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.ts + +/** + * `AbortSignal.timeout()`'s upper bound, and therefore every transport's. Duplicated from + * `@dexpace/core`'s `http/request-options.ts:12`, which is not exported: the two must agree, and + * `RequestOptionsBuilder.timeoutMs`'s own rejection message is the wording copied below so a caller + * who trips either one reads the same sentence. + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + +/** + * Rejects a transport-wide default timeout that no transport could honour. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range the thing this value ends up in accepts. `RequestOptionsBuilder.timeoutMs` has checked + * exactly this since audit #67 / #76, on HTTP-35's reading that a timeout a setter accepted and a + * transport then refused is a failure belonging at the call site. `defaultTimeoutMs` was left + * unchecked on both transports and so became the last path by which `1.5`, `0` or `2**32` reached + * `composeSignal` — where Node throws `RangeError` and Bun 1.3.14 accepts the first two, so the same + * misconfiguration was a failed send on one runtime and a silently different deadline on the other + * (audit #67 / #82). + * + * A `TypeError`, matching the construction-time refusals both transports already raise for a + * caller misconfiguration (`undiciTransport`'s two) and deliberately outside the `IoError` tree — + * though nothing retries a factory, the conformance row asserts the same shape for both, and a + * transport is easier to reason about when every construction-time refusal is one class. + * + * @param value - the configured default, or `undefined` for none. + * @throws `TypeError` when a defined value is zero, negative, not finite, not an integer, or + * greater than `2**32 - 1`. + * + * @internal + */ +export function requireValidDefaultTimeoutMs(value: number | undefined): void { + if ( + value === undefined || + (Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { + return; + } + throw new TypeError( + `defaultTimeoutMs must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, ` + + `got ${String(value)}: it is handed to AbortSignal.timeout(), which accepts nothing else`, + ); +} diff --git a/packages/transport-shared/src/dispatch-classification.test.ts b/packages/transport-shared/src/dispatch-classification.test.ts new file mode 100644 index 0000000..9a61ef1 --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.test.ts @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.test.ts +// Exercises: TRANSPORT-20 (a failure that produced no response is the retryable transport failure), +// RETRY-2 (the retryable set is an allow-list, so a permanent misconfiguration outside the IoError +// tree is non-retryable for free), TRANSPORT-8 (an argument the native client can never accept is +// told apart from an exchange that failed) +import {describe, expect, test} from 'bun:test'; +import {isIoError, TransportFailureError} from '@dexpace/core'; +import { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; + +/** An error carrying a native `code`, the shape undici and Bun both attach one to. */ +function coded(message: string, code: string): Error { + return Object.assign(new Error(message), {code}); +} + +describe('isPermanentDispatchFailure', () => { + test("undici's two argument-validation codes are permanent", () => { + expect( + isPermanentDispatchFailure( + coded( + 'Invalid URL protocol: the URL must start with `http:` or `https:`.', + 'UND_ERR_INVALID_ARG', + ), + ), + ).toBe(true); + expect( + isPermanentDispatchFailure(coded('expect', 'UND_ERR_NOT_SUPPORTED')), + ).toBe(true); + }); + + test("Bun's coded fetch refusal for an unsupported scheme is permanent", () => { + // Bun 1.3.14, measured: `fetch('ftp://…')` rejects with this exact shape, where Node's + // undici-backed `fetch` rejects with `fetch failed` and an `unknown scheme` cause instead. + const error = Object.assign( + new TypeError('protocol must be http:, https: or s3:'), + {code: 'ERR_INVALID_ARG_VALUE'}, + ); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test("a causeless TypeError is undici's argument validation, so it is permanent", () => { + // Node's global `fetch` throws these out of the `Request` constructor, before any dispatch. + for (const message of [ + "'CONNECT' HTTP method is unsupported.", + "'BAD METHOD' is not a valid HTTP method.", + 'Request with GET/HEAD method cannot have body.', + ]) { + expect(isPermanentDispatchFailure(new TypeError(message))).toBe(true); + } + }); + + test('a scheme refusal reported through `fetch failed` is permanent', () => { + const error = new TypeError('fetch failed', { + cause: new Error('unknown scheme'), + }); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test('a network failure reported through `fetch failed` is NOT permanent', () => { + const error = new TypeError('fetch failed', { + cause: coded('getaddrinfo ENOTFOUND example.invalid', 'ENOTFOUND'), + }); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a blocked port stays retryable (TRANSPORT-20 probes one by name)', () => { + // `http://127.0.0.1:1` is the dead-port probe §17 names for TRANSPORT-20, and port 1 is on + // WHATWG's blocked list, so Node's `fetch` refuses it before connecting and says so in the + // cause. Classifying that reason as permanent would turn the SDK's headline retryable case + // terminal, which is why the reason table excludes it explicitly. + const error = new TypeError('fetch failed', {cause: new Error('bad port')}); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a plain connection failure and a non-Error rejection stay retryable', () => { + expect( + isPermanentDispatchFailure( + coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'), + ), + ).toBe(false); + expect(isPermanentDispatchFailure('a string nobody typed')).toBe(false); + }); +}); + +describe('toDispatchFailure', () => { + test('a permanent misconfiguration is a TypeError outside the IoError tree', () => { + const cause = coded('invalid request method', 'UND_ERR_INVALID_ARG'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TypeError); + // RETRY-2's allow-list is what makes this non-retryable; the class is how it stays outside it. + expect(isIoError(mapped)).toBe(false); + expect(mapped.cause).toBe(cause); + }); + + test('an exchange failure is the retryable TransportFailureError, cause intact', () => { + const cause = coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(isIoError(mapped)).toBe(true); + expect(mapped.message).toBe('connect ECONNREFUSED 127.0.0.1:1'); + expect(mapped.cause).toBe(cause); + }); + + test('a permanent verdict taken from the cause names the cause in its message', () => { + // `fetch failed` names nothing; the reason that made the verdict is the useful half. + const mapped = toDispatchFailure( + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + 'fetch failed', + ); + expect(mapped.message).toBe('fetch failed: unknown scheme'); + }); + + test('an error already in the SDK vocabulary is passed through untouched', () => { + // A request-body producer failure racing the dispatch arrives here already classified; the + // table knows nothing about the producer and must not re-answer for it. + const already = new TransportFailureError('producer exploded'); + expect(toDispatchFailure(already, 'fetch failed')).toBe(already); + }); + + test('a non-Error rejection falls back to the caller-supplied message', () => { + const mapped = toDispatchFailure(Symbol('nope'), 'fetch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(mapped.message).toBe('fetch failed'); + }); +}); diff --git a/packages/transport-shared/src/dispatch-classification.ts b/packages/transport-shared/src/dispatch-classification.ts new file mode 100644 index 0000000..cd838de --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.ts @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.ts +import {DexpaceError, TransportFailureError} from '@dexpace/core'; + +/** + * Error codes a native client uses for "these arguments can never work", as opposed to "this + * exchange failed". + * + * `UND_ERR_INVALID_ARG` and `UND_ERR_NOT_SUPPORTED` are undici's two argument-validation codes, + * raised by `Dispatcher.request` before a socket is touched (`lib/core/errors.js` in 6.28.0): a + * non-`http(s)` origin, `CONNECT` as a method, a non-token method, a per-request + * `Proxy-Authorization` on a `ProxyAgent`. `ERR_INVALID_ARG_VALUE`, `ERR_INVALID_ARG_TYPE` and + * `ERR_INVALID_URL` are the Node-style codes Bun's `fetch` sets on the same class of refusal — Bun + * 1.3.14 rejects an `ftp://` URL with a `TypeError` carrying `ERR_INVALID_ARG_VALUE`, where Node's + * undici-backed `fetch` rejects with a causing network error instead. Measured on both, 2026-09-05. + */ +const TERMINAL_ARGUMENT_CODES: ReadonlySet = new Set([ + 'UND_ERR_INVALID_ARG', + 'UND_ERR_NOT_SUPPORTED', + 'ERR_INVALID_ARG_VALUE', + 'ERR_INVALID_ARG_TYPE', + 'ERR_INVALID_URL', +]); + +/** + * WHATWG network-error reasons that describe the *request* rather than the exchange. + * + * undici's `fetch` funnels every failure into one `TypeError('fetch failed', {cause})` + * (`lib/web/fetch/index.js:230`), so the top-level error cannot tell a refused scheme from a + * refused connection — the cause's message is the only discriminator the runtime offers. These + * three are `makeNetworkError` reasons raised before any dispatch (`:620`, `:793`, `:962` in + * 6.28.0); a scheme this SDK cannot speak is the same permanent misconfiguration undici's + * *dispatcher* reports as `UND_ERR_INVALID_ARG`. + * + * `'bad port'` is deliberately **not** here. WHATWG blocks a fixed list of ports, `1` among them, + * so on Node's `fetch` the canonical dead-port probe (`http://127.0.0.1:1`) arrives with that + * reason — and TRANSPORT-20's own conformance sentence is "connect to a dead port; assert the + * retryable type". Adding it would turn that row, and the SDK's headline retryable case, terminal. + */ +const TERMINAL_NETWORK_REASONS: ReadonlySet = new Set([ + 'unknown scheme', + 'URL scheme must be a HTTP(S) scheme', + 'about scheme is not supported', +]); + +function errorCode(error: unknown): string | undefined { + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' ? code : undefined; +} + +function hasTerminalCode(error: unknown): boolean { + const code = errorCode(error); + return code !== undefined && TERMINAL_ARGUMENT_CODES.has(code); +} + +/** + * Whether a native rejection is a permanent misconfiguration rather than a failed exchange. + * + * Three positive recognitions, in one place so the two shipped adapters cannot answer differently + * for the same condition (the `ftp://` row asserts they do not). Everything else falls through to + * retryable, which is both the safe default and the behaviour every adapter had before audit #67 / + * #82 — TRANSPORT-20 makes "no response was produced" a MUST-retryable, so a rejection this table + * does not recognise must stay one. + * + * 1. A **terminal argument code** on the error or its immediate cause, per the + * `TERMINAL_ARGUMENT_CODES` table above. This is the whole undici-dispatcher leg, and Bun's + * `fetch`. + * 2. A **`TypeError` with no `cause`**. undici's `fetch` — which is also Node's global `fetch` — + * builds every *network* rejection with a cause, and every argument rejection as a bare + * `TypeError` thrown out of the `Request`/`Headers` constructors before a dispatch is attempted: + * an unsupported method, a non-token method, a body on a GET. The presence of a cause is + * therefore the runtime's own line between the two, and it needs no message matching. + * 3. A cause whose message is one of the `TERMINAL_NETWORK_REASONS` above — the scheme refusals + * that undici's `fetch` can only report through its fixed `fetch failed` message. + * + * @param error - whatever the native call rejected with. + * @returns `true` when no retry of the same request could succeed. + * + * @internal + */ +export function isPermanentDispatchFailure(error: unknown): boolean { + if (hasTerminalCode(error)) return true; + if (!(error instanceof Error)) return false; + const {cause} = error; + if (error instanceof TypeError && cause === undefined) return true; + if (hasTerminalCode(cause)) return true; + return cause instanceof Error && TERMINAL_NETWORK_REASONS.has(cause.message); +} + +/** + * The message to put on the mapped error: the native message, plus the cause's when the native + * layer's own message is a fixed placeholder. `fetch failed` names nothing on its own, and the + * reason that made the verdict permanent is the only useful thing to say. + */ +function describe(error: unknown, fallbackMessage: string): string { + if (!(error instanceof Error)) return fallbackMessage; + const {cause} = error; + if (!(cause instanceof Error) || error.message.includes(cause.message)) { + return error.message; + } + return `${error.message}: ${cause.message}`; +} + +/** + * Maps one native dispatch rejection onto the SDK's error vocabulary. + * + * A permanent misconfiguration becomes a bare `TypeError` carrying the native error as `cause`, + * deliberately **outside** the `IoError` tree: `retry/classify.ts` is an allow-list that returns + * `true` for every `IoError`, so a condition no retry can fix is non-retryable for free (RETRY-2), + * and `TypeError` is already what both transports raise for a caller misconfiguration caught at + * construction. Anything else becomes the retryable `TransportFailureError` TRANSPORT-20 requires. + * + * An error that already descends from `DexpaceError` is returned unchanged: it was classified at + * its own source — a request-body producer failure racing the dispatch is the live case — and + * re-classifying it here would answer for a layer this table knows nothing about. + * + * @param error - whatever the native call rejected with. + * @param fallbackMessage - the message to use when the rejection is not an `Error` at all. + * @returns the error to throw; the caller always throws it. + * + * @internal + */ +export function toDispatchFailure( + error: unknown, + fallbackMessage: string, +): Error { + if (error instanceof DexpaceError) return error; + if (isPermanentDispatchFailure(error)) { + return new TypeError(describe(error, fallbackMessage), {cause: error}); + } + return new TransportFailureError( + error instanceof Error ? error.message : fallbackMessage, + {cause: error}, + ); +} diff --git a/packages/transport-shared/src/header-mapping.test.ts b/packages/transport-shared/src/header-mapping.test.ts index 4ee6219..6d02976 100644 --- a/packages/transport-shared/src/header-mapping.test.ts +++ b/packages/transport-shared/src/header-mapping.test.ts @@ -4,7 +4,11 @@ // TRANSPORT-12 (per-header graceful degradation), TRANSPORT-14 (lenient inbound copy, obs-text preserved, control-byte header dropped) import {describe, expect, test} from 'bun:test'; import {Headers} from '@dexpace/core'; -import {degradeInboundHeaders, mapOutboundHeaders} from './header-mapping.js'; +import { + CONTROL_BYTE, + degradeInboundHeaders, + mapOutboundHeaders, +} from './header-mapping.js'; describe('mapOutboundHeaders', () => { test('drops framing headers the native client computes', () => { @@ -67,7 +71,47 @@ describe('mapOutboundHeaders graceful degradation (TRANSPORT-12)', () => { }); }); +describe('CONTROL_BYTE (TRANSPORT-14)', () => { + // Asserted on the character class itself, not through `degradeInboundHeaders`, because the two + // gates that reject an inbound value are redundant by construction: the regex below and + // `Headers.addInbound`'s own `hasForbiddenInboundValueByte`, whose throw the `try`/`catch` turns + // into the same drop. A value only the second one caught looked identical from outside, which is + // how `\x0A` stayed out of this class from Phase 8a to audit #67 / #82. + test('every C0 control byte except HTAB is refused, LF included', () => { + for (let code = 0x00; code <= 0x1f; code += 1) { + const value = `v${String.fromCharCode(code)}alue`; + expect([code, CONTROL_BYTE.test(value)]).toEqual([code, code !== 0x09]); + } + expect(CONTROL_BYTE.test('v\x7falue')).toBe(true); + }); + + test('LF is refused whether or not it is preceded by CR', () => { + // The obs-fold shape (`\r\n ` continuation) and a bare LF are both header injection on the + // inbound path; RFC 9110 5.5 forbids either from reaching a field value. + expect(CONTROL_BYTE.test('one\nvalue')).toBe(true); + expect(CONTROL_BYTE.test('one\r\n two')).toBe(true); + }); + + test('HTAB and obs-text are carried, not refused', () => { + // TRANSPORT-14's own SHOULD: a non-ASCII byte in a value is preserved rather than stripped, and + // HTAB is legal whitespace inside a field value (RFC 9110 5.5). + expect(CONTROL_BYTE.test('one\ttwo')).toBe(false); + expect(CONTROL_BYTE.test('café')).toBe(false); + }); +}); + describe('degradeInboundHeaders', () => { + test('drops a header whose value carries a line feed, keeps the rest', () => { + const {headers, dropped} = degradeInboundHeaders([ + ['x-injected', 'value\nx-forged: yes'], + ['x-good', 'value'], + ]); + expect(headers.get('x-injected')).toBeUndefined(); + expect(headers.get('x-forged')).toBeUndefined(); + expect(headers.get('x-good')).toBe('value'); + expect(dropped).toEqual(['x-injected']); + }); + test('drops a header whose value carries a control byte, keeps the rest', () => { const {headers, dropped} = degradeInboundHeaders([ ['x-bad', 'v\x01alue'], diff --git a/packages/transport-shared/src/header-mapping.ts b/packages/transport-shared/src/header-mapping.ts index fbf72db..d570e4c 100644 --- a/packages/transport-shared/src/header-mapping.ts +++ b/packages/transport-shared/src/header-mapping.ts @@ -2,8 +2,25 @@ // packages/transport-shared/src/header-mapping.ts import {Headers} from '@dexpace/core'; +/** + * Every byte TRANSPORT-14 refuses in an *inbound* header value: the C0 controls except HTAB, plus + * DEL. Deliberately the same character class as `@dexpace/core`'s `hasForbiddenInboundValueByte`, + * which `Headers.addInbound` applies a few lines later \u2014 obs-text (\u2265 0x80) is carried, HTAB is + * carried, everything else below 0x20 is not. + * + * `\x0A` was missing from Phase 8a until audit #67 / #82: the class read `\x0B-\x1F`, excepting LF + * alongside the intended HTAB. Nothing observable changed, because `addInbound` rejected the value + * anyway and the `try`/`catch` in {@link degradeInboundHeaders} recorded the same drop \u2014 which is + * exactly why it survived, and why the test for this constant reads the class directly rather than + * going through that function. + * + * Exported for that test only. The package barrel deliberately does not re-export it: it is one + * half of a redundant pair, not plumbing another transport should reach for. + * + * @internal + */ /* eslint-disable no-control-regex -- RFC 9110 requires testing for ASCII control characters */ -const CONTROL_BYTE = /[\x00-\x08\x0B-\x1F\x7F]/u; +export const CONTROL_BYTE = /[\x00-\x08\x0A-\x1F\x7F]/u; const NON_ASCII_OR_CONTROL = /[\x00-\x1F\x7F-\uFFFF]/u; /* eslint-enable no-control-regex -- re-enable */ diff --git a/packages/transport-shared/src/index.ts b/packages/transport-shared/src/index.ts index 9c88c43..ebfcf58 100644 --- a/packages/transport-shared/src/index.ts +++ b/packages/transport-shared/src/index.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/index.ts export {abortToSdkError} from './abort-mapping.js'; +export {hasNoResponseBody} from './body-less.js'; export { isMaterializable, materializeBody, @@ -8,6 +9,11 @@ export { pumpBody, type BodyPump, } from './body-pump.js'; +export {requireValidDefaultTimeoutMs} from './default-timeout.js'; +export { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; export {createDropLogger, type HeaderDropLogging} from './drop-log.js'; export { degradeInboundHeaders, diff --git a/packages/transport-shared/src/signal-fork.test.ts b/packages/transport-shared/src/signal-fork.test.ts index 6fa63e8..37ff026 100644 --- a/packages/transport-shared/src/signal-fork.test.ts +++ b/packages/transport-shared/src/signal-fork.test.ts @@ -1,14 +1,19 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/signal-fork.test.ts // Exercises: SEAM-16 (an abort after delivery must not reach the native client), SEAM-13/TRANSPORT-7 -// (an abort before delivery must) +// (an abort before delivery must), TRANSPORT-9 (the transport can cancel a native call it abandons, +// with or without a caller signal) import {describe, expect, test} from 'bun:test'; import {forkSignal} from './signal-fork.js'; describe('forkSignal', () => { - test('returns no signal when the caller supplied none', () => { + test('still yields a live signal when the caller supplied none', () => { + // The transport's own cancellation handle. Until audit #67 / #82 this returned `undefined`, so + // a send with no caller signal and no timeout dispatched with none -- and a request-body + // producer that failed mid-flight had no way to take the native call down with it. const fork = forkSignal(undefined); - expect(fork.signal).toBeUndefined(); + expect(fork.signal).toBeInstanceOf(AbortSignal); + expect(fork.signal.aborted).toBe(false); expect(() => { fork.detach(); }).not.toThrow(); @@ -19,15 +24,17 @@ describe('forkSignal', () => { const fork = forkSignal(controller.signal); const reason = new Error('caller changed their mind'); controller.abort(reason); - expect(fork.signal?.aborted).toBe(true); - expect(fork.signal?.reason).toBe(reason); + expect(fork.signal.aborted).toBe(true); + // Carried verbatim because `isTimeoutSignal` reads `reason.name`: a fork that invented its own + // reason would turn every per-call timeout into a CancellationError (TRANSPORT-4). + expect(fork.signal.reason).toBe(reason); }); test('an already-aborted source forks as already aborted', () => { const controller = new AbortController(); controller.abort(new Error('too late')); const fork = forkSignal(controller.signal); - expect(fork.signal?.aborted).toBe(true); + expect(fork.signal.aborted).toBe(true); }); test('an abort after detach never reaches the fork (SEAM-16)', () => { @@ -36,6 +43,34 @@ describe('forkSignal', () => { fork.detach(); fork.detach(); // idempotent controller.abort(new Error('after delivery')); - expect(fork.signal?.aborted).toBe(false); + expect(fork.signal.aborted).toBe(false); + }); +}); + +describe('forkSignal.abort (TRANSPORT-9)', () => { + test('cancels the native call with the reason the transport gave up for', () => { + const fork = forkSignal(undefined); + const reason = new Error('producer exploded'); + fork.abort(reason); + expect(fork.signal.aborted).toBe(true); + expect(fork.signal.reason).toBe(reason); + }); + + test('cancels a fork that has a source too, without touching the source', () => { + const controller = new AbortController(); + const fork = forkSignal(controller.signal); + fork.abort(new Error('producer exploded')); + expect(fork.signal.aborted).toBe(true); + // The caller's own signal is not the transport's to abort; only the fork it dispatched with. + expect(controller.signal.aborted).toBe(false); + }); + + test('is a no-op after detach, so a delivered body is never torn out (SEAM-16)', () => { + // The latch is what keeps the second direction of the fork from becoming the very violation + // the first direction exists to prevent. + const fork = forkSignal(undefined); + fork.detach(); + fork.abort(new Error('too late to matter')); + expect(fork.signal.aborted).toBe(false); }); }); diff --git a/packages/transport-shared/src/signal-fork.ts b/packages/transport-shared/src/signal-fork.ts index 5561906..55c84d3 100644 --- a/packages/transport-shared/src/signal-fork.ts +++ b/packages/transport-shared/src/signal-fork.ts @@ -2,15 +2,39 @@ // packages/transport-shared/src/signal-fork.ts /** - * A caller signal, forwarded to the native client only for as long as the transport wants it. + * A caller signal, forwarded to the native client only for as long as the transport wants it — and + * a handle the transport can pull itself. * * @internal */ export interface ForkedSignal { - /** Hand this to the native client instead of the caller's own signal. */ - readonly signal: AbortSignal | undefined; - /** Stops forwarding. Idempotent; later aborts of the source no longer reach the native client. */ + /** + * Hand this to the native client instead of the caller's own signal. + * + * Always present, even when the caller supplied no signal and no timeout was composed. That is + * not symmetry for its own sake: {@link ForkedSignal.abort} is the only way a transport can + * cancel a native call it has decided to abandon, and a send with no caller signal is exactly + * the case where a failed request-body producer would otherwise leave one running forever + * (TRANSPORT-9, SEAM-30). A controller nobody ever aborts costs one allocation and is + * indistinguishable, to the native client, from no signal at all. + */ + readonly signal: AbortSignal; + /** + * Stops forwarding, and latches the fork: a later {@link ForkedSignal.abort} is a no-op too. + * Idempotent. Called at delivery, which is the moment the response stops being the transport's. + */ detach(): void; + /** + * Cancels the in-flight native call, so a response that arrives afterwards is refused rather + * than stranded with its body neither read nor released (TRANSPORT-9). + * + * A no-op after {@link ForkedSignal.detach}, which is what keeps this from becoming the SEAM-16 + * violation the fork exists to prevent: once a body has been handed to the caller, nothing in + * this transport may close it. + * + * @param reason - the abort reason; the failure that made the transport give up. + */ + abort(reason: unknown): void; } /** @@ -24,29 +48,37 @@ export interface ForkedSignal { * delivery keeps cancellation live for the whole in-flight window (SEAM-13, TRANSPORT-7) and inert * afterwards. * + * The fork is two-way. It carries the caller's abort *in*, and it lets the transport cancel the + * native call *out* — the second direction added by audit #67 / #82, because a request-body + * producer that fails while the native call is still pending has to take that call down with it. + * * @param source - the composed caller/timeout signal, if any. - * @returns the signal to dispatch with, plus the detach the transport calls on delivery. + * @returns the signal to dispatch with, the detach the transport calls on delivery, and the abort + * it calls when it abandons the exchange. * * @internal */ export function forkSignal(source: AbortSignal | undefined): ForkedSignal { - if (source === undefined) { - return {signal: undefined, detach: () => undefined}; - } const controller = new AbortController(); - if (source.aborted) { - controller.abort(source.reason); - return {signal: controller.signal, detach: () => undefined}; - } + let detached = false; const forward = (): void => { - controller.abort(source.reason); + controller.abort(source?.reason); }; - source.addEventListener('abort', forward, {once: true}); + if (source !== undefined) { + if (source.aborted) forward(); + else source.addEventListener('abort', forward, {once: true}); + } return { signal: controller.signal, - // removeEventListener is idempotent, so a detach on both the success and failure path is safe. detach: () => { - source.removeEventListener('abort', forward); + detached = true; + // removeEventListener is idempotent and a no-op for a listener never added, so a detach on + // both the success and failure path, with or without a source, is safe. + source?.removeEventListener('abort', forward); + }, + abort: (reason: unknown) => { + if (detached) return; + controller.abort(reason); }, }; } diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md index 5858f98..54edeee 100644 --- a/packages/transport-undici/README.md +++ b/packages/transport-undici/README.md @@ -55,11 +55,18 @@ terminal `CancellationError`, and so does a `send()` issued after `close()` — documented `SEAM-15` post-close mode. It cannot succeed over a dispatcher that no longer exists, so it is not reported as a retryable failure. -## Proxy support and its one real limit +## Proxy support and its two real limits `ProxyOptions` routes here in full: address, Basic credentials, and `NO_PROXY`/`nonProxyHosts` bypass globs, which route over a separate direct `Agent` rather than being tunnelled anyway. +**`type` must be `http`.** undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and reads its `uri` as +a URL, so `socks4`/`socks5` — which `ProxyType` admits and core resolves from `ALL_PROXY` +(`CFG-22`) — are refused at construction with a `TypeError` naming the type. Before 2026-09-05 they +reached `new ProxyAgent({uri: 'socks5://…'})` and escaped this factory as an undici +`InvalidArgumentError`, untyped and undocumented. Neither shipped transport can carry SOCKS; +`docs/deviations.md` records the gap. + **A custom `challengeHandler` cannot be dispatched**, and the limitation is surfaced rather than silently misbehaving (`TRANSPORT-30`): @@ -79,17 +86,43 @@ silently misbehaving (`TRANSPORT-30`): ## Behavior worth knowing -- File bodies (`body.kind === 'file'`, e.g. `@dexpace/body-file`'s `fileBody()`) dispatch straight - off the file honoring `start`/`count`, one fewer userspace copy than the `fetch` transport - (`TRANSPORT-28`; a literal kernel zero-copy path does not exist on Node — see the Deviation - Ledger). Recognition is structural, on `kind` alone: this package does not depend on - `@dexpace/body-file`. +- File bodies (`body.kind === 'file'`, e.g. `@dexpace/body-file`'s `fileBody()`) are written + through the descriptor's own `writeTo`, exactly as in the `fetch` transport — buffered below + 1 MB, streamed above it. Until 2026-09-05 this transport instead handed + `createReadStream(path, {start, end})` to undici: one fewer userspace copy, and no `writeTo`, so + `BODY-13`'s `transferred === count` check never ran and a file truncated between `stat` and + `send` uploaded its remaining bytes and returned 200. `content-length` is dropped outbound, so + the framing could not catch it either. `TRANSPORT-28`'s zero-copy clause is a SHOULD that no + user-space path in either client can honour anyway (Deviation Ledger item 13); its MUSTs — a file + body is replayable, and exactly its declared byte range reaches the wire — are honoured by the + descriptor. Recognition, where it is still needed, stays structural: this package does not depend + on `@dexpace/body-file`. - Redirects are pinned off (`maxRedirections: 0`) even behind a bring-your-own dispatcher that may carry a redirect interceptor. The pipeline is the single redirect authority. - `Connection` is **not** dropped outbound — §17's own note is that an undici-class transport - forwards it. `Content-Length`, `Host`, and `Transfer-Encoding` are. + forwards it — but only with a value undici will carry (`close` or `keep-alive`, matched + case-insensitively). Any other value is dropped, because undici rejects it outright. +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped because undici computes them; + `Expect`, `Keep-Alive` and `Upgrade` because undici refuses them + (`InvalidArgumentError`/`NotSupportedError` out of its own argument validation, before anything + reaches the wire). So is any header name outside RFC 9110 `token` — `@dexpace/core` admits every + printable ASCII byte in a name, undici does not. Every one of these is a drop logged by name + (`TRANSPORT-12`), never a failed send. - Destroying the dispatcher mid-flight surfaces as the terminal `CancellationError`, while a timeout on the same path stays the retryable `TransportFailureError` (`TRANSPORT-8`). +- An argument undici refuses outright — a non-`http(s)` origin such as `ftp://`, `CONNECT` as a + method, a per-request `Proxy-Authorization` on a bring-your-own `ProxyAgent` — is a bare + `TypeError` outside the `IoError` tree, so `retry/classify.ts`'s allow-list makes it non-retryable + (`RETRY-2`). A failed *exchange* stays the retryable `TransportFailureError` (`TRANSPORT-20`). The + table that tells them apart moved to `@dexpace/transport-shared` on 2026-09-05 so + `@dexpace/transport-fetch` answers identically (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. undici's dispatcher always hands back a + `BodyReadable`, so this transport `dump()`s the one it declines to expose — returning the + connection to the pool — rather than wrapping an empty stream the caller would have to read to + discover was empty. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `undiciTransport()`, raised + before any dispatcher is allocated, not a failure on the first send (`HTTP-35`). - `Response.protocol` is always `HTTP_1_1`: undici's `ResponseData` does not surface the negotiated version. A Deviation Ledger row, not a silent gap. diff --git a/packages/transport-undici/src/undici-transport.conformance.test.ts b/packages/transport-undici/src/undici-transport.conformance.test.ts index 866b18f..24f0ef1 100644 --- a/packages/transport-undici/src/undici-transport.conformance.test.ts +++ b/packages/transport-undici/src/undici-transport.conformance.test.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/transport-undici/src/undici-transport.conformance.test.ts // Runs the shared TRANSPORT-N suite (@dexpace/transport-conformance) against undiciTransport(). +import {createProxyOptions} from '@dexpace/core'; import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; import {undiciTransport} from './undici-transport.js'; @@ -9,4 +10,20 @@ runTransportConformanceSuite('undiciTransport', () => undiciTransport(), { supportsProxy: true, // TRANSPORT-11's own note: an undici-class transport forwards `Connection` rather than dropping it. dropsConnectionHeader: false, + // TRANSPORT-30: core resolves `ALL_PROXY=socks5://host:1080` to this type (CFG-22), and undici's + // `ProxyAgent` is an HTTP CONNECT tunnel that cannot carry it. + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => + undiciTransport({defaultTimeoutMs: value}), + unsupportedProxy: { + type: 'socks5', + build: () => + undiciTransport({ + proxy: createProxyOptions({ + type: 'socks5', + host: '127.0.0.1', + port: 1080, + }), + }), + }, }); diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts index 862a718..c8fe3b5 100644 --- a/packages/transport-undici/src/undici-transport.test.ts +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -9,9 +9,10 @@ // native body), TRANSPORT-20 (a permanent argument error is terminal, a no-response failure is // retryable), TRANSPORT-28 (a file body dispatches its declared byte range), SEAM-14, // TRANSPORT-19 (a header-mapping throw leaves no started body producer stranded), SEAM-30 (so no -// producer rejection reaches Node's default unhandledRejection policy) +// producer rejection reaches Node's default unhandledRejection policy), TRANSPORT-9 (a producer that +// loses the race cancels the dispatch it raced, so no response is stranded) import {createRequire} from 'node:module'; -import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {mkdtemp, readFile, rm, writeFile} from 'node:fs/promises'; import {createServer, type Server} from 'node:http'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; @@ -348,25 +349,41 @@ describe('undiciTransport disposal (TRANSPORT-15/16)', () => { }); }); +/** + * A `Dispatcher` that records every `request()` it is handed and answers 200 with an empty body, so + * a row can assert what reached undici's argument validation rather than what reached the wire. + * Returns the recording array; the caller builds the transport around it. + */ +function recordingDispatcher(sink: Dispatcher.RequestOptions[]): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + sink.push(options); + return Promise.resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; +} + +/** The flat `[name, value, ...]` array one recorded dispatch carried. */ +function dispatchedHeaders( + sent: Dispatcher.RequestOptions | undefined, +): string[] { + return (sent?.headers ?? []) as string[]; +} + describe('undiciTransport dispatch', () => { test('TRANSPORT-2/11: redirects are pinned off and Connection is forwarded, not dropped', async () => { const dispatched: Dispatcher.RequestOptions[] = []; - const recorder = { - request: (options: Dispatcher.RequestOptions) => { - dispatched.push(options); - return Promise.resolve({ - statusCode: 200, - headers: {}, - body: { - destroy: () => undefined, - [Symbol.asyncIterator]: () => ({ - next: () => Promise.resolve({done: true, value: undefined}), - }), - }, - } as unknown as Dispatcher.ResponseData); - }, - close: () => Promise.resolve(), - } as unknown as Dispatcher; + const recorder = recordingDispatcher(dispatched); const transport = undiciTransport({dispatcher: recorder}); const request = Request.newBuilder() @@ -383,79 +400,174 @@ describe('undiciTransport dispatch', () => { const sent = dispatched[0]; expect(sent?.maxRedirections).toBe(0); expect(sent?.path).toBe('/anything?q=1'); - const headers = sent?.headers as string[]; + const headers = dispatchedHeaders(sent); expect(headers).toContain('Connection'); expect(headers).not.toContain('Content-Length'); }); }); -describe('undiciTransport body and adaptation paths', () => { - test('TRANSPORT-28: a file body dispatches exactly its declared byte range', async () => { - const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); +describe('undiciTransport outbound header degradation (TRANSPORT-11/12)', () => { + test('every header undici refuses is dropped before dispatch, and logged by name', async () => { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); try { - const path = join(dir, 'payload.bin'); - await writeFile(path, 'ABCDEFGH'); - // The structural recognition contract, built by hand: this package must narrow on - // `kind === 'file'` alone, never on an instanceof against @dexpace/body-file, which it - // deliberately does not depend on. - const descriptor: FileBodyDescriptor = { - kind: 'file', - mediaType: 'application/octet-stream', - contentLength: 4, - replayable: true, - path, - start: 2, - count: 4, - writeTo: () => - Promise.reject(new Error('the transport must not call writeTo here')), - }; + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + const headers = dispatchedHeaders(dispatched[0]); + // Asserted against the argument array, not the wire: undici validates in `new Request(...)`, + // so a name that reaches this array is a name that would have thrown (audit #67 / #81). + expect(headers).toEqual(['X-Pass-Through', 'survives']); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + expect(capture.dropped).toContain(name); + } + } finally { + capture.restore(); + } + }); + + test('a Connection value undici cannot carry is dropped, close and keep-alive are not', async () => { + const carried: string[] = []; + const dropped: string[] = []; + for (const value of ['close', 'Keep-Alive', 'upgrade']) { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); + try { + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers(Headers.newBuilder().set('Connection', value).build()) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + if (dispatchedHeaders(dispatched[0]).includes('Connection')) { + carried.push(value); + } + // `createDropLogger` lower-cases the name it logs, so the field is `connection`. + if (capture.dropped.includes('connection')) dropped.push(value); + } finally { + capture.restore(); + } + } + // `Keep-Alive` mixed-cased on purpose: undici lower-cases the value before comparing + // (`lib/core/request.js:401`), so this transport must too or it would drop a header undici + // would have carried. + expect(carried).toEqual(['close', 'Keep-Alive']); + expect(dropped).toEqual(['upgrade']); + }); +}); + +/** + * A `kind: 'file'` descriptor over a real path whose `writeTo` streams its own declared range and + * counts its calls -- the shape `@dexpace/body-file`'s `fileBody()` produces. Built by hand because + * TRANSPORT-28's recognition contract is structural: this package narrows on `kind` alone and must + * never `instanceof` against a package it does not depend on. + */ +function fileDescriptor( + path: string, + range: {start: number; count: number}, + writes: {count: number}, +): FileBodyDescriptor { + return { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: range.count, + replayable: true, + path, + start: range.start, + count: range.count, + async writeTo(sink: WritableStream): Promise { + writes.count += 1; + const writer = sink.getWriter(); + if (range.count === 0) { + writer.releaseLock(); + return; + } + const bytes = (await readFile(path)).subarray( + range.start, + range.start + range.count, + ); + await writer.write(new Uint8Array(bytes)); + writer.releaseLock(); + }, + }; +} + +/** Runs `body` against a fresh temporary file holding `ABCDEFGH`, and removes the directory after. */ +async function withPayloadFile( + run: (path: string) => Promise, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, 'ABCDEFGH'); + return await run(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + +describe('undiciTransport body and adaptation paths', () => { + test('TRANSPORT-28/BODY-13: a file body dispatches its declared range THROUGH its own writeTo', async () => { + await withPayloadFile(async path => { + const writes = {count: 0}; const transport = undiciTransport(); const request = Request.newBuilder() .method('POST') .url(`${origin}/upload`) - .body(descriptor) + .body(fileDescriptor(path, {start: 2, count: 4}, writes)) .build(); received.length = 0; await (await transport.send(request)).close(); await transport.close(); expect(received[0]).toBe('CDEF'); - } finally { - await rm(dir, {recursive: true, force: true}); - } + // The `writes` count is the whole point of the row. Until audit #67 / #81 this transport + // handed `createReadStream(path, {start, end})` to undici and left `writeTo` uncalled, so the + // same four bytes reached the wire with BODY-13's `transferred === count` invariant -- + // the only thing that can see a file truncated after its length was captured -- never run. + expect(writes.count).toBe(1); + }); }); test('a zero-count file body dispatches as an empty body, not a stream error', async () => { - const dir = await mkdtemp(join(tmpdir(), 'undici-empty-file-body-')); - try { - const path = join(dir, 'payload.bin'); - await writeFile(path, 'ABCDEFGH'); - // createReadStream throws ERR_OUT_OF_RANGE the moment `end` falls below `start`, which is what - // `start + count - 1` computes for count 0 -- the empty range needs its own branch. - const descriptor: FileBodyDescriptor = { - kind: 'file', - mediaType: 'application/octet-stream', - contentLength: 0, - replayable: true, - path, - start: 4, - count: 0, - writeTo: () => Promise.resolve(), - }; + await withPayloadFile(async path => { + // `isMaterializable` admits `contentLength === 0`, and `materializeBody` returns an empty + // buffer without opening anything, so the empty range needs no branch of its own. It did when + // the file body went to `createReadStream`, which throws ERR_OUT_OF_RANGE the moment `end` + // (start + count - 1) falls below `start`. + const writes = {count: 0}; const transport = undiciTransport(); received.length = 0; const response = await transport.send( Request.newBuilder() .method('POST') .url(`${origin}/upload`) - .body(descriptor) + .body(fileDescriptor(path, {start: 4, count: 0}, writes)) .build(), ); await response.close(); await transport.close(); expect(received[0]).toBe(''); - } finally { - await rm(dir, {recursive: true, force: true}); - } + expect(writes.count).toBe(1); + }); }); }); @@ -575,7 +687,117 @@ describe('undiciTransport failure classification (TRANSPORT-20)', () => { }); }); +describe('undiciTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + /** + * A `Dispatcher` whose `request()` resolves only after `delayMs`, recording the signal it was + * handed. Nothing awaits that promise once the producer has lost the race, so the signal is the + * only thing that can still stop the exchange. + */ + function lateDispatcher( + seen: {signal?: AbortSignal | null; settled: boolean}, + delayMs: number, + ): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + seen.signal = options.signal as AbortSignal | null; + return new Promise(resolve => { + setTimeout(() => { + seen.settled = true; + resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + dump: () => Promise.resolve(), + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, delayMs); + }); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; + } + + test('a producer that loses the race takes the pending dispatch down with it', async () => { + // Until audit #67 / #82 this send dispatched with `signal: null` -- the fork only existed when + // the caller supplied a signal or a timeout was composed -- so undici kept dispatching after + // `send()` rejected and whatever came back was dropped with its `BodyReadable` neither read nor + // destroyed, holding the pooled connection open. + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 30)}); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(seen.settled).toBe(false); + expect(seen.signal?.aborted).toBe(true); + expect((seen.signal?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + await transport.close(); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 0)}); + const request = Request.newBuilder().url(`${origin}/anything`).build(); + const response = await transport.send(request); + await response.close(); + // The fork is latched at delivery, so its abort direction can no longer reach a body the + // caller now owns. + expect(seen.signal?.aborted).toBe(false); + await transport.close(); + }); +}); + describe('undiciTransport proxy dispatch (TRANSPORT-30)', () => { + test('TRANSPORT-30: a SOCKS proxy is refused at the factory, before any Agent is built', () => { + // Both SOCKS values `ProxyType` admits, because core resolves both from the environment + // (`socks4`/`socks4a` and `socks:`/`socks5`/`socks5h`, `config/proxy.ts:372-380`) and undici's + // `ProxyAgent` carries neither -- it is an HTTP CONNECT tunnel and reads its `uri` as a URL. + const {agents, restore} = captureOwnedAgents(); + try { + for (const type of ['socks4', 'socks5'] as const) { + let thrown: unknown; + try { + undiciTransport({ + proxy: createProxyOptions({type, host: '127.0.0.1', port: 1080}), + }); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // TRANSPORT-30's "discoverable": the refusal names the type, and it is not the raw + // `InvalidArgumentError: Invalid URL protocol: socks5:` undici used to let escape. + expect((thrown as Error).message).toContain(type); + expect(thrown).not.toBeInstanceOf(IoError); + } + // The check runs before `new undici.Agent(...)`, so a refused construction leaks nothing -- + // there is no transport for the caller to have called `close()` on. + expect(agents.length).toBe(0); + } finally { + restore(); + } + }); + test('a per-request Proxy-Authorization is dropped when a proxy is configured', async () => { // ProxyAgent.dispatch throws InvalidArgumentError on ANY per-request Proxy-Authorization -- a // deliberate undici security fix -- so forwarding one would turn every proxied send into a hard diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index a901c97..0fc67a2 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -1,6 +1,5 @@ // SPDX-License-Identifier: MIT // packages/transport-undici/src/undici-transport.ts -import {createReadStream} from 'node:fs'; import {createRequire} from 'node:module'; import {Readable} from 'node:stream'; import type {ReadableStream as NodeReadableStream} from 'node:stream/web'; @@ -13,8 +12,8 @@ import { Status, TransportFailureError, type Body, - type FileBodyDescriptor, type ProxyOptions, + type ProxyType, type Request, type RequestOptions, type Transport, @@ -24,11 +23,14 @@ import { createDropLogger, degradeInboundHeaders, forkSignal, + hasNoResponseBody, isMaterializable, mapOutboundHeaders, materializeBody, producerFailure, pumpBody, + requireValidDefaultTimeoutMs, + toDispatchFailure, type BodyPump, type ForkedSignal, type HeaderDropLogging, @@ -52,13 +54,26 @@ const require = createRequire(import.meta.url); const undici = require('undici/index.js') as typeof import('undici'); /** - * TRANSPORT-11's outbound drop set for this transport. `connection` is deliberately absent — §17's - * own note is that an undici-class transport forwards it rather than dropping it. + * TRANSPORT-11's outbound drop set for this transport: the three framing headers undici computes + * itself, plus the three it refuses outright. + * + * `expect`, `keep-alive` and `upgrade` are undici's unconditional rejections + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`). Until 2026-09-05 they were absent, so a caller who set any of + * them got a failed send out of `undiciTransport().send()` where the fetch twin dispatched + * (audit #67 / #81). TRANSPORT-12 says the header is what gives way, not the request. + * + * `connection` is deliberately absent — §17's own note is that an undici-class transport forwards + * it rather than dropping it. undici carries only `close` and `keep-alive`, so a third value is + * dropped per-request by {@link isUnsendableHeader} rather than by name here. */ const UNDICI_FORBIDDEN_HEADERS: readonly string[] = [ 'content-length', 'host', 'transfer-encoding', + 'expect', + 'keep-alive', + 'upgrade', ]; /** @@ -87,11 +102,23 @@ export interface UndiciTransportOptions { * (SEAM-14); supplying it together with `proxy` is a construction-time error. */ readonly dispatcher?: Dispatcher; - /** Proxy configuration; the transport constructs and owns the resulting `ProxyAgent`. */ + /** + * Proxy configuration; the transport constructs and owns the resulting `ProxyAgent`. + * + * `type` must be `http`. undici's `ProxyAgent` is an HTTP `CONNECT` tunnel, so the `socks4` and + * `socks5` values `ProxyType` also admits — which core resolves from `ALL_PROXY` — are refused at + * construction rather than at the first send (TRANSPORT-30). + */ readonly proxy?: ProxyOptions; /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ readonly headerDropLogging?: HeaderDropLogging; - /** A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. */ + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link undiciTransport} rather than by the first send (HTTP-35). + */ readonly defaultTimeoutMs?: number; /** `Agent` options, used only when no `dispatcher` is supplied. */ readonly agentOptions?: Agent.Options; @@ -107,6 +134,38 @@ interface DispatcherSet { readonly owned: readonly Dispatcher[]; } +/** + * The one `ProxyOptions.type` this transport can build a dispatcher for. + * + * undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and takes its `uri` as a URL, so a `socks5://` + * or `socks4://` one throws `InvalidArgumentError('Invalid URL protocol: …')` out of its + * constructor. Core resolves `ALL_PROXY=socks5://host:1080` to `type: 'socks5'` quite legitimately + * (`config/proxy.ts:372-380`, CFG-22), so the configuration can express a proxy neither shipped + * transport can honour, and until 2026-09-05 the way you found out was an undici error escaping a + * public factory untyped and undocumented (audit #67 / #81, `docs/deviations.md`). + */ +const SUPPORTED_PROXY_TYPE: ProxyType = 'http'; + +/** + * TRANSPORT-30's "make the limitation discoverable rather than silently misbehaving", applied at + * the earliest point that can: construction, before any dispatcher is allocated. + * + * A `TypeError`, matching {@link selectDispatchers}' other construction-time refusal and + * deliberately outside the `IoError` tree — `retry/classify.ts` is an allow-list, so a + * misconfiguration that no retry can fix is non-retryable for free (RETRY-2). + * + * @param proxy - the configured proxy. + * @throws `TypeError` when `proxy.type` is anything but `http`. + */ +function requireSupportedProxyType(proxy: ProxyOptions): void { + if (proxy.type === SUPPORTED_PROXY_TYPE) return; + throw new TypeError( + `unsupported proxy type \`${proxy.type}\`: undici's ProxyAgent is an HTTP CONNECT tunnel and ` + + 'cannot carry a SOCKS proxy, and @dexpace/transport-fetch has no proxy support at all. ' + + 'Configure an http proxy, or route SOCKS outside this SDK.', + ); +} + /** * The proxy URI plus its Basic credential, kept apart. `formatProxyOptions` is deliberately *not* * used here: it masks credentials as `***:***` for logging, and feeding that to `ProxyAgent` would @@ -137,6 +196,9 @@ function selectDispatchers(options: UndiciTransportOptions): DispatcherSet { const byo = options.dispatcher; return {proxied: byo, direct: byo, owned: []}; } + // Before anything is allocated: a refusal after `new undici.Agent(...)` would leak the direct + // agent on the way out, with no transport for the caller to close it through. + if (options.proxy !== undefined) requireSupportedProxyType(options.proxy); // Agent, not Pool: a Pool is bound to one origin at construction, but a general-purpose Transport // must reach whatever origin each Request names. const direct = new undici.Agent(options.agentOptions); @@ -146,7 +208,52 @@ function selectDispatchers(options: UndiciTransportOptions): DispatcherSet { return {proxied, direct, owned: [proxied, direct]}; } -/** undici's flat `[name, value, name, value, ...]` form -- the only shape that keeps a repeated name repeated (HTTP-14). */ +/** + * RFC 9110 `token`, which is what undici's `isValidHTTPToken` enforces on every header name + * (`lib/core/util.js:547-587`): VCHAR minus the delimiters `"(),/:;<=>?@[\]{}`. Anchored and + * one-or-more, so the empty name undici also rejects fails here too. + * + * `@dexpace/core` is deliberately laxer: `hasForbiddenNameByte` admits every printable ASCII byte + * (`http/ascii-validation.ts:29-33`), so `X Custom` is a model-valid name that no native client on + * this platform will carry. + */ +const RFC9110_TOKEN = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/u; + +/** The only two `Connection` values undici carries; anything else is `InvalidArgumentError`. */ +const FORWARDABLE_CONNECTION_VALUES: ReadonlySet = new Set([ + 'close', + 'keep-alive', +]); + +/** + * Whether undici's own request validation would reject this header, which for TRANSPORT-12 means it + * is dropped rather than allowed to fail the send. + * + * Only the *name* grammar and the `connection` value are checked. undici's value grammar + * (`headerCharRegex`, `lib/core/util.js:598`) admits HTAB, 0x20-0x7E **and** obs-text 0x80-0xFF, + * which is strictly wider than the outbound value rule `mapOutboundHeaders` has already applied + * (HTTP-18), so no value that reaches here can fail there. + * + * @param name - the header name, as the caller spelled it. + * @param value - the header value. + * @returns `true` when the header must be dropped instead of dispatched. + */ +function isUnsendableHeader(name: string, value: string): boolean { + if (!RFC9110_TOKEN.test(name)) return true; + return ( + name.toLowerCase() === 'connection' && + !FORWARDABLE_CONNECTION_VALUES.has(value.toLowerCase()) + ); +} + +/** + * undici's flat `[name, value, name, value, ...]` form -- the only shape that keeps a repeated name + * repeated (HTTP-14) -- with every header undici would refuse degraded to a logged drop. + * + * The per-header guard is the same degrade `@dexpace/transport-fetch` gets for free from wrapping + * `Headers.append` in a `try`/`catch` (`fetch-transport.ts:159-166`): undici validates at dispatch + * rather than at construction, so this transport has to ask the question itself (TRANSPORT-12). + */ function toUndiciHeaders( request: Request, forbidden: readonly string[], @@ -155,8 +262,14 @@ function toUndiciHeaders( const {sent, dropped} = mapOutboundHeaders(request.headers, forbidden, { bodyDerivedMediaType: request.body?.mediaType, }); - logDrops(dropped); - return [...sent.entries()].flat(); + const degraded = [...dropped]; + const flat: string[] = []; + for (const [name, value] of sent.entries()) { + if (isUnsendableHeader(name, value)) degraded.push(name); + else flat.push(name, value); + } + logDrops(degraded); + return flat; } /** @@ -172,44 +285,28 @@ const NATIVE_CANCEL_CODES: ReadonlySet = new Set([ 'UND_ERR_CLOSED', ]); -/** - * undici's codes for "these arguments can never work", as opposed to "this exchange failed". Both are - * raised by argument validation and are perfectly reproducible, so classifying them as - * `TransportFailureError` would hand `classify.ts` an always-retryable verdict (it returns `true` for - * every `IoError`) and spend a caller's whole retry budget re-proving a permanent misconfiguration. - * The commonest way to reach one is a bring-your-own `ProxyAgent` plus a per-request - * `Proxy-Authorization`: `UNDICI_PROXIED_FORBIDDEN_HEADERS` only drops that header when this - * transport constructed the proxy itself, so with a BYO dispatcher it reaches `dispatch` and is - * rejected outright. - */ -const TERMINAL_ARGUMENT_CODES: ReadonlySet = new Set([ - 'UND_ERR_INVALID_ARG', - 'UND_ERR_NOT_SUPPORTED', -]); - -function errorCode(error: unknown): string | undefined { - const code = (error as {code?: unknown} | null | undefined)?.code; - return typeof code === 'string' ? code : undefined; -} - function isNativeCancel(error: unknown): boolean { - const code = errorCode(error); - return code !== undefined && NATIVE_CANCEL_CODES.has(code); + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' && NATIVE_CANCEL_CODES.has(code); } /** - * Maps one dispatch failure onto the SDK's error vocabulary. Extracted from `#dispatch` so the four + * Maps one dispatch failure onto the SDK's error vocabulary. Extracted from `#dispatch` so the * branches read as one classification table rather than as control flow wrapped around a call. * + * Only the first two branches are this transport's own. The permanent-versus-retryable question the + * third asks is `@dexpace/transport-shared`'s {@link toDispatchFailure}, because the two shipped + * adapters answered it oppositely for the same condition until audit #67 / #82: `ftp://` is + * `UND_ERR_INVALID_ARG` here and a `fetch failed` with an `unknown scheme` cause over there, and + * only this transport treated it as permanent. A shared table is what keeps that from recurring — + * the same reason `abort-mapping.ts` exists. + * * @param error - whatever the dispatch rejected with. - * @param signal - the forked signal the dispatch was given, if any. + * @param signal - the forked signal the dispatch was given. * @returns the error to throw; never returns normally without one. */ -function toDispatchError( - error: unknown, - signal: AbortSignal | undefined, -): Error { - if (signal?.aborted) return abortToSdkError(signal, error); +function toDispatchError(error: unknown, signal: AbortSignal): Error { + if (signal.aborted) return abortToSdkError(signal, error); if (isNativeCancel(error)) { // TRANSPORT-8: terminal, never retryable -- the dispatcher this send was routed over no longer // exists, so a retry over it cannot succeed. @@ -217,22 +314,7 @@ function toDispatchError( cause: error, }); } - const code = errorCode(error); - if (code !== undefined && TERMINAL_ARGUMENT_CODES.has(code)) { - // Deliberately outside the IoError tree: `classify.ts` is an allow-list, so anything that is not - // an IoError, a timeout, or a retryable status is non-retryable for free (RETRY-2). `TypeError` - // matches `selectDispatchers`, which already reports a caller misconfiguration that way. - return new TypeError( - error instanceof Error - ? error.message - : 'undici rejected the request arguments', - {cause: error}, - ); - } - return new TransportFailureError( - error instanceof Error ? error.message : 'undici dispatch failed', - {cause: error}, - ); + return toDispatchFailure(error, 'undici dispatch failed'); } /** What undici accepts as a request body; `undefined` is not one of them, `null` is. */ @@ -245,33 +327,29 @@ interface PreparedBody { } /** - * TRANSPORT-28's recognition contract, in one named place: a plain string-literal check, never a - * cross-package `instanceof` against `@dexpace/body-file` (which this package does not depend on). - * `Body.kind` is a union on one interface rather than a discriminated union of interfaces, so the - * narrowing has to be spelled out as a predicate. + * Prepares one request body for dispatch: buffer it if it is small and replayable, stream it + * otherwise. Exactly the two decisions the fetch twin makes, in the same order, and deliberately so + * — there is **no** `kind === 'file'` branch here. + * + * There was one until 2026-09-05. It handed `createReadStream(path, {start, end})` to undici, which + * is a genuinely shorter path to the wire — one fewer userspace copy — and it bypassed the + * descriptor's own `writeTo`, so `@dexpace/body-file`'s `transferred === count` invariant never ran + * (BODY-13). `content-length` is dropped outbound, so undici framed the body chunked and the wire + * could not detect the short write either: a file truncated between `stat` and `send` POSTed its + * remaining bytes and resolved 200, where the fetch transport raised `TransportFailureError` + * (audit #67 / #81). A file body is now written through `writeTo` like any other, which is what + * makes BODY-13 hold on both transports and what the shared conformance row asserts. + * + * TRANSPORT-28's SHOULD is not thereby abandoned so much as re-described: no user-space path in + * either client reaches `sendfile(2)`, which `docs/deviations.md` item 13 has recorded since Phase + * 8a, and the clause's MUST — a file body is replayable, and exactly its declared byte range reaches + * the wire — is honoured by the descriptor itself, on both transports, by the same code. + * + * The zero-count case needs no branch either: `isMaterializable` admits `contentLength === 0`, and + * `materializeBody` returns an empty buffer without ever opening a read stream. */ -function isFileBody(body: Body): body is FileBodyDescriptor { - return body.kind === 'file'; -} - async function prepareBody(body: Body | undefined): Promise { if (body === undefined) return {init: null, pump: undefined}; - if (isFileBody(body)) { - // An empty range is not a degenerate read stream: `createReadStream` throws ERR_OUT_OF_RANGE the - // moment `end` (start + count - 1) falls below `start`, so a zero-count file body has to become - // an explicit empty body rather than a stream nobody can open. - if (body.count === 0) return {init: new Uint8Array(0), pump: undefined}; - // TRANSPORT-28: dispatch straight off the file, honoring start/count, rather than routing the - // bytes through a userspace TransformStream first. The closest available approximation of the - // reference's zero-copy path -- see the Deviation Ledger for why a literal one does not exist. - return { - init: createReadStream(body.path, { - start: body.start, - end: body.start + body.count - 1, - }), - pump: undefined, - }; - } if (isMaterializable(body, MAX_MATERIALIZED_BODY_BYTES)) { try { return {init: await materializeBody(body), pump: undefined}; @@ -331,7 +409,12 @@ function adaptResponse( const raw: [string, string][] = []; for (const [name, value] of Object.entries(result.headers)) { if (value === undefined) continue; - // An array means a genuinely repeated header (Set-Cookie); keep each value its own entry. + // An array means a genuinely repeated header -- undici arrays ANY name it saw more than once, + // not just `Set-Cookie`. Keep each value its own entry: `WWW-Authenticate` and + // `Proxy-Authenticate` arrive this way from a server following RFC 7616 3.3, and collapsing them + // to the first would hide every challenge after it (audit #67 / #74). `@dexpace/transport-fetch` + // comma-joins the same response, which RFC 9110 5.3 makes equivalent; the conformance row + // `a repeated inbound header keeps every value` asserts the two agree on the list. if (Array.isArray(value)) for (const each of value) raw.push([name, each]); else raw.push([name, value]); } @@ -346,7 +429,15 @@ function adaptResponse( .protocol(Protocol.HTTP_1_1) .status(Status.of(result.statusCode)) .headers(headers) - .body(toDemandDrivenStream(result.body)) + // undici's dispatcher always hands back a `BodyReadable`, even for a 204, a 304 or a HEAD -- + // so wrapping it unconditionally gave a caller an empty stream it had to read to discover + // was empty, where the fetch twin on Node gave `null`. The two adapters now decide by the + // same rule; `#exchange` dumps whatever this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, result.statusCode) + ? null + : toDemandDrivenStream(result.body), + ) .build() ); } @@ -388,6 +479,9 @@ class UndiciTransport implements Transport { #closing: Promise | undefined; constructor(options: UndiciTransportOptions) { + // Before `selectDispatchers` allocates anything: a refusal afterwards would leak the agents it + // built, with no transport for the caller to close them through (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); this.#dispatchers = selectDispatchers(options); this.#proxy = options.proxy; this.#logDrops = createDropLogger( @@ -450,7 +544,7 @@ class UndiciTransport implements Transport { // which is exactly the in-flight window this check is about. const dispatched = context.fork.signal; - if (dispatched?.aborted) { + if (dispatched.aborted) { // TRANSPORT-9 / SEAM-30: this response will never reach a caller, so this producer closes it. await result.body.dump().catch(() => undefined); await pump?.abandon(dispatched.reason); @@ -460,6 +554,12 @@ class UndiciTransport implements Transport { try { // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. const response = adaptResponse(request, result, this.#logDrops); + if (response.body === null) { + // Nothing references the `BodyReadable` any more, and an undrained one holds the pooled + // connection open until the dispatcher times it out (TRANSPORT-25, SEAM-30). `dump` reads + // and discards, which is what returns the socket to the pool. + await result.body.dump().catch(() => undefined); + } this.#reportProxyChallenge(response); return response; } catch (error) { @@ -493,9 +593,9 @@ class UndiciTransport implements Transport { method: request.method, headers: context.headers, body: context.body, - // `?? null` rather than an omitted key: `exactOptionalPropertyTypes` makes an explicit - // `undefined` a distinct, rejected value here, and undici reads `null` as "no signal". - signal: context.fork.signal ?? null, + // Always a real signal since audit #67 / #82: the fork is this transport's own + // cancellation handle, not merely a relay for the caller's. + signal: context.fork.signal, // TRANSPORT-1: pinned explicitly rather than inherited -- a BYO dispatcher may carry a // redirect interceptor, and the pipeline is the single redirect authority. maxRedirections: 0, @@ -503,8 +603,16 @@ class UndiciTransport implements Transport { producerFailure(pump?.done), ]); } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const mapped = toDispatchError(error, context.fork.signal); await pump?.abandon(error); - throw toDispatchError(error, context.fork.signal); + // TRANSPORT-9: when the producer lost the race, undici is still dispatching. Nothing awaits + // it any more, so a response that arrives later would be dropped with its `BodyReadable` + // neither read nor destroyed, holding the pooled connection. Pulling the fork takes the + // dispatch down instead; on the path where undici itself rejected it is inert. + context.fork.abort(error); + throw mapped; } } @@ -586,7 +694,10 @@ if (typeof Symbol.asyncDispose === 'symbol') { * * @param options - optional transport settings. * @returns a transport ready to send; release it with `close()`. - * @throws `TypeError` when both `dispatcher` and `proxy` are supplied. + * @throws `TypeError` when both `dispatcher` and `proxy` are supplied; when `proxy.type` is + * anything but `http` — undici's `ProxyAgent` cannot carry a SOCKS proxy, and neither can + * `@dexpace/transport-fetch`, which has no proxy option at all; or when `defaultTimeoutMs` is not + * an integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s range. * * @public */ diff --git a/scripts/verify-consumer-types.mjs b/scripts/verify-consumer-types.mjs index 9a874e2..48a10ed 100644 --- a/scripts/verify-consumer-types.mjs +++ b/scripts/verify-consumer-types.mjs @@ -141,7 +141,7 @@ import { type AuthStepSettings, type AuthTiers, type BackoffSettings, - type BasicCredential, + BasicCredential, type BearerCredential, BearerToken, bearerTokensEqual, @@ -159,7 +159,7 @@ import { type DeserializationErrorOptions, type Deserializer, type DigestAlgorithm, - type DigestCredential, + DigestCredential, type DispatchContext, type ExchangeContext, type ExecutionContext, @@ -333,8 +333,8 @@ export function assemble(transport: Transport): Runtime { const settings: AuthStepSettings = { credentials: { apiKey: {credential: new ApiKeyCredential('k'), prefix: 'ApiKey'}, - basic: {username: 'u', password: 'p'}, - digest: {username: 'u', password: 'p', algorithmPreference: ['SHA-256' satisfies DigestAlgorithm]}, + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p', ['SHA-256' satisfies DigestAlgorithm]), bearer: {provider, marginMs: 5_000}, }, tiers: { @@ -379,8 +379,8 @@ export function narrow(error: unknown): string | undefined { if (error instanceof AuthResolutionError) return error.requiredSchemes?.[0]; return undefined; } -export function credentialSet(set: AuthCredentialSet): BasicCredential | undefined { - return set.basic; +export function credentialSet(set: AuthCredentialSet): string | undefined { + return set.basic?.username; } export function bearerCredential(c: BearerCredential): TokenProvider { return c.provider; diff --git a/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts index c1a1952..a97008a 100644 --- a/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts +++ b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts @@ -9,7 +9,12 @@ // These run the invariants through the fully composed retry+redirect+auth+logging pipeline over a // real socket, which is what this suite adds over 5a's and Phase 2's own unit-level coverage. import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; -import {CancellationError, Request} from '@dexpace/core'; +import { + CancellationError, + HttpStatusError, + Request, + retryAttempts, +} from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; import {rejectionOf} from './fixtures/settle.js'; @@ -20,10 +25,14 @@ let server: XcutFixtureServer; * Walks everything a surfaced failure can nest a prior error under, visiting by identity so a cyclic * chain terminates -- the same discipline `XCUT-9` puts on the classifier. * - * The walk is necessary because 5a's engine folds the retry trail into a `SuppressedError` - * (`RETRY-34`), so the cancellation that ended a backoff wait arrives as `.error` beneath a wrapper - * rather than as the top-level throwable. Asserting on the top level alone would test the wrapping, - * not the invariant. + * Kept after #72 made the top-level assertion possible, because the two answer different questions. + * `CancellationError` carries the raw `AbortSignal.reason` as its `cause`, so the walk is what + * proves the ambient abort was not swallowed on the way out, and it is also what catches a + * `TimeoutError` hiding one hop down where `XCUT-3` forbids one. What it must no longer be is the + * ONLY assertion: 5a's engine folded the retry trail into a `SuppressedError` (`RETRY-34`), the + * cancellation arrived as `.error` beneath that wrapper, and a chain walk was the only way to find + * it -- which is precisely the defect, since a caller writing `catch (e) { e instanceof + * CancellationError }` has no walk. Every row below asserts the top level too. */ function* chainOf(error: unknown): Generator { const seen = new Set(); @@ -176,13 +185,17 @@ describe('XCUT-3: an inter-attempt wait is promptly cancellable', () => { }, 50).unref(); const surfaced = await rejectionOf(pending); - // Asserted on the chain rather than the top-level type, because the top level is a - // `SuppressedError` pairing the cancellation with the prior attempt's failure. What the chain - // must carry is the SDK's OWN type: until 2026-09-02 the retry engine surfaced `signal.reason` - // verbatim, so a cancelled backoff arrived as a bare DOMException `AbortError` while the - // transport path mapped the identical abort to `CancellationError` -- one requirement, two - // types, depending on which layer noticed. Both layers now map through the same shape, so this - // asserts the type as well as XCUT-3's letter. + // XCUT-1's conformance clause, at the top level and unqualified: "assert the surfaced error is + // the cancellation type". Two separate defects had to be fixed for this line to hold. Until + // 2026-09-02 the engine surfaced `signal.reason` verbatim, so a cancelled backoff arrived as a + // bare DOMException `AbortError` while the transport mapped the identical abort to + // `CancellationError` -- one requirement, two types, depending on which layer noticed. Until + // 2026-09-05 the mapping was then undone one line later by the retry trail's `SuppressedError` + // wrapper, and a cancelled backoff ALWAYS has a non-empty trail, so this row was false for + // every reachable case. + expect(surfaced).toBeInstanceOf(CancellationError); + // The chain still has to carry the raw abort (the ambient flag survived the mapping) and must + // not carry a timeout, which is XCUT-3's own letter. expect(carriesCancellation(surfaced)).toBe(true); expect(carriesSdkCancellationError(surfaced)).toBe(true); expect(carriesTimeout(surfaced)).toBe(false); @@ -211,3 +224,31 @@ describe('XCUT-3: an inter-attempt wait is promptly cancellable', () => { await pipeline.close(); }); }); + +describe('RETRY-34: the trail survives the unwrapped cancellation', () => { + test('the attempt the cancelled wait was scheduled for stays reachable', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + const surfaced = await rejectionOf(pending); + + // Surfacing the cancellation unwrapped is not allowed to LOSE the 500 that provoked the retry + // in the first place -- that was the one thing the `SuppressedError` wrapper did buy. It rides + // in the trail instead, and the retired response is a buffered `HttpStatusError` (RECOV-16). + const priors = retryAttempts(surfaced); + expect(priors).toHaveLength(1); + expect(priors[0]).toBeInstanceOf(HttpStatusError); + expect((priors[0] as HttpStatusError).status).toBe(500); + + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/fixtures/server.ts b/tests/conformance/xcut/fixtures/server.ts index 2ac2dfa..754cb65 100644 --- a/tests/conformance/xcut/fixtures/server.ts +++ b/tests/conformance/xcut/fixtures/server.ts @@ -95,6 +95,16 @@ function route( res.writeHead(302, {location: `${crossOrigin}/echo`}); res.end(); return; + case '/redirect-secret-target': { + // A method-preserving 307 whose Location carries a credential-shaped query value, echoed back + // from the caller's own `?secret=` so the test owns the string it then greps the log for. + // XCUT-19's rejected-redirect row needs the secret in the redirect TARGET specifically: that is + // the URL `NonReplayableBodyError` names, and the seed URL alone never reaches that message. + const secret = url.searchParams.get('secret') ?? 'secret'; + res.writeHead(307, {location: `/echo?access_token=${secret}`}); + res.end(); + return; + } case '/fail-500': res.writeHead(500, {'content-type': 'text/plain'}); res.end('server error'); diff --git a/tests/conformance/xcut/retry-safety.conformance.test.ts b/tests/conformance/xcut/retry-safety.conformance.test.ts index 67aa4d7..0be9dd0 100644 --- a/tests/conformance/xcut/retry-safety.conformance.test.ts +++ b/tests/conformance/xcut/retry-safety.conformance.test.ts @@ -2,15 +2,25 @@ // tests/conformance/xcut/retry-safety.conformance.test.ts // Exercises: XCUT-10 (retry-SAFETY is decided at the retry step independently of retryability, and // applies uniformly to protocol AND transport failures -- the gate must not special-case a transport -// error that never reached the server). +// error that never reached the server), XCUT-1 (the class of the surfaced error does not depend on +// how many attempts the pillar spent), RETRY-34 (the earlier attempts stay reachable beside it). // -// The five rows are the ones XCUT-10's own conformance clause names, run for the first time against +// The five XCUT-10 rows are the ones its own conformance clause names, run for the first time against // the composed pipeline rather than 5a's unit-level harness. Each asserts on dispatches that actually // reached the terminal transport, which is the only vantage point where "was it re-sent?" is visible. +// The two XCUT-1/RETRY-34 rows below need the same vantage point for the opposite reason: the budget +// they vary is only observable as a dispatch count. import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; -import {Request, streamBody, stringBody} from '@dexpace/core'; +import { + Request, + retryAttempts, + streamBody, + stringBody, + TransportFailureError, +} from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; let server: XcutFixtureServer; @@ -27,6 +37,16 @@ function retrying(): {settings: {maxAttempts: number; initialDelayMs: number}} { return {settings: {maxAttempts: 3, initialDelayMs: 1}}; } +/** + * A GET at a closed port. Retry-SAFE (idempotent, body-less) and retryABLE (the transports map a + * refused connection to `TransportFailureError`, an `IoError`), so the pillar spends its whole + * budget and every attempt fails the same way -- which is what makes the surfaced CLASS the only + * variable between the two budgets below. + */ +function unreachable(): Request { + return Request.newBuilder().url('http://127.0.0.1:1/').method('GET').build(); +} + describe('XCUT-10: retry-safety on a body-less request follows method idempotence', () => { test('retries a body-less GET against a retryable protocol failure', async () => { const pipeline = buildComposedPipeline({retry: retrying()}); @@ -104,3 +124,44 @@ describe('XCUT-10: retry-safety on a body-bearing request follows body replayabi await pipeline.close(); }); }); + +describe('XCUT-1/RETRY-34: what a retrying pipeline surfaces when it gives up', () => { + test('the same failure surfaces as TransportFailureError for maxAttempts 1 and for 3', async () => { + const once = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + const thrice = buildComposedPipeline({retry: retrying()}); + + const afterOne = await rejectionOf(once.runtime.send(unreachable())); + const afterThree = await rejectionOf(thrice.runtime.send(unreachable())); + + expect(once.dispatches()).toBe(1); + expect(thrice.dispatches()).toBe(3); + // The row #72 exists for. Until 2026-09-05 the three-attempt case surfaced a `SuppressedError` + // holding the transport failure at `.error`, so one condition had two surfaced classes and the + // discriminator was the attempt budget -- something no caller writing `catch` can see. + expect(afterOne).toBeInstanceOf(TransportFailureError); + expect(afterThree).toBeInstanceOf(TransportFailureError); + + await once.close(); + await thrice.close(); + }); + + test('the earlier attempts are reachable beside it, oldest first, self excluded', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const surfaced = await rejectionOf(pipeline.runtime.send(unreachable())); + + // RETRY-34 through the composed pipeline: three sends, so two priors, and the surfaced instance + // is not a member of its own trail. + const priors = retryAttempts(surfaced); + expect(pipeline.dispatches()).toBe(3); + expect(priors).toHaveLength(2); + expect(priors.every(prior => prior instanceof TransportFailureError)).toBe( + true, + ); + expect(priors).not.toContain(surfaced); + + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/security-by-default.conformance.test.ts b/tests/conformance/xcut/security-by-default.conformance.test.ts index dd32f93..93572c6 100644 --- a/tests/conformance/xcut/security-by-default.conformance.test.ts +++ b/tests/conformance/xcut/security-by-default.conformance.test.ts @@ -3,6 +3,8 @@ // Exercises: XCUT-17 (redirect credential hygiene -- Authorization stripped before EVERY re-issue, // origin-scoped credentials additionally stripped cross-origin), XCUT-16 (no credential is ever // stamped over a non-HTTPS transport, and the refusal lands BEFORE any token fetch). +// Exercises: XCUT-19 (default-deny log redaction, clause (a) userinfo and clause (b) query values) on +// the rejected-redirect path, with OBS-11, OBS-12 and REDIR-28 as the requirements it lands under. // // These run over a real two-origin socket pair through the composed retry+redirect+auth+logging // pipeline. 5b's own tests decide the hop in isolation against constructed inputs; this is the first @@ -18,15 +20,30 @@ // XCUT-19 default-deny redaction-> packages/core/src/observability/redaction.test.ts // XCUT-20 observability never throws -> packages/core/src/observability/logging-step.test.ts // XCUT-21 CSPRNG cnonce -> packages/core/src/auth/digest.test.ts (AUTH-20) +// +// Also exercises: XCUT-16/AUTH-28 on the CHALLENGE-REPLAY path -- a hop the outbound pass guarded +// stays guarded, so a `challengeHook` that answers a 401 by downgrading to `http://` is refused +// whatever header it carries the credential in (audit #67 / #71). import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; import { createAuthDescriptor, createAuthRequirement, createBearerToken, + createLogger, Headers, + NonReplayableBodyError, + NOOP_LOGGER, + NameKeyCredential, PlaintextCredentialError, + Protocol, Request, + setGlobalLogger, + streamBody, type AuthStepSettings, + type Method, + Response, + Status, + type Transport, } from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; @@ -106,6 +123,60 @@ describe('XCUT-17: origin-scoped credentials are additionally stripped cross-ori }); }); +describe('XCUT-19: a rejected redirect logs no raw URL (OBS-11, OBS-12, REDIR-28)', () => { + const SECRET = 'SUPERSECRETTOKEN'; + + test('redacts the redirect target inside the rejection cause', async () => { + // A one-shot body makes the 307 unfollowable, so `decide()` fails with the target interpolated + // into the error message -- the one field on this path that `redactUrl` did not already cover. + const seed = Request.newBuilder() + .method('POST') + .url(`${server.url}/redirect-secret-target?secret=${SECRET}`) + .body( + streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ), + ) + .build(); + const pipeline = buildComposedPipeline({ + redirect: {allowedMethods: new Set(['GET', 'HEAD', 'POST'])}, + }); + const records: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + records.push(new Map(fields)); + }), + ); + + try { + const rejected = await rejectionOf(pipeline.runtime.send(seed)); + + expect(rejected).toBeInstanceOf(NonReplayableBodyError); + const rejections = records.filter( + r => r.get('event') === 'http.redirect.rejected', + ); + expect(rejections).toHaveLength(1); + expect(String(rejections[0]?.get('cause'))).toContain('access_token=***'); + // Nothing the whole composed pipeline emitted -- not the redirect events, not the + // request/response pair around them -- carries the secret in clear text. + for (const record of records) { + for (const field of record.values()) { + expect(String(field)).not.toContain(SECRET); + } + } + } finally { + setGlobalLogger(NOOP_LOGGER); + await pipeline.close(); + } + }); +}); + describe('XCUT-16: a credential is never stamped over a non-HTTPS transport', () => { test('refuses a bearer credential over http:// before fetching the token', async () => { let providerInvocations = 0; @@ -159,3 +230,101 @@ describe('XCUT-16: a credential is never stamped over a non-HTTPS transport', () await pipeline.close(); }); }); + +/** + * A transport that answers everything with the same 401 challenge, so the replay path can be driven + * without a TLS fixture. `XCUT-16`'s replay clause needs an outbound hop that is HTTPS — the guard + * cannot have run otherwise — and the plaintext fixture server above cannot provide one. The stubbed + * transport is the same device `error-taxonomy.conformance.test.ts` uses for the inputs a live socket + * cannot produce; everything above the transport is still the real composed pipeline. + */ +class ChallengingTransport implements Transport { + send(request: Request): Promise { + return Promise.resolve( + Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(401)) + .headers( + Headers.newBuilder() + .setInbound('WWW-Authenticate', 'Basic realm="x"') + .build(), + ) + .build(), + ); + } + + async close(): Promise { + // Nothing to release: this transport never opens anything. + } +} + +describe('XCUT-16: a guarded hop stays guarded across a challenge replay', () => { + /** `X-Api-Key`, not `Authorization`: the header this step is configured to stamp. */ + function apiKeyAuth(replacement: (request: Request) => Request): { + auth: AuthStepSettings; + transport: Transport; + } { + return { + auth: { + credentials: { + apiKey: { + credential: new NameKeyCredential('x-api-key', 'SECRET'), + headerName: 'X-Api-Key', + }, + }, + tiers: { + operation: createAuthDescriptor([createAuthRequirement('API_KEY')]), + }, + challengeHook: (_response, request) => + Promise.resolve(replacement(request)), + }, + transport: new ChallengingTransport(), + }; + } + + test('refuses a replacement that downgrades to http:// and carries the key in X-Api-Key', async () => { + // The reported hole: the replay guard tested two header NAMES, and neither of them is the one + // `ApiKeyCredentialConfig.headerName` told this step to stamp. The credential went out in clear + // text with the whole suite green. + const {auth, transport} = apiKeyAuth(request => + request + .newBuilder() + .url('http://example.com/echo') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + // One dispatch: the guarded outbound pass. The replay never reached the transport. + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); + + test('refuses a downgraded replacement even with no credential header on it at all', async () => { + // The rule is "this hop was guarded", not "this replacement looks credentialed" — a hook is free + // to invent a carrier no enumeration of header names would know to look for. + const {auth, transport} = apiKeyAuth(request => + Request.newBuilder() + .url('http://example.com/echo') + .method(request.method) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); diff --git a/tests/node-conformance/README.md b/tests/node-conformance/README.md index b9c62df..ef802e0 100644 --- a/tests/node-conformance/README.md +++ b/tests/node-conformance/README.md @@ -51,10 +51,16 @@ Node). **A phase that touches a runtime-divergent surface adds a case here, not only to `bun test`** (§5.9:378). Since Phase 4 that has meant most phases — pipelines, retry, redirect, auth, serde, SSE, pagination, -configuration, observability, the two concrete transports, and the RxJS bridge all have cases here. Two are -worth naming as the shape to aim for: 8a's `fetch`/`undici` transports, where this stops being precautionary -and becomes the point, and 8b's RxJS bridge, whose reason for being hand-written is a cancellation path the -runtime decides. +configuration, observability, the two concrete transports, and the RxJS bridge all have cases here — as, since +audit #67's #77, do the five Web Streams bridges that had none: `BufferedSource.toReadableStream`, +`BufferedSink.toWritableStream`, `TeeSink.toWritableStream` in `io-byte-stream.test.mjs`, and the +`withRequestLogging` / `withResponseLogging` body taps in `body-lifecycle.test.mjs`. Every one of them is a +hand-written underlying source or sink object, so what they exercise is the runtime's own pull scheduling, +cancel dispatch and reader-lock bookkeeping. Three are worth naming as the shape to aim for: 8a's +`fetch`/`undici` transports, where this stops being precautionary and becomes the point; 8b's RxJS bridge, +whose reason for being hand-written is a cancellation path the runtime decides; and #77's multipart +`Content-Type`, where Bun's `Response.formData()` accepted a header Node's `Response.formData()` rejected — +the Bun rows were green over a body no Node peer could parse, and only the case here reproduced it. ## Which cases exist diff --git a/tests/node-conformance/auth.test.mjs b/tests/node-conformance/auth.test.mjs index bd48a3b..e29c28c 100644 --- a/tests/node-conformance/auth.test.mjs +++ b/tests/node-conformance/auth.test.mjs @@ -39,6 +39,7 @@ import assert from 'node:assert/strict'; import {getEventListeners} from 'node:events'; import {describe, it} from 'node:test'; import { + BasicCredential, CancellationError, Request, authStep, @@ -216,7 +217,7 @@ describe('challenge response lifecycle over Node Web Streams (AUTH-30/AUTH-31/AU const tiers = { client: createAuthDescriptor([createAuthRequirement('BASIC')]), }; - const credentials = {basic: {username: 'u', password: 'p'}}; + const credentials = {basic: new BasicCredential('u', 'p')}; it('closes the original 401 before re-driving, and leaves the replacement response open', async () => { const challenged = challengeResponse( diff --git a/tests/node-conformance/body-lifecycle.test.mjs b/tests/node-conformance/body-lifecycle.test.mjs index ec37f82..931f5ec 100644 --- a/tests/node-conformance/body-lifecycle.test.mjs +++ b/tests/node-conformance/body-lifecycle.test.mjs @@ -24,6 +24,14 @@ import { stringBody, toHttpError, } from '@dexpace/core'; +// The two logging taps are `@internal` -- `body/index.ts` holds them and the public barrel deliberately +// does not, so they are reached by direct `dist/` file path, exactly as `io-byte-stream.test.mjs` reaches +// `io/`. Still the BUILT artifact, never `src/`. +import {withRequestLogging} from '../../packages/core/dist/body/request-body-logging.js'; +import {withResponseLogging} from '../../packages/core/dist/body/response-body-logging.js'; + +/** `Response` above is the SDK's model class, which shadows the platform global this file also needs. */ +const PlatformResponse = globalThis.Response; function streamOf(bytes) { return new ReadableStream({ @@ -227,3 +235,179 @@ describe('toHttpError buffering on Node', () => { assert.deepEqual([...(await response.bytes())], [1, 2, 3]); }); }); + +describe("the multipart Content-Type parses in Node's own FormData reader (HTTP-51)", () => { + // The reproducer for the boundary-quoting fix, and the reason it belongs here rather than only in + // `bun test`: Bun's `Response.formData()` tolerates an unquoted `boundary=a,b`, Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`. Two independent parsers + // disagreeing about a header this SDK generates is precisely what this tree exists to catch. + for (const boundary of ['a,b', 'bound ary', 'a:b', 'a=b', 'a?b', '(a)/b']) { + it(`round-trips a body framed with ${JSON.stringify(boundary)}`, async () => { + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const parsed = await new PlatformResponse(await collect(body), { + headers: {'content-type': body.mediaType}, + }).formData(); + assert.equal(parsed.get('field'), 'value'); + }); + } + + it('leaves a boundary that is already a bare token unquoted', () => { + const body = multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1'); + assert.equal(body.mediaType, 'multipart/form-data; boundary=plain-1'); + }); +}); + +describe('an exact-length copy refuses a zero-length delivery on Node (HTTP-39/BODY-10)', () => { + it('raises rather than forwarding a chunked-encoding terminator to the sink', async () => { + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths = []; + + await assert.rejects( + streamBody(source, undefined, 3).writeTo( + new WritableStream({write: c => void chunkLengths.push(c.length)}), + ), + error => error.name === 'SourceContractViolationError', + ); + assert.deepEqual(chunkLengths, [2]); + }); + + it('still allows a declared length of 0 over a source that just closes', async () => { + let closed = false; + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo( + new WritableStream({ + write: c => void c, + close: () => void (closed = true), + }), + ); + assert.equal(closed, true); + }); +}); + +describe('withRequestLogging over Node Web Streams (BODY-17..21)', () => { + it('mirrors into the tap while the full untruncated payload reaches the primary', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2, 3, 4, 5])), + 2, + ); + assert.deepEqual([...(await collect(logged))], [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2]); + }); + + it('clears the tap between writes so a retry does not accumulate stale bytes (BODY-18)', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([9, 9])), + 8, + ); + await collect(logged); + await collect(logged); + assert.deepEqual([...logged.snapshot()], [9, 9]); + }); + + it('aborts the real sink when the delegate refuses before ever touching the adapter', async () => { + // A `ConsumedBodyError` on a second write reaches neither handler on the adapter stream, so without + // the wrapper's own catch the primary writer stays open and locked forever -- a held connection. + // Whether an abort dispatched on a writer reaches the underlying sink's algorithm, and does so + // instead of the close algorithm, is runtime plumbing rather than logic. + const logged = withRequestLogging( + streamBody(streamOf([1, 2, 3]), undefined, 3), + 8, + ); + await collect(logged); // consumes the single-use delegate + + let abortReason = 'NOT-ABORTED'; + let closed = false; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + await assert.rejects( + logged.writeTo(destination), + error => error.name === 'ConsumedBodyError', + ); + assert.equal(abortReason.name, 'ConsumedBodyError'); + // Never closed: a broken message must not be committed downstream as a well-formed short one. + assert.equal(closed, false); + }); +}); + +describe('withResponseLogging over Node Web Streams (BODY-22..28)', () => { + function chunked(...chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); + } + + async function readAll(stream) { + const out = []; + for await (const chunk of stream) out.push(...chunk); + return out; + } + + it('serves the prefix then the still-live tail, one pull at a time (BODY-24)', async () => { + const logged = withResponseLogging(chunked([1, 2], [3, 4, 5]), 3); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + }); + + it('cancelling the tail stream cancels the delegate exactly once (BODY-27)', async () => { + let cancels = 0; + const delegate = chunked([1, 2], [3, 4]); + const inner = delegate.cancel.bind(delegate); + delegate.cancel = async reason => { + cancels += 1; + return inner(reason); + }; + const logged = withResponseLogging(delegate, 1); + + await (await logged.read()).cancel(); + await logged.close(); + assert.equal(cancels, 1); + }); + + it('close() leaves the tap inert instead of poisoning it with a detached-reader TypeError', async () => { + // Node reports a released reader as `TypeError [ERR_INVALID_STATE]: Invalid state: The reader is not + // attached to a stream`. That message used to be cached as this wrapper's drain failure and reported + // by error() forever, over a capture that never failed. + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + await logged.close(); + + assert.deepEqual([...logged.snapshot()], []); + await new Promise(resolve => setTimeout(resolve, 0)); + assert.equal(logged.error(), null); + await assert.rejects( + logged.read(), + error => error.name === 'ClosedResourceError', + ); + }); + + it('a fits-cap capture stays repeatably readable after close (BODY-23, BODY-28)', async () => { + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + await logged.close(); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + assert.equal(logged.error(), null); + }); +}); diff --git a/tests/node-conformance/io-byte-stream.test.mjs b/tests/node-conformance/io-byte-stream.test.mjs index c9165ce..c86f68c 100644 --- a/tests/node-conformance/io-byte-stream.test.mjs +++ b/tests/node-conformance/io-byte-stream.test.mjs @@ -224,3 +224,163 @@ describe('BufferedSink and TeeSink over a Node WritableStream', () => { ); }); }); + +// The five Web Streams bridges had no case here at all until #77. Every one of them is a hand-written +// `ReadableStream`/`WritableStream` underlying-source or -sink object, so what they exercise is the +// runtime's own pull scheduling, cancel dispatch and reader-lock bookkeeping — the three things §5.9 +// names and the three that two independent Streams implementations are most likely to differ on. + +describe('BufferedSource.toReadableStream on Node (IO-16)', () => { + it('pulls one chunk at a time instead of draining the source eagerly', async () => { + let pulls = 0; + const stream = new ReadableStream({ + pull(controller) { + pulls += 1; + if (pulls > 4) { + controller.close(); + return; + } + controller.enqueue(Uint8Array.from([pulls])); + }, + }); + const bridge = BufferedSource.overStream(stream).toReadableStream(); + const reader = bridge.getReader(); + + const first = await reader.read(); + assert.deepEqual([...first.value], [1]); + // Node's default queuing strategy reads one chunk ahead, so at most one pull beyond the one just + // served. The assertion that matters is that the whole 4-chunk source has not been materialized. + assert.ok(pulls <= 2, `expected at most 2 pulls, saw ${pulls}`); + await reader.cancel(); + }); + + it('closes the bridge at natural EOF without tearing down the owning source', async () => { + // IO-19: closing the source here would invalidate every outstanding peek/slice view, defeating the + // bridge's most natural usage — take a preview, hand the bridge to the transport, read the preview + // afterwards. Only an explicit cancel closes the source (next case). + const source = BufferedSource.overStream(streamOfChunks([[1, 2], [3]])); + const preview = source.peek(); + const collected = []; + for await (const chunk of source.toReadableStream()) + collected.push(...chunk); + + assert.deepEqual(collected, [1, 2, 3]); + assert.deepEqual([...(await preview.readBytes())], [1, 2, 3]); + assert.equal(source.closed, false); + await source.close(); + }); + + it('cancelling the bridge closes the source AND releases the caller stream lock', async () => { + const stream = streamOfChunks([[1, 2, 3]]); + const source = BufferedSource.overStream(stream); + assert.equal(stream.locked, true); + + await source.toReadableStream().cancel(); + assert.equal(source.closed, true); + // cancel() cancels the stream but never releases the reader's lock; only releaseLock() does, and a + // leaked lock on a connection-backed source is a held socket. + assert.equal(stream.locked, false); + }); + + it('a mid-stream read failure closes the source rather than stranding the lock', async () => { + // The Streams spec does NOT invoke `cancel` on an errored stream, so the bridge has to close the + // source itself on this path. A runtime that dispatched cancel here would hide the bug. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + }, + pull() { + throw new Error('mid-stream read failure'); + }, + }); + const source = BufferedSource.overStream(stream); + const reader = source.toReadableStream().getReader(); + + assert.deepEqual([...(await reader.read()).value], [1, 2, 3]); + await assert.rejects(reader.read(), /mid-stream read failure/); + assert.equal(source.closed, true); + assert.equal(stream.locked, false); + }); +}); + +describe('BufferedSink.toWritableStream on Node (IO-16)', () => { + it('carries a pipeTo through to the destination and closes it', async () => { + // `pipeTo` closes its destination on natural EOF, and IO-16 says closing the bridge closes the + // sink, which closes the caller's stream. Three closes chained through two runtimes' plumbing. + const written = []; + let closed = false; + const destination = new WritableStream({ + write: chunk => void written.push(...chunk), + close: () => void (closed = true), + }); + const sink = BufferedSink.overStream(destination); + + await streamOfChunks([[1, 2], [3]]).pipeTo(sink.toWritableStream()); + assert.deepEqual(written, [1, 2, 3]); + assert.equal(sink.closed, true); + assert.equal(closed, true); + }); + + it('aborting the bridge aborts the sink and carries the reason, rather than closing it', async () => { + // Collapsing an abort into a graceful close commits a cancelled request body downstream as a + // well-formed complete one, so the peer cannot tell an aborted upload from a successful short one. + let closed = false; + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + const sink = BufferedSink.overStream(destination); + const writer = sink.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('user cancelled'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.equal(closed, false); + assert.equal(sink.closed, true); + }); + + it('drops a zero-length chunk rather than forwarding a chunked-encoding terminator', async () => { + const {stream, written} = collectingStream(); + const sink = BufferedSink.overStream(stream); + const writer = sink.toWritableStream().getWriter(); + await writer.write(new Uint8Array(0)); + await writer.write(Uint8Array.from([7])); + await writer.close(); + assert.deepEqual([...written()], [7]); + }); +}); + +describe('TeeSink.toWritableStream on Node (IO-16, IO-26)', () => { + it('routes through the tee, so bytes written to the bridge still reach the tap', async () => { + // Handing callers the PRIMARY's bridge instead would let every byte written through it bypass the + // tap, silently producing an empty capture. + const {stream, written} = collectingStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 2); + + await streamOfChunks([ + [1, 2], + [3, 4, 5], + ]).pipeTo(tee.toWritableStream()); + assert.deepEqual([...written()], [1, 2, 3, 4, 5]); + assert.deepEqual([...tee.snapshot()], [1, 2]); + }); + + it('forwards an abort to the primary while the tap survives to record what was attempted', async () => { + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + abort: reason => void (abortReason = reason), + }); + const tee = new TeeSink(BufferedSink.overStream(destination), 4); + const writer = tee.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('deadline exceeded'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.deepEqual([...tee.snapshot()], [1, 2, 3]); + }); +}); diff --git a/tests/node-conformance/observability.test.mjs b/tests/node-conformance/observability.test.mjs index 27300a1..dcac9d3 100644 --- a/tests/node-conformance/observability.test.mjs +++ b/tests/node-conformance/observability.test.mjs @@ -5,15 +5,25 @@ // * AsyncLocalStorage store propagation across native Node promises, microtasks, and macrotask timers (OBS-10, OBS-24). // * activateSpan / activateSpanForCorrelation scope restoration and MDC push on Node (OBS-22, OBS-23). // * W3C trace/span identifier randomness via globalThis.crypto.getRandomValues on Node (OBS-26, OBS-27). +// * What a caller's async context holds AFTER `await runtime.send()` resolves (OBS-22, OBS-23, OBS-29, +// audit #67 / #80). Node's AsyncLocalStorage is the mechanism under test, not merely the host: the +// leak this pins was `enterWith` installing on the caller's async resource with the restore closure +// running on a later one, and Bun's suite passed over it for nine phases. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; import { NOOP_SPAN, + PipelineBuilder, + Protocol, + Request, + Response, + Status, activateSpan, activateSpanForCorrelation, createInstrumentationBundle, createLogger, getActiveSpan, + loggingStep, } from '@dexpace/core'; describe('observability on Node.js native runtime floor', () => { @@ -109,3 +119,88 @@ describe('observability on Node.js native runtime floor', () => { assert.equal(emitted[0]['span.id'], spanId); }); }); + +/** A transport literal: two methods, no socket. */ +const okTransport = { + send: async request => + Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(null) + .build(), + close: async () => undefined, +}; + +/** A recording tracer whose spans carry a spanContext, so OBS-23's correlation push actually fires. */ +function recordingTracer() { + const started = []; + return { + started, + tracer: { + startSpan(name) { + const record = {name, ended: 0}; + started.push(record); + const span = { + isRecording: true, + setAttribute: () => span, + recordException: () => span, + end: () => { + record.ended += 1; + }, + spanContext: () => ({ + traceId: '4bf92f3577b34da6a3ce929d0e0e4736', + spanId: '00f067aa0ba902b7', + }), + }; + return span; + }, + }, + }; +} + +describe('async context after Runtime.send on Node.js (OBS-22, OBS-23, OBS-29)', () => { + it('leaves the caller the active span and diagnostic fields it had before the call', async () => { + const {tracer, started} = recordingTracer(); + const emitted = []; + // A pipeline built the public way, with the LOGGING pillar inside it: that step is what pushes + // OBS-23's trace.id/span.id, and before 2026-09-05 the push outlived the call. + const runtime = new PipelineBuilder(okTransport, { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append( + loggingStep({ + granularity: 'headers', + logger: createLogger(() => undefined), + }), + ) + .build(); + const request = Request.newBuilder().url('https://example.com/one').build(); + + assert.equal(getActiveSpan(), NOOP_SPAN); + const response = await runtime.send(request); + assert.equal(response.status.code, 200); + + assert.equal(getActiveSpan(), NOOP_SPAN); + createLogger((level, fields) => { + emitted.push(Object.fromEntries(fields)); + }) + .atLevel('info') + .event('application.event.after.send') + .emit(); + assert.equal(emitted.length, 1); + assert.equal(emitted[0]['trace.id'], undefined); + assert.equal(emitted[0]['span.id'], undefined); + + // OBS-29's 1:1 binding, which the leak also broke: the second call opens its own operation span. + await runtime.send(request); + const operationSpans = started.filter( + span => span.name === 'http.client.operation', + ); + assert.equal(operationSpans.length, 2); + assert.deepEqual( + operationSpans.map(span => span.ended), + [1, 1], + ); + }); +}); diff --git a/tests/node-conformance/retry.test.mjs b/tests/node-conformance/retry.test.mjs index 8fb48d7..c686241 100644 --- a/tests/node-conformance/retry.test.mjs +++ b/tests/node-conformance/retry.test.mjs @@ -9,9 +9,13 @@ // `AbortSignal.timeout()`, whose class and `name` are the runtime's, not this package's -- if // Node named it anything but `TimeoutError`, every timed-out request would silently stop being // retried and `bun test` would still be green. -// 2. RETRY-34's suppressed trail goes through `suppress()`, which picks the native `SuppressedError` -// or the shape-compatible fallback depending on the runtime. Bun has the global; the declared -// floor (`engines.node >=20.3`) does not. The trail's SHAPE has to be identical either way. +// 2. RETRY-34's trail used to go through `suppress()`, which picks the native `SuppressedError` or +// the shape-compatible fallback depending on the runtime -- Bun has the global, the declared +// floor (`engines.node >=20.3`) does not, and this suite's matrix runs both legs. #72 took that +// branch off the retry path entirely: the final attempt's own error is surfaced and the trail +// rides in a side table. So the assertion moved with it, from "the wrapper has the same shape on +// either runtime" to "neither runtime produces a wrapper", which is the stronger claim and the +// one a reintroduced `suppress()` would break differently on Node 20 than on Node 24. // 3. RETRY-35/RECOV-16's "release the discarded response" rides on Web Streams: a retired response is // drained to EOF by `toHttpError()`, an abandoned one is cancelled by `Response.close()`. Node's // `cancel()`/`pull()` timing is an independent implementation of Bun's. @@ -24,7 +28,14 @@ // `dist/` file path, per this suite's import rule. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; -import {Protocol, Request, Response, Status} from '@dexpace/core'; +import { + CancellationError, + Protocol, + Request, + Response, + retryAttempts, + Status, +} from '@dexpace/core'; import { isRetryableFailure, RETRYABLE_STATUSES, @@ -123,13 +134,12 @@ describe('retry classification on the declared Node floor', () => { }); describe('the retry engine on the declared Node floor', () => { - it('folds the suppressed trail into the same shape whether or not the runtime has SuppressedError', async () => { + it('surfaces the final attempt error itself, on a runtime that may lack SuppressedError', async () => { // Timeout aborts, because they are the retryable throwable this suite can build without reaching - // into another `dist/` module -- two of them exhaust the budget and produce a two-entry trail. - const dispatch = scriptedDispatch([ - failure(new DOMException('timed out', 'TimeoutError')), - failure(new DOMException('timed out again', 'TimeoutError')), - ]); + // into another `dist/` module -- two of them exhaust the budget and produce a one-entry trail. + const first = new DOMException('timed out', 'TimeoutError'); + const last = new DOMException('timed out again', 'TimeoutError'); + const dispatch = scriptedDispatch([failure(first), failure(last)]); const outcome = await runWithRetry( GET, @@ -139,9 +149,13 @@ describe('the retry engine on the declared Node floor', () => { assert.equal(dispatch.calls.length, 2); assert.equal(outcome.kind, 'failure'); - assert.equal(outcome.error.name, 'SuppressedError'); - assert.ok('error' in outcome.error); - assert.ok('suppressed' in outcome.error); + assert.equal(outcome.error, last); + assert.notEqual(outcome.error.name, 'SuppressedError'); + // RETRY-34's trail, read through the accessor as a CONSUMER reaches it -- the `@dexpace/core` + // specifier and the built `dist/`, not the engine's own module path. + const priors = retryAttempts(outcome.error); + assert.equal(priors.length, 1); + assert.equal(priors[0], first); }); it('releases a discarded response through the drain route, over Node Web Streams (RETRY-35)', async () => { @@ -217,5 +231,9 @@ describe('the retry engine on the declared Node floor', () => { assert.equal(outcome.kind, 'failure'); // The point of the case: it returned instead of sleeping out the full 60s backoff. assert.ok(defaultClock.monotonic() - startedAt < 5_000); + // XCUT-1 over a REAL AbortSignal and a REAL timer, which is the half the unit suite's injected + // clock cannot reach. The trail is non-empty here by construction, so before #72 this was a + // `SuppressedError` and the assertion below was false. + assert.ok(outcome.error instanceof CancellationError); }); }); diff --git a/tests/node-conformance/rx-bridge.test.mjs b/tests/node-conformance/rx-bridge.test.mjs index 2c1e90a..ae7a9dc 100644 --- a/tests/node-conformance/rx-bridge.test.mjs +++ b/tests/node-conformance/rx-bridge.test.mjs @@ -14,6 +14,10 @@ // suspended pull if the runtime's `ReadableStream` cancellation rejects/resolves the pending read. // 3. `pages$` unsubscribed mid-walk must close the in-hand page's response body (PAGE-11/PAGE-26) through // the same generator-return path. +// 4. The adapter's ownership transfer converges THREE release paths on one resource -- `sseEvents$`'s +// `release`, the iterator's `return()`, and `SseStream`'s own quiet release inside it. Whether they +// collapse to a single resource close is decided by Node's `ReadableStream` cancel semantics and by when +// Node resumes a generator parked in `return()`, not by this package. Bun agreeing proves nothing here. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; import {firstValueFrom, toArray} from 'rxjs'; @@ -258,3 +262,105 @@ describe('pageItems$/pages$ over Node (PAGE-8, ASYNC-6)', () => { assert.deepEqual(closed, [1], 'page 1 body must be released'); }); }); + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both the levels + * `sseStreamFrom` uses: the reader `BufferedSource` takes, and the stream `Response.close()` cancels. + * + * A structural double, because `ResponseBuilder.body()` stores what it is handed. The platform stream's own + * `cancel` hook cannot do this job: Node invokes it at most once per stream and never after the producer has + * closed the controller, so a second release would collapse into the first and read as clean. + */ +function countingBody(bytes, counts) { + return { + get locked() { + return bytes.locked; + }, + getReader() { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => real.releaseLock(), + cancel: reason => { + counts.source += 1; + return real.cancel(reason); + }, + }; + }, + cancel: reason => { + counts.response += 1; + return bytes.cancel(reason); + }, + }; +} + +/** As {@link openSseStream}/{@link closedSseStream}, but reporting every release the owned resource sees. */ +function countingSseStream(text, ended) { + const counts = {source: 0, response: 0, socket: 0}; + const bytes = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (ended) controller.close(); + }, + cancel() { + counts.socket += 1; + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, counts)) + .build(); + return {stream: sseStreamFrom(response), counts}; +} + +describe('SSE ownership transfer releases once on Node (ASYNC-21 departure, SSE-28)', () => { + it('end-of-source reaches each half of the owned resource exactly once', async () => { + const {stream, counts} = countingSseStream( + 'data: one\n\ndata: two\n\n', + true, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + assert.equal(events.length, 2); + await settle(); + // `socket: 0` is not a miss -- the producer ended the byte stream, so Node has nothing left to tear down. + assert.deepEqual(counts, {source: 1, response: 1, socket: 0}); + }); + + it('unsubscribing while a pull is suspended reaches each half exactly once', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + const subscription = sseEvents$(stream).subscribe({next: () => undefined}); + + await settle(); + assert.deepEqual(counts, {source: 0, response: 0, socket: 0}); + + subscription.unsubscribe(); + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); + + it('a throwing mapper reaches each half exactly once, across three release paths (SSE-36)', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + + // `runMapper`'s explicit close, the adapter's `release`, and the mapping generator's `return()` unwinding + // into the facade's quiet release all run. Exactly one of them may reach the resource. + await assert.rejects(() => + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); +}); diff --git a/tests/node-conformance/seams.test.mjs b/tests/node-conformance/seams.test.mjs index ba8f420..1ddc013 100644 --- a/tests/node-conformance/seams.test.mjs +++ b/tests/node-conformance/seams.test.mjs @@ -12,7 +12,7 @@ // 20.3.0. 20.3.0 is the first release carrying both. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; -import {composeSignal, isTimeoutSignal} from '@dexpace/core'; +import {composeSignal, isTimeoutSignal, RequestOptions} from '@dexpace/core'; describe('composeSignal on the declared Node floor', () => { it('returns a distinct AbortSignal.any() result when both a signal and a timeout are supplied', () => { @@ -101,3 +101,37 @@ describe('Web Crypto on the declared Node floor', () => { ); }); }); + +describe('composeSignal timeout range on Node (HTTP-35)', () => { + // Runtime-divergent by measurement, 2026-09-05: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are accepted on Bun, and a + // negative delay raises `RangeError` on Node against `TypeError` on Bun. `bun test` therefore + // cannot assert either half of this, which is what puts the case here rather than only in + // `packages/core/src/seams/transport.test.ts`. Added by audit #67 / #76, which moved the range + // check onto `RequestOptionsBuilder.timeoutMs` for this reason. + it('accepts every timeout RequestOptionsBuilder accepts, at both ends of the range', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + assert.equal(accepted, value); + assert.ok( + composeSignal(undefined, accepted) instanceof AbortSignal, + `composeSignal must accept the timeout ${value}, which the model admits`, + ); + } + }); + + it('would raise RangeError on the values the model now rejects', () => { + for (const value of [1.5, 2 ** 32]) { + assert.throws( + () => composeSignal(undefined, value), + RangeError, + `Node's AbortSignal.timeout() must still reject ${value}; the model is what keeps it unreachable`, + ); + assert.throws(() => RequestOptions.newBuilder().timeoutMs(value), { + name: 'RequestOptionsValidationError', + }); + } + }); +}); diff --git a/tests/node-conformance/serde.test.mjs b/tests/node-conformance/serde.test.mjs index cdf48f1..a10141b 100644 --- a/tests/node-conformance/serde.test.mjs +++ b/tests/node-conformance/serde.test.mjs @@ -261,3 +261,122 @@ describe("decodeResponse's close-failure path on the declared Node floor (SERDE- ); }); }); + +/** + * Fails the case instead of hanging it. `node --test` has no default per-test timeout, so a + * regression in the abort race would park the runner for as long as CI allows rather than reporting + * anything. The timer is ref'd, which also holds the loop open while the abort is in flight. + */ +async function settleWithin(promise, ms) { + let timer; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout( + () => reject(new Error(`did not settle within ${ms}ms`)), + ms, + ); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** Whatever `promise` rejected with, or the string marker when it resolved. */ +async function rejection(promise) { + try { + await promise; + return 'RESOLVED'; + } catch (e) { + return e; + } +} + +describe('an abort landing on a PENDING read or write (SERDE-3, audit #67 / #79)', () => { + // Runtime-divergent twice over. `AbortSignal` and Web Streams are independent implementations + // here, and the two disagree on what a reader release does to an outstanding read: measured + // 2026-09-05, Bun 1.3.14 rejects it with an `AbortError` and Node 20.3/26 with + // `TypeError: Invalid state: Releasing reader`. Neither may reach the caller in place of its own + // abort reason, and neither may escape as an unhandled rejection — which `node --test` would + // report as a failure of this file even if every assertion below passed. + it('settles deserializeFrom with the caller reason and unlocks the source', async () => { + let cancelled = false; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('{"id"')); + }, + // Parks the drain inside its second `read()`: the state a between-chunks check cannot see. + pull() { + return new Promise(() => {}); + }, + cancel() { + cancelled = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + const settled = rejection( + jsonSerde().deserializer.deserializeFrom( + source, + {schema: identity, typeName: 'Dto'}, + {signal: controller.signal}, + ), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(source.locked, false, 'the caller must get its source back'); + assert.equal(cancelled, false, 'the source is caller-owned (SERDE-3)'); + }); + + it('settles serializeTo with the caller reason and unlocks the sink', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write() { + return new Promise(() => {}); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + const settled = rejection( + jsonSerde().serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(sink.locked, false, 'the caller must get its sink back'); + assert.equal(closed, false, 'the sink is caller-owned (SERDE-3)'); + assert.equal(aborted, false); + }); + + it('leaves a completed drain untouched when the signal never fires', async () => { + const controller = new AbortController(); + const value = await jsonSerde().deserializer.deserializeFrom( + streamOf(Buffer.from('{"id"'), Buffer.from(':42}')), + {schema: identity}, + {signal: controller.signal}, + ); + + assert.deepEqual(value, {id: 42}); + // The listener is removed on the way out, so a later abort reaches nothing at all — an + // unremoved one would reject a promise nobody is waiting on any more. + controller.abort(new Error('too late')); + }); +}); diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index c6e16f6..92541c0 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -11,18 +11,27 @@ // `fileBody()` crossing a real transport has no home inside either package's own suite. // // Exercises: TRANSPORT-1 (redirects not followed), TRANSPORT-4/20 (timeout and no-response classification), +// TRANSPORT-20 with RETRY-2 (an unsupported URL scheme is a permanent misconfiguration outside the IoError +// tree -- Node and Bun report it with entirely different error shapes), // TRANSPORT-17 (a single-use body written once, its bytes on the wire), TRANSPORT-24 (vendor status codes), -// TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), +// TRANSPORT-11/12 (a header the native layer refuses is dropped, not a failed send -- Node's undici-backed +// `fetch` rejects three names Bun's forwards), +// TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), BODY-13 (a truncate-after-stat +// short write fails the send on the streamed path, which only this runtime can assert), +// TRANSPORT-24/25 (a 204 and a HEAD carry a null body on this runtime as well -- Node's `fetch` +// returns null where Bun's returns a stream, and undici's dispatcher always returns a readable), +// HTTP-35 (a defaultTimeoutMs AbortSignal.timeout() cannot take is refused at the factory -- Node +// throws RangeError for two of the values Bun accepts), // TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 // (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). import assert from 'node:assert/strict'; import {createServer} from 'node:http'; import {after, before, describe, it} from 'node:test'; -import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; import {createHash} from 'node:crypto'; -import {Headers, Request, RequestOptions} from '@dexpace/core'; +import {Headers, isIoError, Request, RequestOptions} from '@dexpace/core'; import {fileBody} from '@dexpace/body-file'; import {fetchTransport} from '@dexpace/transport-fetch'; import {undiciTransport} from '@dexpace/transport-undici'; @@ -66,6 +75,9 @@ function fixtureBytes(size) { const sha = bytes => createHash('sha256').update(bytes).digest('hex'); +/** `/fixed-length`'s payload; its length is what the HEAD response advertises and never delivers. */ +const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + // Every hook and test lives inside this suite rather than at the file root, and that is // load-bearing on the declared floor. Under Node 20.3.0 -- `engines.node`, and the floor leg of // CI's node-conformance matrix -- an async ROOT-level `before` does not finish before subtests @@ -96,6 +108,21 @@ describe('the transport adapters on the Node runtime', () => { res.end('vendor status body'); return; } + if (pathname === '/no-content') { + res.writeHead(204); + res.end(); + return; + } + if (pathname === '/fixed-length') { + // `node:http` suppresses the body for a HEAD request by itself and keeps the declared + // length, which is the trap: the header promises bytes no response will deliver. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return; + } const chunks = []; req.on('data', chunk => chunks.push(chunk)); req.on('end', () => { @@ -121,9 +148,14 @@ describe('the transport adapters on the Node runtime', () => { }); }); + // `makeTransport` takes the transport-wide default timeout so the HTTP-35 case can build a + // misconfigured transport; every other case passes nothing and gets today's shape. for (const [name, makeTransport] of [ - ['transport-fetch', () => fetchTransport()], - ['transport-undici', () => undiciTransport()], + ['transport-fetch', defaultTimeoutMs => fetchTransport({defaultTimeoutMs})], + [ + 'transport-undici', + defaultTimeoutMs => undiciTransport({defaultTimeoutMs}), + ], ]) { describe(`${name} on the Node runtime`, () => { it('returns a 302 raw and never follows it (TRANSPORT-1)', async () => { @@ -140,6 +172,42 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('drops the headers the native layer refuses rather than failing the send (TRANSPORT-11/12)', async () => { + // The one case in this file where Bun and Node disagree about the *outcome*, not merely the + // implementation. `Expect`, `Keep-Alive` and `Upgrade` are undici's three unconditional + // rejections, and Node's global `fetch` is undici-backed, so on this runtime an undropped + // one rejects the send: `TypeError: fetch failed` through transport-fetch, which can only + // be classified as the RETRYABLE TransportFailureError, and a terminal TypeError through + // transport-undici. Bun's `fetch` instead forwards `Expect`/`Keep-Alive` to the wire and + // hangs on `Upgrade`, so the Bun conformance rows for these names prove a weaker claim than + // this one does (audit #67 / #81, measured 2026-09-05). + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder() + .url(`${origin}/echo`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(), + RequestOptions.newBuilder().timeoutMs(2_000).build(), + ); + const echoed = JSON.parse(await response.text()); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + assert.equal(echoed.headers[name], undefined, name); + } + assert.equal(echoed.headers['x-pass-through'], 'survives'); + } finally { + await transport.close(); + } + }); + it('surfaces a vendor status with a readable body (TRANSPORT-24)', async () => { const transport = makeTransport(); try { @@ -174,6 +242,47 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('reports a null body for a 204 and a HEAD, on this runtime too (TRANSPORT-24/25)', async () => { + // The one place the WHATWG null-body rule can be checked against the runtime the SDK ships + // to. Node's `fetch` returns `null` for 204/304/HEAD by itself, Bun 1.3.14's returns a live + // `ReadableStream` for all three, and undici's dispatcher always hands back a + // `BodyReadable` -- so the Bun conformance rows prove the adapters normalise Bun's answers + // and this proves they did not normalise into Bun's shape (audit #67 / #82). + const transport = makeTransport(); + try { + const empty = await transport.send( + Request.newBuilder().url(`${origin}/no-content`).build(), + ); + assert.equal(empty.status.code, 204); + assert.equal(empty.body, null); + await empty.close(); + + const head = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(`${origin}/fixed-length`) + .build(), + ); + assert.equal(head.status.code, 200); + assert.equal(head.body, null); + // The advertised length survives; only the body a GET would have returned is absent. + assert.equal( + head.headers.get('content-length'), + String(FIXED_LENGTH_BODY.length), + ); + await head.close(); + + // A body-less decision that also nulled an ordinary response would pass every assertion + // above, so the same route is read once more over GET. + const full = await transport.send( + Request.newBuilder().url(`${origin}/fixed-length`).build(), + ); + assert.equal(await full.text(), FIXED_LENGTH_BODY); + } finally { + await transport.close(); + } + }); + it('exposes the response body as a stream that close() releases (TRANSPORT-25)', async () => { const transport = makeTransport(); try { @@ -188,6 +297,39 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('refuses an unhonourable defaultTimeoutMs at the factory (HTTP-35)', async () => { + // Runtime-divergent, and the reason the check exists at all: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2**32)` throw `RangeError` on Node and are accepted by Bun 1.3.14, + // so before audit #67 / #82 the same misconfigured transport failed every send here and + // silently used a different deadline there. The factory now answers identically on both, + // which is what this pins on the runtime that used to be the strict one. + for (const value of [0, -1, 1.5, 2 ** 32, Number.NaN]) { + assert.throws( + () => makeTransport(value), + error => { + assert.ok( + error instanceof TypeError, + `expected a TypeError for ${value}, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + assert.ok(error.message.includes(String(value)), error.message); + return true; + }, + ); + } + // And a legitimate default still builds something that sends. + const transport = makeTransport(30_000); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/echo`).build(), + ); + assert.equal(response.status.code, 200); + await response.close(); + } finally { + await transport.close(); + } + }); + it('classifies a per-call timeout as retryable, not cancellation (TRANSPORT-4)', async () => { const transport = makeTransport(); try { @@ -206,6 +348,38 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('classifies an unsupported URL scheme as permanent, not retryable (TRANSPORT-20, RETRY-2)', async () => { + // Runtime-divergent in the strongest sense: the two runtimes do not merely word this + // differently, they use different error shapes. Node's undici-backed `fetch` rejects + // `ftp://` with `TypeError: fetch failed` carrying `Error: unknown scheme` as its cause -- + // byte-identical, at the top level, to a DNS or connect failure -- while Bun 1.3.14 rejects + // with `TypeError [ERR_INVALID_ARG_VALUE]: protocol must be http:, https: or s3:` and no + // cause at all. The Bun conformance row therefore proves nothing about this runtime, which + // is the runtime the SDK ships to. undici's dispatcher agrees with itself on both + // (`UND_ERR_INVALID_ARG`) and is here for the pairing (audit #67 / #82). + const transport = makeTransport(); + try { + await assert.rejects( + transport.send( + Request.newBuilder().url('ftp://example.com/anything').build(), + ), + error => { + // `classify.ts` is an allow-list over `IoError`, so the class IS the retry verdict: + // a `TransportFailureError` here would spend the caller's whole budget re-proving a + // URL no retry can fix. + assert.ok( + error instanceof TypeError, + `expected a TypeError, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + return true; + }, + ); + } finally { + await transport.close(); + } + }); + it('classifies a dead port as a retryable transport failure (TRANSPORT-20)', async () => { const transport = makeTransport(); try { @@ -345,6 +519,62 @@ describe('the transport adapters on the Node runtime', () => { await transport.close(); } }); + + // BODY-13's short-write clause on the STREAMED request-body path, which is this tree's to + // hold for two independent reasons. The shared conformance row + // (`run-suite.ts`'s "a file body truncated after its length was captured") drives the + // buffered path only: above the adapters' 1,000,000-byte materialize bound the producer + // failure aborts a `TransformStream` mid-pull, and Bun 1.3.14's `Readable.fromWeb` leaks + // that abort reason as unhandled rejections, which `bun:test` scores against an unrelated + // row. Node's bridge does not. And this is the only layer where a real `fileBody()` — whose + // `transferred === count` invariant is the thing under test — meets a real transport. + // + // Until audit #67 / #81 the undici transport handed `createReadStream(path, …)` to undici + // and never called `writeTo` at all, so this case reported 200 with ten bytes on the wire + // while transport-fetch raised. `content-length` is dropped outbound, so the framing cannot + // catch it either. + it('fails a send whose file was truncated after its length was captured (BODY-13)', async () => { + const declared = 1_100_000; + const shortDir = await mkdtemp( + join(tmpdir(), 'dexpace-filebody-short-'), + ); + const shortPath = join(shortDir, 'payload.bin'); + const transport = makeTransport(); + try { + await writeFile(shortPath, fixtureBytes(declared)); + const body = fileBody(shortPath); + assert.equal(body.contentLength, declared); + await truncate(shortPath, 10); + await assert.rejects( + transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(body) + .build(), + ), + error => { + assert.equal(error.name, 'TransportFailureError'); + // BODY-13 names transferred-of-total. The streamed path rethrows the producer's own + // message; the buffered one carries it as a cause, so both are searched. + const chain = []; + for (let at = error; at instanceof Error; at = at.cause) { + chain.push(at.message); + } + assert.ok( + chain.some(message => + message.includes(`transferred 10 of ${declared} bytes`), + ), + `no transferred-of-total in ${JSON.stringify(chain)}`, + ); + return true; + }, + ); + } finally { + await transport.close(); + await rm(shortDir, {recursive: true, force: true}); + } + }); }); }); }