diff --git a/.changeset/core-http-domain-model.md b/.changeset/2026-08-10-core-http-domain-model.md similarity index 100% rename from .changeset/core-http-domain-model.md rename to .changeset/2026-08-10-core-http-domain-model.md diff --git a/.changeset/seam-foundations.md b/.changeset/2026-08-11-seam-foundations.md similarity index 52% rename from .changeset/seam-foundations.md rename to .changeset/2026-08-11-seam-foundations.md index ee18c98..58cc4d5 100644 --- a/.changeset/seam-foundations.md +++ b/.changeset/2026-08-11-seam-foundations.md @@ -2,6 +2,6 @@ "@dexpace/core": minor --- -Add the seam foundations: the `Transport` contract with its `composeSignal`/`isTimeoutSignal` cancellation helpers and `CancellationError`, the operation-input projection (`OperationDescriptor`, `buildRequest`, `OperationAssemblyError`), and `DexpaceError` as the new root of the error taxonomy above `DomainModelError`. +Add the seam foundations: the `Transport` contract with its `composeSignal`/`isTimeoutSignal` cancellation helpers and `CancellationError`, the operation-input projection (`OperationDescriptor`, `buildRequest`, `OperationAssemblyError`), and `DexpaceError` as the root of the error taxonomy. -`DomainModelError` now extends `DexpaceError` instead of `Error`. This is additive — every existing leaf keeps its parent, its behavior, and its `instanceof DomainModelError` narrowing. +Every existing error leaf keeps its behavior and its message. The taxonomy is two levels: a leaf's own superclass is `DexpaceError` itself, and a family is grouped with an exported type guard rather than an intermediate class. diff --git a/.changeset/2026-08-25-body-lifecycle-review-fixes.md b/.changeset/2026-08-25-body-lifecycle-review-fixes.md new file mode 100644 index 0000000..3d3b291 --- /dev/null +++ b/.changeset/2026-08-25-body-lifecycle-review-fixes.md @@ -0,0 +1,24 @@ +--- +"@dexpace/core": minor +--- + +Body lifecycle review fixes. + +Security: + +- Body media types are validated as header-safe at construction (`byteArrayBody`, `stringBody`, `streamBody`, and every part rendered into a multipart body), using the same predicate as outbound header-value validation (HTTP-26). A CR/LF in a media type was previously interpolated verbatim into a multipart part header, which allowed arbitrary header injection, arbitrary part content, and a forged closing boundary while the declared content length still matched the corrupted bytes (HTTP-51). +- `StreamBody.writeTo` now refuses a chunk that would carry the body past its declared `contentLength` *before* writing it, and aborts the sink rather than closing it on any length mismatch. Overrun bytes previously reached the sink and were reported only afterwards, leaving them on the socket behind a stamped `Content-Length` (HTTP-39/BODY-10). + +Correctness: + +- A body write failure is no longer masked by the close that follows it. All five `Body` implementations share one writer scope that aborts on failure and never lets a close error replace the primary one (RECOV-12), so retry classification still sees the I/O failure in the cause chain (RETRY-2). +- `TypedResponse.value()` memoizes a parser that throws synchronously; it previously re-ran the handler and re-read the single-use body (HTTP-44). +- `HttpStatusError.preview()` decodes with the charset declared by the response media type, falling back to UTF-8, and never throws a `RangeError` on an unknown label (HTTP-42). +- `withRequestLogging(...).materialize()` gives the new wrapper its own tap buffer instead of aliasing the original's, so one wrapper's write can no longer rewrite another's captured preview (BODY-21). +- `withResponseLogging` treats a zero-length delegate chunk as a stream-contract violation, matching `RetentionWindow` under IO-17 (BODY-25), and `snapshot()` now starts the lazy drain the way `read()` does (BODY-22). +- `Response.close()` marks the response closed only once the release actually succeeds, memoized so concurrent closers share one cancel — the shape `BufferedSink.close()` already uses (BODY-15, HTTP-43). + +Public API: + +- New `FormBodyValidationError`, reported by `isBodyError`. A form field whose value cannot be rendered is now raised instead of silently dropped from the body. +- `FormUrlEncodedInput` accepts the new `FormUrlEncodedValue` (`string | number | boolean | bigint | null`); primitives render rather than vanish (HTTP-38/BODY-35). diff --git a/.changeset/2026-08-25-body-lifecycle.md b/.changeset/2026-08-25-body-lifecycle.md new file mode 100644 index 0000000..6c296c6 --- /dev/null +++ b/.changeset/2026-08-25-body-lifecycle.md @@ -0,0 +1,7 @@ +--- +"@dexpace/core": minor +--- + +Add the core Body domain interface and implementations (ByteArrayBody, StringBody, FormUrlEncodedBody, StreamBody, MultipartBody, materialize, TypedResponse, HttpStatusError, toHttpError, withRequestLogging, withResponseLogging). + +`RequestBuilder.body` and `ResponseBuilder.body` narrow from `unknown` to `Body | undefined` and `ReadableStream | null` respectively — a breaking parameter-type change per `styleguide/typescript/10-api-design.md`. Resolving Phase 3b's open D1 finding (`docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`, "Open Findings — Phase 3b Validation Review"): kept as **minor** rather than major because `@dexpace/core` is still pre-1.0 (`0.0.0`), where a 0.x breaking change is conventionally released as minor (semver's own carve-out for initial development, https://semver.org/#spec-item-4). Revisit at 1.0. diff --git a/.changeset/2026-08-25-io-contracts.md b/.changeset/2026-08-25-io-contracts.md new file mode 100644 index 0000000..36054cf --- /dev/null +++ b/.changeset/2026-08-25-io-contracts.md @@ -0,0 +1,5 @@ +--- +"@dexpace/core": patch +--- + +Internal: byte-streaming primitives for product-spec §5 (IO-1–IO-42). No public API change. diff --git a/.changeset/2026-08-26-add-the-node-runtime-conformance-suite.md b/.changeset/2026-08-26-add-the-node-runtime-conformance-suite.md new file mode 100644 index 0000000..aaefe70 --- /dev/null +++ b/.changeset/2026-08-26-add-the-node-runtime-conformance-suite.md @@ -0,0 +1,23 @@ +--- +--- + +Add the Node-runtime conformance suite. + +No published package changes. + +Deliberately empty — `changeset --empty` — rather than absent. Every file in this change is repository +infrastructure that ships to nobody: `test/node-conformance/`, `.github/workflows/ci.yml`, `bunfig.toml`, +`eslint.config.js`, the root `package.json` scripts, `CLAUDE.md`, and the phase docs. Zero files under +`packages/` were touched, so there is nothing for `@dexpace/core` to bump and a `patch` here would put a line +in the published changelog that means nothing to a consumer reading it. + +The empty changeset records that the judgement was made, which is the difference between "this change needs no +release" and "somebody forgot a changeset". Verified before writing it: +`git show --stat --name-only e3d0b18 | grep '^packages/'` returns nothing. + +What the change does, for anyone reading this file from the repository rather than the changelog: `bun test` +runs the unit suite on Bun and proves nothing about the runtime the SDK ships to. 319 of 516 unit tests +exercise a runtime-divergent surface — Web Streams, `AbortSignal`, async iteration, `ByteQueue`'s `Uint8Array` +handling — against two assertions of Node coverage that touched none of it. `test/node-conformance/` adds 30 +`node --test` cases over the built artifact, wired as `test:node` and run by CI as a matrix over the declared +`engines.node` floor and current LTS. Closes checkpoint §5.9 / roadmap finding E5. diff --git a/.changeset/2026-08-26-execution-context.md b/.changeset/2026-08-26-execution-context.md new file mode 100644 index 0000000..dd8302d --- /dev/null +++ b/.changeset/2026-08-26-execution-context.md @@ -0,0 +1,46 @@ +--- +'@dexpace/core': patch +--- + +Add the execution-context model for product-spec §7 (`CTX-1`–`CTX-20`, `XCUT-14`). No public API change. + +Everything this adds lives under `packages/core/src/context/` and none of it is re-exported from +`src/index.ts` — `packages/core/etc/core.api.md` is byte-identical before and after. `patch` rather than an +empty changeset because files under `packages/` did change: the published tarball carries the new +`dist/context/*.js`, and a consumer stepping through the package in a debugger will see them. + +What landed: `ExecutionContext` as a three-member discriminated union — `DispatchContext` (before any +request), `RequestContext` (an outbound request assembled), `ExchangeContext` (a response arrived, terminal) — +with `promoteToRequest`/`promoteToExchange` as the pure promotion chain and `createDispatchContext`/ +`createRequestContext`/`createExchangeContext` as the off-chain factories `CTX-5`/`CTX-6` require. +`InstrumentationBundle` plus the `noopInstrumentationBundle` disabled-tracing default. `ContextStore`, a +bounded keyed registry with `install`/`installIfAbsent`/`find`/`close`, and `DuplicateContextKeyError`. + +Three design calls worth recording: + +- **Call keys are `Symbol()`, not a counter or a UUID.** `CTX-4`'s uniqueness requirement cannot lean on any + field of the instrumentation bundle, because `noopInstrumentationBundle`'s fields are all constants shared + by every context that takes the default. A fresh `Symbol()` per call is distinct across the process and + across all three context flavors by construction, and `ContextInit.key` is the pin that makes two contexts + deliberately share one store slot (`CTX-5`). +- **The store's cap drains in a loop, and holds strong references.** `XCUT-14` names context registries first + among the caller-keyed process-lived maps that MUST carry a hard cap and drain back under it after each + insert — an unbounded one is a memory-exhaustion vector, not merely a leak. The loop (rather than a single + check-then-evict) is what makes an insert burst converge. `Map`, never `WeakMap`/`WeakRef`: a registered + context keeps its whole `Request`+`Response` graph reachable on purpose, so the cap is the backstop rather + than the collector (`CTX-19`). +- **Promotions never touch a store.** `context.ts` does not import `store.ts`, which is what satisfies + `CTX-17`'s negative half structurally — constructing a head context must not auto-register it. Wiring the + store into the promotions would invert the layering and make every promotion a global side effect. The + positive half — the first store entry, installed by the first promotion — is Phase 4c's `Runtime.send()`. + +Two known deviations, both already in the deferral register (`docs/work/mvp/2026-09-04-open-items-dissolution.md`): + +- `contextStore` is a module-level mutable singleton, which + `docs/knowledge/harvested/variables-and-declarations.md:22` bans. Accepted because threading a store handle through + builder → runtime → every step would be a wide API change for no observable gain; logged in the design's + Deviation Ledger for Phase 10. Tests build their own `new ContextStore()` rather than asserting through the + singleton, which is shared by every file in a `bun test` run. +- `activeSpan` and `tracerFactory` stay typed `unknown`, and `activeSpan` is `undefined` rather than a no-op + span object. `CTX-14`/`CTX-15` ship as the bundle's frozen shape and the disabled default only; real W3C + Trace Context generation waits for the Phase 7 tracing adapter that gets to define `Span`. diff --git a/.changeset/2026-08-26-max-retries-range-check.md b/.changeset/2026-08-26-max-retries-range-check.md new file mode 100644 index 0000000..76d0272 --- /dev/null +++ b/.changeset/2026-08-26-max-retries-range-check.md @@ -0,0 +1,14 @@ +--- +"@dexpace/core": patch +--- + +Tighten `RequestOptionsBuilder.maxRetries` validation: a defined value must now be a non-negative +integer. `Infinity`, `NaN`, and fractional values were previously accepted and now throw +`RequestOptionsValidationError`, the same way a negative value already did. + +A retry ceiling is a count of wire sends, so a non-finite one is as out of range as a negative one — +and worse in effect: a negative value still fails a downstream `>= 1` guard, while `Infinity` or +`NaN` makes a retry driver's `attempt >= ceiling` test permanently false and its loop unbounded. +HTTP-35's requirement is that an out-of-range retry count is a loud error at the call site that +supplied it, never a value reinterpreted somewhere downstream; this closes the half of that +requirement the setter did not implement. diff --git a/.changeset/2026-08-26-phase3-conformance-fixes.md b/.changeset/2026-08-26-phase3-conformance-fixes.md new file mode 100644 index 0000000..b35f88d --- /dev/null +++ b/.changeset/2026-08-26-phase3-conformance-fixes.md @@ -0,0 +1,51 @@ +--- +"@dexpace/core": minor +--- + +Phase 3 conformance fixes, from a review of the shipped `io/` and `body/` layers against the phase 3a/3b plans. + +Correctness: + +- The request-body logging tee now forwards **both** teardown paths to the sink it was handed. Its adapter stream + declared `write` and `close` but no `abort`, and a `WritableStream`'s default abort algorithm is a no-op — so a + delegate failure aborted the adapter and stopped there, leaving the caller's sink open, still locked, and never + told the message was broken. A truncated body could be committed downstream as a complete one. `writeTo` also + releases the writer when a delegate refuses before ever touching the adapter, which is what a `ConsumedBodyError` + on a second write does (BODY-17, RECOV-12). +- `StreamBody.writeTo` no longer cancels the caller's stream when the sink fails. The unknown-length path used + `pipeTo`'s default `preventCancel: false`, which cancels the *source* on a destination failure — taking + cancellation ownership away from the caller on exactly the failure path, and disagreeing with the + declared-length path, which only releases its reader. Both paths now leave the caller's stream alone (BODY-8). +- Every `Body` variant is frozen at construction. `readonly` is erased at run time, so `contentLength` could be + reassigned after construction and desynchronized from the bytes `writeTo` emits — the same declared-length drift + `MultipartBody` shares one framing routine to prevent, left open on the field a transport stamps into + `Content-Length` (HTTP-1, XCUT-15, HTTP-51). +- `Response` regained the private constructor and `createResponse` friend hook that the body-lifecycle rewrite + dropped. `Response` is exported as a value, so a public field-wise constructor let a caller construct around + `build()`'s required-field validation, and it appeared in the published `.d.ts` (HTTP-2). +- `TeeSink.write` validates its count. `IO-3`'s guard existed as three byte-for-byte copies and the tee — the + fourth size-taking surface — had none, so a negative count was rejected only indirectly, and not at all on its + `count === 0` and short-source early returns. The guard is now single-sourced in `io/limits.ts`. +- `withResponseLogging` enforces the zero-length-chunk contract on the exceeds-cap tail path as well as the drain. + A rule held in one regime and not the other made the same violating upstream pass or fail depending only on how + big the body happened to be (BODY-25). + +Public API: + +- `Response` and the response-body logging wrapper no longer declare `[Symbol.asyncDispose]`; `close()` is the only + teardown interface, matching every other resource-owning class in the package. The symbol postdates the declared + `engines.node` floor (`>=18.17`), where it evaluates to `undefined` and binds the method to the string + `"undefined"`, and its type reached the package only through a dev-only global — so a consumer compiling against + the published `.d.ts` on this package's own declared `lib` failed with + `TS2550: Property 'asyncDispose' does not exist on type 'SymbolConstructor'`. It returns, on all seven resource + owners at once, when the runtime floor moves. +- Every public symbol now carries TSDoc. The committed API report had accumulated 62 `(undocumented)` members, + including 11 of `Response`/`ResponseBuilder`'s own that a wholesale file rewrite had dropped; it is back to zero. + +Internal: + +- `http/charset.ts`'s `decodeText` is renamed `decodeBodyText`. It shares a name with `io/text-codec.ts`'s + `decodeText` while deliberately disagreeing with it: this one delegates every label to `TextDecoder` (so + `iso-8859-1` follows the WHATWG mapping onto windows-1252) and consumes a leading BOM, which is right for a whole + message body; the other implements true ISO-8859-1 for IO-13's round-trip and sets `ignoreBOM` so a mid-stream + BOM survives as ordinary data (SSE-12). Reaching for the wrong one silently changes bytes. diff --git a/.changeset/2026-08-26-phase3-review-pass-2.md b/.changeset/2026-08-26-phase3-review-pass-2.md new file mode 100644 index 0000000..8cdae5d --- /dev/null +++ b/.changeset/2026-08-26-phase3-review-pass-2.md @@ -0,0 +1,44 @@ +--- +"@dexpace/core": minor +--- + +Phase 3 review pass 2. Five defects, each in the same class as one pass 1 already fixed — the earlier fixes +were correct but did not reach every site the same reasoning applies to. + +Correctness: + +- `Response.bytes()`, `Response.text()` and `toHttpError()` now acquire the body reader **inside** the try, so + the response is closed even when the read cannot start. `getReader()` itself throws a `TypeError` when an + external consumer already holds the lock — which `BODY-15` explicitly forbids assuming away, and which + `Response.close()` was already hardened for — so the one failure `BODY-16`'s close guarantee most needs to + cover was the one that skipped the close entirely and held the connection open. +- `MultipartBody.writeTo` verifies the bytes it writes against its own declared `contentLength`. The shared + framing routine keeps the framing consistent but takes each part's own `contentLength` on trust, and + `MultipartPart.body` is the public `Body` interface — so a caller implementation reporting one length and + writing another desynchronized the value a transport stamps into `Content-Length` from what reaches the + socket. An overrunning chunk is now refused before it is written, and a short total raises inside the writer + scope so the sink is aborted rather than cleanly closed (HTTP-51, same shape as `StreamBody`'s HTTP-39 check). +- `withRequestLogging` closes the primary sink when a delegate resolves without closing the adapter. It is the + only place that takes a writer on behalf of someone else's `Body`, so a delegate that ignored `writeTo`'s + close-the-sink contract stranded the caller's sink open and locked with nothing thrown to notice it by. +- A foreign primitive source that over-reports its transferred count now raises `SourceContractViolationError`. + It previously surfaced as `EndOfStreamError: delivered 2 of 99 bytes` — a foreign source's broken accounting + reported as an exhausted stream, which is the exact confusion `IO-17` forbids and which the under-report + direction was already guarded against (IO-17). + +Documentation: + +- `multipartBody`'s `boundary` parameter and `MultipartBodyBuilder.boundary` now state the obligation a + caller-supplied delimiter carries. RFC 2046 requires the sender to pick a boundary that appears in no part, + and that half cannot be checked here — a `StreamBody` part's bytes do not exist until the write, and a partial + scan would read as a complete guarantee. The generated default (32 random characters from Web Crypto) is the + mitigation, and is why it is the default. + +Tooling: + +- New blocking gate `verify:consumer-types`: compiles a throwaway consumer against the built `.d.ts` using the + `lib` and `target` read from `tsconfig.base.json`, with `types: []`. This is the gate whose absence let pass + 1's `Symbol.asyncDispose` defect ship — `typecheck` passes on dev-only ambient globals, `build` emits + regardless, `api` only compares a report, `lint:publish` checks resolution and export shape rather than + whether declarations resolve, and `verify:dual-consumption` runs `node`, not `tsc`. Verified to fail on the + reintroduced defect and pass once reverted. diff --git a/.changeset/2026-08-26-raise-the-node-floor-to-20-3.md b/.changeset/2026-08-26-raise-the-node-floor-to-20-3.md new file mode 100644 index 0000000..bdba70f --- /dev/null +++ b/.changeset/2026-08-26-raise-the-node-floor-to-20-3.md @@ -0,0 +1,39 @@ +--- +"@dexpace/core": minor +--- + +Raise `engines.node` from `>=18.17` to `>=20.3`, and `lib`/`target` from `ES2022` to `ES2023` with it. + +The declared floor was not real. `MultipartBody` generates its boundary from `crypto.getRandomValues`, and Node +exposes `globalThis.crypto` unflagged only from **19.0.0** — never to an ES module on any 18.x release, verified +on both 18.17.0 and 18.20.8. Every `multipartBody(...)` call threw `ReferenceError: crypto is not defined` on the +version `engines.node` promised. `bun test` could not see it, because Bun supplies the global; the Node +conformance suite caught it the first time it ran the built artifact on the pinned floor. + +The floor is `>=20.3` rather than `>=20.0` because `AbortSignal.any()` — `composeSignal`'s own floor-defining +call, backported to 18.17.0 — reached the 20.x line only in 20.3.0. Confirmed by running the suite against a +pinned 20.0.0, where `composeSignal` fails with `AbortSignal.any is not a function`. + +Raising the floor was chosen over the two alternatives that keep Node 18. A `node:crypto` fallback puts a +Node-only specifier in a package documented as running on browsers, Deno, Bun and Workers, and cannot be reached +synchronously from the constructor that needs it. A non-crypto fallback RNG silently downgrades the +unguessable-boundary mitigation `HTTP-51` leans on against multipart injection, on exactly the runtime CI pins. +Node 18 reached end of life in April 2025, so no supported runtime is dropped. + +Also in this change: + +- `verify:runtime-floor`'s pairing table moves its `es2023` row to `>=20.3`, with the built-ins the SDK calls — + not the syntax it emits — named as the reason the floor sits above the language level's own minimum. +- The `node-conformance` CI matrix pins `20.3.0` in place of `18.17.0`. +- The conformance suite gains a case asserting `globalThis.crypto.getRandomValues` is a function **in ESM**, so + this floor cannot regress silently. Node 18 exposed `crypto` to CommonJS while leaving it undefined in ES + modules, so a CJS probe would have reported the old floor as satisfied. +- `seams.test.mjs` holds the event loop open with a ref'd deadline while awaiting an `AbortSignal.timeout()` + abort. That timer is unref'd on every Node version by design, so with nothing else scheduled the loop drained + before it fired and Node 18.17.0's test runner cancelled the rest of the file. Newer runners kept the loop + alive through handles of their own, which is why this passed on current LTS and failed only on the floor. +- `sdk-design-nodejs/02`'s runtime-requirement line is corrected; it had claimed Node ≥18.17 supplies + `globalThis.crypto.subtle`. + +`Symbol.asyncDispose` is still not declared anywhere. The symbols reached the 20.x line in 20.4.0, one patch +above this floor, and re-adding them remains checkpoint §5.4's job across all seven resource owners at once. diff --git a/.changeset/2026-08-26-recovery-chain-primitives.md b/.changeset/2026-08-26-recovery-chain-primitives.md new file mode 100644 index 0000000..a444a15 --- /dev/null +++ b/.changeset/2026-08-26-recovery-chain-primitives.md @@ -0,0 +1,24 @@ +--- +'@dexpace/core': patch +--- + +Add the recovery-chain primitives for product-spec §8.2 (`RECOV-1`–`RECOV-16`). No public API change. + +Everything this adds lives under `packages/core/src/recovery/` plus two package-root helpers, and none of it is +re-exported from `src/index.ts` — `packages/core/etc/core.api.md` is byte-identical before and after. `patch` +rather than an empty changeset because files under `packages/` did change: the published tarball carries the +new `dist/recovery/*.js` and `dist/suppress.js`, and a consumer stepping through the package in a debugger will +see them. + +What landed: `Outcome` with `success`/`failure`/`fold`; `RequestRecoveryChain` and `ResponseRecoveryChain` +(defensive copies on both, concurrency-safe by construction); `dispatchWithRecovery`, whose single `try`/`catch` +wraps both the request chain and the transport hop so no throwable from either can bypass the recovery hooks; +`wrapCancellation`; and `statusMappingStep`, a thin response step over Phase 3b's unchanged `toHttpError()`. +`assertNever` joins `invariant.ts` as the codebase's first discriminated-union `default` case. + +One consumer-visible-in-principle detail worth recording: `RECOV-12` pairs a step's throwable with a close +failure, which is what `SuppressedError` is for — and `SuppressedError` reached Node only in 24.0.0, against +this package's `>=20.3` floor. Rather than raise the floor and drop Node 18, 20 and 22 for one error class, +`suppress()` uses the native class where the runtime has one and returns a shape-compatible stand-in (`name`, +`error`, `suppressed`) where it does not. Code that catches one of these should read its fields, not test +`instanceof SuppressedError`. diff --git a/.changeset/2026-08-26-retry-pillar-and-engine.md b/.changeset/2026-08-26-retry-pillar-and-engine.md new file mode 100644 index 0000000..0f4a5a6 --- /dev/null +++ b/.changeset/2026-08-26-retry-pillar-and-engine.md @@ -0,0 +1,118 @@ +--- +'@dexpace/core': patch +--- + +Add the retry pillar for product-spec §9 (`RETRY-1`–`RETRY-45`) and appendix C's `RECOV-17`–`RECOV-34`, plus +the Phase 7a `config/` prerequisite slice and the shared `FakeTransport`. No public API change. + +Everything this adds lives under `packages/core/src/{retry,config,testing}/` and none of it is re-exported +from `src/index.ts` — `packages/core/etc/core.api.md` is byte-identical before and after. `patch` rather than +an empty changeset because files under `packages/` did change: the published tarball carries the new +`dist/retry/*.js`, `dist/config/*.js`, and `dist/testing/*.js`, and a consumer stepping through the package in +a debugger will see them. (The one behavior change a caller can observe from outside — tightening +`RequestOptionsBuilder.maxRetries` to a non-negative integer — ships under its own changeset.) + +Public-barrel promotion of `retryStep` and the step-authoring surface is deliberately **not** in this release. +A caller cannot assemble a working pipeline until the standard-resilience preset exists, and publishing +`retryStep` alone would freeze `StepDescriptor`/`Stage`/`PipelineBuilder` shapes that still had latitude to +move. Phase 5c owns that promotion. + +## What landed + +`packages/core/src/retry/`, eight files, no folder barrel: + +- **`classify.ts`** — the two orthogonal axes (`RETRY-1`–`RETRY-8`, `RETRY-37`). Retryability is an + ALLOW-list over an iterative, identity-tracking cause walk; `isResendable` is the second axis over + `Body.replayable` and Phase 1's `isIdempotent`. +- **`backoff.ts`, `pacing.ts`** — the pure math and the server-hint parser, split away from the imperative + loop. +- **`settings.ts`** — `RETRY-12`'s defaults, `RECOV-34`'s construction validation, and `totalTimeoutMs` as an + opt-in. +- **`engine.ts`** — the one attempt loop both adapters reach. +- **`attempt-stamp.ts`, `retry-step.ts`, `retry-dispatch.ts`** — per-attempt stamping and the two thin + adapters: the `RETRY` pillar step and the recovery-chain wrapper. + +Plus `recovery/idempotency-key.ts` (`RECOV-32`) and `testing/fake-transport.ts`, which closes the roadmap's +twice-punted `FakeTransport` deferral. + +Two files outside those folders changed, both additively. `StepContext` gains `signal` and `options` +(`PIPE-13`/`PIPE-17`): `Cursor` already carried both and threaded them into terminal dispatch, but no step +could read either, so `RETRY-26`'s cancellable wait and `RETRY-32` were unimplementable and `PIPE-17`'s +"readable by any step" MUST was unsatisfied outright — which is also the wire `RETRY-41`'s per-call +`maxRetries` override (`HTTP-35`) had been missing since Phase 1 designed the knob. + +## Executed out of numeric order: the Phase 7a prerequisite slice + +`config/clock.ts` (`CFG-15`–`CFG-17`), `config/http-date.ts` (`CFG-29`–`CFG-31`), and `config/retryable.ts` +(`CFG-35`) are built here, verbatim from Phase 7a's plan Tasks 1–3, because 5a's Global Constraints ban +shipping the private copies that would otherwise be needed: Task 8 consumes the `Clock` seam, Task 4 imports +the shared RFC 1123 parser, and Task 2 re-exports the shared retryable-status set instead of defining it a +second time. Phase 7a's Tasks 4–10 are untouched, and none of the three enters the public barrel — 7a's Task +10 still owns that decision. + +## Design calls worth recording + +- **One retry loop, reached by both adapters.** `RETRY-13`/`RETRY-14` and `RECOV-30` require the pillar stack + and the recovery-chain stack not to drift. `runWithRetry` is the single choke point both call, so the + schedule, the classifier, and the budget cannot diverge — structural, not a discipline. Every piece of + per-call state is a local (`RETRY-42`/`RECOV-28`), so concurrent invocations sharing one config cannot + clobber each other's attempt count or start instant. +- **`RETRY-25`'s fatal-error exclusion needs no code.** Because classification is an allow-list, a + stack-overflow `RangeError` is non-retryable for never having been opted in, not for having been screened + out. A caller `AbortError` is likewise non-retryable for free (`RETRY-23`), while `TimeoutError` is + explicitly listed (`RETRY-24`) — keying off the abort reason's `name` draws that line more precisely than + the class hierarchy the reference describes. +- **The pacing parser is total, and a failure never maps to `0`.** `RETRY-16` makes never-throwing the + defining property; every malformed, negative, or out-of-range value maps to `null` ("no hint", fall back to + backoff). `0` is reserved for a validly-parsed instant already in the past (`RETRY-17`) — mapping a + malformed header to `0` would hammer a server that just asked for room. `X-RateLimit-Reset` receives + `RECOV-25`'s positive [100%, 120%] jitter so a fleet released at one reset instant does not stampede; a + literal `Retry-After` receives none (`RETRY-20`). +- **`RETRY-36`'s remap applies only to responses the engine DISCARDS.** A response surviving the gates is + returned live and unread: `toHttpError()` drains the body and drops the headers irreversibly, and 4c's + pillar signature must return a `Response`. This is also why the pacing hint is read BEFORE the retire step + — that ordering is load-bearing, not stylistic. +- **`RETRY-27`'s budget clause is implemented as three separate checks, deliberately.** A delay that would + push cumulative elapsed time past the budget SUPPRESSES the retry and surfaces the last failure; the + `Math.min` clamp beside it is the requirement's separately-listed belt-and-braces clause and narrows + nothing except across clock drift between two `elapsed()` reads. It ships because the requirement lists it + separately, not because a test can drive it. +- **A non-finite retry ceiling is guarded at three layers.** Unlike a negative value, which still fails a + downstream `>= 1` guard, `Infinity` or `NaN` makes `attempt >= ceiling` permanently false and the loop + unbounded. The setter, the step's per-call derivation, and a `runWithRetry` precondition each reject it — + the precondition being the one choke point both adapters pass through. +- **`RETRY-41`'s "clamp a negative retry count to the default" is implemented as a REJECTION.** It collides + head-on with `HTTP-35`, also a MUST, which rejects precisely so the value cannot be silently reinterpreted + downstream. The port takes `HTTP-35`'s line on both surfaces; recorded in the design's Deviation Ledger. +- **The inter-attempt wait delegates to `Clock.sleep`.** `CFG-17` already races the timer against the signal, + clears it on both exits (`RETRY-45`'s scheduler hygiene, which has no scheduler object to own in this + port), and rejects promptly for a signal that aborted earlier. Hand-rolling a second `setTimeout`-plus- + listener would put the wait outside the injected seam and force real timers into a suite that must stay + deterministic. Cancellation RESOLVES rather than propagates, so the loop's next iteration observes the + signal and stops through its own `RETRY-32` path. +- **`RETRY-33`'s "every terminal path returns an Outcome" is honored literally.** An attempt that throws is + folded into a failure outcome carrying the trail rather than left to surface as a bare rejected promise, + which would drop `RETRY-34`'s suppressed attempts on the floor. The trail folds through Phase 4b's + `suppress()` helper, not `new SuppressedError(...)`: the native class reached Node only in 24.0.0 and this + package's floor is `>=20.3`. Argument order is controlled explicitly — native `using` disposal builds the + pair the other way round, making the LATER error primary. +- **`RETRY-30`'s trampoline requirement is satisfied by the language.** An `await` loop is already iterative, + so N retries build no continuation chain and no stack growth. +- **`PIPE-36` is satisfied structurally.** `retryStep()` is a factory returning a descriptor with + `stage: 'RETRY'` baked in — no class to subclass, no way for a caller to relocate a shipped pillar family + out of its pillar. 4c deferred this to "whichever future phase ships the first real pillar step family"; + this is that phase. +- **`countingResponse()` counts release by BOTH routes it can happen** — `cancel()` for an abandoned + response, `pull()`-to-EOF for one `toHttpError()` drained. A helper counting `cancel()` alone reads zero on + exactly the `RETRY-35` path it exists to prove. + +## Known gaps, each recorded rather than left silent + +- **`RETRY-29`** (opt-in server-driven retry-classification override) is a `MAY` and is unscheduled: it + widens the classifier's input surface to server-controlled values and wants an explicit trust decision, not + a default. +- **`RECOV-33`** (client-identity header step) belongs with the `CFG-*` work and is Phase 7a's Task 9. +- **`RETRY-40`'s "log the failure" clause and the two SHOULD-level structured events** (`retry.attemptFailed`, + `retry.exhausted`) are not implemented here. 5a executes before 7b, so an `observability/logger.js` import + would not resolve; 7b in turn needs this phase's `FakeTransport`, so the cycle only breaks in this + direction. Phase 7b's Task 9 owns them, named in `engine.ts`'s retrofit note. diff --git a/.changeset/2026-08-26-stage-based-pipeline.md b/.changeset/2026-08-26-stage-based-pipeline.md new file mode 100644 index 0000000..0dbbd6a --- /dev/null +++ b/.changeset/2026-08-26-stage-based-pipeline.md @@ -0,0 +1,52 @@ +--- +'@dexpace/core': patch +--- + +Add the stage-based pipeline for product-spec §8.1 (`PIPE-1`–`PIPE-40`). No public API change. + +Everything this adds lives under `packages/core/src/pipeline/` and none of it is re-exported from +`src/index.ts` — `packages/core/etc/core.api.md` is byte-identical before and after. `patch` rather than an +empty changeset because files under `packages/` did change: the published tarball carries the new +`dist/pipeline/*.js`, and a consumer stepping through the package in a debugger will see them. + +What landed: `Stage` and `STAGE_ORDER`, the fixed total order from `PRE_REDIRECT` out to the reserved terminal +`SEND`, with `PILLAR_STAGES` marking the slots that admit at most one step. `Step`, `StepContext`, `Next` and +`StepDescriptor` as the step contract. `PipelineBuilder`, the surgical-edit API — `append`/`prepend`/ +`appendAll`/`prependAll`/`insertAfter`/`insertBefore`/`replace`/`remove`/`reload` — flattening into an +immutable `Runtime` at `build()`. `Cursor`, one instance per call, driving the flattened array. Five typed +errors: `PillarCollisionError`, `AnchorNotFoundError`, `CrossStageEditError`, `CursorAlreadyAdvancedError`, +`ReservedStageError`. + +Design calls worth recording: + +- **`Runtime` implements `Transport` itself (`PIPE-26`), and its `close()` is a deliberate no-op + (`PIPE-27`).** Phase 2's `Transport` SPI has a single `send`, so there is no second async entry point to + delegate through. The pipeline never owns the transport it wraps, so closing the pipeline must not close it. +- **Continuations are one-shot, and a fork is a closure, not a second cursor.** `next` and every `fork()` + handle are one-shot closures over one private recursive dispatcher indexed by array position + (`PIPE-15`/`PIPE-16`); reusing an already-invoked handle rejects with `CursorAlreadyAdvancedError`. There is + deliberately no settable start position — a step that must re-drive the chain calls `ctx.fork()` again. The + dispatcher shares one mutable in-flight request, so a `PIPE-14` substitution sticks for every later step + *and* the terminal dispatch. +- **`Stage` is a string-literal union, not an enum.** `erasableSyntaxOnly` bars enums, and `Stage` carries no + behavior beyond ordering, which `STAGE_ORDER` alone provides. Adding a stage later is one splice into that + array — no existing `Stage` value changes, so there is no numeric-gap renumbering to design around. +- **`prependAll` reverses its batch and `appendAll` does not.** The asymmetry falls out of prepending each + element individually, and is the documented one `PIPE-38` allows rather than an oversight. `reload` is the + transactional bulk path (`PIPE-23`): fully validated before any existing content is touched, so a rejected + batch leaves the builder untouched instead of half-applied. +- **`replace` is the sanctioned way past a pillar collision.** `PIPE-5` exempts it from the pillar check; + re-seating the *same* `type` symbol anywhere is an idempotent no-op rather than a second step (`PIPE-6`), + which is also what keeps the bulk paths from seating two steps where `append` would seat one. +- **`send()` closes `CTX-17`'s positive half.** The first promotion installs into Phase 4a's `contextStore`, + the exchange promotion replaces it under the same key, and the `finally` evicts whichever context was + installed last. `exchangeSource()` is exported (still `@internal`) so its two branches can be asserted as + the pure function they are: when a step substituted the outbound request, the exchange is promoted from an + off-chain rebuild around the request that was *actually sent*, pinned to the same call key and carrying the + same instrumentation bundle by reference. Promoting straight off the original would pair the response with a + request that never left the process. + +One deferral, recorded in `docs/work/mvp/2026-09-04-open-items-dissolution.md`: `StepContext` carries neither the per-call `options` nor the +`AbortSignal`. `Cursor` holds both and threads them into the terminal dispatch (`PIPE-17`), but the +"readable by any step" clause has no reader until Phase 5a's retry engine, which adds both fields as one +additive amendment. diff --git a/.changeset/2026-08-27-codec-json.md b/.changeset/2026-08-27-codec-json.md new file mode 100644 index 0000000..bb6378d --- /dev/null +++ b/.changeset/2026-08-27-codec-json.md @@ -0,0 +1,18 @@ +--- +'@dexpace/codec-json': minor +--- + +Initial release of the reference JSON wire codec: `jsonSerde()`, the `Tristate` PATCH replacer (on by +default, opt-out is an explicit `{tristate: false}`), and the `tristate()` / `tristateObject()` decode +combinators. Depends on nothing beyond a `@dexpace/core` peer — the schema that witnesses each decode +is the caller's, so no schema library is a dependency of either package. + +Encoding details worth knowing at the call site: a top-level `undefined`, function, or symbol raises +`SerializationError` rather than encoding as the `null` literal — all three are unencodable values, +and substituting `null` would send a PATCH server a meaningful "clear this field" the caller never +wrote. Nested occurrences keep ordinary `JSON.stringify` behaviour. The `SERDE-20` top-level Tristate +degradation (a top-level Absent or Null still encodes as `null`) is resolved by the serializer before +`JSON.stringify` runs, because a replacer cannot tell the top-level value from a key literally named +`""`; a caller composing their own `JSON.stringify(v, tristateReplacer)` therefore gets the nested and +array-element behaviour but must route through `jsonSerde()` for the top level. + diff --git a/.changeset/2026-08-27-configuration-and-platform-primitives.md b/.changeset/2026-08-27-configuration-and-platform-primitives.md new file mode 100644 index 0000000..e33ba8a --- /dev/null +++ b/.changeset/2026-08-27-configuration-and-platform-primitives.md @@ -0,0 +1,7 @@ +--- +"@dexpace/core": minor +--- + +Add the configuration subsystem and the shared platform primitives (Phase 7a): the layered `Configuration` model with its `ConfigurationBuilder`, substitutable env/property seams, never-throw typed accessors, copy-on-write `derive`, the process-wide global slot, and the well-known `CFG_KEY_*` constants; the injectable `Clock` seam and `defaultClock`; RFC 1123 `formatHttpDate`/`parseHttpDate`; the shared `isRetryableStatus`/`RETRYABLE_STATUSES` classifier; `randomUuid`; the `ProxyOptions` model with `createProxyOptions`, `formatProxyOptions`, `shouldBypassProxy`, and `resolveProxyOptions`; and the `BuildInfo` descriptor behind `getBuildInfo`. + +`@dexpace/core`'s own version is now compiled in at build time by `scripts/gen-version.mjs`, which the package's `prebuild` step runs — so a runtime-emitted identifier reports the real version rather than an `unknown` placeholder, with no runtime `package.json` read on any runtime. diff --git a/.changeset/2026-08-27-configuration-review-pass-2.md b/.changeset/2026-08-27-configuration-review-pass-2.md new file mode 100644 index 0000000..db5869f --- /dev/null +++ b/.changeset/2026-08-27-configuration-review-pass-2.md @@ -0,0 +1,61 @@ +--- +"@dexpace/core": minor +--- + +Phase 7a review pass 2 (adversarial). Fourteen defects found by enumerating boundaries, failure paths, and +lifetimes against the running code. The public API surface is unchanged — `etc/core.api.md` is +byte-identical — but several of these change observable behavior, so they are recorded here. + +Security and availability: + +- `shouldBypassProxy` no longer compiles bypass globs to a regular expression. Translating `*` to `.*` + produced adjacent unanchored runs, and a non-matching host then drove catastrophic backtracking: the + operator-supplied `NO_PROXY` entry `*a*a*a*a*a*a*a*a*a*b` against a 60-character host blocked the event + loop for 38 seconds. A two-pointer wildcard walk replaces it — 0.02ms on that case, `O(pattern × text)` at + worst (CFG-23). +- `getBuildInfo().identityTokens` is now header-safe at its source. The runtime identity is read from + ambient values (`process.version`, `Deno.version.deno`, `navigator.userAgent`) that were returned + untrimmed and unvalidated, so a single non-ASCII byte in a browser `navigator.userAgent` made the default + `clientIdentityStep` reject **every** outbound request with a `HeaderValidationError`. An unusable value + now falls back to `unknown` (CFG-36, RECOV-33, NFR-15). +- `RETRYABLE_STATUSES` is genuinely immutable. The `ReadonlySet` type is compile-time only and + `Object.freeze` does not seal a `Set`'s internal slots, so `(RETRYABLE_STATUSES as Set).add(418)` + succeeded and permanently rewrote the process-wide retry classifier for the whole program. `add`, `delete`, + and `clear` now throw (CFG-35, RETRY-1). + +Correctness: + +- `resolveProxyOptions` honors an explicitly written default port. The WHATWG URL parser normalizes a + special scheme's default port to the empty string, so `HTTP_PROXY=http://proxy:80` and + `HTTPS_PROXY=https://proxy:443` — the two most common proxy configurations there are — both resolved to + `null` and routed direct. CFG-25 bans *guessing* an absent port, not honoring one the operator wrote; a + URL with no port at all is still rejected (CFG-25). +- `resolveProxyOptions` no longer throws a `URIError` on a literal `%` in proxy credentials. The + percent-decode sat outside the parse `try`, and an un-encoded password containing `%` is ordinary operator + input (CFG-24). +- The layered lookup is total against any seam. A `Record`-backed source — `process.env` included — resolves + a key named `__proto__`, `constructor`, or `toString` through `Object.prototype`, so `getString` returned a + *function* typed as `string | undefined` and `getInt`/`getBoolean`/`getDuration` died on a raw `TypeError`. + A seam that throws escaped unwrapped through the same accessors. Both now fall through as "this layer + supplies nothing" (CFG-5, CFG-6, CFG-7, CFG-11). +- `Clock.sleep` rejects a duration above `2 ** 31 - 1` ms instead of firing almost immediately. `setTimeout` + silently clamps a larger delay to `1`, so `sleep(2 ** 31)` returned in 7ms rather than waiting 24.8 days — + an overflowed retry backoff became no backoff at all (CFG-17). +- `Clock.sleep(0)` yields to the event loop rather than only to the microtask queue. The previous + `Promise.resolve()` short-circuit let a zero-backoff loop spin 4.1 million times in 300ms without a pending + `setTimeout(fn, 0)` ever running (CFG-17). +- `formatHttpDate` rejects an instant outside the four-digit-year span RFC 1123 renders. `padStart(4, '0')` + emitted the malformed `00-1` for year −1 and `275760` for `Date`'s upper limit, neither of which survived a + round-trip back through `parseHttpDate` (CFG-29). +- The proxy port accepts only a bare run of decimal digits. Bare `Number()` also read `0x10` as port 16, + `1e2` as 100, `0b11` as 3, and `80.0`/`+80`, silently connecting to a port the operator never wrote + (CFG-25). +- An IPv6 proxy address resolves to the same bare form from either configuration tier, rather than bracketed + from the environment URL and bare from the system property (CFG-22, CFG-24). +- An empty user name means no credentials on both tiers, so a blank `https.proxyUser` no longer fabricates a + masked `***:***@` for a proxy that has none (CFG-24). +- `randomUuid` names its missing dependency when a runtime exposes no global WebCrypto, instead of reporting + `TypeError: Cannot read properties of undefined (reading 'getRandomValues')` (CFG-32). +- `setGlobalConfiguration` rejects a present-but-wrong value rather than only a null one, matching every + other CFG-37 guard in the module (CFG-37). +- `Configuration.getInt` normalizes `-0` onto `0`. diff --git a/.changeset/2026-08-27-configuration-review-pass-3.md b/.changeset/2026-08-27-configuration-review-pass-3.md new file mode 100644 index 0000000..96736ab --- /dev/null +++ b/.changeset/2026-08-27-configuration-review-pass-3.md @@ -0,0 +1,30 @@ +--- +"@dexpace/core": patch +--- + +Phase 7a review pass 3 (readability and convention). No behavior changes. Two public parameter names change, +which is the whole of the `etc/core.api.md` diff: + +- `Clock.sleep(ms, signal)` becomes `Clock.sleep(durationMs, signal)`. A bare `ms` is a unit with no concept + attached, and the report carried it two lines above `composeSignal(userSignal, timeoutMs)` — the same + package stating the same kind of quantity two different ways + (`docs/knowledge/harvested/naming-conventions.md:36`). +- `Configuration.getDuration(key, fallback)` becomes `getDuration(key, fallbackMs)`. The accessor returns + and accepts milliseconds, and said so only in prose while its own private collaborator is named + `parseDurationMs`. + +Positional callers are unaffected; only the name shown in editor hints and the emitted `.d.ts` changes. + +The rest of the pass is documentation and test strength, with nothing observable to a consumer. The +documentation fixes worth naming, because each was a comment that had stopped matching its code: + +- `Clock.sleep`'s TSDoc claimed the timer was cleared "on both the resolve and the abort path". Only the + abort path clears a timer; the resolve path detaches the abort listener. +- `randomUuid` carried a comment describing an `unknown` widening that no longer exists, and pointed at + `setGlobalConfiguration` for a shape it no longer shares. +- `composeHeaders`'s doc block sat on the interface declared above it, so the function was undocumented and + the interface was described as if it wrote headers. +- Every `Configuration` and `ConfigurationBuilder` `@throws` said "when `x` is absent"; every guard is a + `typeof` shape check, which is what the module's own comment says they are. +- The package barrel justified not exporting `deepEqual`/`deepHash` partly on "in-package consumers import + the module directly". They have no in-package consumer, which `docs/work/mvp/2026-09-04-open-items-dissolution.md` G16 already recorded. diff --git a/.changeset/2026-08-27-pagination-engine.md b/.changeset/2026-08-27-pagination-engine.md new file mode 100644 index 0000000..4ea8669 --- /dev/null +++ b/.changeset/2026-08-27-pagination-engine.md @@ -0,0 +1,89 @@ +--- +'@dexpace/core': minor +--- + +Add the pagination engine for product-spec §12 (`PAGE-1`–`PAGE-36`). The public surface is `Paginator` +with its two views, the `Page` resource, the `PageInfo` / `pageInfo()` pair and the +`PaginationStrategy` interface, three built-in strategies (`cursorStrategy()`, `pageNumberStrategy()`, +`linkHeaderStrategy()`), the fetcher-driven front end `paginateWithFetchers()` with `PagingOptions` and +`FetcherPage`, and the `PaginationError` leaf. + +The engine drives a `Transport` directly and stays serde-agnostic: item extraction is a caller-supplied +callback on every built-in strategy, never a `Serde`. Resilience composes from outside — 4c's `Runtime` is +itself a `Transport`, so a full retry/redirect/auth pipeline drops in as the `transport` field with no +pagination-side change, recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` §J5. The query splice and the `Link` +tokenizer stay internal; publishing them would stand a second URL-manipulation surface next to Phase 1's +`QueryParams`, which is the confusion the one-encoder rule exists to avoid. + +What landed under `packages/core/src/pagination/`: `page.ts`, `strategy.ts`, `paginator.ts`, +`strategies.ts`, `link-header.ts`, `query-splice.ts`, `fetchers.ts`, and `errors.ts`, plus +`test/node-conformance/pagination.test.mjs`. + +Three files changed outside it. `packages/core/src/http/query-params.ts` now exports +`encodeQueryComponent`/`decodeQueryComponent` (both `@internal`, so the API report is unaffected) — +`PAGE-22` restates HTTP-29's encoding rule verbatim, and two encoders in one codebase is a drift bug +waiting to happen. `packages/core/src/testing/fake-transport.ts` gains `sentOptions`/`sentSignals` +accessors and an init-object overload on `countingResponse()`. And `tsconfig.base.json` adds +`ESNext.Disposable` to `lib` — see the caveat below, because that one reaches consumers. + +Five design calls worth recording: + +- **Each page is closed *before* any of its items are yielded**, not in a `finally` after. `PAGE-11` + mandates the ordering and `sdk-design-nodejs/07` §7.1's illustrative snippet shows the opposite — it + closes after the yield, which holds the response open for the entire item walk and still passes the + requirement's stated conformance test. Materialized items survive close (`PAGE-2`), so closing first + costs nothing and means abandoning iteration mid-page can never strand a connection, however long the + consumer takes. An erratum callout was added to §7.1; recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` §J1. +- **`PaginationStrategy.parse` is asynchronous, against `PAGE-5`'s literal wording.** The requirement says + a strategy reads what it needs "synchronously inside parse"; this runtime has no synchronous body read, + because the bytes may not have arrived. Every enforceable part of the intent — isolated, non-mutating, + one read, no retained body — survives the promise and is stated on the interface, since none of it is + expressible in the type system. Recorded at §J2. It cannot be "fixed" back to a synchronous signature. +- **The query splice is hand-rolled rather than `URLSearchParams` or `QueryParams`.** Both re-serialize + the *whole* query through their own canonical encoding on every mutation: untouched parameters get + reordered and re-encoded, against `PAGE-21`'s byte-for-byte rule, and a space becomes `+` rather than + the `%20` this port standardizes on. `query-splice.ts` tokenizes the raw query substring and copies + every untargeted byte through, sharing only the component *encoder* — the part `PAGE-22` and `HTTP-29` + genuinely agree on (§J4). +- **`Link` parsing is a scanner, not a regular expression.** The separator rules are context-sensitive in + two directions at once: a comma splits link-values only outside both angle brackets and quoted strings, + and a semicolon splits parameters under the same condition — and quoted strings support `\"` escapes, so + quote tracking cannot be a simple toggle. A target that fails to resolve is end-of-stream, not an error + (`PAGE-19`), which is one of the few places in this codebase where swallowing an exception is the + specified behavior rather than a smell. +- **`items()` is re-iterable and `pages()` is single-use.** The asymmetry is deliberate: each `items()` + walk closes every page before yielding, so a second iteration simply drives a second fetch sequence + (`PAGE-8`), while `pages()` hands out live connection-owning objects whose re-iteration would + double-consume unclosed resources (`PAGE-14`). `paginateWithFetchers()` is single-use for the same + reason — a second loop would re-run `first()` and break `PAGE-34`'s "exactly once" (§J6). + +Limits worth knowing at the call site: + +- **`Page` declares `implements AsyncDisposable` unconditionally, and this package's declared `lib` grew + `ESNext.Disposable` to make that compile.** A consumer compiling the published `.d.ts` needs the same + lib entry (`"ESNext.Disposable"`, or `esnext`) or `Page` will not typecheck for them. This is also the + one place the SDK is now internally inconsistent about explicit resource management: `Response` + (`HTTP-38`) and Phase 6b's `SseStream` install `[Symbol.asyncDispose]` behind a runtime guard precisely + because the declared `engines.node` floor is `>=20.3` and the symbol landed in 20.4, where a computed + key evaluating to `undefined` binds the method to the string `"undefined"` instead. `Page` takes the + unguarded route (§J3), so on the declared floor `await using page = ...` does not dispose and the class + carries a stray `"undefined"` method. `test/node-conformance/pagination.test.mjs` does not catch this — + its `page[Symbol.asyncDispose]` lookup coerces the key the same way the class definition did, so the + assertion passes on 20.3 without exercising anything. Resolving this one way or the other is a + floor-bump decision, not a pagination one. +- **Cancellation cannot reach a response the engine never received** (`PAGE-33`). If `signal` aborts + before the transport delivers, releasing that response is the transport's job. A request already + dispatched may still complete after the abort; when it does, the engine closes and discards it rather + than yielding it. +- **`PaginationError` is reserved for engine misuse** — a non-positive `maxPages` at construction + (`PAGE-9`), or a second iterator on a single-use view (`PAGE-14`). Transport, parse, and close failures + propagate as whatever the underlying layer raised, because `PAGE-28` requires the original cause to + surface rather than a pagination-flavored wrapper (§J8). +- **Ownership transfers to the page.** A fetcher builds a `Page` and must not close its response; the + engine closes it as the consumer advances and at exhaustion. A fetcher that throws *before* building the + page still owns whatever response it opened — the engine never saw it and has no handle to close it + with. +- **Two built-in strategies defend against servers that never signal termination.** `cursorStrategy` + treats an empty-string cursor as end-of-stream alongside `null`, and `pageNumberStrategy` stops on an + empty item list before any arithmetic runs. Both would otherwise walk forever against a server that + keeps answering past the end. diff --git a/.changeset/2026-08-27-redirect-pillar-step.md b/.changeset/2026-08-27-redirect-pillar-step.md new file mode 100644 index 0000000..5cc4c9e --- /dev/null +++ b/.changeset/2026-08-27-redirect-pillar-step.md @@ -0,0 +1,75 @@ +--- +'@dexpace/core': patch +--- + +Add the redirect-following pillar step for product-spec §10 (`REDIR-1`–`REDIR-27`) and close `PIPE-40`. No +public API change. + +Everything this adds lives under `packages/core/src/redirect/` and none of it is re-exported from +`src/index.ts` — `packages/core/etc/core.api.md` is byte-identical before and after. `patch` rather than an +empty changeset because files under `packages/` did change: the published tarball carries the new +`dist/redirect/*.js`, and a consumer stepping through the package in a debugger will see them. + +One file landed outside `redirect/`: `packages/core/src/recovery/release.ts`, which is +`releaseQuietly`/`withReleaseFailure` extracted unchanged from `retry/engine.ts`. The redirect step needs +the same "a teardown failure never becomes primary" discipline `RECOV-12` already required of retry, and +the helper's identity guard is subtle enough that a second copy would drift. `engine.ts` now imports what +it used to define; its behavior and its suite are unchanged. + +What landed: `codes.ts` (the recognized `{301,302,303,307,308}` set and per-code method eligibility), +`cross-origin.ts` (the RFC 6454 origin tuple compared against the seed, plus the credential-suppression +marker header), `settings.ts` (validated, frozen policy with a defensively copied allowed-method set), +`decide.ts` (the pure per-hop decision), `redirect-step.ts` (the `REDIRECT` pillar adapter), and +`strip-marker-step.ts` (a `POST_AUTH` guard plus `withRedirect()`). Two new operational error leaves, +`NonReplayableBodyError` and `SchemeDowngradeError`, both `@internal` for now. + +Four design calls worth recording: + +- **The cross-origin suppression signal is a real header, not an in-process marker.** A `WeakSet` + keyed by object identity is unforgeable and never touches the wire, but stage order is + `REDIRECT → RETRY → AUTH` and 5a's attempt-stamping builds a fresh per-attempt `Request` copy when + enabled — an identity-keyed signal would silently stop matching exactly when a retry sits between + redirect and auth, which is when cross-origin credential suppression matters most. Stamping preserves + headers, so a header survives the intermediate copy. +- **A second, always-bundled step strips that marker independently of whether an auth step exists.** + `REDIR-11` itself names the porter caveat: in the reference only the auth step strips the signal, so a + pipeline with none forwards it to the transport. 5b ships before 5c, so that is not a future concern + here — it is a live leak this phase would otherwise ship. `stripCrossOriginMarkerStep()` occupies 4c's + inert `POST_AUTH` extension slot, so nothing in 4c or 5c had to change. +- **Two origin-shaped checks, two deliberately different reference points.** Cross-origin classification + compares against the **seed** origin for the whole chain (`REDIR-8`), so a foreign host cannot hand the + credential back by redirecting to the seed's own origin. The scheme-downgrade guard compares the + **current hop** against its target (`REDIR-15`), so an HTTPS→HTTP→HTTPS chain flags only the hop that + actually downgraded. Conflating them silently breaks one or the other. +- **A failing release never replaces the error it was supposed to let through.** `Response.close()` + rethrows whatever cancelling the body raised, so the two error paths that close before propagating + (`decideOrClose`, and the `'fail'` branch's `SchemeDowngradeError`) route through + `withReleaseFailure`: the decision error stays primary and the release failure rides along as + `suppressed`. The third close — releasing a superseded hop before the next drive — is deliberately + left bare, because there is no primary error to preserve and `PIPE-40` makes the release itself part + of the contract. +- **Location resolution ends with an explicit `http:`/`https:` gate.** WHATWG `URL` parses + `javascript:`, `data:`, `file:`, and `mailto:` without complaint, and the downgrade guard waves all of + them through (none is `http:`). Without the gate the step would dispatch a server-supplied + `javascript:` target. The `catch` around `new URL(raw, base)` is a genuinely narrow path, not the + general garbage guard it looks like: with a base supplied, a non-URL string resolves as a relative + reference rather than throwing. + +One normative conflict, resolved and recorded rather than silently picked: **`PIPE-40` and `REDIR-22` +disagree, both at `MUST`, about the non-replayable-body path.** `PIPE-40` lists it among the paths whose +in-flight response is "returned unclosed"; `REDIR-22`(b) lists the same trigger among those "closed before +the error propagates". `REDIR-6` settles the control flow — that path "MUST fail with a clear error" — so it +throws, and a response never returned cannot be returned unclosed. 5b closes and throws; the contradiction +is in the design's Deviation Ledger and deferred to Phase 10, which owns the erratum either way. + +Two known gaps, both recorded in the phase checklist: + +- **`REDIR-28`'s structured hop/loop/downgrade log events, and `REDIR-15`'s "surface it observably" clause + on a permitted downgrade, are not implemented here.** Phase 5b executes before Phase 7b, so + `redirect-step.ts` cannot import `observability/`, and 7b needs this step for its own retrofit test — + the dependency cannot run the other way. Phase 7b's Task 9 owns them, named in `redirectStep()`'s TSDoc. +- **`REDIR-20`'s predicate override is read as scoped to code/method eligibility only.** A configured + predicate replaces the built-in follow decision; it does not bypass userinfo stripping, credential + hygiene, the downgrade guard, the replayability gate, or loop/cap detection, all of which the same spec + document states as unconditional `MUST`s. Logged in the design's Deviation Ledger for Phase 10 and + flagged for re-confirmation at Phase 9's conformance sweep. diff --git a/.changeset/2026-08-27-resilience-auth.md b/.changeset/2026-08-27-resilience-auth.md new file mode 100644 index 0000000..43900e2 --- /dev/null +++ b/.changeset/2026-08-27-resilience-auth.md @@ -0,0 +1,116 @@ +--- +'@dexpace/core': minor +--- + +Ship the authentication layer (product-spec §11, `AUTH-1`–`AUTH-38`) and promote the pillar-authoring surface +to the public barrel. **This is the first release with new public API since Phase 1.** + +`minor`, not `patch`: `packages/core/etc/core.api.md` gains the whole pipeline-authoring surface plus the auth +configuration types its signatures name, and `RequestOptions` gains one member. Nothing is removed or +narrowed, so no consumer breaks. + +## What a caller can now do + +```ts +import { + ApiKeyCredential, + createAuthDescriptor, + createAuthRequirement, + standardResilience, +} from '@dexpace/core'; + +const client = standardResilience(transport, { + auth: { + credentials: {apiKey: {credential: new ApiKeyCredential(process.env.API_KEY ?? '')}}, + tiers: {client: createAuthDescriptor([createAuthRequirement('API_KEY')])}, + }, +}); +``` + +`standardResilience()` installs redirect, retry, and auth in that order — `AUTH-27`'s "redirect wraps retry +wraps auth" — so auth re-resolves and re-stamps per redirect hop and per retry attempt. `PipelineBuilder`, +`retryStep`, `redirectStep`, and `authStep` are exported for hand-assembling a pipeline instead, and +`PipelineBuilder.seedFrom(runtime, 'flatten' | 'nest')` composes one pipeline onto another with the choice +explicit rather than accidental (`PIPE-35`). + +## What landed + +The scheme-agnostic descriptor/resolver model (`AuthScheme`, `AuthRequirement`, `AuthDescriptor`, +`resolveAuthRequirement`), the credential types (`BearerToken`, `ApiKeyCredential`, `NameKeyCredential`, +`TokenProvider`), a total RFC 7235 challenge parser, a dependency-free MD5, the Basic/Digest/static-key +stamping handlers, a single-flight three-zone bearer token cache, and one AUTH pillar step tying them +together. `RequestOptions` gains `auth?: AuthDescriptor`, which fills `AUTH-4`'s most-specific `perCall` tier. + +Zero runtime dependencies still (`SEAM-1`). SHA-256 and the Digest client nonce go through +`globalThis.crypto`, and Basic stamping through `globalThis.btoa`, never `node:crypto` — the package stays +portable to browsers, Deno, and Workers. MD5 is hand-rolled because Web Crypto deliberately excludes it and +RFC 7616 still requires it for interop. + +## Design calls worth recording + +- **Basic and Digest never stamp preemptively.** Both are phrased in §11 entirely in terms of answering a + parsed challenge, and Digest structurally cannot stamp before seeing the server's `realm`/`nonce`. `OAUTH2` + and `API_KEY` do stamp preemptively; `NO_AUTH` never stamps. Flagged as an interpretation, not a certainty + — Phase 9's conformance sweep re-checks it. +- **One auth step with one pluggable challenge hook, not three mechanisms.** `AUTH-27` mandates exactly one + step, yet `AUTH-30`, `AUTH-23`–`AUTH-26`, and `AUTH-34`–`AUTH-37` read as three. Reconciled as one step, + one `challengeHook` extension point, and a scheme-dependent default body. A caller may override the hook + entirely — for a custom OAuth2 grant, say — and it takes precedence over every scheme default. +- **The cross-origin marker suppresses the WHOLE hop, not just the outbound pass.** The redirect step + (Phase 5b) marks a cross-origin re-issue; the auth step is that marker's intended consumer. It skips the + HTTPS guard, skips stamping, clears the marker so it never reaches the wire — and declines to answer a 401 + on that hop, because answering it would stamp exactly the credential the outbound pass withheld, onto a + server-chosen foreign host. +- **A `TokenProvider` takes no arguments and must carry its own deadline.** `AUTH-34` coalesces every + concurrent caller racing on a missing or expiring token onto ONE fetch, so that fetch belongs to no single + request — handing it one caller's signal would let a stranger's cancellation reject callers who never + aborted, and let a request that merely finished tear down a refresh others were joined to. Each caller + instead races its own wait against its own signal, cancelling the wait without cancelling the work. Because + nothing could ever populate a signal parameter, the type has none: write providers as + `() => fetchToken({signal: AbortSignal.timeout(5_000)})`. +- **`ChallengeHook` receives the call's signal.** Unlike a token fetch, a hook is not shared between callers, + so the same reasoning that withholds the signal above positively requires passing it here — a hook running a + custom OAuth2 refresh grant is network I/O on the request path. The hook's third parameter is optional and + additive: an existing two-argument hook still type-checks. `authStep` also declines to spend a second wire + send on the replay once the caller has aborted, and skips the hook entirely when the call was already + abandoned before the challenge arrived — matching the redirect and retry pillars. +- **A Digest challenge this client cannot echo is declined, not answered.** A received header may legally + carry non-ASCII (`Digest realm="café"` is a real RFC 7616 shape), but an outbound header value may not, and + loosening that is the request-splitting defence. Such a challenge is now reported as unsatisfiable, so the + 401 surfaces unchanged rather than the step throwing. A non-ASCII configured Digest *username* is caller + misconfiguration and is rejected up front; RFC 7616 `username*` encoding is not yet supported. +- **Every credential type is a nominal class that redacts its secret.** `BearerToken`, `ApiKeyCredential`, and + `NameKeyCredential` each hold their secret in a `#` field, so `console.log`, `util.inspect`, + `JSON.stringify`, and `Object.keys` all see a redacted form and never the value. Build them through + `createBearerToken`/the constructors — an object literal is not assignable, which is also what stops a + `TokenProvider` handing back a token that skipped the non-blank validation. +- **One clock for the whole pipeline.** `AuthStepSettings.clock` is the `now()` half of the same `Clock` + `RetryStepOptions.clock` takes, so one instance drives both pillars and a test cannot fake time for one and + forget the other. +- **`challengeHook` is the only challenge-reaction extension point.** There is deliberately no + `handlers` field: the built-in Basic and Digest handlers are internal, so a caller-supplied list could only + replace them wholesale, never compose with them. A hook covers the custom-scheme case with a shape a caller + can actually satisfy. +- **One bearer strategy, not two.** The reference ships a synchronous single-flight policy and a separate + async three-zone policy because it has two pipeline execution stories. This port has one, so the three-zone + policy ships unconditionally and `AUTH-34`'s non-blocking cached read is its fresh-zone branch. Same shape + as the retry engine's `RETRY-28` collapse. +- **`AUTH-31`'s replayability gate applies to every replacement — and gates only the replay.** The reference + gates only its sync step and recommends a port extend it; one unified step leaves exactly one place to apply + it. A non-replayable body skips the re-drive, but the challenge is still handled, so a 401 on a streaming + upload still evicts the token the server rejected instead of leaving it cached for every later request. +- **A refresh margin is validated, and so is a token's expiry.** `bearerMarginMs`, `BearerCredential.marginMs`, + and `createBearerToken`'s `expiresAt` must all be finite. Expiry is evaluated as `now + margin > expiresAt`, + which is `false` for `NaN` — an unvalidated margin (`Number(process.env.MARGIN_MS)` on an unset variable) + made the cache read a long-dead token as fresh and serve it forever without ever calling the provider again. +- **A failed background token refresh can never fail the request that triggered it.** `AUTH-37` says so + unconditionally, so the failure is swallowed unconditionally — including a programmer-error-shaped one. The + alternative re-raised it into a promise nobody awaits, which does not surface at the fault: it terminates the + host process asynchronously, unattributable to any request, while the request that triggered it had already + been served a valid token. +- **A failing response release never masks the error it was unwinding from.** If the challenge hook throws and + closing the 401's body then also fails, the hook's error stays primary and the teardown failure rides along + as `suppressed` (`RECOV-12`), matching the redirect and retry pillars. + +`standardResilience()` leaves the `LOGGING` slot empty — Phase 7b installs `loggingStep()` there and gives +`AUTH-37`'s failed-background-refresh case somewhere to be recorded. `SERDE` stays reserved. diff --git a/.changeset/2026-08-27-serde-seam.md b/.changeset/2026-08-27-serde-seam.md new file mode 100644 index 0000000..c5b6a2d --- /dev/null +++ b/.changeset/2026-08-27-serde-seam.md @@ -0,0 +1,18 @@ +--- +'@dexpace/core': minor +--- + +Add the serde seam. `Serde`/`Serializer`/`Deserializer` are reshaped around an explicit schema +witness supplied at each decode call, closing `SEAM-21` — `Serde` is no longer generic in a payload +type, because a bundle is per wire format, not per DTO. Ships alongside it: `Tristate` and its +helpers for PATCH three-state fields, the `SerializationError`/`DeserializationError` leaves with an +`isSerdeError` guard, `serdeBody()` (the serde's own media type becomes the default `Content-Type`), +and the `decodeResponse()`/`decodeSuccessResponse()` response handlers. + +`decodeResponse()` passes through every error already in the SDK's typed tree rather than re-typing +it, so a stream failure raised by this SDK's I/O layer reaches the caller unwrapped (`SERDE-12`). A +foreign transport's stream error is indistinguishable from a non-conforming codec leaking one and is +still surfaced as `DeserializationError`; both handlers' `@throws` state that limit and name the +affected transports. A body already locked by another consumer raises a plain `TypeError`, matching +`Response.bytes()`, instead of being reported as a malformed payload. + diff --git a/.changeset/2026-08-27-sse-subsystem.md b/.changeset/2026-08-27-sse-subsystem.md new file mode 100644 index 0000000..b8ae246 --- /dev/null +++ b/.changeset/2026-08-27-sse-subsystem.md @@ -0,0 +1,73 @@ +--- +'@dexpace/core': minor +--- + +Add the Server-Sent Events subsystem for product-spec §13 (`SSE-1`–`SSE-41`). The public surface is +`sseStreamFrom()` and the `SseStream` facade it returns, `typedSseStream()` with the `MapperOutcome` +union and its `mapperValue()` / `MAPPER_SKIP` / `MAPPER_DONE` constructors, the `SseEvent` value with +`makeSseEvent()` / `sseEventsEqual()` / `sseEventToString()` / `isSseEventEmpty()`, and two error leaves, +`SseStreamError` and `SseLineTooLongError`. + +Pull-based with no read-ahead (`SSE-39`): one consumer pull drives at most one parse, and nothing is +buffered speculatively. No reconnection and no `Last-Event-ID` continuity (`SSE-38`) — both remain the +caller's responsibility, and both are now gate-enforced rather than merely documented. + +The line reader and the parser stay internal. They are driven only through the facade, and publishing them +would publish a way to violate `SSE-17`'s non-ownership contract by accident: neither closes the +`BufferedSource` it reads, because lifecycle belongs to `SseStream` alone. + +What landed under `packages/core/src/sse/`: `event.ts` (the frozen value and its operations), +`line-reader.ts` (byte-level line framing plus the opt-in cap), `parser.ts` (the field grammar and +dispatch rules), `stream.ts` (the resource-owning facade and `sseStreamFrom()`), `typed.ts` (the mapper +adapter), and `errors.ts`. Outside the package: `scripts/verify-sse-37.mjs` with its own test, a CI step +that runs it, and `test/node-conformance/sse.test.mjs`. + +Four design calls worth recording: + +- **SSE frames its own lines rather than reusing `BufferedSource.readUtf8Line()`.** Phase 3a's primitive + treats `\n` and `\r\n` as terminators but keeps a lone `\r` as line *content* (`IO-14`); `SSE-2` + requires the opposite, where a lone CR terminates a line by itself. Both contracts are normative for + their own subsystem, so reshaping the frozen Phase 3a surface for one consumer was the wrong trade. The + duplication is deliberate and recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` §I2 so Phase 10's deviation review does + not read it as accidental. The awkward case it exists to get right is a `\r` ending one chunk whose `\n` + begins the next: the pending CR is held until the following byte — or EOF — is known, so the pair + resolves to a single terminator. +- **`SSE-37`/`SSE-38` are enforced by a script, not by a type.** Nothing in the type system would catch + somebody "helpfully" adding a reconnect loop or a `Last-Event-ID` header, so `verify:sse-37` scans + `src/sse/` for serde imports and for reconnection markers. It scans **comments-stripped** source on + purpose: the requirement forbids the code path, not the documentation of its absence, and "this + subsystem never reconnects; that is the caller's job" is the single most likely sentence to appear in a + TSDoc there. A gate that fails on its own requirement's explanation is a gate the next person deletes + instead of the comment. +- **`[Symbol.asyncDispose]` is installed at run time, not declared on the class.** The declared + `engines.node` floor is `>=20.3` and the symbol landed in Node 20.4, where a computed key that evaluates + to `undefined` binds the method to the string `"undefined"` instead — wrong, silent, and only at run + time. Declaring the member would also break consumers compiling the published `.d.ts` on a plain + `ES2023` lib. `SseStream` therefore installs it behind a `typeof Symbol.asyncDispose === 'symbol'` + guard, matching `Response` (`HTTP-38`). Recorded at §I3; it becomes an unconditional `implements + AsyncDisposable` when the floor moves past 20.4. Note that Phase 6c's `Page` resolves the same question + the other way — see that changeset. +- **`MapperOutcome` is a sibling of Phase 4b's `Outcome`, not a third variant on it.** `Outcome` + is a two-branch success/failure union threaded through the recovery chain; widening it with `skip` and + `done` would force every `fold` call site in `src/recovery/` to handle variants that can never occur + there. What `sdk-design-nodejs/07` §7.2 asks to reuse is the *idiom* — a `kind`-discriminated union over + frozen literals. + +Limits worth knowing at the call site: + +- **The line cap is opt-in and off by default** (`SSE-19`), matching the reference's own absence of a cap. + Set `maxLineBytes` to bound memory against a server that never sends a terminator; exceeding it raises + `SseLineTooLongError`, which carries `limitBytes` as a field so a log aggregator indexes it without + parsing the message. +- **`signal` adds a trigger, not a code path.** Aborting closes the stream, which is all the cancellation + a pull-based reader needs: an iterator sitting *between* pulls ends cleanly (`SSE-27`), and one blocked + *in* a read surfaces an `IoError` (`SSE-31`). Both paths release the owned resource exactly once. +- **A release failure on a clean terminal path is swallowed and reported out-of-band** (`SSE-30`), because + throwing would discard events already delivered. `onReleaseFailure` receives it and defaults to a no-op; + Phase 7 wires a real `Logger` there without reshaping the class. An explicit `close()` still propagates. +- **A bodyless response is rejected rather than yielding an empty stream** (`SSE-32`). It is a server or + caller mistake, and silently producing zero events would hide it behind a successful-looking loop that + does nothing. +- **`SSE-41`'s reactive `Observable` view is not here.** It is a MAY, and the roadmap scopes §18's + async-runtime adapters to Phase 8b (`@dexpace/rx`). Deferral recorded at §I1. `SSE-21`'s hash-equality + clause has no JavaScript analogue; value equality ships as `sseEventsEqual()` (§I4). diff --git a/.changeset/2026-08-28-async-runtime-bridge.md b/.changeset/2026-08-28-async-runtime-bridge.md new file mode 100644 index 0000000..940fa46 --- /dev/null +++ b/.changeset/2026-08-28-async-runtime-bridge.md @@ -0,0 +1,17 @@ +--- +"@dexpace/rx": minor +--- + +Add `@dexpace/rx`, the RxJS async-runtime bridge (Phase 8b, `SSE-41` / the non-collapsed `ASYNC-*` subset): + +- `sseEvents$(stream)` and `typedSse$(stream, mapper)` — single-subscription `Observable` views of Phase 6b's + `SseStream` and `typedSseStream`. A second `subscribe()` surfaces `SseStream`'s own `SSE-26` guard through the + error channel rather than inventing a new restriction. +- `pageItems$(paginator)` and `pages$(paginator)` — cold, repeatable `Observable` views of Phase 6c's + `Paginator`, one independent fetch sequence per subscription (`PAGE-8`). +- Unsubscribing reaches the source even while a pull is suspended (`ASYNC-6`), so an idle SSE stream releases its + response body immediately instead of at the server's next event. This is the one clause RxJS's own + `from(asyncIterable)` does not satisfy, so the package ships a small internal bridge in its place; the + conformance suite pins both behaviors. +- Source errors reach the error channel unwrapped (`ASYNC-13`); no new error class. +- `rxjs` and `@dexpace/core` are peer dependencies, and the package has zero runtime dependencies (`SEAM-1`). diff --git a/.changeset/2026-08-28-instrumentation-and-observability.md b/.changeset/2026-08-28-instrumentation-and-observability.md new file mode 100644 index 0000000..85a62c8 --- /dev/null +++ b/.changeset/2026-08-28-instrumentation-and-observability.md @@ -0,0 +1,14 @@ +--- +"@dexpace/core": minor +"@dexpace/logging-pino": minor +"@dexpace/logging-debug": minor +--- + +Add the instrumentation and observability subsystem (Phase 7b): +- The `Logger` / `LogEvent` structured logging facade and `createLogger` builder, with zero-allocation `NOOP_LOGGER`, four severity levels, 4-tier precedence folding, safe total field rendering with 8 KiB truncation, at-most-once single emission, and global logger slot (`getGlobalLogger`/`setGlobalLogger`). +- AsyncLocalStorage-backed diagnostic context (MDC) allow-list filtering (`trace.id`, `span.id`). +- Redaction policy for URLs and headers with default-deny allow-listing. +- OpenTelemetry-compatible tracing SPI (`Tracer`, `Span`, `SpanContext`, `Scope`, `activateSpan`, `activateSpanForCorrelation`) and W3C Trace Context generation (`createInstrumentationBundle`). +- Metrics SPI (`Counter`, `Histogram`, `Meter`, `NOOP_METER`). +- The `LOGGING` pillar step (`loggingStep`, `LOGGING_STEP_TYPE`) with configurable granularity (`none`, `headers`, `body`), bounded body previews, asymmetric `OBS-20` failure containment, and installation into `standardResilience()`. +- Adapter packages: `@dexpace/logging-pino` and `@dexpace/logging-debug`. diff --git a/.changeset/2026-08-28-transport-adapters.md b/.changeset/2026-08-28-transport-adapters.md new file mode 100644 index 0000000..6ba8114 --- /dev/null +++ b/.changeset/2026-08-28-transport-adapters.md @@ -0,0 +1,15 @@ +--- +"@dexpace/core": minor +"@dexpace/transport-fetch": minor +"@dexpace/transport-undici": minor +"@dexpace/transport-shared": minor +"@dexpace/body-file": minor +--- + +Add the transport adapters (Phase 8a) — the first code in this SDK that puts bytes on the wire: +- `@dexpace/transport-fetch`: a `Transport` over the runtime's global `fetch`, with zero dependencies beyond its `@dexpace/core` peer. No `proxy` option exists at all (an absent option, not a silently ignored one), and `close()` is a sanctioned no-op over a runtime global it does not own. +- `@dexpace/transport-undici`: the full-featured `Transport`, taking exactly one external dependency. Ownership-aware `close()` over the dispatchers it constructed (never a bring-your-own one), `NO_PROXY` bypass routed over a separate direct `Agent`, direct file-body dispatch honoring `start`/`count`, and a native-internal cancel told apart from a timeout. +- `@dexpace/body-file`: the concrete `fileBody()` factory, with fail-fast `node:fs` construction validation, a fresh handle per write, and short-write detection. Transports recognize it structurally through `body.kind === 'file'`, never a cross-package `instanceof`. +- `@dexpace/transport-shared`: the header drop/degrade pass, drop-log dedup policy, abort-to-SDK-error mapping, request-body pump, and delivery-detached signal fork — `@internal` exports both transports share so the one algorithm exists once rather than twice. +- `@dexpace/core` gains `TransportFailureError` (the canonical retryable no-response failure, an `IoError` subtype) and the type-only `FileBodyDescriptor` plus a `'file'` member on `Body['kind']`. `IoError` is promoted from `@internal` to `@public` as its base class. Note for TypeScript consumers: widening `Body['kind']` is additive for anyone *implementing* `Body`, but an exhaustive `switch (body.kind)` with a `never` default will stop compiling until it handles `'file'`. +- Both transports are proven against one shared `TRANSPORT-N` conformance suite and are `AsyncDisposable`, so `await using` is a single teardown path. Both also keep a handler on a streaming request body's producer for the whole send: a producer that fails *after* the response was delivered (an early `413`, say) is an observed rejection rather than one that reaches the runtime's default `unhandledRejection` policy. `@dexpace/transport-undici` additionally reports undici's argument-validation failures outside the `IoError` tree, so a permanent misconfiguration is terminal rather than retried to exhaustion. diff --git a/.changeset/2026-08-29-guard-symbol-asyncdispose-installs.md b/.changeset/2026-08-29-guard-symbol-asyncdispose-installs.md new file mode 100644 index 0000000..579110a --- /dev/null +++ b/.changeset/2026-08-29-guard-symbol-asyncdispose-installs.md @@ -0,0 +1,25 @@ +--- +"@dexpace/core": minor +"@dexpace/transport-fetch": minor +"@dexpace/transport-undici": minor +--- + +Guard every `[Symbol.asyncDispose]` install behind a runtime check, so disposal is never promised on a Node version that does not have the symbol. + +`Page`, `FetchTransport`, and `UndiciTransport` each declared `[Symbol.asyncDispose]` as a plain computed class member. `Symbol.asyncDispose` arrived in Node **20.4**, but every package here declares `engines.node: ">=20.3"`. On the declared floor the computed key evaluates to `undefined`, so the method was bound to the string key `"undefined"` — leaving a junk prototype entry and **no working disposal**, while the emitted `.d.ts` promised `AsyncDisposable` unconditionally. `SseStream` was already guarded, and `Response` carries a regression test asserting the absence of exactly this junk key (`http/response.test.ts`); these three sites had reintroduced it. + +The installs now match `SseStream`: `Object.defineProperty` behind `typeof Symbol.asyncDispose === 'symbol'`. Disposal works unchanged on Node 20.4+. + +Breaking, in the type system only: + +- `Page` no longer declares `implements AsyncDisposable`, and its `.d.ts` no longer declares `[Symbol.asyncDispose]`. +- `fetchTransport()` returns `Transport` rather than `Transport & AsyncDisposable`. +- `undiciTransport()` returns `Transport` rather than `Transport & AsyncDisposable`. + +`await using page = ...` / `await using transport = ...` therefore no longer type-checks. This is deliberate: the declaration was only ever true on Node 20.4+, and on the floor it type-checked a call that silently did nothing — for `undiciTransport` that meant leaking every pooled connection. Call `close()` instead, which has always been the real teardown path and is unchanged. Consumers pinned to Node 20.4+ who want `await using` back can reach the installed symbol through a cast. + +The floor will not be raised to `>=20.4` to restore the declaration. `NFR-10` requires a capability that needs a newer runtime to be isolated into its own unit declaring that higher floor, never to raise the floor of the general-purpose core; it also requires the emitted-artifact target and the visible-API level to agree, which is the clause the unguarded member violated. `>=20.3` is in any case derived rather than chosen — it is the lowest Node that runs what these packages emit, set by `globalThis.crypto` (absent from ESM on every Node 18 release) and `AbortSignal.any()` (20.3.0). The guarded install is the permanent shape. + +`Paginator.pages()`'s published TSDoc is corrected to match: it had discharged `PAGE-12`'s "consumers MUST be told to wrap the view in a scoped/auto-close construct" clause by naming `await using` alongside `for await`, which no longer type-checks. It now names the two constructs that do give the guarantee — a `for await` loop, or `.return()` from a `finally` when you drive the iterator by hand — and says why `await using` is not a third. + +Kept as **minor** rather than major because these packages are pre-1.0 (`0.0.0`), per the same semver initial-development carve-out the earlier `Body` narrowing used. diff --git a/.changeset/2026-08-30-undici-teardown-and-producer-ordering.md b/.changeset/2026-08-30-undici-teardown-and-producer-ordering.md new file mode 100644 index 0000000..b8e9fee --- /dev/null +++ b/.changeset/2026-08-30-undici-teardown-and-producer-ordering.md @@ -0,0 +1,20 @@ +--- +"@dexpace/transport-undici": patch +--- + +Fix two teardown defects in `undiciTransport()`. + +`close()` no longer strands owned dispatchers when one fails to release. It previously walked the +owned set with a bare `for … await` loop, so the first rejecting `destroy()` aborted the walk — and +because the set is walked in reverse, a configured proxy meant the `ProxyAgent` actually holding the +pooled connections was the one left leaked. Every owned dispatcher is now destroyed before any +failure is reported, and the failure surfaces as a `TransportFailureError` carrying the underlying +cause (an `AggregateError` when more than one dispatcher failed) rather than a raw `undici` error +escaping a public method untyped. + +`send()` now maps request headers before preparing the request body. `prepareBody()` starts a +streaming producer eagerly while header mapping reads `request.body.mediaType` — a getter on a +caller-supplied `Body` that may throw. In the old order such a throw left a live producer that +nothing could abandon, whose own later rejection reached Node's default `unhandledRejection` policy +(TRANSPORT-19, SEAM-30). `@dexpace/transport-fetch` already evaluated the two in this order and is +unaffected; both transports now carry a regression test pinning it. diff --git a/.changeset/2026-09-02-clock-sleep-honours-any-finite-duration.md b/.changeset/2026-09-02-clock-sleep-honours-any-finite-duration.md new file mode 100644 index 0000000..1169025 --- /dev/null +++ b/.changeset/2026-09-02-clock-sleep-honours-any-finite-duration.md @@ -0,0 +1,32 @@ +--- +"@dexpace/core": patch +--- + +`Clock.sleep` now honors any finite, non-negative duration by chaining timers, instead of rejecting +a duration longer than one `setTimeout` delay can carry. + +`setTimeout` clamps a delay above 2^31 − 1 ms and *silently* rewrites it to `1`, so an oversized +sleep used to return in about a millisecond — an overflowed retry backoff became no backoff at all. +Phase 7a repaired that by **rejecting** any such duration with an `InvariantViolation`. That fixed +the silent clamp but created a second problem: `RETRY-18`/`RECOV-26` require a server pacing hint to +be clamped to a 365-day ceiling, roughly fourteen times what one timer can carry, so a conformant +retry could produce a delay the clock refused. `Clock.sleep` sliced into `MAX_SLEEP_MS` chunks keeps +the original intent — never a silent clamp — and honors `RETRY-18` exactly. + +**Consumer-visible changes:** + +- A duration above 2^31 − 1 ms now waits, where it previously rejected. Nothing that worked before + stops working. +- A negative or non-finite duration now rejects with `RangeError` rather than the internal + `InvariantViolation`, which was never exported and so could not be caught by class. +- A cancelled sleep now rejects with `CancellationError` carrying the caller's abort reason as + `cause`, rather than the raw reason — the same mapping the transports and the retry engine already + apply, so one cancellation type surfaces wherever the abort was observed. A timeout-aborted signal + yields `TransportFailureError`, keeping `XCUT-3`'s distinction. `CFG-17`'s "re-assert the + cancellation status" clause is unaffected: `AbortSignal.aborted` is latched, so a downstream + handler observes the cancelled state whatever object is thrown. + +**If you implement `Clock` yourself**, honor long durations too — passing `durationMs` straight to +`setTimeout` reintroduces the silent clamp. The interface's `@remarks` now says so. + +Closes `docs/work/mvp/2026-09-04-open-items-dissolution.md` V13. diff --git a/.changeset/2026-09-02-config-and-auth-diagnostics.md b/.changeset/2026-09-02-config-and-auth-diagnostics.md new file mode 100644 index 0000000..edf79ae --- /dev/null +++ b/.changeset/2026-09-02-config-and-auth-diagnostics.md @@ -0,0 +1,23 @@ +--- +"@dexpace/core": patch +--- + +Three configuration and auth failures that resolved silently now emit a structured warning through +`getGlobalLogger()`. All three were deferred to "once a `Logger` seam exists"; Phase 7b shipped one, +and these are the call sites that never got wired to it. + +- **`AUTH-37`** — a failed background bearer-token refresh emits `http.auth.bearerRefreshFailed` with + the provider's error as the cause, then continues exactly as before. The requirement is + log-and-continue; only the continue half was implemented. +- **`CFG-24`** — a proxy URL rejected by `resolveProxyOptions` emits `http.proxy.configRejected` + naming the variable it came from (`HTTPS_PROXY`/`HTTP_PROXY`) and which gate rejected it + (`unparseable`, `scheme`, `port`, `host`). A typo'd proxy variable previously routed every request + direct with nothing to read anywhere. The URL itself is never logged — it can carry `user:pass@`, + and CFG-22 masks credentials in every rendering. +- **`CFG-5`/`CFG-11`** — a caller-supplied configuration source that throws emits + `config.sourceFailed` naming the layer and the key. The lookup still falls through to the caller's + default, because CFG-5's never-throw clause is the stronger obligation; what changes is that the + operator can now see why. + +Resolution behaviour is unchanged in all three cases. Every emission is wrapped so a failing logger +cannot fail the operation (OBS-20). diff --git a/.changeset/2026-09-02-context-keys-render-distinctly.md b/.changeset/2026-09-02-context-keys-render-distinctly.md new file mode 100644 index 0000000..9c04680 --- /dev/null +++ b/.changeset/2026-09-02-context-keys-render-distinctly.md @@ -0,0 +1,14 @@ +--- +"@dexpace/core": patch +--- + +Default-constructed execution-context keys now carry a serial number in their description — +`Symbol('dispatch-context#7')` rather than `Symbol('dispatch-context')`. + +`CTX-8` is stated in appendix C as an error "whose **message** identifies the key", and +`DuplicateContextKeyError`'s message renders `String(key)`. Every default key of a flavor rendered +identically, so the message named the *kind* of key and never *which* key — the error's typed +`.key` field carried the identity, but the message did not. The identity is still the `Symbol()` +itself; only the label changed, so `CTX-4`/`CTX-5`/`CTX-6`'s uniqueness is untouched. + +Recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` A5. diff --git a/.changeset/2026-09-02-core-side-cancellation-mapping.md b/.changeset/2026-09-02-core-side-cancellation-mapping.md new file mode 100644 index 0000000..16832f2 --- /dev/null +++ b/.changeset/2026-09-02-core-side-cancellation-mapping.md @@ -0,0 +1,19 @@ +--- +"@dexpace/core": patch +--- + +A cancellation observed inside core now surfaces as `CancellationError`, the same type the transport +layer already produced for the identical abort (XCUT-1, `docs/work/mvp/2026-09-04-open-items-dissolution.md` N1). + +The retry engine's `RETRY-32` exit handed back `config.signal.reason` verbatim, and the bearer +cache's `raceAbort` rejected with it, so a caller writing +`catch (e) { if (e instanceof CancellationError) … }` handled a cancelled transport dispatch and +silently missed a cancelled backoff or a cancelled token fetch — those arrived as a bare +`DOMException` named `AbortError`. Both now map through the same shape, keeping the caller's own +abort reason as the error's `cause`. + +`XCUT-3` is why the mapping is not unconditional: a signal aborted by `AbortSignal.timeout()` +surfaces `TransportFailureError`, so a timeout stays distinguishable from a cancellation. + +`Clock.sleep` is deliberately unchanged — `CFG-17` requires it to reject with the caller's reason +exactly as given, and the retry loop absorbs that rejection rather than surfacing it. diff --git a/.changeset/2026-09-02-cursor-honours-an-aborted-signal.md b/.changeset/2026-09-02-cursor-honours-an-aborted-signal.md new file mode 100644 index 0000000..ec4445d --- /dev/null +++ b/.changeset/2026-09-02-cursor-honours-an-aborted-signal.md @@ -0,0 +1,23 @@ +--- +"@dexpace/core": patch +--- + +The pipeline cursor now checks the caller's `AbortSignal` at every step boundary, so a cancelled call +stops walking instead of running every installed step and only failing at the transport hop. + +`Cursor` accepted the signal, threaded it to the terminal transport, and never looked at it in +between — against `concurrency-and-async.md`'s "check the signal at the top of each loop iteration or +before each expensive step". Each pillar guarded its *own* loop (`RETRY-32`, redirect's per-hop +check), but the walk itself was unguarded, so an already-aborted call could still do real work on the +way down — the auth step's bearer-token refresh being the concrete case. + +**Consumer-observable:** a call whose signal is already aborted now rejects before any step runs, so +no wire send happens and no response is produced. It previously dispatched and, on the redirect path, +handed the first hop back open. An abort raised *during* a hop is unchanged: the redirect step's own +guard runs before it forks again, so the in-flight response is still returned unclosed (`PIPE-40`). + +The abort is mapped through the same helper `docs/work/mvp/2026-09-04-open-items-dissolution.md` N1 added, so it surfaces as +`CancellationError` with the caller's own reason as `cause` — never a bare `DOMException` — and a +timeout-aborted signal still surfaces `TransportFailureError`, keeping `XCUT-3`'s distinction. + +Closes `docs/work/mvp/2026-09-04-open-items-dissolution.md` V15 and Section T's `F9`. diff --git a/.changeset/2026-09-02-delete-the-dead-seams-barrel.md b/.changeset/2026-09-02-delete-the-dead-seams-barrel.md new file mode 100644 index 0000000..1f3c495 --- /dev/null +++ b/.changeset/2026-09-02-delete-the-dead-seams-barrel.md @@ -0,0 +1,11 @@ +--- +"@dexpace/core": patch +--- + +Delete `packages/core/src/seams/index.ts`, an internal folder-level barrel from Phase 2 that nothing +imported. Its only reference anywhere in the workspace was the comment in `packages/core/src/index.ts` +explaining why the public barrel deliberately did not re-export it. + +No published surface changes: `packages/core/package.json`'s `exports` names `.` only, so the file +was never reachable by a consumer, and every symbol it re-exported is already named directly on the +public barrel. Closes `docs/work/mvp/2026-09-04-open-items-dissolution.md` H12. diff --git a/.changeset/2026-09-02-dropped-header-log-levels.md b/.changeset/2026-09-02-dropped-header-log-levels.md new file mode 100644 index 0000000..d4de3ee --- /dev/null +++ b/.changeset/2026-09-02-dropped-header-log-levels.md @@ -0,0 +1,17 @@ +--- +"@dexpace/transport-shared": patch +--- + +Give `createDropLogger`'s verbosity policy real levels (OBS-19, TRANSPORT-13). Every mode used to +emit at `verbose`, so the policy was configurable in name only. + +- `'all'` now warns on every occurrence. +- `'first-per-name'` — the default for both `fetchTransport()` and `undiciTransport()` — now warns + the **first** drop of each header name and emits later drops of that name at `verbose`. It + previously suppressed later drops entirely; OBS-19's conformance text asks for "exactly one WARN + then verbose lines", so they are emitted rather than dropped. +- `'quiet'` is unchanged and still writes nothing, which is TRANSPORT-13's own third mode. + +The visible effect is that a caller-set header the transport cannot encode — dropped rather than +thrown, per TRANSPORT-12 — is now audible at a level a production logger enables. Before this, the +drop was indistinguishable from nothing having happened. diff --git a/.changeset/2026-09-02-http-status-error-survives-a-failing-close.md b/.changeset/2026-09-02-http-status-error-survives-a-failing-close.md new file mode 100644 index 0000000..877b9e6 --- /dev/null +++ b/.changeset/2026-09-02-http-status-error-survives-a-failing-close.md @@ -0,0 +1,20 @@ +--- +"@dexpace/core": patch +--- + +`toHttpError` no longer lets a failing `close()` replace the `HttpStatusError` it was about to +build. The drain ended in a bare `finally { await response.close() }`; `Response.close()` memoizes +its release promise, so a response whose close had already failed handed the same rejection back +from inside that `finally`, and it replaced the result. A 5xx then surfaced as the raw close error +with `error instanceof HttpStatusError` false — making the `@throws HttpStatusError on 4xx/5xx` tag +on `decodeSuccessResponse`, `statusMappingStep` and the retry engine untrue. + +Release now goes through `releaseQuietly`/`withReleaseFailure`, the same pair every other subsystem +uses (RECOV-12): + +- a **read** failure stays primary, with the release failure suppressed under it; +- a **successful** read returns the `HttpStatusError` even when the release failed, carrying that + failure as the error's `cause` rather than dropping it. + +Fixing it at `toHttpError` covers all four callers at once. Recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` H14 (of +which `P1` is the same defect under a second letter). diff --git a/.changeset/2026-09-02-numeric-range-checks-are-the-full-range.md b/.changeset/2026-09-02-numeric-range-checks-are-the-full-range.md new file mode 100644 index 0000000..6cf31e2 --- /dev/null +++ b/.changeset/2026-09-02-numeric-range-checks-are-the-full-range.md @@ -0,0 +1,24 @@ +--- +"@dexpace/core": patch +--- + +Close the public numeric range checks that guarded only their lower bound, completing the sweep +`docs/work/mvp/2026-09-04-open-items-dissolution.md` P2 asked for. + +- `RequestOptionsBuilder.timeoutMs` now rejects `Infinity` and `NaN`. It previously rejected only + `<= 0`, so a non-finite deadline degraded silently to "no deadline" instead of failing at the call + site that supplied it (HTTP-35). Fractional milliseconds are still accepted — a timeout is a + duration, not a count. +- `retrySettings`'s `multiplier` now rejects a non-finite value. `Infinity >= 1` passed, and made + the second backoff delay `Infinity`. +- `retrySettings`'s `maxAttempts` now requires an integer. `2.5` passed a `Number.isFinite` check + and is not a count of wire sends. +Retry durations are deliberately **not** given an upper bound. An earlier revision of this change +bounded `initialDelayMs`/`maxDelayMs`/`fixedDelayMs` at `Clock`'s `MAX_SLEEP_MS`, because +`Clock.sleep` rejected anything longer; `Clock.sleep` now chains timers to honor any finite duration +(see the separate clock changeset), so such a bound would reject a wait the platform can perform — +and would make `RETRY-18`'s 365-day pacing ceiling unconfigurable. + +The sweep's full result, including the surfaces found already whole (`HttpRange`, +`redirectSettings.maxHops`, the auth margins, `Paginator.maxPages`, `ContextStore`'s cap), is +recorded in P2's note. diff --git a/.changeset/2026-09-02-publish-the-catchable-errors-and-the-redirect-guard.md b/.changeset/2026-09-02-publish-the-catchable-errors-and-the-redirect-guard.md new file mode 100644 index 0000000..c413190 --- /dev/null +++ b/.changeset/2026-09-02-publish-the-catchable-errors-and-the-redirect-guard.md @@ -0,0 +1,40 @@ +--- +"@dexpace/core": minor +--- + +Publish ten symbols that the emitted `.d.ts` already told consumers about but no package exported. + +**The redirect guard** (`docs/work/mvp/2026-09-04-open-items-dissolution.md` U7). `withRedirect(builder, overrides?)` and +`stripCrossOriginMarkerStep()` are now public. `redirectStep()` marks a cross-origin hop with an +internal header and relies on a second `POST_AUTH` step to strip it before dispatch (`REDIR-11(c)`); +that step was `@internal`, so `withRedirect`'s own instruction — "a caller who installs +`redirectStep()` directly is responsible for installing the guard too" — named an obligation no +consumer could discharge. `standardResilience()` and `PipelineBuilder.seedFrom()` were the only safe +routes to a redirect pipeline; `withRedirect(builder)` is now the direct one. + +**Eight catchable error classes** (`docs/work/mvp/2026-09-04-open-items-dissolution.md` U9): `PillarCollisionError`, +`ReservedStageError`, `AnchorNotFoundError`, `CrossStageEditError`, `CursorAlreadyAdvancedError`, +`EndOfStreamError`, `SchemeDowngradeError` and `NonReplayableBodyError`. Each is the subject of a +`@throws` tag on a public symbol, and each shipped into the `.d.ts` — so a consumer read the tag, +reached for `instanceof`, and had nothing to reach for. `error.name` was the only handle. + +`InvariantViolation` stays unexported and `@internal`: it signals a bug rather than a condition, and +extends `Error` rather than `DexpaceError`. Its `@throws` tags on public symbols now read as prose — +"an assertion failure (a caller bug, not a catchable condition)" — instead of naming a class nobody +can catch. `DuplicateContextKeyError` likewise stays behind the `@internal` `ContextStore`. + +**Two new error classes, both from `XCUT-8`** (`docs/work/mvp/2026-09-04-open-items-dissolution.md` N2/V14): + +- `HttpStatusValidationError` — `HttpStatusError`'s constructor now validates that `status` is an + integer in HTTP-11's 400–599 band and throws this otherwise. The class documented that invariant + and never enforced it, so `new HttpStatusError(200, …)` built the "successful exception" `XCUT-8` + forbids. **This is the one behavioural break in this changeset**: a caller constructing an + `HttpStatusError` out of band now gets a throw. `toHttpError` is unaffected — it is the total form + and still returns `null` for any status outside the band. +- `RetryDiscardedResponseError` — the retry engine's trail entry for a response it discarded whose + status is outside 400–599, reachable only by widening `RetrySettings.retryableStatuses` to include + a non-error code. The engine used to fabricate `new HttpStatusError(, …)` there, so + core itself built the object the requirement forbids and the trail claimed an HTTP failure that had + not occurred. A discarded 4xx/5xx still yields `HttpStatusError` exactly as before. + +Otherwise additive: nothing else was removed or narrowed. diff --git a/.changeset/2026-09-02-redirect-loop-and-malformed-location-events.md b/.changeset/2026-09-02-redirect-loop-and-malformed-location-events.md new file mode 100644 index 0000000..0c1c270 --- /dev/null +++ b/.changeset/2026-09-02-redirect-loop-and-malformed-location-events.md @@ -0,0 +1,20 @@ +--- +"@dexpace/core": patch +--- + +`redirectStep()` now emits the last two of `REDIR-28`'s four structured events: +`http.redirect.loopDetected` and `http.redirect.malformedLocation`. Phase 7b shipped the hop, +rejection and permitted-downgrade events; these two were blocked because `decide()`'s +`'return-current'` outcome was a bare `{kind}` that could not tell loop detection from a hop cap +from ordinary termination. + +`Decision`'s `'return-current'` variant now carries a `reason` — `'not-a-redirect'`, +`'not-eligible'`, `'malformed-location'`, `'loop-detected'` or `'hop-cap'`. `decide()` and +`Decision` are `@internal` and appear in no API report, so no published surface changes. + +The malformed-Location event logs the header **raw**, unredacted. That is `REDIR-28`'s own carve-out: +the value failed to parse into a URL, so there is nothing for the redactor to key off. A deployment +whose upstreams may send credential-bearing malformed `Location` values should account for it. + +Closes `docs/work/mvp/2026-09-04-open-items-dissolution.md` G3, and the "Redirect's loop-detected and malformed-Location events" +row in Section D. diff --git a/.changeset/2026-09-02-unknown-charset-resolves-to-undefined.md b/.changeset/2026-09-02-unknown-charset-resolves-to-undefined.md new file mode 100644 index 0000000..eb20736 --- /dev/null +++ b/.changeset/2026-09-02-unknown-charset-resolves-to-undefined.md @@ -0,0 +1,20 @@ +--- +"@dexpace/core": patch +--- + +`MediaType.charset` now returns `undefined` for an encoding label the runtime does not recognize, +closing the "or unknown" half of `HTTP-24` — whose own conformance text reads +`charset=bogus` → null. It previously returned the label verbatim, so `text/plain;charset=bogus` +answered `'bogus'` and a caller had no way to reach the requirement's fallback without exception +handling of its own. + +"Unknown" is resolved against the runtime's WHATWG Encoding registry: a label +`new TextDecoder(label)` refuses is one nothing in this SDK could decode with, and it is the same +resolution `decodeBodyText` already performs a layer down, so the two cannot disagree about what is +decodable. Recognized labels keep their original case (`HTTP-23`). + +The raw parameter is unchanged and still reachable: `parameter('charset')` returns `'bogus'`, and +`render()` still round-trips it verbatim (`HTTP-25`). Behaviour downstream is unchanged too — +`resolveCharset` already fell back to UTF-8 for a label `TextDecoder` rejected. + +Recorded at `docs/work/mvp/2026-09-04-open-items-dissolution.md` A1. diff --git a/.changeset/2026-09-04-flatten-the-domain-model-error-tier.md b/.changeset/2026-09-04-flatten-the-domain-model-error-tier.md new file mode 100644 index 0000000..4e4eb48 --- /dev/null +++ b/.changeset/2026-09-04-flatten-the-domain-model-error-tier.md @@ -0,0 +1,13 @@ +--- +"@dexpace/core": minor +--- + +Remove `DomainModelError` as a class tier. The ten HTTP domain-model error leaves — `RequiredFieldError`, `HeaderValidationError`, `MediaTypeParseError`, `ProtocolParseError`, `UrlConstructionError`, `RequestOptionsValidationError`, `EtagParseError`, `HttpRangeValidationError`, `RequestConditionsValidationError` and `RequestBodyNotAllowedError` — now extend `DexpaceError` directly, and a new `@public` `isDomainModelError` type guard groups them. + +**This is a breaking change to published API.** `DomainModelError` was a barrel export and a runtime value, so `instanceof` narrowing on it was live public API, and it is gone. The migration is one line: `if (error instanceof DomainModelError)` becomes `if (isDomainModelError(error))`, which narrows to the same ten-class union, so nothing downstream of the check changes. + +The class earned its removal by doing nothing: it was an empty marker (`export class DomainModelError extends DexpaceError {}`), nothing in the SDK ever narrowed on it, and the corpus caps custom error hierarchies at two levels. Core had already stopped feeding it — `HttpStatusValidationError` landed as a two-level leaf under `DexpaceError` rather than as an eleventh leaf on the tier. + +**The taxonomy stays mixed, and that is deliberate.** `TransportFailureError extends IoError` is still three levels. `TRANSPORT-20` is a MUST requiring the canonical transport failure to be a subtype of the platform IO exception so existing `catch (IOException)` sites keep matching, and `retry/classify.ts` walks the cause chain with `current instanceof IoError` to make such a failure unconditionally retryable; removing that tier would break a MUST and a live retry path (`docs/deviations.md` item 17). This change removes the one gratuitous three-level tier and leaves the one the specification requires — it does not make the tree uniformly two-level. + +Kept as **minor** rather than major because `@dexpace/core` is still pre-1.0 (`0.0.0`), where a 0.x breaking change is conventionally released as minor (semver's own carve-out for initial development, https://semver.org/#spec-item-4). Revisit at 1.0. diff --git a/.changeset/2026-09-04-operation-auth-tier-slot.md b/.changeset/2026-09-04-operation-auth-tier-slot.md new file mode 100644 index 0000000..a9311c6 --- /dev/null +++ b/.changeset/2026-09-04-operation-auth-tier-slot.md @@ -0,0 +1,36 @@ +--- +"@dexpace/core": minor +--- + +Give `AuthTiers.operation` a source: `RequestOptions` gains `operationAuth`, a second per-call slot +that `effectiveTiers()` folds into AUTH-4's middle tier (AUTH-4, AUTH-5, AUTH-6, AUTH-7). Additive — +no signature changed and no behavior changed for a caller who does not set it. + +`AuthTiers` has always resolved `perCall ?? operation ?? client`, and nothing in the workspace could +write the middle slot. The cost was measured rather than assumed: `examples/petstore/FINDINGS.md` §4 +found that a consumer with per-operation descriptors had to fold them itself — +`const auth = call.auth ?? operation?.auth` — which reimplements the top two-thirds of AUTH-4's +precedence chain in consumer code, and leaves core unable to tell a caller's genuine per-call +override from an operation's declared requirement once they arrive in the same slot. Every generated +SDK would have carried that fold. + +```ts +const options = RequestOptions.newBuilder() + .auth(callerOverride) // may be undefined + .operationAuth(operation.auth) // the operation table's static declaration + .build(); +``` + +`effectiveTiers()` applies each slot only when present, so a configured tier is never overwritten +with `undefined` — spreading an absent `perCall` would have erased one the AUTH step was constructed +with. + +The other option the spike named — carrying the operation descriptor as a separate `StepContext` +field — was not taken. `StepContext.options` already travels from `Runtime.send` through every retry +attempt and redirect hop, and is where `authStep` reads the per-call descriptor today; a parallel +carrier for the same lifetime would have widened the pipeline's plumbing for one consumer. + +Verified by deleting the fold it exists to remove: `examples/petstore/src/service-core.ts` now fills +both slots and lets core resolve the chain, and the spike's canary passes unchanged. + +See `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1. diff --git a/.changeset/2026-09-04-per-operation-auth-tier-disposition.md b/.changeset/2026-09-04-per-operation-auth-tier-disposition.md new file mode 100644 index 0000000..0c6a938 --- /dev/null +++ b/.changeset/2026-09-04-per-operation-auth-tier-disposition.md @@ -0,0 +1,30 @@ +--- +"@dexpace/core": patch +--- + +Document `AuthTiers.operation`'s settled disposition on the field itself (AUTH-4, AUTH-5, AUTH-6, +AUTH-7). Documentation only: no behavior, no types, and no exported symbol changed, so +`packages/core/etc/core.api.md` is byte-identical — the field is still +`readonly operation?: AuthDescriptor | undefined;`. + +The tier was previously described as having "no shipped source yet", with a pointer to the +roadmap's Deferred Items Log. Both halves were wrong. "Yet" read as pending work, and the pointer +dangled: that log moved out of the roadmap into `docs/deferred-items.md` on 2026-08-31 and the +roadmap's own section is a stub. + +Nothing is pending. `resolveAuthRequirement` selects `perCall ?? operation ?? client`, so the +`operation` tier resolves correctly the moment a caller populates the `AuthTiers` it passes — the +tier is live, not dead code. What does not exist is an automatic source: filling it would take a +per-operation configuration layer, a code generator or a client surface, and no phase on this +roadmap ships one, so `client` and `operation` alike stay construction-time configuration. +`AUTH-4` through `AUTH-7` are mechanically satisfied either way, which is why this is a missing +source rather than an unmet requirement. + +The per-call half of the same question shipped in Phase 5c and is unaffected: `RequestOptions.auth` +reaches the AUTH step through `StepContext.options` and is merged into the tier set at resolution +time. + +The reasoning now lives in the code a consumer reads in the published `.d.ts` and needs no pointer, +so the row for it was closed. The register that held it, `docs/deferred-items.md`, was dissolved the +same day; the live gap this leaves — a published tier core gives consumers no way to fill — is +tracked as `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1. diff --git a/.changeset/2026-09-04-per-operation-span.md b/.changeset/2026-09-04-per-operation-span.md new file mode 100644 index 0000000..3eea5d0 --- /dev/null +++ b/.changeset/2026-09-04-per-operation-span.md @@ -0,0 +1,33 @@ +--- +"@dexpace/core": minor +--- + +`Runtime.send()` now opens one span per logical operation (OBS-29), and `createRuntime` takes an +optional context init supplying the instrumentation bundle it comes from. Additive — a runtime built +without one behaves exactly as before. + +`OBS-29` requires that one tracer instance correspond 1:1 to a single logical operation. The port +had spans, but `PIPE-2` fixes the LOGGING pillar step *inside* the RETRY and REDIRECT pipelines, so +every span it opened was per transmission attempt and per redirect hop — the right scope for an +attempt, the wrong one for an operation, and nowhere for the per-attempt and retries-exhausted events +to attach. `send()` is the only place in this package that runs exactly once per logical operation, +so the operation span is opened there, outside every pillar, and the LOGGING step's spans become its +children. + +```ts +const runtime = createRuntime(steps, transport, { + instrumentation: createInstrumentationBundle(() => myTracer), +}); +``` + +Ended exactly once, on exactly one of two paths: `end()` on success, or `recordException(error)` then +`end()` on failure — `OBS-29`'s mutually-exclusive succeeded/failed pair, under span names. + +**No span is opened when one is already active.** `Runtime implements Transport` (PIPE-26), so a +runtime can be another runtime's terminal transport, and a caller may have activated a span of their +own; in both cases the outermost one is the logical operation. An empty pipeline (PIPE-9) opens none +either, since it allocates no context. + +The remaining gap is the vocabulary, not the scope: the spec names `operationStarted` / +`operationSucceeded` / `operationFailed` and this port spells them as a span's lifecycle. That +shape difference is recorded in `docs/deviations.md`. diff --git a/.changeset/2026-09-04-proxy-challenge-handler-slot-rationale.md b/.changeset/2026-09-04-proxy-challenge-handler-slot-rationale.md new file mode 100644 index 0000000..4d572c4 --- /dev/null +++ b/.changeset/2026-09-04-proxy-challenge-handler-slot-rationale.md @@ -0,0 +1,27 @@ +--- +"@dexpace/core": patch +--- + +Document `ProxyOptions.challengeHandler`'s settled disposition on the field itself, in both +`ProxyOptions` and `ProxyOptionsInit` (CFG-22, TRANSPORT-30, SEAM-1). Documentation only: no +behavior, no types, and no exported symbol changed, so `packages/core/etc/core.api.md` is +byte-identical — both entries are still `readonly challengeHandler?: unknown;`. + +The slot was previously described as having "no protocol behind it yet", which read as pending +work. It is not pending. The field is required by `CFG-22`'s field list (a MUST) and gives +`TRANSPORT-30`'s SHOULD-warn clause a subject, but **nothing dispatches through it and nothing is +going to**: undici's `ProxyAgent` takes its credential solely from its own constructor and rejects +a per-request `Proxy-Authorization` with `InvalidArgumentError`, and that constructor runs before +any challenge exists, so a handler-minted credential can never reach the exchange that provoked it. +`@dexpace/transport-undici` answers the requirement by discoverability instead — a WARN at +construction, a second WARN on the first real `407`, Basic proxy auth through +`ProxyOptions.credentials`, and the `407` returned untouched. + +The TSDoc also records why the type stays `unknown` rather than becoming a declared signature: the +only concrete argument a handler could take is the native client's own response type, which +`SEAM-1`'s zero-runtime-dependency rule forbids core from naming, and a transport-neutral challenge +shape invented here would be a contract with no implementation behind it. + +The reasoning now lives in the code a consumer reads in the published `.d.ts` rather than in a +register, so the row for it has been removed from the deferral register; the full platform audit +remains at `docs/deviations.md` item 13. diff --git a/.changeset/2026-09-04-publish-client-identity-step.md b/.changeset/2026-09-04-publish-client-identity-step.md new file mode 100644 index 0000000..e32342b --- /dev/null +++ b/.changeset/2026-09-04-publish-client-identity-step.md @@ -0,0 +1,35 @@ +--- +"@dexpace/core": minor +--- + +Publish `clientIdentityStep` and `ClientIdentitySettings` on the package barrel (RECOV-33, NFR-15). +Purely additive: `packages/core/etc/core.api.md` gains ten lines and loses none, and the step's +behavior, defaults and error paths are unchanged. + +`RECOV-33`'s identity-stamping step has been implemented and tested since Phase 7a, and unreachable +for just as long — tagged `@internal`, absent from the barrel, and installed by nothing. +`standardResilience` does not install it, so the step's own TSDoc instruction ("a caller adds it to +their own pipeline") named an action no caller could take. Every other step factory was already +public: `authStep`, `retryStep`, `redirectStep`, `loggingStep`, `stripCrossOriginMarkerStep`. + +The blocker that kept it internal is gone and had been for two phases. The barrel comment claimed +its `StepDescriptor` return type was "part of the still-internal pipeline authoring surface", which +stopped being true when Phase 5c promoted `StepDescriptor`, `Stage`, `Step`, `StepContext` and +`PipelineBuilder`. Exporting the step therefore names no forgotten export, and api-extractor accepts +it unchanged. + +Its file stays at `packages/core/src/config/client-identity-step.ts`. A `@public` symbol named on the +barrel against its own module path has an invisible folder, and relocating it to `recovery/` would +trade its one outbound `→ pipeline/` edge for a new `→ config/` one for `./build-info.js`. + +Usage: + +```ts +import {clientIdentityStep, PipelineBuilder} from '@dexpace/core'; + +const runtime = new PipelineBuilder(transport) + .append(clientIdentityStep({tokens: ['acme-sdk/1.2.3'], mode: 'append'})) + .build(); +``` + +See `docs/work/mvp/2026-09-04-open-items-dissolution.md` K1 (fixed) and K11 (closed). diff --git a/.changeset/2026-09-04-publish-io-error-taxonomy.md b/.changeset/2026-09-04-publish-io-error-taxonomy.md new file mode 100644 index 0000000..4eae301 --- /dev/null +++ b/.changeset/2026-09-04-publish-io-error-taxonomy.md @@ -0,0 +1,37 @@ +--- +"@dexpace/core": minor +--- + +Publish the four flat I/O error leaves, `isIoError`, and the `SuppressedErrorLike` type. Additive — +no class changed, no hierarchy moved, and nothing was renamed. + +Newly on the barrel: `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError`, +`isIoError`, and `SuppressedErrorLike` (a type, not the class). `IoError`, `EndOfStreamError` and +`TransportFailureError` were already there. + +A caller receives these today and had no name to catch them by. `decodeResponse`'s guard is a single +`e instanceof DexpaceError` pass-through — anything already in this SDK's typed tree is never +re-typed — so a body stream that fails with a `ClosedResourceError` or an `AllocationLimitError` +delivers exactly that class, identity preserved, to a caller who could not `import` it. `isIoError` +is the category catch a deliberately flat error tree cannot offer through `instanceof`: + +```ts +import {isIoError} from '@dexpace/core'; + +try { + await decodeResponse(response, deserializer, {schema}); +} catch (error) { + if (isIoError(error)) retry(); +} +``` + +`SuppressedErrorLike` is exported as a type because `instanceof SuppressedError` is **not** a valid +test on this package's declared `engines.node >=20.3` floor, where the global is absent. A caller +narrowing a decode failure whose release also failed needs the structural shape — `name` is +`'SuppressedError'`, `.error` is the primary throwable, `.suppressed` rides along — not a class. + +This completes the taxonomy the previous release started: `DomainModelError` was flattened and +replaced with a `@public isDomainModelError` guard, making "two levels, plus an exported guard per +family" the settled shape. `isIoError` is that guard for `io/`, and `isBodyError` was already public. + +See `docs/work/mvp/2026-09-04-open-items-dissolution.md` H8. diff --git a/.changeset/2026-09-04-serde-seam-unification.md b/.changeset/2026-09-04-serde-seam-unification.md new file mode 100644 index 0000000..ab528a3 --- /dev/null +++ b/.changeset/2026-09-04-serde-seam-unification.md @@ -0,0 +1,47 @@ +--- +"@dexpace/core": minor +"@dexpace/codec-json": minor +--- + +**Breaking to the `Serde` SPI, taken deliberately before the first published version.** Every decode +entry point now takes a `DecodeTarget`, the two stream-driving methods take `{signal}`, and +`DecodeTarget` gains an `admitsNull` opt-in. + +```ts +// before +deserialize(data: Uint8Array, schema: Schema, typeName?: string): T; +deserializeFrom(source: ReadableStream, schema: Schema, typeName?: string): Promise; +serializeTo(value: unknown, sink: WritableStream): Promise; + +// after +deserialize(data: Uint8Array, target: DecodeTarget): T; +deserializeFrom(source: ReadableStream, target: DecodeTarget, options?: {signal?: AbortSignal}): Promise; +serializeTo(value: unknown, sink: WritableStream, options?: {signal?: AbortSignal}): Promise; +``` + +Migration is mechanical: `d.deserialize(bytes, schema, 'Dto')` becomes +`d.deserialize(bytes, {schema, typeName: 'Dto'})`. + +**One spelling, both layers.** `decodeResponse`/`decodeSuccessResponse` already bundled the +schema/label pair as `DecodeTarget`; the SPI took the same pair positionally. A codec author +implemented one shape while a caller used the other, and +`docs/knowledge/harvested/api-design.md:14` points at the object form for both. `DecodeTarget` now +lives on the seam, where a third-party codec implements against it, and the handler layer re-exports +it — one type, not two. + +**`{signal}` where an API drives a stream it did not open.** That is the project-wide rule, stated +once and applied here: `deserializeFrom` and `serializeTo` drive caller-owned streams and now accept +a signal; the buffered-bytes APIs (`serialize`, `serializeToString`, `toHttpError`, +`Response.bytes()`) correctly take none, and neither do `decodeResponse`/`decodeSuccessResponse`, +which hand the live stream to the codec and never read it. The abort reaches the drain loop and +leaves the caller's stream unlocked, uncancelled and unclosed (SERDE-3). The CPU-bound parse after +the drain is not interruptible by any signal — `JSON.parse` has no incremental form. + +**`DecodeTarget.admitsNull` (SERDE-13).** A wire `null` at the top level is still rejected before the +schema runs, unconditionally, because a schema *value* carries no nullability a codec could read and +moving the check later would let `{parse: (i) => i}` launder a `null` into a non-null `T`. Set +`admitsNull: true` to state what the schema cannot — that `T` includes `null` — and the check is +skipped. This is what makes a legitimately nullable success body decodable, and the one way +`tristate(inner)` can serve as a top-level target rather than a field combinator. + +See `docs/work/mvp/2026-09-04-open-items-dissolution.md` H9, H10 and H15. diff --git a/.claude/skills/ci-preflight/SKILL.md b/.claude/skills/ci-preflight/SKILL.md new file mode 100644 index 0000000..0843dbb --- /dev/null +++ b/.claude/skills/ci-preflight/SKILL.md @@ -0,0 +1,172 @@ +--- +name: ci-preflight +description: Use before pushing a branch, opening or updating a PR, or whenever asked whether CI will pass, to "run the CI checks", "check CI locally", or to verify a phase is done. Runs every blocking step of .github/workflows/ci.yml against the working tree, reports all failures at once, then resolves them. +--- + +# CI Preflight + +## Overview + +`.github/workflows/ci.yml` is 20 named steps across two jobs — 17 in `ci`, 3 in the +`node-conformance` matrix — and every one of them is blocking. Every one of them can run +locally, so a red CI run is always avoidable — `bun test` passing is not evidence, and it is +the single most common reason work gets handed over broken. + +One command runs all of them, in CI's order: + +```bash +node .claude/skills/ci-preflight/run-ci.mjs +``` + +~2.5 minutes warm on a green tree. Full output per step goes to +`node_modules/.cache/ci-preflight/.log`; only a summary and a tail of each failure +reach stdout, so a red run costs a few hundred tokens rather than the ~40k that all the +raw `bun run` calls would. + +Do not hand-run the individual commands instead. Two things go wrong when you do: + +- **Order is load-bearing.** `test`, `api`, `lint:publish` and every `verify:*` gate resolve + `@dexpace/core` by package name, which lands in `packages/core/dist/`. Run any of them + before `build` and they either fail with unresolved-module noise or — worse — pass green + against yesterday's artifact. +- **You will stop at the first failure.** The point is to hand the user the whole list. + +## The workflow + +1. **Run it.** Add `--skip-install` only if you have not touched `package.json` since the + last install. **Before you push, add `--clean`** — a warm tree is blind to a whole class of + defect CI hits on its first step. (The pinned Bun needs no flag; it is the default.) +2. **All green** → say so plainly: CI is all good, naming the count the runner itself prints + (`CI preflight: all N steps passed.`), not a number from this file. + Nothing else to do. +3. **Anything red** → report the findings to the user *first*: which gates failed, what each + one means, and the fix you intend. One line per finding, not a transcript dump. +4. **Then resolve them**, using the playbook below. +5. **Re-verify.** Re-run the affected gates while iterating + (`--only lint,api --skip-install`), then **one full `--clean` run before reporting done**. + A subset pass is not a green CI — fixes cross gate boundaries constantly (a lint fix edits + an export, which moves the API report, which fails `api`). + +Report honestly at every step: if a gate still fails, say so with its output. Never describe +a subset run as a full one. + +**Resolve means fix the defect, not silence the gate.** Lowering `coverageThreshold`, +deleting a failing test, adding an `eslint-disable`, or regenerating an `.api.md` to bless an +unintended export are all ways to make the runner green while shipping the bug. Where the +real fix is a judgment call — a deliberate spec deviation, a moved runtime floor, an +intentional public-API change — stop and ask. This repo is structured specifically to +prevent silent gaps (`CLAUDE.md`, "Requirement-ID conventions"); a suppression needs a stated +reason and an owner. + +## Two failure modes that read as success + +Both of these will make you report a passing gate that CI rejects. + +- **A compile error in core masks every lint finding.** `typecheck`, `lint` and `build` all + run `build:core` first, so one bad type in `packages/core/src/` makes all three fail with + the *same* `tsc` error and `gts lint .` never executes. Fix the compile error, then re-run + `lint` — the formatting and rule findings are still there, unseen. +- **Your Bun is not CI's Bun** — handled by default, but know why. `.bun-version` is what + `setup-bun` resolves, and Bun's `fetch` and `node:http` are independent implementations that + change between releases; a test can pass on yours and fail on CI with no code difference at + all. **The runner pins every step to `.bun-version` via mise automatically**, nested + `bun run` chains included. If mise cannot supply it, the run continues but says loudly that + it is measuring the wrong runtime — that banner is not decoration, and a green run under it + is not a green CI. `--path-bun` opts out deliberately, which is worth doing only to check + whether a newer Bun fixes something. + + PR #52 hit this twice in one run, both invisible on Bun 1.4.0: `node:http` emitted a + response carrying *both* `Transfer-Encoding: chunked` and `Content-Length` with an unchunked + body (undici rejected it, Bun's own `fetch` hung to a 5s timeout), and `fetch` served a + poisoned pooled connection to a later row, failing a timeout assertion ~30 rows from its + cause. Reproducing each took one command on the pinned version and was guesswork without it. +- **A warm tree hides missing build prerequisites.** CI checks out a tree with no `dist/` in + it; yours almost never is one. A package whose `exports` point at `dist/`, imported by name + from another package's `src/` with nothing building it first, resolves fine locally against + the leftovers of your last build and fails on a fresh clone. Every gate goes green here and + CI dies on step 2. **`--clean` is the answer** — it sweeps every `dist/` and `*.tsbuildinfo` + first, so the run starts where CI starts. It costs ~40s of rebuild. + + This is not hypothetical. PR #52 failed exactly this way: Phase 8a made + `@dexpace/transport-shared` the second published package imported by name from another + package's `src/`, `typecheck` and `lint` still pre-built only core, and a warm preflight + passed every step on the commit CI rejected. Fixed by `build:deps` — see CLAUDE.md, and + keep that list current when a new package crosses the same line. +- **The coverage floor fails silently.** `bun test` enforces `bunfig.toml`'s + `coverageThreshold` (0.8) by **exit code alone**. It prints no threshold message, and the + summary still reads `0 fail`. The runner prints a `note:` when it detects this; without + that note you would read the tail and conclude the step passed. (The + `--coverage-threshold` CLI flag is ignored — bunfig is what gates.) + +## Resolution playbook + +`fix:` lines the runner prints come from here. Steps are listed in run order. + +| Step | A failure means | First move | +|---|---|---| +| `install` | `bun.lock` disagrees with a `package.json`. The tree CI installs is not yours, so nothing after it is measuring the right thing — the runner stops here. | `bun install`, then commit `bun.lock`. | +| `typecheck` | `tsc --noEmit` over all 9 projects. | `Cannot find module '@dexpace/…'` means a build prerequisite is missing from `build:deps`, not a bad import — check with `--clean`. Otherwise a real fix; usual suspects: a missing `.js` extension on a relative import (NodeNext), a type import without `import type` (`verbatimModuleSyntax`), an enum/namespace/parameter property (`erasableSyntaxOnly`). | +| `lint` | Formatting **and** type-aware rules; formatting is an error, not a warning. | `bun run fix` first — it clears every prettier finding. Hand-fix what survives: 70-line function cap, `max-depth` 3, `max-params` 3, explicit return types on exported members. Every `eslint-disable` needs a `-- reason`. | +| `build` | Emit failed. **Blocks the eleven gates below it**, which the runner reports `SKIP`. | Fix this before reading anything else; the skipped gates are unknown, not passing. | +| `test` | A failing test, *or* the silent coverage floor (see above). | If the tail says `0 fail`, it is coverage — find the file that dropped below 0.8 in the printed table and test it. Otherwise fix the test or the code. | +| `test:scripts` | A gate's own logic broke, or the knowledge corpus shifted under an assertion that pins its shape. | `node --test scripts/.test.mjs` for detail. If it is `knowledge.test.mjs`'s ID-less-topic count, a corpus edit gave a previously ID-less topic its first requirement ID — confirm that was intended, then move the number in the test, `CLAUDE.md` and `knowledge-lookup/SKILL.md` together. Otherwise fix the gate; never relax the assertion to match a degraded gate. | +| `api` | The committed `etc/.api.md` no longer matches the built surface, or an export lacks TSDoc. | Intended export change: `cd packages/ && bun run api:local`, then commit the regenerated report. `(undocumented)` in the diff means the export needs a `@public` block, plus `@throws` naming each catchable error class. **Unintended** change: revert the export, don't bless the report. | +| `lint:publish` | `publint` + `attw` on every built package's `exports` map, `types`/`main` fields, and declaration resolution. | Fix the manifest. `cjs-resolves-to-esm` is already ignored by design (ESM-only); every other rule is real. | +| `verify:dual-consumption` | A built package is no longer importable and runnable by plain `node` through its package name. | Usually a broken `exports` map or a subpath that ships no JS. | +| `verify:consumer-types` | The built `.d.ts` does not compile on the declared `lib` with `types: []` — i.e. a dev-only global (`@types/bun`) leaked into the public surface. | Remove the dependency on the dev global, or declare it. This gate exists because exactly that defect passed all four gates above it. | +| `verify:seam-1` | A package gained a runtime dependency outside the allow-list, or dropped its committed empty `dependencies` object (an omitted field is a violation too). | Remove the dependency — SEAM-1 is the constraint, not the gate. `@dexpace/core` is a **peer** of the satellites, never a dependency. | +| `verify:sse-37` | Core's SSE code reached for serde or a codec package. | Remove the import; SSE-37/38 forbid the coupling. | +| `verify:runtime-floor` | `engines.node` and the `target`/`lib` a package compiles to have drifted apart. | Move both together, deliberately — never raise one to silence this. | +| `verify:test-partition` | One of the five strings that keep `tests/conformance/` (Bun) and `tests/node-conformance/` (`node --test`) apart has drifted — see CLAUDE.md's hard rule. Every way this breaks is silent: Bun runs `node:test` files and reports them **passing**, Bun ignores an unrecognized `[test]` key with no warning, and `node --test` over a glob matching nothing exits 0. | The assertion names the file and the string. Fix all five together — `bunfig.toml`, `package.json`, `eslint.config.js`, `run-ci.mjs`, `tests/node-conformance/README.md` — never one alone. Unlike the gates around it this one reads files only, so it still reports through a red `build` rather than going `SKIP`. | +| `verify:reproducible-build` | Two clean builds of an identical source tree disagreed (NFR-12) — either in an emitted `dist/` file or in an `npm pack` tarball. | The assertion names what differed. A wall-clock or random value reaching a build-time codegen step is the usual cause; `packages/core/scripts/gen-version.mjs` is the only such step today, and injecting a `Date.now()` there is this gate's own negative test (it fails naming `packages/core/dist/generated/version.js` and `npm-pack:dexpace-core-0.0.0.tgz`). If the log instead ends in `tsc` errors, the **build inside the gate** failed and there is no difference to read — fix that first. The gate sweeps every `dist/` and rebuilds twice itself, so it is last in the job and leaves the tree freshly built. | +| `audit` | A high-severity advisory in production dependencies. | `bun audit --prod` for detail. Note the tree is tiny (zero runtime deps by design), so a hit here is usually a transitive dev-dep misclassification worth reading carefully. | +| `test:node` | Bun-vs-Node runtime divergence, almost always in `packages/core/src/io/` — Web Streams, `AbortSignal`, `Uint8Array` chunking. | Fix against Node's semantics. A phase touching a runtime-divergent surface should be *adding* cases here; see `tests/node-conformance/README.md`. | + +**A `timeout` verdict is not the same finding as a red one.** Every step is capped at +`STEP_TIMEOUT_MS` (10 minutes) in `run-ci.mjs`, and a step that hits the cap is reported `timeout` +whether it hung or was merely slow. Nearly all of them finish in seconds; +`verify:reproducible-build` is the exception, performing **two full swept builds plus two `npm pack` +passes** — ~34s observed warm, but a cold or loaded machine multiplies that, and it is the one step +with any real chance of approaching the cap. If *it* comes back `timeout`, raise `STEP_TIMEOUT_MS` +and re-run before reading the verdict as a reproducibility defect. + +## Local-vs-CI divergences worth stating + +The runner reproduces CI's steps, not CI's machine. Two gaps survive, and both belong in +your report when they matter: + +- **Node version.** `test:node` runs on whatever `node` is active; CI runs it twice, on the + `engines.node` floor (**20.3.0**) and on `lts/*`. A green local run on a newer Node does + not prove the floor. `--node-floor` runs the floor leg via `mise`/`fnm`/`nvm` (downloading + the toolchain once); the runner prints a note when the active major is not 20. + + **Run it whenever the change adds or edits a file under `tests/node-conformance/`**, touches + `io/`, reaches for a new built-in, or moves the floor. This gap is not theoretical: Phase + 8a's `transport.test.mjs` passed on Node 26 and failed 20 of 22 cases on 20.3.0, because an + async *root-level* `before` hook does not complete before subtests inside a `describe` when + a file's only root children are suites — fixed in Node 22, and invisible to every other + gate. Own hooks from an enclosing `describe`, never the file root. +- **Bun version.** Closed by default — the runner pins to `.bun-version` itself. The gap + reopens only when mise cannot supply that version, and the run says so in a banner. + +CI also runs `node-conformance` only after the `ci` job succeeds — so locally, a `test:node` +failure alongside other failures is the same signal, just surfaced earlier. + +Not in CI at all, so the runner does not include it: changesets (a consumer-facing change +still needs `bun run changeset`). `test:scripts` used to be on this list; Phase 10 wired it +into the `ci` job, so the runner covers it now. + +## Runner flags + +| Flag | Effect | +|---|---| +| `--only a,b` | Run just these step ids. The iteration loop; still respects order and the build-gates-everything rule. | +| `--clean` | Sweep every `dist/` and `*.tsbuildinfo` first, so the run starts from the tree CI checks out. The pre-push default. ~40s. | +| `--path-bun` | Run on PATH's bun instead of `.bun-version`'s. The pinned Bun is the default; use this only to test a newer one. | +| `--skip-install` | Skip the frozen-lockfile install. Safe when `package.json` is untouched. | +| `--node-floor` | Also run `test:node` under Node 20.3.0 via mise/fnm/nvm. | +| `--tail N` | Lines of a failing log to print (default 30). Raise for a wall of tsc errors. | +| — | Each step is capped at 10 minutes (`STEP_TIMEOUT_MS`) and reported `timeout` if it hangs. A gate *can* hang rather than fail — a conformance test holding the event loop open on an unclosed server does exactly that. It can also just be slow: `verify:reproducible-build` builds the workspace twice and packs it twice, so raise the cap rather than diagnosing a `timeout` there as a real failure. | +| `--list` | Step ids and the command each runs. | + +Exit code is 0 only when every selected step ran and passed. `SKIP` is never a pass. diff --git a/.claude/skills/ci-preflight/run-ci.mjs b/.claude/skills/ci-preflight/run-ci.mjs new file mode 100644 index 0000000..c080e99 --- /dev/null +++ b/.claude/skills/ci-preflight/run-ci.mjs @@ -0,0 +1,478 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/ci-preflight/run-ci.mjs +// +// Runs every blocking step of `.github/workflows/ci.yml` against the working tree, in CI's own +// order, and reports all failures at once rather than stopping at the first. +// +// Two things make this more than a shell alias for sixteen `bun run` calls: +// +// * Ordering is load-bearing. `bun test`, `api`, `lint:publish` and every `verify:*` gate resolve +// `@dexpace/core` by package name, which lands in `packages/core/dist/`. Run them before +// `build` and they either fail with unresolved-module noise or, worse, pass green against +// yesterday's artifact. CI is safe because its Build step precedes its Test step; a human +// running gates ad hoc is not. +// * A failed `build` invalidates the eleven gates downstream of it. Running them anyway produces +// eleven spurious findings that all say "cannot resolve @dexpace/core". They are reported SKIP +// here, so the summary names the one real defect. +// +// Logs go to node_modules/.cache/ci-preflight/.log — full output stays on disk, only the +// summary and a tail of each failure reach stdout. + +import {spawnSync} from 'node:child_process'; +import { + globSync, + mkdirSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import {argv, cwd, env, exit, stdout, version} from 'node:process'; + +// Mirrors ci.yml step for step. `ci` is the workflow's own step name, so a failure here can be +// matched to the job that would have caught it. `fix` is the mechanical remedy where one exists. +const STEPS = [ + { + id: 'install', + ci: 'Install (frozen lockfile)', + cmd: 'bun install --frozen-lockfile', + tier: 'install', + fix: 'bun install (then commit the updated bun.lock)', + }, + { + // First of the gates, mirroring ci.yml: pure Node over Markdown, no build, + // so a corpus mistake reports in seconds rather than after the pipeline. + id: 'verify:knowledge-structure', + ci: 'Knowledge-corpus structure check', + cmd: 'bun run verify:knowledge-structure', + tier: 'gate', + }, + {id: 'typecheck', ci: 'Typecheck', cmd: 'bun run typecheck', tier: 'build'}, + { + id: 'lint', + ci: 'Lint', + cmd: 'bun run lint', + tier: 'build', + fix: 'bun run fix', + }, + {id: 'build', ci: 'Build', cmd: 'bun run build', tier: 'build'}, + { + id: 'test', + ci: 'Test (with coverage)', + cmd: 'bun test --coverage', + tier: 'gate', + // `bun test` fails the bunfig coverage floor by exit code ALONE -- it prints no threshold + // message, and the summary above it still reads "0 fail". Read the tail without this note and + // the obvious conclusion is that the step passed. (The `--coverage-threshold` CLI flag is + // ignored; bunfig.toml's `coverageThreshold` is the one that gates.) + diagnose: output => + /^\s*0 fail\s*$/m.test(output) + ? 'every test passed, so this is the coverage floor in bunfig.toml (0.8), not a failing' + + ' test. Find the file that dropped below it in the table above.' + : null, + }, + { + // Tier `static`, not `gate`. `tier` has exactly one consumer -- the build-failed SKIP rule at + // the bottom of this file, which tests for `gate` -- so `static` means "runs even when `build` + // is red". Correct here and for verify:test-partition below: both read files and resolve no + // workspace package by name, so a failed build does not make either meaningless the way it does + // the gates around them. + id: 'test:scripts', + ci: 'Gate self-tests (scripts/*.test.mjs)', + cmd: 'bun run test:scripts', + tier: 'static', + }, + { + id: 'api', + ci: 'API surface check', + cmd: 'bun run api', + tier: 'gate', + fix: 'cd packages/ && bun run api:local, then commit etc/.api.md', + }, + { + id: 'lint:publish', + ci: 'Package health (publint + attw)', + cmd: 'bun run lint:publish', + tier: 'gate', + }, + { + id: 'verify:dual-consumption', + ci: 'Dual JS/TS consumption check', + cmd: 'bun run verify:dual-consumption', + tier: 'gate', + }, + { + id: 'verify:consumer-types', + ci: 'Consumer typecheck against the published .d.ts', + cmd: 'bun run verify:consumer-types', + tier: 'gate', + }, + { + id: 'verify:seam-1', + ci: 'SEAM-1 zero-dependency check', + cmd: 'bun run verify:seam-1', + tier: 'gate', + }, + { + id: 'verify:sse-37', + ci: 'Verify SSE-37/SSE-38', + cmd: 'bun run verify:sse-37', + tier: 'gate', + }, + { + id: 'verify:runtime-floor', + ci: 'Runtime-floor consistency check', + cmd: 'bun run verify:runtime-floor', + tier: 'gate', + }, + { + // Tier `static` — see `test:scripts` above. + id: 'verify:test-partition', + ci: 'Test-partition check (tests/ vs tests/node-conformance/)', + cmd: 'bun run verify:test-partition', + tier: 'static', + fix: 'the assertion names the string that drifted — change all five together, never one alone', + }, + { + // Resolves `@dexpace/core` and `@dexpace/codec-json` by name, so it needs the built dist/. + id: 'test:examples', + ci: 'Example canaries', + cmd: 'bun run test:examples', + tier: 'gate', + }, + { + // Tier `static` — see `test:scripts` above: it reads source text and resolves no workspace + // package, so a swept dist/ cannot make it red. + id: 'verify:import-cycles', + ci: 'Import-cycle check', + cmd: 'bun run verify:import-cycles', + tier: 'static', + fix: 'the failure names every file on the cycle — break it by moving the shared declaration into a module both sides import, not by making one edge type-only', + }, + { + // Last among the gates, matching ci.yml: it sweeps every dist/ and rebuilds twice, so running it + // earlier would pull the tree out from under any step that resolves a workspace package by name. + id: 'verify:reproducible-build', + ci: 'Reproducible-build check (NFR-12)', + cmd: 'bun run verify:reproducible-build', + tier: 'gate', + }, + {id: 'audit', ci: 'Dependency audit', cmd: 'bun run audit', tier: 'gate'}, + { + id: 'test:node', + ci: 'node-conformance (matrix)', + cmd: 'bun run test:node', + tier: 'gate', + }, +]; + +// engines.node across every publishable package, and the floor leg of ci.yml's node-conformance +// matrix. The other leg is `lts/*`, which resolves at run time and so cannot be pinned here. +const NODE_FLOOR = '20.3.0'; +// Comfortably past the slowest gates (`api` ~50s, `verify:reproducible-build` ~34s warm) without +// letting a hung one stall the run. `verify:reproducible-build` is the one worth watching as this +// grows: it does two full swept builds and two `npm pack` passes, so a cold or loaded machine +// multiplies its wall time. If it ever reports `timeout`, raise this rather than reading the verdict +// as a reproducibility defect. +const STEP_TIMEOUT_MS = 10 * 60 * 1000; +// setup-bun resolves this file, so it is the Bun every CI step actually runs on. +const PINNED_BUN = readFileSync('.bun-version', 'utf8').trim(); +const LOG_DIR = 'node_modules/.cache/ci-preflight'; + +function parseArgs(args) { + const opts = { + only: null, + skipInstall: false, + tail: 30, + nodeFloor: false, + clean: false, + pinnedBun: true, + }; + for (let i = 0; i < args.length; i++) { + const arg = args[i]; + if (arg === '--list') opts.list = true; + else if (arg === '--skip-install') opts.skipInstall = true; + else if (arg === '--node-floor') opts.nodeFloor = true; + else if (arg === '--clean') opts.clean = true; + else if (arg === '--pinned-bun') opts.pinnedBun = true; + else if (arg === '--path-bun') opts.pinnedBun = false; + else if (arg === '--only') + opts.only = (args[++i] ?? '').split(',').filter(Boolean); + else if (arg.startsWith('--only=')) + opts.only = arg.slice(7).split(',').filter(Boolean); + else if (arg === '--tail') opts.tail = Number(args[++i]); + else if (arg.startsWith('--tail=')) opts.tail = Number(arg.slice(7)); + else if (arg === '--help' || arg === '-h') opts.help = true; + else { + console.error(`unknown argument: ${arg}\nRun with --help.`); + exit(2); + } + } + return opts; +} + +const HELP = `Usage: node .claude/skills/ci-preflight/run-ci.mjs [options] + +Runs every blocking step of .github/workflows/ci.yml against the working tree. + + --only a,b Run only these step ids (see --list). Ordering and the + build-gates-everything rule still apply. + --skip-install Skip the frozen-lockfile install. + --node-floor Additionally run test:node under Node ${NODE_FLOOR}, CI's floor + leg. Needs mise, fnm, or nvm; downloads the toolchain once. + --clean Delete every dist/ and *.tsbuildinfo first, so the run starts + from the state CI checks out. Catches missing build + prerequisites that a warm tree hides. Costs ~40s. + --path-bun Run on PATH's bun instead of .bun-version's (${PINNED_BUN}). + The pinned Bun is the DEFAULT: Bun's fetch and node:http differ + between releases enough to pass locally and fail on CI. Use this + only to check whether a newer Bun fixes something. + --tail N Lines of a failing step's log to print (default 30). + --list List step ids and exit. + +Exit code is 0 only when every step selected ran and passed.`; + +function selectSteps(opts) { + let steps = STEPS; + if (opts.only) { + const known = new Set(STEPS.map(s => s.id)); + const unknown = opts.only.filter(id => !known.has(id)); + if (unknown.length > 0) { + console.error( + `unknown step id(s): ${unknown.join(', ')}\nKnown: ${[...known].join(', ')}`, + ); + exit(2); + } + steps = STEPS.filter(s => opts.only.includes(s.id)); + } + if (opts.skipInstall) steps = steps.filter(s => s.id !== 'install'); + return steps; +} + +// The pinned Bun is the default, not an opt-in. `.bun-version` is what `setup-bun` resolves, and +// Bun's `fetch` and `node:http` are independent implementations that move between releases -- a +// rehearsal on a different one is not a rehearsal. Phase 8a lost a CI round to exactly that: three +// transport rows that pass on 1.4.0 fail on the pinned 1.3.14, two of them from malformed HTTP +// framing the newer Bun emits correctly. +// +// Prepending to PATH rather than wrapping each command in `mise x`: a root script like `typecheck` +// shells out to `bun run build:deps`, which shells out again. Only the environment reaches all of +// them. +function pinnedBunEnv() { + const active = spawnSync('bun', ['--version'], {encoding: 'utf8'}); + if (active.status === 0 && active.stdout.trim() === PINNED_BUN) { + stdout.write(`bun: ${PINNED_BUN} on PATH already matches .bun-version\n`); + return null; + } + const probe = spawnSync('mise', ['where', `bun@${PINNED_BUN}`], { + encoding: 'utf8', + }); + if (probe.status !== 0) { + // Loud, because the run that follows is measuring a runtime CI will not use. Not fatal: a + // preflight on the wrong Bun still catches everything that is not runtime-specific, and + // refusing to run at all would be worse than running with the caveat stated. + stdout.write( + `\n!! bun: .bun-version pins ${PINNED_BUN}; PATH has ` + + `${active.stdout.trim() || 'an unknown version'}, and mise cannot supply the pinned one.\n` + + ' Steps will run on the WRONG Bun — runtime-specific failures may not reproduce.\n' + + ` Fix with: mise install bun@${PINNED_BUN}\n\n`, + ); + return null; + } + const bin = `${probe.stdout.trim()}/bin`; + stdout.write( + `bun: pinning every step to ${PINNED_BUN} from .bun-version ` + + `(PATH has ${active.stdout.trim() || 'unknown'})\n`, + ); + return {...env, PATH: `${bin}:${env.PATH ?? ''}`}; +} + +function run(step, tail, childEnv) { + const started = Date.now(); + // `2>&1` inside the shell rather than two piped streams: spawnSync hands back stdout and stderr + // as separate buffers, and concatenating them puts bun's own `$ script` echo *after* the compiler + // error it preceded. The tail is the part that gets read, so it has to be in real order. + const result = spawnSync(`${step.cmd} 2>&1`, { + shell: true, + encoding: 'utf8', + maxBuffer: 64 * 1024 * 1024, + // A gate CAN hang rather than fail: a `node --test` file whose teardown hook never runs holds + // the event loop open on an unclosed server and waits forever. Without a cap the whole preflight + // stalls behind it, which reads as "still running" and is the one outcome worse than a red run. + timeout: STEP_TIMEOUT_MS, + killSignal: 'SIGKILL', + ...(childEnv ? {env: childEnv} : {}), + }); + const seconds = Math.round((Date.now() - started) / 1000); + const output = `$ ${step.cmd}\n\n${result.stdout ?? ''}${result.stderr ?? ''}`; + const log = `${LOG_DIR}/${step.id.replace(/[:/]/g, '-')}.log`; + writeFileSync(log, output); + const lines = output.trimEnd().split('\n'); + const timedOut = + result.error?.code === 'ETIMEDOUT' || result.signal === 'SIGKILL'; + const ok = result.status === 0 && !timedOut; + return { + ...step, + seconds, + log, + ok, + timedOut, + status: timedOut ? 'timeout' : result.status, + note: timedOut + ? `no output for ${STEP_TIMEOUT_MS / 60000} minutes — killed. A hang here is usually a test` + + ' holding the event loop open (an unclosed server, a teardown hook that never ran), not a' + + ' slow gate.' + : (step.diagnose?.(output) ?? null), + tail: lines.slice(-tail).join('\n'), + }; +} + +function report(results, skipped, opts) { + stdout.write('\n'); + for (const r of results) { + const mark = r.ok ? 'PASS' : 'FAIL'; + const where = r.ok ? '' : ` ${r.log}`; + stdout.write( + ` ${mark} ${r.id.padEnd(24)} ${String(r.seconds).padStart(3)}s${where}\n`, + ); + } + for (const s of skipped) { + stdout.write( + ` SKIP ${s.id.padEnd(24)} build failed — gate not meaningful\n`, + ); + } + + const failed = results.filter(r => !r.ok); + stdout.write('\n'); + if (failed.length === 0 && skipped.length === 0) { + stdout.write(`CI preflight: all ${results.length} steps passed.\n`); + return 0; + } + + stdout.write( + `CI preflight: ${failed.length} FAILED — ${failed.map(f => f.id).join(', ')}\n`, + ); + for (const f of failed) { + stdout.write( + `\n${'='.repeat(72)}\n${f.id} (ci.yml step: "${f.ci}", exit ${f.status})\n`, + ); + if (f.note) stdout.write(`note: ${f.note}\n`); + if (f.fix) stdout.write(`fix: ${f.fix}\n`); + stdout.write(`${'='.repeat(72)}\n${f.tail}\n`); + stdout.write(`[last ${opts.tail} lines; full log: ${f.log}]\n`); + } + return 1; +} + +// The three globs below duplicate `package.json`'s `test:node`, deliberately: this leg runs the +// suite under a pinned Node, so it cannot go through `bun run`. They are three of the five strings +// that hold the `tests/` partition (CLAUDE.md's hard rule) and are checked by +// `scripts/verify-test-partition.mjs` -- a stale glob here makes `node --test` match nothing and +// exit 0, which reads as a clean floor run over zero cases. +function runNodeFloor(opts, childEnv) { + const managers = [ + [ + 'mise', + `mise x node@${NODE_FLOOR} -- node --test tests/node-conformance/*.test.mjs`, + ], + [ + 'fnm', + `fnm exec --using=${NODE_FLOOR} node --test tests/node-conformance/*.test.mjs`, + ], + [ + 'nvm', + `bash -lc 'nvm exec ${NODE_FLOOR} node --test tests/node-conformance/*.test.mjs'`, + ], + ]; + const found = managers.find( + ([bin]) => spawnSync('command', ['-v', bin], {shell: true}).status === 0, + ); + if (!found) { + stdout.write( + `\nnode-floor: no mise/fnm/nvm on PATH — Node ${NODE_FLOOR} leg not exercised.\n`, + ); + return null; + } + stdout.write( + `\nnode-floor: running test:node under Node ${NODE_FLOOR} via ${found[0]}...\n`, + ); + return run( + { + id: 'test:node@floor', + ci: `node-conformance (${NODE_FLOOR})`, + cmd: found[1], + }, + opts.tail, + childEnv, + ); +} + +const opts = parseArgs(argv.slice(2)); +if (opts.help) { + stdout.write(`${HELP}\n`); + exit(0); +} +if (opts.list) { + for (const s of STEPS) stdout.write(`${s.id.padEnd(24)} ${s.cmd}\n`); + exit(0); +} + +// CI checks out a tree with no build artifacts in it; a working tree almost never is one. That gap +// hides a whole class of defect -- a package whose `exports` point at `dist/` being imported by name +// from another package's `src/` without anything building it first. Every gate passes locally +// against the stale `dist/` left over from the last build, and the fresh clone CI runs cannot +// resolve the module at all. Sweeping the artifacts is what makes the preflight a real rehearsal. +function cleanArtifacts() { + const targets = [ + ...globSync('packages/*/dist'), + ...globSync('packages/*/*.tsbuildinfo'), + ]; + for (const target of targets) rmSync(target, {recursive: true, force: true}); + stdout.write( + `clean: removed ${targets.length} build artifact(s) — starting from CI's state\n`, + ); +} + +mkdirSync(LOG_DIR, {recursive: true}); +const steps = selectSteps(opts); +if (opts.clean) cleanArtifacts(); +const childEnv = opts.pinnedBun ? pinnedBunEnv() : null; +stdout.write( + `CI preflight — ${steps.length} step(s) from .github/workflows/ci.yml, in ${cwd()}\n`, +); + +const results = []; +const skipped = []; +let buildFailed = false; +for (const step of steps) { + if (buildFailed && step.tier === 'gate') { + skipped.push(step); + continue; + } + stdout.write(` ... ${step.id}\n`); + const result = run(step, opts.tail, childEnv); + results.push(result); + if (!result.ok && step.id === 'build') buildFailed = true; + // A frozen-lockfile failure means the dependency tree on disk is not the one CI installs. + // Everything after it would be measuring the wrong tree. + if (!result.ok && step.id === 'install') { + stdout.write( + '\ninstall failed — the tree on disk is not the tree CI builds. Stopping.\n', + ); + break; + } +} + +if (opts.nodeFloor && !buildFailed) { + const floor = runNodeFloor(opts, childEnv); + if (floor) results.push(floor); +} else if (!opts.nodeFloor && results.some(r => r.id === 'test:node')) { + const major = Number(version.slice(1).split('.')[0]); + if (major !== 20) { + stdout.write( + `\nnote: test:node ran on Node ${version}; CI also runs it on ${NODE_FLOOR} (the` + + ' engines.node floor). Re-run with --node-floor to exercise that leg.\n', + ); + } +} + +exit(report(results, skipped, opts)); diff --git a/.claude/skills/housekeeping/SKILL.md b/.claude/skills/housekeeping/SKILL.md new file mode 100644 index 0000000..b40764f --- /dev/null +++ b/.claude/skills/housekeeping/SKILL.md @@ -0,0 +1,173 @@ +--- +name: housekeeping +description: Use when asked to tidy docs/, check whether CLAUDE.md or README.md still match the code, file phase documents left in docs/superpowers/, find broken links or dangling open-items citations, or verify the documentation before handing work over. Probes the repository for documentation drift, reports it, and applies the mechanical repairs. +--- + +# Housekeeping + +## Overview + +Documentation drifts because nothing checks it. `CLAUDE.md` claimed "two published packages +today" for nine phases while the workspace grew to eleven; `README.md` was two lines with a +spelling error; two shipped package READMEs opened with a code sample that had stopped +compiling. Every one of those is checkable against the repository in a few lines of script, +and none of them was checked. + +This skill is that check. Two stages, and the order is not optional. + +```bash +node .claude/skills/housekeeping/probe.mjs # read-only. Report. Always first. +node .claude/skills/housekeeping/apply.mjs # dry run: prints the moves it would make +node .claude/skills/housekeeping/apply.mjs --write +``` + +It is a hand-run tool, not a CI step. Run it before claiming documentation is current, +after landing a phase, and whenever `docs/superpowers/` has something in it. + +## Stage 1 — probe + +Read-only, and tested to be: `probe.test.mjs` snapshots `git status --porcelain` around a +run and asserts it did not move. Exit code is 0 by default; `--strict` exits 1 when +anything is found, so it can be promoted to a gate without changing what it reports. + +```bash +node .claude/skills/housekeeping/probe.mjs +node .claude/skills/housekeeping/probe.mjs --strict +node .claude/skills/housekeeping/probe.mjs --only=links,citations +``` + +Eight checks. Each derives the repository fact **once, from the repository**, and compares +every document that states it against that one derivation — never one document against +another. + +| Check | Finds | +|---|---| +| `inbox` | Files in `docs/superpowers/` that belong under `docs/work//phaseN/` | +| `root` | Markdown at the repository root that belongs under `docs/` | +| `claims` | `CLAUDE.md` and `README.md` against the real package list, the real `verify:*` gate list, the real named-CI-step count, the real API-report count and the real `docs/` tree; plus `docs/README.md` against the tree it indexes | +| `readmes` | A publishable package with no README, one under 800 bytes, or one declaring `@dexpace/core` as a dependency rather than a peer | +| `links` | Broken relative links in `docs/`, `CLAUDE.md`, `README.md` and every package README | +| `registers` | An aggregate `## Open Findings` / `## Deferred Items Log` / `## Open Items` left in a specification document instead of a register at the `docs/` root | +| `citations` | An `open-items.md ` citation — in either spelling, the pre-dissolution one or the archive path — matching neither a `### ` heading in `docs/work/mvp/2026-09-04-open-items-dissolution.md` nor a `## Purged item IDs` row in `docs/work/mvp/2026-09-04-register-retirement-purge.md`. Both are dated archives; the register itself was dissolved on 2026-09-04 | +| `guard` | The frozen list and the writable surface overlapping — the one way the apply stage could eat a normative document | + +A **separate** check needs the built packages and so runs on its own: + +```bash +bun run build && node .claude/skills/housekeeping/check-fences.mjs +``` + +It extracts every ` ```typescript ` fence that imports from `@dexpace/*` and typechecks the +lot against `dist/`. A fence with no such import is an illustrative fragment; a fence with a +relative import is package-local. Both are skipped, and an import of a package absent from +this workspace (`pino`, `debug`, `zod`) is reported rather than failed. + +## Stage 2 — apply + +**Only after reading the probe's report.** The apply stage does exactly one thing: drains +`docs/superpowers/` into `docs/work//phaseN/` with `git mv`, so `git log --follow` +resolves each file across the move. + +```bash +node .claude/skills/housekeeping/apply.mjs # dry run +node .claude/skills/housekeeping/apply.mjs --write +node .claude/skills/housekeeping/apply.mjs --write --delivery=v2 +``` + +It refuses the whole batch if any path is frozen, and refuses if a target already exists, +rather than half-applying. + +Everything else the probe reports — a stale count in `CLAUDE.md`, a missing README, a broken +link, a dangling citation — is **prose, and you edit it**. That is deliberate. A tool that +rewrites a sentence to make its own check pass produces documentation that is true and +useless at the same time. The probe tells you what is wrong and where; the judgement about +what the sentence should say is yours. + +Two things `--write` does not do, and says so when it finishes: + +1. **Repoint references.** Re-run the probe's `links` and `citations` checks and fix what + they report, in the same commit — a comment that no longer matches the code is corrected + with the change that staled it (`docs/knowledge/harvested/documentation.md:34`). +2. **Commit.** A migration is its own commit, `git mv` only, so history follows every file. + +## What it must never write + +``` +docs/knowledge/ docs/product-spec/ docs/product-spec.md + docs/sdk-design-nodejs/ docs/sdk-design-nodejs.md +``` + +This is a guard, not a promise. `guard.mjs` exports `assertWritable` and +`assertAllWritable`, and the two places this skill writes both go through one of them: +`apply.mjs`'s batch pre-check before any `git mv`, and `check-fences.mjs`'s scratch +directory, which it opens by deleting. `guard.test.mjs` proves the four ways a naive +implementation fails — a sibling whose name merely starts with a frozen one +(`docs/product-spec-draft/`), a `..` segment that lands inside after normalization, an +absolute path, and a **symlink** whose target is inside a frozen tree while its own path is +not. The frozen list itself is pinned by a test, so widening it is a reviewed diff rather +than a silent constant change. + +Both call sites are covered by a test that fails when the call is deleted: +`apply.test.mjs`'s `--delivery=../product-spec` case and `check-fences.test.mjs`'s frozen +`dir` case. That matters because deleting `apply.mjs`'s `assertAllWritable` once left the +entire suite green. + +The reasons are per-tree and are in [`docs/README.md`](../../../docs/README.md). The one +worth repeating: `docs/knowledge/harvested/` **cannot** absorb a hand edit, because a +`` sha digests the whole source file rather than the entry — an edit inside an entry +changes no sha, and the next harvest regenerates or duplicates it with nothing to notice. +A finding about a harvested rule goes in `docs/knowledge/notes/`, by hand, by a human. + +## Where the rules come from + +Not from this skill's opinion. The documentation rules are the corpus's: + +```bash +bun run knowledge --topic documentation # 21 harvested styleguide rules +``` + +The four this skill mechanises: + +- `documentation.md:28` — every publishable package ships a README whose top gets a new + engineer from zero to one working call, without reading source, in about 30 seconds. +- `documentation.md:32` — each fact in exactly one authoritative place, linked from + everywhere else. This is why `docs/sdk-documentation/` does not restate the API report or + the TSDoc, and why the probe checks links rather than duplicating content. +- `documentation.md:34` — a comment that no longer matches the code is updated or deleted in + the same commit as the change that staled it, never deferred. +- `documentation.md:50` — the documentation build typechecks the code fences, so worked + examples cannot silently drift. That is `check-fences.mjs`. + +## Its own tests + +```bash +node --test .claude/skills/housekeeping/*.test.mjs +``` + +Seventy-seven cases across the guard, the probe, the apply stage and the fence check. Each +check has a **pair**: a throwaway fixture tree it reports clean over, and a mutation of that +tree it must fire on — the shape `scripts/verify-seam-1.test.mjs:6`, +`verify-knowledge-structure.test.mjs:4` and `verify-test-partition.test.mjs:4` already use +here. An earlier version asserted only that the live tree was clean, and replacing the +bodies of seven of the eight checks with `return;` left it fully green. They are +**not** in `bun run test:scripts`, which globs `scripts/*.test.mjs` — promoting them is a +one-line glob change, and the argument for it is the same one that made `test:scripts` +blocking in Phase 10: a gate whose own logic degrades still exits 0, so nothing else +notices. Tracked in `docs/work/mvp/2026-09-04-open-items-dissolution.md` U5. No count is written here on purpose: +`node --test .claude/skills/housekeeping/*.test.mjs` reports it. + +## Structure + +``` +.claude/skills/housekeeping/ + SKILL.md this file + fixture.mjs builds the throwaway repositories the tests probe + guard.mjs the frozen-path guard; both write sites go through it + guard.test.mjs 13 cases: prefix, traversal, absolute, symlink + probe.mjs stage 1 — eight read-only checks + probe.test.mjs 38 cases; every check has a fixture that must fire + apply.mjs stage 2 — git mv only, guarded, dry by default + apply.test.mjs 17 cases; the CLI half spawns the real script + check-fences.mjs typechecks the documentation's code fences against dist/ + check-fences.test.mjs 9 cases over the fence classifier +``` diff --git a/.claude/skills/housekeeping/apply.mjs b/.claude/skills/housekeeping/apply.mjs new file mode 100644 index 0000000..3e61f65 --- /dev/null +++ b/.claude/skills/housekeeping/apply.mjs @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/apply.mjs +// +// The only stage that writes. It does exactly one mechanical thing — drain the +// `docs/superpowers/` inbox into `docs/work//phaseN/` with `git mv`, so history +// follows each file — and refuses everything else. +// +// node .claude/skills/housekeeping/apply.mjs # dry run, prints the plan +// node .claude/skills/housekeeping/apply.mjs --write # performs it +// node .claude/skills/housekeeping/apply.mjs --write --delivery=v2 +// node .claude/skills/housekeeping/apply.mjs --root=/fixture # for the tests +// +// Everything the probe reports that is NOT a file move — a stale count in `CLAUDE.md`, a +// missing package README, a broken link — is prose, and prose is edited by whoever ran the +// probe. A tool that rewrites a sentence to make its own check pass is how documentation +// becomes true and useless at the same time. + +import {existsSync, mkdirSync} from 'node:fs'; +import {execFileSync} from 'node:child_process'; +import {dirname, join, posix} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {assertAllWritable} from './guard.mjs'; + +function resolveRepoRoot() { + const here = dirname(fileURLToPath(import.meta.url)); + return execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd: here, + encoding: 'utf8', + }).trim(); +} + +function git(root, ...args) { + return execFileSync('git', ['-c', 'core.quotePath=false', ...args], { + cwd: root, + encoding: 'utf8', + }); +} + +/** + * Where a phase document belongs, from its own name. + * + * `2026-07-28-phase8a-transport-design.md` → `phase8/phase8a/`. A file naming a whole phase + * with no sub-phase letter — a segmentation design, a shared checklist — sits at the + * `phaseN/` level. A file naming no phase at all sits directly under the delivery. + * + * Exported for the tests: the mapping is the part of this stage that can be wrong quietly. + */ +export function targetDirectory(filename, delivery = 'mvp') { + const base = posix.basename(filename); + const sub = /-phase(\d+)([a-z])-/.exec(base); + if (sub) + return `docs/work/${delivery}/phase${sub[1]}/phase${sub[1]}${sub[2]}`; + const whole = /-phase(\d+)[-.]/.exec(base); + if (whole) return `docs/work/${delivery}/phase${whole[1]}`; + if (/-scaffold-milestone/.test(base)) return `docs/work/${delivery}/scaffold`; + return `docs/work/${delivery}`; +} + +/** + * The inbox, tracked and untracked alike. + * + * `--others --exclude-standard` is the point: the inbox's NORMAL state is a file + * `brainstorming` has just written and nobody has staged. A tracked-only listing reported + * "the inbox is empty" over exactly the case this stage exists for. + */ +function inboxFiles(root) { + return git( + root, + 'ls-files', + '--cached', + '--others', + '--exclude-standard', + '--', + 'docs/superpowers/**', + ) + .trim() + .split('\n') + .filter(Boolean) + .filter(f => posix.basename(f) !== 'README.md'); +} + +/** Which of `files` git does not track yet. `git mv` cannot move those. */ +function untrackedAmong(root, files) { + if (files.length === 0) return []; + const cached = new Set( + git(root, 'ls-files', '--cached', '--', 'docs/superpowers/**') + .trim() + .split('\n') + .filter(Boolean), + ); + return files.filter(f => !cached.has(f)); +} + +/** The moves this run would perform, unperformed. */ +export function plan(delivery = 'mvp', root = resolveRepoRoot()) { + return inboxFiles(root).map(from => ({ + from, + to: posix.join(targetDirectory(from, delivery), posix.basename(from)), + })); +} + +/** + * Every reason this batch cannot be performed, as messages. + * + * Two collision classes, not one. `existsSync` catches a target already in the tree; the + * `seen` map catches two inbox files that land on the SAME target — which is the likely + * one, because `specs/` and `plans/` are the two directories the inbox actually uses and a + * design and its plan can share a basename. Without it the dry run printed "2 move(s) + * planned" and `--write` performed the first, then died on `git mv: destination exists` + * with an uncaught stack and a half-applied index. + * + * Exported so the tests can drive it without moving anything. + */ +export function batchRefusals(moves, root) { + const refusals = []; + const seen = new Map(); + for (const move of moves) { + if (seen.has(move.to)) { + refusals.push( + `two inbox files collide on ${move.to}: ${seen.get(move.to)} and ${move.from}. ` + + 'Rename one before collecting; the date prefix is what usually differs.', + ); + continue; + } + seen.set(move.to, move.from); + if (existsSync(join(root, move.to))) { + refusals.push(`${move.to} already exists (from ${move.from})`); + } + } + return refusals; +} + +function main(argv) { + const write = argv.includes('--write'); + const deliveryArg = argv.find(a => a.startsWith('--delivery=')); + const delivery = deliveryArg?.slice('--delivery='.length) ?? 'mvp'; + const rootArg = argv.find(a => a.startsWith('--root=')); + const root = rootArg?.slice('--root='.length) ?? resolveRepoRoot(); + + const moves = plan(delivery, root); + if (moves.length === 0) { + process.stdout.write('the inbox is empty; nothing to collect.\n'); + return 0; + } + + // Guard the WHOLE batch before performing any of it, so a refusal cannot leave the tree + // between two states. `--delivery=../product-spec` is what this stops. + assertAllWritable( + moves.flatMap(m => [m.from, m.to]), + root, + ); + + const untracked = untrackedAmong( + root, + moves.map(m => m.from), + ); + if (untracked.length > 0) { + for (const file of untracked) { + process.stderr.write( + `refusing: ${file} is not tracked; \`git mv\` cannot move it\n`, + ); + } + process.stderr.write( + `run \`git add ${untracked.join(' ')}\` first, then re-run. A phase document is worth ` + + 'a commit of its own before it moves, so history follows it across the collection.\n', + ); + return 1; + } + + const refusals = batchRefusals(moves, root); + if (refusals.length > 0) { + for (const message of refusals) + process.stderr.write(`refusing: ${message}\n`); + return 1; + } + + const done = []; + try { + for (const {from, to} of moves) { + process.stdout.write( + `${write ? 'git mv' : ' would move'} ${from} -> ${to}\n`, + ); + if (!write) continue; + mkdirSync(join(root, dirname(to)), {recursive: true}); + git(root, 'mv', from, to); + done.push(`${from} -> ${to}`); + } + } catch (error) { + // A mid-batch failure must say how far it got. Without this the operator is left with a + // raw stack and an index in an unknown state. + process.stderr.write( + `\n${String(done.length)} of ${String(moves.length)} move(s) were performed before ` + + 'this failed:\n', + ); + for (const line of done) process.stderr.write(` ${line}\n`); + process.stderr.write( + `\n${error instanceof Error ? error.message : String(error)}\n` + + 'The tree is half-collected. `git status` shows the completed moves; finish or ' + + 'revert them before re-running.\n', + ); + return 1; + } + + if (!write) { + process.stdout.write( + `\n${String(moves.length)} move(s) planned. Re-run with --write to perform them.\n`, + ); + return 0; + } + + process.stdout.write( + `\n${String(moves.length)} file(s) collected. Two things this stage did NOT do:\n` + + " 1. Repoint references to the old paths. Run the probe's link and citation checks,\n" + + ' then fix what they report — in the same commit, per documentation.md:34.\n' + + ' 2. Commit. A migration is its own commit, git mv only, so `git log --follow` works.\n', + ); + return 0; +} + +if (import.meta.url === `file://${process.argv[1]}`) { + process.exitCode = main(process.argv.slice(2)); +} diff --git a/.claude/skills/housekeeping/apply.test.mjs b/.claude/skills/housekeeping/apply.test.mjs new file mode 100644 index 0000000..3b215da --- /dev/null +++ b/.claude/skills/housekeeping/apply.test.mjs @@ -0,0 +1,351 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/apply.test.mjs +// +// Two halves, for the two ways this stage goes wrong. +// +// `targetDirectory` decides where a phase document lands, and getting it wrong is quiet: +// the file moves, `git log --follow` still works, and it is simply in the wrong place. The +// cases below are every shape the 62-file migration of 2026-08-31 actually produced. +// +// The rest spawns the real script against throwaway fixture trees, following +// `scripts/verify-seam-1.test.mjs:6` — because a suite that only calls the exported helpers +// passes just as happily when the CLI has stopped refusing anything, and that is exactly +// what happened: deleting `assertAllWritable` and its import, the guard's only production +// call site, left the whole suite green. + +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {spawnSync} from 'node:child_process'; +import {existsSync, mkdirSync, writeFileSync} from 'node:fs'; +import {dirname, join} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {batchRefusals, plan, targetDirectory} from './apply.mjs'; +import {makeFixture, removeFixture} from './fixture.mjs'; + +const SCRIPT = fileURLToPath(new URL('./apply.mjs', import.meta.url)); + +function run(root, ...args) { + return spawnSync(process.execPath, [SCRIPT, `--root=${root}`, ...args], { + encoding: 'utf8', + }); +} + +function inbox(root, path, text = '# doc\n') { + mkdirSync(join(root, dirname(path)), {recursive: true}); + writeFileSync(join(root, path), text); +} + +function stage(root, ...paths) { + spawnSync('git', ['add', ...paths], {cwd: root, encoding: 'utf8'}); +} + +// --- the mapping -------------------------------------------------------------------------- + +test('a sub-phase document nests under its phase', () => { + const cases = [ + ['2026-07-28-phase8a-transport-design.md', 'docs/work/mvp/phase8/phase8a'], + ['2026-07-28-phase8b-async-runtime.md', 'docs/work/mvp/phase8/phase8b'], + [ + '2026-07-24-phase3a-io-contracts-checklist.md', + 'docs/work/mvp/phase3/phase3a', + ], + ['2026-07-26-phase5c-auth.md', 'docs/work/mvp/phase5/phase5c'], + [ + 'docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md', + 'docs/work/mvp/phase4/phase4b', + ], + ]; + for (const [name, expected] of cases) { + assert.equal(targetDirectory(name), expected, name); + } +}); + +test('a whole-phase document sits at the phase level, not inside a sub-phase', () => { + const cases = [ + ['2026-07-28-phase8-segmentation-design.md', 'docs/work/mvp/phase8'], + [ + '2026-07-26-phase4-execution-context-and-pipelines-checklist.md', + 'docs/work/mvp/phase4', + ], + ['2026-07-23-phase1-core-http-domain-model.md', 'docs/work/mvp/phase1'], + ['2026-07-28-phase10-deviation-reconciliation.md', 'docs/work/mvp/phase10'], + ]; + for (const [name, expected] of cases) { + assert.equal(targetDirectory(name), expected, name); + } +}); + +test('phase10 is not read as phase1', () => { + // The reason the whole-phase pattern anchors on `[-.]` after the digits: `\d+` is greedy, + // but a lazier reading of `-phase1` inside `-phase10-` would file ten phases under one. + assert.equal( + targetDirectory('2026-07-28-phase10-deviation-reconciliation-design.md'), + 'docs/work/mvp/phase10', + ); + assert.notEqual( + targetDirectory('2026-07-28-phase10-x.md'), + 'docs/work/mvp/phase1', + ); +}); + +test('the scaffold milestone gets its own directory', () => { + for (const name of [ + '2026-07-23-scaffold-milestone.md', + '2026-07-23-scaffold-milestone-design.md', + '2026-07-23-scaffold-milestone-checklist.md', + ]) { + assert.equal(targetDirectory(name), 'docs/work/mvp/scaffold', name); + } +}); + +test('a document belonging to no phase sits directly under the delivery', () => { + assert.equal( + targetDirectory('2026-07-23-nodejs-sdk-v1-roadmap-design.md'), + 'docs/work/mvp', + ); + assert.equal( + targetDirectory('2026-07-25-checkpoint-scaffold-through-phase3a.md'), + 'docs/work/mvp', + ); +}); + +test('the delivery is a parameter, so a later effort is a sibling of mvp', () => { + assert.equal( + targetDirectory('2026-09-01-phase1-x-design.md', 'v2'), + 'docs/work/v2/phase1', + ); + assert.equal( + targetDirectory('2026-09-01-phase1a-x-design.md', 'v2'), + 'docs/work/v2/phase1/phase1a', + ); + assert.equal(targetDirectory('2026-09-01-roadmap.md', 'v2'), 'docs/work/v2'); +}); + +// --- batch refusals --------------------------------------------------------------------- + +test('batchRefusals catches two inbox files landing on ONE target', () => { + const root = makeFixture(); + try { + const moves = [ + { + from: 'docs/superpowers/specs/2026-09-01-phase1-x.md', + to: 'docs/work/mvp/phase1/2026-09-01-phase1-x.md', + }, + { + from: 'docs/superpowers/plans/2026-09-01-phase1-x.md', + to: 'docs/work/mvp/phase1/2026-09-01-phase1-x.md', + }, + ]; + const refusals = batchRefusals(moves, root); + assert.equal(refusals.length, 1, JSON.stringify(refusals)); + assert.match(refusals[0], /two inbox files collide on/); + assert.match( + refusals[0], + /specs\/2026-09-01-phase1-x\.md and .*plans\/2026-09-01-phase1-x\.md/, + ); + } finally { + removeFixture(root); + } +}); + +test('batchRefusals catches a target already in the tree', () => { + const root = makeFixture({ + overrides: { + 'docs/work/mvp/phase1/2026-09-01-phase1-x.md': '# already here\n', + }, + }); + try { + const refusals = batchRefusals( + [ + { + from: 'docs/superpowers/specs/2026-09-01-phase1-x.md', + to: 'docs/work/mvp/phase1/2026-09-01-phase1-x.md', + }, + ], + root, + ); + assert.equal(refusals.length, 1); + assert.match(refusals[0], /already exists/); + } finally { + removeFixture(root); + } +}); + +// --- the CLI ------------------------------------------------------------------------------ + +test('an empty inbox collects nothing', () => { + const root = makeFixture(); + try { + const {status, stdout} = run(root); + assert.equal(status, 0); + assert.match(stdout, /the inbox is empty/); + } finally { + removeFixture(root); + } +}); + +test('a dry run plans without moving', () => { + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + stage(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + const {status, stdout} = run(root); + assert.equal(status, 0); + assert.match( + stdout, + /would move .*phase11-thing-design\.md -> docs\/work\/mvp\/phase11\//, + ); + assert.ok( + existsSync( + join(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'), + ), + 'a dry run must not move anything', + ); + } finally { + removeFixture(root); + } +}); + +test('--write performs the move with git mv', () => { + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + stage(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + const {status, stdout} = run(root, '--write'); + assert.equal(status, 0, stdout); + assert.ok( + existsSync( + join(root, 'docs/work/mvp/phase11/2026-09-01-phase11-thing-design.md'), + ), + ); + assert.ok( + !existsSync( + join(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'), + ), + ); + assert.match(stdout, /did NOT do/); + } finally { + removeFixture(root); + } +}); + +test('an UNTRACKED inbox file is refused with the git add to run', () => { + // `git mv` cannot move what git does not track, and the inbox's normal state is exactly + // that. The old code neither reported nor refused it — `plan()` never saw the file. + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + const {status, stderr} = run(root, '--write'); + assert.equal(status, 1); + assert.match(stderr, /is not tracked; `git mv` cannot move it/); + assert.match( + stderr, + /run `git add docs\/superpowers\/specs\/2026-09-01-phase11-thing-design\.md`/, + ); + assert.ok( + existsSync( + join(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'), + ), + ); + } finally { + removeFixture(root); + } +}); + +test('two same-basename inbox files are refused BEFORE anything moves', () => { + // Reproduced end to end before the fix: the dry run printed "2 move(s) planned", `--write` + // performed the first, then `git mv` fatalled with an uncaught stack and a half-applied + // index. `specs/` and `plans/` are the two directories the inbox actually uses. + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing.md'); + inbox(root, 'docs/superpowers/plans/2026-09-01-phase11-thing.md'); + stage( + root, + 'docs/superpowers/specs/2026-09-01-phase11-thing.md', + 'docs/superpowers/plans/2026-09-01-phase11-thing.md', + ); + + const {status, stderr} = run(root, '--write'); + assert.equal(status, 1, 'the batch must be refused'); + assert.match(stderr, /two inbox files collide on/); + assert.ok( + existsSync( + join(root, 'docs/superpowers/specs/2026-09-01-phase11-thing.md'), + ) && + existsSync( + join(root, 'docs/superpowers/plans/2026-09-01-phase11-thing.md'), + ), + 'nothing may move when the batch is refused', + ); + assert.ok( + !existsSync(join(root, 'docs/work/mvp/phase11')), + 'no target may be created', + ); + } finally { + removeFixture(root); + } +}); + +test('a delivery that escapes into a frozen tree is refused by the guard', () => { + // The guard's only production call site. Deleting it left every other test green. + const root = makeFixture({ + overrides: {'docs/product-spec/04-core.md': '# normative\n'}, + }); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + stage(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + + const {status, stderr} = run(root, '--write', '--delivery=../product-spec'); + assert.notEqual(status, 0, 'a frozen destination must not be written'); + assert.match(stderr, /FrozenPathError/); + assert.match(stderr, /refusing to write/); + assert.ok( + existsSync( + join(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'), + ), + 'nothing may move when the guard refuses', + ); + } finally { + removeFixture(root); + } +}); + +// --- plan() ------------------------------------------------------------------------------- + +test('the plan keeps the filename, date prefix included', () => { + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + inbox(root, 'docs/superpowers/plans/2026-09-01-roadmap-v2.md'); + stage(root, 'docs/superpowers'); + + const moves = plan('mvp', root); + assert.equal(moves.length, 2, JSON.stringify(moves)); + for (const {from, to} of moves) { + assert.equal(to.split('/').pop(), from.split('/').pop(), from); + assert.ok(to.startsWith('docs/work/mvp/'), to); + } + } finally { + removeFixture(root); + } +}); + +test('plan() sees an UNTRACKED inbox file', () => { + const root = makeFixture(); + try { + inbox(root, 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md'); + assert.equal(plan('mvp', root).length, 1); + } finally { + removeFixture(root); + } +}); + +test('the inbox README is never collected', () => { + const root = makeFixture(); + try { + assert.deepEqual(plan('mvp', root), []); + } finally { + removeFixture(root); + } +}); diff --git a/.claude/skills/housekeeping/check-fences.mjs b/.claude/skills/housekeeping/check-fences.mjs new file mode 100644 index 0000000..d5628b3 --- /dev/null +++ b/.claude/skills/housekeeping/check-fences.mjs @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/check-fences.mjs +// +// Typechecks the code fences in the documentation against the BUILT packages. +// +// bun run build && node .claude/skills/housekeeping/check-fences.mjs +// +// The harvested styleguide asks for exactly this — "the documentation build typechecks the +// code fences inside `@example` tags so worked examples cannot silently drift out of sync +// with the API" (docs/knowledge/harvested/documentation.md:50). Nothing else in this +// repository does it: `api:ci` diffs signatures, `verify:consumer-types` compiles the +// emitted `.d.ts`, and neither reads a README. Two shipped transport READMEs opened with a +// sample that had not compiled since Phase 10, which is why this exists. +// +// Two kinds of fence are skipped, and the rules are deliberate: +// +// - **No `@dexpace/*` specifier** — an illustrative fragment: an interface quote, an +// expression sample. Compiling it in isolation would prove nothing. +// - **A relative import** — package-local. It only means anything from inside the +// package it documents, and cannot resolve from a scratch directory. +// +// The first rule tests for the SPECIFIER, not for an `import` line carrying it. A +// single-line `/^import .*'@dexpace\//m` silently reclassified every fence whose import +// list wraps — which was the documentation's four largest worked examples, `write-a-transport.md`, +// `errors.md`, `write-a-paging-strategy.md` and `write-a-serde.md`. Breaking one of them +// still printed PASS. +// +// An import of an uninstalled OPTIONAL peer (`pino`, `debug`) or of a schema library named +// only as an illustration (`zod`) is reported and not counted as a failure: those packages +// are legitimately absent from this workspace. + +import {mkdirSync, readFileSync, rmSync, writeFileSync} from 'node:fs'; +import {execFileSync} from 'node:child_process'; +import {basename, dirname, join} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {assertWritable} from './guard.mjs'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const ROOT = execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd: HERE, + encoding: 'utf8', +}).trim(); + +const SCRATCH = '.housekeeping-fences'; +const FENCE = /```(?:typescript|ts)\n([\s\S]*?)```/g; +const ABSENT_MODULES = ['pino', 'debug', 'zod']; + +const DEFAULT_FILES = () => + execFileSync( + 'git', + [ + 'ls-files', + '--', + 'README.md', + 'docs/sdk-documentation/*.md', + 'packages/*/README.md', + ], + {cwd: ROOT, encoding: 'utf8'}, + ) + .trim() + .split('\n') + .filter(Boolean); + +/** + * Extracts the runnable fences into `dir`, returning one entry per written file. + * + * `root` is a parameter so `check-fences.test.mjs` can drive this over a throwaway tree + * rather than over the repository it is testing. + */ +export function extract(dir, files, root = ROOT) { + // This function opens by DELETING `dir`. `main` only ever passes the module constant, so + // no caller reaches it with anything else today — but an exported function whose first + // act is a recursive remove must not be one guard away from eating a normative tree, and + // the tests below pass a `dir` of their own. + assertWritable(dir, root); + rmSync(join(root, dir), {recursive: true, force: true}); + mkdirSync(join(root, dir), {recursive: true}); + + const written = []; + for (const file of files) { + const text = readFileSync(join(root, file), 'utf8'); + let index = 0; + for (const match of text.matchAll(FENCE)) { + index++; + const code = match[1]; + if (!/'@dexpace\//.test(code)) continue; + if (/from '\.\.?\//.test(code)) continue; + const line = text.slice(0, match.index).split('\n').length; + const name = + `${basename(dirname(file))}-${basename(file, '.md')}-${index}.ts` + .replace(/[^\w.-]/g, '_') + .replace(/^[.-]+/, 'root-'); + writeFileSync(join(root, dir, name), code); + written.push({ + name, + from: `${file}:${line}`, + importLines: importLines(code), + }); + } + } + + writeFileSync( + join(root, dir, 'tsconfig.json'), + `${JSON.stringify( + { + compilerOptions: { + module: 'NodeNext', + moduleResolution: 'NodeNext', + target: 'ES2022', + lib: ['ES2022', 'DOM', 'DOM.AsyncIterable'], + strict: true, + // On for the IMPORT diagnostics only — see `importDiagnostics`. An unused import + // in a worked example is a defect: it tells a reader they need a symbol they do + // not. An unused *local* is not: `const body = serdeBody(value, serde);` exists + // to show the shape of what comes back, and has nothing to do afterwards. + noUnusedLocals: true, + noEmit: true, + skipLibCheck: true, + types: [], + }, + include: ['*.ts'], + }, + null, + 1, + )}\n`, + ); + return written; +} + +/** `1`-based line numbers of every physical line inside an import declaration. */ +function importLines(code) { + const lines = code.split('\n'); + const inside = new Set(); + let open = false; + for (const [index, line] of lines.entries()) { + if (open || /^\s*import\b/.test(line)) { + inside.add(index + 1); + // A declaration ends at the `;`, or at the `from '…'` for a braceless one. + open = !/;\s*$/.test(line); + } + } + return inside; +} + +const UNUSED = /^(.+?)\((\d+),\d+\): error TS(6133|6192):/; + +/** + * Is this diagnostic about an unused LOCAL rather than an unused import? + * + * `noUnusedLocals` covers both and TypeScript has no flag that separates them, so the split + * happens here: TS6192 is always an import declaration; TS6133 is one only when the symbol + * it names sits on a line inside one. + */ +function isUnusedLocal(line, written) { + const match = UNUSED.exec(line); + if (match === null) return false; + if (match[3] === '6192') return false; // "All imports in import declaration are unused" + const entry = written.find(w => match[1].endsWith(w.name)); + if (entry === undefined) return false; + return !entry.importLines.has(Number(match[2])); +} + +function main() { + const files = process.argv.slice(2).filter(a => !a.startsWith('--')); + const targets = files.length > 0 ? files : DEFAULT_FILES(); + + // `finally`, so an interrupted or throwing run does not leave `.housekeeping-fences/` + // behind. `.gitignore` hides it either way, which is why this ranks where it does. + try { + const written = extract(SCRATCH, targets); + + let output = ''; + try { + execFileSync( + './node_modules/.bin/tsc', + ['-p', `${SCRATCH}/tsconfig.json`], + { + cwd: ROOT, + encoding: 'utf8', + }, + ); + } catch (error) { + output = `${error.stdout ?? ''}${error.stderr ?? ''}`; + } + + const lines = output.split('\n').filter(Boolean); + const absent = lines.filter(l => + ABSENT_MODULES.some(m => l.includes(`TS2307: Cannot find module '${m}'`)), + ); + const unusedLocal = lines.filter(l => isUnusedLocal(l, written)); + const real = lines.filter( + l => !absent.includes(l) && !unusedLocal.includes(l), + ); + + process.stdout.write( + `${String(written.length)} runnable fence(s) from ${String(targets.length)} file(s)\n`, + ); + for (const line of real) process.stdout.write(`${line}\n`); + if (absent.length > 0) { + process.stdout.write( + `\n${String(absent.length)} import(s) of a package absent from this workspace, ` + + `ignored: ${ABSENT_MODULES.join(', ')}\n`, + ); + } + if (unusedLocal.length > 0) { + process.stdout.write( + `${String(unusedLocal.length)} unused local(s), ignored: a worked example may bind a ` + + 'value to show its shape. Unused IMPORTS are still failures.\n', + ); + } + + if (real.length > 0) { + process.stdout.write('\nFENCE CHECK: FAIL\n'); + return 1; + } + process.stdout.write('\nFENCE CHECK: PASS\n'); + return 0; + } finally { + rmSync(join(ROOT, SCRATCH), {recursive: true, force: true}); + } +} + +if (import.meta.url === `file://${process.argv[1]}`) { + process.exitCode = main(); +} diff --git a/.claude/skills/housekeeping/check-fences.test.mjs b/.claude/skills/housekeeping/check-fences.test.mjs new file mode 100644 index 0000000..464b6c8 --- /dev/null +++ b/.claude/skills/housekeeping/check-fences.test.mjs @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/check-fences.test.mjs +// +// The classifier decides which fences are checked at all, so getting it wrong is silent by +// construction: the run reports PASS over the examples it skipped. It did. A single-line +// `/^import .*'@dexpace\//m` reclassified every fence whose import list wraps — the +// documentation's four largest worked examples — and breaking one of them still printed +// PASS with exit 0. + +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import { + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {dirname, join} from 'node:path'; +import {extract} from './check-fences.mjs'; +import {FrozenPathError} from './guard.mjs'; + +function withTree(files, body) { + const root = mkdtempSync(join(tmpdir(), 'fences-')); + try { + for (const [path, text] of Object.entries(files)) { + mkdirSync(join(root, dirname(path)), {recursive: true}); + writeFileSync(join(root, path), text); + } + return body(root); + } finally { + rmSync(root, {recursive: true, force: true}); + } +} + +const WRAPPED = `# doc + +\`\`\`typescript +import { + Request, + Response, + type Transport, +} from '@dexpace/core'; + +export const t: Transport = null as never; +\`\`\` +`; + +const SINGLE_LINE = `# doc + +\`\`\`typescript +import {Request} from '@dexpace/core'; +\`\`\` +`; + +const FRAGMENT = `# doc + +\`\`\`typescript +interface Transport { + close(): Promise; +} +\`\`\` +`; + +const RELATIVE = `# doc + +\`\`\`typescript +import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; +import {myTransport} from '../src/index.js'; +\`\`\` +`; + +test('a fence whose import list WRAPS is runnable, not a fragment', () => { + withTree({'a.md': WRAPPED}, root => { + const written = extract('out', ['a.md'], root); + assert.equal(written.length, 1, 'a wrapped import must not be skipped'); + assert.match( + readFileSync(join(root, 'out', written[0].name), 'utf8'), + /@dexpace\/core/, + ); + }); +}); + +test('a single-line import is runnable', () => { + withTree({'a.md': SINGLE_LINE}, root => { + assert.equal(extract('out', ['a.md'], root).length, 1); + }); +}); + +test('a fence with no @dexpace specifier is an illustrative fragment', () => { + withTree({'a.md': FRAGMENT}, root => { + assert.deepEqual(extract('out', ['a.md'], root), []); + }); +}); + +test('a fence importing a RELATIVE path is package-local and skipped', () => { + withTree({'a.md': RELATIVE}, root => { + assert.deepEqual(extract('out', ['a.md'], root), []); + }); +}); + +test('every written entry records where it came from, with a line number', () => { + withTree({'docs/a.md': `intro\n\n${SINGLE_LINE}`}, root => { + const [entry] = extract('out', ['docs/a.md'], root); + assert.equal(entry.from, 'docs/a.md:5'); + assert.ok(entry.name.endsWith('.ts')); + }); +}); + +test('a snippet from a root-level file gets a non-dotfile name', () => { + // `basename(dirname('README.md'))` is `.`, and tsconfig's `include: ["*.ts"]` does not + // match a dotfile — the whole run reported "No inputs were found". + withTree({'README.md': SINGLE_LINE}, root => { + const [entry] = extract('out', ['README.md'], root); + assert.ok(!entry.name.startsWith('.'), entry.name); + assert.match(entry.name, /^root-README-1\.ts$/); + }); +}); + +test('the generated tsconfig turns on the flags the check depends on', () => { + withTree({'a.md': SINGLE_LINE}, root => { + extract('out', ['a.md'], root); + const config = JSON.parse( + readFileSync(join(root, 'out', 'tsconfig.json'), 'utf8'), + ); + assert.equal(config.compilerOptions.strict, true); + assert.equal(config.compilerOptions.noUnusedLocals, true); + assert.equal(config.compilerOptions.module, 'NodeNext'); + assert.deepEqual(config.compilerOptions.types, []); + }); +}); + +test('importLines marks a wrapped declaration, so an unused LOCAL is told from an unused IMPORT', () => { + withTree({'a.md': WRAPPED}, root => { + const [entry] = extract('out', ['a.md'], root); + // The declaration spans lines 1-5; the binding on 7 is a local. + assert.deepEqual( + [...entry.importLines].sort((a, b) => a - b), + [1, 2, 3, 4, 5], + ); + assert.ok(!entry.importLines.has(7)); + }); +}); + +test('extract refuses a frozen output directory', () => { + // It opens by deleting `dir`. No caller passes anything but the module constant today — + // these tests are the first to pass a `dir` at all, which is why the guard is here. + withTree({'a.md': SINGLE_LINE}, root => { + assert.throws( + () => extract('docs/product-spec', ['a.md'], root), + FrozenPathError, + ); + assert.throws( + () => extract('docs/knowledge/harvested', ['a.md'], root), + FrozenPathError, + ); + }); +}); diff --git a/.claude/skills/housekeeping/fixture.mjs b/.claude/skills/housekeeping/fixture.mjs new file mode 100644 index 0000000..81cd11d --- /dev/null +++ b/.claude/skills/housekeeping/fixture.mjs @@ -0,0 +1,156 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/fixture.mjs +// +// Builds a throwaway repository the probe and the apply stage can be pointed at, so a test +// can assert a check FIRES rather than asserting the live tree happens to be clean. +// +// The distinction is not academic. Before these fixtures existed, replacing the bodies of +// seven of the eight checks with `return;` left the whole suite green, and so did deleting +// `apply.mjs`'s only `assertAllWritable` call. `scripts/verify-seam-1.test.mjs:6`, +// `verify-knowledge-structure.test.mjs:4` and `verify-test-partition.test.mjs:4` had each +// already reached that conclusion in this repository and say so in their own headers. +// +// Only used by tests, so `node:fs` and `node:child_process` are fine here — the +// zero-`node:` invariant governs `packages/*/src`, not tooling. + +import {execFileSync} from 'node:child_process'; +import {mkdirSync, mkdtempSync, rmSync, writeFileSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {dirname, join} from 'node:path'; + +/** A CI workflow with a known shape: two jobs, three named steps. */ +const WORKFLOW = `name: CI + +on: + push: + branches: [main] + pull_request: + +jobs: + ci: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install + run: bun install + + - name: Verify seam + run: bun run verify:seam-1 + + node-conformance: + needs: ci + runs-on: ubuntu-latest + steps: + - name: Node conformance + run: bun run test:node +`; + +/** + * The counts a fixture's own documents must state to be clean: + * 2 packages (1 publishable, 1 private), 1 API report, 3 named CI steps, 2 jobs. + */ +export const CLEAN_CLAIMS = [ + 'Two packages, one is published and one is `private`.', + 'One committed report.', + 'Three named steps across two jobs.', + 'Gates: `verify:seam-1`.', + '`@dexpace/thing` and `@dexpace/secret` are the packages.', +].join('\n\n'); + +function write(root, path, text) { + mkdirSync(join(root, dirname(path)), {recursive: true}); + writeFileSync(join(root, path), text); +} + +/** + * A minimal repository the probe reports clean over. + * + * `overrides` replaces or adds files after the clean tree is written; a value of `null` + * deletes. `untracked` is written after `git add`, so it stays untracked. + */ +export function makeFixture({overrides = {}, untracked = {}} = {}) { + const root = mkdtempSync(join(tmpdir(), 'housekeeping-fixture-')); + + write( + root, + 'package.json', + `${JSON.stringify( + { + name: 'fixture', + private: true, + scripts: {'verify:seam-1': 'true', test: 'true'}, + }, + null, + 2, + )}\n`, + ); + write(root, '.github/workflows/ci.yml', WORKFLOW); + + write( + root, + 'packages/thing/package.json', + `${JSON.stringify( + {name: '@dexpace/thing', peerDependencies: {'@dexpace/core': '*'}}, + null, + 2, + )}\n`, + ); + write(root, 'packages/thing/etc/thing.api.md', '# API\n'); + write( + root, + 'packages/thing/README.md', + `# @dexpace/thing\n\n${'Long enough to clear the thin-README floor. '.repeat(25)}\n`, + ); + write( + root, + 'packages/secret/package.json', + `${JSON.stringify({name: '@dexpace/secret', private: true}, null, 2)}\n`, + ); + + write( + root, + 'CLAUDE.md', + `# CLAUDE.md\n\n${CLEAN_CLAIMS}\n\ndocs/README.md, docs/work, docs/sdk-documentation, docs/superpowers.\n`, + ); + write(root, 'README.md', `# fixture\n\n${CLEAN_CLAIMS}\n`); + write( + root, + 'docs/README.md', + '# docs\n\nEntries: README.md, work, sdk-documentation, superpowers.\n', + ); + // The register was dissolved on 2026-09-04 and archived here; item IDs stay reserved and still + // resolve, so this is where the citation check reads them from. + write( + root, + 'docs/work/mvp/2026-09-04-open-items-dissolution.md', + '# Open items — dissolved\n\n### A1 — a real item — **WATCH**\n\nBody.\n', + ); + write(root, 'docs/superpowers/README.md', '# inbox\n'); + write(root, 'docs/work/mvp/phase1/2026-01-01-phase1-thing.md', '# phase 1\n'); + write(root, 'docs/sdk-documentation/architecture.md', '# architecture\n'); + + for (const [path, text] of Object.entries(overrides)) { + if (text === null) { + rmSync(join(root, path), {force: true, recursive: true}); + continue; + } + write(root, path, text); + } + + execFileSync('git', ['init', '-q'], {cwd: root}); + execFileSync('git', ['config', 'user.email', 'fixture@example.invalid'], { + cwd: root, + }); + execFileSync('git', ['config', 'user.name', 'fixture'], {cwd: root}); + execFileSync('git', ['add', '-A'], {cwd: root}); + execFileSync('git', ['commit', '-qm', 'fixture'], {cwd: root}); + + for (const [path, text] of Object.entries(untracked)) write(root, path, text); + + return root; +} + +export function removeFixture(root) { + rmSync(root, {recursive: true, force: true}); +} diff --git a/.claude/skills/housekeeping/guard.mjs b/.claude/skills/housekeeping/guard.mjs new file mode 100644 index 0000000..b93d73a --- /dev/null +++ b/.claude/skills/housekeeping/guard.mjs @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/guard.mjs +// +// The frozen-path guard. Three trees and two files in `docs/` are read-only to this +// skill, and that has to be a check rather than a paragraph of good intent: the apply +// stage moves files and rewrites Markdown, and a glob that widens by one segment is +// exactly how a maintenance tool eats a normative document. +// +// Every write the skill performs goes through `assertWritable` first. `guard.test.mjs` +// is what proves it, including the three ways a naive prefix test gets it wrong: a +// sibling whose name merely starts with a frozen one, a `..` segment that lands inside +// after normalization, and an absolute path. + +import {realpathSync} from 'node:fs'; +import {dirname, relative, resolve, sep} from 'node:path'; + +/** + * The five entries this skill must never write to. + * + * `docs/knowledge/` covers both `harvested/` and `notes/`. `harvested/` cannot absorb a + * hand edit at all — a `` sha digests the whole source file rather than the entry, so + * an edit inside one changes no sha and the next harvest regenerates or duplicates it + * silently. `notes/` is hand-written and could in principle be edited; it is frozen here + * because the CLI reads the two as one corpus and a note's key citation couples them. + */ +export const FROZEN = Object.freeze([ + 'docs/knowledge', + 'docs/product-spec', + 'docs/sdk-design-nodejs', + 'docs/product-spec.md', + 'docs/sdk-design-nodejs.md', +]); + +/** Raised instead of writing. Carries the offending path so a caller can report it. */ +export class FrozenPathError extends Error { + constructor(path, frozen) { + super( + `refusing to write ${path}: it is under the frozen entry '${frozen}'. ` + + 'The housekeeping skill reads the normative and harvested trees; it never ' + + 'writes to them. See docs/README.md.', + ); + this.name = 'FrozenPathError'; + this.path = path; + this.frozen = frozen; + } +} + +/** + * `path` with every symlink in it resolved, as far as the filesystem actually goes. + * + * `resolve()` is purely lexical, so on its own it answers the wrong question: with + * `docs/work` a symlink to `docs/product-spec`, `docs/work/mvp/x.md` lexically escapes the + * frozen tree and physically lands inside it — and `mkdirSync(…, {recursive: true})` + * follows the link, so a `git mv` would write there while the guard said yes. + * + * A target that does not exist yet is the normal case for a move, so this walks up to the + * nearest ancestor that does, resolves that, and re-attaches the tail. + */ +function realpathOfNearestAncestor(path) { + const segments = []; + let current = path; + for (;;) { + try { + return resolve(realpathSync.native(current), ...segments.reverse()); + } catch { + const parent = dirname(current); + // Root reached without anything existing: nothing to resolve, answer lexically. + if (parent === current) return path; + segments.push(current.slice(parent.length + 1)); + current = parent; + } + } +} + +/** + * Which frozen entry `candidate` falls under, or `null`. + * + * Resolved against `repoRoot` and compared **segment-wise**, never as a raw string + * prefix: `docs/product-spec-draft/x.md` starts with `docs/product-spec` as characters + * and is not under it as a path. `..` is normalized away first, so a path that spells its + * way in cannot spell its way past the check — and symlinks are resolved, so a path that + * *links* its way in cannot either. + */ +export function frozenEntryFor(candidate, repoRoot = process.cwd()) { + const root = resolve(repoRoot); + const target = realpathOfNearestAncestor(resolve(root, candidate)); + for (const entry of FROZEN) { + const frozenAbs = realpathOfNearestAncestor(resolve(root, entry)); + if (target === frozenAbs) return entry; + const rel = relative(frozenAbs, target); + // Inside iff the relative path neither escapes upward nor is absolute. + if ( + rel !== '' && + !rel.startsWith(`..${sep}`) && + rel !== '..' && + !rel.startsWith(sep) + ) { + return entry; + } + } + return null; +} + +/** `true` when `candidate` is a frozen entry or lives under one. */ +export function isFrozen(candidate, repoRoot = process.cwd()) { + return frozenEntryFor(candidate, repoRoot) !== null; +} + +/** + * Throws `FrozenPathError` when `candidate` is frozen; returns it otherwise, so a call + * site reads `writeFileSync(assertWritable(p), text)` and cannot forget the check. + */ +export function assertWritable(candidate, repoRoot = process.cwd()) { + const frozen = frozenEntryFor(candidate, repoRoot); + if (frozen !== null) throw new FrozenPathError(candidate, frozen); + return candidate; +} + +/** + * Guards a whole batch before performing any of it, so a run cannot half-apply and leave + * the tree between two states. + */ +export function assertAllWritable(candidates, repoRoot = process.cwd()) { + const refused = candidates + .map(path => ({path, frozen: frozenEntryFor(path, repoRoot)})) + .filter(({frozen}) => frozen !== null); + if (refused.length > 0) { + throw new FrozenPathError(refused[0].path, refused[0].frozen); + } + return candidates; +} diff --git a/.claude/skills/housekeeping/guard.test.mjs b/.claude/skills/housekeeping/guard.test.mjs new file mode 100644 index 0000000..f7749c8 --- /dev/null +++ b/.claude/skills/housekeeping/guard.test.mjs @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/guard.test.mjs +// +// The guard is the one part of this skill that must not be wrong, because everything it +// protects is a document no other copy of exists. These cases are the three ways a naive +// `startsWith` implementation fails, plus proof that the mutable half stays mutable. + +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {mkdirSync, mkdtempSync, rmSync, symlinkSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join, resolve} from 'node:path'; +import { + FROZEN, + FrozenPathError, + assertAllWritable, + assertWritable, + frozenEntryFor, + isFrozen, +} from './guard.mjs'; + +const ROOT = '/repo'; + +test('every frozen entry is itself refused', () => { + for (const entry of FROZEN) { + assert.equal(frozenEntryFor(entry, ROOT), entry, entry); + } +}); + +test('a file under a frozen tree is refused, at any depth', () => { + assert.equal( + frozenEntryFor('docs/product-spec/04-core.md', ROOT), + 'docs/product-spec', + ); + assert.equal( + frozenEntryFor('docs/knowledge/harvested/documentation.md', ROOT), + 'docs/knowledge', + ); + assert.equal( + frozenEntryFor('docs/knowledge/notes/pagination.md', ROOT), + 'docs/knowledge', + ); + assert.equal( + frozenEntryFor('docs/sdk-design-nodejs/10-deliberate-deviations.md', ROOT), + 'docs/sdk-design-nodejs', + ); +}); + +test('a SIBLING whose name merely starts with a frozen one is writable', () => { + // The failure a raw string prefix test would produce, and the reason the comparison is + // segment-wise. `verify-knowledge-structure.mjs` guards the same shape for source roots. + assert.equal(isFrozen('docs/product-spec-draft/04-core.md', ROOT), false); + assert.equal(isFrozen('docs/knowledge-notes.md', ROOT), false); + assert.equal(isFrozen('docs/sdk-design-nodejs-old/01.md', ROOT), false); + assert.equal(isFrozen('docs/product-spec.md.bak', ROOT), false); +}); + +test('a `..` segment that lands inside is refused', () => { + assert.equal( + frozenEntryFor('docs/work/../product-spec/04-core.md', ROOT), + 'docs/product-spec', + ); + assert.equal( + frozenEntryFor('docs/sdk-documentation/../knowledge/x.md', ROOT), + 'docs/knowledge', + ); +}); + +test('a `..` segment that escapes upward is not mistaken for containment', () => { + assert.equal( + isFrozen('docs/product-spec/../work/mvp/phase1/x.md', ROOT), + false, + ); + assert.equal(isFrozen('docs/knowledge/../README.md', ROOT), false); +}); + +test('an absolute path is resolved, not treated as relative', () => { + assert.equal( + frozenEntryFor(resolve(ROOT, 'docs/product-spec/04.md'), ROOT), + 'docs/product-spec', + ); + // An absolute path outside the repository is nobody's business but is certainly not frozen. + assert.equal(isFrozen('/elsewhere/docs/product-spec/04.md', ROOT), false); +}); + +test('everything the skill is allowed to write stays writable', () => { + for (const path of [ + 'docs/README.md', + 'docs/work/mvp/2026-09-04-open-items-dissolution.md', + 'docs/first-release.md', + 'docs/deviations.md', + 'docs/sdk-documentation/architecture.md', + 'docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model.md', + 'docs/superpowers/specs/2026-09-01-x-design.md', + 'docs/assets/dexpace-wordmark-dark.svg', + 'CLAUDE.md', + 'README.md', + 'packages/core/README.md', + ]) { + assert.equal(isFrozen(path, ROOT), false, path); + assert.equal(assertWritable(path, ROOT), path); + } +}); + +test('assertWritable throws a FrozenPathError naming both paths', () => { + assert.throws( + () => assertWritable('docs/product-spec/04-core.md', ROOT), + error => { + assert.ok(error instanceof FrozenPathError); + assert.equal(error.name, 'FrozenPathError'); + assert.equal(error.frozen, 'docs/product-spec'); + assert.match(error.message, /refusing to write/); + return true; + }, + ); +}); + +test('assertAllWritable refuses the whole batch before performing any of it', () => { + const batch = ['docs/README.md', 'docs/product-spec/04-core.md', 'CLAUDE.md']; + assert.throws(() => assertAllWritable(batch, ROOT), FrozenPathError); + assert.deepEqual(assertAllWritable(['docs/README.md', 'CLAUDE.md'], ROOT), [ + 'docs/README.md', + 'CLAUDE.md', + ]); +}); + +test('the frozen list is exactly the five docs/README.md names', () => { + // Pinned deliberately: widening this list is a decision about what a maintenance tool + // may edit, and it must be a reviewed diff here rather than a silent constant change. + assert.deepEqual( + [...FROZEN], + [ + 'docs/knowledge', + 'docs/product-spec', + 'docs/sdk-design-nodejs', + 'docs/product-spec.md', + 'docs/sdk-design-nodejs.md', + ], + ); +}); + +test('a SYMLINK into a frozen tree is refused', () => { + // Lexically `docs/work/...` escapes every frozen entry; physically it lands inside + // `docs/product-spec`. `apply.mjs`'s `mkdirSync(…, {recursive: true})` follows the link, + // so a purely lexical guard says yes and `git mv` writes into the normative tree. + const root = mkdtempSync(join(tmpdir(), 'guard-symlink-')); + try { + mkdirSync(join(root, 'docs/product-spec'), {recursive: true}); + symlinkSync( + join(root, 'docs/product-spec'), + join(root, 'docs/work'), + 'dir', + ); + + assert.equal( + frozenEntryFor('docs/work/mvp/phase9/x.md', root), + 'docs/product-spec', + 'a symlinked path into a frozen tree must be refused', + ); + assert.throws( + () => assertWritable('docs/work/mvp/phase9/x.md', root), + FrozenPathError, + ); + } finally { + rmSync(root, {recursive: true, force: true}); + } +}); + +test('a real docs/work directory stays writable', () => { + // The other half: resolving symlinks must not make the ordinary tree frozen. + const root = mkdtempSync(join(tmpdir(), 'guard-real-')); + try { + mkdirSync(join(root, 'docs/product-spec'), {recursive: true}); + mkdirSync(join(root, 'docs/work/mvp/phase9'), {recursive: true}); + assert.equal(isFrozen('docs/work/mvp/phase9/x.md', root), false); + assert.equal(isFrozen('docs/product-spec/04.md', root), true); + } finally { + rmSync(root, {recursive: true, force: true}); + } +}); + +test('a target whose ancestors do not exist yet is still judged', () => { + // The normal case for a move: nothing at the destination. + const root = mkdtempSync(join(tmpdir(), 'guard-absent-')); + try { + assert.equal( + frozenEntryFor('docs/product-spec/new/deep/x.md', root), + 'docs/product-spec', + ); + assert.equal(isFrozen('docs/work/mvp/phase1/x.md', root), false); + } finally { + rmSync(root, {recursive: true, force: true}); + } +}); diff --git a/.claude/skills/housekeeping/probe.mjs b/.claude/skills/housekeeping/probe.mjs new file mode 100644 index 0000000..3d936d8 --- /dev/null +++ b/.claude/skills/housekeeping/probe.mjs @@ -0,0 +1,894 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/probe.mjs +// +// Read-only. Reports drift; never writes. Eight checks, each of which caught something +// real the first time it ran (`docs/open-items.md` Section U). +// +// node .claude/skills/housekeeping/probe.mjs # report, exit 0 +// node .claude/skills/housekeeping/probe.mjs --strict # exit 1 when anything is found +// node .claude/skills/housekeeping/probe.mjs --only=links,claims +// node .claude/skills/housekeeping/probe.mjs --root=/path/to/a/fixture/tree +// +// The eight are deliberately independent: a repository fact is derived once, from the +// repository, and every document that states it is checked against that one derivation. +// Nothing here reads a number out of one document and compares it to another. +// +// `--root` exists for `probe.test.mjs`, which builds throwaway fixture trees and asserts +// each check FIRES — the shape `scripts/verify-seam-1.test.mjs:6` and +// `verify-test-partition.test.mjs:4` already use here. A suite that only asserts the live +// tree is clean passes just as happily over a check whose body has become `return;`, and +// seven of these eight were in exactly that state when it was written. + +import {existsSync, readFileSync, readdirSync, statSync} from 'node:fs'; +import {execFileSync} from 'node:child_process'; +import {dirname, join, posix} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {isFrozen} from './guard.mjs'; + +function resolveRepoRoot() { + const here = dirname(fileURLToPath(import.meta.url)); + return execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd: here, + encoding: 'utf8', + }).trim(); +} + +/** + * Everything a check needs, bound to one repository root. + * + * The root is a parameter rather than a module constant so a fixture tree can be probed; + * that is the only reason this indirection exists. + */ +function createContext(root) { + const findings = []; + return { + root, + findings, + read: path => readFileSync(join(root, path), 'utf8'), + exists: path => existsSync(join(root, path)), + /** + * Tracked paths matching `globs`. + * + * `-c core.quotePath=false` because `git ls-files` C-quotes any path with a non-ASCII + * byte by default (`"docs/caf\303\251.md"`), and a quoted path fed back to `readFileSync` + * is an `ENOENT` that takes the whole run down with a raw stack instead of a finding. + */ + tracked: (...globs) => + execFileSync( + 'git', + ['-c', 'core.quotePath=false', 'ls-files', '--', ...globs], + {cwd: root, encoding: 'utf8'}, + ) + .trim() + .split('\n') + .filter(Boolean) + // `ls-files` lists the INDEX, so a file deleted in the working tree and not yet staged is + // still listed. Every caller goes on to read what it gets back, and an `ENOENT` there takes + // the whole run down with a raw stack instead of producing a finding -- which is what + // happened the first time a check ran against a tree with an uncommitted deletion in it. + .filter(path => existsSync(join(root, path))), + /** Tracked paths PLUS untracked, non-ignored ones. */ + present: (...globs) => + execFileSync( + 'git', + [ + '-c', + 'core.quotePath=false', + 'ls-files', + '--others', + '--exclude-standard', + '--cached', + '--', + ...globs, + ], + {cwd: root, encoding: 'utf8'}, + ) + .trim() + .split('\n') + .filter(Boolean), + finding: (check, severity, message) => + findings.push({check, severity, message}), + }; +} + +// --------------------------------------------------------------------------------------- +// Spelled-out numerals. +// +// Every count claim in `CLAUDE.md` and `README.md` is written as an English word -- +// "eleven packages", "nine committed reports", "Twenty named CI steps". The first version +// of this file matched `(\d+)` only, so it protected exactly one sentence in the whole +// repository, and appending "Two published packages today, and that is the whole +// workspace." to `CLAUDE.md` -- the precise drift SKILL.md names as this tool's reason for +// existing -- still printed `no drift found`. +// --------------------------------------------------------------------------------------- + +const ONES = [ + 'zero', + 'one', + 'two', + 'three', + 'four', + 'five', + 'six', + 'seven', + 'eight', + 'nine', + 'ten', + 'eleven', + 'twelve', + 'thirteen', + 'fourteen', + 'fifteen', + 'sixteen', + 'seventeen', + 'eighteen', + 'nineteen', +]; +const TENS = { + twenty: 20, + thirty: 30, + forty: 40, + fifty: 50, + sixty: 60, + seventy: 70, + eighty: 80, + ninety: 90, +}; + +/** The number `token` denotes, or `null` when it is not a numeral at all. */ +export function parseNumeral(token) { + const word = token.toLowerCase(); + if (/^\d+$/.test(word)) return Number(word); + const ones = ONES.indexOf(word); + if (ones !== -1) return ones; + if (Object.hasOwn(TENS, word)) return TENS[word]; + const compound = /^([a-z]+)-([a-z]+)$/.exec(word); + if (compound && Object.hasOwn(TENS, compound[1])) { + const unit = ONES.indexOf(compound[2]); + if (unit > 0 && unit < 10) return TENS[compound[1]] + unit; + } + return null; +} + +/** A digit run or a word that might be a numeral; `parseNumeral` decides which. */ +const NUMBER = String.raw`(\d+|[A-Za-z]+(?:-[A-Za-z]+)?)`; + +function numberedPattern(tail) { + return new RegExp(`${NUMBER}\\s+${tail}`, 'gi'); +} + +/** + * The count claims the two documents make, each anchored on its own subject. + * + * `required` is the point. A count that is merely *checked when found* is not checked at + * all: deleting the sentence passes, and so does rewording it past the pattern. Every row + * here must appear in each document that lists it. + */ +function countClaims(facts) { + return [ + { + id: 'packages', + label: 'packages in the workspace', + actual: facts.packages.length, + pattern: numberedPattern(String.raw`(?:\*\*)?packages\b`), + required: ['CLAUDE.md', 'README.md'], + }, + { + id: 'publishable', + label: 'publishable packages', + actual: facts.publishable.length, + pattern: numberedPattern( + String.raw`(?:(?:is|are)\s+)?publish(?:ed|able)\b`, + ), + required: ['CLAUDE.md', 'README.md'], + }, + { + id: 'private', + label: 'private packages', + actual: facts.privatePackages.length, + pattern: numberedPattern( + String.raw`(?:more\s+)?(?:is|are)\s+(?:\*\*)?\x60?private\b`, + ), + required: ['CLAUDE.md', 'README.md'], + }, + { + id: 'api-reports', + label: 'committed API reports', + actual: facts.apiReports.length, + pattern: numberedPattern(String.raw`committed\s+(?:API\s+)?reports?\b`), + required: ['CLAUDE.md'], + }, + { + id: 'ci-steps', + label: 'named CI steps', + actual: facts.namedSteps.length, + pattern: numberedPattern(String.raw`named\s+(?:CI\s+)?steps?\b`), + required: ['CLAUDE.md', 'README.md'], + }, + { + id: 'ci-jobs', + label: 'CI jobs', + actual: facts.jobs.length, + pattern: numberedPattern(String.raw`jobs?\b`), + required: ['CLAUDE.md', 'README.md'], + }, + ]; +} + +// --------------------------------------------------------------------------------------- +// The repository facts. Derived once; every check below compares a document to THESE. +// --------------------------------------------------------------------------------------- + +function repositoryFacts(ctx) { + const packages = readdirSync(join(ctx.root, 'packages')) + .filter(name => ctx.exists(`packages/${name}/package.json`)) + .map(dir => { + const manifest = JSON.parse(ctx.read(`packages/${dir}/package.json`)); + return { + dir, + name: manifest.name, + private: manifest.private === true, + hasReadme: ctx.exists(`packages/${dir}/README.md`), + readmeBytes: ctx.exists(`packages/${dir}/README.md`) + ? statSync(join(ctx.root, 'packages', dir, 'README.md')).size + : 0, + apiReport: ctx.tracked(`packages/${dir}/etc/*.api.md`), + peersCore: Object.hasOwn( + manifest.peerDependencies ?? {}, + '@dexpace/core', + ), + dependsOnCore: Object.hasOwn( + manifest.dependencies ?? {}, + '@dexpace/core', + ), + }; + }); + + const workflow = ctx.read('.github/workflows/ci.yml'); + // A named step is a `- name:` under `steps:`. Counting `run:` would miss the matrix + // legs and counting `-` would count `uses:` setup steps, which are not gates. + // Jobs are the 2-space keys inside the `jobs:` block only. Counting every 2-space key + // in the file also counts `on:`'s `pull_request:`, which is how this first read 3. + const jobsBlock = workflow.slice(workflow.indexOf('\njobs:\n')); + const jobs = [...jobsBlock.matchAll(/^ {2}([a-z][a-z0-9-]*):$/gm)].map( + m => m[1], + ); + const namedSteps = [...workflow.matchAll(/^ {6}- name: (.+)$/gm)].map(m => + m[1].trim(), + ); + + const scripts = Object.keys( + JSON.parse(ctx.read('package.json')).scripts ?? {}, + ); + + // Tracked, so an editor swap file or an untracked scratch directory in `docs/` cannot + // manufacture an `act` finding against every document that "omits" it. + const docsEntries = [ + ...new Set(ctx.tracked('docs/*.md', 'docs/**').map(f => f.split('/')[1])), + ].sort(); + + return { + packages, + publishable: packages.filter(p => !p.private), + privatePackages: packages.filter(p => p.private), + apiReports: ctx.tracked('packages/*/etc/*.api.md'), + jobs, + namedSteps, + scripts, + verifyScripts: scripts.filter(s => s.startsWith('verify:')), + docsEntries, + }; +} + +// --------------------------------------------------------------------------------------- +// 1. docs/superpowers/ is an inbox. Anything in it is unfiled. +// --------------------------------------------------------------------------------------- + +function checkInbox(ctx) { + // `present`, not `tracked`: the inbox's NORMAL state is a file `brainstorming` has just + // written and nobody has staged. A tracked-only sweep reports the empty tree the skill + // exists to notice. + const stray = ctx + .present('docs/superpowers/**') + .filter(f => posix.basename(f) !== 'README.md'); + for (const file of stray) { + ctx.finding( + 'inbox', + 'act', + `${file} is still in the inbox. It belongs under docs/work//phaseN/ — ` + + 'see the collection rules in docs/README.md.', + ); + } +} + +// --------------------------------------------------------------------------------------- +// 2. Documents at the repository root that belong under docs/. +// --------------------------------------------------------------------------------------- + +const ROOT_MARKDOWN_ALLOWED = new Set([ + 'README.md', + 'CLAUDE.md', + // #58's scope; permitted the moment they exist. + 'CONTRIBUTING.md', + 'CODE_OF_CONDUCT.md', + 'SECURITY.md', + 'CHANGELOG.md', + 'LICENSE.md', +]); + +function checkRootDocuments(ctx) { + // `git ls-files -- '*.md'` matches at any depth: a pathspec glob crosses `/`. The root + // is what this check is about, so filter to files with no directory component. + for (const file of ctx.tracked('*.md').filter(f => !f.includes('/'))) { + if (ROOT_MARKDOWN_ALLOWED.has(file)) continue; + ctx.finding( + 'root', + 'act', + `${file} sits at the repository root. A register belongs in docs/, a phase record ` + + 'under docs/work/. The root carries README.md, CLAUDE.md and the community-health ' + + 'files only.', + ); + } +} + +// --------------------------------------------------------------------------------------- +// 3+4. Claims in CLAUDE.md, README.md and the community-health files, against the facts. +// --------------------------------------------------------------------------------------- + +/** + * `CONTRIBUTING.md` and `SECURITY.md` make the same class of ungated claim about this + * repository — the CI step count, the gate list, the package facts. They do not exist on + * this branch (issue #58 adds them), so each is checked only if present. + */ +const CLAIM_DOCUMENTS = [ + 'CLAUDE.md', + 'README.md', + 'CONTRIBUTING.md', + 'SECURITY.md', +]; + +/** + * The documents that must name every package, for the same reason `required` exists on a + * count claim: these two carry the workspace table, so a package missing from them is drift. + * + * `CONTRIBUTING.md` and `SECURITY.md` are deliberately not here. Neither enumerates the + * workspace — `CONTRIBUTING.md` states the shape once and points at `CLAUDE.md` for the + * table, and `SECURITY.md` names only the packages that carry a security surface. Requiring + * the full list of them would report eighteen findings against two files that are correct, + * which is how a checker teaches people to ignore it. + */ +const PACKAGE_ROSTER_DOCUMENTS = ['CLAUDE.md', 'README.md']; + +/** + * The prose a document asserts in its own voice. + * + * Fenced code is not prose, and a double-quoted span is reported speech: `CLAUDE.md`'s + * documentation-upkeep section quotes the historical drift it fixed — `"two published + * packages" against eleven` — which is a description of a past claim, not a present one. + * Matching inside either turns a document that explains its own history into a document + * that fails its own check. + */ +function assertedProse(text) { + return text.replace(/^```[\s\S]*?^```$/gm, '').replace(/"[^"]*"/g, ' '); +} + +function checkClaims(ctx, facts) { + const claims = countClaims(facts); + + for (const doc of CLAIM_DOCUMENTS) { + if (!ctx.exists(doc)) continue; + const text = ctx.read(doc); + const prose = assertedProse(text); + + for (const pkg of PACKAGE_ROSTER_DOCUMENTS.includes(doc) + ? facts.packages + : []) { + if (!text.includes(pkg.name)) { + ctx.finding( + 'claims', + pkg.private ? 'note' : 'act', + `${doc} never names ${pkg.name}${pkg.private ? ' (private)' : ''}. ` + + `The workspace has ${String(facts.packages.length)} packages: ` + + `${String(facts.publishable.length)} publishable, ` + + `${String(facts.privatePackages.length)} private.`, + ); + } + } + + for (const claim of claims) { + // Every document that states a count is checked; only the documents in `required` + // must state it. `CONTRIBUTING.md` need not carry a package count — but if it does, + // being wrong is the same defect it is anywhere else. + const required = claim.required.includes(doc); + let stated = 0; + for (const match of prose.matchAll(claim.pattern)) { + const value = parseNumeral(match[1]); + if (value === null) continue; // an adjective, not a numeral + stated++; + if (value !== claim.actual) { + ctx.finding( + 'claims', + 'act', + `${doc} states "${match[0].trim()}" but the repository has ` + + `${String(claim.actual)} ${claim.label}.`, + ); + } + } + if (stated === 0 && required) { + ctx.finding( + 'claims', + 'act', + `${doc} states no count of ${claim.label} (${String(claim.actual)}). ` + + 'A count that is only checked when it happens to be found is not checked: ' + + 'deleting or rewording the sentence passes.', + ); + } + } + + // Only enforced on a document that claims to enumerate the gates. README.md delegates + // to CLAUDE.md and the preflight command by design, so listing them there is optional — + // but a document that lists SOME must list all, which is exactly how `verify:sse-37` + // went unmentioned for four phases. + // Two or more is a list; one is a citation. README.md naming `verify:seam-1` once as + // an example of the zero-dependency rule is not a claim to enumerate the gates. + const mentioned = facts.verifyScripts.filter(s => text.includes(s)).length; + if (doc === 'CLAUDE.md' || mentioned >= 2) { + for (const script of facts.verifyScripts) { + if (!text.includes(script)) { + ctx.finding( + 'claims', + 'act', + `${doc} lists verification gates but not \`${script}\`, which is blocking in ` + + '.github/workflows/ci.yml.', + ); + } + } + } + + if (doc === 'CLAUDE.md') { + for (const entry of facts.docsEntries) { + if (!text.includes(`docs/${entry}`)) { + ctx.finding( + 'claims', + 'note', + `CLAUDE.md's documentation map omits docs/${entry}.`, + ); + } + } + } + } + + // docs/README.md is the index; every entry in docs/ must appear in it. + const index = ctx.read('docs/README.md'); + for (const entry of facts.docsEntries) { + if (!index.includes(entry)) { + ctx.finding( + 'claims', + 'act', + `docs/README.md does not list docs/${entry}.`, + ); + } + } +} + +// --------------------------------------------------------------------------------------- +// 5. A README on every publishable package. Private ones are exempt. +// --------------------------------------------------------------------------------------- + +const README_THIN_BYTES = 800; + +function checkPackageReadmes(ctx, facts) { + for (const pkg of facts.publishable) { + if (!pkg.hasReadme) { + ctx.finding( + 'readmes', + 'act', + `packages/${pkg.dir}/README.md is missing. The harvested styleguide requires one on ` + + 'every publishable package (docs/knowledge/harvested/documentation.md:28).', + ); + continue; + } + if (pkg.readmeBytes < README_THIN_BYTES) { + ctx.finding( + 'readmes', + 'note', + `packages/${pkg.dir}/README.md is ${String(pkg.readmeBytes)} bytes. The bar is ` + + 'zero to one working call in about 30 seconds, without reading source ' + + '(documentation.md:28-30).', + ); + } + if (pkg.dependsOnCore) { + ctx.finding( + 'readmes', + 'act', + `packages/${pkg.dir} declares @dexpace/core as a dependency. It must be a peer ` + + '(SEAM-1, the dual-package hazard).', + ); + } + } +} + +// --------------------------------------------------------------------------------------- +// 6. Broken relative links in docs/, CLAUDE.md, README.md and the package READMEs. +// --------------------------------------------------------------------------------------- + +const LINK = /\[[^\]]*\]\(([^)\s]+)(?:\s+"[^"]*")?\)/g; + +function linkedFiles(ctx) { + // BOTH `docs/` globs, through a Set. Git's pathspec `**/` does not match zero + // directories, so `docs/**/*.md` alone silently drops every file at the TOP of `docs/` — + // the index this skill ships and all three registers, 67 relative links unchecked. Git's + // `*` does cross `/`, so `docs/*.md` alone happens to cover both; listing the pair and + // deduping says which coverage is intended instead of resting on that subtlety, and the + // Set is what stops the overlap reporting one broken link twice. + return [ + ...new Set([ + ...ctx.tracked('docs/*.md', 'docs/**/*.md'), + ...ctx.tracked('*.md').filter(f => !f.includes('/')), + ...ctx.tracked('packages/*/README.md'), + ]), + ]; +} + +function checkLinks(ctx) { + for (const file of linkedFiles(ctx)) { + const text = ctx.read(file); + // A regex literal inside a fenced block is not a link. + const parts = text.split(/(^```[\s\S]*?^```$)/m); + for (let i = 0; i < parts.length; i += 2) { + for (const match of parts[i].matchAll(LINK)) { + const raw = match[1]; + if (/^(https?:|mailto:|#)/.test(raw)) continue; + const target = raw.split('#')[0]; + if (target === '') continue; + const resolved = posix.normalize( + posix.join(posix.dirname(file), decodeURIComponent(target)), + ); + if (!ctx.exists(resolved)) { + ctx.finding( + 'links', + 'act', + `${file} links ${raw}, which resolves to a path that does not exist.`, + ); + } + } + } + } +} + +// --------------------------------------------------------------------------------------- +// 7. Register text that landed in a specification document instead of a register. +// --------------------------------------------------------------------------------------- + +const SPEC_TREES = ['docs/work/**/*.md', 'docs/sdk-documentation/*.md']; +const AGGREGATE_HEADINGS = [ + /^##\s+Open Findings\b/m, + /^##\s+Deferred Items Log\s*$/m, + /^##\s+Open Items\s*$/m, +]; + +function checkRegisterLeakage(ctx) { + for (const file of ctx.tracked(...SPEC_TREES)) { + const text = ctx.read(file); + for (const heading of AGGREGATE_HEADINGS) { + const match = heading.exec(text); + if (!match) continue; + // The roadmap keeps pointer stubs under these names; a stub is a paragraph, not a table. + const after = text.slice(match.index, match.index + 600); + if (/^\*\*Moved out on /m.test(after)) continue; + const line = text.slice(0, match.index).split('\n').length; + ctx.finding( + 'registers', + 'act', + `${file}:${String(line)} carries "${match[0].trim()}". An aggregate register belongs ` + + "at the docs/ root — open-items.md or deviations.md. A phase's " + + 'own dated `## Deferred Items` section stays in place; the aggregate does not.', + ); + } + } +} + +// --------------------------------------------------------------------------------------- +// 8. Every `open-items.md ` citation resolves to a real item. +// +// "Real" spans TWO dated notes since 2026-09-04, and NEITHER is a live register. `docs/open-items.md` +// was dissolved that day and moved whole to `DISSOLUTION_NOTE`; the retirement table it used to end +// with had already moved to `PURGE_NOTE`. Item IDs stay reserved and still resolve because they are +// cited from source comments, which no gate rewrites — so this check outlives the register it was +// written for, and reads the archive instead. +// --------------------------------------------------------------------------------------- + +// An optional `[Q-T].` qualifier: Sections Q, R, S and T carry the relocated reviews' OWN row +// numbering, so three `F` namespaces coexist and a bare `F8` resolves to any of them. The register's +// Section index states the qualified form; option 2 (renumbering the dated records) was rejected in +// favour of teaching this check instead. +// Matches BOTH spellings: the pre-2026-09-04 `open-items.md K11` and the archive path the +// dissolution rewrote them to, `…/2026-09-04-open-items-dissolution.md K11`. Widened rather than +// swapped: `docs/work/` is never retro-edited, so the old spelling survives in every phase record, +// and a regex that matched only one of the two would have gone quietly blind to most of the corpus +// -- it did, for one run, reporting 9 citations where there were 61. +const CITATION = + /open-items(?:-dissolution)?\.md`?[ ]*(?:§)?\s*(?:([Q-T])\.)?([A-Z]\d+)/g; + +// `## Section — …`. Sections Q-T hold their rows as `| | …` table rows rather than as +// `### ` headings, so a qualified citation has to be resolved against the section's own rows. +const SECTION_HEADING = /^## Section ([A-Z])(?:[^\n]*)$/gm; +const TABLE_ROW_ID = /^\|\s*([A-Z]\d+)\s*\|/gm; +const QUALIFIED_SECTIONS = 'QRST'; + +/** + * The register itself, archived. `docs/open-items.md` was dissolved on 2026-09-04 — every open + * question in it decided, and what remained moved here whole. The `### ` headings live on, so a + * citation written when the register was live still resolves. + */ +const DISSOLUTION_NOTE = 'docs/work/mvp/2026-09-04-open-items-dissolution.md'; + +// The second namespace of resolvable IDs, and it is NOT in the register any more. A resolved item's +// BODY was removed and replaced by one row in `## Retired items`; on 2026-09-04 that table — and +// `deferred-items.md`'s `## Delivered and retired` beside it — was deleted outright, and its 102 rows +// were reproduced verbatim into the dated note below. The IDs were never released, so a source comment +// citing `K10` or `T.F9` must still resolve; deleting the table without moving the namespace left 25 +// citations pointing at nothing. The note is therefore read exactly as the table was, bare +// (`| \`K10\` |`) and section-qualified (`| \`T.F9\` |`) alike. +// +// ONLY the `## Purged item IDs` table is parsed. The note's second table, `## Purged rows without an +// item ID`, holds the Section D / Section R / `L1 —` rows that never carried an ID; the old code +// ignored them because their first cell is not one (`D — …` fails `[A-Z]\d+` immediately, `L1 — …` +// fails the closing `|`), and slicing to the one heading keeps that true by construction rather than +// by luck. +const PURGE_NOTE = 'docs/work/mvp/2026-09-04-register-retirement-purge.md'; +const PURGED_HEADING = /^## Purged item IDs\s*$/m; +const PURGED_ROW = /^\|\s*`?(?:([Q-T])\.)?([A-Z]\d+)`?\s*\|/gm; + +/** + * The IDs the note's `## Purged item IDs` table reserves, as `{bare: Set, qualified: Map}`. + * + * Empty maps when the note carries no such section — and the caller passes `''` when the note is + * absent entirely — so the check degrades to the pre-retirement behaviour rather than throwing. + * `ctx.read` is `readFileSync`: an absent path is an `ENOENT` that takes the whole run down with a + * raw stack instead of producing a finding, which is why the call site guards on `ctx.exists`. + */ +function purgedIds(note) { + const heading = PURGED_HEADING.exec(note); + const bare = new Set(); + const qualified = new Map(); + if (heading === null) return {bare, qualified}; + const from = heading.index + heading[0].length; + const next = note.slice(from).search(/^## /m); + const body = next === -1 ? note.slice(from) : note.slice(from, from + next); + for (const [, section, id] of body.matchAll(PURGED_ROW)) { + if (section === undefined) bare.add(id); + else { + if (!qualified.has(section)) qualified.set(section, new Set()); + qualified.get(section).add(id); + } + } + return {bare, qualified}; +} + +/** + * Row IDs per qualifiable section, as `{Q: Set('D1', …), R: Set('E1', …), …}`. + * + * Read from the table rows, not from headings: a relocated review's rows have no `###` of their own, + * which is the whole reason a bare citation into one is ambiguous. + */ +function qualifiableRows(register) { + const bounds = []; + for (const match of register.matchAll(SECTION_HEADING)) { + bounds.push({letter: match[1], start: match.index}); + } + const rows = new Map(); + for (const [i, section] of bounds.entries()) { + if (!QUALIFIED_SECTIONS.includes(section.letter)) continue; + const end = bounds[i + 1]?.start ?? register.length; + const body = register.slice(section.start, end); + rows.set( + section.letter, + new Set([...body.matchAll(TABLE_ROW_ID)].map(m => m[1])), + ); + } + return rows; +} + +function citedFiles(ctx) { + return ctx + .tracked( + 'packages/**', + 'tests/**', + 'scripts/**', + 'docs/**', + '*.md', + '.claude/**', + ) + .filter(f => /\.(md|mts|ts|mjs|js)$/.test(f)) + .filter(f => !f.startsWith('.changeset/')); // frozen release history +} + +/** + * Every register citation in the repository, with where it sits. + * + * Exported because three documents used to state three different, all-wrong counts of it. + * There is one derivation, and `--only=citations` prints it. + */ +export function registerCitations(ctx) { + const register = ctx.exists(DISSOLUTION_NOTE) + ? ctx.read(DISSOLUTION_NOTE) + : ''; + const ids = new Set( + [...register.matchAll(/^### ([A-Z]\d+)\b/gm)].map(m => m[1]), + ); + const rows = qualifiableRows(register); + // A purged item has no heading and no section row left anywhere in the register — only its row in + // the note. Merged in rather than checked separately: a citation does not know, and must not care, + // whether the item it names is still live, and the two namespaces are one to every caller. + const purged = purgedIds(ctx.exists(PURGE_NOTE) ? ctx.read(PURGE_NOTE) : ''); + for (const id of purged.bare) ids.add(id); + for (const [section, set] of purged.qualified) { + if (!rows.has(section)) rows.set(section, new Set()); + for (const id of set) rows.get(section).add(id); + } + const sites = []; + for (const file of citedFiles(ctx)) { + const text = ctx.read(file); + for (const match of text.matchAll(CITATION)) { + const [, qualifier, id] = match; + sites.push({ + file, + line: text.slice(0, match.index).split('\n').length, + id, + qualifier, + cited: qualifier === undefined ? id : `${qualifier}.${id}`, + resolves: + qualifier === undefined + ? ids.has(id) + : (rows.get(qualifier)?.has(id) ?? false), + }); + } + } + return {ids, rows, sites}; +} + +function checkRegisterCitations(ctx) { + const {sites} = registerCitations(ctx); + for (const site of sites.filter(s => !s.resolves)) { + const where = + site.qualifier === undefined + ? `which has no \`### \` heading and no \`## Purged item IDs\` row in ${PURGE_NOTE}` + : `which Section ${site.qualifier} carries as neither a table row nor a purged row in ${PURGE_NOTE}`; + ctx.finding( + 'citations', + 'act', + `${site.file}:${String(site.line)} cites open-items.md ${site.cited}, ${where}. ` + + 'Item IDs are permanent; a dangling one means the citation, not the register, is wrong.', + ); + } +} + +// --------------------------------------------------------------------------------------- +// 9. The frozen guard is intact, and nothing the skill may write is frozen. +// --------------------------------------------------------------------------------------- + +const WRITABLE_SURFACE = [ + 'docs/README.md', + 'docs/first-release.md', + 'docs/deviations.md', + 'docs/sdk-documentation', + 'docs/work', + 'docs/superpowers', + 'CLAUDE.md', + 'README.md', +]; + +function checkGuard(ctx) { + for (const path of WRITABLE_SURFACE) { + if (isFrozen(path, ctx.root)) { + ctx.finding( + 'guard', + 'act', + `${path} is on the writable surface AND matches a frozen entry.`, + ); + } + } + for (const path of [ + 'docs/product-spec/04.md', + 'docs/knowledge/notes/x.md', + 'docs/sdk-design-nodejs.md', + ]) { + if (!isFrozen(path, ctx.root)) { + ctx.finding( + 'guard', + 'act', + `the guard does not refuse ${path}. Run guard.test.mjs.`, + ); + } + } +} + +// --------------------------------------------------------------------------------------- + +const CHECKS = { + inbox: checkInbox, + root: checkRootDocuments, + claims: checkClaims, + readmes: checkPackageReadmes, + links: checkLinks, + registers: checkRegisterLeakage, + citations: checkRegisterCitations, + guard: checkGuard, +}; + +export const CHECK_NAMES = Object.freeze(Object.keys(CHECKS)); + +export function probe(only, root = resolveRepoRoot()) { + const ctx = createContext(root); + const facts = repositoryFacts(ctx); + for (const name of only ?? CHECK_NAMES) { + const check = CHECKS[name]; + if (check === undefined) throw new Error(`unknown check '${name}'`); + check(ctx, facts); + } + return {facts, findings: ctx.findings, ctx}; +} + +function main(argv) { + const strict = argv.includes('--strict'); + const onlyArg = argv.find(a => a.startsWith('--only=')); + const only = onlyArg?.slice('--only='.length).split(','); + const rootArg = argv.find(a => a.startsWith('--root=')); + + const { + facts, + findings: found, + ctx, + } = probe(only, rootArg?.slice('--root='.length) ?? resolveRepoRoot()); + + process.stdout.write('housekeeping probe — read-only\n\n'); + process.stdout.write( + `repository: ${String(facts.packages.length)} packages ` + + `(${String(facts.publishable.length)} publishable, ${String(facts.privatePackages.length)} private), ` + + `${String(facts.apiReports.length)} API reports, ` + + `${String(facts.namedSteps.length)} named CI steps across ${String(facts.jobs.length)} jobs, ` + + `${String(facts.scripts.length)} package scripts\n`, + ); + + if (only?.includes('citations')) { + const {ids, sites} = registerCitations(ctx); + const outside = sites.filter(s => s.file !== DISSOLUTION_NOTE); + const core = sites.filter(s => s.file.startsWith('packages/core/src/')); + process.stdout.write( + `citations: ${String(sites.length)} total, ${String(outside.length)} outside the ` + + `register, ${String(core.length)} in packages/core/src/, ` + + `${String(new Set(sites.map(s => s.cited)).size)} distinct IDs against ` + + `${String(ids.size)} items ` + + `(${String(sites.filter(s => s.qualifier !== undefined).length)} section-qualified)\n`, + ); + } + process.stdout.write('\n'); + + if (found.length === 0) { + process.stdout.write('no drift found.\n'); + return 0; + } + + const byCheck = new Map(); + for (const f of found) { + if (!byCheck.has(f.check)) byCheck.set(f.check, []); + byCheck.get(f.check).push(f); + } + for (const [check, items] of byCheck) { + process.stdout.write(`## ${check} (${String(items.length)})\n`); + for (const item of items) { + process.stdout.write(` [${item.severity}] ${item.message}\n`); + } + process.stdout.write('\n'); + } + process.stdout.write( + `${String(found.length)} finding(s). This stage writes nothing — read them, then run ` + + 'apply.mjs for the mechanical ones.\n', + ); + return strict ? 1 : 0; +} + +if (import.meta.url === `file://${process.argv[1]}`) { + process.exitCode = main(process.argv.slice(2)); +} diff --git a/.claude/skills/housekeeping/probe.test.mjs b/.claude/skills/housekeeping/probe.test.mjs new file mode 100644 index 0000000..35f1454 --- /dev/null +++ b/.claude/skills/housekeeping/probe.test.mjs @@ -0,0 +1,918 @@ +// SPDX-License-Identifier: MIT +// .claude/skills/housekeeping/probe.test.mjs +// +// Tests the CHECKS, not a copy of their logic, and not the live tree's cleanliness. +// +// The first version of this file asserted only that each check returned no findings +// against the real repository. That passes just as happily over a check whose body has +// become `return;` — proved by mutation: seven of the eight were replaced with `return;` +// and the suite stayed 29/29 green. `scripts/verify-seam-1.test.mjs:6`, +// `verify-knowledge-structure.test.mjs:4` and `verify-test-partition.test.mjs:4` each +// reached the same conclusion earlier in this repository and build fixture trees instead. +// So does this. +// +// Every check therefore has a pair: a fixture that must be clean, and a mutation of it +// that must fire. The live-tree assertions stay at the end, because they are still what +// says the repository is in order today. + +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {execFileSync} from 'node:child_process'; +import {dirname} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {CHECK_NAMES, parseNumeral, probe, registerCitations} from './probe.mjs'; +import {CLEAN_CLAIMS, makeFixture, removeFixture} from './fixture.mjs'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const ROOT = execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd: HERE, + encoding: 'utf8', +}).trim(); + +/** Runs `checks` over a fixture built from `spec` and returns its findings. */ +function onFixture(spec, checks) { + const root = makeFixture(spec); + try { + return probe(checks, root).findings; + } finally { + removeFixture(root); + } +} + +function messages(findings) { + return findings.map(f => f.message); +} + +// --- numerals --------------------------------------------------------------------------- + +test('parseNumeral reads digits, words and compounds, and rejects prose', () => { + assert.equal(parseNumeral('0'), 0); + assert.equal(parseNumeral('20'), 20); + assert.equal(parseNumeral('nine'), 9); + assert.equal(parseNumeral('Eleven'), 11); + assert.equal(parseNumeral('Twenty'), 20); + assert.equal(parseNumeral('twenty-four'), 24); + assert.equal(parseNumeral('ninety-nine'), 99); + // The words that made the digits-only version protect one sentence in the repository. + assert.equal(parseNumeral('published'), null); + assert.equal(parseNumeral('several'), null); + assert.equal(parseNumeral('twenty-zero'), null); +}); + +// --- claims ----------------------------------------------------------------------------- + +test('a clean fixture reports nothing', () => { + assert.deepEqual(messages(onFixture({}, undefined)), []); +}); + +test('claims: a count stated as a WORD and wrong is caught', () => { + // The exact drift SKILL.md names as this tool's reason for existing. + const found = onFixture( + { + overrides: { + 'CLAUDE.md': `# CLAUDE.md\n\n${CLEAN_CLAIMS}\n\nSeven packages, actually.\n\ndocs/README.md docs/work docs/sdk-documentation docs/superpowers\n`, + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /states "Seven packages" but the repository has 2/.test(f.message), + ), + `expected a word-count finding, got: ${JSON.stringify(messages(found))}`, + ); +}); + +test('claims: a count stated as a DIGIT and wrong is caught', () => { + const found = onFixture( + { + overrides: { + 'README.md': `# fixture\n\n${CLEAN_CLAIMS.replace('Three named steps', '7 named steps')}\n`, + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /states "7 named steps" but the repository has 3 named CI steps/.test( + f.message, + ), + ), + JSON.stringify(messages(found)), + ); +}); + +test('claims: a DELETED count claim is caught — presence is asserted', () => { + const found = onFixture( + { + overrides: { + 'CLAUDE.md': `# CLAUDE.md\n\n${CLEAN_CLAIMS.replace('Three named steps across two jobs.', 'Some steps across two jobs.')}\n\ndocs/README.md docs/work docs/sdk-documentation docs/superpowers\n`, + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /CLAUDE\.md states no count of named CI steps \(3\)/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('claims: a quoted historical count is reported speech, not a claim', () => { + // CLAUDE.md's own documentation-upkeep section quotes the drift it fixed. + const found = onFixture( + { + overrides: { + 'CLAUDE.md': `# CLAUDE.md\n\n${CLEAN_CLAIMS}\n\nIt used to say "two published packages" and that was wrong.\n\ndocs/README.md docs/work docs/sdk-documentation docs/superpowers\n`, + }, + }, + ['claims'], + ); + assert.deepEqual(messages(found), []); +}); + +test("claims: a community-health file's counts are checked when it exists", () => { + // #58 adds CONTRIBUTING.md and SECURITY.md; they make the same class of claim about this + // repository. Neither must STATE a count — but a count they do state is checked. + const found = onFixture( + { + overrides: { + 'CONTRIBUTING.md': + '# contributing\n\n`@dexpace/thing` and `@dexpace/secret`. Seven named steps across two jobs.\n', + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /^CONTRIBUTING\.md states "Seven named steps"/.test(f.message), + ), + JSON.stringify(messages(found)), + ); + assert.ok( + !found.some(f => /CONTRIBUTING\.md states no count/.test(f.message)), + 'a community-health file must not be REQUIRED to carry a count', + ); +}); + +test('claims: a community-health file is not required to NAME every package', () => { + // The roster lives in CLAUDE.md and README.md. CONTRIBUTING.md states the shape once and + // points at CLAUDE.md for the table; SECURITY.md names only the packages that carry a + // security surface. Requiring the full list of either reported eighteen findings against + // two correct files, which is how a checker teaches people to ignore it. + const found = onFixture( + { + overrides: { + 'CONTRIBUTING.md': + '# contributing\n\nTwo packages, one of them published.\n', + 'SECURITY.md': '# security\n\nReport privately.\n', + }, + }, + ['claims'], + ); + assert.ok( + !found.some(f => + /CONTRIBUTING\.md never names|SECURITY\.md never names/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('claims: an absent community-health file fires nothing', () => { + // Neither exists on this branch. The check must not report on a file that is not there. + const found = onFixture({}, ['claims']); + assert.ok(!found.some(f => /CONTRIBUTING\.md|SECURITY\.md/.test(f.message))); +}); + +test('claims: an unnamed package is caught', () => { + const found = onFixture( + { + overrides: { + 'README.md': `# fixture\n\n${CLEAN_CLAIMS.replace('`@dexpace/thing` and ', '')}\n`, + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => /README\.md never names @dexpace\/thing/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('claims: a gate list that omits a blocking gate is caught', () => { + const found = onFixture( + { + overrides: { + 'package.json': `${JSON.stringify( + { + name: 'fixture', + private: true, + scripts: { + 'verify:seam-1': 'true', + 'verify:brand-new': 'true', + test: 'true', + }, + }, + null, + 2, + )}\n`, + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /CLAUDE\.md lists verification gates but not `verify:brand-new`/.test( + f.message, + ), + ), + JSON.stringify(messages(found)), + ); +}); + +test('claims: docs/README.md omitting an entry is caught', () => { + const found = onFixture( + { + overrides: { + 'docs/README.md': '# docs\n\nEntries: README.md.\n', + }, + }, + ['claims'], + ); + assert.ok( + found.some(f => + /docs\/README\.md does not list docs\/work/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('claims: an UNTRACKED entry in docs/ does not manufacture findings', () => { + // Derived from `git ls-files`, so an editor swap file cannot fire four `act` findings. + const found = onFixture( + { + untracked: { + 'docs/.notes.md.swp': 'x', + 'docs/validation-prompts/a.md': '# scratch\n', + }, + }, + ['claims'], + ); + assert.deepEqual(messages(found), []); +}); + +// --- inbox ------------------------------------------------------------------------------ + +test('inbox: an UNTRACKED phase document is caught', () => { + // The inbox's normal state: `brainstorming` has just written a file and nobody staged it. + const found = onFixture( + { + untracked: { + 'docs/superpowers/specs/2026-09-01-phase11-thing-design.md': + '# design\n', + }, + }, + ['inbox'], + ); + assert.ok( + found.some(f => + /phase11-thing-design\.md is still in the inbox/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('inbox: a tracked phase document is caught too', () => { + const found = onFixture( + { + overrides: { + 'docs/superpowers/plans/2026-09-01-phase11-thing.md': '# plan\n', + }, + }, + ['inbox'], + ); + assert.equal(found.length, 1, JSON.stringify(messages(found))); +}); + +test('inbox: the inbox README is never reported', () => { + assert.deepEqual(messages(onFixture({}, ['inbox'])), []); +}); + +// --- root ------------------------------------------------------------------------------- + +test('root: a stray register at the repository root is caught', () => { + const found = onFixture( + {overrides: {'open-items.md': '# a second register\n'}}, + ['root'], + ); + assert.ok( + found.some(f => + /^open-items\.md sits at the repository root/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('root: the allowed root files are not reported', () => { + const found = onFixture( + { + overrides: { + 'CONTRIBUTING.md': '# contributing\n', + 'SECURITY.md': '# security\n', + }, + }, + ['root'], + ); + assert.deepEqual(messages(found), []); +}); + +// --- readmes ---------------------------------------------------------------------------- + +test('readmes: a publishable package with no README is caught', () => { + const found = onFixture({overrides: {'packages/thing/README.md': null}}, [ + 'readmes', + ]); + assert.ok( + found.some(f => /packages\/thing\/README\.md is missing/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('readmes: a thin README is a note, not an act', () => { + const found = onFixture( + {overrides: {'packages/thing/README.md': '# thing\n'}}, + ['readmes'], + ); + assert.equal(found.length, 1); + assert.equal(found[0].severity, 'note'); + assert.match(found[0].message, /bytes\. The bar is/); +}); + +test('readmes: core declared as a dependency rather than a peer is caught', () => { + const found = onFixture( + { + overrides: { + 'packages/thing/package.json': `${JSON.stringify( + {name: '@dexpace/thing', dependencies: {'@dexpace/core': '*'}}, + null, + 2, + )}\n`, + }, + }, + ['readmes'], + ); + assert.ok( + found.some(f => /declares @dexpace\/core as a dependency/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('readmes: a private package needs no README', () => { + assert.deepEqual(messages(onFixture({}, ['readmes'])), []); +}); + +// --- links ------------------------------------------------------------------------------ + +test('links: a broken link in a nested docs file is caught', () => { + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': + '# a\n\n[gone](./nowhere.md)\n', + }, + }, + ['links'], + ); + assert.ok( + found.some(f => /architecture\.md links \.\/nowhere\.md/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('links: a broken link at the TOP of docs/ is caught', () => { + // `docs/**/*.md` alone misses every file here — git's `**/` does not match zero + // directories — which left the index and all three registers unchecked. + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': + '# Open Items\n\n### A1 — x — **WATCH**\n\n[gone](./nowhere.md)\n', + }, + }, + ['links'], + ); + assert.ok( + found.some(f => /links \.\/nowhere\.md/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('links: a broken link in a package README is caught', () => { + const found = onFixture( + { + overrides: { + 'packages/thing/README.md': `# thing\n\n[gone](./etc/nope.md)\n${'x '.repeat(500)}`, + }, + }, + ['links'], + ); + assert.ok( + found.some(f => + /packages\/thing\/README\.md links \.\/etc\/nope\.md/.test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('links: a link inside a fenced block is not a link', () => { + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': + '# a\n\n```js\nconst LINK = /\\[[^\\]]*\\]\\(([^)]+)\\)/g;\n```\n', + }, + }, + ['links'], + ); + assert.deepEqual(messages(found), []); +}); + +test('links: an external or anchor-only link is skipped', () => { + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': + '# a\n\n[x](https://example.invalid/nope) [y](#section) [z](mailto:a@b.invalid)\n', + }, + }, + ['links'], + ); + assert.deepEqual(messages(found), []); +}); + +// --- registers -------------------------------------------------------------------------- + +test('registers: an aggregate register in a specification document is caught', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/phase1/2026-01-01-phase1-thing.md': + '# phase 1\n\n## Deferred Items Log\n\n| Item |\n|---|\n| a |\n', + }, + }, + ['registers'], + ); + assert.ok( + found.some(f => /carries "## Deferred Items Log"/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('registers: a "Moved out on" pointer stub is not a register', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/phase1/2026-01-01-phase1-thing.md': + '# phase 1\n\n## Deferred Items Log\n\n**Moved out on 2026-08-31.** See docs/deferred-items.md.\n', + }, + }, + ['registers'], + ); + assert.deepEqual(messages(found), []); +}); + +// --- citations -------------------------------------------------------------------------- + +// Assembled at run time, never written as one literal. The citation check scans +// `.claude/**`, so a contiguous `open-items.md ` in THIS file is a citation the live +// tree sees — and a deliberately-dangling one would make the suite fail on itself. +const REGISTER = 'open-items.md'; +const cite = id => `See \`docs/${REGISTER}\` ${id} for the rest.`; + +test('citations: a dangling register citation is caught', () => { + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('Z9')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md Z9, which has no/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('citations: a resolving citation is not reported', () => { + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('A1')}\n`, + }, + }, + ['citations'], + ); + assert.deepEqual(messages(found), []); +}); + +// Sections Q-T hold the relocated reviews' OWN row numbering, so three `F` namespaces coexist and a +// bare `F8` resolves to any of them. The register's Section index states the qualified form; +// these four cases are the mechanism. +const QUALIFIED_REGISTER = + '# Open Items\n\n### A1 — a real item — **WATCH**\n\nBody.\n\n' + + '## Section S — a relocated review\n\n' + + '| # | Sev | Finding |\n|---|---|---|\n| F8 | minor | a row, not a heading |\n\n' + + '## Section T — another relocated review\n\n' + + '| # | Sev | Finding |\n|---|---|---|\n| F9 | major | a different row |\n'; + +const citeQualified = (section, id) => + `See \`docs/${REGISTER}\` ${section}.${id} for the rest.`; + +test("citations: a qualified citation resolves against that section's table rows", () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': + QUALIFIED_REGISTER, + 'docs/sdk-documentation/architecture.md': `# a\n\n${citeQualified('S', 'F8')}\n`, + }, + }, + ['citations'], + ); + assert.deepEqual(messages(found), []); +}); + +test('citations: a qualifier naming the wrong section is caught', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': + QUALIFIED_REGISTER, + // F8 is Section S's row, not Section T's. Both sections exist and both carry an `F` + // namespace, which is exactly the confusion the qualifier is for. + 'docs/sdk-documentation/architecture.md': `# a\n\n${citeQualified('T', 'F8')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => + /cites open-items\.md T\.F8, which Section T carries as neither a table row nor a purged row/.test( + f.message, + ), + ), + JSON.stringify(messages(found)), + ); +}); + +test('citations: a qualified ID is NOT satisfied by a `### ` heading elsewhere', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': + QUALIFIED_REGISTER, + // A1 has a heading, but Section S does not carry it as a row, so `S.A1` must not resolve. + 'docs/sdk-documentation/architecture.md': `# a\n\n${citeQualified('S', 'A1')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md S\.A1/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('citations: a bare ID still resolves against `### ` headings only', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': + QUALIFIED_REGISTER, + // F8 exists as a Section S ROW. Unqualified, it must not resolve -- the bare namespace is + // the `###` headings, and this is what makes the qualifier load bearing rather than optional. + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('F8')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md F8, which has no/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +// A resolved item's body was REMOVED from the register and replaced by one row in `## Retired items`. +// On 2026-09-04 that table was deleted and its rows reproduced verbatim into a dated note, so the +// second namespace of resolvable IDs now lives OUTSIDE the register. The ID is still never released, +// so every citation of it must keep resolving — the check reads the note's `## Purged item IDs` table +// as that second source, bare and section-qualified alike. +const PURGE_NOTE = 'docs/work/mvp/2026-09-04-register-retirement-purge.md'; + +// The register after the purge: one live heading, and a Section T whose rows are all gone. +const PURGED_REGISTER = + '# Open Items\n\n### A1 — a live item — **WATCH**\n\nBody.\n\n' + + '## Section T — a relocated review\n\n' + + 'All rows retired.\n'; + +// The note, with all three of its tables. Only the FIRST is a namespace, and the other two are the +// two ways that can go wrong. `## Purged rows without an item ID` holds rows whose first cell is not +// an ID at all (`D — …`, `L1 — …`); those fall out of the row pattern wherever it is pointed, which is +// what the real note relies on. `## Purged deferral rows` is the one that would actually leak: give it +// a row keyed like an item ID and a whole-file scan reserves it, while the slice does not. +const PURGE_NOTE_TEXT = + '# Register retirement purge\n\n' + + 'Prose.\n\n---\n\n' + + '## Purged item IDs\n\n' + + '| ID | Title | Resolution | Date | Evidence |\n|---|---|---|---|---|\n' + + '| `K10` | a fixed item | fixed | 2026-09-02 | `src/x.ts` |\n' + + '| `T.F9` | a retired review row | fixed | 2026-09-02 | `src/y.ts` |\n\n---\n\n' + + '## Purged rows without an item ID\n\n' + + '| Row | Subject | Resolution | Date | Evidence |\n|---|---|---|---|---|\n' + + '| L1 — `OBS-19` | a Section L half | fixed | 2026-09-02 | `src/w.ts` |\n' + + '| D — a Section D row | — | shipped | 2026-09-02 | `src/z.ts` |\n' + + '| R — residual: a thing | — | shipped | 2026-09-02 | `src/v.ts` |\n\n---\n\n' + + '## Purged deferral rows\n\n' + + '| Row key | Origin phase | Delivered by | Evidence | Date |\n|---|---|---|---|---|\n' + + '| `Y7` | Phase 0 | Phase 9 | `src/u.ts` | 2026-09-02 |\n'; + +const PURGED = { + 'docs/open-items.md': PURGED_REGISTER, + [PURGE_NOTE]: PURGE_NOTE_TEXT, +}; + +test("citations: a purged ID resolves against the note's `## Purged item IDs` table", () => { + const found = onFixture( + { + overrides: { + ...PURGED, + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('K10')}\n`, + }, + }, + ['citations'], + ); + assert.deepEqual(messages(found), []); +}); + +test('citations: a purged Q-T row resolves through its section qualifier', () => { + const found = onFixture( + { + overrides: { + ...PURGED, + // Section T no longer carries F9 as a table row -- only the note does. + 'docs/sdk-documentation/architecture.md': `# a\n\n${citeQualified('T', 'F9')}\n`, + }, + }, + ['citations'], + ); + assert.deepEqual(messages(found), []); +}); + +test('citations: an ID in neither the headings nor the note still dangles', () => { + const found = onFixture( + { + overrides: { + ...PURGED, + // K11 is neither a `### ` heading nor a purged row. The note widens the ID set; + // it must not turn the check into a no-op. + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('K11')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => + new RegExp( + String.raw`cites open-items\.md K11, which has no \`### \` heading and no ` + + String.raw`\`## Purged item IDs\` row in ${PURGE_NOTE}`, + ).test(f.message), + ), + JSON.stringify(messages(found)), + ); +}); + +test('citations: the finding names the note, never the deleted `## Retired items` table', () => { + const found = onFixture( + { + overrides: { + ...PURGED, + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('K11')}\n${citeQualified('T', 'F8')}\n`, + }, + }, + ['citations'], + ); + assert.equal(found.length, 2, JSON.stringify(messages(found))); + for (const message of messages(found)) { + assert.ok(message.includes(PURGE_NOTE), message); + assert.ok(!/Retired items/.test(message), message); + } +}); + +test('citations: a row under `## Purged rows without an item ID` is not an ID', () => { + const found = onFixture( + { + overrides: { + ...PURGED, + // `L1` is the first cell of a row in the SECOND table. It reserves no ID -- the row is keyed + // by section and title -- so a citation of it must still dangle. Parsing the whole note + // instead of the one section is what this case forbids. + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('L1')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md L1, which has no/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('citations: only the `## Purged item IDs` section is a namespace', () => { + const found = onFixture( + { + overrides: { + ...PURGED, + // `Y7` is the first cell of a row in the note's THIRD table, which is a deferral audit and + // reserves no open-item ID. Only the one section is sliced, so a citation of it dangles. + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('Y7')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md Y7, which has no/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('citations: the note being absent degrades rather than throws', () => { + // The default fixture writes no note at all. `ctx.read` is `readFileSync`, so an unguarded read of + // the missing path is an ENOENT that replaces every finding with a raw stack; the guard is what + // makes this a normal run with the pre-purge namespace only. + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('A1')}\n${cite('K10')}\n`, + }, + }, + ['citations'], + ); + // Spelled through `REGISTER` for the same reason `cite` is: a contiguous `open-items.md K10` in + // this file is a citation the live tree resolves, and asserting on one would couple this suite to + // whatever the real note happens to hold. + assert.deepEqual(messages(found), [ + `docs/sdk-documentation/architecture.md:4 cites ${REGISTER} K10, which has no ` + + `\`### \` heading and no \`## Purged item IDs\` row in ${PURGE_NOTE}. ` + + 'Item IDs are permanent; a dangling one means the citation, not the register, is wrong.', + ]); +}); + +test('citations: a note with no `## Purged item IDs` heading degrades too', () => { + const found = onFixture( + { + overrides: { + 'docs/work/mvp/2026-09-04-open-items-dissolution.md': PURGED_REGISTER, + [PURGE_NOTE]: '# a note that lost its table\n\nProse only.\n', + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('K10')}\n`, + }, + }, + ['citations'], + ); + assert.ok( + found.some(f => /cites open-items\.md K10, which has no/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +test('registerCitations is the single derivation, and reports where each site is', () => { + const root = makeFixture({ + overrides: { + 'docs/sdk-documentation/architecture.md': `# a\n\n${cite('A1')}\n`, + }, + }); + try { + const {ctx} = probe([], root); + const {ids, sites} = registerCitations(ctx); + assert.deepEqual([...ids], ['A1']); + assert.equal(sites.length, 1); + assert.deepEqual(sites[0], { + file: 'docs/sdk-documentation/architecture.md', + line: 3, + id: 'A1', + // `undefined` rather than absent: an unqualified citation still carries the field, so a + // caller can tell "bare" from "qualified" without re-parsing the text. + qualifier: undefined, + cited: 'A1', + resolves: true, + }); + } finally { + removeFixture(root); + } +}); + +// --- non-ASCII paths -------------------------------------------------------------------- + +test('a non-ASCII filename does not take the run down', () => { + // `git ls-files` C-quotes it by default, and a quoted path fed to readFileSync is an + // ENOENT that replaces every finding with a raw stack. + const found = onFixture( + { + overrides: { + 'docs/sdk-documentation/café.md': '# café\n\n[gone](./nowhere.md)\n', + }, + }, + ['links'], + ); + assert.ok( + found.some(f => /café\.md links \.\/nowhere\.md/.test(f.message)), + JSON.stringify(messages(found)), + ); +}); + +// --- plumbing --------------------------------------------------------------------------- + +test('every check name is selectable, and an unknown one is refused', () => { + const root = makeFixture(); + try { + for (const name of CHECK_NAMES) { + assert.doesNotThrow(() => probe([name], root), name); + } + assert.throws(() => probe(['nope'], root), /unknown check 'nope'/); + } finally { + removeFixture(root); + } +}); + +test('CHECK_NAMES is the eight the documentation states', () => { + assert.deepEqual( + [...CHECK_NAMES], + [ + 'inbox', + 'root', + 'claims', + 'readmes', + 'links', + 'registers', + 'citations', + 'guard', + ], + ); +}); + +// --- the live tree ---------------------------------------------------------------------- + +function gitStatus() { + return execFileSync('git', ['status', '--porcelain'], { + cwd: ROOT, + encoding: 'utf8', + }); +} + +test('the probe writes nothing', () => { + const before = gitStatus(); + probe(); + assert.equal( + gitStatus(), + before, + 'the working tree changed during a probe run', + ); +}); + +test('facts are derived from the repository, not from a document', () => { + const {facts} = probe([]); + assert.ok(facts.packages.length >= 2, 'found no packages'); + assert.equal( + facts.packages.length, + facts.publishable.length + facts.privatePackages.length, + 'every package is publishable or private, never both or neither', + ); + assert.ok(facts.publishable.every(p => p.name.startsWith('@dexpace/'))); + assert.ok(facts.namedSteps.length > 0, 'parsed no named CI steps'); + assert.ok(facts.jobs.length >= 1, 'parsed no CI jobs'); + assert.ok( + facts.jobs.every(j => !j.includes(' ')), + 'a parsed job name looks like prose, so the jobs: block regex has drifted', + ); + assert.ok(facts.scripts.includes('test'), 'parsed no package scripts'); + assert.ok(facts.verifyScripts.every(s => s.startsWith('verify:'))); + assert.ok(facts.docsEntries.includes('README.md'), 'docs/ has no index'); +}); + +test('every finding names a check and a severity the report can group by', () => { + for (const f of probe().findings) { + assert.ok(CHECK_NAMES.includes(f.check), `bad check: ${f.check}`); + assert.ok( + ['act', 'note'].includes(f.severity), + `bad severity: ${f.severity}`, + ); + assert.ok(f.message.length > 20, 'a finding must say what to do'); + } +}); + +test('the live tree is clean on every check', () => { + for (const name of CHECK_NAMES) { + assert.deepEqual(messages(probe([name]).findings), [], name); + } +}); diff --git a/.claude/skills/knowledge-lookup/SKILL.md b/.claude/skills/knowledge-lookup/SKILL.md new file mode 100644 index 0000000..218b4e3 --- /dev/null +++ b/.claude/skills/knowledge-lookup/SKILL.md @@ -0,0 +1,263 @@ +--- +name: knowledge-lookup +description: Use when starting a phase or a numbered task from a docs/work/mvp/ plan file, implementing or reviewing against a requirement ID (HTTP-7, SEAM-1, RETRY-13, NFR-5), resolving a styleguide citation such as "styleguide 6.7" or "ch08", auditing a subsystem against every rule the corpus holds for it, or recording what an implementation found in docs/knowledge/ — which is a note under notes/, never an edit to harvested/. +--- + +# Knowledge Lookup + +## Overview + +`docs/knowledge/` is 39 topics across two trees and ~1457 harvested entries, past what belongs +in context. `bun run knowledge` filters it. A requirement-ID query runs ~120–580 tokens +(median ~230) against a topic file of ~1800–5200 (median ~2300): roughly 9× smaller, and +much more than that when the ID you want lives in a file you'd never have guessed. + +Every entry is one bullet plus a `` line, and a stable key — `/<8 hex>`, printed +after the section name. The `` usually carries role, source path, line range and sha, which +is the citation a test-file header or deferral note needs; a Conflicts entry carries two sources +and no sha, and a note carries a manual `sha:` marker. Copy what is there. + +**Two trees, one query surface.** + +| Tree | Holds | Rule | +|---|---|---| +| `docs/knowledge/harvested/` | What the source documents say. Roles `spec`, `design`, `styleguide`. | Generated by `knowledge-harvest`. **Never hand-edit it.** A `` sha digests the whole source file, not the entry, so your edit changes no sha and the next harvest deletes or duplicates it. | +| `docs/knowledge/notes/` | What the implementation found. Role `review`, a manual `sha:` marker. | Hand-written. Small by design — a note earns its place only by overriding a harvested rule. | + +**How to read a result:** a role of `spec`, `design`, or `styleguide` states what the +documents say; a role of `review` states what the implementation found, and it overrides the +first. `verify:knowledge-structure` is the CI gate that keeps the two apart. + +## Start of a phase: run these two + +```bash +bun run knowledge --origin note --brief # what the implementation found +bun run knowledge --section conflicts --brief # what two documents still disagree about +``` + +They are different sets and you need both. The first is every note: the places where a plan's +Global Constraints may assert as settled something the implementation found to be otherwise. +The second is every recorded design-vs-styleguide contradiction — a resolved one prints +`[overridden by notes/…]` on its location line, and one with no such tag is **still open**. A +plan that assumes an open conflict is settled is the failure both queries exist to catch, and +nothing else in this workflow surfaces either. + +Different filters AND together; multiple values inside one filter OR. So +`--req A --req B --topic headers` means "(cites A or B) and is in a headers file". + +## Check the result is real before trusting it + +**A `--req` hit is not proof the corpus knows anything.** 256 of the 645 canonical IDs resolve +*only* to an appendix-B conformance roll-up — one sentence naming three to five IDs and +stating none of them. It exits 0, so nothing else will warn you. (`--coverage` reports the +substantive / roll-up-only / uncited split per prefix; run it rather than trusting a +remembered number.) + +The CLI tags these `[appendix-B roll-up]` and prints a WARNING when every hit is one. When you +see it, follow the roll-up path below — all three steps, not just the first. + +### The roll-up path + +1. `bun run knowledge --req ` came back all roll-up. +2. Get the canonical text: + + ```bash + grep -n '^| HTTP-10 ' docs/product-spec/appendix-c-consolidated-normative-requirement-index.md + ``` + + The leading `| ` and the trailing space are load-bearing — `grep 'HTTP-1'` matches HTTP-10 + through HTTP-19. +3. Read the owning chapter. The row from step 2 carries a Subsystem cell; the prefix maps to a + chapter file in `docs/product-spec/`: + + | Prefix | Chapter | + |---|---| + | SEAM | `03-pluggable-seams-and-extension-model.md` | + | HTTP | `04-core-http-domain-model.md` | + | IO | `05-i-o-contracts.md` | + | BODY | `06-request-and-response-body-lifecycle.md` | + | CTX | `07-execution-context-model.md` | + | PIPE, RECOV | `08-execution-pipelines.md` | + | RETRY | `09-retry-and-resilience.md` | + | REDIR | `10-redirect-handling.md` | + | AUTH | `11-authentication.md` | + | PAGE | `12-pagination.md` | + | SSE | `13-server-sent-events-and-streaming.md` | + | SERDE | `14-serialization-serde.md` | + | OBS | `15-instrumentation-and-observability.md` | + | CFG | `16-configuration.md` | + | TRANSPORT | `17-transport-adapter-conformance-contract.md` | + | ASYNC | `18-asynchronous-runtime-adapter-contract.md` | + | XCUT | `19-cross-cutting-invariants-and-policies.md` | + | NFR | `20-non-functional-requirements-and-quality-bar.md` | + + A prefix not listed here is new; derive its chapter from the Subsystem cell. + +## Two entry points + +**ID-first — you have requirement IDs.** This is the plan-task case. Plan tasks list their IDs +in the task header; pass the whole set in one call, never six. + +1. `bun run knowledge --req HTTP-13,HTTP-14,HTTP-15` — what the corpus concluded. Design-role + entries quote `docs/sdk-design-nodejs/` inline, so this usually covers the TypeScript + mapping too; add `--role design` to isolate them. Open the design doc only to follow a line + range. +2. `grep -n '^| ' docs/product-spec/appendix-c-…md` — canonical text, when the query came + back a roll-up or you need the normative wording verbatim. + +**Topic-first — you have an area, or a styleguide citation.** + +```bash +bun run knowledge --list-topics # topics, entry, ID and note counts +bun run knowledge --topic pipeline --section rules --brief cursor fork +``` + +**15 of the 38 harvested topics carry no requirement ID at all** — every styleguide-derived +one, including `data-modeling`, `error-handling`, `assertions`, `testing`, `api-design`. +ID-first cannot reach them. `--list-topics` shows which; don't work from a memorised list. + +For "styleguide 6.7" / "ch08", use `--chapter`: + +```bash +bun run knowledge --chapter 6 interface class # styleguide 6.7 → the classes chapter +``` + +Entries record a chapter file and line range, never a section number, so `--chapter 6.7` +queries chapter 6 and tells you it dropped the `.7`. Narrow with bare words instead. + +## Auditing: the one case where a broad query is right + +A lookup wants the smallest answer. An audit wants a **complete group of rules**, because a +rule it never read is a rule it never checked. The CLI applies no cap, so a group query is +complete as soon as the group is. + +Two forms: + +```bash +bun run knowledge --topic api-design,documentation,type-system --section rules --brief +bun run knowledge --prefix HTTP --section rules --brief +``` + +`--prefix` takes a whole ID family and is validated against appendix C, so a typo fails +loudly instead of returning nothing. + +**One topic is not a group.** A search of `Rules` for the word "public" hits twelve topics; an +audit that queries `api-design` alone reports clean over an incomplete set. Use the recorded +groups, and extend this table rather than improvising a group per audit — an audit is only +repeatable if its group is written down. + +| Group | Topics | +|---|---| +| API surface | `api-design`, `http-domain-model`, `documentation`, `type-system`, `module-organization`, `error-handling`, `tooling-and-quality-gates`, `styleguide-overview` | + +To build a group that is not in the table: for an ID-bearing subsystem use `--prefix`, which is +exact. Otherwise grep the corpus for the subject word (`bun run knowledge --section rules +--brief `), take the topic files that came back, and add the row here before running the +audit — an audit whose group is not written down cannot be repeated. The API-surface group's +`--section rules` is ~183 entries and ~12k tokens; budget for it. + +### The audit loop + +1. **Read the notes first** — the phase-start query above. A rule that already carries a note + is already known-broken; don't re-report it. A harvested entry that prints + `[overridden by notes/…]` is the same signal inline. +2. **Read the group.** One query, `--section rules`, from the table above. +3. **Check the system** against each rule. +4. **Write a note for each broken rule**, in `docs/knowledge/notes/.md`, naming the + rule by the key the query printed (`api-design/e0f4662b`). Backtick the key: that is how + the CLI links the two, so the harvested entry then prints `[overridden by notes/…]` and + `bun run knowledge --key api-design/e0f4662b` resolves it. The key changes exactly when the + rule's text changes — including on a re-harvest that rewords it, which is when the note + needs revisiting; `bun run knowledge:drift` reports a citation that has gone stale. Never + edit the harvested entry. + +A note's shape: a topic heading, a section, one bullet, role `review`, a source path, a manual +`sha:manual-` marker. + +```markdown +# pagination — notes + +Hand-written. `../harvested/pagination.md` is what the documents say; this is what the +implementation found, and it wins. + +## Superseded +- **What we found**, superseding `pagination/81881061`. … + review · `docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md` · high · sha:manual-6c-erratum +``` + +Which section: `## Superseded` when following the harvested rule would cause damage, +`## Conflicts` when you are recording which of two documents won, `## Reference` when the note +only points somewhere (the deviation register's pointer is the one example). A note is not for +a preference. + +**Re-harvesting.** Always `--corpus docs/knowledge/harvested`; the skill's default is +`docs/knowledge/`, which no query reads and the structure gate rejects. If the harvest emits a +`supersede` resolution, move it to `notes/` by hand — a `## Superseded` entry under +`harvested/` fails CI. + +## Never read a whole topic file + +A filtered query answers the question for a fraction of the tokens, and `grep` is not a +substitute — it has no section, role, or exact-token ID matching. Two exceptions, both narrow: + +- **An unnarrowed `--topic` costs more than the file.** Add `--section`, `--chapter`, or bare + words, or read the file — but don't run a bare `--topic` on a subsystem topic. (Two stated + exceptions: an audit group, which is deliberately broad and narrowed by `--section rules`; + and a topic `--list-topics` shows to be tiny, such as a note-only one.) +- **Following up a located entry, when you need the exact bytes.** The bullet sits at the + printed line; read from it to the next `` (usually line+1, but a multi-paragraph entry + runs longer), and stop at the section boundary. Reach for this last: a neighbouring entry is + related only about half the time. Prefer `--req` on the ID cluster (appendix C numbers + related requirements together) or `--topic X --section rules`, whose median section is small. + +## Quick reference + +| Flag | Effect | +|---|---| +| `--req HTTP-7,HTTP-8` | Entries citing any of these. Exact-token: never matches `HTTP-70`. | +| `--prefix HTTP` | A whole ID family. The audit filter; validated against appendix C. | +| `--key pagination/81881061` | The one entry with that key. How a note's citation is resolved; an unknown key means the rule was reworded. | +| `--origin harvested\|note` | Which tree. `note` is the phase-start query. | +| `--topic pipeline,retry` | Topic files by substring — matches broadly and silently. | +| `--section rules,…` | rules, constraints, conclusions, reference, conflicts, superseded. | +| `--role spec\|design\|styleguide\|review` | Provenance role. `review` only ever appears in `notes/`. | +| `--chapter 6` | Styleguide chapter. The only way in from a "styleguide N.M" citation. | +| `--grep ` / bare words | Case-insensitive; regex is real, bare words are literal. | +| `--brief` | Drop `` lines, ~30% smaller — but you lose the citation. | +| `--json` | Records, each with `origin`, `key` and a `rollup` boolean. | +| `--list-topics` | Every topic with entry, distinct-ID and note counts. | +| `--list-reqs` | ID → location map. **~6k tokens, bigger than any topic file.** Prefer `--coverage`. | +| `--coverage` | Substantive vs roll-up-only vs uncited, per prefix. A report, not a gate. | + +Zero matches exits 1 and names what does exist — nearest IDs, available topics, harvested +chapters. Follow it rather than guessing again. `--help` for the rest. + +## Citing what you find + +The `` line is the citation, but it comes in two tiers: + +- **spec / design** — repo-relative, quote verbatim: + `` docs/product-spec/09-retry-and-resilience.md:28 · sha:9efbe276001e `` +- **styleguide** — an absolute path to a sibling repo on the harvest machine + (`/home/…/styleguide/typescript/11-testing.md:110-114`). **Strip the machine prefix** before + committing it: `styleguide/typescript/11-testing.md:110-114`. Pasting it raw produces a + citation that resolves on one laptop. + +Shape is not uniform: Conflicts entries carry two sources and no sha; a note carries a manual +`sha:` marker and sometimes no line range. Copy what is there, don't assume four fields. + +Drop `--brief` whenever the result will be cited. + +## Common mistakes + +| Mistake | Fix | +|---|---| +| Editing an entry under `harvested/` to record what you found | Write a note in `notes/` naming the rule by its key. The harvested sha is per-file; your edit is invisible to the next harvest and CI rejects it. | +| Trusting a `--req` hit that is all roll-up | Watch for the WARNING; follow all three steps of the roll-up path. | +| Six sequential `--req` calls for one task | One comma-separated call. | +| Auditing one topic and reporting "clean" | One topic is not a group. Use the group table, or `--prefix`. | +| ID-first on `data-modeling` / `error-handling` / `testing` | Those carry zero IDs. Topic- or chapter-first. | +| Pasting a styleguide `` path verbatim | Strip the machine prefix first. | +| `--topic X` with nothing else | Not a filter; costs more than the file — unless `--list-topics` shows the topic is tiny. | +| Looking for the deviation register in the corpus | It is not harvested — a register goes stale on the next append. `bun run knowledge --topic deliberate-deviations` gives the pointer; read §10 itself. | +| Treating `--coverage` or `knowledge:drift` as a gate | Hand-run reports. The only CI gate here is `verify:knowledge-structure`. | diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..f08d440 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,66 @@ +# .github/dependabot.yml +# +# Version updates for the Bun workspace. +# +# Preconditions this repository meets. Dependabot's `bun` ecosystem went GA on +# 2025-02-13 -- https://github.blog/changelog/2025-02-13-dependabot-version-updates-now-support-the-bun-package-manager-ga/ +# -- and needs Bun >= 1.1.39 and the text-based `bun.lock`, not the legacy +# binary `bun.lockb`. `.bun-version` pins 1.3.14 and `bun.lock` is JSON +# (`"lockfileVersion": 1` on its second line). +# +# One hard limit and one caution, stated here rather than diagnosed later from +# a confusing PR: +# +# * HARD: `bun` covers VERSION updates only. Dependabot issues no SECURITY +# updates for this ecosystem, so an advisory against a dependency will not +# arrive as a PR. `bun run audit` (`bun audit --audit-level=high --prod`) +# is a CI step and stays the thing that catches those. +# * CAUTION: treat lockfile updates under a workspace layout as unproven. +# dependabot/dependabot-core#14223 is open (2026-02-19), titled "Dependabot +# does not fix bun.lock in environment which using npm workspace", and the +# symptom matches this repository's shape -- `workspaces.packages: +# ["packages/*"]` in the root package.json, eleven packages. It is NOT a +# confirmed defect of the configuration below: the reporter's linked config +# declares `package-ecosystem: "npm"` with `enable-beta-ecosystems: true`, +# not `bun`, so the published repro does not exercise this file. (#11602, +# closed, is the older single-package report.) If it does bite, it shows up +# as a no-op PR, or as a manifest bump with a stale lockfile -- the second +# kind fails CI at `bun install --frozen-lockfile`, the first step of the +# run, and that failure is the tooling, not the bump. Re-run `bun install` +# locally and commit `bun.lock` onto the PR branch. +# +# This file is inert until it reaches the repository's DEFAULT branch -- +# Dependabot reads its configuration only from there, and the default is `main` +# while the MVP work integrates on `mvp` (see CONTRIBUTING.md, "Branching"). +# To activate it before that merge, cherry-pick it onto `main` AND add +# `target-branch: 'mvp'` to the entry below, so the PRs land where the work is. +# Drop that line again once `mvp` has merged. +# +# No `github-actions` ecosystem block: deliberately out of scope for the ticket +# that added this file. The three actions in use -- actions/checkout@v4, +# oven-sh/setup-bun@v2, actions/setup-node@v4 -- are pinned by major tag. +version: 2 + +updates: + - package-ecosystem: 'bun' + directory: '/' + schedule: + interval: 'weekly' + day: 'monday' + # 5 is Dependabot's own default, written out so it reads as a decision + # rather than an omission. Deliberately not RAISED: every PR here runs the + # full 20-step CI, including a double clean build for the reproducibility + # gate, and the caution above means some fraction of them may be no-ops. + open-pull-requests-limit: 5 + commit-message: + prefix: 'chore' + include: 'scope' + groups: + # One PR for the routine drift. A major bump is excluded, so it arrives on + # its own branch and the breaking change gets reviewed alone. + minor-and-patch: + patterns: + - '*' + update-types: + - 'minor' + - 'patch' diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 899ac6f..495ce86 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,6 +19,16 @@ jobs: - name: Install (frozen lockfile) run: bun install --frozen-lockfile + # First of the gates: pure Node over Markdown, no build, no dist/, so a + # corpus mistake reports in seconds instead of after the whole pipeline. + # Keeps docs/knowledge/'s two trees apart — no hand-written entry under + # harvested/, whose `` shas digest whole source files and so cannot + # record an edit; the next harvest would delete it silently. The companion + # drift report is deliberately NOT here: 16 of the 47 sources are a + # sibling styleguide repository that no CI checkout has. + - name: Knowledge-corpus structure check + run: bun run verify:knowledge-structure + - name: Typecheck run: bun run typecheck @@ -28,8 +38,29 @@ jobs: - name: Build run: bun run build + # `bun run test`, not a bare `bun test`: the root script passes both test + # trees explicitly (`./packages ./tests`), and bunfig's `root = "packages"` + # means a bare invocation silently skips `tests/`. See CLAUDE.md. - name: Test (with coverage) - run: bun test --coverage + run: bun run test --coverage + + # `examples/` is not a workspace member and is not in the root test script's two trees, so + # until 2026-09-04 the petstore canary -- the witness for the codegen target surface, and the + # only assertion anywhere that an unsatisfiable AUTH requirement raises before the transport is + # touched -- compiled and lint-checked but never ran. Its own step rather than a widening of + # `bun run test`: examples are not workspace packages and their coverage does not belong in the + # 80% floor's denominator. + - name: Example canaries + run: bun run test:examples + + # The gates' own tests (`scripts/*.test.mjs`), on `node --test`. Deliberately outside + # `bun run test`: bunfig scopes discovery and the 80% coverage floor to `packages`, and that + # floor is a statement about `packages/core`, not about repo tooling. What this protects is a + # gate's logic silently degrading — a bad glob, a swallowed assertion — which no other step + # would notice, since a degraded gate still exits 0. Closes open-items H13, whose trigger had + # already fired: `knowledge.test.mjs` was failing on `main` and nothing ran it. + - name: Gate self-tests (scripts/*.test.mjs) + run: bun run test:scripts - name: API surface check run: bun run api @@ -47,18 +78,53 @@ jobs: - name: Dual JS/TS consumption check run: bun run verify:dual-consumption + - name: Consumer typecheck against the published .d.ts + run: bun run verify:consumer-types + - name: SEAM-1 zero-dependency check run: bun run verify:seam-1 + - name: Verify SSE-37/SSE-38 (no serde dependency, no reconnect path in core SSE) + run: bun run verify:sse-37 + - name: Runtime-floor consistency check run: bun run verify:runtime-floor + # Reads the five files that must agree on `tests/node-conformance/`. The rule and the reasons + # it exists live in CLAUDE.md, "HARD RULE — the `tests/` partition"; what matters here is only + # that every way it breaks is silent, so nothing else in this workflow would catch it. + - name: Test-partition check (tests/ vs tests/node-conformance/) + run: bun run verify:test-partition + + # docs/knowledge/harvested/module-organization.md:20 treats an import cycle as a bug rather + # than a style nit, and :22 requires it be gated here. Hand-written and dependency-free like + # every other verify:* gate, so it cannot be skipped by a missing install (docs/open-items.md + # K12). Type-only edges count, deliberately. + - name: Import-cycle check + run: bun run verify:import-cycles + + # NFR-12. Deliberately last in this job: it sweeps every dist/ and rebuilds + # the workspace twice, so it would otherwise pull the rug from under any + # step above that resolves a workspace package through its dist/. + - name: Reproducible-build check (NFR-12) + run: bun run verify:reproducible-build + - name: Dependency audit run: bun run audit - node-floor-conformance: + node-conformance: needs: ci runs-on: ubuntu-latest + strategy: + # Report both versions rather than stopping at the first failure: "broken on the floor" and + # "broken on LTS" are different diagnoses and the matrix exists to tell them apart. + fail-fast: false + matrix: + # The declared floor AND current LTS, which is the "in addition to current LTS" half of + # sdk-design-nodejs/09:52-54 that a floor-only pin left unexercised (checkpoint 5.9). + # `lts/*` resolves at run time, so this does not go stale as LTS moves. + node: ['20.3.0', 'lts/*'] + name: node-conformance (${{ matrix.node }}) steps: - uses: actions/checkout@v4 @@ -74,7 +140,7 @@ jobs: - uses: actions/setup-node@v4 with: - node-version: 18.17.0 + node-version: ${{ matrix.node }} - - name: Verify the built artifact against the declared minimum Node version (NFR-10/NFR-17) - run: node scripts/verify-node-floor.mjs + - name: Node-runtime conformance against the built artifact (NFR-10/NFR-17) + run: bun run test:node diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..ab97fe7 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,109 @@ +# .github/workflows/release.yml +# +# NFR-16 (SHOULD): published artifacts are cryptographically signed for provenance, with signing +# enforced on the release/CI path and gracefully optional in local builds. This workflow is that +# path. `docs/first-release.md` tracks the requirement and the blockers below. +# +# THIS WORKFLOW IS INERT UNTIL AN `NPM_TOKEN` REPOSITORY SECRET EXISTS. +# Without one, `changesets/action` cannot authenticate to the registry, so the publish step is a +# no-op: the action still opens and maintains the "Version Packages" pull request, but nothing is +# ever pushed to npm. That is deliberate. The workflow is authored ahead of the first release so +# the release path is reviewable now rather than improvised under time pressure later. +# +# Two manifest prerequisites are ALSO unmet as of 2026-09-02, and the first real publish fails +# without them: +# 1. No package.json in the workspace carries a `repository` field. npm rejects `--provenance` +# without one that resolves to the source repository. +# 2. `.changeset/config.json` sets `"access": "restricted"`. Provenance attestations go to a +# public transparency log and require a public package. +# Neither is fixed here; both are recorded so the first release does not discover them at the +# registry. +# +# How a release happens once the above are satisfied: +# 1. A change lands on `main` carrying a changeset under `.changeset/` (see CLAUDE.md — write +# one with `bun run changeset`, which names the file `YYYY-MM-DD-.md`). +# 2. This workflow runs. `changesets/action` finds unreleased changesets and opens (or updates) +# a "Version Packages" pull request that applies the version bumps and rewrites CHANGELOGs. +# 3. Merging that pull request runs this workflow again. This time there are no changesets left, +# so the action runs the `publish` command below instead, which publishes every package whose +# version is not yet on the registry. +name: Release + +on: + push: + branches: [main] + +# Two pushes to main must never publish concurrently: the second would race the first's registry +# writes and could publish a half-versioned set. `cancel-in-progress` stays false on purpose — a +# publish that has already started must be allowed to finish rather than be killed mid-registry. +concurrency: + group: release-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: false + +permissions: + # `contents: write` and `pull-requests: write` let the action open, update and push to the + # "Version Packages" pull request. `id-token: write` is what makes provenance possible at all: + # npm exchanges the GitHub OIDC token for the signed attestation. + contents: write + pull-requests: write + id-token: write + +jobs: + release: + # Belt and braces with the `on.push.branches` filter above: releases run from `main` only. + # Work on `mvp` or any other branch is never versioned or published, even if this file is + # copied to another trigger later. + if: github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + # changesets needs the full history to work out what has already been released. + fetch-depth: 0 + + # Pinned exactly the way .github/workflows/ci.yml pins it: the version in .bun-version, not + # whatever the runner ships. Bun's fetch/node:http behaviour differs enough between releases + # that an unpinned runner has produced green-here/red-there results before (see CLAUDE.md). + - uses: oven-sh/setup-bun@v2 + with: + bun-version-file: .bun-version + + - name: Install (frozen lockfile) + run: bun install --frozen-lockfile + + # Build once here so a broken tree fails before the action starts touching the registry. + # This is not the real gate: each package's own `prepublishOnly` re-runs + # `bun run build && bun run api:ci && publint . && attw --pack .` at publish time, and npm + # aborts that package's publish if any of the four fails. Everything those scripts need + # (Bun, @microsoft/api-extractor, publint, @arethetypeswrong/cli) is a root devDependency the + # frozen install above provides. + - name: Build + run: bun run build + + - name: Version pull request, or publish + uses: changesets/action@v1 + with: + # The lockfile-pinned changesets binary, invoked directly. + # + # NOT `bun run changeset publish`: that routes through scripts/changeset.mjs, a wrapper + # whose only job is renaming a newly *created* changeset file. `publish` is already in + # its passthrough list, so on this path the wrapper adds a process layer and a second + # `bunx` resolution and changes nothing. + # + # NOT `bun x changeset publish` either: `bun x` falls back to fetching from the registry + # when local resolution misses, and a release path must never publish using tooling the + # lockfile did not pin. `bun install --frozen-lockfile` ran immediately above, so + # node_modules/.bin/changeset is guaranteed present and guaranteed to be the pinned + # @changesets/cli. + publish: node_modules/.bin/changeset publish + version: node_modules/.bin/changeset version + title: 'chore: version packages' + commit: 'chore: version packages' + env: + # changesets/action writes ~/.npmrc from NPM_TOKEN when it is set, and skips publishing + # entirely when it is not — which is what makes this workflow inert today. + NPM_TOKEN: ${{ secrets.NPM_TOKEN }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # Applies to every `npm publish` the action drives, so provenance is enforced on the + # release path rather than being a flag someone can forget at one call site (NFR-16). + NPM_CONFIG_PROVENANCE: 'true' diff --git a/.gitignore b/.gitignore index 4a4078f..fab9800 100644 --- a/.gitignore +++ b/.gitignore @@ -115,3 +115,8 @@ dist # api-extractor scratch output (the committed report lives in packages/*/etc/) packages/*/temp/ + +# Scratch directory the housekeeping skill's fence check writes and removes +# (.claude/skills/housekeeping/check-fences.mjs). Present only mid-run; ignored so an +# interrupted run cannot leave an untracked tree. +.housekeeping-fences/ diff --git a/CLAUDE.md b/CLAUDE.md index 07d77fc..c1de24c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,8 +8,29 @@ A Node.js/TypeScript HTTP SDK platform, built as a **port of a language-agnostic spec in `docs/product-spec/` is normative and numbered; the code exists to satisfy it. Work here is spec-driven, not feature-driven: before implementing anything, find the requirement IDs it must satisfy. -Bun workspace. One published package today — `@dexpace/core` (`packages/core`) — with more planned per -`docs/sdk-design-nodejs/02-package-and-workspace-layout.md`. +Bun workspace, **eleven packages**. Nine are published; two are `private` and exist only to serve the build. +Every gate below runs over all of them, not over core alone — `verify:seam-1` in particular asserts zero +runtime dependencies for each, which is `NFR-2`'s "core plus at most one external library per optional +capability". + +| Package | Provides | External dependencies | +|---|---|---| +| `@dexpace/core` | Models, pipeline, seams, resilience pillars, SSE, pagination, configuration, observability | **none** | +| `@dexpace/transport-fetch` | `fetchTransport()` over the runtime's global `fetch` | none | +| `@dexpace/transport-undici` | `undiciTransport()` — pools, proxies, real `close()` | `undici` | +| `@dexpace/transport-shared` | `@internal` plumbing both transports need identically | none | +| `@dexpace/codec-json` | `jsonSerde()` — the reference wire codec | none | +| `@dexpace/body-file` | `fileBody()` over `node:fs` | none | +| `@dexpace/logging-pino` | `createPinoLogger()` | `pino` (optional peer) | +| `@dexpace/logging-debug` | `createDebugLogger()` | `debug` (optional peer) | +| `@dexpace/rx` | `Observable` views of SSE and pagination | `rxjs` (peer) | +| `@dexpace/shrink-test` | **private.** Proves the published bundles survive minify + tree-shake (`NFR-9`) | — | +| `@dexpace/transport-conformance` | **private.** The shared `TRANSPORT-N` suite both transports run | — | + +**`@dexpace/core` is a peer of every other package, never a dependency.** Two copies of core in one install +defeat the branded symbols and identity checks the seams rely on — the dual-package hazard — and +`verify:seam-1` is what enforces it. The layout is `docs/sdk-design-nodejs/02-package-and-workspace-layout.md`; +what each package is *for*, and how they compose, is `docs/sdk-documentation/architecture.md`. ## Commands @@ -18,54 +39,290 @@ All run from the repo root unless noted. ```bash bun install --frozen-lockfile -bun run typecheck # tsc --noEmit against packages/core/tsconfig.json -bun run lint # gts lint . — formatting AND type-aware rules; fatal -bun run fix # gts fix . — autofixes formatting/lint -bun run build # tsc -p packages/core/tsconfig.build.json → dist/ -bun test # coverage is on by default (bunfig.toml), 80% line floor +bun run build:core # tsc -b of core's declarations; incremental +bun run build:deps # build:core + the other packages another package's src or tests/ + # imports BY NAME; a prerequisite of the four below +bun run typecheck # build:deps, then tsc --noEmit per package +bun run lint # build:deps, then gts lint . — formatting AND type-aware rules; fatal +bun run fix # build:deps, then gts fix . — autofixes formatting/lint +bun run build # build:deps, then plain tsc for the rest → each package's dist/ +bun run test # BOTH test trees (see below); needs `build` first; coverage on, 80% line floor +bun run test:node # Node-runtime conformance against the BUILT artifact; needs `build` first ``` +**Anything that resolves a workspace package by name needs that package's `dist/` to exist**, from Phase 6a +on — a consumer reaches it only through its published entry point, and both `tsc` and Bun follow the +`types`/`main` fields there. `typecheck`, `lint`, `fix`, and `build` each run `build:deps` first for that +reason, so every one of them works on a fresh clone. Both legs are `tsc`, so a warm repeat is close to free. +Do not drop that prefix to "save a step": without it `typecheck` fails with unresolved-module errors the +moment `dist/` is absent, which is exactly what a CI runner sees. + +**`build:deps` is the list, and it grows.** It is core, `@dexpace/transport-shared`, `@dexpace/codec-json` and +`@dexpace/transport-fetch` today. A package belongs in it the moment another package's `src/` — or the top-level +`tests/` tree — imports it *by name* and its `exports` point at `dist/`. +Phase 8a proved the cost of missing one: `transport-shared` landed as the second such package, `build:core` +stayed the prefix, and CI failed on `typecheck` at the first fresh clone while every local gate stayed green +against a warm `dist/`. Phase 9 grew it twice over for one reason: `packages/shrink-test/src/` imports `codec-json` and +`transport-fetch` by name, and so does `tests/conformance/xcut/`. `@dexpace/transport-conformance` is +deliberately absent — it is `private` and its `exports` name `./src/index.ts`, so it resolves unbuilt. Check the +graph, not this sentence: + +```bash +for d in packages/*/; do grep -rhoE "from '@dexpace/[a-z-]+'" "$d/src" | sort -u; done +grep -rhoE "from '@dexpace/[a-z-]+'" tests | sort -u # the second test tree counts too +``` + +`node .claude/skills/ci-preflight/run-ci.mjs --clean` is what catches a missing entry — it sweeps every +`dist/` and `*.tsbuildinfo` first, so the run starts from the tree CI checks out rather than a warm one. +It also pins every step to `.bun-version`'s Bun by default (via mise, falling back to PATH's with a loud +banner): CI resolves that file, and Bun's `fetch`/`node:http` differ enough between releases that Phase 8a's +transport rows passed on 1.4.0 and failed three ways on the pinned 1.3.14. `--clean` plus that default is the +difference between "the gates pass here" and "CI will be green". + +**There are two test trees, and `bun run test` is the only command that runs both.** Colocated unit tests +live under `packages/*/src/`; everything that crosses a process, a network, or a *runtime* boundary lives +under `tests/`. Styleguide 11-testing scopes that rule to process and network boundaries; this repo reads a +**runtime** boundary the same way, and the Node suite is why — see the hard rule below. The root script is +`bun test ./packages ./tests` — two trees, one process, one coverage report, one exit code. + +`tests/` in turn holds one subdirectory per **runner**, and they are not interchangeable: + +``` +tests/ + conformance/xcut/ # Bun runner, part of `bun run test` + node-conformance/ # node --test, run by `bun run test:node`, against the built dist/ +``` + +**A bare `bun test` silently runs only the first tree.** `bunfig.toml`'s `[test] root = "packages"` governs +discovery, so a bare invocation never visits `tests/` and reports green over a suite it never opened, with +no "0 files matched" to notice. Explicit `./`-prefixed paths override the root — a plain `tests/...` +argument is treated as a name filter and matches nothing, which is its own quiet failure. The coverage floor +does still fire on the combined run (confirmed by raising `coverageThreshold` and watching it exit 1), so +CI's Test step is `bun run test --coverage` rather than the bare form. + +**Neither form reaches `tests/node-conformance/`**, and that is enforced by a config key rather than by the +file system — read the hard rule below before touching it. + +**Either form needs `bun run build` to have run first**, from Phase 6a on: `@dexpace/codec-json`'s tests reach +core through its published entry point, which Bun resolves to `packages/core/dist/`. On a fresh clone they +cannot resolve core at all; against a stale `dist/` they report green over yesterday's core. CI is safe — its +Build step precedes its Test step. The root `test` script deliberately does not build first, so the inner loop +stays fast; rebuild when you have changed `packages/core/src/`. + +`test:node` is a separate, thin layer under `tests/node-conformance/` that runs the same built package under +`node --test`, because Bun's Web Streams / `AbortSignal` / `Uint8Array` behavior is an independent +implementation of Node's and `src/io/` is where they diverge. **A phase that touches a runtime-divergent +surface adds a case there, not only to `bun run test`** — see `tests/node-conformance/README.md`. Cases sit +flat in that directory and are named `*.test.mjs`; the runner glob does not descend. + Single test file or single test: ```bash bun test packages/core/src/http/media-type.test.ts -bun test -t 'rejects blank input' # filter by test name +bun test -t 'rejects blank input' # filter by test name +bun test ./tests/conformance/xcut # a tests/ path needs the ./ prefix ``` -API surface (report is committed at `packages/core/etc/core.api.md`): +API surface — **nine committed reports**, one per publishable package, at `packages//etc/.api.md`. +The two private packages have none: `api-extractor` runs only where something is published. ```bash -cd packages/core && bun run api:local # regenerate the report after changing exports -cd packages/core && bun run api:ci # verify it matches — this is what CI runs +cd packages/core && bun run api:local # regenerate that package's report after changing its exports +bun run api # verify all NINE match — this is what CI runs ``` +`bun run api` chains `api:ci` across `core`, `codec-json`, `logging-pino`, `logging-debug`, `body-file`, +`transport-shared`, `transport-fetch`, `transport-undici` and `rx`, in that order. A new publishable package +adds itself to that chain and to `lint:publish`. + Release-shape and invariant gates: ```bash -bun run lint:publish # publint + attw against the built package -bun run verify:dual-consumption # plain `node` imports the built package and runs it -bun run verify:seam-1 # asserts @dexpace/core has zero runtime dependencies +bun run lint:publish # publint + attw against every built package +bun run verify:dual-consumption # plain `node` imports each built package and exercises it end to end +bun run verify:consumer-types # the built .d.ts compiles on the declared `lib` with types: [] +bun run test:node # CI runs this as a matrix over engines.node's floor and current LTS +bun run verify:seam-1 # zero runtime dependencies in EVERY package, plus the @dexpace/core + # peer-dependency rule that guards the dual-package hazard +bun run verify:sse-37 # no serde dependency and no reconnect path in core SSE bun run verify:runtime-floor # tsconfig target vs package engines.node consistency +bun run verify:test-partition # the five files that keep tests/ and tests/node-conformance/ apart +bun run verify:import-cycles # no import cycle in any package's src/; type-only edges count +bun run verify:knowledge-structure # docs/knowledge/'s two trees stay separate (see below) +bun run verify:reproducible-build # two clean builds of one source tree agree, dist/ and tarball (NFR-12); + # in CI it runs after every step that resolves a package through dist/, + # because it sweeps and rebuilds them all +bun run test:scripts # the gates' OWN tests (node --test scripts/*.test.mjs) bun run audit # bun audit --audit-level=high --prod ``` -**Every one of these is a blocking CI step** (`.github/workflows/ci.yml`). Run the full set before claiming -work is done — `bun test` passing is not sufficient evidence. +**Every one of these is a blocking CI step.** `.github/workflows/ci.yml` is **22 named steps across two +jobs** — 19 in `ci`, 3 in the `node-conformance` matrix that runs after it. Run the full set before claiming +work is done; `bun run test` passing is not sufficient evidence, and the one command that runs all of them in +CI's order is: -## Documentation hierarchy +```bash +node .claude/skills/ci-preflight/run-ci.mjs --clean +``` + +The per-step reasoning, including why `verify:reproducible-build` must run last and why `test:scripts` is +blocking at all, is `docs/sdk-documentation/quality-gates.md`. + +`test:scripts` tests the *gates themselves* — the knowledge CLI, `verify-seam-1.mjs`, `verify-sse-37.mjs`, +`verify-knowledge-structure.mjs`, `verify-test-partition.mjs`. Phase 10 made it a blocking CI step, closing +`docs/work/mvp/2026-09-04-open-items-dissolution.md` H13. It was not one before, and the proof that it should have been is that +`knowledge.test.mjs` had been failing on `main` since `36c3f96` with nobody noticing. A gate whose own logic +degrades still exits 0, so nothing else in the run would. + +### HARD RULE — the `tests/` partition + +`tests/` holds two suites. They must never run together. `tests/conformance/` runs on Bun, as part of +`bun run test`. `tests/node-conformance/` runs on `node --test`, through `bun run test:node`, against the +built `dist/`. It **must not** run on Bun. That is the only reason the tree exists. -Four distinct trees, easy to confuse: +Before Phase 10, the file system held this separation. The Node tree was at `test/`, and no Bun command +could reach it. One path — `tests/node-conformance/` — now holds it instead, written into five files that +must agree: -| Path | Role | +| File | What it holds | |---|---| -| `docs/product-spec/` | **Normative.** Numbered requirements (`HTTP-7`, `SEAM-1`, `RETRY-13`, `NFR-5`, …). The source of truth. | -| `docs/sdk-design-nodejs/` | How each spec area maps to idiomatic TypeScript. Non-normative but binding by convention. | -| `docs/knowledge/` | Harvested styleguide + spec knowledge, topic-indexed (`INDEX.md`). Cited as "styleguide 6.7", "ch08". | -| `docs/superpowers/specs/` + `plans/` | Per-phase design doc, task-by-task implementation plan, and a requirement-coverage checklist. | +| `bunfig.toml` | `[test] pathIgnorePatterns` — keeps the Node tree out of `bun test` | +| `package.json` | the `test:node` glob — the only command that runs the Node tree | +| `eslint.config.js` | the `.mjs` override — without it, `console`, `URL`, and the Web Streams globals fail `no-undef` | +| `.claude/skills/ci-preflight/run-ci.mjs` | three globs in the `--node-floor` path | +| `tests/node-conformance/README.md` | the membership rule, and the paths that name the tree | + +**The key is `pathIgnorePatterns`. The key is not `testPathIgnorePatterns`.** Bun accepts an unknown +`[test]` key without complaint. A wrong key gives no warning, does not fail, and does not stop the run. Bun +then collects the Node suite. Bun runs `node:test` files without an error and reports them as passing. The +run reports success over a suite that proves nothing about Node. Measured on `bun run test`, pinned Bun +1.3.14, 2026-09-04: with the key, 165 files and 2216 tests; without it, 179 files. Thirteen of the fourteen +extra files pass silently; the run goes red only because the fourteenth trips an unrelated timer assertion in +`tests/node-conformance/config-primitives.test.mjs`, which points nowhere near the cause. Treat the exit code as an accident, not a control. + +Never change one of these five files alone. Change one, then change all of them. Then run +`node scripts/verify-test-partition.mjs`. That gate catches the wrong key name, and CI blocks on it. + +Do not remove the bunfig key and narrow the root script to `bun test ./packages ./tests/conformance` +instead. That protects the root script only. A command typed by hand, such as `bun test ./tests`, would +still collect the Node suite. The gate checks for this. + +Keep `[test] root = "packages"`. It controls discovery for a bare `bun test`, and it keeps +`scripts/*.test.mjs` out of *that* run's coverage floor. It is a second mechanism, and it is independent. +It does not replace the ignore glob, which is what governs the explicit `./tests` path the root script +passes. The gate checks this too. + +## Documentation hierarchy + +`docs/README.md` is the index and the contract; this is the working summary. Every entry in `docs/` is below, +because the one that used to be omitted — the open-items register — was the largest file in the tree +until it was dissolved on 2026-09-04. + +| Path | Role | Writable? | +|---|---|---| +| `docs/product-spec/` + `docs/product-spec.md` | **Normative.** Numbered requirements (`HTTP-7`, `SEAM-1`, `RETRY-13`, `NFR-5`, …). The source of truth; the `.md` is its table of contents. | **frozen** | +| `docs/sdk-design-nodejs/` + `docs/sdk-design-nodejs.md` | How each spec area maps to idiomatic TypeScript. Non-normative but binding by convention. §10 is the **normative deviation ledger**. | **frozen** | +| `docs/knowledge/harvested/` | Harvested styleguide + spec knowledge, topic-indexed (`INDEX.md`). Cited as "styleguide 6.7", "ch08". Generated; never hand-edited. | **frozen** | +| `docs/knowledge/notes/` | What the implementation found, hand-written, role `review`. Overrides a harvested entry. | **frozen** | +| `docs/sdk-documentation/` | **As-built.** How the packages compose, which one to install, worked cross-package examples. Eleven files; `architecture.md` is the front door. | yes | +| `docs/work//phaseN/` | Per-phase design doc, implementation plan and requirement-coverage checklist. `mvp/` is the only delivery so far. | yes | +| `docs/superpowers/` | The **inbox** the `brainstorming` and `writing-plans` skills hard-code. Drained into `docs/work/`; never a citation target. | yes | +| [`docs/work/mvp/2026-09-04-open-items-dissolution.md`](docs/work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record, not a register.** The dissolved open-items register, moved here whole on 2026-09-04 with every open question in it decided. Nothing is appended to it; its item IDs stay reserved and still resolve, because they are cited from source. | no — archive | +| `docs/first-release.md` | **Live.** Release-readiness: what the release path already does, and the blockers that must clear before a first publish. Was the `NFR-16` row of the dissolved deferral register. | yes | +| `docs/deviations.md` | **Register.** The as-built audit of §10, and where a deviation found outside a phase lands. | yes | +| `docs/audit-67-decisions.md` | **Ledger.** Decisions taken during the audit #67 remediation run, and the release-machinery work it deferred. | yes | +| `docs/assets/` | Vendored wordmark SVGs the root `README.md` renders. | yes | + +**Frozen means a maintenance tool refuses to write there**, not merely that you should not. The +`housekeeping` skill's guard (`.claude/skills/housekeeping/guard.mjs`) enforces it and `guard.test.mjs` +proves it; the per-tree reasons are in `docs/README.md`. + +**Which register. There are two, and neither is a general-purpose one, from 2026-09-04.** All three +registers were dissolved that day. A **deviation** — a place this port deliberately differs from the +reference contract — goes to `docs/deviations.md`. A **release blocker**, or a decision that is only free +before the first version bump, goes to `docs/first-release.md`. **Everything else goes where it is +enforced**: a gate, a test, or a TSDoc comment on the thing it concerns. Do not open a third register; the +lesson of the two that were dissolved is that a concern only a register remembered was a concern nothing +acted on — twenty items came to name a phase that had shipped without doing the work. A **deviation** goes in the owning phase spec's own `## Deviation Ledger` section, is consolidated +into §10, and is audited by `deviations.md` — which is also where a deviation with no owning phase lands, +since §10 sits in a frozen tree. + +**Never renumber an open-item ID, and never reuse one.** They are cited from source comments, tests, +changesets and the `docs/` tree — `K11` in `packages/core/src/index.ts`, `V13` in `config/clock.ts`, `H8` in +`io/index.ts`. Since the register was dissolved they resolve against two dated archives rather than a live +file: the `### ` headings in `docs/work/mvp/2026-09-04-open-items-dissolution.md`, and the +`## Purged item IDs` table in `docs/work/mvp/2026-09-04-register-retirement-purge.md`. The probe's citation +check reads both, and matches the old `open-items.md K11` spelling as well as the archive path, because +`docs/work/` is never retro-edited and every phase record still carries the old one. + +**Do not write the number of them into a document.** Three documents once stated three different, all-wrong +counts (`docs/work/mvp/2026-09-04-open-items-dissolution.md` U10). One command derives it, from the same regex and file set the check uses: + +```bash +node .claude/skills/housekeeping/probe.mjs --only=citations +``` + +That is also the check that every citation still resolves. U6 records what it found the first time it ran. `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md` is the fastest way to locate a requirement ID. +### Querying `docs/knowledge/` + +`docs/knowledge/` is two trees and 39 topics — never read a topic file whole when a filtered query +answers the question. `bun run knowledge` parses both trees into entries and filters them; a requirement-ID +query returns ~170 tokens against a ~5700-token file read. + +```bash +bun run knowledge --req HTTP-13,HTTP-14,HTTP-15 # a whole task's IDs in one call (exact-token) +bun run knowledge --origin note --brief # start of a phase: everything the implementation found +bun run knowledge --prefix HTTP --section rules # an audit group: a whole ID family, uncapped +bun run knowledge --chapter 6 interface class # a "styleguide 6.7" citation +``` + +Different filters AND together, values within one filter OR; `--help` lists the rest. Each result carries its +`` provenance line — the citation for test-file headers and deferral notes, though styleguide paths are +absolute to a sibling repo and need their machine prefix stripped first — and a stable key, `/<8 hex>`, +digested from the entry text, which is how a note names the rule it overrides. **A `--req` hit is not proof of +knowledge:** 256 of 645 IDs are named only by an appendix-B conformance roll-up, tagged `[appendix-B roll-up]` +in output; 385 have a substantive entry and 4 are cited nowhere at all (`--coverage` breaks this down). 15 of +the 38 harvested topics carry no requirement ID at all and are reachable only via `--topic`/`--chapter` +(`--list-topics`). Every count in this paragraph moves when the corpus is edited, so +`scripts/knowledge.test.mjs` pins all four against the live corpus and its failure message names the two docs +to update alongside. No CI step gates corpus *content*; CI does run the CLI's own suite (`test:scripts`), +which parses the real corpus. The `.claude/skills/knowledge-lookup` skill carries the full workflow. + +**Two trees, and the split is a CI gate.** `docs/knowledge/README.md` is the contract; the short version is +that `harvested/` is `knowledge-harvest`'s output and is never hand-edited, because a `` sha digests the +whole source file rather than the entry — an edit inside an entry changes no sha, and the next harvest +regenerates or duplicates it. Record what the implementation found in `docs/knowledge/notes/.md` +instead: role `review`, a manual `sha:` marker, and a backticked `/<8 hex>` key naming the harvested +rule, which makes that rule print `[overridden by notes/…]` in every query result. + +`bun run verify:knowledge-structure` (blocking, in CI) keeps the trees apart: no `review` or invented role +under `harvested/`, no `Superseded` entry there, every harvested entry carrying a `` that cites one of +the three source roots `SOURCES.md` names, every note carrying `review`, and no `.md` stranded at the root of +`docs/knowledge/` — the CLI reads the two trees only, so a file there is invisible rather than wrong, and that +is exactly where a `--corpus`-less harvest run lands. + +`bun run knowledge:drift` is the hand-run companion, deliberately not in CI. It reports source drift +(`OK` / `DRIFT` / `NOT VERIFIABLE` / `UNREADABLE` per `SOURCES.md` row) and stale note citations (a key no +entry carries any more). Not a gate: the styleguide root is a sibling repository at an absolute path, so 16 of +the 47 sources are absent from any CI checkout, and drift is normal — a design chapter a phase edits to record +an outcome *should* drift, and the fix is a re-harvest. + +**A ledger is not harvested.** `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` +is a ledger that every phase appends to, so any harvest of it is a stale fraction of +it. Read it directly; `notes/deliberate-deviations.md` is the pointer. When you re-harvest, point the skill +at the harvested tree — `--corpus docs/knowledge/harvested` — and hand-move any `supersede` entry it emits +into `notes/`. + +**Citations into the corpus** are written `docs/knowledge/harvested/.md:`. `docs/work/` is the +exception: its phase designs, plans and checklists are dated records of what was true when they were written, +are never retro-edited, and so still carry pre-split paths — 206 of them, across 33 files, measured +2026-09-02 with `grep -rhoE "docs/knowledge/[a-z0-9-]+\.md" docs/work | wc -l` (`docs/work/mvp/2026-09-04-open-items-dissolution.md` +O3, which carried 207 until the same date). + ## Requirement-ID conventions (enforced by review, not tooling) - Every source file opens with `// SPDX-License-Identifier: MIT` on **line 1** (NFR-13). @@ -96,8 +353,13 @@ catch: factories instead. - **Required fields go through `requireField()`** from `builder.ts` — never a bespoke `if (!x) throw`. It single-sources HTTP-4's `` `${name} is required` `` message. -- **Typed errors only.** Everything descends from `DomainModelError`; no bare `throw new Error(...)`. Each - subclass sets `this.name = new.target.name`, and wrap-and-rethrow always passes `{cause}`. +- **Typed errors only.** Everything descends from `DexpaceError`, the single root; no bare + `throw new Error(...)`. Each subclass sets `this.name = new.target.name`, and wrap-and-rethrow always + passes `{cause}`. **Two levels, not three** — a leaf's own superclass is `DexpaceError` itself. Group a + family with an exported type guard (`isDomainModelError`, `isIoError`, `isBodyError`), never with an + intermediate class; `DomainModelError` was exactly that intermediate class and was removed on 2026-09-04. + The one surviving middle tier is `TransportFailureError extends IoError`, which `TRANSPORT-20` requires + and `retry/classify.ts`'s cause-walk is load-bearing on. - **Getters return frozen or freshly-copied values.** `Request.url` clones on every access because the native `URL` is mutable — the one place a frozen class still leaks mutability (HTTP-5). @@ -131,11 +393,50 @@ Anything the barrel exports needs a TSDoc block with `@public`, plus `@throws` n class on operations that throw. `api-extractor` will otherwise flag it, and the committed report records it as `(undocumented)`. After changing exports: rebuild, run `api:local`, and commit the regenerated report. -Consumer-facing changes need a changeset (`bunx changeset`). +Consumer-facing changes need a changeset — `bun run changeset`, not `bunx changeset`. The wrapper +(`scripts/changeset.mjs`) forwards every argument to the CLI, then renames the file it generates from +`@changesets/write`'s random `human-id` name to `YYYY-MM-DD-.md`, the same name shape every document +under `docs/work/mvp/` carries. The slug is prompted for, defaulting to the changeset's own first +sentence. Nothing reads the filename back — the CLI globs `.changeset/*.md` and decides from the +frontmatter — so a hand-written changeset just needs to be named the same way. ## Phase workflow -Work proceeds phase by phase against `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`. Each +Work proceeds phase by phase against `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`. Each phase has a design spec, an implementation plan with numbered tasks (TDD: write the failing test, confirm it fails, implement, confirm it passes, commit), and a checklist mapping every requirement ID to the task that satisfies it. When asked to implement or validate a phase, read all three before touching code. + +Starting a numbered task means starting with what the corpus already knows about its requirement IDs — invoke +the `knowledge-lookup` skill, which carries both entry points (ID-first via appendix C, topic-first for the +styleguide-derived areas that carry no IDs). + +**A phase's documents are written into `docs/superpowers/` and do not stay there.** The Superpowers +`brainstorming` and `writing-plans` skills hard-code `docs/superpowers/{specs,plans}/` +(`brainstorming/SKILL.md:100`, `writing-plans/SKILL.md:18`); they are installed globally, shared across +projects, and this repository cannot change them. So that directory is an inbox, and the `housekeeping` skill +collects from it into `docs/work//phaseN/`. Cite the `docs/work/` path — the one the document will +have for the rest of its life — never the staging path. + +## Documentation upkeep + +Nothing gates `CLAUDE.md` or `README.md`, and both had drifted for nine phases before 2026-08-31: "two +published packages" against eleven, two API reports against nine, a gate list missing `verify:sse-37`, a +documentation table missing the largest file in the tree. The `housekeeping` skill is the check. + +```bash +node .claude/skills/housekeeping/probe.mjs # read-only. Eight checks. Always first. +node .claude/skills/housekeeping/apply.mjs # dry run; --write performs the git mv calls +bun run build && node .claude/skills/housekeeping/check-fences.mjs # typecheck every doc code fence +``` + +It derives each repository fact once — the package list, the `verify:*` gates, the named CI steps, the API +reports, the `docs/` tree — and checks every document that states it against that one derivation. It also +finds phase documents left in the inbox, Markdown stranded at the repository root, a publishable package +with no README, a broken relative link, an aggregate register left in a specification document, and a +an open-item citation that resolves to nothing. + +It is a **hand-run** tool, not a CI step. Run it after landing a phase, and before claiming the documentation +is current. Its apply stage moves files; the prose it reports on is edited by you. It refuses to write to +`docs/knowledge/`, `docs/product-spec/`, `docs/sdk-design-nodejs/` or the two sibling tables of contents, and +that refusal is a tested guard rather than a stated intention. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..594b673 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,40 @@ +# Code of Conduct + +## Our pledge + +We as members, contributors, and maintainers pledge to make participation in +the dexpace Node.js SDK a harassment-free experience for everyone, regardless +of age, body size, visible or invisible disability, ethnicity, sex +characteristics, gender identity and expression, level of experience, +education, socio-economic status, nationality, personal appearance, race, +religion, or sexual identity and orientation. + +## Our standards + +Examples of behavior that contributes to a positive environment: + +- Showing empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Focusing on what is best for the community + +Examples of unacceptable behavior: + +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information without explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the maintainers at +[oaljarrah@dexpace.org](mailto:oaljarrah@dexpace.org). All complaints will be +reviewed and investigated promptly and fairly. Maintainers are obligated to +respect the privacy and security of the reporter of any incident. + +## Attribution + +This Code of Conduct is adapted from the +[Contributor Covenant](https://www.contributor-covenant.org), version 2.1. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..565f107 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,75 @@ +# Contributing + +Thanks for your interest in the Dexpace Node.js SDK. External pull requests +are welcome — this page covers everything you need to get a change merged. + +## Setup + +The repository is a [Bun](https://bun.sh)-managed workspace of eleven +packages, nine of them published. One install provisions everything along +with the dev toolchain. The Bun version is pinned in `.bun-version`, which +CI resolves — use it: + +```bash +git clone https://github.com/dexpace/nodejs-sdk.git +cd nodejs-sdk +bun install --frozen-lockfile +``` + +## Quality gates + +Every pull request must pass the same 20 steps CI runs, across two jobs and +on both Node 20.3 and current LTS. One command runs all of them locally, in +CI's own order: + +```bash +node .claude/skills/ci-preflight/run-ci.mjs --clean +``` + +Run it before opening a PR; `--clean` starts it from the tree CI checks out +rather than a warm one. A consumer-facing change also needs a changeset — +`bun run changeset`, not `bunx changeset`, because the wrapper renames the +generated file — and a change to a package's exports needs its API report +regenerated with `api:local` in that package and committed. + +## Conventions + +The full convention set lives in [`CLAUDE.md`](CLAUDE.md). The essentials: + +- **Branch off `mvp`, not `main`.** `mvp` is the integration branch and + merges into `main` when the MVP is complete; GitHub still offers `main` + as the base, so change it. +- **`bun run build` before `bun run test`.** Every package reaches + `@dexpace/core` through `packages/core/dist/`; without a build the tests + cannot resolve it, and against a stale one they pass over yesterday's core. +- **`bun run test` is the only invocation that reaches both test trees** + (`bun test ./packages ./tests`) — a bare `bun test` silently runs + `packages/` alone. `bun run test:node` is the separate Node-runtime suite. +- **ESM-only, NodeNext**: relative imports carry `.js` even in `.ts` source, + type-only imports need `import type`, and `erasableSyntaxOnly` rules out + enums and namespaces. +- **No new runtime dependencies.** Every published package ships a + hard-committed empty `dependencies`; new third-party needs belong behind + the `Transport` or `Serde` seams, or in a new adapter package (SEAM-1, + gate-enforced). +- **MIT licence header** (`// SPDX-License-Identifier: MIT`) on line 1 of + every source file, src and tests alike; functions capped at 70 lines. + +## Commit messages + +Use the prefixes the history already follows: + +| Prefix | Use for | +|----------|----------------------------------| +| `feat:` | new features | +| `fix:` | bug fixes | +| `chore:` | refactors and cleanup | +| `docs:` | documentation-only changes | +| `test:` | tests only | +| `ci:` | CI configuration | + +## Reporting issues + +Open one at [github.com/dexpace/nodejs-sdk/issues](https://github.com/dexpace/nodejs-sdk/issues). +For security vulnerabilities, follow [`SECURITY.md`](SECURITY.md) instead of +opening a public issue. diff --git a/LICENSE b/LICENSE.md similarity index 86% rename from LICENSE rename to LICENSE.md index d75d8bf..1724c32 100644 --- a/LICENSE +++ b/LICENSE.md @@ -1,6 +1,6 @@ -MIT License +# MIT License -Copyright (c) 2026 dexpace +Copyright (c) 2026 dexpace and Omar Aljarrah Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -9,13 +9,13 @@ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/README.md b/README.md index 638b778..3f4323d 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,310 @@ -# nodejs-sdk -NodeJS SDK paltform by dexpace +

+ + + dexpace + +

+ +

Dexpace Node.js SDK

+ +[![CI](https://github.com/dexpace/nodejs-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/dexpace/nodejs-sdk/actions/workflows/ci.yml) +[![Node >=20.3](https://img.shields.io/badge/node-%3E%3D20.3-blue.svg)](https://nodejs.org/) +[![TypeScript strict](https://img.shields.io/badge/typescript-strict-blue.svg)](https://www.typescriptlang.org/tsconfig#strict) +[![Lint: gts](https://img.shields.io/badge/lint-gts-blue.svg)](https://github.com/google/gts) +![License: MIT](https://img.shields.io/badge/license-MIT-green.svg) + +A toolkit for building Node.js HTTP client libraries. It provides immutable request and response +models, a staged policy pipeline, pluggable transports, and an authentication pillar that speaks +OAuth bearer tokens and RFC 7616 Digest. Everything is typed end to end under `strict` plus +type-aware lint, ships ESM only, and targets Node 20.3 or later. + +The SDK is deliberately not an HTTP client. It defines the contracts — `Transport`, `Serde`, +`PaginationStrategy`, `Logger` — and supplies the models, policies and observability hooks that +surround them; the networking itself arrives through a transport package of your choosing. Pick the +adapter that fits your dependency budget, or write your own: the interface is two methods. + +## Packages + +A Bun workspace of eleven packages. Nine are published; `@dexpace/core` is a **peer** of every one of +the others, never a dependency, so a consumer can never end up with two copies of it. + +| Package | Provides | Third-party dependencies | +|---|---|---| +| `@dexpace/core` | Models, pipeline, seams, resilience pillars, SSE, pagination, configuration, observability | **none** | +| `@dexpace/transport-fetch` | `fetchTransport()` over the runtime's global `fetch` | none | +| `@dexpace/transport-undici` | `undiciTransport()` — connection pools, proxies, real `close()` | `undici` | +| `@dexpace/transport-shared` | Plumbing both transports need identically; not installed directly | none | +| `@dexpace/codec-json` | `jsonSerde()` — the reference wire codec, PATCH tri-state included | none | +| `@dexpace/body-file` | `fileBody()` — a file-backed request body over `node:fs` | none | +| `@dexpace/logging-pino` | `createPinoLogger()` | `pino` (optional peer) | +| `@dexpace/logging-debug` | `createDebugLogger()` | `debug` (optional peer) | +| `@dexpace/rx` | `Observable` views of SSE and pagination | `rxjs` (peer) | + +Two more are `private` and never published: `@dexpace/shrink-test`, which proves the published +bundles survive minify and tree-shake, and `@dexpace/transport-conformance`, the shared `TRANSPORT-N` +suite both transports run so they cannot drift apart. + +Install the core plus whichever transport you need: + +```sh +bun add @dexpace/core @dexpace/transport-fetch +``` + +## Quick start + +### A minimal request + +```typescript +import {Request} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const transport = fetchTransport(); + +const response = await transport.send( + Request.newBuilder().url('https://httpbin.org/get').build(), +); +try { + console.log(response.status.code, await response.text()); +} finally { + await response.close(); // the caller owns the body, always +} +``` + +### A POST with a JSON body + +```typescript +import {Request, serdeBody} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; + +const request = Request.newBuilder() + .method('POST') + .url('https://httpbin.org/post') + .body(serdeBody({hello: 'world'}, jsonSerde())) // sets Content-Type: application/json + .build(); +``` + +### A configured pipeline + +`standardResilience()` returns a `Runtime` pre-wired with all four pillars in the order `AUTH-27` +requires — redirect wraps retry wraps auth — so a retry re-resolves credentials and a redirect hop +re-stamps them. Every slot is optional; an omitted one takes that pillar's own defaults. + +```typescript +import { + createAuthDescriptor, + createAuthRequirement, + createBearerToken, + standardResilience, +} from '@dexpace/core'; +import {undiciTransport} from '@dexpace/transport-undici'; + +declare function mintToken(): Promise; + +const client = standardResilience(undiciTransport({agentOptions: {connections: 32}}), { + retry: {settings: {maxAttempts: 5, totalTimeoutMs: 30_000}}, + redirect: {maxHops: 3}, + auth: { + credentials: { + bearer: {provider: async () => createBearerToken(await mintToken()), marginMs: 60_000}, + }, + tiers: {client: createAuthDescriptor([createAuthRequirement('OAUTH2')])}, + }, +}); +``` + +`PipelineBuilder` enforces stage ordering and the one-step-per-pillar rule, and supports surgical +edits anchored on a step's `type` symbol: `insertBefore`, `insertAfter`, `replace`, `remove`. +`PipelineBuilder.seedFrom(runtime, 'flatten' | 'nest')` layers your own steps onto the preset. + +### Streaming and replayable bodies + +```typescript +import {byteArrayBody, materialize, streamBody, stringBody} from '@dexpace/core'; +import {fileBody} from '@dexpace/body-file'; + +declare const stream: ReadableStream; + +byteArrayBody(new Uint8Array([1, 2, 3])); // replayable +stringBody('{"hello":"world"}', 'application/json'); // replayable +fileBody('upload.bin', {start: 0, count: 4096}); // replayable; fresh handle per send +const once = streamBody(stream); // single-use: a retry cannot re-send it + +const many = await materialize(once); // buffer it once, deliberately, to make it retryable +``` + +Buffering an arbitrarily large upload to make it retryable is a decision for the caller who knows how +large it is, not for the retry engine — so a retryable body arrives at the retry pillar already +retryable. + +## Architecture + +A request flows down through ordered `Step`s and back up through their post-processing. The terminal +stage hands it to a `Transport`. + +``` +caller → Runtime ──┬─ PRE_REDIRECT · REDIRECT · POST_REDIRECT + ├─ PRE_RETRY · RETRY · POST_RETRY + ├─ PRE_AUTH · AUTH · POST_AUTH + ├─ PRE_LOGGING · LOGGING · POST_LOGGING + ├─ PRE_SERDE · SERDE · POST_SERDE + └─ SEND → Transport → wire +``` + +Sixteen stages in `STAGE_ORDER`. Five of them — `REDIRECT`, `RETRY`, `AUTH`, `LOGGING`, `SERDE` — are +**pillars**: each admits exactly one step and raises on a second. The `PRE_`/`POST_` stages around +them stack, and are the user-extensible slots. + +`Runtime` implements `Transport`, so a pipeline is substitutable wherever a transport is — which is +what makes nesting, `seedFrom`, and driving a `Paginator` over a full pipeline work. + +Bottom-up, the layers are: + +1. **Bodies.** A request `Body` is a producer: `writeTo(sink)` emits on demand, `replayable` decides + whether a retry may re-send. A response body is a `ReadableStream` the **caller** owns and closes. +2. **Models.** `Request`, `Response`, `Headers`, `QueryParams`, `RequestOptions` and + `RequestConditions` are frozen at construction and reachable only through a builder, so validation + cannot be routed around and behaviour is identical under every transport. +3. **Context.** `DispatchContext` promotes to `RequestContext` then `ExchangeContext`, carrying one + `InstrumentationBundle` throughout; propagation is `AsyncLocalStorage`-based. +4. **Pipeline.** `Step`, `Next`, `StepContext`, `StepDescriptor`, `PipelineBuilder`, `Runtime`. +5. **Transport.** `send()` and `close()`. That is the whole contract. + +## Inside `@dexpace/core` + +| Module | Surface | +|---|---| +| `http/` | `Request`, `Response`, `Headers`, `HeaderName`, `Status`, `Protocol`, `MediaType`, `ETag`, `HttpRange`, `QueryParams`, `RequestOptions`, `RequestConditions` | +| `body/` | `byteArrayBody`, `stringBody`, `formUrlEncodedBody`, `multipartBody`, `streamBody`, `serdeBody`, `materialize`, `TypedResponse`, `HttpStatusError`, `toHttpError` | +| `pipeline/` | `Stage`, `STAGE_ORDER`, `PILLAR_STAGES`, `Step`, `Next`, `StepContext`, `StepDescriptor`, `PipelineBuilder`, `Runtime` | +| `retry/` | `retryStep`, `RetrySettings`, `BackoffSettings` — exponential backoff with jitter, `Retry-After` awareness, injectable `Clock`/`random` | +| `redirect/` | `redirectStep`, `RedirectSettings`, `RedirectPredicate` — loop detection, hop cap, downgrade guard, credential stripping | +| `auth/` | `authStep`, `standardResilience`, `createAuthDescriptor`, `createAuthRequirement`, `ApiKeyCredential`, `NameKeyCredential`, `BearerToken`, RFC 7235 challenges, RFC 7616 Digest | +| `serde/` | `Serde`, `Serializer`, `Deserializer`, `Schema`, `Tristate`, `decodeResponse`, `decodeSuccessResponse` | +| `sse/` | `sseStreamFrom`, `SseStream`, `SseEvent`, `typedSseStream` — WHATWG-compliant, bounded line buffer | +| `pagination/` | `Paginator`, `Page`, `PaginationStrategy`, `cursorStrategy`, `pageNumberStrategy`, `linkHeaderStrategy`, `paginateWithFetchers` | +| `config/` | `Configuration`, `ConfigurationBuilder`, `Clock`, `ProxyOptions`, `getBuildInfo`, HTTP-date parsing | +| `observability/` | `Logger`, `createLogger`, `LogEvent`, `Tracer`, `Span`, `Meter`, `loggingStep`, URL redaction, no-op singletons | +| `context/` | `DispatchContext` → `RequestContext` → `ExchangeContext`, `InstrumentationBundle` | +| `seams/` | `Transport`, `Serde`, `OperationDescriptor`, `buildRequest`, `composeSignal`, `isTimeoutSignal` | + +## Highlights + +- **Zero runtime dependencies, and it is a gate.** `@dexpace/core` takes none, and + `bun run verify:seam-1` asserts that for **every** package in the workspace plus the + `@dexpace/core`-as-peer rule that guards the dual-package hazard. +- **Immutable models, no public constructors.** Builders only; the emitted `.d.ts` declares each + constructor `private`, so a consumer cannot construct around `build()`'s validation. Deriving + deep-copies every collection rather than aliasing. +- **Pluggable everything, registered nothing.** `Transport`, `Serde`, `Schema`, + `PaginationStrategy`, `Logger`, `Tracer`, `Meter` and `Clock` are duck-typed — a conforming object + is a valid implementation, with no registry, no discovery and no install step. +- **Retry done right.** Exponential backoff with jitter, server pacing hints (`Retry-After`, + `X-RateLimit-Reset`) in a fixed precedence, an opt-in total-timeout budget, and deterministic tests + through an injectable `Clock`. +- **Redirects done right.** Loop detection, hop cap, `Authorization` stripped across origins, + HTTPS→HTTP downgrade refused by default, and the transport pinned to never follow a hop itself, so + the pipeline is the single redirect authority. +- **Real auth.** OAuth bearer with serialized concurrent refresh, an RFC 7235 `WWW-Authenticate` + parser, RFC 7616 Digest (MD5, MD5-sess, SHA-256, SHA-256-sess), Basic and key credential — with + credentials refused over plaintext and redacted in every `toString` and inspect path. +- **PATCH tri-state.** `Tristate` distinguishes absent, null and present, so `{}` and + `{"x": null}` stop being the same wire message. Wired into `@dexpace/codec-json` by default. +- **Server-Sent Events and pagination.** A WHATWG-compliant SSE parser with a bounded line buffer and + no reconnect path in core (gate-enforced), and a `Paginator` that walks item-by-item or page-by-page + over pluggable strategies. +- **Observability that costs nothing when off.** `NOOP_LOGGER`, `NOOP_TRACER` and `NOOP_METER` are + the defaults; a suppressed event never builds its field map. +- **Proven against Node, not just Bun.** A separate conformance suite runs the built artifact under + `node --test`, as a matrix over the declared floor and current LTS, because Bun's Web Streams and + `AbortSignal` are an independent implementation. + +## Development + +A [Bun](https://bun.sh) workspace, pinned by `.bun-version` (1.3.14). One install provisions every +package. + +```bash +git clone https://github.com/dexpace/nodejs-sdk.git +cd nodejs-sdk +bun install --frozen-lockfile +``` + +```bash +bun run build # every package's dist/ +bun run typecheck # tsc --noEmit, per package +bun run lint # gts — formatting AND type-aware rules, both fatal +bun run test # both Bun test trees, one coverage report, 80% line floor +bun run test:node # the built artifact under node --test +bun run api # every committed etc/*.api.md matches +``` + +Twenty-two named CI steps across two jobs, every one blocking +([`.github/workflows/ci.yml`](.github/workflows/ci.yml)). Run all of them locally before claiming +work is done: + +```bash +node .claude/skills/ci-preflight/run-ci.mjs --clean +``` + +`--clean` sweeps every `dist/` and `*.tsbuildinfo` first, so the run starts from the tree CI checks +out rather than a warm one, and pins every step to `.bun-version`'s Bun. Both matter: a transport +suite has passed on one Bun release and failed three ways on the pinned one. + +## Conventions + +The full contract is in [`CLAUDE.md`](CLAUDE.md); the documentation map is +[`docs/README.md`](docs/README.md). The short version: + +- **Spec-driven, not feature-driven.** [`docs/product-spec/`](docs/product-spec/) is normative and + numbered; the code exists to satisfy it. Before implementing anything, find the requirement IDs. +- **ESM only, `NodeNext`.** Relative imports carry `.js` even in `.ts` source; + `verbatimModuleSyntax` is on. No enums, no namespaces, no parameter properties. +- **`#private` fields, private constructors, `Object.freeze(this)`.** Every domain model follows one + construction pattern; deviating breaks invariants no tool catches. +- **Typed errors only.** Everything descends from `DexpaceError`; wrap-and-rethrow always passes + `{cause}`. +- **Lint is type-aware and strict.** 70-line function cap, `max-depth` 3, `max-params` 3, explicit + return types on exported functions. Formatting is an error, not a warning. Every + `eslint-disable` must carry a stated reason. +- **Every gap is recorded.** A finding goes in [`docs/work/mvp/2026-09-04-open-items-dissolution.md`](docs/work/mvp/2026-09-04-open-items-dissolution.md), and so does a + deferral — as an open item carrying the trigger that would discharge it, since the separate deferral + register was dissolved on 2026-09-04. A deliberate divergence goes in the deviation ledger. Silent gaps + are the failure mode this project is structured to prevent. + +As-built documentation — how the packages compose, and worked examples across a package boundary — +is [`docs/sdk-documentation/`](docs/sdk-documentation/). + +## Releases + +Releases start from `main` only. The workflow is +[`.github/workflows/release.yml`](.github/workflows/release.yml). It runs on each push to `main`. +It reads the pending changesets and opens a "Version Packages" pull request. When that pull request +merges, the workflow publishes the packages. + +The workflow does not run on `mvp` or on any other branch. Work on those branches is not released. +Changesets written there wait until the branch merges into `main`. + +Each package is at version `0.0.0`. The first release starts from that version. + +Publishing is blocked at this time. The block is deliberate. Three conditions must be true before +the first publish can succeed: + +1. The repository must have an `NPM_TOKEN` secret. Without it, the workflow opens the pull request + but does not publish. +2. The maintainers must decide the access level of the `@dexpace` scope. + [`.changeset/config.json`](.changeset/config.json) sets `"access": "restricted"`. Restricted + packages are private. npm does not attach provenance to a private package. The workflow sets + `NPM_CONFIG_PROVENANCE` for `NFR-16`, so a publish with the current setting fails. To publish + with provenance, set the access to `public`. To stay private, remove the provenance setting and + record `NFR-16` as a deviation. +3. The source repository must be public. npm issues provenance attestations for public source only. + +[`docs/first-release.md`](docs/first-release.md) records all three under `NFR-16`, together with what +the release path already does. + +The sibling repositories do not share one answer yet. `dexpace/python-sdk` publishes to PyPI with +trusted publishing and PEP 740 attestations; PyPI has no private tier, so those packages are public +by construction. `dexpace/dexpace-react` is `UNLICENSED`, sets `"access": "restricted"`, and its +release policy names `npm publish --provenance`, which is the same conflict as this repository. +This SDK is MIT-licensed, like the Python SDK. Until the maintainers decide, this repository keeps +the current settings and stays unpublished. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..4759430 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,41 @@ +# Security Policy + +## Supported versions + +Nothing has shipped yet: every package in the workspace is at `0.0.0` and +none has been published to npm, so there is no released version to support +and no patched release to point at. Until the first release, the supported +revision is the tip of `mvp` — report against a commit SHA. + +## Reporting a vulnerability + +Please **do not** open a public issue for security vulnerabilities. + +Instead, report privately by email to +[oaljarrah@dexpace.org](mailto:oaljarrah@dexpace.org) with `[SECURITY]` in +the subject line. + +Include what you can of the following: + +- The affected package(s), and the commit SHA and Bun/Node.js versions you + reproduced against +- A description of the vulnerability and its impact +- Steps or a proof of concept to reproduce it + +You can expect an acknowledgement within a few days. Please allow time for +a fix to land and be released before disclosing publicly. + +## Scope notes + +- The SDK is a **toolkit**, not a service: `@dexpace/core` executes no + network I/O of its own, and reaches into `node:` exactly once, for + `AsyncLocalStorage`. Transport-level vulnerabilities (TLS, connection + handling, message parsing) belong to whatever sits behind the `Transport` + seam — the runtime's global `fetch`, or `undici` for + `@dexpace/transport-undici` — report those upstream. +- In scope here: credential handling and challenge parsing + (`packages/core/src/auth/`), header/URL redaction in logging + (`packages/core/src/observability/redaction.ts`), redirect safety + (`Authorization` stripped on every re-issue, `Cookie` and + `Proxy-Authorization` cross-origin — `packages/core/src/redirect/decide.ts`), + and body capture (`packages/core/src/body/`, `@dexpace/body-file`). diff --git a/bun.lock b/bun.lock index 5ea441b..ab11554 100644 --- a/bun.lock +++ b/bun.lock @@ -7,27 +7,201 @@ "devDependencies": { "@arethetypeswrong/cli": "^0.18", "@changesets/cli": "^2", + "@dexpace/body-file": "workspace:*", + "@dexpace/codec-json": "workspace:*", "@dexpace/core": "workspace:*", + "@dexpace/logging-debug": "workspace:*", + "@dexpace/logging-pino": "workspace:*", + "@dexpace/rx": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@dexpace/transport-fetch": "workspace:*", + "@dexpace/transport-shared": "workspace:*", + "@dexpace/transport-undici": "workspace:*", "@eslint-community/eslint-plugin-eslint-comments": "^4", - "@microsoft/api-extractor": "^7", + "@microsoft/api-extractor": "catalog:", "@types/bun": "latest", "eslint": "^9", - "fast-check": "^3", + "fast-check": "catalog:", "globals": "^17.8.0", "gts": "^7", + "mitata": "^1", "publint": "^0.3", - "typescript": "^5.8", + "rxjs": "catalog:", + "typescript": "catalog:", "typescript-eslint": "^8", }, }, + "packages/body-file": { + "name": "@dexpace/body-file", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + }, + }, + "packages/codec-json": { + "name": "@dexpace/codec-json", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + }, + }, "packages/core": { "name": "@dexpace/core", "version": "0.0.0", "devDependencies": { - "expect-type": "^1.4.0", + "expect-type": "catalog:", + }, + }, + "packages/logging-debug": { + "name": "@dexpace/logging-debug", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "debug": ">=4.0.0", + }, + "optionalPeers": [ + "debug", + ], + }, + "packages/logging-pino": { + "name": "@dexpace/logging-pino", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "pino": ">=8.0.0", + }, + "optionalPeers": [ + "pino", + ], + }, + "packages/rx": { + "name": "@dexpace/rx", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "rxjs": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "rxjs": "^7.8.0", + }, + }, + "packages/shrink-test": { + "name": "@dexpace/shrink-test", + "version": "0.0.0", + "devDependencies": { + "@dexpace/codec-json": "workspace:*", + "@dexpace/core": "workspace:*", + "@dexpace/transport-fetch": "workspace:*", + "esbuild": "^0.28.2", + }, + }, + "packages/transport-conformance": { + "name": "@dexpace/transport-conformance", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + }, + }, + "packages/transport-fetch": { + "name": "@dexpace/transport-fetch", + "version": "0.0.0", + "dependencies": { + "@dexpace/transport-shared": "workspace:*", + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + }, + }, + "packages/transport-shared": { + "name": "@dexpace/transport-shared", + "version": "0.0.0", + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", + }, + }, + "packages/transport-undici": { + "name": "@dexpace/transport-undici", + "version": "0.0.0", + "dependencies": { + "@dexpace/transport-shared": "workspace:*", + "undici": "^6.21.1", + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:", + }, + "peerDependencies": { + "@dexpace/core": "workspace:*", }, }, }, + "overrides": { + "fast-uri": "^3.1.5", + "js-yaml": "^4.3.1", + "tmp": "^0.2.6", + }, + "catalog": { + "@microsoft/api-extractor": "^7", + "expect-type": "^1.4.0", + "fast-check": "^3", + "rxjs": "^7.8.0", + "typescript": "^5.8", + }, "packages": { "@andrewbranch/untar.js": ["@andrewbranch/untar.js@1.0.3", "", {}, "sha512-Jh15/qVmrLGhkKJBdXlK1+9tY4lZruYjsgkDFj08ZmDiWVBLJcqkok7Z0/R0In+i1rScBpJlSvrTS2Lm41Pbnw=="], @@ -79,8 +253,80 @@ "@colors/colors": ["@colors/colors@1.5.0", "", {}, "sha512-ooWCrlZP11i8GImSjTHYHLkvFDP48nS4+204nGb1RiX/WXYHmJA2III9/e2DWVabCESdW7hBAEzHRqUn9OUVvQ=="], + "@dexpace/body-file": ["@dexpace/body-file@workspace:packages/body-file"], + + "@dexpace/codec-json": ["@dexpace/codec-json@workspace:packages/codec-json"], + "@dexpace/core": ["@dexpace/core@workspace:packages/core"], + "@dexpace/logging-debug": ["@dexpace/logging-debug@workspace:packages/logging-debug"], + + "@dexpace/logging-pino": ["@dexpace/logging-pino@workspace:packages/logging-pino"], + + "@dexpace/rx": ["@dexpace/rx@workspace:packages/rx"], + + "@dexpace/shrink-test": ["@dexpace/shrink-test@workspace:packages/shrink-test"], + + "@dexpace/transport-conformance": ["@dexpace/transport-conformance@workspace:packages/transport-conformance"], + + "@dexpace/transport-fetch": ["@dexpace/transport-fetch@workspace:packages/transport-fetch"], + + "@dexpace/transport-shared": ["@dexpace/transport-shared@workspace:packages/transport-shared"], + + "@dexpace/transport-undici": ["@dexpace/transport-undici@workspace:packages/transport-undici"], + + "@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ=="], + + "@esbuild/android-arm": ["@esbuild/android-arm@0.28.2", "", { "os": "android", "cpu": "arm" }, "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg=="], + + "@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.2", "", { "os": "android", "cpu": "arm64" }, "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A=="], + + "@esbuild/android-x64": ["@esbuild/android-x64@0.28.2", "", { "os": "android", "cpu": "x64" }, "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q=="], + + "@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw=="], + + "@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw=="], + + "@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw=="], + + "@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg=="], + + "@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.2", "", { "os": "linux", "cpu": "arm" }, "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w=="], + + "@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug=="], + + "@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.2", "", { "os": "linux", "cpu": "ia32" }, "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ=="], + + "@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ=="], + + "@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA=="], + + "@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ=="], + + "@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA=="], + + "@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg=="], + + "@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.2", "", { "os": "linux", "cpu": "x64" }, "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ=="], + + "@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw=="], + + "@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.2", "", { "os": "none", "cpu": "x64" }, "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw=="], + + "@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ=="], + + "@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw=="], + + "@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q=="], + + "@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g=="], + + "@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ=="], + + "@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.2", "", { "os": "win32", "cpu": "ia32" }, "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA=="], + + "@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.2", "", { "os": "win32", "cpu": "x64" }, "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g=="], + "@eslint-community/eslint-plugin-eslint-comments": ["@eslint-community/eslint-plugin-eslint-comments@4.7.2", "", { "dependencies": { "escape-string-regexp": "^4.0.0", "ignore": "^7.0.5" }, "peerDependencies": { "eslint": "^6.0.0 || ^7.0.0 || ^8.0.0 || ^9.0.0 || ^10.0.0" } }, "sha512-LF03qURSwEWm2dz5wtdDCzNk+7Opl0X7q6I3undsaIuNsEiNvRV3BCtqu14Q/6Pzg1tBj44LcxpW2EpSLZStZw=="], "@eslint-community/eslint-utils": ["@eslint-community/eslint-utils@4.10.1", "", { "dependencies": { "eslint-visitor-keys": "^3.4.3" }, "peerDependencies": { "eslint": "^6.0.0 || ^7.0.0 || >=8.0.0" } }, "sha512-cuadcxVFE8sDK6iWJbs8Sn0av2Nrh2QSGQhVlBW9AaAHqHwjWsZHT8LJ4hFGPh7ASBV2deFdM7H/DPjulmh8rg=="], @@ -281,6 +527,8 @@ "es-errors": ["es-errors@1.3.0", "", {}, "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw=="], + "esbuild": ["esbuild@0.28.2", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.2", "@esbuild/android-arm": "0.28.2", "@esbuild/android-arm64": "0.28.2", "@esbuild/android-x64": "0.28.2", "@esbuild/darwin-arm64": "0.28.2", "@esbuild/darwin-x64": "0.28.2", "@esbuild/freebsd-arm64": "0.28.2", "@esbuild/freebsd-x64": "0.28.2", "@esbuild/linux-arm": "0.28.2", "@esbuild/linux-arm64": "0.28.2", "@esbuild/linux-ia32": "0.28.2", "@esbuild/linux-loong64": "0.28.2", "@esbuild/linux-mips64el": "0.28.2", "@esbuild/linux-ppc64": "0.28.2", "@esbuild/linux-riscv64": "0.28.2", "@esbuild/linux-s390x": "0.28.2", "@esbuild/linux-x64": "0.28.2", "@esbuild/netbsd-arm64": "0.28.2", "@esbuild/netbsd-x64": "0.28.2", "@esbuild/openbsd-arm64": "0.28.2", "@esbuild/openbsd-x64": "0.28.2", "@esbuild/openharmony-arm64": "0.28.2", "@esbuild/sunos-x64": "0.28.2", "@esbuild/win32-arm64": "0.28.2", "@esbuild/win32-ia32": "0.28.2", "@esbuild/win32-x64": "0.28.2" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA=="], + "escalade": ["escalade@3.2.0", "", {}, "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA=="], "escape-string-regexp": ["escape-string-regexp@4.0.0", "", {}, "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA=="], @@ -303,8 +551,6 @@ "espree": ["espree@10.4.0", "", { "dependencies": { "acorn": "^8.15.0", "acorn-jsx": "^5.3.2", "eslint-visitor-keys": "^4.2.1" } }, "sha512-j6PAQ2uUr79PZhBjP5C5fhl8e39FmRnOjsD5lGnWrFU8i2G776tBK7+nP8KuQUTTyAZUwfQqXAgrVH5MbH9CYQ=="], - "esprima": ["esprima@4.0.1", "", { "bin": { "esparse": "./bin/esparse.js", "esvalidate": "./bin/esvalidate.js" } }, "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A=="], - "esquery": ["esquery@1.7.0", "", { "dependencies": { "estraverse": "^5.1.0" } }, "sha512-Ap6G0WQwcU/LHsvLwON1fAQX9Zp0A2Y6Y/cJBl9r/JbW90Zyg4/zbG6zzKa2OTALELarYHmKu0GhpM5EO+7T0g=="], "esrecurse": ["esrecurse@4.3.0", "", { "dependencies": { "estraverse": "^5.2.0" } }, "sha512-KmfKL3b6G+RXvP8N1vr3Tq1kL/oCFgn2NYXEtqP8/L3pKapUA4G8cFVaoF3SU323CD4XypR/ffioHmkti6/Tag=="], @@ -333,7 +579,7 @@ "fast-levenshtein": ["fast-levenshtein@2.0.6", "", {}, "sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw=="], - "fast-uri": ["fast-uri@3.1.4", "", {}, "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw=="], + "fast-uri": ["fast-uri@3.1.6", "", {}, "sha512-7Ical1vFEMr0onbVzEDIreM22I4khW+fzyQPwvAFWBp1iwdshSZRsL4jjRvPG9JP1uiqMHRto+YU6R2/CzDz5Q=="], "fastq": ["fastq@1.20.1", "", { "dependencies": { "reusify": "^1.0.4" } }, "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw=="], @@ -429,7 +675,7 @@ "js-tokens": ["js-tokens@4.0.0", "", {}, "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="], - "js-yaml": ["js-yaml@4.3.0", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q=="], + "js-yaml": ["js-yaml@4.3.2", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA=="], "json-buffer": ["json-buffer@3.0.1", "", {}, "sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ=="], @@ -483,6 +729,8 @@ "minimist-options": ["minimist-options@4.1.0", "", { "dependencies": { "arrify": "^1.0.1", "is-plain-obj": "^1.1.0", "kind-of": "^6.0.3" } }, "sha512-Q4r8ghd80yhO/0j1O3B2BjweX3fiHg9cdOwjJd2J76Q135c+NDxGCqdYKQ1SKBuFfgWbAUzBfvYjPUEeNgqN1A=="], + "mitata": ["mitata@1.0.34", "", {}, "sha512-Mc3zrtNBKIMeHSCQ0XqRLo1vbdIx1wvFV9c8NJAiyho6AjNfMY8bVhbS12bwciUdd1t4rj8099CH3N3NFahaUA=="], + "mri": ["mri@1.2.0", "", {}, "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA=="], "ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="], @@ -507,8 +755,6 @@ "optionator": ["optionator@0.9.4", "", { "dependencies": { "deep-is": "^0.1.3", "fast-levenshtein": "^2.0.6", "levn": "^0.4.1", "prelude-ls": "^1.2.1", "type-check": "^0.4.0", "word-wrap": "^1.2.5" } }, "sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g=="], - "os-tmpdir": ["os-tmpdir@1.0.2", "", {}, "sha512-D2FR03Vir7FIu45XBY20mTb+/ZSWB00sjU9jdQXt83gDrI4Ztz5Fs7/yy74g2N5SVQY4xY1qDr4rNddwYRVX0g=="], - "outdent": ["outdent@0.5.0", "", {}, "sha512-/jHxFIzoMXdqPzTaCpFzAAWhpkSjZPF4Vsn6jAfNpmbH/ymsmd7Qc6VE9BGn0L6YMj6uwpQLxCECpus4ukKS9Q=="], "p-filter": ["p-filter@2.1.0", "", { "dependencies": { "p-map": "^2.0.0" } }, "sha512-ZBxxZ5sL2HghephhpGAQdoskxplTwr7ICaehZwLIlfL6acuVgZPm8yBNuRAFBGEqtD/hmUeq9eqLg2ys9Xr/yw=="], @@ -589,7 +835,7 @@ "run-parallel": ["run-parallel@1.2.0", "", { "dependencies": { "queue-microtask": "^1.2.2" } }, "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA=="], - "rxjs": ["rxjs@6.6.7", "", { "dependencies": { "tslib": "^1.9.0" } }, "sha512-hTdwr+7yYNIT5n4AMYp85KA6yw2Va0FLa3Rguvbpa4W3I5xynaBZo41cM3XM+4Q6fRMj3sBYIR1VAmZMXYJvRQ=="], + "rxjs": ["rxjs@7.8.2", "", { "dependencies": { "tslib": "^2.1.0" } }, "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA=="], "sade": ["sade@1.8.1", "", { "dependencies": { "mri": "^1.1.0" } }, "sha512-xal3CZX1Xlo/k4ApwCFrHVACi9fBqJ7V+mwhBsuf/1IOKbBy098Fex+Wa/5QMubw09pSZ/u8EY8PWgevJsXp1A=="], @@ -657,7 +903,7 @@ "tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="], - "tmp": ["tmp@0.0.33", "", { "dependencies": { "os-tmpdir": "~1.0.2" } }, "sha512-jRCJlojKnZ3addtTOjdIqoRuPEKBvNXcGYqzO6zWZX8KfKEpnGY5jfggJQ3EjKuu8D4bJRr0y+cYJFmYbImXGw=="], + "tmp": ["tmp@0.2.7", "", {}, "sha512-e0votIpp4Uo2AJYSzVHV6xCcawuiez3DzqDAbrTc3YxBkplN6e+dM13ZeIcZnDg/QpSuU2zfZ3rzwY8ukEnaXw=="], "to-regex-range": ["to-regex-range@5.0.1", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="], @@ -667,7 +913,7 @@ "ts-declaration-location": ["ts-declaration-location@1.0.7", "", { "dependencies": { "picomatch": "^4.0.2" }, "peerDependencies": { "typescript": ">=4.0.0" } }, "sha512-EDyGAwH1gO0Ausm9gV6T2nUvBgXT5kGoCMJPllOaooZ+4VvJiKBdZE7wK18N1deEowhcUptS+5GXZK8U/fvpwA=="], - "tslib": ["tslib@1.14.1", "", {}, "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg=="], + "tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], "type-check": ["type-check@0.4.0", "", { "dependencies": { "prelude-ls": "^1.2.1" } }, "sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew=="], @@ -677,6 +923,8 @@ "typescript-eslint": ["typescript-eslint@8.65.0", "", { "dependencies": { "@typescript-eslint/eslint-plugin": "8.65.0", "@typescript-eslint/parser": "8.65.0", "@typescript-eslint/typescript-estree": "8.65.0", "@typescript-eslint/utils": "8.65.0" }, "peerDependencies": { "eslint": "^8.57.0 || ^9.0.0 || ^10.0.0", "typescript": ">=4.8.4 <6.1.0" } }, "sha512-/ggrHAwyjENDusvyxbuqxAC2dTnZg/Z8F+fgQtYIz+L6n/9HfSlEZcFGV/NsMNa6CkGk0xUjUAFwC0vHOflvIA=="], + "undici": ["undici@6.28.0", "", {}, "sha512-LIY910g9TI13YS95lrMFrs8Rm/u/irgHeTWoKCoteeJ04CUJ92eEfj0rVn+7VKMPBpUPiUoBKfhNyLI23EE/KA=="], + "undici-types": ["undici-types@8.3.0", "", {}, "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ=="], "unicode-emoji-modifier-base": ["unicode-emoji-modifier-base@1.0.0", "", {}, "sha512-yLSH4py7oFH3oG/9K+XWrz1pSi3dfUrWEnInbxMfArOfc1+33BlGPQtLsOYwvdMy11AwUBetYuaRxSPqgkq+8g=="], @@ -773,6 +1021,8 @@ "inquirer/ansi-escapes": ["ansi-escapes@4.3.2", "", { "dependencies": { "type-fest": "^0.21.3" } }, "sha512-gKXj5ALrKWQLsYG9jlTRmR/xKluxHV+Z9QEwNIgCfM1/uwPMCuzVVnh5mwTd+OuBZcwSIMbqssNWRm1lE51QaQ=="], + "inquirer/rxjs": ["rxjs@6.6.7", "", { "dependencies": { "tslib": "^1.9.0" } }, "sha512-hTdwr+7yYNIT5n4AMYp85KA6yw2Va0FLa3Rguvbpa4W3I5xynaBZo41cM3XM+4Q6fRMj3sBYIR1VAmZMXYJvRQ=="], + "js-yaml/argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="], "marked-terminal/chalk": ["chalk@5.6.2", "", {}, "sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA=="], @@ -789,8 +1039,6 @@ "read-pkg-up/type-fest": ["type-fest@0.8.1", "", {}, "sha512-4dbzIzqvjtgiM5rw1k5rEHtBANKmdudhGyBEajN01fEyhaAIhsoKNy6y7+IN93IfpFtwY9iqi7kD+xwKhQsNJA=="], - "read-yaml-file/js-yaml": ["js-yaml@3.15.0", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog=="], - "restore-cursor/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="], "strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="], @@ -819,6 +1067,8 @@ "inquirer/ansi-escapes/type-fest": ["type-fest@0.21.3", "", {}, "sha512-t0rzBq87m3fVcduHDUFhKmyyX+9eo6WQjZvf51Ea/M0Q7+T374Jp1aUiyUl0GKxp8M/OETVHSDvmkyPgvX+X2w=="], + "inquirer/rxjs/tslib": ["tslib@1.14.1", "", {}, "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg=="], + "read-pkg-up/find-up/locate-path": ["locate-path@5.0.0", "", { "dependencies": { "p-locate": "^4.1.0" } }, "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g=="], "read-pkg/normalize-package-data/hosted-git-info": ["hosted-git-info@2.8.9", "", {}, "sha512-mxIDAb9Lsm6DoOJ7xH+5+X4y1LU/4Hi50L9C5sIswK3JzULS4bwk1FvjdBgvYR4bzT4tuUQiC15FE2f5HbLvYw=="], diff --git a/bunfig.toml b/bunfig.toml index dc5be8b..91a1346 100644 --- a/bunfig.toml +++ b/bunfig.toml @@ -1,4 +1,33 @@ [test] +# The rule these two keys enforce, and why it exists, live in ONE place: CLAUDE.md, "HARD RULE -- +# the `tests/` partition". Read that before changing either. What belongs here is only what is local +# to this file. +# +# `root` scopes discovery for a BARE `bun test`. It does NOT govern an explicit path argument, and +# the root script passes `./packages ./tests`, so for the run CI actually performs its effect is to +# keep `scripts/*.test.mjs` -- repo tooling, run via `bun run test:scripts` -- out of neither: those +# are outside both paths already. It matters for a bare invocation, and for the coverage floor, +# which is a statement about `packages/*/src`. +# +# `pathIgnorePatterns` is what excludes the Node suite on that explicit `./tests` path, and its +# spelling is the whole game. Bun accepts an unrecognized `[test]` key in silence -- no warning, no +# error, no effect -- so `testPathIgnorePatterns` reads as configured and does nothing. (Naming the +# wrong key here is safe: `scripts/verify-test-partition.mjs` checks for a DECLARATION, not a +# mention, so this file is free to warn about the trap it sits next to.) +# +# Measured on `bun run test`, pinned Bun 1.3.14, 2026-09-04: with the key, 165 files and exit 0; +# without it, 179 files -- the 14 Node files collected by a runner that cannot prove anything about +# Node. That run exits 1 rather than 0, but only by accident: 13 of the 14 pass silently under Bun +# and the 14th fails on an unrelated timer assertion in config-primitives.test.mjs, pointing nowhere +# near the real cause. +root = "packages" +pathIgnorePatterns = ["tests/node-conformance/**"] coverage = true coverageThreshold = 0.8 coverageSkipTestFiles = true +# Exclude the BUILT artifact. `@dexpace/codec-json` is a separate package and reaches core only +# through its public entry point, which Bun resolves to `packages/core/dist/index.js` -- so from +# Phase 6a on, running the suite instruments core twice: once as `src/` (the statement this floor is +# about) and once as `dist/`, where only the handful of exports the codec touches are ever reached. +# Left in, the duplicate halves the reported number without a line of real coverage changing. +coveragePathIgnorePatterns = ["**/dist/**"] diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..6b9507d --- /dev/null +++ b/docs/README.md @@ -0,0 +1,118 @@ +# `docs/` + +Eight trees and three registers, each with one owner and one job — counting `knowledge/`'s +`harvested/` and `notes/` as the two that `CLAUDE.md` treats them as, and `assets/` as one. This file +is the index; the rule is that nothing in `docs/` is unowned, and nothing is written by two things. + +| Entry | Owns | Written by | Housekeeping may write? | +|---|---|---|---| +| [`product-spec/`](./product-spec/) + [`product-spec.md`](./product-spec.md) | **Normative.** The numbered requirements — `HTTP-7`, `SEAM-1`, `RETRY-13`, `NFR-5`, … — that the code exists to satisfy | A human, deliberately | **No — frozen** | +| [`sdk-design-nodejs/`](./sdk-design-nodejs/) + [`sdk-design-nodejs.md`](./sdk-design-nodejs.md) | How each spec area maps to idiomatic TypeScript. Non-normative but binding by convention. §10 is the **normative deviation ledger** | A human, deliberately | **No — frozen** | +| [`knowledge/harvested/`](./knowledge/harvested/) | Harvested styleguide and spec knowledge, topic-indexed. Cited as "styleguide 6.7", "ch08" | The `knowledge-harvest` skill. **Never hand-edited** | **No — frozen** | +| [`knowledge/notes/`](./knowledge/notes/) | What the implementation found, overriding a harvested entry. Role `review`, manual `sha:` | A human | **No — frozen** | +| [`sdk-documentation/`](./sdk-documentation/) | **As-built.** How the packages compose, which one to install, worked cross-package examples | A human, or the skill on request | Yes | +| [`work/`](./work/) | Process records: per-phase design, plan and checklist, one directory per phase under a unit of delivery | The phase that produced them; **collected** here by the skill | Yes — `git mv` only | +| [`superpowers/`](./superpowers/) | Nothing, for long. The **inbox** the Superpowers skills write into | `brainstorming`, `writing-plans` | Yes — it drains it | +| [the dissolved open-items register](./work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record.** Everything the register held when it was dissolved on 2026-09-04, with every open question in it decided. Item IDs stay reserved and still resolve | — | No — nothing is appended | +| [`first-release.md`](./first-release.md) | Release readiness: what the release path already does, the confirmed mechanics, and the blockers that must clear before a first publish. Was the `NFR-16` row of `deferred-items.md`, which was dissolved on 2026-09-04 — its five still-live rows are archived under *Live deferrals* in [`work/mvp/2026-09-04-register-retirement-purge.md`](./work/mvp/2026-09-04-register-retirement-purge.md) | The maintainer | Yes — edited as blockers clear | +| [`deviations.md`](./deviations.md) | The as-built audit of §10, and the landing point for a deviation found outside a phase | An audit or review | Yes — appends | +| [`audit-67-decisions.md`](./audit-67-decisions.md) | Decision ledger for the audit #67 remediation run: cross-task decisions, rejected alternatives, and the release-machinery work deferred from it | The remediation supervisor | Yes — appends | +| [`assets/`](./assets/) | Vendored wordmark SVGs the root `README.md` renders | Copied from `dexpace/morphic` | Yes | + +## Frozen means frozen + +`knowledge/`, `product-spec/`, `sdk-design-nodejs/` and the two sibling tables of contents are +**read-only** to routine maintenance. The `housekeeping` skill refuses to write to them, and that +refusal is a testable guard, not a paragraph of good intent +(`.claude/skills/housekeeping/guard.mjs`, `guard.test.mjs`). + +Each has its own reason: + +- **`product-spec/`** is what the code is measured against. A tool editing the yardstick is a + category error. +- **`sdk-design-nodejs/`** carries §10, the normative deviation ledger, whose numbering + `deviations.md` is keyed to. Amending it is a deliberate act; the audit beside it is where a + maintenance pass writes instead (the dissolved register's U4). +- **`knowledge/harvested/`** cannot absorb a hand edit. Its `` shas digest the whole source + file, not the entry, so an edit inside an entry changes no sha and the next harvest regenerates or + duplicates it silently. Record the finding in `knowledge/notes/` instead. +- **`knowledge/notes/`** is hand-written and could in principle be edited; it is grouped with + `harvested/` because the CLI reads the two as one corpus and a note's key citation couples them. + Whether that grouping is right is an open question (the dissolved register's U1). + +## The three registers, and which one a thing goes in + +The boundary is **when** the item was created, not what it is about. + +- An **open item** is a discovery made *after* the work: "this is not what the checklist says it + is." → wherever it is enforced: a gate, a test, or a TSDoc comment on the thing it concerns +- A **deferral** is a decision made *before* the work: "not this phase, that one." → the same place, or + `first-release.md` when the deadline is the first version bump + too, from 2026-09-04, as an open item stating the trigger that would discharge it. The separate + `deferred-items.md` register was dissolved that day; the five deferrals still live at the time are + archived under *Live deferrals* in + [`work/mvp/2026-09-04-register-retirement-purge.md`](./work/mvp/2026-09-04-register-retirement-purge.md), + which is an archive of record and never an intake +- A **deviation** is a place the port differs from the reference contract on purpose. → the owning + phase's `## Deviation Ledger` section, consolidated into §10; `deviations.md` audits §10 and + catches what has no owning phase. + +The same requirement ID could legitimately appear in two, back when there were three files. `AUTH-37` +was deferred to Phase 7b in the deferral register and recorded at the dissolved register's G12 as a live silent +swallow; both rows are discharged now, the log half having landed on 2026-09-02, which is the shape a +requirement took as it moved between them. With the deferral register gone, a requirement in that +position now carries one open item that states both halves. + +Register letters and item numbers in the dissolved register's are **permanent**: they are cited across the +repository from source comments, tests, changesets and this tree. A new review appends the next +letter; nothing is ever renumbered or reused. `node .claude/skills/housekeeping/probe.mjs +--only=citations` both counts them and checks that every one still resolves — the count lives in that +command, not in a sentence here, because three documents once carried three different wrong ones +(the dissolved register's U10). + +## `work/` and the inbox + +`docs/work//phaseN/` is the archive. `mvp/` is the only delivery so far and holds every +phase to date; a later effort becomes a sibling. + +``` +work/mvp/ + 2026-07-23-nodejs-sdk-v1-roadmap-design.md # belongs to no phase + 2026-07-25-checkpoint-scaffold-through-phase3a.md + scaffold/ + phase1/ … phase10/ + phase6/2026-07-28-phase6-segmentation-design.md # spans the phase + phase6/phase6a/ phase6b/ phase6c/ # one per sub-phase +``` + +A phase directory is `phaseN`, no hyphen. A phase with sub-phases nests one directory per sub-phase. +A document spanning a whole phase — a segmentation design, a shared checklist — sits at the `phaseN/` +level. Every file keeps its `YYYY-MM-DD-` prefix, which carries ordering the directory name does not. + +New documents do **not** land there directly. The `brainstorming` and `writing-plans` skills hard-code +`docs/superpowers/{specs,plans}/`, they are installed globally, and this repository cannot change +them — so that directory stays as an inbox and the `housekeeping` skill collects from it. See +[`superpowers/README.md`](./superpowers/README.md). + +## Querying the corpus + +`docs/knowledge/` is two trees and 39 topics. Never read a topic file whole when a filtered query +answers the question — a requirement-ID query returns ~170 tokens against a ~5700-token file read. + +```bash +bun run knowledge --req HTTP-13,HTTP-14,HTTP-15 # a whole task's IDs in one call +bun run knowledge --origin note --brief # everything the implementation found +bun run knowledge --topic documentation # the 21 rules the housekeeping skill obeys +bun run knowledge --chapter 6 interface class # a "styleguide 6.7" citation +``` + +`bun run verify:knowledge-structure` keeps the two trees apart and is a blocking CI step. +`bun run knowledge:drift` is the hand-run companion, deliberately not in CI: 16 of the 47 sources are +a sibling styleguide repository no CI checkout has. + +## Keeping this file true + +`.claude/skills/housekeeping/` probes every claim here against the repository — the tree itself, +`CLAUDE.md`'s package and gate counts, `README.md`'s, a README on every publishable package, broken +relative links, and register text that landed in a specification document. It is a hand-run tool, not +a CI step. Run it before claiming the documentation is current. diff --git a/docs/assets/dexpace-wordmark-dark.svg b/docs/assets/dexpace-wordmark-dark.svg new file mode 100644 index 0000000..e3a3c8a --- /dev/null +++ b/docs/assets/dexpace-wordmark-dark.svg @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/docs/assets/dexpace-wordmark-light.svg b/docs/assets/dexpace-wordmark-light.svg new file mode 100644 index 0000000..727bee4 --- /dev/null +++ b/docs/assets/dexpace-wordmark-light.svg @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md new file mode 100644 index 0000000..94c08f8 --- /dev/null +++ b/docs/audit-67-decisions.md @@ -0,0 +1,665 @@ +# Audit #67 remediation — decision ledger + +Supervisor-owned record for the remediation run of the 2026-09-04 audit +([umbrella #67](https://github.com/dexpace/nodejs-sdk/issues/67), subtasks #68–#82). One entry per +cross-task decision: which issue raised it, what was decided, the alternatives rejected, and which later +issues it constrains. Deferred work is listed at the end so the release pass can recover it. Umbrella +branch: `audit/remediation-67`, base `mvp`. Task branches: `audit/67/-`. + +A decision that departs from the spec text is also a dated row in +[`deviations.md`](./deviations.md) under "Deviations recorded outside a phase"; this file records the +*choice*, that file records the *deviation*. + +## Ground rules fixed before wave 1 + +### D0 — Where a remediation deviation is written (raised by #68, #69, #71, #72, #74, #75) +Several subtask issues say "record the reading in the Phase Nx ledger section". Those sections live in +`docs/work/mvp/`, which CLAUDE.md declares a dated record that is never retro-edited. **Decision:** every +deviation or reading this run records goes to `docs/deviations.md` under "Deviations recorded outside a +phase", dated, with `file:line` evidence and "Found by: audit #67 / #". Phase ledger sections are +not touched. *Rejected:* appending to phase ledgers (retro-edits a dated record; §10 is frozen so it is +not an option either). *Constrains:* every later subtask. + +### D1 — Release machinery is out of scope (raised by the run's own brief) +No changesets, no version bumps, no `docs/first-release.md` edits. Each PR lists what it skipped under +"Deferred — release machinery"; the consolidated list is at the end of this file. + +### D2 — Wave overlap adjustments +- Wave 1 (#68, #69) both edit the "Deviations recorded outside a phase" table in `docs/deviations.md` and + `packages/core/src/context/instrumentation.ts`. Kept concurrent: #68 edits existing rows/lines and the + `tracerFactory` TSDoc; #69 only *appends* rows at the end of the table and changes the single + `activeSpan` line. The supervisor resolves the adjacent-hunk conflict at merge. +- Wave 3 becomes #72, #74, #75 (all M3); #73 moves to wave 4 with #76, #77. Reason: #72 and #73 both + reshape `packages/core/src/retry/engine.ts` (`withTrail` vs. the per-attempt re-send of the template), + and #73's layering choice is easier to make on top of #72's landed trail shape. +- Wave 6 splits: #81 first, then #82. Both edit `undici-transport.ts` and add rows to + `packages/transport-conformance/src/run-suite.ts` + `fixtures.ts`; #82's "make undici match" clause + depends on #81's drop-set rewrite. + +## Decisions taken for #69 (M1) — the maintainer-decision items + +### D3 — CTX-15: fix, not ledger +`noopInstrumentationBundle.activeSpan` becomes `NOOP_SPAN`. One line plus a test. #80 lists the same item; +it is done here, #80 skips it. *Rejected:* ledger row (the fix is smaller than the row). + +**D3 outcome (2026-09-04, #69, PR #84).** The size estimate was wrong: importing `NOOP_SPAN` into +`context/instrumentation.ts` closed an import cycle with `observability/tracing.ts` (which imports +`InstrumentationBundle`), and `verify:import-cycles` counts type-only edges. Fixed as that gate prescribes: +`SpanContext`, `Span`, `Tracer`, `NOOP_SPAN`, `NOOP_TRACER` moved to a new leaf module +`packages/core/src/observability/span.ts`; `tracing.ts` re-exports them, so no import path and no API +report changed. The decision itself stands. *Constrains #80:* a `file:line` citation into `tracing.ts` for +those five names is now stale; CTX-15 is done, skip it. + +**Trap for every later subtask.** gts turns on `stripInternal`, and TypeScript tests it by substring-scanning +every leading comment of a declaration, line comments included. A module header that merely *mentions* the +`@internal` tag deletes the first exported declaration from the emitted `.d.ts` with no `tsc` diagnostic; +the failure surfaces one package later as an unresolved name in core's own `dist/`. Do not write that tag in +a file-level comment. + +**#68 outcome (2026-09-04, PR #83).** Round 1 corrected the TSDoc and guides; round 2 requested because +five `docs/deviations.md` anchors it found stale (items 2, 3, 4, 8, 11) were left unfixed as "outside the +partition" — they are the issue's own acceptance criterion. The false "nothing consumes either yet" claim at +`context/instrumentation.ts:8-13` was assigned to #68 in round 2. *Constrains #78:* item 17 now states the +cause-walk matches `IoError` and `TransportFailureError` only, and names #78 as the decider. *Constrains +#80:* the OBS-29 row is marked in progress with the 1:1 binding recorded as met at `pipeline/runtime.ts`; +the open part is caller-reachability of the operation span. *Constrains #71:* `auth.md`'s credential-shape +example is untouched and is #71's. + +### D4 — PIPE-37: ledger the gap, do not implement in M1 +No `PRE_REDIRECT` status-mapping pipeline step exists; `statusMappingStep` is a `ResponseStep`. Recorded +as a row in `deviations.md` naming the gap, the Phase 4→5 hand-off that dropped it, and the petstore +spike finding 2 as the same work. Implementing it is real pipeline work with public surface, outside a +docs-only milestone, and opening a tracking issue is a remote action this run is not authorised to take — +the maintainer opens it if wanted. *Constrains:* none of #70–#82 depends on it. + +### D5 — REDIR-3: keep the current-hop-method reading, pin it, ledger it +Spec text says "original request method". The port evaluates eligibility against the method of the +request being redirected at *this* hop. The two differ only after an opted-in 303 rewrote POST→GET and a +later 301/302 arrives: the port follows it (GET is in the default set), the literal reading would refuse +it. **Decision:** keep the port's reading — the rewritten GET is idempotent and body-less, the 303 rewrite +is opt-in, and refusing would make `allow303` half-useful — pin it with a test (`allow303: true`, POST, +303 then 301) and record the reading as a `deviations.md` row. *Rejected:* switching to the literal reading +(behaviour change inside a docs milestone; stricter without a safety gain). + +### D6 — PAGE-19: WHATWG relative resolution is the intended reading +`; rel=next` resolves against the page URL under WHATWG rules, so it is a followable relative +reference, not an unparseable one. A target that fails `new URL(target, base)` still ends the stream. +Pin with a test and ledger the reading. *Rejected:* adding an ad-hoc "looks unparseable" heuristic. + +### D7 — HTTP-46, IO-13, BODY-9, BODY-34, IO-38, transport `reasonPhrase`: ledger rows +All six are recorded as rows (evidence + reading). `reasonPhrase` sits beside §10 item 13; #82 reads it as +already done and does not re-ledger it. + +## Wave 1 — landed 2026-09-04 +PR #83 (#68) and PR #84 (#69) merged into the umbrella at `840f355`. One conflict (the OBS-29 row of +`deviations.md`): kept #68's "in progress, see #80" text and carried #69's moved `span.ts` citation. Merged +tree preflighted before the merge in a throwaway worktree (byte-identical result): all 20 steps passed. + +## Wave 2 — landed 2026-09-05 +PR #85 (#70) and PR #86 (#71) merged into the umbrella. One conflict (the import list of +`tests/conformance/xcut/security-by-default.conformance.test.ts`), unioned. Merged tree preflighted in a +throwaway worktree before the merge (byte-identical result): all 20 steps passed. Run paused here by the +maintainer; wave 3 (#72, #74, #75) not yet cut. + +## Decisions taken for wave 2 (M2) + +### D8 — #70: redact inside the error messages, and keep the raw URLs on the error properties +`SchemeDowngradeError` and `NonReplayableBodyError` build their messages from `redactUrl(url)`; `fromUrl` / +`toUrl` stay raw for program use. Reason: the message is what every logger, `cause` chain and consumer +`console.error` renders, so redacting at the source protects paths this SDK does not own, not only +`http.redirect.rejected`. If `emitRejected` can also carry the redacted URL fields the other redirect events +carry, add them — but the message fix is the required one. *Rejected as sole fix:* logging `error.name` plus +fields and dropping the message (leaves the raw message reachable through `cause` on the thrown error). +*Constrains:* none. + +### D9 — #71: credential classes, and "once guarded, always guarded" for the replay +- `BasicCredential` and `DigestCredential` become classes with `#password`, `toString` and the + `nodejs.util.inspect.custom` override, following whatever shape `auth/credential.ts` already uses for + `ApiKeyCredential` / `BearerToken` (class plus `createX()` factory if that is the pattern there). Public + shape change, free before the first version bump; `api:local` on core. +- Replay guard: if the original request required HTTPS (the step guarded it), `requireHttps` runs on the + replacement request unconditionally, regardless of which header names it carries. *Rejected:* building the + set of credential-carrying header names from configuration (misses a `challengeHook` that invents a + header). AUTH-8 names bearer, API-key and name-key only; the wider reading is a `deviations.md` row (D0). +- `docs/sdk-documentation/auth.md`'s credential-shape example is rewritten here (#68 left it). +*Constrains #74:* it edits `auth-step.ts` next wave on top of this; the guard site moves. + +**D8 outcome (2026-09-04, #70, PR #85).** Both messages built from `redactUrl()`; raw URLs stay on +`targetUrl` / `fromUrl` / `toUrl`; `http.redirect.rejected` gained `url.full` (redacted) like the sibling +events. A non-URL string handed to either public constructor now renders `[malformed url]` in the message +(OBS-15 totality read as the safe default; pinned). New fixture route `/redirect-secret-target` in +`tests/conformance/xcut/fixtures/server.ts` (307 to `/echo?access_token=`), reusable. +*Constrains #74:* build any URL-naming message from `redactUrl()` at the constructor. *Constrains #72, #78:* +`docs/sdk-documentation/errors.md` redirect section was edited here. + +**D9 outcome (2026-09-05, #71, PR #86).** `BasicCredential` / `DigestCredential` are classes in +`auth/credential.ts` with `#password`, read inside the package through an `@internal` `credentialPassword()` +friend hook; `DigestCredential` takes `algorithmPreference` as a third positional. No construction-time +validation on the classes — AUTH-14/AUTH-16 stay single-sourced in `basicHandler()` / `digestHandler()`, +which `authStep()` builds at construction, so a blank password still fails there. Replay guard keys on +`OutboundPlan.guarded` ("once guarded, always guarded"); `guardReplayScheme` now takes a `ReplayGuardInput` +bundle. Two `deviations.md` rows: AUTH-8 widened to every credential type; XCUT-16 guard deliberately wider +than its letter. `scripts/verify-consumer-types.mjs`'s fixture changed because no structural +`BasicCredential` shape exists any more. *Constrains #74:* `auth-step.ts` conflict surface is the +`./credential.js` import block, `buildHandlers`, `OutboundPlan`/`planOutbound`, `guardReplayScheme`, +`ChallengeDrive`; `credential.ts` now has a type-only import of `./digest.js`, so a new edge from +`digest.ts` back into `credential.ts` closes a cycle. A stubbed `ChallengingTransport` exists in +`security-by-default.conformance.test.ts` for clauses needing an `https://` hop. + +**Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is +inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. + +## Wave 3 — landed 2026-09-05 +PR #87 (#75), PR #89 (#74) and PR #88 (#72) merged into the umbrella at `2ea8b3e`, in that order. One +conflict (the `deviations.md` table tail: #75's `ASYNC-21` row and #74's `AUTH-22` row), unioned. Merged tree +preflighted in a throwaway worktree before the merge (byte-identical result): all 20 steps passed. The three +stale `retry/*` citations in `deviations.md` item 3 that #72 shifted were re-anchored on the umbrella after the +merge (`engine.ts:367`, `retry-step.ts:151`, `retry-dispatch.ts:55`). + +**Trap for every later subtask (git identity).** The #72 agent committed with +`git -c user.email="oaljarrah@dexpace.org"`, lifted from the harness's "user's email address" context line — +that address is the Claude login, and GitHub attributes it to a different account. Rewritten with +`--reset-author` and force-pushed before the merge; nothing on the umbrella carries it. Contract item 10 now +forbids any author override; the supervisor checks `git log --format=%ae` on a branch before merging it. + +## Decisions taken for wave 3 (M3) — pre-taken 2026-09-05, before dispatch + +### D10 — #72: the final typed error is surfaced as-is; the trail rides in a side table, read through `retryAttempts()` +The surfaced error of `retryStep` / `dispatchWithRetry` is the final attempt's own error, class untouched: +`instanceof TransportFailureError` holds for `maxAttempts` 1 and 3 alike, and an abort during backoff surfaces +the `CancellationError` that `abortToSdkError` built (`retry/engine.ts:385`), which `withTrail` at `:386` was +undoing. Earlier attempts' errors are reachable through a new `@public` accessor exported from core, +`retryAttempts(error: unknown): readonly unknown[]` — oldest first, the surfaced instance itself excluded +(RETRY-34's skip-self clause), `[]` for an error that carries no trail — backed by a module-private `WeakMap` +that the engine writes once per terminal failure. **Not a deviation, a correction:** RETRY-34 says the prior +failures are "attached to the surfaced exception as suppressed", which is Java's `addSuppressed` — the +surfaced exception stays what it is and grows a list. Wrapping it in `SuppressedError` made the surfaced +*type* a function of how many attempts ran, which is what XCUT-1's "assert the surfaced error is the +cancellation type" clause catches. No `deviations.md` row. `suppress()` stays for its RECOV-12 job. +*Rejected:* an own property (`attempts` / `errors` / `suppressed`) defined on the surfaced error — a foreign +error may be frozen or non-extensible, so `defineProperty` in the engine's failure path can itself throw; a +primitive thrown value cannot carry one at all; and `.suppressed` already means "the one secondary" on +`SuppressedErrorLike`. *Rejected:* a `RetryExhaustedError` wrapper (hides `CancellationError`, the row XCUT-1 +is about). *Rejected:* threading the trail through `cause` (`cause` is already the raw abort reason at +`:383`, and it means "why", not "before"). A primitive surfaced value is passed through unchanged with no +trail entry rather than wrapped. Update the `retryStep` TSDoc, `retry-dispatch.ts:45`'s `@throws` prose, +`docs/sdk-documentation/pipelines.md`'s retry section, the "suppressed trail" wording at +`docs/sdk-documentation/errors.md:188`, and `write-a-response-handler.md` so the RECOV-12 wrapper is documented +as the *only* place a `SuppressedError` is built. `api:local` on core. *Constrains #78:* the classify +cause-walk sees the typed error directly now, never through a `SuppressedError.error` hop. *Constrains #73:* +the trail accessor is the shape it layers on. + +### D11 — #74: parse every challenge header; emit `cnonce` for `-sess` regardless of `qop`; empty `realm`/`nonce` are unsatisfiable +- **Repeated `WWW-Authenticate` / `Proxy-Authenticate`.** `pickChallengeHeader` reads `headers.getAll(name)` + and parses each value with `parseChallenges`, concatenating the lists in wire order — parse-each rather + than comma-join, so a malformed later value cannot poison the parse of an earlier one. `rank` selects across + the concatenation. Conformance row in `packages/transport-conformance` (`run-suite.ts` + `fixtures.ts`): a + fixture route sending two `WWW-Authenticate` headers, asserting the *parsed challenge list* is identical + through both transports — `getAll` legitimately returns one comma-joined entry through fetch and two entries + through undici, and the list after parsing is the only thing the transport is answerable for. Fix the + `Set-Cookie`-only comment at `undici-transport.ts:334`; touch nothing else in that file (#81 owns it). +- **`-sess` without `qop`: emit `cnonce`.** RFC 7616 §3.4 says of `cnonce` "This parameter MUST be used by all + implementations", and §3.4.2 folds it into A1 for every `-sess` algorithm; a `-sess` response without it is + unverifiable by construction, which is what the port sends today (`digest.ts:337-343` hashes a cnonce the + header at `:386-387` omits). `nc` and `qop` stay conditional on a negotiated `qop`. AUTH-22's "emit + cnonce/nc/qop only when qop is negotiated" is RFC 2617's RFC 2069-compatibility form, which predates + `-sess`. Departure from AUTH-22's letter: one `deviations.md` row (D0). *Rejected:* declining the challenge — + it turns every `-sess`-without-`qop` server into a guaranteed 401 for no security gain, and the value is + already computed. `computeDigestResponse` vector for `MD5-sess` with no `qop`. +- **Empty `realm` or `nonce`** is unsatisfiable: `parseDigestChallenge` requires non-empty strings, the + challenge is declined and the next one tried (AUTH-25's "return no header when it cannot satisfy any"). No + row; AUTH-12's verbatim storage is unchanged, the check sits at selection. +- No new core exports. `api:local` on core only if a `@public` TSDoc changes. No changeset (D1). +*Constraints inherited:* D9 (the `auth-step.ts` surface #71 reshaped: `buildHandlers`, `OutboundPlan`, +`planOutbound`, `guardReplayScheme`, `ChallengeDrive`; `credential.ts` imports `./digest.js` type-only, so no +edge from `digest.ts` back into `credential.ts`); D8 (any URL-naming message is built from `redactUrl()`). + +### D12 — #75: keep the ownership transfer, and ledger it +`sseEvents$` / `typedSse$` keep passing `() => stream.close()` as `fromAsyncIterable`'s `release`. The +issue's "spec-faithful one-line change" is neither: (1) `SseStream` self-releases on **any** iterator +termination by SSE-30's own design — `#iterate`'s `finally` runs `#releaseQuietly()` when the runtime calls +`return()` (`packages/core/src/sse/stream.ts:136-138`), and `fromAsyncIterable` must call `iterator.return()` +exactly once (ASYNC-6), so the socket closes with or without the callback; a `for await` with `break` closes it +the same way. ASYNC-21's "MUST NOT close the caller-owned source" presumes a source whose iterator return does +not release, which this port's `SseStream` deliberately is not. (2) The release-*before*-`return()` ordering +is what settles an in-flight pull on unsubscribe (`packages/rx/src/from-async-iterable.ts:44-48`): an async +generator's `return()` queues behind a suspended `next()`, so dropping the callback would leave an unsubscribe +during a stalled read pending until the server sends a byte. Pagination passes no release because its pulls +are bounded HTTP exchanges; SSE's are not. Removing the callback would change only the failure channel and the +ordering, not whether the source closes — and it would reintroduce the hang. **Recorded as a deviation** from +ASYNC-21's non-closing clause: one `deviations.md` row (D0) naming `sse.ts:35,57-59`, `from-async-iterable.ts:103-108`, +the two reasons above, and the Phase 8b checklist gist at +`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67` that dropped the clause (a +dated record; not retro-edited). Tests in `packages/rx/src/sse.test.ts`: early unsubscribe, source error, and +end-of-source each close the underlying resource **exactly once** (count the resource's `close`, not the +facade's — `SseStream.close()` is idempotent by SSE-28, so the facade count proves nothing); plus the +unsubscribe-during-suspended-pull case settles. TSDoc on both functions states the transfer outright +("subscribing hands the stream to the adapter; do not call `close()` yourself, and do not iterate it +afterwards"), and `packages/rx/README.md` says the same beside its `for await` guidance. `api:local` on rx. +*Rejected:* dropping `release` (above). *Rejected:* a caller-facing `{ownership}` option (two behaviours to +document for a case with one correct answer). No changeset (D1). + + +**D10 outcome (2026-09-05, #72, PR #88).** `withTrail` became `attachTrail`; the trail lives in a new leaf +module `packages/core/src/retry/attempt-trail.ts` (`recordAttempts` internal, `retryAttempts` public, one +shared frozen empty list). An empty trail *deletes* a prior entry for a reused error instance; symbols are +excluded as weak keys rather than probed. Round 2 removed a public claim that `retryAttempts(e).length + 1` is +the send count — false on three reachable paths (the RETRY-32 gate, `stampAttempt` throwing, a non-abort +`Clock.sleep` rejection), and the `pipelines.md` example was unsound even narrowed to +`TransportFailureError`, because `abortToSdkError` synthesizes one for a timeout signal. Two engine cases pin +sends against trail length. `tests/node-conformance/retry.test.mjs` was outside the partition and edited anyway: +its case asserted the wrapper by name and was red the moment D10 landed. *Correction to D10's text:* +`classify.ts` never walked `.error`, and it ran per attempt before the terminal wrap, so the "hop" the +constraint on #78 described was not live; the end state it names is right. + +**D11 outcome (2026-09-05, #74, PR #89).** `pickChallengeHeader` returns every value (`getAll`), parsed per value +by a new `challengesOf`; a test with an unterminated quoted string in the first value discriminates parse-each +from comma-join. `cnonce` emitted for any `-sess` algorithm (row in `deviations.md`). Empty `realm`/`nonce` +decline (`!== ''`, not trimmed). **Deviation from D11's letter, accepted:** the conformance row cannot assert the +*parsed* list — `parseChallenges` is `@internal` and D11 forbade a new core export, and a real `authStep` drive is +blocked by AUTH-28 on the plain-`http` fixture — so `run-suite.ts`'s `challengeList()` splits the joined +`getAll` at `, Digest ` boundaries, scoped to the fixture's own two challenges and documented as not a parser. +The transport is answerable only for surfacing both values, which the row proves (red against a one-line +fixture). Found and fixed on the way: `run-suite.ts`'s header claimed TRANSPORT-10..14 were asserted elsewhere +while labelling rows with them; `node:http`'s `writeHead` rejects a `readonly string[]` header value, and +`gts lint` did not catch it — only `typecheck` did. *Constrains #82:* the new fixture route +`/repeated-challenge` and `registerInboundHeaderRows` exist; the `TRANSPORT-14` label is the closest MUST. + +**D12 outcome (2026-09-05, #75, PR #87).** No behaviour change. Ten `sse.test.ts` cases count the release the +*owned resource* sees (a structural `ReadableStream` double counting `reader.cancel()` and `body.cancel()` +separately) plus one in `tests/node-conformance/rx-bridge.test.mjs`. Measured counterfactual: deleting both +`release` arguments turns the two suspended-pull cases and two pre-existing idle-unsubscribe assertions red while +every exactly-once count stays green — so only the suspended-pull case discriminates the design; the counts pin +against a future double release, and the row says which is which. The row also closes `SSE-41`'s "documented +source ownership" clause, which Phase 8b marked done on documentation that named unsubscription only. New +`packages/rx/README.md` section "Who owns the stream"; `rx.api.md` regenerated byte-identical (prose only). + +### Wave 3 partition +| Task | Owns | Shared, append-only | +|---|---|---| +| #72 | `packages/core/src/retry/**`, the retry exports in `packages/core/src/index.ts`, `packages/core/etc/core.api.md` (retry names), `docs/sdk-documentation/pipelines.md` retry section, `errors.md:188`, `write-a-response-handler.md`, `tests/conformance/xcut/retry-safety.*` and `cancellation-and-timeout.*` | — | +| #74 | `packages/core/src/auth/{digest,auth-step,challenge}.ts` and tests, `undici-transport.ts:334` comment only, `packages/transport-conformance/src/{run-suite,fixtures}.ts`, `docs/sdk-documentation/auth.md` | `docs/deviations.md` (one row at table end) | +| #75 | `packages/rx/**` | `docs/deviations.md` (one row at table end) | +Known merge seams: the `deviations.md` table tail (#74 + #75), and `core.api.md` if #74 changes a `@public` +TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. + +## Wave 4 — landed 2026-09-05 +PR #90 (#73), PR #91 (#77) and PR #92 (#76) merged into the umbrella at `36c3d04`, in that order. One +conflict (the `deviations.md` table tail: #73's `RETRY-44` row against #76's `HTTP-35` and `HTTP-31` rows), +unioned. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all 20 +steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com` before merging. + +**Traps for every later subtask, added this wave.** +- `stripInternal` is wider than D3 recorded: **any** leading comment of a declaration that contains the + `@internal` substring — inside backticks, in ordinary prose — strips that declaration from the `.d.ts`. + #73 lost `dispatchWithRecovery` from `orchestrator.d.ts` by writing "Both are `@internal`" in its TSDoc. +- `bun run api:local` exits 0 on `ae-unresolved-link` *warnings*; `api:ci` (what `bun run api` and CI run) + exits 1 on them. A `{@link}` to an `@internal` name from a `@public` block therefore looks clean locally + and is red in CI. Backticked prose, as for the `.message` trap. +- `gts lint` and a green `bun test` do not type-check what `tsc` does: #74's `readonly string[]` header value + and #77's `Promise`-as-`BodyInit` both surfaced only on the preflight's `typecheck` step. +- `String.prototype.isWellFormed()` exists on the Node floor but not on `lib: ES2023`; use + `/\p{Surrogate}/u` (now in `http/rfc3986.ts`). + +## Decisions taken for wave 4 (#73 M3, #76 + #77 M4) — pre-taken 2026-09-05, before dispatch + +### D13 — #73: the request recovery chain runs once, above the retry loop +`dispatchWithRetry` applies `config.requestChain` **once** and hands the prepared request to `runWithRetry`; +each attempt then re-executes transport + response chain over `stampAttempt`'s fresh copy of that prepared +request. That is the layering `idempotencyKeyStep`'s `@public` TSDoc has claimed since `1f48926` ("runs ONCE per +call, upstream of retry"), and it is what RECOV-32's "one stable key across every retry" and RETRY-38's +"preserving any idempotency key" both presuppose — a key the engine re-sends from a pre-chain template is not +preserved, it is regenerated. **Reading of RETRY-44:** its "downstream chain" is whatever sits below the retry +loop; in the recovery stack that is transport + response chain, and its "upstream steps MUST NOT mutate the +shared in-flight request" clause is satisfied by construction, because upstream steps no longer run between +attempts at all. One `deviations.md` row records the reading (D0), because the port's own test named RETRY-44 +as the reason the request chain re-ran (`retry-dispatch.test.ts:67`, to be renamed). A request-chain failure +happens before the loop and is not retried: it never reached the wire, and it still passes through the response +chain once so RECOV-10/11's outcome handling is unchanged. *Rejected:* memoizing the key on the template +(`WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice +would replay the key and have the server drop a real second call. *Rejected:* re-running the chain over the +*prepared* request each attempt — the chain reads its own output, which is the mutation RETRY-44 forbids in +different clothes, and every step would have to be proven idempotent. Enumerate the shipped `RequestStep`s in +`recovery/` and state in the ledger outcome that none needs per-attempt re-execution; the attempt ordinal is +the engine's (RETRY-38). Tests: N attempts, one `generate()` call, the same header value on every wire send; +rename the RETRY-44 test to what it now proves. TSDoc on `dispatchWithRetry` and the orchestrator; no +public shape change (`dispatchWithRetry` is `@internal`). `api:local` if `idempotencyKeyStep`'s prose moves. +*Constrains #78:* `engine.ts` is edited here; wave 5 lands on top. + +### D14 — #76: reject at the setter, in the `DexpaceError` tree, and say so in `@throws` +- **Path params:** `Object.hasOwn(pathParams, name)`; a missing own property is `OperationAssemblyError` + (SEAM-27's "every placeholder MUST have a supplied value"). `{constructor}` with `{}` is the pin. +- **Dates:** `ifModifiedSince` / `ifUnmodifiedSince` throw `RequestConditionsValidationError` on + `Number.isNaN(date.getTime())`. +- **`timeoutMs`:** the setter rejects a non-integer and anything above `2**32 - 1` with + `RequestOptionsValidationError` — HTTP-35 puts the range check at the setter, and `AbortSignal.timeout()`'s + range is the only one a transport can honour. Rewrite the TSDoc paragraph that argues a fractional + millisecond is meaningful and flip the test that pins it. *Rejected:* rounding/clamping in `composeSignal` + (hides the caller's error where HTTP-35 says to surface it). Add `@throws` to `composeSignal` for whatever + it can still raise. +- **Lone surrogates:** `String.prototype.isWellFormed()` at the call site that supplied the value — + `QueryParams` builder `add`/`set` (name and value) and `substitutePathParams` — throwing the error class + that call site already throws for invalid input (`OperationAssemblyError` for path params; for query params + the class the builder uses today, or `UrlConstructionError` if it has none — no new error class). Then + prove by test that no `URIError` can escape `encode()`, `equals()` or `buildRequest`; if one still can, + report the path rather than adding a second mechanism. +- **`getAll`:** one shared frozen empty array in `Headers` and `QueryParams`; a test that the present-name + array is frozen too. +- **`Headers.equals`:** direct cases (name order, value order, subset, case). +- **`TeeSink`:** `Number.isInteger(tapLimit) || tapLimit === Number.POSITIVE_INFINITY`, else the error the + constructor already throws for a negative limit. +- `api:local` on core after the `@throws` edits. No changeset (D1). Do not touch `http/media-type.ts` (#77's). + +### D15 — #77: contract violation on an empty chunk, quote the boundary, gate the tap on `closed` +- **Empty chunks:** `#writeExactly` throws `SourceContractViolationError` on `value.length === 0` for a + positive request, same wording as `io/retention-window.ts:177-183`; a declared length of 0 stays a + legitimate empty write (BODY-10). Tests: an empty-only source, and an empty chunk between real chunks. +- **Boundary:** keep RFC 2046 `bchars` as the accepted grammar (HTTP-51 says reject what *violates* it, not + narrow it) and **quote** the `boundary=` parameter whenever it is not a pure `tchar` token, using + `http/media-type.ts`'s existing token/quoted-string rendering (export an internal helper from that file if + the class API does not reach it; #77 owns `media-type.ts` this wave). Round-trip test through + `Response.formData()` on Bun, and the same case in `tests/node-conformance/body-lifecycle.test.mjs`. + *Rejected:* narrowing `validateBoundary` to `tchar` (rejects boundaries RFC 2046 allows for a problem the + renderer owns). +- **Logging tap after `close()`:** `startDrain`, `snapshot` and `read` check `state.closed` first — + `snapshot()` returns the captured prefix without starting a drain, `read()` rejects with + `ClosedResourceError`, `error()` reports only a genuine drain failure. Tests: close-then-snapshot, + close-then-read, close-then-error. +- **Node conformance:** cases for `toReadableStream`, `toWritableStream`, `TeeSink`'s bridge, + `withRequestLogging` and `withResponseLogging` go into the **existing** topic files + (`io-byte-stream.test.mjs`, `body-lifecycle.test.mjs`), not one new file per bridge — the tree is + topic-named and flat, and its README lists members. Pull, cancel and lock behaviour is what to assert. +- `api:local` on core if a `@throws` changes. No changeset (D1). + + +**D13 outcome (2026-09-05, #73, PR #90).** `orchestrator.ts` split into `prepareRequest()` (request chain, +once, RECOV-2's throw-to-`Failure` applied) and `dispatchPrepared()` (transport + response chain + unwrap), +both `@internal`; `dispatchWithRecovery` is their composition, behaviour unchanged. `dispatchWithRetry` +applies the chain once and retries `dispatchPrepared` over `stampAttempt`'s copy; a request-chain failure +gets one trip through the response chain and no retry. `engine.ts` needed no functional edit (D13's +constraint on #78 was over-cautious). The shipped `RequestStep` list is exactly one — `idempotencyKeyStep`; +the issue's "client identity, auth stamps" are pipeline-stack `StepDescriptor`s, already re-driven per +attempt by `retryStep`'s `ctx.fork()`, which is RETRY-44 correctly applied to *that* stack. *Correction to +D13's text:* "one stable key across every retry" is `idempotency-key.ts`'s own TSDoc, not RECOV-32's, whose +letter is "invoked at most once per applicable request"; the row records it as a reading. + +**D14 outcome (2026-09-05, #76, PR #92).** All seven bullets landed test-first. Departures, accepted: +`/\p{Surrogate}/u` instead of `isWellFormed()` (not on `lib: ES2023`); `QueryParams.parse` substitutes +U+FFFD rather than throwing, because HTTP-31 is a MUST that `parse` never throws (row); `QueryParamsBuilder` +has no `set`, so `add` only; `tests/node-conformance/seams.test.mjs` edited outside the partition because +`AbortSignal.timeout()` is runtime-divergent (Bun accepts `1.5` and `2**32`; Node rejects both). Round 2 +fixed `http.md`'s now-false fractional-timeout sentence, the two error-class TSDocs, consolidated +`EMPTY_VALUE_LIST` into `http/builder.ts`, and appended the HTTP-35 and HTTP-31 rows. `core.api.md` +byte-identical (signatures unchanged). *Handed to #79:* the last `URIError` escape, `pagination/query-splice.ts` +with a server-supplied cursor. *Handed to #81/#82:* `defaultTimeoutMs` unvalidated on both transports. + +**D15 outcome (2026-09-05, #77, PR #91).** Empty chunk → `SourceContractViolationError` via +`assertNonEmptyChunk`, applied for `declared === 0` too (the only reading that also honours "never an +infinite spin"; recorded in a source comment, as IO-17 and BODY-25 did for the same qualifier — no row). +Boundary rendered through `MediaType.of(...).render()`, so nothing new is exported from `media-type.ts`. +Tap gated on `closed` with the checks ordered fits-cap → tail-consumed → closed, because a literal +closed-first gate breaks BODY-23's repeatable read. 22 Node-conformance cases across the two existing topic +files; **Bun's `Response.formData()` accepts `boundary=a,b`, Node's rejects it**, so the Bun round-trip rows +are regression guards and only the Node tree reproduces the bug — the clearest case yet for that tree +existing. The unknown-length `pipeTo` path still forwards empty chunks; out of HTTP-39's scope, noted. + +### Wave 4 partition +| Task | Owns | +|---|---| +| #73 | `packages/core/src/recovery/**`, `packages/core/src/retry/{retry-dispatch,engine}.ts` + tests, `tests/node-conformance/recovery-chain.test.mjs` and `retry.test.mjs` if a case belongs there, `docs/deviations.md` (one row at table end), `core.api.md` if `idempotencyKeyStep`'s TSDoc moves | +| #76 | `packages/core/src/http/{headers,query-params,request-options,request-conditions,rfc3986}.ts`, `packages/core/src/seams/{operation,transport}.ts`, `packages/core/src/io/tee-sink.ts` + their tests, `core.api.md` (`@throws` on those) | +| #77 | `packages/core/src/body/{stream-body,multipart-body,response-body-logging}.ts` + tests, `packages/core/src/http/media-type.ts` (helper export only), `tests/node-conformance/{io-byte-stream,body-lifecycle}.test.mjs` + that tree's README, `core.api.md` if a `@throws` changes | +Known merge seams: `core.api.md` (up to three), which the supervisor regenerates on the merged tree rather than +hand-merging. No two tasks share a source file. + +## Wave 5 — landed 2026-09-05 +PR #93 (#78), PR #95 (#80) and PR #94 (#79) merged into the umbrella at `4576658`, in that order. No conflicts: +#78's item 17 section edit and #80's OBS-29 row edit plus two appended rows sit in disjoint regions of +`deviations.md`. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all +20 steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com`. + +**Traps added this wave.** `gts --fix` deletes an `eslint-disable-next-line` whose next line is another +comment (the directive binds to the comment and becomes unused): prose first, directive last. And +`packages/core/src/io/index.ts` is a dead barrel nothing imports, whose file-level comment carries the +`@internal` substring and so ships a broken `dist/io/index.d.ts` today; harmless only because the modules it +re-exports emit `export {}`. Left for the release pass (deleting it is a design call). + +## Decisions taken for wave 5 (M4: #78, #79, #80) — pre-taken 2026-09-05, before dispatch + +### D16 — #78: `instanceof IoError` stays; it means "transport-layer failure", and item 17 says so +The classifier keeps `current instanceof IoError` (option b). The four flat leaves are SDK-internal contract +and lifecycle failures, deterministic on re-send: `SourceContractViolationError` and `ClosedResourceError` +are caller programming errors, `AllocationLimitError` is a cap the same request will hit again, and +`EndOfStreamError` is the exact-length-copy contract inside this package — a *wire* truncation is the +transport's to surface, as `TransportFailureError`, which is why `TRANSPORT-20` makes that class an `IoError` +and the leaves not. RETRY-2's "an I/O error" is read as that boundary. Five `classify.test.ts` cases pin one +answer per class; `docs/deviations.md` item 17's rationale paragraph and the anchor-correction block are +rewritten to state the rule (the row itself stays; #68 left the decision to #78), and `io/index.ts`'s +comment says what the cause-walk actually matches. *Rejected:* switching to `isIoError` and deciding per leaf +(only `EndOfStreamError` was ever a candidate, and it is the wrong layer to decide wire truncation). +- `delayOverride` returning a non-finite number (`NaN`, `±Infinity`) is treated exactly like one that throws + (RETRY-40): the computed schedule is used and the loop continues; log through the same path a throwing + override uses. A finite negative override keeps today's behaviour (inline, no wait — pinned already). +- `computeDelay`: `initialDelayMs === 0` short-circuits to `0` before the power is taken, so `0 * Infinity` + never happens; keep the `Math.min` saturation for the positive case. A test at the overflow attempt. +- No changeset (D1); `api:local` only if a `@public` TSDoc changes. *Constrains #80:* item 17 is a section + edit mid-file in `deviations.md`; #80 edits the OBS-29 row and appends — different regions. + +### D17 — #79: race the read against the signal; close on every paginator exit; a present `null` is a `DeserializationError` +- **Abort:** `deserializeFrom` and `serializeTo` race each pending `reader.read()` / `writer.write()` against + the signal (abort listener added once, removed in `finally`), then release the lock as today, so the + documented "an aborted call never leaves the caller's source locked" becomes true rather than narrowed. A + module-private helper inside `packages/codec-json` — core exports no abort-race utility and codec-json has + no dependencies. Mid-drain tests in `json-serde.test.ts` and in `tests/node-conformance/serde.test.mjs` + (Web Streams + `AbortSignal`: runtime-divergent, so the Node case is required, not optional). + *Rejected:* narrowing `write-a-serde.md` and `seams/serde.ts`'s promise to "checked between chunks". +- **Paginator:** the two invariants use the same close-then-rethrow discipline as `parseOrClose`; test with + `== null` so `items: null` fails the invariant, not the spread. The error class is the one `invariant()` + already throws; the message names the invariant. Tests: `parse → undefined`, `parse → {items: null}`, + resource `close` called exactly once (PAGE-27). +- **`tristate()`:** runtime `=== null` check after `inner.parse`, throwing `DeserializationError` (SERDE-14's + fourth state is a decode failure, not a programmer error). +- **SSE:** `#releaseWithInFlightError` no longer calls `onReleaseFailure`; the release failure rides as + `suppressed` only (SSE-30 scopes the hook to the clean-terminal path). Adjust the test that expected both. +- **Handed over from #76:** the one `URIError` still escaping core is `pagination/query-splice.ts` + (`spliceQueryParam` / `readQueryParam`, reached from `strategies.ts:44,79,85` with a server-supplied cursor; + repro `spliceQueryParam(new URL('https://h/?a=1'), 'cursor', '\uD800')`). Handle it here with the same + `hasLoneSurrogate` helper #76 added to `http/rfc3986.ts`, throwing the error class the strategies already use + for a malformed cursor; add the test. +- Two packages: `api:local` in `core` and `codec-json` if a `@public` TSDoc changes. No changesets (D1). + +### D18 — #80: `run`, not `enterWith`, in the runtime; a public `instrumentation` option; document the config wiring +- **Context leak:** `Runtime.send()` wraps its awaited body in `AsyncLocalStorage.run()` for both stores + (diagnostic fields, active span), so what the caller observes after `await send()` is what it had before. + The handle-based `pushDiagnosticFields` / `AsyncScopedStore.enter` stay for callers, with TSDoc stating + that the returned restore works only inside the continuation that called it; the runtime stops using them. + Tests: log *after* `await send()` and assert no `trace.id`; the second `send()` gets its own + `http.client.operation` span (OBS-29 — this is the leak that suppressed it). +- **Public path:** `PipelineBuilder`'s constructor gains an options bag `{instrumentation?, operationName?}` + (second parameter, optional, so no caller breaks), threaded to `createRuntime`'s `contextInit`; + `StandardResilienceOptions` gains the same two fields and forwards them. `operationName` reaches + `promoteToRequest` (CTX-16). Rewrite `tracerFactory`'s TSDoc and the `deviations.md` OBS-29 row to "met" + with the new `file:line`. `api:local` on core. The example in + `.changeset/2026-09-04-per-operation-span.md` is a dated release note: leave it, list it as deferred (D1). +- **Store hygiene:** `contextStore.install` moves inside the `try`, or the `try` opens before + `startOperationSpan`; `span.end()` runs once, behind an `ended` flag. Test: `store.size` equal before and + after a send whose `tracerFactory` throws. +- **Body-drain diagnostics:** both catches emit `http.instrumentation.bodyCaptureFailed` through the + existing `safeEmit` with `cause`, then return the empty capture as today (OBS-20). +- **`DEXPACE_LOG_LEVEL`:** the global configuration's default stays empty — defaulting it to + `defaultConfiguration()` would read the process environment at import time, which is a behaviour change + with no phase behind it. Instead: `docs/sdk-documentation/pipelines.md` (logging section) documents + `setGlobalConfiguration(defaultConfiguration())` as the wiring, and `LoggingStepSettings` gains an optional + `configKey` whose default is the current constant. Check whether an OBS-35 row already exists in + `deviations.md`; if not, append one recording that the default key is baked in (OBS-35's letter) and why + a required key was not chosen (every caller would have to name one to get any logging at all). +- **Smaller items:** `noopInstrumentationBundle.activeSpan` is DONE (D3, #69) — skip. `logging-step.ts`'s + per-request span is ended in a `finally`. `redactUrl(string)`'s WHATWG normalisation (host case, default + port, path) is documented in its TSDoc as inherent to parsing and left as is — re-rendering the original + authority by hand is a second URL renderer for no security gain. +- No changeset (D1). `tests/node-conformance/observability.test.mjs` gets the post-`send()` context case + (`AsyncLocalStorage` behaviour is Node's, and the Bun suite is the one that passed over the leak). + +**Carried to wave 6 (#81/#82):** `defaultTimeoutMs` is unvalidated on both transports (`fetch-transport.ts:73,205,215`, +`undici-transport.ts:95,401,419`) and is now the only path by which an out-of-range delay reaches +`AbortSignal.timeout()`; Node rejects `1.5`/`2**32`/`-1` with `RangeError`, Bun accepts the first two. Validate at the +transport factory the way `RequestOptionsBuilder.timeoutMs` now does (D14), with a conformance row. + + +**D16 outcome (2026-09-05, #78, PR #93).** Rule kept and written down in `classify.ts`, `io/index.ts` and item +17. Six classify cases (five leaves plus `TransportFailureError`, each asserting `isIoError`'s answer beside the +classifier's). Non-finite `delayOverride` is screened at the source and reported through the same +`http.retry.delayOverrideFailed` path a throw uses (`reportOverrideFailure`); finite negatives unchanged. +`computeDelay` short-circuits `initialDelayMs === 0`. Round 2 put the failure semantics on the `@public` +`RetryStepOptions.delayOverride` TSDoc. `core.api.md` byte-identical. + +**D17 outcome (2026-09-05, #79, PR #94).** `codec-json/src/abort-race.ts` races each pending read/write; both +Bun and Node abort cases hung to their deadlines before it. Paginator: `pageOrClose()` runs the PAGE-4 +invariants under the same `closeThenRethrow()` as `parseOrClose`; all four invariants (paginator + page) reject +`null` as well as `undefined`. `tristate()` rejects a present `undefined` too, not only `null` +(`NonNullable` excludes both). **Correction to D17's text:** the SSE double report never came from +`#releaseWithInFlightError` — it came from `#iterate`'s `finally` running `#releaseQuietly()` after the catch +had already released; fixed there. A second double report survives deliberately: `bindAbort`'s hook call plus +the `suppressed` copy when an iterator is parked in a read at abort time (the listener cannot know). Query-splice +surrogates → `UrlConstructionError` (the strategies used no class of their own). Round 2 aligned +`UrlConstructionError`'s TSDoc, both abort clauses in `seams/serde.ts`, and added rule 5 (PAGE-4) to +`write-a-paging-strategy.md`. + +**D18 outcome (2026-09-05, #80, PR #95).** `send()` runs under `runWithSnapshot` (diagnostic store) and +`runWithActiveSpan` (span), both `AsyncLocalStorage.run`; the handle forms stay with TSDoc on their reach. New +`@public` `PipelineOptions {instrumentation?, operationName?}` as `PipelineBuilder`'s optional second argument; +`StandardResilienceOptions extends PipelineOptions`; `seedFrom('flatten')` carries the options through an +`@internal` friend accessor (beyond D18's letter, so the documented derivation path does not drop the bundle). +`install` + span start inside one `try`; `endOnce` latch; LOGGING step's span in a `finally`. +`http.instrumentation.bodyCaptureFailed` at `verbose`; `LoggingStepSettings.configKey`; `pipelines.md` section +"Turning logging on from the environment". OBS-29 row closed, OBS-35 row appended. *Left for a later pass:* +`packages/core/README.md`'s Observability bullet does not yet point at `PipelineOptions.instrumentation`. + +### Wave 5 partition +| Task | Owns | +|---|---| +| #78 | `packages/core/src/retry/{classify,backoff,engine}.ts` + tests, `packages/core/src/io/index.ts` (comment), `docs/deviations.md` **item 17 section only**, `core.api.md` if a TSDoc changes | +| #79 | `packages/codec-json/**`, `packages/core/src/pagination/{paginator,page}.ts` + tests, the `tristate` schema module + test, `packages/core/src/sse/stream.ts` + tests, `docs/sdk-documentation/write-a-serde.md`, `tests/node-conformance/serde.test.mjs`, `codec-json.api.md` | +| #80 | `packages/core/src/observability/{diagnostic-context,logging-step,redaction}.ts`, `packages/core/src/pipeline/{runtime,builder}.ts`, `packages/core/src/auth/preset.ts`, `packages/core/src/context/instrumentation.ts` (TSDoc), their tests, `docs/sdk-documentation/pipelines.md` and the observability guide, `docs/deviations.md` **OBS-29 row edit + one appended row**, `tests/node-conformance/observability.test.mjs`, `core.api.md` | +Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint regions); `core.api.md` +(regenerated on the merged tree). + +## Wave 6a — landed 2026-09-05 +PR #96 (#81) merged into the umbrella at `808f6b0`. Single branch on the umbrella tip, so its own preflight (all 20 +steps passed, at `ca682c5`; round 2 changed a README only) is the merged tree's. Identity checked. + +## Wave 6b — landed 2026-09-05; the run is complete +PR #97 (#82) merged into the umbrella at `901b50e`. Single branch on the umbrella tip, so its own preflight (all +20 steps passed) is the merged tree's. Identity checked. All fifteen subtasks of #67 are merged; milestones 1–5 +are done. The seven `file:line` citations in `deviations.md` item 13 and the §10 `sideEffects` item that #81 +and #82 shifted were re-anchored on the umbrella after this merge. + +**D20 outcome (2026-09-05, #82, PR #97).** Three new `transport-shared` modules: `dispatch-classification.ts` +(allow-list of permanent recognitions — terminal argument codes, a cause-less `TypeError`, three known scheme +messages; `'bad port'` deliberately excluded because Node's `fetch` reports TRANSPORT-20's own dead-port probe +that way), `body-less.ts` (`body === null` for HEAD, 101/103/204/205/304, 2xx CONNECT; each adapter releases +the handle it declines — **Bun's `fetch` also returned a live stream for these, so both adapters needed it**), +`default-timeout.ts` (carried from #76; `TypeError` at both factories). `CONTROL_BYTE` now covers LF. +`ForkedSignal` always returns a live signal and gains `abort(reason)`, so the producer-failure branch can +cancel the native call; `producerFailure` classifies its own rejection as `TransportFailureError` so the shared +table cannot read it as permanent. **One `deviations.md` row beyond the task's letter, accepted:** the +classification departs from TRANSPORT-20's "*any* transport failure that produced no HTTP response", and +`write-a-transport.md` now prescribes the reading to third-party transports. The producer race is proven by an +instrumented-transport test, not a shared row (the suite cannot see the signal handed to the native call). +*Correction to D19's outcome:* `ftp://` on undici was already terminal via `UND_ERR_INVALID_ARG`. + +## Closing state (2026-09-05) +- Umbrella `audit/remediation-67`: fifteen task branches merged, PRs #83–#97 closed as merged, #67 checklist + fully ticked. Base `mvp` untouched at `1f48926`. +- Every merged wave preflighted on its exact tree (all 20 steps) before the merge; every task-branch commit + authored as the repository's configured identity. +- Left deliberately for the release pass: the "Deferred — release machinery" table below (fifteen changesets' + worth of notes), `docs/first-release.md`, the `ProxyType` union question, `packages/core/src/io/index.ts`'s + dead barrel, `packages/core/README.md`'s Observability pointer, the `bindAbort` second report in SSE, and the + unknown-length `pipeTo` path forwarding empty chunks. +- Not done by this run, on purpose: opening a PR from the umbrella into `mvp`, and any `gh issue create` (D4's + PIPE-37 tracking issue). Both are the maintainer's. + +## Decisions taken for wave 6 (M5: #81, then #82) — pre-taken 2026-09-05 + +### D19 — #81: degrade what undici cannot carry; one short-write detector; a typed refusal for SOCKS +- **Header rejections become logged drops (TRANSPORT-11/12).** `UNDICI_FORBIDDEN_HEADERS` gains `expect`, + `keep-alive` and `upgrade`; `connection` stays forwarded (§17's note) but a value other than `close` / + `keep-alive` is dropped and logged; every name is validated against RFC 9110 `token` in `toUndiciHeaders` + and a non-token name is dropped and logged, which is the degrade `fetch-transport.ts:146-152` already does + by `try`/`catch`. Check `FETCH_FORBIDDEN_HEADERS` against the same rows on Node's undici-backed `fetch` and + widen it only if a row proves a rejection. Conformance rows both transports run: `Expect: 100-continue`, + `Upgrade: websocket`, a non-token name — the send succeeds, the header is absent on the wire, the drop is + logged by name. +- **`fileBody` goes through `writeTo` (BODY-13).** The undici file branch stops handing `createReadStream` to + undici and takes the same `pumpBody` path the streamed case uses, so `transferred === count` runs on both + transports and a truncate-after-stat fails with `TransportFailureError` naming transferred-of-total. + *Rejected:* a byte-counting wrapper around the read stream (a second BODY-13 implementation, which is the + drift the shared pump exists to prevent). Conformance row: truncate a 1 MB file to 10 bytes after `stat`, + both transports reject identically. If the pump path changes the framing (`content-length` vs chunked), + say so in the PR and keep whatever the streamed case does today. +- **SOCKS:** `undiciTransport()` refuses `proxy.type !== 'http'` at the factory with the same `TypeError` + shape `toDispatchError` uses for a terminal misconfiguration (deliberately outside the `IoError` tree), + `@throws` documented. `ProxyType` keeps `socks4`/`socks5` — narrowing it is a public-shape decision for the + release pass (D1) — and one `deviations.md` row records "SOCKS is resolved by core and supported by neither + transport". `fetchTransport()` gets the same check if it accepts a proxy at all. +- Rows go in `run-suite.ts` + `fixtures.ts`; #82 lands on top of them (D2). `api:local` on `transport-undici` + (and `transport-fetch` if touched). No changesets (D1). +*Partition:* `packages/transport-undici/**`, `packages/transport-fetch/src/fetch-transport.ts` (forbidden set +only, if a row proves it), `packages/transport-conformance/src/**`, `docs/sdk-documentation/write-a-transport.md`, +`docs/deviations.md` (one appended row), the two transports' `api.md`. + + +**D19 outcome (2026-09-05, #81, PR #96).** `expect`/`keep-alive`/`upgrade` added to BOTH forbidden sets — a row +proved Node's undici-backed `fetch` rejects them with a *retryable* `TransportFailureError`, and Bun 1.3.14 +diverges a third way (forwards two, hangs on `upgrade`). undici gains a per-header RFC 9110 token guard and a +`connection`-value guard, all degrading to logged drops. **The undici file branch was deleted outright** rather +than rerouted, so `prepareBody` is the same two decisions on both transports; framing for a file ≤ 1,000,000 +bytes changed from chunked to `content-length`, matching fetch. SOCKS refused at the factory with a `TypeError`; +`ProxyType` keeps the union (narrowing it would breach `CFG-22`'s MUST for the model); one row. The streamed leg +of the truncate row lives in `tests/node-conformance/transport.test.mjs` because Bun's `Readable.fromWeb` leaks +the abort reason as an unhandled rejection. *Constrains #82:* new registrars `registerNativeRejectionRows`, +`registerFileBodyRows`, `registerProxyRefusalRows`, fixture `fileBodyFixture`, `TransportCapabilities.unsupportedProxy`; +the `fromWeb` leak sits in the producer-race code D20 edits; `ftp://` still reaches `dispatcher.request` unchecked. + +### D20 — #82 (after #81 merges): one classification table; `body === null` when there is none; abort the fork +- **Permanent errors on fetch:** the fetch transport classifies a `TypeError` from an unsupported scheme, an + invalid URL, a forbidden method or an invalid header exactly as the undici transport classifies its + `TERMINAL_ARGUMENT_CODES` — a bare `TypeError` with `{cause}`, outside the `IoError` tree — and the table + that decides it moves to `@dexpace/transport-shared` so the two cannot drift (the precedent is + `abort-mapping.ts`). `fetch failed` with a network `cause` stays `TransportFailureError`. Conformance row: + `ftp://` (or another scheme the runtime refuses) is non-retryable on both — assert `isIoError(e) === false`. +- **Body-less responses (204, 304, HEAD):** `Response.body` is `null` — the WHATWG shape, and what core's + model already types (`http/response.ts:18`); undici stops wrapping `result.body` unconditionally and + hands `null` for those three. Rows for 204, 304 and HEAD assert `body === null`, `contentLength`, and that + `reasonPhrase` is `undefined` or a string (the divergence is D7's row beside §10 item 13; the row does not + re-ledger it). *Rejected:* an empty stream on both (a consumer would have to read to learn there is nothing). +- **`CONTROL_BYTE`** becomes `/[\x00-\x08\x0A-\x1F\x7F]/u`, with a test in `transport-shared` for LF. +- **Producer-failure race:** the producer-failure branch aborts the forked signal before rethrowing, on both + transports, so a native call that resolves afterwards is cancelled rather than stranded with an unread body. + Verify with an instrumented transport whose native call resolves after the producer fails. +- `api:local` on `transport-shared`, `transport-fetch`, `transport-undici` as touched. No changesets (D1). +- **Carried from #76:** validate `defaultTimeoutMs` at both transport factories with the same integer + `1..2**32-1` rule `RequestOptionsBuilder.timeoutMs` uses (D14), typed error, `@throws`, a conformance row. +*Partition:* the three transport packages, `packages/transport-conformance/src/**`, +`docs/sdk-documentation/write-a-transport.md`, the three `api.md`s. + +## Deferred — release machinery (recoverable list) +| Issue / PR | Deferred item | +|---|---| +| #68 / PR #83 | patch notes for `@dexpace/core` and `@dexpace/codec-json`: shipped `.d.ts` prose changed for `Deserializer`, `jsonSerde()`, `InstrumentationBundle.tracerFactory`, `buildRequest`, `RequestConditions.applyTo` | +| #68 / PR #83 | `docs/first-release.md:117` claims `serde.ts:99,170` cite `H15` — `H15` appears nowhere in `serde.ts`; `:159` puts `deserializeFrom` at `:162` and `serializeTo` at `:96` (actual `:221`, `:104`). File suspended under D1 | +| #70 / PR #85 | patch changeset for `@dexpace/core`: redirect error messages now carry redacted URLs (and `[malformed url]` for unparseable input); `http.redirect.rejected` gained `url.full` | +| #71 / PR #86 | minor changeset for `@dexpace/core`: `BasicCredential`/`DigestCredential` become classes (breaking for object-literal callers); patch note for `authStep`'s `@throws PlaintextCredentialError` prose; `docs/first-release.md` untouched though this is its "free before the first bump" class | +| #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | +| #72 / PR #88 | minor changeset for `@dexpace/core`: retry surfaces the final typed error (was a `SuppressedError` wrapper); new `retryAttempts()` export | +| #74 / PR #89 | patch changeset for `@dexpace/core`: every `WWW-Authenticate`/`Proxy-Authenticate` value is read; `cnonce` emitted for `-sess`; empty `realm`/`nonce` declined | +| #75 / PR #87 | changeset for `@dexpace/rx` (issue says minor; only `.d.ts` prose moved, so patch is arguable): SSE ownership transfer documented | +| #73 / PR #90 | patch changeset for `@dexpace/core`: one idempotency key per logical request across retry attempts; `.changeset/2026-08-26-recovery-chain-primitives.md:14`'s "single `try`/`catch`" description of `dispatchWithRecovery` is now two functions (guarantee unchanged) | +| #76 / PR #92 | patch changeset for `@dexpace/core`: `timeoutMs` rejects non-integers and values above `2**32-1` (was accepted, threw at send); lone surrogates rejected at `QueryParamsBuilder.add` and path substitution; `getAll` frozen on every path | +| #77 / PR #91 | patch changeset for `@dexpace/core`: empty chunk in `streamBody` is a contract violation; multipart boundary quoted when not a token; logging tap safe after `close()`; `.d.ts` `@throws` prose changed | +| #78 / PR #93 | patch changeset for `@dexpace/core`: non-finite `delayOverride` ignored with a warning (was a `RangeError` after one send); `computeDelay` returns `0` not `NaN` for `initialDelayMs: 0` at overflow | +| #79 / PR #94 | patch changesets for `@dexpace/core` (paginator closes on malformed `PageInfo`; SSE release failure reported once; splice surrogate typed) and `@dexpace/codec-json` (abortable reads/writes; `tristate` rejects present null) | +| #80 / PR #95 | minor changeset for `@dexpace/core`: `PipelineOptions` (new public shape), `LoggingStepSettings.configKey`, context restored after `send()`; `.changeset/2026-09-04-per-operation-span.md`'s `createRuntime(...)` example is stale | +| #81 / PR #96 | patch changesets for `@dexpace/transport-undici` (header degrade, file body via `writeTo`, SOCKS refusal, file framing change) and `@dexpace/transport-fetch` (three names added to the drop set); `ProxyType` narrowing declined — keep the union | +| #82 / PR #97 | patch changesets for `@dexpace/transport-shared` (`CONTROL_BYTE`, classification table, body-less rule, `ForkedSignal.abort`, `requireValidDefaultTimeoutMs`), `@dexpace/transport-fetch` and `@dexpace/transport-undici` (permanent failures non-retryable; `body === null` for body-less responses; producer failure aborts the native call; `defaultTimeoutMs` validated) | diff --git a/docs/deviations.md b/docs/deviations.md new file mode 100644 index 0000000..0467a45 --- /dev/null +++ b/docs/deviations.md @@ -0,0 +1,528 @@ +# Deviations — the as-built audit, and the landing point for the rest + +Audit of `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` (the Phase 10 +reconciled ledger, 17 items) performed against the **as-built code**, not against the phase specs that +produced it. Every item was re-derived from source; each entry below records the file and line that proves +the claim. + +**Scope of this file, as of 2026-08-31:** two things, in this order. + +1. **The audit.** The deviations that are *permanently uncorrectable* — where restoring the reference + contract's own mechanism is impossible on this platform, forbidden by a project constraint, or would be a + regression. Items found to be **correctable** are deliberately **not** listed here; they were fixed + instead. That is everything below, and it is unchanged. +2. **The collection point.** A deviation found outside a phase, by a review or a maintenance pass, with no + phase ledger to write to and no permission to write to §10. It is appended under + "Deviations recorded outside a phase" at the end of this file, dated, and folded into §10 the next time §10 + is deliberately amended. **That section is no longer empty, and the paragraph that once said it was is + this one.** The 2026-08-31 restructure swept the non-frozen tree and found no unrecorded deviation there, + only three unrecorded *deferrals* (the dissolved register's U2) and six mis-numbered register citations + (the dissolved register's U6) — a clean result, and one that held only because that sweep read the + *registers*. The 2026-09-02 register audit and the 2026-09-04 code audit + ([#67](https://github.com/dexpace/nodejs-sdk/issues/67)) both read something else and both filled the + section: the second went through the shipped **code** and found MUST-level narrowings and undecided + readings that lived only in a phase spec, only in a test comment, or nowhere at all. Read the table below + for what is in it — this paragraph deliberately does not count the rows, because a stated count is the + one thing in a collection point that goes wrong on the next append. + +**This file is the audit and the mutable collection point; §10 is the ledger and it is frozen.** (The +cross-reference was added 2026-08-30, when nothing in the repo linked the two and the numbering they share had +no stated owner. The role below widened on 2026-08-31, when `docs/` gained a stated structure and §10 landed +inside a read-only tree.) + +- **`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` (§10) is the normative + ledger** — the canonical list of deliberate deviations, and the **owner of the item numbers**. Every `## N` + heading and every table row below is keyed to §10's numbering and has no independent identity. **If §10 + renumbers, this file must be renumbered in the same commit.** §10 carries the matching pointer back here. +- **§10 sits in a frozen tree.** `docs/sdk-design-nodejs/` is read-only to routine maintenance and to the + `housekeeping` skill, which refuses to write there — see [`README.md`](./README.md). Editing §10 is a + deliberate, hand-made act. **This file is not frozen**, and that asymmetry is the point: a deviation + discovered by a maintenance pass, a review, or an audit is recorded here on the day it is found, and §10 is + amended when someone deliberately amends it. What must never happen is the finding waiting for the ledger. + The coupling now spans a freeze boundary, and only one side of it can be repaired by the tool that notices + the drift — registered as the dissolved register's U4. +- **This file is the as-built audit of that ledger** — it re-derives each item from source, carries the + `file:line` evidence, and records which of §10's claims did not survive contact with the code. +- **A deviation *produced by a phase* is recorded in neither, at first.** It goes in the owning phase's own + `## Deviation Ledger (for Phase 10)` section — 24 such sections exist under `docs/work/mvp/` — and §10 is + the consolidated **output** of those, not their intake. Those sections are dated provenance and stay where + they are. +- **A deviation found *outside* a phase is recorded here.** A maintenance pass, a review of shipped code, or + an audit has no phase ledger to write to and cannot write to §10. It writes here, and the next deliberate + §10 amendment folds it in. This is the half of the intake rule that did not exist before 2026-08-31, and its + absence is why a finding with no owning phase had nowhere to go. +- **The corpus no longer carries a copy.** `docs/knowledge/deliberate-deviations.md` was a harvested topic + file derived from an older revision of §10 — a third of the register, mis-anchored, two entries false. It + was dropped on 2026-08-31: a register accumulates rows and a harvest of it is one stale revision, so §10 is + read directly. What remains under the corpus is a pointer, `docs/knowledge/notes/deliberate-deviations.md`, + which says exactly that. + +Audited 2026-08-29 against `25-phase-10-deviation-reconciliation` @ `d8217af`; the audit's own changes landed +on that branch as `27fb81f`, which is the tree this file describes. + +**What the audit changed** (committed as `27fb81f`): + +| Was | Outcome | +|---|---| +| Item 11 — `Symbol.asyncDispose` declared as a plain class member on `Page`, `FetchTransport`, `UndiciTransport` | **Code fixed.** All three now install it guarded, matching `SseStream`. On the `>=20.3` floor the computed key evaluated to `undefined`, leaving a junk `"undefined"` prototype entry and no working disposal — verified on real Node 20.3.0, before and after. `implements AsyncDisposable` and the factories' `& AsyncDisposable` return types dropped as untrue on the floor. Changeset added; API reports regenerated | +| Item 11 — the `Page` node-conformance test asserted `typeof page[Symbol.asyncDispose] === 'function'` | **Test fixed.** On the floor that read `page['undefined']`, which *was* the junk method, so the assertion passed over a `Page` that could not be disposed. Now branches on the symbol and asserts the junk key's absence on both matrix legs | +| Item 14 — `NFR-12` recorded as unverifiable | **Closed on evidence.** 644 emitted files **and 9 `npm pack` tarballs** byte-identical across two clean builds. Added `bun run verify:reproducible-build` as a blocking CI step, negative-tested by injecting a `Date.now()` into `gen-version.mjs`. (Widened 2026-08-30: the tarball comparison was a by-hand check of `@dexpace/core` alone at audit time; it is now a second leg inside the gate, over every publishable package, on both builds) | +| Item 7 — "three tiers, not four" | **Ledger corrected.** The code implements all four; only the default production binding of the property layer is empty | +| Item 4 — "never bare structural interfaces" | **Ledger corrected.** 61 exported interfaces vs 58 classes; `Configuration` is builder-built and exported structurally | +| Item 14 — "`npm publish --provenance` is scripted" | **Ledger corrected.** It is not scripted anywhere; only `prepublishOnly` is | + +Everything below is what remains genuinely uncorrectable: **fifteen** of the ledger's seventeen items. Items +**7** and **11** are gone from the count because they were correctable and were corrected; item **14** stays, +because only its `NFR-12` half closed. (`27fb81f`'s commit message says "fourteen"; it counted the split item +14 as gone. The list below is the authoritative count — fifteen `##` sections, fifteen table rows.) + +| Ledger item | Verdict | +|---|---| +| 1 Single execution model | Uncorrectable — platform | +| 2 Byte-stream provider seam | Uncorrectable — platform + `SEAM-1` | +| 3 Retry stacks unified | Uncorrectable — spec-sanctioned by `RETRY-28` | +| 4 Structural-typing encapsulation gap | Uncorrectable — language (its mitigation clause was wrong; corrected) | +| 5 Schema-as-witness | Uncorrectable — language | +| 6 Vendored MD5 | Uncorrectable — platform + `SEAM-1` | +| 8 `AbortSignal` cancellation | Uncorrectable — platform | +| 9 Freeze-once collections | Uncorrectable — and strictly better | +| 10 `NFR-8` not applicable | Uncorrectable — no surface to configure | +| 12 Cross-origin marker header | Uncorrectable — the alternative is broken | +| 13 Transport-adapter platform gaps | Uncorrectable — platform (4 of 5 clauses) | +| 14 `NFR-16` publish provenance | Uncorrectable *here* — needs a real registry (`NFR-12` split out and closed) | +| 15 ETag obs-text does not round-trip | Uncorrectable — MUST outranks SHOULD | +| 16 No async-runtime adapter fragmentation | Uncorrectable — no second ecosystem exists | +| 17 Three-level error tree | Uncorrectable — the subtyping *is* the requirement | + +Items **7**, **11**, and the `NFR-12` half of **14** are absent from this list on purpose. They were correctable, and have been corrected — see the table above. + +--- + +## 1. Single execution model eliminates every thread/CAS/interrupt-flag primitive + +**Verified.** `Transport.send()` is `Promise`-only, one method satisfying `SEAM-11` and `SEAM-16` at once +(`packages/core/src/seams/transport.ts:49`). `ContextStore` holds a plain `Map` +(`packages/core/src/context/store.ts:24`) with a fresh `Symbol()` per call +(`packages/core/src/context/context.ts:112`, defaulted per flavor at `:128` and `:149`). `Next` is +`(request?) => Promise` with no sync twin (`packages/core/src/pipeline/step.ts:23`). +`wrapCancellation` degenerates to `failure(error)` and says so +(`packages/core/src/recovery/cancellation.ts:31`). `ASYNC-18` holds: SSE parses `retryMs` +(`packages/core/src/sse/parser.ts:131`) but never acts on it — reconnection is caller-owned, so no adapter +schedules a delay outside the retry engine. + +**Why it cannot be corrected.** There is no thread to interrupt, no CAS to perform, and no clearable +interrupt flag to restore. `AbortSignal.aborted` is latched by specification — re-asserting it is not merely +unnecessary, it is not expressible. Reintroducing the distinction would mean shipping a synchronous blocking +transport, which Node's I/O model cannot provide without a worker thread and a `SharedArrayBuffer` + +`Atomics.wait` handshake — a mechanism strictly worse than the one it replaced, and one that would break +`SEAM-1`'s zero-dependency floor for the browser/Workers half of the runtime target. + +--- + +## 2. The byte-stream provider seam and its discovery machinery are removed + +**Verified.** `packages/core/src/io/index.ts` exports concrete types only; the sole surviving mention of +"provider" in `io/` is a comment recording that `IO-30`'s *resolution* half was not built +(`packages/core/src/io/factories.ts:13`). There is no registry, no install precedence, no conflict +resolution — `IO-39` ships nothing. + +**Why it cannot be corrected.** `SEAM-3`–`SEAM-10` exist to keep a *third-party* stream library out of a +zero-dependency core. Web Streams are a runtime built-in. A discovery mechanism needs at least two +candidate implementations to discover between; there is exactly one, and adding a second would require a +runtime dependency that `verify:seam-1` fails the build over. The machinery would be ceremony with an empty +registry behind it. + +`SEAM-18`'s three bridge clauses (caller-supplied executor, async-wrapper unwrapping, interruptible blocking +wait) inherit item 1's impossibility — they presuppose the blocking transport that cannot exist. Its one +non-bridge clause survives and is enforced as an ordinary obligation on `send()` +(`packages/core/src/seams/transport.ts:49`). + +--- + +## 3. Two retry stacks collapse into one, with the total-timeout budget explicitly opt-in + +**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:367`) — with exactly two thin +callers: the pillar step (`packages/core/src/retry/retry-step.ts:151`) and the dispatch adapter +(`packages/core/src/retry/retry-dispatch.ts:85`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | +undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for +`RETRY-28` (`packages/core/src/retry/settings.test.ts:20`). + +**Why it cannot be corrected.** This is not a deviation the port chose against the spec — `RETRY-28` is the +spec instructing a unifying port to make the budget opt-in. "Correcting" it means splitting one engine into +two that differ in no observable behavior, then re-deriving `RECOV-17`–`RECOV-34` against the duplicate. +The ledger entry documents conformance, not a gap. + +--- + +## 4. True runtime encapsulation of domain models is not fully achievable + +**Verified.** Domain models use `#private` fields and a TS `private` constructor reached through the +`createX` friend hook, so `build()` cannot be bypassed for a *class* — e.g. +`packages/core/src/http/request-conditions.ts:59-76`. + +**Why it cannot be corrected.** TypeScript's type system is structural and erased. Nothing at runtime +distinguishes a `Headers` instance from an object literal that satisfies the same shape, and no compiler flag +changes that. A nominal-typing emulation (a branded `#private` witness field) would only move the check to +call sites that must then be written to perform it, and would still be defeated by a cast. This is a +language-level ceiling. + +> **Corrected in the ledger 2026-08-29 — the text was wrong, the deviation is not.** Item 4's stated mitigation — "exporting only +> concrete classes, never bare structural interfaces, from each package's public entry point" — is false as +> written. `packages/core/etc/core.api.md` exports interfaces and classes in comparable numbers, and at least +> one interface is a builder-built, validated, frozen type: `Configuration` is `export interface Configuration` +> (`packages/core/src/config/configuration.ts:100`) returned from `ConfigurationBuilder.build()`, and +> `setGlobalConfiguration()` / `resolveProxyOptions()` accept any hand-rolled object of that shape. Most +> other exported interfaces are seams (`Transport`, `Serde`, `Logger`) or options records, where structural +> typing is the point. The *deviation* is real and uncorrectable; the *mitigation sentence* overstated what +> the package actually does and has been narrowed to the `http/` wire-model types. + +--- + +## 5. Schema-as-witness replaces reflective generic-type capture + +**Verified.** `Serde` is not generic in `T` (`packages/core/src/seams/serde.ts:241`); the witness is a +decode-time parameter, `deserialize(data: Uint8Array, target: DecodeTarget): T` +(`packages/core/src/seams/serde.ts:194`, and `deserializeFrom` at `:221` takes the same target), +where `DecodeTarget` bundles the `Schema` witness with its optional diagnostic label +(`:122-124`). The witness moved into that object on 2026-09-04 — the SPI took it positionally as +`(data, schema, typeName?)` until then, which is the signature this row quoted; the deviation is +unchanged by the reshaping, since a schema *value* is still what stands in for a reflected type token. +The codec-configuration knobs are absent and documented as absent — +`packages/codec-json/src/json-serde.ts:244,250` explain that `SERDE-23`'s unknown-field policy belongs +to the schema and that `SERDE-21`/`22` have no coercion setting because there is no coercing codec. +`packages/codec-json/src/conformance.test.ts:8` states outright that no code implements `SERDE-21` or +`SERDE-22`. + +**Why it cannot be corrected.** `SERDE-5`–`SERDE-8` are worded around a reflectively reconstructed type +token. JVM generics erasure leaves a raw `Class` behind; TypeScript erases to nothing — there is no runtime +artifact of `T` at all to reflect over. Nor can the missing knobs be added: `JSON.parse`/`JSON.stringify` +expose no coercion, unknown-field, or date-format hooks to gate. A hand-rolled JSON parser could expose +them, at the cost of correctness, performance, and a large maintenance surface, to gate settings the schema +witness already decides more precisely. + +--- + +## 6. Digest MD5 needs a vendored implementation; SHA-256 does not + +**Verified.** `packages/core/src/auth/md5.ts` is a hand-rolled RFC 1321 implementation whose header states +the reason. SHA-256 goes through `globalThis.crypto.subtle.digest('SHA-256', bytes)` +(`packages/core/src/auth/digest.ts:103`). + +**Why it cannot be corrected.** Web Crypto excludes MD5 by design, on security grounds — it is not an +oversight to work around, and no flag re-enables it. The two alternatives are both closed: an npm MD5 +dependency fails `verify:seam-1`'s zero-runtime-dependency gate, and `node:crypto` would forfeit the +browser/Deno/Workers portability that motivated choosing Web Crypto in the first place. RFC 7616 still +requires MD5/MD5-sess for interop with servers that have not moved to SHA-256, so dropping it is not an +option either. + +--- + +## 8. Cancellation is `AbortController`/`AbortSignal` end-to-end + +**Verified.** `composeSignal` folds a caller signal and a timeout into one via `AbortSignal.any` +(`packages/core/src/seams/transport.ts:74`). The timeout-vs-cancellation split reads the structured +`reason.name` field, explicitly *not* `instanceof`, because `instanceof` is realm-bound +(`packages/core/src/seams/transport.ts:109`). + +**Why it cannot be corrected.** `Promise` has no `cancel()`, unlike `CompletableFuture`; cancellation in +JavaScript is cooperative by construction. The `reason.name` check is not a shortcut around a class +hierarchy that could be built — `AbortSignal.timeout()` and a caller `abort()` both deliver a `DOMException` +through the same signal type, and the runtime chooses the class. There is no seam at which the port could +substitute its own. + +> Minor wording nit: the ledger says "the abort reason's *constructor name*". The code reads the `name` +> **property** (`reason.name === 'TimeoutError'`), which is deliberate and stronger — a constructor-name +> check would break across realms exactly as `instanceof` does. + +--- + +## 9. Frozen collections are computed once, not wrapped on every read + +**Verified.** `Headers` freezes each value array, the lookup map, the casing map, and the insertion order +once at build time, then freezes the instance (`packages/core/src/http/headers.ts:314-319,345`). + +**Why it cannot be corrected.** The reference's per-access unmodifiable wrapper exists to guard a mutable +backing collection. These models are immutable after construction, so there is no window in which a +re-wrap could observe a different value. Restoring per-access wrapping would allocate on every getter to +defend against a mutation that cannot occur. The one genuine residue is already recorded elsewhere in the +project: `Request.url` clones per access because the native `URL` really is mutable. + +--- + +## 10. `NFR-8` (shrinker keep/retain configuration) is not applicable + +**Verified.** `packages/shrink-test/` exists and targets the dual-package `instanceof` hazard +(`packages/shrink-test/src/fixture-app.ts:111`, `run-shrink-guard.test.ts:8` — both line numbers refreshed +2026-08-30 after the fixture grew a third probe). Nothing in the workspace carries a keep-rule, and there is +no reflective lookup for one to protect. + +**Why it cannot be corrected.** A keep-rule names a symbol a static analyzer cannot see is reachable. +`NFR-8`'s premise is JVM reflection; JS bundlers have no equivalent blind spot, and item 2 already retired +the one discovery mechanism the port might have had. There is no symbol to keep-configure, so the +configuration file would be empty by construction. The structurally equivalent JS risk is covered instead. + +**What the guard now also pins, and why it is the standing evidence for a manifest decision (added +2026-08-30).** The `[Symbol.asyncDispose]` repair replaced three class members with a **module-scope +`Object.defineProperty` statement** — a top-level side effect — in four files across three packages that all +declare `"sideEffects": false` (`packages/core/src/sse/stream.ts:209`, +`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:366`, +`packages/transport-undici/src/undici-transport.ts:674`; the manifest field at `packages/core/package.json:25`, +`packages/transport-fetch/package.json:26`, `packages/transport-undici/package.json:26`). That field entitles a +bundler to drop a module whose exports go unused, and nothing forbids a future one from also dropping a +top-level statement it judges inert — which would silently un-install disposal in the shipped artifact while +every type still checks, the same failure shape this guard already exists for. `fixture-app.ts`'s +`probeDisposalSymbol` therefore constructs a `Page` and a `FetchTransport` **inside the bundled, minified, +tree-shaken artifact** and asserts the member is present and callable; `run-shrink-guard.ts` exits non-zero on +any `false` field of `FixtureResult`, so the check is blocking. + +**`sideEffects` was deliberately not narrowed** to the four file paths that carry an install. Narrowing is +more fragile, not less: the list would silently go stale on any file move or rename, and a stale narrow list +fails *open* — the bundler drops the module and nothing complains. The shrink guard tests the property that +actually matters (the install survives a real `bundle + minify + treeShaking` pass) rather than a manifest +proxy for it, and it needs no maintenance when a file moves. Budget note: the added probe took the measured +bundle from 16,671 to 17,689 bytes against a 24 KiB budget (`packages/shrink-test/shrink-test.config.ts`). + +--- + +## 12. The redirect/auth cross-origin marker is a real header, not a `WeakSet` + +**Verified.** `CROSS_ORIGIN_MARKER_HEADER` is set, cleared, and tested per hop +(`packages/core/src/redirect/cross-origin.ts:80,93,104,118`). The auth step reads it once on the outbound +pass (`packages/core/src/auth/auth-step.ts:395`) and gates both branches on the answer — preemptive stamping +at `:401`, and whether to react to a challenge at all at `:786`, not merely whether to stamp. The answer +rides across the dispatch on `OutboundPlan.crossOrigin` (`:375`, and `:370-372` for why), because the marker +itself is cleared from the request before it reaches the wire. An independent `POST_AUTH` backstop strips it +even in a pipeline with no auth step (`packages/core/src/redirect/strip-marker-step.ts`), satisfying +`REDIR-11(c)`'s porter caveat. + +**Why it cannot be corrected.** The reference's in-process marker was tried and withdrawn during Phase 5b's +own drafting: retry's attempt-stamping sits between redirect and auth and produces a **fresh `Request` +copy**, which a `WeakSet` keyed on object identity no longer recognizes — the marker would silently +vanish exactly on the hop that most needs it. Restoring the identity-based marker means either removing +attempt-stamping (breaking `RETRY-38`) or making `Request` mutable (breaking `HTTP-2`/`HTTP-5`). + +The two interpretive questions Phase 10 settled here — `REDIR-20`'s predicate scope and Basic/Digest never +stamping preemptively — are confirmed against the code and stand as decided. Both are security-conservative +readings; reversing either would widen an attack surface for a caller convenience the spec never asked for. + +--- + +## 13. Transport adapters have platform-shaped gaps the reference does not + +**Verified.** `Protocol.HTTP_1_1` is hardcoded in both adapters +(`packages/transport-fetch/src/fetch-transport.ts:201`, `packages/transport-undici/src/undici-transport.ts:429`). +`transport-fetch` documents having no `proxy` option at all +(`packages/transport-fetch/src/fetch-transport.ts:77-83`). The proxy `challengeHandler` is surfaced with a +warning rather than dispatched (`packages/transport-undici/src/challenge-handler.ts:27,50`). + +**Why it cannot be corrected.** Four of the five clauses are closed by the platform, not by choice: + +- **Negotiated protocol version.** Neither `fetch`'s `Response` nor undici's `ResponseData` carries it. + There is no API to read, so the best-effort default is the only honest answer available. +- **Zero-copy `sendfile(2)` (`TRANSPORT-28`, SHOULD).** No user-space path in either client reaches the + syscall; a raw `node:net` transport would be a different product. +- **`TRANSPORT-8`'s native-cancel-vs-timeout distinction.** §17's own text scopes the clause to transports + that *have* an internal cancel path. `transport-fetch` does not, so the clause does not bind it. +- **Proxy `challengeHandler` on undici.** undici's `ProxyAgent` takes its credential solely from its own + constructor and rejects a per-request `Proxy-Authorization` with `InvalidArgumentError` — a deliberate + security fix upstream. The constructor runs before any challenge exists, so a handler-minted credential + can never reach the exchange that provoked it. This is unfixable without vendoring undici internals. Note + that the Phase 8a *plan* specified a retry-with-stamped-credential flow that is simply not implementable + on this platform; the shipped fallback (WARN at construction, WARN on first `407`, Basic via + `ProxyOptions.credentials`, `407` returned untouched) is the correct disposition. + +The fifth clause, `transport-fetch` shipping no proxy support (`TRANSPORT-30`), is a **deliberate scope +boundary rather than an impossibility** — it is achievable, at the cost of depending on `undici` internals, +which would defeat the package's zero-dependency purpose. `@dexpace/transport-undici` is the supported +answer for callers who need proxying. Recorded here for completeness, not as a platform limit. + +--- + +## 14. `NFR-16` — publish provenance + +**Verified.** `prepublishOnly` is wired in all nine publishable packages (e.g. +`packages/core/package.json`). ~~There is **no** release workflow — `.github/workflows/` contains `ci.yml` +only — and the string `provenance` appears in no `package.json`, no workflow, and no `.npmrc` (there is no +`.npmrc`).~~ + +**Superseded 2026-09-02.** `.github/workflows/release.yml` now exists: it triggers on a push to +`main`, runs `changesets/action@v1`, declares `id-token: write`, and sets +`NPM_CONFIG_PROVENANCE: 'true'`. So `provenance` is now scripted, and the "no release workflow" +statement above is false. There is still no `.npmrc`, which is correct — `changesets/action` writes +one from `NPM_TOKEN` at run time. + +**Why it cannot be corrected here.** `NFR-16`'s conformance test is behavioral: "a CI/release build fails an +unsigned publication; a local build without keys still publishes unsigned." Satisfying it requires a real +`npm publish --provenance` against a real registry with a real OIDC token. Nothing in this repository can +produce that evidence; it unblocks at first release and not before. + +> **Corrected in the ledger 2026-08-29.** Item 14 claimed `prepublishOnly` *and* `npm publish --provenance` +> "are scripted (Phase 0 Task 3)". Only the first is. `docs/work/mvp/2026-09-04-open-items-dissolution.md`'s Section D row +> "Publish + provenance CI job" ([`#d-nfr-16-provenance`](./work/mvp/2026-09-04-open-items-dissolution.md#d-nfr-16-provenance)) already recorded this +> accurately ("`prepublishOnly` wired; nothing published yet"); §10 did not, and now does. +> +> ~~**Still actionable, and not done here:** authoring the release workflow with `--provenance` and +> `id-token: write` is doable today — it is only *exercising* it that needs a registry. That is the one +> remaining piece of work this audit identified and deliberately did not perform, because a release workflow +> is an outward-facing artifact whose shape (trigger, environment, tag convention, who may publish) is a +> project decision rather than a defect repair.~~ +> +> **Done 2026-09-02.** `.github/workflows/release.yml` is authored. It is **inert until an `NPM_TOKEN` +> repository secret exists** — without one `changesets/action` cannot authenticate, so it maintains +> the "Version Packages" pull request and publishes nothing. Two prerequisites that blocked the first +> real publish are now one: +> +> - **Fixed 2026-09-02** — no manifest carried a `repository` field, which npm requires before it +> will accept `--provenance`. All nine publishable manifests now carry one; the two private +> packages deliberately do not. +> - **Still open, and a maintainer call** — `.changeset/config.json` sets `"access": "restricted"`, +> which conflicts with provenance: attestations go to a public transparency log and require a +> public package. Publishing privately and publishing with provenance cannot both be true. +> +> The *behavioural* half of `NFR-16` is unchanged and still uncorrectable here, for the reason stated +> above: it needs a real registry and a real OIDC token. +> +> **`NFR-12` was split out of this row and closed on evidence** — 644 emitted files and 9 `npm pack` tarballs +> byte-identical across two clean builds, and a new blocking CI gate +> (`scripts/verify-reproducible-build.mjs`). It is no longer part of this file's scope. *Widened 2026-08-30: +> at audit time the tarball evidence was a by-hand `npm pack` of `@dexpace/core` alone, asserted rather than +> gated. Both legs now run inside the gate — `digestTarballs()` packs every non-`private` package on each of +> the two builds and diffs the SHA-256 maps — so the claim above is verified on every CI run rather than on +> the day it was written.* + +--- + +## 15. A server-issued ETag containing obs-text does not round-trip + +**Verified.** `RequestConditions.applyTo` writes every entity tag through the outbound `Headers` builder's +`set` (`packages/core/src/http/request-conditions.ts:133-146`), which enforces `HTTP-18`'s HTAB + printable +ASCII 0x20–0x7E rule (`packages/core/src/http/ascii-validation.ts:16`). The inbound path is separately laxer +and permits obs-text, exactly as `HTTP-19` requires +(`packages/core/src/http/ascii-validation.ts:41`, `packages/core/src/http/headers.ts:246,262`). + +**Why it should not be corrected.** This one is *technically* correctable — a relaxed emit path for replayed +ETags could be added — and the decision is that it must not be. `HTTP-18` is **MUST**-level and its rationale +is header-injection safety, reinforced by `XCUT-18`, which the product spec treats as a universal invariant +that binds "even if each subsystem individually appears to work." `HTTP-48`'s obs-text permission is +**SHOULD**-level RFC conformance for a rare case, mostly legacy servers. A SHOULD-level nicety does not +outrank a MUST-level cross-cutting security invariant, and adding the relaxed path would create precisely +the two-emit-paths condition that makes splitting defenses fail in practice. Permanent by decision. + +--- + +## 16. Async-runtime adapter fragmentation does not exist + +**Verified.** `packages/rx/` is the only adapter, and its `sseEvents$`/`typedSse$` are documented as +single-subscription because `SseStream` wraps an already-consumed-once response body +(`packages/rx/src/sse.ts:17,41`). No coroutine, reactor, netty, or virtual-thread equivalents exist. + +**Why it cannot be corrected.** The reference's adapter set exists because the JVM has several competing +async ecosystems the SDK must pivot between. Node has one: `Promise`. There is no second ecosystem to bridge +to, so the adapters have no counterpart to be written against. `@dexpace/rx` is sugar over a genuinely +different *data shape* (push-based `Observable`), not the same plumbing under another name — and its +single-subscription behavior is forced by HTTP itself, since a consumed response body cannot be re-read. + +--- + +## 17. `TransportFailureError` adds a third level to a two-level error tree + +**Verified.** `IoError extends DexpaceError` (`packages/core/src/io/errors.ts:13`); the four I/O leaves — +`EndOfStreamError`, `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` — each +extend `DexpaceError` **directly** (lines 29, 51, 67, 83) and are grouped by the `isIoError` predicate +(line 108) rather than by a middle tier. `TransportFailureError extends IoError` (line 132) is the single +three-level branch. + +**Why it cannot be corrected.** `TRANSPORT-20` requires `TransportFailureError` to *be* an `IoError` — the +subtyping is the requirement, not an artifact of modelling. It is also load-bearing, and **what it bears is +a boundary, not a category**: `classify.ts`'s cause-walk tests `current instanceof IoError` +(`packages/core/src/retry/classify.ts:90`), and because the tree is flat that branch matches exactly two of +the six classes in `io/errors.ts` — `IoError` itself and `TransportFailureError`. It does **not** match the +four leaves `isIoError` groups. That is the intended reading, decided by audit #67 / #78 and stated here +because it is this deviation's own consequence: the branch means "the wire failed". A send that produced no +response is `RETRY-4`'s unconditionally-retryable condition, `TRANSPORT-20` names `TransportFailureError` as +what carries it, and the `extends` is what routes it to the retry layer with no edit there. A flat sibling +would have to be enumerated by hand in the retry classifier, and again for every transport added later — +trading one level of depth for an open-ended maintenance obligation that the styleguide's own rule exists +to prevent. Held at exactly three; a fourth level is not sanctioned by this entry. + +**What the other four leaves are, and why they stay outside the branch.** `EndOfStreamError`, +`SourceContractViolationError`, `ClosedResourceError` and `AllocationLimitError` are this package's own +contract and lifecycle failures, and every one of them is deterministic on re-send: a closed resource +(`IO-42`) and a source that returned zero bytes for a positive read (`IO-17`) are caller programming errors, +an allocation cap (`IO-9`) is a limit the same request hits again, and `EndOfStreamError` is the +exact-length-copy contract inside `io/` — a *wire* truncation is the transport's to report, as a +`TransportFailureError`, which is the layer that can tell one from a complete short body. `RETRY-2`'s "an +I/O error" is read as that boundary. Widening the branch to `isIoError` would retry all four; only +`EndOfStreamError` was ever a candidate, and `io/` is the wrong layer to decide whether a stream ended early +because the wire broke. Six cases in `packages/core/src/retry/classify.test.ts` pin one answer per class, +each asserting both what `isIoError` says and what the classifier says, so the disagreement is recorded as a +decision rather than an oversight — and re-parenting any leaf under `IoError` turns four of them red. + +> **Anchor correction 2026-09-04 (audit #67 / #68), rule supplied 2026-09-05 (audit #67 / #78).** The line +> numbers in this section were stale on 2026-09-04 and were re-derived then; `classify.ts`'s branch moved +> from `:73` to `:90` on 2026-09-05 when the paragraph above it was written. The substantive point this +> section used to make — "the cause-walk returns retryable for any `IoError`" — read as covering all five +> classes `isIoError` names, and it never did. #68 recorded the gap and left the answer to #78; #78 chose to +> keep `instanceof IoError` and to say what it means, which is the two paragraphs above. Nothing in the +> deviation itself (the three-level branch, and why it stays) ever turned on that answer. + +## Deviations recorded outside a phase + +Every row here was found by a pass over shipped code rather than produced by a phase. The +`docs/work/mvp/2026-09-04-open-items-dissolution.md` register audit (that file's Section V) opened the +section on 2026-09-02; later reviews and audits append to it. Deliberately uncounted — a stated total is a +number that goes wrong on the next append, which is that file's U10. + +A row here has no owning phase — it was found by a review, an audit, or a maintenance pass over shipped code. +It is recorded on the day it is found rather than waiting for `docs/sdk-design-nodejs/10-…`, which is in a +frozen tree and is amended only deliberately, by hand. When §10 is next amended, a row here becomes a numbered +§10 item and moves into the audit above under that number. + +| Deviation | Found by | Date | Evidence | §10 status | +|---|---|---|---|---| +| **`HTTP-11`'s range classifications are on `Status` only, not mirrored onto `Response`.** The spec places them on both: `docs/product-spec/04-core-http-domain-model.md:23` reads "Status MUST classify by range … **and a response MUST expose these derived from its status**", and appendix C (`appendix-c-consolidated-normative-requirement-index.md:47`) restates it as "Response MUST expose these same classifications derived from its status." The port ships them once, on `Status`, reachable as `response.status.isSuccess`. Six delegating getters on `Response` would duplicate surface that cannot drift, since there would be one implementation behind both — but the letter of the requirement does name the response, so the reading is recorded rather than assumed | the dissolved register's A3, register audit | 2026-09-02 | `packages/core/src/http/status.ts` carries all six; `packages/core/src/http/response.ts` carries `status` and no classification of its own | not yet in §10 | +| **`REDIR-20`'s "fully override" is read as scoped to code/method eligibility, not to the safety mechanics that follow it.** A configured redirect predicate replaces the built-in follow decision; it does **not** bypass userinfo stripping, credential hygiene, the downgrade guard, body replayability, or loop/cap detection. Those are stated as unconditional MUSTs elsewhere in the same chapter and are not "should this kind of redirect be followed" policy — a caller predicate opting to follow a 307 with a single-use body still cannot make that body re-sendable. Genuinely ambiguous wording, decided one way and now recorded as decided | the dissolved register's G4, register audit | 2026-09-02 | `packages/core/src/redirect/decide.ts` consults `settings.predicate` at the eligibility gate and runs every later guard unconditionally; pinned by "the predicate does NOT bypass the safety mechanics" in `decide.test.ts`. Phase 9's conformance sweep was to re-confirm this and closed without doing so | not yet in §10 | +| **`OBS-29` is carried by spans rather than by the named tracer callbacks; both halves are now met, and the operation span is reachable from the public API.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`, and the requirement is discharged under that vocabulary. **Ordering:** `Runtime.send` opens one span before the drive and ends it exactly once, behind an `ended` latch that a throwing `end()` cannot get past (`pipeline/runtime.ts:78-96`), and the per-attempt spans the LOGGING pillar step opens follow the same shape from a single `finally` (`observability/logging-step.ts:469`, then `:457,494`). **1:1 binding:** the span is opened outside every pillar, so a retry's second attempt and a redirect's second hop stay inside the first call's span (`pipeline/runtime.ts:57-67,280-282`); before 2026-09-05 a leaked `enterWith` made the *previous* call's ended span read as active and suppressed the next one, so only the first operation per async context got a span at all — the binding was stated and not delivered. **Caller reachability**, which this row carried as the open half from 2026-09-02 to 2026-09-05, is closed: `PipelineOptions` (`pipeline/builder.ts:32-50`) is `@public`, is the second constructor argument of `PipelineBuilder` (`:71`) and is extended by `StandardResilienceOptions` (`auth/preset.ts:23,115-118`), so `createInstrumentationBundle`'s result has somewhere public to go. What remains a deviation is only the **vocabulary**: a consumer sees `startSpan`/`end`/`recordException`, not `operationStarted`/`operationSucceeded`/`operationFailed`, and appendix C's own note that "pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced" (`appendix-c-consolidated-normative-requirement-index.md:509`) is now out of date for this port | the dissolved register's L1/V2, register audit; anchors re-derived by audit #67 / #68; finished by audit #67 / #80 | 2026-09-02, re-anchored 2026-09-04, closed 2026-09-05 | `packages/core/src/observability/span.ts:54-56` (`Tracer`, one method — declared in `tracing.ts` until 2026-09-04, when audit #67 / #69 moved the inert tracing declarations into their own module to break an import cycle; `tracing.ts` re-exports them, so the public path is unchanged); `packages/core/src/pipeline/runtime.ts:57-67,78-96,206-233,280-282` (the operation span, its single `end()`, and the `run`-scoped stores that keep the binding true across calls); `packages/core/src/pipeline/builder.ts:32-50,71,293` and `packages/core/src/auth/preset.ts:23,115-118` (the public route to a bundle); `observability/logging-step.ts:457,469,494` (the per-attempt span); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | +| **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` and `context/` both carry them, so the rule is applied where it earns its place. Counted qualitatively on purpose — the two figures this cell used to state were wrong by the time anyone read them; re-derive with `grep -rn 'invariant(' packages/core/src/ --include='*.ts' \| grep -v '\.test\.'` | not yet in §10 | +| **`PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path, and the port implements `REDIR-22`.** Two MUSTs name the same trigger and prescribe opposite dispositions. `docs/product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `docs/product-spec/10-redirect-handling.md:22` (`REDIR-22`): "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." The port closes, then throws, on three grounds: `REDIR-6` independently fixes the control flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response never *returned* cannot be "returned unclosed"; specific governs general, since §10 of the spec owns the redirect step's lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return unclosed as it requires. **The erratum this needs is proposed below and is not applied here**, because `docs/product-spec/` is frozen and correcting a normative sentence is the specification owner's act, not a maintenance one | the dissolved register's G1, Phase 5b design's Deviation Ledger | 2026-09-04 | `packages/core/src/redirect/redirect-step.ts` closes before throwing, with the reasoning asserted inline in `redirect-step.test.ts`; nothing in the code waits on the erratum | not yet in §10 | +| **`PIPE-37`'s outermost pre-redirect status-mapping step was never built, and until this row nothing recorded that.** `PIPE-37` (MUST) requires a step whose correctness depends on the single terminal response — status-to-typed-error mapping is its own worked example — to occupy the outermost pre-redirect slot, so it runs outside both the redirect and the retry loop. The port ships the *mapping*, but as `statusMappingStep`, a `ResponseStep` on the response-recovery chain (RECOV-15/RECOV-16), not as a pipeline `Step` carrying `stage: 'PRE_REDIRECT'`. The slot itself exists and is installable — `config/clientIdentityStep` occupies it today — so this is a wiring gap, not a missing mechanism. It has an owner in writing and the owner never took it: Phase 4's checklist marked the row ⏳ and said "the obligation lands on whichever phase wires 4b's `statusMappingStep` into a real pipeline — **Phase 5**", and Phase 5 shipped without it, with no deferral carrying the hand-off forward. That is why a code audit found it and no checklist did. **Ledgered rather than implemented.** Installing a `PRE_REDIRECT` mapping step is public pipeline surface with a real behaviour change behind it, and the petstore spike arrives at the same work from the other side: its finding 2 wants a declarative `StatusErrorMap` applied *at* the `toHttpError` call site rather than wrapped around it, so a mapped error class can see a decoded payload instead of raw bytes. Whoever does one does both | audit #67 / #69 | 2026-09-04 | `packages/core/src/recovery/status-mapping.ts:26` (`statusMappingStep(response: Response): Promise` — a `ResponseStep`, with no `StepDescriptor` and no stage); `packages/core/src/pipeline/stage.ts:39` (`PRE_REDIRECT` heads `STAGE_ORDER`) and `packages/core/src/config/client-identity-step.ts:124` (the one shipped step that occupies it); `docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md:141` (the dropped hand-off); `docs/product-spec/08-execution-pipelines.md:26` (PIPE-37 shares that line with PIPE-25/36/38); `examples/petstore/FINDINGS.md:96-97` | not yet in §10 | +| **`REDIR-3`'s eligibility test reads the CURRENT hop's method, where the spec says the ORIGINAL request's.** `isEligibleByCode` is handed `currentRequest.method`, the method of the request being redirected at this hop. The two readings agree on every chain but one: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 or 302 arriving on that rewritten hop is then followed under the default {GET, HEAD} set, where the literal reading would refuse it because the request that started the chain was a POST. **Kept, deliberately.** The rewritten GET is idempotent and carries no body — REDIR-5 dropped it — so the literal reading buys no wire safety here, only a refusal; and refusing would make `allow303` half-useful, opting into the rewrite but not into anything the rewritten request can reach. Every other guard on the hop is unaffected, since eligibility is step 3 of eight and the userinfo strip, downgrade guard, replayability gate, loop detection and hop cap all run after it regardless. Switching to the literal reading stays a narrow, mechanical change — thread the seed request's method through `RedirectContext` — if a later reading of §10 wants it | audit #67 / #69 | 2026-09-04 | `packages/core/src/redirect/decide.ts:241` passes `currentRequest.method` into `packages/core/src/redirect/codes.ts:69`'s `eligibility.allowedMethods.has(method)`; `docs/product-spec/10-redirect-handling.md:8` is the "ORIGINAL request method" wording. Pinned by "a 303-rewritten GET makes a following 301 eligible under the default method set" in `packages/core/src/redirect/decide.test.ts`, which goes red the moment the reference point moves | not yet in §10 | +| **`PAGE-19`'s own conformance fixture `; rel=next` does not end the stream on this platform — it is followed.** The requirement's normative sentence is "a target that cannot resolve into a valid URL MUST be treated as end-of-stream", and its illustrative conformance note offers `` as an instance. Under WHATWG `URL` — the resolver `strategies.ts` uses, and the only RFC 3986 resolver available without a runtime dependency (SEAM-1) — a supplied base makes that string a perfectly ordinary relative *path* reference: it resolves to `/repo/not%20a%20url`. It resolves, so the port follows it. **The normative half is satisfied exactly as written**: a target that genuinely fails `new URL(target, base)` returns the page with no next request and throws nothing. Only the fixture disagrees, and it disagrees because it was written against a resolver that rejects a space. **Rejected:** an ad-hoc "looks unparseable" heuristic in front of the resolver, which would have to guess at strings RFC 3986 defines, and would make the followable-relative-reference clause of the same requirement wrong instead | audit #67 / #69 | 2026-09-04 | `packages/core/src/pagination/strategies.ts:115-120` (the `new URL(target, response.request.url)` resolve, with the `catch` returning `pageInfo(items)`); `docs/product-spec/12-pagination.md:52` is the conformance note. Both halves pinned in `packages/core/src/pagination/strategies.test.ts` — "the spec's `` fixture is a RELATIVE reference, so it is followed" and "an unresolvable target ends the stream rather than throwing" | not yet in §10 | +| **`Request.equals` compares the body by reference identity, where `HTTP-46` says by value.** `HTTP-46` requires equality to compare "method, headers, and body by value", with only the URL singled out for textual comparison. The port compares method, `url.href` and headers by value as required, and the body with `===`. **Why it stays.** `Body` is a lifecycle object, not a value: a `StreamBody` is single-use (BODY-9) and reading its bytes to compare them consumes it, which would make `equals` destructive — an equality operator that empties its operands is worse than one that under-reports. The variants that *could* compare cheaply (`ByteArrayBody`, `StringBody`) would give a comparison whose cost and semantics depend on which variant a caller happened to build, which is the drift `HTTP-1`'s value-model rules exist to prevent. Reference identity is sound in the safe direction — it never reports two different bodies equal — and it is what `Request.equals`'s own TSDoc has always said it does. Archived as blocked in the dissolution record and in neither §10 nor the register; recorded here so it is somewhere a reader will look | audit #67 / #69 | 2026-09-04 | `packages/core/src/http/request.ts:137` (`this.#body === other.#body`, beside the by-value method, `url.href` and `headers.equals` comparisons on 134-136); `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:82` | not yet in §10 | +| **`IO-13`'s "symmetric write-side encodings" ship for UTF-8 and ISO-8859-1 only; every other charset throws on write.** The read side stays fully general through `TextDecoder`. The write side does not, because `TextEncoder` is UTF-8-only — there is no `TextEncoder('iso-8859-1')` — and SEAM-1 forbids adding an encoding library to `@dexpace/core`. ISO-8859-1 is therefore hand-rolled as the direct code-point-to-byte map, which is also what lets the decode side round-trip it: WHATWG maps the label `iso-8859-1` onto windows-1252, so delegating would break the symmetry `IO-13` is about. Any other label raises `IoError` naming the charset rather than silently re-encoding as UTF-8 and corrupting the bytes on the wire. `IO-13`'s own conformance note names ISO-8859-1 as the non-UTF-8 case, so the two shipped encodings are the two the requirement exercises. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** — it never reached §10 or this file, which is the gap this row closes | audit #67 / #69 | 2026-09-04 | `packages/core/src/io/text-codec.ts:32-51` (`encodeText`, and the `unsupported write charset` throw at :36-39); `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:406` | not yet in §10 | +| **`BODY-9`'s mark/reset replay path for a stream-backed body is not built: `StreamBody` is always single-use.** `BODY-9` is a SHOULD, and it is conditional — replayable "when and only when the stream supports mark/reset". Node's `ReadableStream` has no generic mark/reset to support, so the condition is never met and the SHOULD's own fallback ("otherwise it MUST be single-use") is the branch that applies. `StreamBody.replayable` is a hardcoded `false`, which every consumer already reads: `decide.ts` fails a non-303 redirect carrying one (REDIR-6), and the retry pillar refuses to re-send it. A caller who wants replay materializes first or uses `byteArrayBody`. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/stream-body.ts:24` (`readonly replayable = false`); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:441` | not yet in §10 | +| **`BODY-34`'s one shared preview cap covers the two logging tees only, not `toHttpError`'s error-body capture.** Read literally as "every in-memory capture in the package", `BODY-34` would put the request-side tee, the response-side drain and `toHttpError`'s error buffer behind one configurable value. They cannot share one: `HTTP-52` **fixes** the error-body cap at 1 MiB, and a spec-fixed constant cannot also be the configurable shared setting. The two capture sites `BODY-34` actually names — the request-side tee-capture and the response-side drain-on-first-access — do share one cap, so the requirement's own enumeration is satisfied; only the wider reading is not. `http-status-error.ts` states the split at the constant rather than leaving it to be inferred. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/http-status-error.ts:17-19` (`ERROR_BODY_CAP_BYTES`, with the "Deliberately NOT BODY-34's shared preview cap" note); `packages/core/src/body/response-body-logging.ts:47` and `packages/core/src/body/response-body-logging.ts:105-115` (the shared `cap` and the bounded drain that honours it); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:447` | not yet in §10 | +| **`IO-38`'s cross-thread close visibility has no subject on this platform and is recorded as not applicable, not as satisfied.** The requirement presupposes that a source or buffer instance can reach a second thread, so that closing it there invalidates a slice being read here. None can. Class instances are not structured-cloneable at all — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: it holds a `ReadableStreamDefaultReader`, which is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject there either. This row exists because "not applicable" and "done" look identical in a checklist and are not the same claim. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** | audit #67 / #69 | 2026-09-04 | `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:401`; `packages/core/src/io/buffered-source.ts:47` (the reader it holds); no `worker_threads`, `postMessage` or `structuredClone` call exists anywhere in `packages/core/src/` | not yet in §10 | +| **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:203` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:425-441` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | +| **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | +| **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | +| **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 | +| **`AUTH-22`'s "emit cnonce/nc/qop only when qop is negotiated" is not applied to `cnonce` for a `-sess` algorithm.** A `-sess` HA1 is `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so the client nonce is an *input to the hash* for `MD5-sess` and `SHA-256-sess` whatever `qop` the challenge offered. The port implemented AUTH-22 to the letter: it drew a fresh cnonce, folded it into HA1, and then omitted it from the header whenever `qop` was absent — a response no server can verify, because it has no way to reconstruct HA1. AUTH-30 bounds the re-challenge replay to one 401, so every such exchange simply failed. **`cnonce` is now emitted for any `-sess` algorithm; `nc` and `qop` stay conditional exactly as AUTH-22 says**, because RFC 2069's response input is `H(HA1:nonce:HA2)` and carries no nonce count, so emitting one would advertise a count the response was not computed over. RFC 7616 §3.4 states the wider rule outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's clause is RFC 2617's RFC 2069-compatibility form, written before `-sess` existed, and the requirement's own AUTH-15 mandates both `-sess` algorithms, so the two sentences cannot both be followed. *Rejected:* declining a `-sess`-without-`qop` challenge instead, which turns every such server into a guaranteed 401 for no security gain, when the value the server needs has already been computed | audit #67 / #74 | 2026-09-05 | `packages/core/src/auth/digest.ts:345-350` (the `-sess` HA1 that consumes the cnonce) against `:405-408` (`buildHeaderValue`, where the `else if` now emits it); `docs/product-spec/11-authentication.md:18` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:357` are AUTH-22's wording. Pinned by the `digestHandler -sess without qop (AUTH-17/AUTH-22)` block in `packages/core/src/auth/digest.test.ts` — one row asserting the header carries `cnonce` and neither `nc` nor `qop`, one recomputing the response from the header's OWN cnonce so a value drawn twice would fail — and by the `MD5-sess, no qop` vector in the same file | not yet in §10 | +| **`RETRY-44`'s "downstream chain" is read as everything BELOW the retry point, which in the recovery stack excludes the request chain.** The requirement has two clauses: each attempt re-executes the downstream chain with fresh per-attempt state, and "upstream steps MUST NOT mutate the shared in-flight request between attempts". The port originally read the first clause as covering the *whole* recovery chain and re-ran `RequestRecoveryChain.apply()` per attempt, with a test that said so by name. That makes `packages/core/src/recovery/idempotency-key.ts` generate a fresh key on every attempt, so three attempts of one logical request reach the server as three unrelated writes — the precise failure `RECOV-32` exists to prevent, and the opposite of what that step's own `@public` TSDoc promises. **The chain is now applied once, above the loop; each attempt re-executes transport plus response chain over `stampAttempt`'s fresh copy of the prepared request.** Under this reading both clauses hold and the second holds *by construction*: upstream steps cannot mutate the in-flight request between attempts because they no longer run between attempts. The pillar stack is untouched — there "downstream" is the forked continuation (`ctx.fork()`), and `retryStep` still re-drives it per attempt. *Rejected:* memoizing the key on the template (a `WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice would replay the key and have the server drop a genuine second call. *Rejected:* re-running the chain over the *prepared* request each attempt — the chain would read its own output, which is clause two's mutation in different clothes, and every shipped and caller-written step would have to be proven idempotent. One consequence recorded rather than assumed: the re-send gate (`RETRY-5`/`RECOV-18`) now judges the prepared request rather than the caller's, which is what a retry would actually re-send | audit #67 / #73 | 2026-09-05 | `packages/core/src/retry/retry-dispatch.ts:83-88` (the chain applied once, then `runWithRetry`) against `:27-33` (the per-attempt half); `packages/core/src/recovery/orchestrator.ts:62` (`prepareRequest`) and `:117` (`dispatchPrepared`); `packages/core/src/retry/engine.ts:243` is the gate that now reads the prepared request. `RETRY-44`'s wording is `docs/product-spec/09-retry-and-resilience.md:35` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:306`. Pinned by `packages/core/src/retry/retry-dispatch.test.ts:126` (chain applied once), `:169` (one `generate()`, one key on three sends), `:201` (the `RETRY-38` ordinal varies while the key does not) and `:227` (a request-chain throw is not retried and meets the recovery phase exactly once) | not yet in §10 | +| **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | +| **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | +| **`OBS-35`'s "MUST NOT bake in a default config key name" is satisfied by making the key configurable, not by removing the default.** `OBS-35` (SHOULD) asks for a tolerant, layered log-level resolution and adds one MUST: no baked-in default key name. The port ships `CFG_KEY_LOG_LEVEL` (`DEXPACE_LOG_LEVEL`) as `CFG-14`'s well-known key and, until 2026-09-05, read it unconditionally. It is now `LoggingStepSettings.configKey`'s default: a caller names their own key and the resolution is otherwise identical. **Why the default stays.** A required key would mean no caller gets ambient granularity without naming one first, which trades a MUST about *naming* for a worse default experience, and `CFG-14` — which this port also implements — exists precisely to standardise the name. The layered resolution itself is `CFG-1`'s (override → environment → normalised property → default) and is tolerant as the requirement asks. **A second, quieter half:** the process-wide configuration slot starts empty (`CFG-13`), so no key of any name resolves until a host calls `setGlobalConfiguration(defaultConfiguration())`. That is deliberate — defaulting the slot to a configuration that reads `process.env` would make an import-time environment read the SDK's default behaviour — and it is now documented as the required wiring rather than left to be discovered | audit #67 / #80 | 2026-09-05 | `packages/core/src/observability/logging-step.ts:64-79,94-104` (the setting and the resolution); `packages/core/src/config/configuration.ts:311,354-358,368` (`CFG_KEY_LOG_LEVEL`, `defaultConfiguration`, the empty default slot); `docs/sdk-documentation/pipelines.md` "Turning logging on from the environment"; `docs/product-spec/15-instrumentation-and-observability.md:66` (the requirement) | not yet in §10 | +| **`CFG-22`'s SOCKS proxy types are resolved by the configuration layer and supported by neither shipped transport; the refusal is at the transport factory, and `ProxyType` keeps them.** `CFG-22` (MUST) requires the proxy model to carry "the proxy protocol type (HTTP, SOCKS4, SOCKS5)", and the port implements it in full: `ProxyType` is `'http' \| 'socks4' \| 'socks5'`, and `resolveProxyOptions` maps `ALL_PROXY`/`HTTPS_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto it. Nothing can then send over one. `@dexpace/transport-undici` builds undici's `ProxyAgent`, which is an HTTP `CONNECT` tunnel reading its `uri` as a URL, and `@dexpace/transport-fetch` ships no `proxy` option at all because Node's bare global `fetch` exposes no proxy hook outside undici internals. So a configuration that resolves cleanly has no transport that can honour it. **What changed on 2026-09-05 (audit #67 / #81).** Until then the discovery was `new ProxyAgent({uri: 'socks5://…'})` throwing undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` out of a public factory — untyped, undocumented, and outside the SDK's error vocabulary. `undiciTransport()` now refuses `proxy.type !== 'http'` at construction with a `TypeError` naming the type, before any dispatcher is allocated, deliberately outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable (RETRY-2). That is `TRANSPORT-30`'s "make the limitation discoverable rather than silently misbehaving" applied at the earliest point that can. **Why `ProxyType` still admits `socks4`/`socks5`.** Narrowing a `@public` union is a breaking change and therefore a release-pass decision, which this run is not taking (the run's release machinery is suspended); and `CFG-22`'s MUST is about the *model*, which would then no longer satisfy it. The honest state is a configuration layer that is complete and a transport layer that is not, which is what this row records. A future transport — a `node:net` SOCKS dialer, or a `ProxyAgent` replacement — closes the gap without a model change. | audit #67 / #81 | 2026-09-05 | `packages/core/src/config/proxy.ts:34` (`ProxyType`), `:372-380` (the scheme map); `packages/transport-undici/src/undici-transport.ts:147,160-167,201` (the supported type, the refusal, and where it runs); `packages/transport-fetch/src/fetch-transport.ts:79-82` (no `proxy` option, and why); `docs/product-spec/16-configuration.md:42` (`CFG-22`); `docs/product-spec/17-transport-adapter-conformance-contract.md:48` (`TRANSPORT-30`) | not yet in §10 | +| **`TRANSPORT-20`'s "any transport failure that produced no HTTP response" is read as an exchange that failed, not as a request the native client refused to make.** `TRANSPORT-20` (MUST) names four instances — connection refused, DNS/TLS failure, peer reset, connect/read timeout — and requires the retryable `TransportFailureError`. A scheme the client will not speak (`ftp://`), a forbidden method (`CONNECT`), a method that is not a token, a body on a GET: all of them also produce no HTTP response, so the literal reading makes them retryable too. The port refuses that. `retry/classify.ts:90` is an allow-list over `instanceof IoError`, so retryable would mean the caller's entire retry budget spent re-proving a URL that cannot change between attempts, and the requirement's own enumeration is four ways an *exchange* fails, not four ways an argument is rejected. Such a refusal surfaces as a bare `TypeError` carrying the native error as `cause`, outside the `IoError` tree, which is the same class both transports already raise for a misconfiguration caught at construction. **What changed on 2026-09-05 (audit #67 / #82).** The reading is not new — `@dexpace/transport-undici` has applied it to undici's `UND_ERR_INVALID_ARG` / `UND_ERR_NOT_SUPPORTED` since Phase 8a — but it was recorded only in that phase's checklist, and `@dexpace/transport-fetch` did the opposite for the identical condition: every native rejection became `TransportFailureError`. The decision now lives in one table in `@dexpace/transport-shared` that both adapters call, so the two cannot answer differently again, and `docs/sdk-documentation/write-a-transport.md`'s rule 4 tells a third transport to use it. **The MUST is still the default.** The table is an allow-list of three positive recognitions and everything else falls through to retryable; `'bad port'` is excluded by name, because port 1 is on WHATWG's blocked list and so `TRANSPORT-20`'s own dead-port conformance probe arrives with that reason on Node's `fetch`. | audit #67 / #82 | 2026-09-05 | `packages/transport-shared/src/dispatch-classification.ts:17,40,81,123` (the two tables, the predicate and the mapping, with the `'bad port'` exclusion documented at `:35`); `packages/transport-fetch/src/fetch-transport.ts:340` and `packages/transport-undici/src/undici-transport.ts:308,317` (the two call sites); `packages/core/src/retry/classify.ts:89-90` (the allow-list this is answerable to); `docs/product-spec/17-transport-adapter-conformance-contract.md:38` (the requirement); `docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md:57` (where the reading was recorded before this row). Pinned by `packages/transport-conformance/src/run-suite.ts:387` on both adapters and by `tests/node-conformance/transport.test.mjs:351`, whose runtimes report the same refusal in entirely different shapes | not yet in §10 | + +### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) + +The narrower of the two edits, and the one that leaves both requirements true. `PIPE-40` is the +general rule and needs only to stop naming a trigger that `REDIR-22` has already claimed; `REDIR-22` +is correct as written and should not be touched. + +`docs/product-spec/08-execution-pipelines.md:20`, currently: + +> on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the +> in-flight response MUST be returned unclosed. + +Proposed: + +> on paths that abandon a re-drive and **return** the in-flight response (redirect cycle, budget +> exhausted) that response MUST be returned unclosed. Where a path instead **fails** — a +> non-replayable body under `REDIR-6`, a rejected downgrade — `REDIR-22` governs and the response +> MUST be closed before the error propagates. + +Applying it is a deliberate hand edit to a frozen tree by whoever owns the specification. Until then +the dissolved register's G1 carries the live pointer, and the behaviour is chosen, tested and unaffected +either way. diff --git a/docs/first-release.md b/docs/first-release.md new file mode 100644 index 0000000..a65cac6 --- /dev/null +++ b/docs/first-release.md @@ -0,0 +1,187 @@ +# First release + +Nothing in this repository has been published. All nine publishable packages sit at `version: "0.0.0"`, +[`.github/workflows/release.yml`](../.github/workflows/release.yml) is authored and **inert**, and `NFR-16` +— publish provenance — is the one requirement that cannot be closed without a real registry. This note is +the release-readiness record: what is already wired, what the release mechanics are confirmed to be, the +blockers that must clear before a first publish can succeed, and the changes whose deadline is the first +version bump rather than the publish itself. It is edited as those blockers clear and those decisions are +taken. + +**Where it came from.** This was the `NFR-16` row of `docs/deferred-items.md` until 2026-09-04, the day that +register was dissolved. Four of its ten rows were decided that day (the dissolved register's Section W); the five that +survived beside this one are unscheduled deferrals with a trigger and nothing to act on, and they were +archived into +[`work/mvp/2026-09-04-register-retirement-purge.md`](./work/mvp/2026-09-04-register-retirement-purge.md). +This material earned a file of its own instead, at the `docs/` root beside [the dissolved open-items register](./work/mvp/2026-09-04-open-items-dissolution.md) +and [`deviations.md`](./deviations.md), because it is a live document rather than a dated record — the +blockers below are things someone will do, and this is where they get struck out. It took on a second kind +of content the same day: two the dissolved register's rows whose only stated trigger was this release moved here, for +the same reason, and are the last section below. + +**Where the requirement is ledgered.** `NFR-16` is a **SHOULD**: published artifacts are cryptographically +signed for provenance, with signing enforced on the release/CI path and gracefully optional in local builds. +Item 14 of +[`sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`](./sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md) +is where its intended verification is recorded — run `prepublishOnly` and a real `npm publish --provenance` +— and where it was split from `NFR-12`, which the two-clean-builds gate closed on evidence on 2026-08-29. +the dissolved register's Section D carries the same row under its `d-nfr-16-provenance` anchor, and the workflow's own +header comment points here. + +## What is wired + +**The release workflow, authored 2026-09-02.** `.github/workflows/release.yml` triggers on `push` to `main` +and on nothing else (`:32-34`), with a second `if: github.ref == 'refs/heads/main'` guard on the job +(`:56`) so a copied trigger cannot publish from a branch. It pins Bun from `.bun-version` exactly the way +`ci.yml` does (`:67-69`), installs with `--frozen-lockfile` (`:71-72`), runs `bun run build` so a broken +tree fails before the registry is touched (`:80-81`), and hands `changesets/action@v1` the lockfile-pinned +binary directly — `publish: node_modules/.bin/changeset publish` (`:98`), not `bun run changeset publish` +(which routes through `scripts/changeset.mjs`, a rename wrapper) and not `bun x` (which would fall back to +fetching the CLI from the registry). It sets `permissions: {contents: write, pull-requests: write, id-token: +write}` (`:43-49`) — `id-token: write` is what makes provenance possible at all, since npm exchanges the +GitHub OIDC token for the signed attestation — `NPM_CONFIG_PROVENANCE: 'true'` (`:108`) so every +`npm publish` the action drives is a provenance publish rather than a flag one call site can forget, and a +`concurrency` group keyed on the workflow and ref with `cancel-in-progress: false` (`:39-41`) so two pushes +to `main` cannot publish concurrently and a publish already writing to the registry is never killed +mid-flight. + +*Corrected 2026-08-29, kept because §10 Item 14 once claimed otherwise:* before that date only +`prepublishOnly` was wired. `--provenance` appeared in no `package.json`, no workflow and no `.npmrc`, and +there was no release workflow at all. There is still no `.npmrc`; `changesets/action` writes one from +`NPM_TOKEN` at run time. + +**`prepublishOnly`, on all nine publishable packages.** Each runs +`bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm` (for +example `packages/core/package.json:34`), so npm aborts that package's publish if the build, the API-report +check, the release-shape lint or the types-resolution check fails. The runner satisfies it out of the frozen +install: Bun, `@microsoft/api-extractor`, `publint` and `@arethetypeswrong/cli` are all root +devDependencies. The two `private` packages — `@dexpace/shrink-test` and `@dexpace/transport-conformance` — +carry no `prepublishOnly`, because nothing publishes them. + +**Release mechanics, confirmed by the maintainer 2026-09-02.** Releases run from `main` only, so no branch, +tag or manual dispatch can publish. Every package is still at `version: "0.0.0"`, so the **first +`changeset version` run sets the initial published version for all nine at once** — there is no per-package +history to reconcile and no partially released set to inherit. A change lands on `main` with a changeset, +the workflow opens or updates a "Version Packages" pull request, and merging that pull request runs the +workflow again with no changesets left, which is the run that publishes. + +## Prerequisite already met — the `repository` field + +The row this note replaces recorded two unmet manifest prerequisites. **One landed in `d64a107` and is +confirmed met as of 2026-09-04.** All nine publishable `package.json` files carry + +```json +{"type": "git", "url": "git+https://github.com/dexpace/nodejs-sdk.git", "directory": "packages/"} +``` + +naming the repository `git remote -v` reports as `origin`, so npm's rule that `--provenance` needs a +`repository` resolving to the source repository is satisfied. The two `private` packages carry none, which +is correct: nothing publishes them, so nothing checks them. + +## Blockers + +Three, in the order they will bite. None is fixed by anything in this repository's gates, which is why they +are written down rather than tested for. + +1. **The `NPM_TOKEN` repository secret does not exist.** `changesets/action` writes `~/.npmrc` from + `NPM_TOKEN` when it is set and skips publishing entirely when it is not, so the workflow publishes + nothing today: it will still open and maintain the "Version Packages" pull request, and the publish step + is a silent no-op. **This one is the maintainer's to do** — a repository secret is not a file anyone can + land here. + +2. **`.changeset/config.json` sets `"access": "restricted"`.** Verified still `restricted` on 2026-09-04. + Provenance attestations go to a public transparency log and require a public package, so this conflicts + directly with `NPM_CONFIG_PROVENANCE: 'true'` in the release workflow and the first publish fails at the + registry. It is a decision, not an oversight: set the access to `public` to publish with provenance, or + keep the scope private, drop the provenance setting, and record `NFR-16` as a deviation. The root + [`README.md`](../README.md)'s "Releases" section states the same choice, and adds the third condition the + registry imposes independently — npm issues provenance attestations for public **source** only. + +3. **Exercising the provenance path needs a real registry.** `NFR-16`'s conformance test is behavioral — a + CI/release build fails an unsigned publication, while a local build without keys still publishes unsigned + — so it needs a real registry and a real OIDC token. Nothing short of a first real publish verifies it, + which is why a SHOULD-level requirement is open with the workflow already written. + +## Decisions owed before the first version bump + +The blockers above are what stops a publish from *succeeding*. These two are a different thing and must not +be confused with them: neither breaks a publish, and a release that ignores both works. They are changes +that are free today and expensive after the version bump, so the first `changeset version` run is their +deadline rather than their obstacle. + +**Why they live here.** Both were `UNSCHEDULED` items in [the dissolved open-items register](./work/mvp/2026-09-04-open-items-dissolution.md) — `H10` and +`H15`, Section H, Phase 6a — and both stated the same single trigger: *the pre-publish breaking-change batch, +before the first non-`0.0.0` release*. That batch is a release decision, so it belongs in the +release-readiness record rather than in a register of discoveries made after the work. Their IDs stay +reserved: the `### H10` and `### H15` headings remain in the dissolved register's as `MOVED` stubs pointing here, so +every citation of them still resolves — `packages/core/src/seams/serde.ts:99,170` cite `H15` from TSDoc +`@remarks`, and the Phase 6a checklist cites both. + +**Why the batch has a deadline at all.** Every publishable package is at `version: "0.0.0"`, and semver's +initial-development carve-out — which Phase 3b's validation review already invoked once, for a narrowing of +its own — stops applying at 1.0. "Release mechanics" above is the mechanism: the first `changeset version` +run sets the initial published version for all nine packages at once, so that run is the last moment either +change below is free. After it, each is a major-version break taken against consumers who are already there. + +### `H10` — one concept, two spellings across the seam and the handler layer + +`Deserializer.deserialize(data, schema, typeName?)` takes the schema and its diagnostic label positionally; +`decodeResponse`/`decodeSuccessResponse` bundle the identical pair as `DecodeTarget`. Both ship public, in +the same api-extractor report. + +Each layer's choice is locally right. The positional form is three parameters, inside `max-params`, and +`Deserializer` is an SPI a third-party codec *implements*, where a positional shape is the smaller burden on +the implementer. The object form exists because positionally the handlers would be four parameters, which is +a lint error. The pair is nonetheless globally inconsistent: a codec author implements one spelling while a +caller uses the other. `docs/knowledge/harvested/api-design.md:14` ("optional parameters collected into a +single options object rather than a positional list past two parameters") points at the object form for both. + +**The direction is already decided; only the timing is open.** Recorded on 2026-09-02 so a later reader does +not re-derive it: + +> **Unify on the `DecodeTarget` object form.** `docs/knowledge/harvested/api-design.md:14` points there, +> the handler layer already uses it, and a codec author implementing one spelling while a caller uses the +> other is the cost being paid every day it stays split. + +It was not taken in the Phase 6a review pass because it is a breaking change to a published SPI, and a +review pass is not the place to take one alone. + +### `H15` — no `AbortSignal` on two stream-driving SPI methods + +The project-wide position was decided 2026-09-02 and is stated once: + +> **A signal is required where the API drives a stream it did not open. Buffered-bytes APIs take none.** + +Under that rule `toHttpError` and `Response.bytes()` take buffered bytes and correctly take no signal, and +SSE and pagination are long-lived I/O consumers and correctly do. Two sites fall on the other side of the +line and therefore owe one: + +- `Deserializer.deserializeFrom(source: ReadableStream, …)` — `packages/core/src/seams/serde.ts:162` +- `Serializer.serializeTo(value, sink: WritableStream)` — `:96` + +Both already carry a TSDoc `@remarks` citing the item, so the obligation is visible where the method is read +rather than only in a register. The corpus rules that apply are +`docs/knowledge/harvested/concurrency-and-async.md:18` ("every long-running async API must accept an options +object with `{ signal }`"), `:20` (accepting must be paired with honoring), and `:44` (a signal must reach +the actual I/O primitive). + +**The mitigation is verified rather than assumed, and it bounds the cost of declining.** Abort **is** honored +transitively today: a transport that errors the body stream on abort makes `reader.read()` reject, the read +loop exits promptly, and `deserializeFrom` surfaces the `DOMException` cleanly. What is *not* interruptible +is the CPU-bound `JSON.parse` / `schema.parse` span after the drain completes, which no signal could cancel +without a streaming parser — and `JSON.parse` has no incremental form to build one on. So what is missing is +the parameter, not the behaviour; and adding a parameter to a published SPI is exactly the break that has a +deadline. + +### The decision + +Run the batch before the first `changeset version`, or decline it and carry both as permanent post-1.0 +major-version debt. There is no third option that keeps either change free. + +The two are **one break, not two.** They touch the same file — `packages/core/src/seams/serde.ts` — and +`H15`'s own text says so: *"H10's batch: same file, same break."* Taking either one means the other costs +nothing extra. + +Whatever is taken needs a changeset (`bun run changeset`, not `bunx changeset`) and a regenerated +[`packages/core/etc/core.api.md`](../packages/core/etc/core.api.md) — both changes are to `@dexpace/core`'s +public seam, which the committed API report records and `bun run api` blocks on. diff --git a/docs/knowledge/README.md b/docs/knowledge/README.md new file mode 100644 index 0000000..caf89ce --- /dev/null +++ b/docs/knowledge/README.md @@ -0,0 +1,31 @@ +# docs/knowledge/ + +Two trees, one query surface. `bun run knowledge` reads both; nothing else should read either by hand. + +| Tree | Holds | Rule | +| --- | --- | --- | +| `harvested/` | What the source documents say. Roles `spec`, `design`, `styleguide`. | Generated by the `knowledge-harvest` skill. **Never hand-edited.** | +| `notes/` | What the implementation found. Role `review`, a manual `sha:` marker. | Hand-written. A note overrides the harvested entry it names. | + +**Why `harvested/` is never hand-edited.** A `` line's sha digests the whole *source file*, not the +entry, so every entry harvested from one document carries the same value and an edit to an entry's text +changes no sha. The next harvest cannot see the edit: it regenerates the original text, or writes a +duplicate. A correction made in place is therefore scheduled for silent deletion. Make it in `notes/`. + +**How a note names its target.** Every entry has a stable key, `/<8 hex>`, digested from the entry's +own text. A note cites that key; the CLI resolves it, tags the harvested entry `[overridden by notes/…]`, +and `bun run knowledge --key ` looks one up. The key changes exactly when the rule's text does — so a +re-harvest that rewords a rule breaks the citation on purpose, and `bun run knowledge:drift` reports it. + +**A register is not harvested.** `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` +and `docs/open-items.md` are ledgers every phase appends to; any harvest of one is a stale fraction of it. +Read them directly. `notes/deliberate-deviations.md` is the pointer. + +**Gates.** `bun run verify:knowledge-structure` is blocking in CI and keeps the two trees apart. +`bun run knowledge:drift` is a hand-run report over source shas and note citations; it never fails a build. + +Re-harvesting: `--corpus docs/knowledge/harvested`, always. The skill's default is this directory, which no +query reads — the structure gate rejects a topic file stranded here for that reason. A `supersede` +resolution the harvest emits must be moved to `notes/` by hand, or the gate rejects it too. + +The workflow for reading and writing all of this is `.claude/skills/knowledge-lookup/SKILL.md`. diff --git a/docs/knowledge/deliberate-deviations.md b/docs/knowledge/deliberate-deviations.md deleted file mode 100644 index c0c6983..0000000 --- a/docs/knowledge/deliberate-deviations.md +++ /dev/null @@ -1,39 +0,0 @@ -# deliberate-deviations - -## Rules - -## Constraints - -## Conclusions -- None of the port's Node-idiomatic mechanism substitutions narrow a MUST-level correctness guarantee from the reference contract; each substitutes an equivalent, differently-shaped mechanism where the JVM-specific one the requirement was worded around does not exist in Node. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:3-6` · high · sha:f9ecb6e7d87b -- The synchronous transport seam and the asynchronous transport seam collapse into a single `Promise`-returning `Transport.send()` satisfying both requirements' letter simultaneously, because Node has no blocking-I/O execution model to give the sync/async distinction meaning. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:8-12` · high · sha:f9ecb6e7d87b -- The byte-stream provider seam is no longer pluggable because Web Streams are a runtime standard rather than a third-party library, so `@dexpace/core` implements the byte-stream contracts directly with no discovery/installation machinery. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:13-16` · high · sha:f9ecb6e7d87b -- Async-runtime adapter fragmentation does not exist in the port because `Promise` is Node's only ecosystem-wide async primitive, so no bridge modules equivalent to the JVM reference's coroutine/reactor/netty/virtual-threads adapters are shipped, and the one optional adapter shipped (`@dexpace/rx`) is sugar over a genuinely different push-based data shape rather than plumbing for the request/response pivot. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:17-21` · high · sha:f9ecb6e7d87b -- The two retry stacks collapse into one, with the total-timeout budget made explicitly opt-in, a substitution the spec itself sanctions by requiring that a port that unifies retry entry points MUST make that budget explicitly opt-in. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:22-24` · high · sha:f9ecb6e7d87b -- True runtime encapsulation of domain models is not fully achievable because TypeScript's structural typing means a hand-built object literal can still impersonate a public interface type and bypass builder validation, even though ECMAScript `#private` fields close the "official construction path" hole; this acknowledged language-level limitation is mitigated, not eliminated, by exporting only concrete classes rather than bare structural interfaces from each package's public entry point. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:25-30` · high · sha:f9ecb6e7d87b -- The generic-erasure defense uses schema-as-witness rather than reflective type capture, because TypeScript erases types more completely than JVM generic erasure and leaves no raw class token to reflect over, and this substitution is argued to be at least as strong a guarantee, not a weaker one. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:31-35` · high · sha:f9ecb6e7d87b -- Single-threaded execution collapses the JVM reference's atomic compare-and-set guard for the materialize-once body race into a synchronous check-and-set, correct only if the guard executes before the guarded async function's first `await`, a precondition stated explicitly because it is the one place the simplification could be silently misapplied. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:36-40` · high · sha:f9ecb6e7d87b -- Digest MD5 needs a vendored implementation while SHA-256 does not, because the Web Crypto API deliberately excludes MD5, so the port vendors a small, dependency-free MD5 implementation for RFC 7616 interoperability and uses `crypto.subtle` directly for SHA-256/SHA-256-sess. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:41-43` · high · sha:f9ecb6e7d87b -- Configuration layering has three tiers rather than four because the system-property tier is lost outright, Node having no ambient key/value store distinct from environment variables to fill that slot, and the port does not fabricate one. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:44-46` · high · sha:f9ecb6e7d87b -- Cancellation is `AbortController`/`AbortSignal` end-to-end rather than an interrupt-and-restore-a-flag discipline, composing the same signal type across the transport call, the retry backoff wait, and a derived per-call timeout; since `Promise` has no public `cancel()` unlike `CompletableFuture`, cancellation is cooperative end-to-end, and a `send()` implementation must check `signal.aborted` after resuming from an `await` before treating a resolved value as deliverable. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:47-53` · high · sha:f9ecb6e7d87b -- Frozen collections are computed once rather than wrapped on every read, satisfied by `Object.freeze`-ing each collection exactly once at construction and returning the same frozen reference from every subsequent getter call, cheaper than the reference's per-access unmodifiable-wrapper pattern because the port's models never change after construction. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:54-57` · high · sha:f9ecb6e7d87b -- The dead-code-survival gate targets a different risk than the JVM reference, since JS bundlers have no reflection blind spot to guard against, so `@dexpace/shrink-test` instead targets the dual-package hazard of two copies of `@dexpace/core` breaking cross-package `instanceof` checks after a bundle-and-tree-shake round trip. - design · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:58-62` · high · sha:f9ecb6e7d87b - -## Reference - -## Conflicts - -## Superseded diff --git a/docs/knowledge/INDEX.md b/docs/knowledge/harvested/INDEX.md similarity index 89% rename from docs/knowledge/INDEX.md rename to docs/knowledge/harvested/INDEX.md index 36d9ff7..3cde49d 100644 --- a/docs/knowledge/INDEX.md +++ b/docs/knowledge/harvested/INDEX.md @@ -1,5 +1,7 @@ # Knowledge Index +Generated. The two-tree contract, and why nothing here is hand-edited, is in `../README.md`. + | topic | file | entries | roles | conflicts | last harvest | | --- | --- | --- | --- | --- | --- | | api-design | `api-design.md` | 32 | styleguide | 0 | 2026-07-25 | @@ -10,7 +12,6 @@ | configuration | `configuration.md` | 51 | design, spec | 0 | 2026-07-25 | | cross-cutting-invariants | `cross-cutting-invariants.md` | 7 | spec | 0 | 2026-07-25 | | data-modeling | `data-modeling.md` | 27 | styleguide | 0 | 2026-07-25 | -| deliberate-deviations | `deliberate-deviations.md` | 13 | design | 0 | 2026-07-25 | | documentation | `documentation.md` | 21 | styleguide | 0 | 2026-07-25 | | error-handling | `error-handling.md` | 43 | spec, styleguide | 0 | 2026-07-25 | | execution-context | `execution-context.md` | 33 | spec | 0 | 2026-07-25 | @@ -22,7 +23,7 @@ | naming-conventions | `naming-conventions.md` | 32 | styleguide | 0 | 2026-07-25 | | observability | `observability.md` | 63 | design, spec | 0 | 2026-07-25 | | package-and-dependency-layout | `package-and-dependency-layout.md` | 31 | design, spec | 0 | 2026-07-25 | -| pagination | `pagination.md` | 61 | design, spec | 0 | 2026-07-25 | +| pagination | `pagination.md` | 61 | design, spec | 1 | 2026-07-25 | | performance | `performance.md` | 35 | styleguide | 0 | 2026-07-25 | | pipeline | `pipeline.md` | 83 | design, spec | 1 | 2026-07-25 | | redaction-and-security | `redaction-and-security.md` | 22 | spec | 0 | 2026-07-25 | @@ -41,3 +42,7 @@ | typescript-idioms | `typescript-idioms.md` | 19 | styleguide | 0 | 2026-07-25 | | url-and-query-encoding | `url-and-query-encoding.md` | 20 | design, spec | 0 | 2026-07-25 | | variables-and-declarations | `variables-and-declarations.md` | 14 | styleguide | 0 | 2026-07-25 | + +38 topic files, 1457 entries — 1451 in the type sections plus +6 conflict statements. A conflict statement records what two documents each say; its +resolution, where one exists, is a note under `../notes/`. diff --git a/docs/knowledge/SOURCES.md b/docs/knowledge/harvested/SOURCES.md similarity index 90% rename from docs/knowledge/SOURCES.md rename to docs/knowledge/harvested/SOURCES.md index 2873015..95b8c43 100644 --- a/docs/knowledge/SOURCES.md +++ b/docs/knowledge/harvested/SOURCES.md @@ -1,5 +1,12 @@ # Harvested Sources +Every source `harvested/` is derived from, with the sha256 of the whole file at harvest time — per **file**, +never per entry, which is why an entry here cannot be hand-corrected. See `../README.md`. +`bun run knowledge:drift` compares each digest below against the file on disk; the 16 styleguide rows are +`NOT VERIFIABLE` off the harvest machine, which is expected and never a failure. +`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` was harvested once and +dropped on 2026-08-31: it is a register, not a description. + | source | role | sha256 | last harvest | | --- | --- | --- | --- | | `/home/mohammad/Projects/dexpace/styleguide/typescript/01-formatting-and-tooling.md` | styleguide | `640652667e83` | 2026-07-25 | @@ -49,4 +56,3 @@ | `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md` | design | `d546f9973c4e` | 2026-07-25 | | `docs/sdk-design-nodejs/08-instrumentation-and-configuration.md` | design | `35281a426195` | 2026-07-25 | | `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md` | design | `2d2fd9dcfee4` | 2026-07-25 | -| `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` | design | `f9ecb6e7d87b` | 2026-07-25 | diff --git a/docs/knowledge/api-design.md b/docs/knowledge/harvested/api-design.md similarity index 100% rename from docs/knowledge/api-design.md rename to docs/knowledge/harvested/api-design.md diff --git a/docs/knowledge/assertions.md b/docs/knowledge/harvested/assertions.md similarity index 100% rename from docs/knowledge/assertions.md rename to docs/knowledge/harvested/assertions.md diff --git a/docs/knowledge/authentication.md b/docs/knowledge/harvested/authentication.md similarity index 89% rename from docs/knowledge/authentication.md rename to docs/knowledge/harvested/authentication.md index d85c0a3..227810e 100644 --- a/docs/knowledge/authentication.md +++ b/docs/knowledge/harvested/authentication.md @@ -3,83 +3,83 @@ ## Rules - A port must preserve the security invariants of never leaking credentials over plaintext or cross-origin, an unpredictable Digest cnonce, and secret redaction, along with the deterministic resolution semantics and the exact challenge/retry lifecycle. spec · `docs/product-spec/11-authentication.md:3` · high · sha:efba58233dd1 -- The recognized auth scheme set MUST be exactly {OAUTH2, API_KEY, BASIC, DIGEST, NO_AUTH}, where NO_AUTH is a distinct sentinel meaning 'may run anonymously / skip credential stamping' rather than a wire scheme. +- The recognized auth scheme set MUST be exactly {OAUTH2, API_KEY, BASIC, DIGEST, NO_AUTH}, where NO_AUTH is a distinct sentinel meaning 'may run anonymously / skip credential stamping' rather than a wire scheme (AUTH-1). spec · `docs/product-spec/11-authentication.md:7` · high · sha:efba58233dd1 -- An auth requirement MUST bind exactly one scheme to its own OAuth scopes and params, meaningful only for OAUTH2 and never inspected by resolution but preserved, MUST be immutable such that input collections mutated after construction do not affect the stored value, and MUST have value-based equality over scheme, scopes, and params. +- An auth requirement MUST bind exactly one scheme to its own OAuth scopes and params, meaningful only for OAUTH2 and never inspected by resolution but preserved, MUST be immutable such that input collections mutated after construction do not affect the stored value, and MUST have value-based equality over scheme, scopes, and params (AUTH-2). spec · `docs/product-spec/11-authentication.md:7` · high · sha:efba58233dd1 -- An auth descriptor MUST be a non-empty ordered list of requirements in preference order, MUST reject an empty list at construction, MUST be immutable, and MUST report 'allows anonymous' true if and only if any requirement's scheme is NO_AUTH. +- An auth descriptor MUST be a non-empty ordered list of requirements in preference order, MUST reject an empty list at construction, MUST be immutable, and MUST report 'allows anonymous' true if and only if any requirement's scheme is NO_AUTH (AUTH-3). spec · `docs/product-spec/11-authentication.md:7` · high · sha:efba58233dd1 -- Tier resolution MUST select the single most-specific descriptor present, in the strict order per-call, then operation, then client, and resolve only against that descriptor; a higher tier that is present but unsatisfiable MUST NOT fall through to a lower tier, since it fails because the caller asked for that override explicitly. +- Tier resolution MUST select the single most-specific descriptor present, in the strict order per-call, then operation, then client, and resolve only against that descriptor; a higher tier that is present but unsatisfiable MUST NOT fall through to a lower tier, since it fails because the caller asked for that override explicitly (AUTH-4). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- Within the selected auth descriptor, resolution MUST return the first requirement in declared order whose scheme is satisfiable, where satisfiable means NO_AUTH (always) or membership in the supplied set of available schemes, without inspecting any concrete credential. +- Within the selected auth descriptor, resolution MUST return the first requirement in declared order whose scheme is satisfiable, where satisfiable means NO_AUTH (always) or membership in the supplied set of available schemes, without inspecting any concrete credential (AUTH-5). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- Auth resolution MUST fail with an argument error when all tiers are absent, and with a distinct auth-resolution error carrying the required schemes in preference order and the available schemes when the selected descriptor lists no satisfiable scheme. +- Auth resolution MUST fail with an argument error when all tiers are absent, and with a distinct auth-resolution error carrying the required schemes in preference order and the available schemes when the selected descriptor lists no satisfiable scheme (AUTH-6). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- The auth resolver MUST be stateless, concurrency-safe, and a deterministic pure function of its inputs. +- The auth resolver MUST be stateless, concurrency-safe, and a deterministic pure function of its inputs (AUTH-7). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- Every credential type MUST redact its secret in any string/diagnostic representation without mutating or corrupting the real fields, MAY leave non-secret fields visible, and MUST preserve its variant-specific equality -- the bearer token has value-based equality over its real token and expiry (unaffected by the redacted string form), while the API-key and name-key credentials use reference identity, so two instances with identical fields are not equal. +- Every credential type MUST redact its secret in any string/diagnostic representation without mutating or corrupting the real fields, MAY leave non-secret fields visible, and MUST preserve its variant-specific equality -- the bearer token has value-based equality over its real token and expiry (unaffected by the redacted string form), while the API-key and name-key credentials use reference identity, so two instances with identical fields are not equal (AUTH-8). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- Credential construction MUST validate secret and identity fields as non-blank and reject blanks, for the bearer token, API key, and name-key name and key. +- Credential construction MUST validate secret and identity fields as non-blank and reject blanks, for the bearer token, API key, and name-key name and key (AUTH-9). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- Bearer-token expiry MUST be optional, with null meaning it never locally expires, and MUST be evaluated additively with a grace margin -- expired at reference time now with margin M if and only if expiry is non-null and now plus M is strictly after expiry. +- Bearer-token expiry MUST be optional, with null meaning it never locally expires, and MUST be evaluated additively with a grace margin -- expired at reference time now with margin M if and only if expiry is non-null and now plus M is strictly after expiry (AUTH-10). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- A token provider's fetch errors MUST propagate and MUST NOT be cached, so a subsequent request retries, and async callers MUST observe a provider error through the asynchronous channel (a failed future), never a synchronous throw. +- A token provider's fetch errors MUST propagate and MUST NOT be cached, so a subsequent request retries, and async callers MUST observe a provider error through the asynchronous channel (a failed future), never a synchronous throw (AUTH-11). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- The challenge parser MUST parse RFC 7235 WWW-Authenticate/Proxy-Authenticate values into an ordered list of challenges, honoring multiple comma-separated challenges, quoted-string values containing commas and equals signs, backslash escapes, scheme/param names normalized to lower case, values stored verbatim after unquoting, a bare scheme emitted with an empty parameter map, and a token68 value recorded under a synthetic key. +- The challenge parser MUST parse RFC 7235 WWW-Authenticate/Proxy-Authenticate values into an ordered list of challenges, honoring multiple comma-separated challenges, quoted-string values containing commas and equals signs, backslash escapes, scheme/param names normalized to lower case, values stored verbatim after unquoting, a bare scheme emitted with an empty parameter map, and a token68 value recorded under a synthetic key (AUTH-12). spec · `docs/product-spec/11-authentication.md:16` · high · sha:efba58233dd1 -- The challenge parser MUST be lenient and never throw -- blank input yields an empty list, a malformed challenge recovers to the next top-level comma, an unterminated quoted string terminates at end-of-input, and parameters parsed before a malformed tail are preserved. +- The challenge parser MUST be lenient and never throw -- blank input yields an empty list, a malformed challenge recovers to the next top-level comma, an unterminated quoted string terminates at end-of-input, and parameters parsed before a malformed tail are preserved (AUTH-13). spec · `docs/product-spec/11-authentication.md:16` · high · sha:efba58233dd1 -- Basic stamping MUST produce 'Basic ' plus base64 of UTF-8-encoded username:password, computed once, accept a Basic challenge case-insensitively, emit Authorization or Proxy-Authorization for a proxy challenge, and validate credentials as non-empty, permitting whitespace-only per RFC 7617, which is laxer than the non-blank rule used elsewhere. +- Basic stamping MUST produce 'Basic ' plus base64 of UTF-8-encoded username:password, computed once, accept a Basic challenge case-insensitively, emit Authorization or Proxy-Authorization for a proxy challenge, and validate credentials as non-empty, permitting whitespace-only per RFC 7617, which is laxer than the non-blank rule used elsewhere (AUTH-14). spec · `docs/product-spec/11-authentication.md:17` · high · sha:efba58233dd1 -- Digest stamping MUST support exactly {MD5, MD5-sess, SHA-256, SHA-256-sess} with qop auth or absent, declining auth-int-only challenges, unsupported algorithms, and mutual-auth verification. +- Digest stamping MUST support exactly {MD5, MD5-sess, SHA-256, SHA-256-sess} with qop auth or absent, declining auth-int-only challenges, unsupported algorithms, and mutual-auth verification (AUTH-15). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- A Digest challenge is considered satisfiable if and only if the scheme is Digest (case-insensitive), it carries realm and nonce, qop contains auth or is absent, and the algorithm is supported or absent, defaulting to MD5, preferring the algorithm earliest in the configured preference list regardless of wire order. +- A Digest challenge is considered satisfiable if and only if the scheme is Digest (case-insensitive), it carries realm and nonce, qop contains auth or is absent, and the algorithm is supported or absent, defaulting to MD5, preferring the algorithm earliest in the configured preference list regardless of wire order (AUTH-16). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- Digest stamping MUST compute HA1/HA2/response per RFC 7616/2069 using lower-case hex of the selected algorithm. +- Digest stamping MUST compute HA1/HA2/response per RFC 7616/2069 using lower-case hex of the selected algorithm (AUTH-17). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- The Digest nonce count MUST be tracked per server nonce starting at 00000001 and incrementing only on reuse, rendered as exactly 8 lower-case hex digits using the low 32 bits on overflow. +- The Digest nonce count MUST be tracked per server nonce starting at 00000001 and incrementing only on reuse, rendered as exactly 8 lower-case hex digits using the low 32 bits on overflow (AUTH-18). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- The Digest client nonce MUST be drawn from a cryptographically strong source with at least 128 bits of entropy. +- The Digest client nonce MUST be drawn from a cryptographically strong source with at least 128 bits of entropy (AUTH-20). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- Digest MUST use UTF-8 hash-input encoding when the challenge advertises charset=UTF-8 and ISO-8859-1 otherwise. +- Digest MUST use UTF-8 hash-input encoding when the challenge advertises charset=UTF-8 and ISO-8859-1 otherwise (AUTH-21). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- Digest stamping MUST quote/escape the appropriate fields, leave qop/nc/algorithm unquoted with the full algorithm spelling, use the request-target as the digest-uri, and emit cnonce/nc/qop only when qop is negotiated. +- Digest stamping MUST quote/escape the appropriate fields, leave qop/nc/algorithm unquoted with the full algorithm spelling, use the request-target as the digest-uri, and emit cnonce/nc/qop only when qop is negotiated (AUTH-22). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- The per-nonce counter store SHOULD be bounded, defaulting to 1024 entries, and drained under the cap; evicting a live nonce is harmless because its nc restarts at 1, which is spec-legal for a fresh nonce. +- The per-nonce counter store SHOULD be bounded, defaulting to 1024 entries, and drained under the cap; evicting a live nonce is harmless because its nc restarts at 1, which is spec-legal for a fresh nonce (AUTH-19). spec · `docs/product-spec/11-authentication.md:18` · high · sha:efba58233dd1 -- Composing auth handlers MUST delegate to the first handler in declaration order whose can-handle check passes and MUST defensively copy the handler list; callers order stronger schemes first. +- Composing auth handlers MUST delegate to the first handler in declaration order whose can-handle check passes and MUST defensively copy the handler list; callers order stronger schemes first (AUTH-23). spec · `docs/product-spec/11-authentication.md:19` · high · sha:efba58233dd1 -- Auth handlers MUST be safe for concurrent invocation, with per-handler mutable counters such as Digest nc using thread-safe primitives so concurrent reuse of one nonce still yields correct, non-duplicated counts. +- Auth handlers MUST be safe for concurrent invocation, with per-handler mutable counters such as Digest nc using thread-safe primitives so concurrent reuse of one nonce still yields correct, non-duplicated counts (AUTH-24). spec · `docs/product-spec/11-authentication.md:19` · high · sha:efba58233dd1 -- An auth handler MUST emit Authorization for WWW-Authenticate challenges and Proxy-Authorization for Proxy-Authenticate challenges, selected by an explicit proxy flag, and return no header when it cannot satisfy any offered challenge. +- An auth handler MUST emit Authorization for WWW-Authenticate challenges and Proxy-Authorization for Proxy-Authenticate challenges, selected by an explicit proxy flag, and return no header when it cannot satisfy any offered challenge (AUTH-25). spec · `docs/product-spec/11-authentication.md:19` · high · sha:efba58233dd1 -- Static key-credential stamping MUST write the key into the configured header, default Authorization, and when a prefix is configured, prepend it followed by a single space, with the stamping step stateless after construction. +- Static key-credential stamping MUST write the key into the configured header, default Authorization, and when a prefix is configured, prepend it followed by a single space, with the stamping step stateless after construction (AUTH-26). spec · `docs/product-spec/11-authentication.md:19` · high · sha:efba58233dd1 -- There MUST be exactly one auth step occupying the single AUTH pillar stage, running nested inside both the redirect loop and the retry loop, so auth executes per redirect hop and per retry attempt, with redirect wrapping retry wrapping auth. +- There MUST be exactly one auth step occupying the single AUTH pillar stage, running nested inside both the redirect loop and the retry loop, so auth executes per redirect hop and per retry attempt, with redirect wrapping retry wrapping auth (AUTH-27). spec · `docs/product-spec/11-authentication.md:23` · high · sha:efba58233dd1 -- On any path where a credential will be attached, the auth step MUST reject a non-HTTPS request URL, case-insensitive, before any token fetch or header stamping, failing with an error naming the concrete step and the offending scheme; credentials MUST NOT be stamped over plaintext. +- On any path where a credential will be attached, the auth step MUST reject a non-HTTPS request URL, case-insensitive, before any token fetch or header stamping, failing with an error naming the concrete step and the offending scheme; credentials MUST NOT be stamped over plaintext (AUTH-28). spec · `docs/product-spec/11-authentication.md:23` · high · sha:efba58233dd1 -- On a cross-origin redirect re-issue, differing in scheme, host, or effective port under the RFC 6454 tuple and marked by the redirect step, the auth step MUST NOT stamp the caller's credential, MUST strip the internal cross-origin marker so it never reaches the wire, and MUST skip the HTTPS guard so a deliberately-allowed downgrade hop is forwarded credential-free rather than hard-failing; a same-origin re-issue MUST be re-stamped normally and remains subject to the HTTPS guard. +- On a cross-origin redirect re-issue, differing in scheme, host, or effective port under the RFC 6454 tuple and marked by the redirect step, the auth step MUST NOT stamp the caller's credential, MUST strip the internal cross-origin marker so it never reaches the wire, and MUST skip the HTTPS guard so a deliberately-allowed downgrade hop is forwarded credential-free rather than hard-failing; a same-origin re-issue MUST be re-stamped normally and remains subject to the HTTPS guard (AUTH-29). spec · `docs/product-spec/11-authentication.md:24` · high · sha:efba58233dd1 -- The cross-origin suppression mechanism MUST only be able to suppress credential stamping, never force a credential to be sent. +- The cross-origin suppression mechanism MUST only be able to suppress credential stamping, never force a credential to be sent (AUTH-29). spec · `docs/product-spec/11-authentication.md:24` · high · sha:efba58233dd1 -- On a 401 carrying a WWW-Authenticate header, the auth step MUST consult its challenge hook; if the hook yields a non-null replacement request, the step MUST close the original 401 and drive the replacement through a fresh copy of the downstream chain exactly once, with no further challenge handling on the replacement; the default hook yields no replacement. +- On a 401 carrying a WWW-Authenticate header, the auth step MUST consult its challenge hook; if the hook yields a non-null replacement request, the step MUST close the original 401 and drive the replacement through a fresh copy of the downstream chain exactly once, with no further challenge handling on the replacement; the default hook yields no replacement (AUTH-30). spec · `docs/product-spec/11-authentication.md:25` · high · sha:efba58233dd1 -- A 401 without a WWW-Authenticate header MUST be returned unchanged without consulting the challenge hook. +- A 401 without a WWW-Authenticate header MUST be returned unchanged without consulting the challenge hook (AUTH-33). spec · `docs/product-spec/11-authentication.md:25` · high · sha:efba58233dd1 -- If the challenge hook throws, or its async future completes exceptionally, or the async hook throws synchronously, the auth step MUST close the open 401 response body before propagating. +- If the challenge hook throws, or its async future completes exceptionally, or the async hook throws synchronously, the auth step MUST close the open 401 response body before propagating (AUTH-32). spec · `docs/product-spec/11-authentication.md:25` · high · sha:efba58233dd1 -- The 401 re-challenge replay MUST be gated on request-body replayability -- if the replacement carries a non-replayable body, the step MUST skip the replay, surface the original 401 unchanged, and MUST NOT close that original response, since the caller owns it. +- The 401 re-challenge replay MUST be gated on request-body replayability -- if the replacement carries a non-replayable body, the step MUST skip the replay, surface the original 401 unchanged, and MUST NOT close that original response, since the caller owns it (AUTH-31). spec · `docs/product-spec/11-authentication.md:26` · high · sha:efba58233dd1 -- The bearer auth step MUST stamp Authorization: Bearer using a token cached until a configurable refresh margin before expiry, default 30 seconds, ensuring concurrent requests racing on a missing/expiring token result in at most one provider fetch (single-flight) with a non-blocking hot-path read of a valid cached token. +- The bearer auth step MUST stamp Authorization: Bearer using a token cached until a configurable refresh margin before expiry, default 30 seconds, ensuring concurrent requests racing on a missing/expiring token result in at most one provider fetch (single-flight) with a non-blocking hot-path read of a valid cached token (AUTH-34). spec · `docs/product-spec/11-authentication.md:27` · high · sha:efba58233dd1 -- The bearer auth step MUST reject a null token and a token already expired at fetch time, evaluated with no margin, and MUST NOT cache a thrown provider error. +- The bearer auth step MUST reject a null token and a token already expired at fetch time, evaluated with no margin, and MUST NOT cache a thrown provider error (AUTH-35). spec · `docs/product-spec/11-authentication.md:27` · high · sha:efba58233dd1 -- On a 401 advertising a Bearer challenge, the bearer auth step MUST evict only the exact cached token that produced the 401, matched by the stamped header value, and re-stamp a single retry with a freshly fetched token, preserving a token another request already refreshed, surfacing the 401 unchanged when the rejected request carried no Authorization header or the response advertises no Bearer challenge, and firing the eviction-driven retry regardless of HTTP method. +- On a 401 advertising a Bearer challenge, the bearer auth step MUST evict only the exact cached token that produced the 401, matched by the stamped header value, and re-stamp a single retry with a freshly fetched token, preserving a token another request already refreshed, surfacing the 401 unchanged when the rejected request carried no Authorization header or the response advertises no Bearer challenge, and firing the eviction-driven retry regardless of HTTP method (AUTH-36). spec · `docs/product-spec/11-authentication.md:27` · high · sha:efba58233dd1 -- The async bearer step MUST implement a three-zone expiry policy without blocking the dispatching thread -- fresh tokens are stamped with no refresh, expiring-but-valid tokens are stamped immediately while an off-thread background refresh is kicked off, and expired/missing tokens await a fresh single-flight fetch, coalescing concurrent expiring/missing requests onto one fetch, not caching a failed fetch, and treating a failed background refresh as non-fatal since a valid token was already stamped. +- The async bearer step MUST implement a three-zone expiry policy without blocking the dispatching thread -- fresh tokens are stamped with no refresh, expiring-but-valid tokens are stamped immediately while an off-thread background refresh is kicked off, and expired/missing tokens await a fresh single-flight fetch, coalescing concurrent expiring/missing requests onto one fetch, not caching a failed fetch, and treating a failed background refresh as non-fatal since a valid token was already stamped (AUTH-37). spec · `docs/product-spec/11-authentication.md:27` · high · sha:efba58233dd1 -- In the async auth path, the HTTPS-guard failure and any challenge hook error SHOULD be delivered through the asynchronous channel, a failed future, rather than synchronously thrown. +- In the async auth path, the HTTPS-guard failure and any challenge hook error SHOULD be delivered through the asynchronous channel, a failed future, rather than synchronously thrown (AUTH-38). spec · `docs/product-spec/11-authentication.md:27` · high · sha:efba58233dd1 - The cryptographically-strong client nonce (at least 128 bits of entropy) must use `crypto.getRandomValues()` rather than `Math.random()`, since it must come from a CSPRNG, never a non-cryptographic RNG. design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:68-70` · high · sha:b0e2bb42d809 @@ -97,9 +97,9 @@ ## Reference - Authentication has two largely independent halves -- a scheme-agnostic descriptor/resolver model that decides which auth alternative an operation requires, and a stamping/challenge half that puts credentials on the wire and reacts to server challenges. spec · `docs/product-spec/11-authentication.md:3` · high · sha:efba58233dd1 -- 401 eviction/refresh matching for bearer tokens is done on the stamped header string, not credential equality, so value equality is not required for the key credentials. +- 401 eviction/refresh matching for bearer tokens is done on the stamped header string, not credential equality, so value equality is not required for the key credentials (AUTH-36). spec · `docs/product-spec/11-authentication.md:8` · high · sha:efba58233dd1 -- The reference implementation enforces the 401 replayability gate on the synchronous auth step only; the async auth step does not currently apply a replayability gate and closes the original 401 before re-driving unconditionally, and a faithful port SHOULD apply the same gate on both paths. +- The reference implementation enforces the 401 replayability gate on the synchronous auth step only; the async auth step does not currently apply a replayability gate and closes the original 401 before re-driving unconditionally, and a faithful port SHOULD apply the same gate on both paths (AUTH-31). spec · `docs/product-spec/11-authentication.md:26` · high · sha:efba58233dd1 - An auth challenge is a parsed RFC 7235 WWW-Authenticate/Proxy-Authenticate directive, a scheme plus a parameter map, that a server returns on a 401/407 to indicate how a client may authenticate. spec · `docs/product-spec/appendix-a-glossary.md:7` · high · sha:f0b3d2058626 diff --git a/docs/knowledge/cancellation-and-timeouts.md b/docs/knowledge/harvested/cancellation-and-timeouts.md similarity index 100% rename from docs/knowledge/cancellation-and-timeouts.md rename to docs/knowledge/harvested/cancellation-and-timeouts.md diff --git a/docs/knowledge/concurrency-and-async.md b/docs/knowledge/harvested/concurrency-and-async.md similarity index 99% rename from docs/knowledge/concurrency-and-async.md rename to docs/knowledge/harvested/concurrency-and-async.md index 9c42c05..6b6de95 100644 --- a/docs/knowledge/concurrency-and-async.md +++ b/docs/knowledge/harvested/concurrency-and-async.md @@ -69,7 +69,7 @@ spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:33-33` · high · sha:f1bf00174456 - An adapter that owns an executor should shut it down gracefully on close — stopping new work and waiting for in-flight tasks rather than interrupting them — escalating to forceful shutdown only if the closing thread is itself interrupted, with callers needing eager abort using the interrupt/structured-cancellation path. spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:34-34` · high · sha:f1bf00174456 -- The async transport SPI should provide a no-op default close so lightweight/functional implementations need not implement lifecycle management, while any implementation that owns resources overrides it to follow the idempotent/ownership-aware/interrupt-safe close contract; behavior of executeAsync after close is undefined. +- The async transport SPI should provide a no-op default close so lightweight/functional implementations need not implement lifecycle management, while any implementation that owns resources overrides it to follow the idempotent/ownership-aware/interrupt-safe close contract; behavior of executeAsync after close is undefined (SEAM-15). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:35-35` · high · sha:f1bf00174456 - Components documented as shared/reusable across concurrent requests (pipeline steps, auth handlers, redactors, factories) must be safe for concurrent invocation, with per-call mutable state kept on the call's local state and any shared mutable state synchronized. spec · `docs/product-spec/19-cross-cutting-invariants-and-policies.md:28` · high · sha:d6123be82c9e diff --git a/docs/knowledge/configuration.md b/docs/knowledge/harvested/configuration.md similarity index 100% rename from docs/knowledge/configuration.md rename to docs/knowledge/harvested/configuration.md diff --git a/docs/knowledge/cross-cutting-invariants.md b/docs/knowledge/harvested/cross-cutting-invariants.md similarity index 100% rename from docs/knowledge/cross-cutting-invariants.md rename to docs/knowledge/harvested/cross-cutting-invariants.md diff --git a/docs/knowledge/data-modeling.md b/docs/knowledge/harvested/data-modeling.md similarity index 100% rename from docs/knowledge/data-modeling.md rename to docs/knowledge/harvested/data-modeling.md diff --git a/docs/knowledge/documentation.md b/docs/knowledge/harvested/documentation.md similarity index 100% rename from docs/knowledge/documentation.md rename to docs/knowledge/harvested/documentation.md diff --git a/docs/knowledge/error-handling.md b/docs/knowledge/harvested/error-handling.md similarity index 100% rename from docs/knowledge/error-handling.md rename to docs/knowledge/harvested/error-handling.md diff --git a/docs/knowledge/execution-context.md b/docs/knowledge/harvested/execution-context.md similarity index 88% rename from docs/knowledge/execution-context.md rename to docs/knowledge/harvested/execution-context.md index 9519865..cc4ab7e 100644 --- a/docs/knowledge/execution-context.md +++ b/docs/knowledge/harvested/execution-context.md @@ -1,77 +1,77 @@ # execution-context ## Rules -- The execution context model MUST provide three context flavors forming a one-way promotion chain mirroring the call lifecycle -- a dispatch stage before any request, a request stage with an outgoing request assembled, and an exchange stage after a response arrives -- with promotion advancing dispatch to request to exchange only and the exchange stage terminal. +- The execution context model MUST provide three context flavors forming a one-way promotion chain mirroring the call lifecycle -- a dispatch stage before any request, a request stage with an outgoing request assembled, and an exchange stage after a response arrives -- with promotion advancing dispatch to request to exchange only and the exchange stage terminal (CTX-1). spec · `docs/product-spec/07-execution-context-model.md:7` · high · sha:5a9eacfb1c53 -- Each promotion in the execution context chain MUST be additive and non-mutating, producing a new instance without modifying the source, carrying forward the same instrumentation bundle and call key, and adding exactly one new artifact (the request when promoting dispatch to request, the response when promoting request to exchange). +- Each promotion in the execution context chain MUST be additive and non-mutating, producing a new instance without modifying the source, carrying forward the same instrumentation bundle and call key, and adding exactly one new artifact (the request when promoting dispatch to request, the response when promoting request to exchange) (CTX-2). spec · `docs/product-spec/07-execution-context-model.md:8` · high · sha:5a9eacfb1c53 -- The entire context promotion chain MUST share one call key -- a promotion carries the source's call key forward verbatim so all three flavors register under the identical store slot and successive promotions overwrite one entry. +- The entire context promotion chain MUST share one call key -- a promotion carries the source's call key forward verbatim so all three flavors register under the identical store slot and successive promotions overwrite one entry (CTX-3). spec · `docs/product-spec/07-execution-context-model.md:9` · high · sha:5a9eacfb1c53 -- A directly-constructed (off-chain) context without an explicit key MUST receive a fresh call-unique key using the same uniqueness guarantee as promoted contexts, and default construction MUST mint globally distinct keys across the whole process and all three flavors. +- A directly-constructed (off-chain) context without an explicit key MUST receive a fresh call-unique key using the same uniqueness guarantee as promoted contexts, and default construction MUST mint globally distinct keys across the whole process and all three flavors (CTX-5 / CTX-6). spec · `docs/product-spec/07-execution-context-model.md:14` · high · sha:5a9eacfb1c53 -- Because the call key participates in value-equality, two default-constructed contexts with otherwise identical fields are not equal; callers needing value-equality between contexts MUST be able to pin an explicit shared key. +- Because the call key participates in value-equality, two default-constructed contexts with otherwise identical fields are not equal; callers needing value-equality between contexts MUST be able to pin an explicit shared key (CTX-5). spec · `docs/product-spec/07-execution-context-model.md:14` · high · sha:5a9eacfb1c53 -- Registration MUST happen at promotion time, not at head-context construction -- constructing the initial dispatch context MUST NOT auto-register it, so a dispatch context never promoted leaves no store entry and its close is a harmless no-op. +- Registration MUST happen at promotion time, not at head-context construction -- constructing the initial dispatch context MUST NOT auto-register it, so a dispatch context never promoted leaves no store entry and its close is a harmless no-op (CTX-17). spec · `docs/product-spec/07-execution-context-model.md:15` · high · sha:5a9eacfb1c53 -- The context store MUST support an unconditional overwrite operation (install-or-replace, never throwing) used by promotion, and a reject-on-duplicate insert operation (install only if absent) that admits exactly one winner under concurrency and fails all others with an error naming the key. +- The context store MUST support an unconditional overwrite operation (install-or-replace, never throwing) used by promotion, and a reject-on-duplicate insert operation (install only if absent) that admits exactly one winner under concurrency and fails all others with an error naming the key (CTX-8). spec · `docs/product-spec/07-execution-context-model.md:20` · high · sha:5a9eacfb1c53 -- Closing a context MUST evict the store entry conditionally on reference identity, removing the slot only when the current occupant is the closing context (never by value equality), and removing a non-existent or already-replaced slot MUST be a well-defined no-op. +- Closing a context MUST evict the store entry conditionally on reference identity, removing the slot only when the current occupant is the closing context (never by value equality), and removing a non-existent or already-replaced slot MUST be a well-defined no-op (CTX-9). spec · `docs/product-spec/07-execution-context-model.md:21` · high · sha:5a9eacfb1c53 -- Only the context currently occupying the shared store slot (the furthest-reached link in the promotion chain) evicts on close; closing an intermediate link that was already promoted is a no-op. +- Only the context currently occupying the shared store slot (the furthest-reached link in the promotion chain) evicts on close; closing an intermediate link that was already promoted is a no-op (CTX-10). spec · `docs/product-spec/07-execution-context-model.md:21` · high · sha:5a9eacfb1c53 -- Looking up an unknown key MUST return an explicit absent result rather than throw, and removing an unknown or already-removed key MUST be a no-op, so double-close and cleanup-path closes are well-defined. +- Looking up an unknown key MUST return an explicit absent result rather than throw, and removing an unknown or already-removed key MUST be a no-op, so double-close and cleanup-path closes are well-defined (CTX-18). spec · `docs/product-spec/07-execution-context-model.md:22` · high · sha:5a9eacfb1c53 -- The cap-draining strategy SHOULD be a post-insert drain loop (drain until at or under the cap) rather than a single check-then-evict, so concurrent insert bursts converge to the bound instead of overshooting. +- The cap-draining strategy SHOULD be a post-insert drain loop (drain until at or under the cap) rather than a single check-then-evict, so concurrent insert bursts converge to the bound instead of overshooting (CTX-12). spec · `docs/product-spec/07-execution-context-model.md:27` · high · sha:5a9eacfb1c53 -- Each context MUST carry a correlation/instrumentation bundle exposing at minimum a trace id, a span id, trace flags, trace state, a trace-id encoding flavor, validity and remoteness flags, an active span, and a per-operation tracer factory, W3C Trace Context compatible for cross-service propagation. +- Each context MUST carry a correlation/instrumentation bundle exposing at minimum a trace id, a span id, trace flags, trace state, a trace-id encoding flavor, validity and remoteness flags, an active span, and a per-operation tracer factory, W3C Trace Context compatible for cross-service propagation (CTX-14). spec · `docs/product-spec/07-execution-context-model.md:31` · high · sha:5a9eacfb1c53 -- A disabled-tracing/no-op instrumentation bundle MUST be available as the default, with reserved invalid sentinels (all-zero trace id, all-zero span id, zero flags, empty state), isValid false, isRemote false, a no-op span, and a no-op tracer factory. +- A disabled-tracing/no-op instrumentation bundle MUST be available as the default, with reserved invalid sentinels (all-zero trace id, all-zero span id, zero flags, empty state), isValid false, isRemote false, a no-op span, and a no-op tracer factory (CTX-15). spec · `docs/product-spec/07-execution-context-model.md:31` · high · sha:5a9eacfb1c53 -- A context SHOULD carry an optional operation name (a schema-defined operation id, or absent), MUST carry it forward unchanged across every promotion, and MUST keep it advisory only, exposed to the tracing seam without influencing the request, dispatch decision, or store key. +- A context SHOULD carry an optional operation name (a schema-defined operation id, or absent), MUST carry it forward unchanged across every promotion, and MUST keep it advisory only, exposed to the tracing seam without influencing the request, dispatch decision, or store key (CTX-16). spec · `docs/product-spec/07-execution-context-model.md:32` · high · sha:5a9eacfb1c53 -- The per-operation tracer factory SHOULD default to a no-op emitting nothing so untraced call sites pay zero tracing cost, and its factory method MUST be safe to invoke concurrently. +- The per-operation tracer factory SHOULD default to a no-op emitting nothing so untraced call sites pay zero tracing cost, and its factory method MUST be safe to invoke concurrently (CTX-20). spec · `docs/product-spec/07-execution-context-model.md:32` · high · sha:5a9eacfb1c53 -- When folding thread-local diagnostic context into a log event, only allow-listed keys are folded; the default allow-list is exactly {trace.id, span.id}, a null (absent) allow-list folds every present key, and keys with null values are skipped, to prevent arbitrary application context from leaking into SDK-owned events. +- When folding thread-local diagnostic context into a log event, only allow-listed keys are folded; the default allow-list is exactly {trace.id, span.id}, a null (absent) allow-list folds every present key, and keys with null values are skipped, to prevent arbitrary application context from leaking into SDK-owned events (OBS-10). spec · `docs/product-spec/15-instrumentation-and-observability.md:20-20` · high · sha:1b678eca176d -- Adapters that move work/callbacks onto another thread should propagate the caller's diagnostic logging context across the hop (capture on the boundary thread, reinstate on the executing/callback thread) so post-hop log events retain correlation; this is an observability guarantee, not a functional one, so an adapter that omits it still executes exchanges correctly. +- Adapters that move work/callbacks onto another thread should propagate the caller's diagnostic logging context across the hop (capture on the boundary thread, reinstate on the executing/callback thread) so post-hop log events retain correlation; this is an observability guarantee, not a functional one, so an adapter that omits it still executes exchanges correctly (ASYNC-8). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:20-20` · high · sha:f1bf00174456 -- When an adapter reinstates a captured context, it must first save the executing thread's prior context, install the captured context only for the work's duration, and restore the prior context afterward — including when the work throws — so a reused/pooled thread's own context is never clobbered. +- When an adapter reinstates a captured context, it must first save the executing thread's prior context, install the captured context only for the work's duration, and restore the prior context afterward — including when the work throws — so a reused/pooled thread's own context is never clobbered (ASYNC-9). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:21-21` · high · sha:f1bf00174456 -- When an adapter propagates logging context, capture must occur at the point that identifies the logical caller — per-subscription for cold/reusable stream or promise objects, per-task-submission for executor decorators — not at object-construction time, so a reused async object picks up the live context of each use. +- When an adapter propagates logging context, capture must occur at the point that identifies the logical caller — per-subscription for cold/reusable stream or promise objects, per-task-submission for executor decorators — not at object-construction time, so a reused async object picks up the live context of each use (ASYNC-10). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:22-22` · high · sha:f1bf00174456 -- When an adapter propagates logging context, capture and restore must be safe when no logging-context backend is installed: an absent context captures as empty, and reinstating an empty context clears the target thread's context rather than raising. +- When an adapter propagates logging context, capture and restore must be safe when no logging-context backend is installed: an absent context captures as empty, and reinstating an empty context clears the target thread's context rather than raising (ASYNC-11). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:23-23` · high · sha:f1bf00174456 -- On runtimes where a newly created worker does not inherit the spawning thread's logging context (lightweight threads or plain thread-local contexts), an adapter that propagates logging context must explicitly transfer it at the thread-creation boundary, distinct from any carrier-hop guarantee the runtime provides. +- On runtimes where a newly created worker does not inherit the spawning thread's logging context (lightweight threads or plain thread-local contexts), an adapter that propagates logging context must explicitly transfer it at the thread-creation boundary, distinct from any carrier-hop guarantee the runtime provides (ASYNC-12). spec · `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:24-24` · high · sha:f1bf00174456 ## Constraints -- Each call's store key MUST be unique per call and MUST NOT be derived from the trace identifier, or the trace+span pair, alone, so two concurrent calls sharing a trace id or even a span id receive distinct keys and never evict each other. +- Each call's store key MUST be unique per call and MUST NOT be derived from the trace identifier, or the trace+span pair, alone, so two concurrent calls sharing a trace id or even a span id receive distinct keys and never evict each other (CTX-4). spec · `docs/product-spec/07-execution-context-model.md:13` · high · sha:5a9eacfb1c53 -- Contexts MUST be immutable and shareable without external synchronization, and the store MUST be thread-safe such that contexts with distinct call keys can be registered, overwritten, and removed concurrently without external locking. +- Contexts MUST be immutable and shareable without external synchronization, and the store MUST be thread-safe such that contexts with distinct call keys can be registered, overwritten, and removed concurrently without external locking (CTX-7). spec · `docs/product-spec/07-execution-context-model.md:19` · high · sha:5a9eacfb1c53 -- The context store MUST be bounded, enforcing a maximum number of tracked entries and draining back to at or below that cap after each insert, as a backstop so a caller who fails to close a context on an exception path leaks at most the cap's worth of entries. +- The context store MUST be bounded, enforcing a maximum number of tracked entries and draining back to at or below that cap after each insert, as a backstop so a caller who fails to close a context on an exception path leaks at most the cap's worth of entries (CTX-11). spec · `docs/product-spec/07-execution-context-model.md:26` · high · sha:5a9eacfb1c53 -- The context store MUST keep the pinned request/response graph reachable while a context remains stored; reimplementations MUST NOT hold contexts by weak or soft references, and MUST treat the bounded cap, not garbage collection, as the leak backstop. +- The context store MUST keep the pinned request/response graph reachable while a context remains stored; reimplementations MUST NOT hold contexts by weak or soft references, and MUST treat the bounded cap, not garbage collection, as the leak backstop (CTX-19). spec · `docs/product-spec/07-execution-context-model.md:26` · high · sha:5a9eacfb1c53 -- Eviction victim selection in the context store is arbitrary -- the store provides no ordering and no guarantee that any particular entry, including the just-inserted one, survives an insert that trips the cap, and a port MUST NOT rely on any specific entry surviving. +- Eviction victim selection in the context store is arbitrary -- the store provides no ordering and no guarantee that any particular entry, including the just-inserted one, survives an insert that trips the cap, and a port MUST NOT rely on any specific entry surviving (CTX-13). spec · `docs/product-spec/07-execution-context-model.md:27` · high · sha:5a9eacfb1c53 -- Because the default no-op instrumentation bundle shares constant identifiers across every untraced call, call-key derivation MUST remain call-unique even when every bundle field is identical. +- Because the default no-op instrumentation bundle shares constant identifiers across every untraced call, call-key derivation MUST remain call-unique even when every bundle field is identical (CTX-4). spec · `docs/product-spec/07-execution-context-model.md:31` · high · sha:5a9eacfb1c53 ## Conclusions -- Trace ids cannot be used alone for call-key derivation because a disabled-tracing context shares one constant trace id across every untraced call, an inbound distributed trace shares one trace id across many spans, and a tracer may reuse a span id. +- Trace ids cannot be used alone for call-key derivation because a disabled-tracing context shares one constant trace id across every untraced call, an inbound distributed trace shares one trace id across many spans, and a tracer may reuse a span id (CTX-4). spec · `docs/product-spec/07-execution-context-model.md:13` · high · sha:5a9eacfb1c53 -- Value-equality removal is rejected for context store eviction because contexts are value-equal, so a value-equality remove could let a stale context evict a structurally-identical live sibling. +- Value-equality removal is rejected for context store eviction because contexts are value-equal, so a value-equality remove could let a stale context evict a structurally-identical live sibling (CTX-9). spec · `docs/product-spec/07-execution-context-model.md:21` · high · sha:5a9eacfb1c53 -- The context store is bounded rather than left unbounded because a registered context strongly pins the full request-and-response graph, including a possibly-unread body holding a connection. +- The context store is bounded rather than left unbounded because a registered context strongly pins the full request-and-response graph, including a possibly-unread body holding a connection (CTX-11 / CTX-19). spec · `docs/product-spec/07-execution-context-model.md:26` · high · sha:5a9eacfb1c53 ## Reference - A single in-flight call's correlation state is modeled as a one-way promotion chain of three immutable context flavors, each carrying a shared instrumentation bundle and a single call-unique key, registered in a bounded process-wide store keyed by that call key. spec · `docs/product-spec/07-execution-context-model.md:3` · high · sha:5a9eacfb1c53 -- The operation name is introduced at the request stage as an argument to the dispatch-to-request promotion. +- The operation name is introduced at the request stage as an argument to the dispatch-to-request promotion (CTX-2). spec · `docs/product-spec/07-execution-context-model.md:8` · high · sha:5a9eacfb1c53 -- The reference implementation's default call key appends a process-wide monotonic counter to a traceId:spanId rendering. +- The reference implementation's default call key appends a process-wide monotonic counter to a traceId:spanId rendering (CTX-4). spec · `docs/product-spec/07-execution-context-model.md:13` · high · sha:5a9eacfb1c53 ## Conflicts diff --git a/docs/knowledge/function-design.md b/docs/knowledge/harvested/function-design.md similarity index 100% rename from docs/knowledge/function-design.md rename to docs/knowledge/harvested/function-design.md diff --git a/docs/knowledge/http-domain-model.md b/docs/knowledge/harvested/http-domain-model.md similarity index 100% rename from docs/knowledge/http-domain-model.md rename to docs/knowledge/harvested/http-domain-model.md diff --git a/docs/knowledge/io-and-byte-streams.md b/docs/knowledge/harvested/io-and-byte-streams.md similarity index 100% rename from docs/knowledge/io-and-byte-streams.md rename to docs/knowledge/harvested/io-and-byte-streams.md diff --git a/docs/knowledge/message-bodies.md b/docs/knowledge/harvested/message-bodies.md similarity index 99% rename from docs/knowledge/message-bodies.md rename to docs/knowledge/harvested/message-bodies.md index e2b0fa3..d88c88d 100644 --- a/docs/knowledge/message-bodies.md +++ b/docs/knowledge/harvested/message-bodies.md @@ -21,7 +21,7 @@ spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:8-8` · high · sha:c2bf15dc8a06 - A materialize-once operation MUST return the same body unchanged when already replayable, and otherwise drain the body's write output exactly once into an in-memory buffer and return a replayable buffer-backed body, after which the original MUST be treated as consumed (BODY-3 / HTTP-37). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:9-9` · high · sha:c2bf15dc8a06 -- A single-use body MUST fail loudly on a second write, never silently emitting zero bytes, and the consume-once guard MUST be race-safe so that under concurrent writes at most one proceeds and the losers observe a clear error (BODY-3 / HTTP-37). +- A single-use body MUST fail loudly on a second write, never silently emitting zero bytes, and the consume-once guard MUST be race-safe so that under concurrent writes at most one proceeds and the losers observe a clear error (BODY-3 / HTTP-37 / BODY-6 / BODY-7). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:9-9` · high · sha:c2bf15dc8a06 - A single-use body that owns a closeable source MUST release that source as part of its single write, so skipping materialization does not leak it (BODY-8). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:10-10` · high · sha:c2bf15dc8a06 @@ -89,7 +89,7 @@ design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:38-42` · high · sha:b0e2bb42d809 ## Reference -- The reference implementation of the consume-once guard for a single-use body is an atomic compare-and-set (BODY-3 / HTTP-37). +- The reference implementation of the consume-once guard for a single-use body is an atomic compare-and-set (BODY-3 / HTTP-37 / BODY-7). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:9-9` · high · sha:c2bf15dc8a06 - In the reference implementation, the buffered-source-backed single-use body drains and closes its source during write, while the raw byte-stream-backed bodies do not close their stream during write — the rewindable variant keeps it open to replay and the one-shot variant leaves the caller-supplied stream unclosed per its documented ownership (BODY-8). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:10-10` · high · sha:c2bf15dc8a06 diff --git a/docs/knowledge/module-organization.md b/docs/knowledge/harvested/module-organization.md similarity index 100% rename from docs/knowledge/module-organization.md rename to docs/knowledge/harvested/module-organization.md diff --git a/docs/knowledge/naming-conventions.md b/docs/knowledge/harvested/naming-conventions.md similarity index 100% rename from docs/knowledge/naming-conventions.md rename to docs/knowledge/harvested/naming-conventions.md diff --git a/docs/knowledge/observability.md b/docs/knowledge/harvested/observability.md similarity index 100% rename from docs/knowledge/observability.md rename to docs/knowledge/harvested/observability.md diff --git a/docs/knowledge/package-and-dependency-layout.md b/docs/knowledge/harvested/package-and-dependency-layout.md similarity index 100% rename from docs/knowledge/package-and-dependency-layout.md rename to docs/knowledge/harvested/package-and-dependency-layout.md diff --git a/docs/knowledge/pagination.md b/docs/knowledge/harvested/pagination.md similarity index 92% rename from docs/knowledge/pagination.md rename to docs/knowledge/harvested/pagination.md index 880bd9d..05c07a3 100644 --- a/docs/knowledge/pagination.md +++ b/docs/knowledge/harvested/pagination.md @@ -17,9 +17,9 @@ spec · `docs/product-spec/12-pagination.md:19-19` · high · sha:ba759edd34ec - A Page MUST be a closeable resource owning exactly one underlying response, whoever pulls a page owns closing it, closing the page MUST release that response's body/connection, and a component that hands a caller a live page MUST NOT itself close the response. spec · `docs/product-spec/12-pagination.md:20-20` · high · sha:ba759edd34ec -- A pagination strategy's parse output MUST carry items plus a next-request value where a null/absent next-request is the single exclusive end-of-stream signal, parse MUST always return a well-formed non-null result, termination MUST never be signaled by throwing or a side channel, and an empty items list with a non-null next-request is a valid non-terminal page. +- A pagination strategy's parse output MUST carry items plus a next-request value where a null/absent next-request is the single exclusive end-of-stream signal, parse MUST always return a well-formed non-null result, termination MUST never be signaled by throwing or a side channel, and an empty items list with a non-null next-request is a valid non-terminal page (PAGE-4). spec · `docs/product-spec/12-pagination.md:26-26` · high · sha:ba759edd34ec -- A pagination strategy MUST read everything it needs from the response synchronously inside parse since the body is single-use, MUST NOT retain the response or its body beyond the call, MUST NOT close or mutate the response, and strategies MUST be immutable and safe to share concurrently. +- A pagination strategy MUST read everything it needs from the response synchronously inside parse since the body is single-use, MUST NOT retain the response or its body beyond the call, MUST NOT close or mutate the response, and strategies MUST be immutable and safe to share concurrently (PAGE-5). spec · `docs/product-spec/12-pagination.md:27-27` · high · sha:ba759edd34ec - The pagination engine MUST accept a page cap bounding a server that never advances its cursor, the cap counts exchanges/pages not items, the engine MUST stop fetching once the cap is reached even if the strategy reports a next-request, and the cap MUST be validated as strictly positive at construction rather than lazily. spec · `docs/product-spec/12-pagination.md:31-31` · high · sha:ba759edd34ec @@ -81,7 +81,7 @@ design · `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:7-10` · high · sha:d546f9973c4e - Close-on-abandon for pagination relies on JavaScript's iterator protocol automatically calling `.return()` on an async iterator when a `for await...of` loop exits early via break, return, or exception, resuming execution at the enclosing `finally` block, unlike Kotlin's `Iterator`/`Sequence` protocol which has no built-in early-termination cleanup hook and requires a bespoke `CloseablePages` wrapper. design · `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:12-19` · high · sha:d546f9973c4e -- The page-level view's two-outstanding-pages buffering requirement is implemented as a one-slot look-ahead buffer held in the generator's own closure, released via the same `finally` mechanism. +- The page-level view holds the currently delivered page in the generator's `held` binding, releasing it upon advancing before dispatching the next request and releasing the last held page at exhaustion or early termination via the enclosing `finally` block. design · `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:32-34` · high · sha:d546f9973c4e - The port rejects `URLSearchParams` for verbatim query-parameter splicing because it re-serializes the entire query string through its own canonical encoding on every mutation, reordering and re-encoding untouched parameters and encoding space as `+` rather than the RFC 3986 `%20` the port's query model otherwise standardizes on. design · `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:36-41` · high · sha:d546f9973c4e @@ -127,13 +127,11 @@ spec · `docs/product-spec/appendix-b-conformance-test-checklist.md:15` · high · sha:0451cc7f3bb4 - The pagination conformance suite verifies that per-call options reach every page exchange (PAGE-36). spec · `docs/product-spec/appendix-b-conformance-test-checklist.md:16` · high · sha:0451cc7f3bb4 -- The port's item-level generator wraps `yield* page.items` in a `finally` block that awaits `page.close()`, so an early `break` in the consumer's loop automatically triggers page closure with no wrapper type or documented convention required. **Erratum (Phase 6c, 2026-07-28): correct about the `.return()`-on-abandon mechanism, wrong about close ordering — see Conflicts below.** +- The port's item-level generator wraps `yield* page.items` in a `finally` block that awaits `page.close()`, so an early `break` in the consumer's loop automatically triggers page closure with no wrapper type or documented convention required. design · `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:19-31` · high · sha:d546f9973c4e ## Conflicts -- **Item-view close ordering.** The Rules entry above (`PAGE-11`, `docs/product-spec/12-pagination.md:38`) requires the item-level view to eager-close each page **before** yielding any of that page's items, after copying them. The Reference entry above (`sdk-design-nodejs/07` §7.1) shows the opposite ordering — `yield*` inside a `try`, `close()` in the `finally` — which holds the response open for the entire time a consumer walks that page's items. **`PAGE-11` governs**, per the standing tie-breaker that the normative spec wins over an illustrative snippet; the cost is zero because `PAGE-2` guarantees materialized items survive close. Resolved in Phase 6c, which implements copy-items → close → yield and writes the erratum into `sdk-design-nodejs/07` §7.1. - - The reason this needed recording rather than silently correcting: **the conformance test is weaker than the requirement.** Appendix B's `PAGE-11` check ("take one item from a multi-item first page and stop; assert the first page's response was closed") *passes* under the snippet's ordering, because an early `break` drives `.return()` and therefore the `finally`. Following the design doc would have shipped a MUST violation the checklist could not catch. Phase 6c's `lifecycle.test.ts` adds the assertion appendix B does not make — that the close is observed *before* the first item is yielded. - review · `docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md` · high · sha:manual-6c-erratum +- **spec vs design: item-view close ordering** — `PAGE-11` requires the item-level view to eager-close each page **before** yielding any of that page's items, after copying them. The §7.1 snippet shows the opposite ordering — `yield*` inside a `try`, `close()` in the `finally` — which holds the response open for the entire time a consumer walks that page's items. + spec `docs/product-spec/12-pagination.md:38` · design `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md:19-31` · resolved 2026-07-28 ## Superseded diff --git a/docs/knowledge/performance.md b/docs/knowledge/harvested/performance.md similarity index 100% rename from docs/knowledge/performance.md rename to docs/knowledge/harvested/performance.md diff --git a/docs/knowledge/pipeline.md b/docs/knowledge/harvested/pipeline.md similarity index 89% rename from docs/knowledge/pipeline.md rename to docs/knowledge/harvested/pipeline.md index d4f58d6..a92c441 100644 --- a/docs/knowledge/pipeline.md +++ b/docs/knowledge/harvested/pipeline.md @@ -1,115 +1,115 @@ # pipeline ## Rules -- Steps MUST execute in a single fixed total order derived from stage assignment -- a step in a lower-ordered stage runs before (wraps) a step in a higher-ordered stage on the inbound path and observes the response later on the outbound path, and this cross-stage order is deterministic and independent of insertion order. +- Steps MUST execute in a single fixed total order derived from stage assignment -- a step in a lower-ordered stage runs before (wraps) a step in a higher-ordered stage on the inbound path and observes the response later on the outbound path, and this cross-stage order is deterministic and independent of insertion order (PIPE-1). spec · `docs/product-spec/08-execution-pipelines.md:9` · high · sha:33e9443472ce -- The runtime MUST preserve the pillar precedence chain REDIRECT to RETRY to AUTH to LOGGING to SERDE (outer to inner), plus an outermost pre-redirect slot outside both loops and a terminal SEND hop innermost, and a step's placement relative to these boundaries determines whether it sees per-hop/per-attempt responses or only the single terminal response. +- The runtime MUST preserve the pillar precedence chain REDIRECT to RETRY to AUTH to LOGGING to SERDE (outer to inner), plus an outermost pre-redirect slot outside both loops and a terminal SEND hop innermost, and a step's placement relative to these boundaries determines whether it sees per-hop/per-attempt responses or only the single terminal response (PIPE-2). spec · `docs/product-spec/08-execution-pipelines.md:10` · high · sha:33e9443472ce -- The stage list SHOULD interleave user-extensible slots around each pillar (a pre and post slot) and SHOULD use sparse numeric order keys so new stages can be inserted without renumbering. +- The stage list SHOULD interleave user-extensible slots around each pillar (a pre and post slot) and SHOULD use sparse numeric order keys so new stages can be inserted without renumbering (PIPE-3). spec · `docs/product-spec/08-execution-pipelines.md:11` · high · sha:33e9443472ce -- Installing a distinct second step onto an occupied pillar stage, via any add or a bulk reload, MUST fail fast naming both step types and pointing at the replace path, rather than silently overwriting. +- Installing a distinct second step onto an occupied pillar stage, via any add or a bulk reload, MUST fail fast naming both step types and pointing at the replace path, rather than silently overwriting (PIPE-5). spec · `docs/product-spec/08-execution-pipelines.md:12` · high · sha:33e9443472ce -- Re-installing the same step onto its pillar stage MUST be idempotent, distinguished by reference identity, not value equality. +- Re-installing the same step onto its pillar stage MUST be idempotent, distinguished by reference identity, not value equality (PIPE-6). spec · `docs/product-spec/08-execution-pipelines.md:12` · high · sha:33e9443472ce -- An empty pipeline MUST dispatch directly to the terminal transport, threading the caller's per-call options, and SHOULD do so without allocating per-call cursor state. +- An empty pipeline MUST dispatch directly to the terminal transport, threading the caller's per-call options, and SHOULD do so without allocating per-call cursor state (PIPE-9). spec · `docs/product-spec/08-execution-pipelines.md:16` · high · sha:33e9443472ce -- The built runtime MUST be immutable after construction, and each send MUST allocate its own per-call cursor so concurrent calls share no mutable pipeline state. +- The built runtime MUST be immutable after construction, and each send MUST allocate its own per-call cursor so concurrent calls share no mutable pipeline state (PIPE-10). spec · `docs/product-spec/08-execution-pipelines.md:16` · high · sha:33e9443472ce -- Steps MUST be safe for concurrent invocation, with per-request mutable state living in the per-call cursor, never on the step. +- Steps MUST be safe for concurrent invocation, with per-request mutable state living in the per-call cursor, never on the step (PIPE-11). spec · `docs/product-spec/08-execution-pipelines.md:16` · high · sha:33e9443472ce -- Each step MUST be bidirectional -- it receives the inbound request, may invoke the rest of the chain, may inspect or substitute the outbound response, and may short-circuit by returning a synthetic response without invoking the chain. +- Each step MUST be bidirectional -- it receives the inbound request, may invoke the rest of the chain, may inspect or substitute the outbound response, and may short-circuit by returning a synthetic response without invoking the chain (PIPE-12). spec · `docs/product-spec/08-execution-pipelines.md:17` · high · sha:33e9443472ce -- Invoking the next step MUST advance a monotonic cursor and invoke it; when exhausted it MUST dispatch the current in-flight request to the terminal transport, threading the caller's per-call options, and the cursor MUST only move forward within a single un-forked drive. +- Invoking the next step MUST advance a monotonic cursor and invoke it; when exhausted it MUST dispatch the current in-flight request to the terminal transport, threading the caller's per-call options, and the cursor MUST only move forward within a single un-forked drive (PIPE-13). spec · `docs/product-spec/08-execution-pipelines.md:17` · high · sha:33e9443472ce -- A substituted request MUST propagate to every downstream step and the terminal dispatch. +- A substituted request MUST propagate to every downstream step and the terminal dispatch (PIPE-14). spec · `docs/product-spec/08-execution-pipelines.md:17` · high · sha:33e9443472ce -- A step that drives the downstream chain more than once (retry re-attempting, redirect following a hop, auth retrying after a challenge) MUST fork a fresh cursor for each re-drive rather than reusing the same next handle; reusing the handle resumes past already-visited steps and MUST be treated as a defect. +- A step that drives the downstream chain more than once (retry re-attempting, redirect following a hop, auth retrying after a challenge) MUST fork a fresh cursor for each re-drive rather than reusing the same next handle; reusing the handle resumes past already-visited steps and MUST be treated as a defect (PIPE-15). spec · `docs/product-spec/08-execution-pipelines.md:18` · high · sha:33e9443472ce -- A port MUST provide an equivalent cursor-fork primitive and its wrapping pillar steps MUST use it. +- A port MUST provide an equivalent cursor-fork primitive and its wrapping pillar steps MUST use it (PIPE-15). spec · `docs/product-spec/08-execution-pipelines.md:18` · high · sha:33e9443472ce -- A forked cursor MUST resume from the same position as its parent, carry the current in-flight request, and share the immutable options, with forks advancing independently. +- A forked cursor MUST resume from the same position as its parent, carry the current in-flight request, and share the immutable options, with forks advancing independently (PIPE-16). spec · `docs/product-spec/08-execution-pipelines.md:18` · high · sha:33e9443472ce -- The caller's per-call options MUST be carried unchanged for the entire call, including across every re-drive fork, readable by any step, and threaded into the terminal dispatch; options MUST be immutable/shared, not copied-and-diverged per fork. +- The caller's per-call options MUST be carried unchanged for the entire call, including across every re-drive fork, readable by any step, and threaded into the terminal dispatch; options MUST be immutable/shared, not copied-and-diverged per fork (PIPE-17). spec · `docs/product-spec/08-execution-pipelines.md:19` · high · sha:33e9443472ce -- A wrapping step that re-drives the chain MUST release each superseded intermediate response, closing its body before the next drive, and MUST NOT close the response it ultimately hands back to the caller, so close-responsibility passes outward. +- A wrapping step that re-drives the chain MUST release each superseded intermediate response, closing its body before the next drive, and MUST NOT close the response it ultimately hands back to the caller, so close-responsibility passes outward (PIPE-40). spec · `docs/product-spec/08-execution-pipelines.md:20` · high · sha:33e9443472ce -- On paths that abandon a re-drive (redirect cycle, non-replayable body, budget exhausted), the in-flight response MUST be returned unclosed. +- On paths that abandon a re-drive (redirect cycle, non-replayable body, budget exhausted), the in-flight response MUST be returned unclosed (PIPE-40). spec · `docs/product-spec/08-execution-pipelines.md:20` · high · sha:33e9443472ce -- Non-pillar stages MUST hold an ordered sequence where append adds to the tail and prepend to the head, preserving relative order through build and any re-bucketing edit. +- Non-pillar stages MUST hold an ordered sequence where append adds to the tail and prepend to the head, preserving relative order through build and any re-bucketing edit (PIPE-7). spec · `docs/product-spec/08-execution-pipelines.md:24` · high · sha:33e9443472ce -- The surgical insert-after/insert-before and replace edits MUST act relative to the first existing instance of an anchor type, and the inserted/replacing step MUST declare the same stage as the anchor; a cross-stage insert/replace MUST be rejected. +- The surgical insert-after/insert-before and replace edits MUST act relative to the first existing instance of an anchor type, and the inserted/replacing step MUST declare the same stage as the anchor; a cross-stage insert/replace MUST be rejected (PIPE-18 / PIPE-19). spec · `docs/product-spec/08-execution-pipelines.md:24` · high · sha:33e9443472ce -- Remove MUST delete every instance of a step type, preserving relative order, and be a no-op when the type is absent. +- Remove MUST delete every instance of a step type, preserving relative order, and be a no-op when the type is absent (PIPE-20). spec · `docs/product-spec/08-execution-pipelines.md:24` · high · sha:33e9443472ce -- An insert-relative or replace edit whose anchor type is absent MUST fail identifying the missing type. +- An insert-relative or replace edit whose anchor type is absent MUST fail identifying the missing type (PIPE-21). spec · `docs/product-spec/08-execution-pipelines.md:24` · high · sha:33e9443472ce -- Every mutation that re-buckets steps by stage MUST re-derive the flattened order deterministically, so the observable ordering after an edit equals building the same set from scratch. +- Every mutation that re-buckets steps by stage MUST re-derive the flattened order deterministically, so the observable ordering after an edit equals building the same set from scratch (PIPE-22). spec · `docs/product-spec/08-execution-pipelines.md:25` · high · sha:33e9443472ce -- A bulk reload MUST be all-or-nothing -- a pillar collision leaves the existing collection completely unchanged rather than a partial rebuild. +- A bulk reload MUST be all-or-nothing -- a pillar collision leaves the existing collection completely unchanged rather than a partial rebuild (PIPE-23). spec · `docs/product-spec/08-execution-pipelines.md:25` · high · sha:33e9443472ce -- The standard-resilience preset MUST install into empty pillar slots only, validating up front that no target pillar is occupied and rejecting the whole call, installing nothing, if any pillar is occupied. +- The standard-resilience preset MUST install into empty pillar slots only, validating up front that no target pillar is occupied and rejecting the whole call, installing nothing, if any pillar is occupied (PIPE-24). spec · `docs/product-spec/08-execution-pipelines.md:25` · high · sha:33e9443472ce -- build() MUST produce the ordered sequence by flattening stages in declaration order, skipping SEND, into an immutable runtime that exposes a read-only, ordered view of its steps. +- build() MUST produce the ordered sequence by flattening stages in declaration order, skipping SEND, into an immutable runtime that exposes a read-only, ordered view of its steps (PIPE-25). spec · `docs/product-spec/08-execution-pipelines.md:26` · high · sha:33e9443472ce -- The shipped pillar families SHOULD lock their stage assignment so a subclass cannot relocate out of its pillar. +- The shipped pillar families SHOULD lock their stage assignment so a subclass cannot relocate out of its pillar (PIPE-36). spec · `docs/product-spec/08-execution-pipelines.md:26` · high · sha:33e9443472ce -- A step whose correctness depends on the single terminal response, such as status-to-typed-error mapping, MUST occupy the outermost pre-redirect slot so it runs outside both loops, and on a non-error status MUST return the response untouched. +- A step whose correctness depends on the single terminal response, such as status-to-typed-error mapping, MUST occupy the outermost pre-redirect slot so it runs outside both loops, and on a non-error status MUST return the response untouched (PIPE-37). spec · `docs/product-spec/08-execution-pipelines.md:26` · high · sha:33e9443472ce -- The runtime MUST itself implement the transport SPI, delegating execute/execute-async to its own send/send-async (with and without options), so a configured pipeline can stand in wherever a transport is expected and options survive the indirection. +- The runtime MUST itself implement the transport SPI, delegating execute/execute-async to its own send/send-async (with and without options), so a configured pipeline can stand in wherever a transport is expected and options survive the indirection (PIPE-26). spec · `docs/product-spec/08-execution-pipelines.md:30` · high · sha:33e9443472ce -- Closing the pipeline MUST be a no-op with respect to the underlying transport -- the pipeline never owns its transport and MUST NOT close it. +- Closing the pipeline MUST be a no-op with respect to the underlying transport -- the pipeline never owns its transport and MUST NOT close it (PIPE-27). spec · `docs/product-spec/08-execution-pipelines.md:30` · high · sha:33e9443472ce -- The runtime SHOULD offer convenience constructors for a step-less pipeline forwarding directly to a transport and a standard pipeline installing the default resilience pillars, sync being redirect+retry+instrumentation and async being retry+instrumentation with a caller-supplied scheduler for non-blocking backoff. +- The runtime SHOULD offer convenience constructors for a step-less pipeline forwarding directly to a transport and a standard pipeline installing the default resilience pillars, sync being redirect+retry+instrumentation and async being retry+instrumentation with a caller-supplied scheduler for non-blocking backoff (PIPE-39). spec · `docs/product-spec/08-execution-pipelines.md:31` · high · sha:33e9443472ce -- The async runtime MUST reuse the identical stage identities and staging policy as the sync runtime; the two MUST NOT each re-derive ordering independently. +- The async runtime MUST reuse the identical stage identities and staging policy as the sync runtime; the two MUST NOT each re-derive ordering independently (PIPE-28). spec · `docs/product-spec/08-execution-pipelines.md:35` · high · sha:33e9443472ce -- An async step MUST NOT throw synchronously to signal a transport/async failure -- it MUST return a future completing exceptionally -- and MAY throw synchronously only for caller-bug argument validation. +- An async step MUST NOT throw synchronously to signal a transport/async failure -- it MUST return a future completing exceptionally -- and MAY throw synchronously only for caller-bug argument validation (PIPE-29). spec · `docs/product-spec/08-execution-pipelines.md:35` · high · sha:33e9443472ce -- The async runtime MUST defensively normalize any synchronous exception from a step's async entry point, or the empty-pipeline dispatch, into an exceptionally-completed future, while fatal/unrecoverable errors propagate synchronously and MUST NOT be swallowed. +- The async runtime MUST defensively normalize any synchronous exception from a step's async entry point, or the empty-pipeline dispatch, into an exceptionally-completed future, while fatal/unrecoverable errors propagate synchronously and MUST NOT be swallowed (PIPE-30). spec · `docs/product-spec/08-execution-pipelines.md:35` · high · sha:33e9443472ce -- The async terminal response-mapping operator MUST, on success, apply the handler then close the response, tolerating idempotent double-close; on failure it MUST unwrap async-wrapper exceptions to the original cause and MUST close any response accompanying a failure to avoid leaking the body. +- The async terminal response-mapping operator MUST, on success, apply the handler then close the response, tolerating idempotent double-close; on failure it MUST unwrap async-wrapper exceptions to the original cause and MUST close any response accompanying a failure to avoid leaking the body (PIPE-31). spec · `docs/product-spec/08-execution-pipelines.md:36` · high · sha:33e9443472ce -- The sync-to-async bridge MUST require a caller-supplied executor with no default, run the wrapped synchronous pipeline as a single opaque unit on that executor so its steps stay synchronous on the worker and do not gain per-step concurrency, and thread per-call options into the wrapped send. +- The sync-to-async bridge MUST require a caller-supplied executor with no default, run the wrapped synchronous pipeline as a single opaque unit on that executor so its steps stay synchronous on the worker and do not gain per-step concurrency, and thread per-call options into the wrapped send (PIPE-33). spec · `docs/product-spec/08-execution-pipelines.md:40` · high · sha:33e9443472ce -- Cancelling the sync-to-async bridge's future with interruption MUST interrupt the worker running the in-flight send, and cancelling without interruption MUST complete as cancelled without interrupting. +- Cancelling the sync-to-async bridge's future with interruption MUST interrupt the worker running the in-flight send, and cancelling without interruption MUST complete as cancelled without interrupting (PIPE-33). spec · `docs/product-spec/08-execution-pipelines.md:40` · high · sha:33e9443472ce -- The async-to-sync bridge MUST block on the async result per call while preserving options and MUST honor thread interruption -- on interrupt it restores the flag, cancels the in-flight future, and surfaces an interrupted-I/O error. +- The async-to-sync bridge MUST block on the async result per call while preserving options and MUST honor thread interruption -- on interrupt it restores the flag, cancels the in-flight future, and surfaces an interrupted-I/O error (PIPE-34). spec · `docs/product-spec/08-execution-pipelines.md:40` · high · sha:33e9443472ce -- The builder SHOULD provide two unambiguous ways to seed from an existing pipeline -- FLATTEN, which copies its steps and transport so they run in the same loops, versus NEST, which treats it as an opaque transport so the new steps run once outside the nested loops -- and a port MUST make the flatten-vs-nest choice explicit rather than accidental. +- The builder SHOULD provide two unambiguous ways to seed from an existing pipeline -- FLATTEN, which copies its steps and transport so they run in the same loops, versus NEST, which treats it as an opaque transport so the new steps run once outside the nested loops -- and a port MUST make the flatten-vs-nest choice explicit rather than accidental (PIPE-35). spec · `docs/product-spec/08-execution-pipelines.md:41` · high · sha:33e9443472ce -- The response-side outcome MUST be a closed sum type with exactly two variants -- a success carrying a response and a failure carrying a throwable -- mutually exclusive and jointly exhaustive, with derivable accessors and a fold that applies exactly one of two branches at most once per call. +- The response-side outcome MUST be a closed sum type with exactly two variants -- a success carrying a response and a failure carrying a throwable -- mutually exclusive and jointly exhaustive, with derivable accessors and a fold that applies exactly one of two branches at most once per call (RECOV-1). spec · `docs/product-spec/08-execution-pipelines.md:45` · high · sha:33e9443472ce -- The unified orchestrator MUST catch every throwable from any request-chain step and from the transport invocation, convert it into a Failure, and thread it through the response recovery chain; no throwable from the pre-request phase or the transport may bypass the recovery hooks. +- The unified orchestrator MUST catch every throwable from any request-chain step and from the transport invocation, convert it into a Failure, and thread it through the response recovery chain; no throwable from the pre-request phase or the transport may bypass the recovery hooks (RECOV-2). spec · `docs/product-spec/08-execution-pipelines.md:46` · high · sha:33e9443472ce -- The request recovery chain MUST apply its ordered steps as a sequential left-to-right fold where the output of step N is the input of step N+1; an empty chain returns the input unchanged, and a throwing step aborts the remainder and propagates. +- The request recovery chain MUST apply its ordered steps as a sequential left-to-right fold where the output of step N is the input of step N+1; an empty chain returns the input unchanged, and a throwing step aborts the remainder and propagates (RECOV-3). spec · `docs/product-spec/08-execution-pipelines.md:47` · high · sha:33e9443472ce -- Response steps (response-to-response) MUST run only when the current outcome is a Success; on a Failure the entire response-step phase is skipped. +- Response steps (response-to-response) MUST run only when the current outcome is a Success; on a Failure the entire response-step phase is skipped (RECOV-4). spec · `docs/product-spec/08-execution-pipelines.md:48` · high · sha:33e9443472ce -- Recovery steps MUST be applied to every outcome, successes and failures, sequentially and always, observing the terminal outcome including a failure a response step just produced by throwing. +- Recovery steps MUST be applied to every outcome, successes and failures, sequentially and always, observing the terminal outcome including a failure a response step just produced by throwing (RECOV-5). spec · `docs/product-spec/08-execution-pipelines.md:48` · high · sha:33e9443472ce -- The fold order MUST be all response steps first (on the success path), then all recovery steps, in declared order within each group. +- The fold order MUST be all response steps first (on the success path), then all recovery steps, in declared order within each group (RECOV-6). spec · `docs/product-spec/08-execution-pipelines.md:48` · high · sha:33e9443472ce -- If a response step throws, its throwable MUST be converted into a Failure fed to the subsequent recovery steps, never propagated out of the response chain, so error-mapping steps flow through recovery exactly like a transport error. +- If a response step throws, its throwable MUST be converted into a Failure fed to the subsequent recovery steps, never propagated out of the response chain, so error-mapping steps flow through recovery exactly like a transport error (RECOV-7). spec · `docs/product-spec/08-execution-pipelines.md:49` · high · sha:33e9443472ce -- If a recovery step throws, its throwable MUST be wrapped into a Failure fed to the next recovery step, never aborting the remaining recovery steps, and the chain's apply operation MUST NOT throw under any input. +- If a recovery step throws, its throwable MUST be wrapped into a Failure fed to the next recovery step, never aborting the remaining recovery steps, and the chain's apply operation MUST NOT throw under any input (RECOV-8). spec · `docs/product-spec/08-execution-pipelines.md:49` · high · sha:33e9443472ce -- Recovery steps SHOULD surface errors by returning a Failure rather than throwing. +- Recovery steps SHOULD surface errors by returning a Failure rather than throwing (RECOV-9). spec · `docs/product-spec/08-execution-pipelines.md:49` · high · sha:33e9443472ce -- The orchestrator's dispatch MUST unwrap the final outcome by returning the contained response on Success, or rethrowing the contained throwable unchanged on Failure with no wrapping or substitution; any typed-exception surfacing must be done by a recovery step constructing the error and returning a Failure. +- The orchestrator's dispatch MUST unwrap the final outcome by returning the contained response on Success, or rethrowing the contained throwable unchanged on Failure with no wrapping or substitution; any typed-exception surfacing must be done by a recovery step constructing the error and returning a Failure (RECOV-10). spec · `docs/product-spec/08-execution-pipelines.md:50` · high · sha:33e9443472ce -- When wrapping a cancellation/interruption throwable into a Failure, the wrapping helper MUST re-assert the cancellation signal on the current context before returning, so code later blocked on the outcome still observes the cancellation. +- When wrapping a cancellation/interruption throwable into a Failure, the wrapping helper MUST re-assert the cancellation signal on the current context before returning, so code later blocked on the outcome still observes the cancellation (RECOV-11). spec · `docs/product-spec/08-execution-pipelines.md:50` · high · sha:33e9443472ce -- When a response or recovery step throws while holding a Success response, the pipeline MUST close/release that in-hand response before wrapping the throwable, attaching any close error as suppressed so it never masks the primary, releasing the response exactly once. +- When a response or recovery step throws while holding a Success response, the pipeline MUST close/release that in-hand response before wrapping the throwable, attaching any close error as suppressed so it never masks the primary, releasing the response exactly once (RECOV-12). spec · `docs/product-spec/08-execution-pipelines.md:51` · high · sha:33e9443472ce -- When a step handed a Success deliberately returns a different outcome, whether a Success-to-Failure transform or a substitute Success, the pipeline MUST NOT auto-close the discarded original response; the transforming step owns releasing the response it drops. +- When a step handed a Success deliberately returns a different outcome, whether a Success-to-Failure transform or a substitute Success, the pipeline MUST NOT auto-close the discarded original response; the transforming step owns releasing the response it drops (RECOV-13). spec · `docs/product-spec/08-execution-pipelines.md:51` · high · sha:33e9443472ce -- A chain's step lists MUST behave as immutable after construction, and the response recovery chain MUST defensively copy both its lists at construction. +- A chain's step lists MUST behave as immutable after construction, and the response recovery chain MUST defensively copy both its lists at construction (RECOV-14). spec · `docs/product-spec/08-execution-pipelines.md:52` · high · sha:33e9443472ce -- Recovery chain steps MUST be safe for concurrent invocation, with per-request state in the passed context or the value being transformed, never on the step. +- Recovery chain steps MUST be safe for concurrent invocation, with per-request state in the passed context or the value being transformed, never on the step (RECOV-14). spec · `docs/product-spec/08-execution-pipelines.md:52` · high · sha:33e9443472ce -- The status-to-typed-exception mapping step MUST treat only 400..599 as errors, mapping to the matching typed exception which becomes a Failure, and return all other statuses unchanged. +- The status-to-typed-exception mapping step MUST treat only 400..599 as errors, mapping to the matching typed exception which becomes a Failure, and return all other statuses unchanged (RECOV-15). spec · `docs/product-spec/08-execution-pipelines.md:53` · high · sha:33e9443472ce -- Before mapping an error-status response, both initially and on a re-sent error response, the error body MUST be buffered into a bounded (1 MiB), replayable in-memory copy so the connection is released promptly and the body remains readable on the Failure, with the same bound shared across all buffering paths and the cap a hard truncation with no marker. +- Before mapping an error-status response, both initially and on a re-sent error response, the error body MUST be buffered into a bounded (1 MiB), replayable in-memory copy so the connection is released promptly and the body remains readable on the Failure, with the same bound shared across all buffering paths and the cap a hard truncation with no marker (RECOV-16). spec · `docs/product-spec/08-execution-pipelines.md:53` · high · sha:33e9443472ce - Pillar stages are validated at composition time to admit at most one step (PIPE-4/PIPE-5), distinguished by reference identity for idempotent re-installation (PIPE-6). design · `docs/sdk-design-nodejs/05-pipeline-architecture.md:20-21` · high · sha:16ad31311df7 @@ -119,11 +119,11 @@ design · `docs/sdk-design-nodejs/05-pipeline-architecture.md:50-53` · high · sha:16ad31311df7 ## Constraints -- A pillar stage MUST admit at most one step; the configurable pillars are REDIRECT, RETRY, AUTH, LOGGING, and SERDE. +- A pillar stage MUST admit at most one step; the configurable pillars are REDIRECT, RETRY, AUTH, LOGGING, and SERDE (PIPE-4). spec · `docs/product-spec/08-execution-pipelines.md:12` · high · sha:33e9443472ce -- The terminal SEND stage MUST be reserved for the transport hop, MUST NOT hold a user step, and flattening MUST skip it. +- The terminal SEND stage MUST be reserved for the transport hop, MUST NOT hold a user step, and flattening MUST skip it (PIPE-8). spec · `docs/product-spec/08-execution-pipelines.md:12` · high · sha:33e9443472ce -- The async standard pipeline MUST NOT follow HTTP redirects at the pipeline layer, since there is no async redirect pillar; a 3xx surfaces verbatim unless redirect following is enabled on the transport, and a port MUST document this asymmetry with the sync standard pipeline. +- The async standard pipeline MUST NOT follow HTTP redirects at the pipeline layer, since there is no async redirect pillar; a 3xx surfaces verbatim unless redirect following is enabled on the transport, and a port MUST document this asymmetry with the sync standard pipeline (PIPE-32). spec · `docs/product-spec/08-execution-pipelines.md:36` · high · sha:33e9443472ce - A port MUST NOT collapse the stage-based pipeline and recovery-chain primitives into one layer -- the stage pipeline owns ordering and re-drive-with-fork, while the recovery chain owns the sum-type fold and the uniform-failure guarantee. spec · `docs/product-spec/08-execution-pipelines.md:59` · high · sha:33e9443472ce @@ -133,13 +133,13 @@ ## Conclusions - The stage-based pipeline and the recovery-chain primitives share one backoff calculator and one pacing-header parser so their retry behavior cannot drift. spec · `docs/product-spec/08-execution-pipelines.md:3` · high · sha:33e9443472ce -- A closed two-variant outcome is what lets one code path handle a throwable and a response identically. +- A closed two-variant outcome is what lets one code path handle a throwable and a response identically (RECOV-1). spec · `docs/product-spec/08-execution-pipelines.md:45` · high · sha:33e9443472ce - The stage-based pipeline is used as the composition surface for assembling a client because it is where redirect, retry, auth, logging/instrumentation, and serialization concerns are ordered as pillar steps, where per-call cursors and forks drive re-attempts, and where a configured pipeline becomes a transport others can nest, and it has a real async mirror. spec · `docs/product-spec/08-execution-pipelines.md:57` · high · sha:33e9443472ce - The recovery-chain primitives are used as the resilience layer when a concern must observe every outcome uniformly, in particular error-mapping, retry, or rescue logic that must see a transport failure and a response failure through one code path and must never let a pre-transport throw bypass it; the recovery layer is synchronous, with its async equivalent expressed through the stage-based async pipeline. spec · `docs/product-spec/08-execution-pipelines.md:59` · high · sha:33e9443472ce -- The recovery-aware retry stack enforces a total-timeout budget that the stage-based retry step intentionally omits, and a port unifying retry entry points MUST make that budget explicitly opt-in. +- The recovery-aware retry stack enforces a total-timeout budget that the stage-based retry step intentionally omits, and a port unifying retry entry points MUST make that budget explicitly opt-in (RETRY-28). spec · `docs/product-spec/08-execution-pipelines.md:59` · high · sha:33e9443472ce - The stage-based pipeline (§8.1 of the spec) is structurally identical to the "onion" middleware composition pattern already implemented by every Koa-descended Node HTTP framework, including Koa itself, tRPC's middleware, and Apollo Server's plugin model. design · `docs/sdk-design-nodejs/05-pipeline-architecture.md:3-9` · high · sha:16ad31311df7 @@ -153,11 +153,11 @@ ## Reference - The SDK has two cooperating pipeline layers -- the stage-based pipeline, the user-facing dispatch runtime where cross-cutting concerns become discrete bidirectional steps on a fixed totally-ordered list of named stages, and the recovery-chain primitives, the resilience layer beneath resilience steps threading a closed two-variant outcome through a fold so every failure is observed uniformly. spec · `docs/product-spec/08-execution-pipelines.md:3` · high · sha:33e9443472ce -- The SERDE pillar is a reserved stage slot with no shipped behavior. +- The SERDE pillar is a reserved stage slot with no shipped behavior (PIPE-2). spec · `docs/product-spec/08-execution-pipelines.md:10` · high · sha:33e9443472ce -- Append-all MUST preserve the batch's iteration order within a stage, while prepend-all (each element prepended individually) results in the reversed batch order; a port MUST document this asymmetry. +- Append-all MUST preserve the batch's iteration order within a stage, while prepend-all (each element prepended individually) results in the reversed batch order; a port MUST document this asymmetry (PIPE-38). spec · `docs/product-spec/08-execution-pipelines.md:26` · high · sha:33e9443472ce -- In the reference implementation the request recovery chain does not defensively copy, retaining the caller's read-only list reference directly, an asymmetry a porter must not assume away; a port SHOULD copy there too. +- In the reference implementation the request recovery chain does not defensively copy, retaining the caller's read-only list reference directly, an asymmetry a porter must not assume away; a port SHOULD copy there too (RECOV-14). spec · `docs/product-spec/08-execution-pipelines.md:52` · high · sha:33e9443472ce - A pipeline step is a function of type `(request: Request, next: Next) => Promise`, where `Next = () => Promise`. design · `docs/sdk-design-nodejs/05-pipeline-architecture.md:12-16` · high · sha:16ad31311df7 diff --git a/docs/knowledge/redaction-and-security.md b/docs/knowledge/harvested/redaction-and-security.md similarity index 100% rename from docs/knowledge/redaction-and-security.md rename to docs/knowledge/harvested/redaction-and-security.md diff --git a/docs/knowledge/redirect-handling.md b/docs/knowledge/harvested/redirect-handling.md similarity index 88% rename from docs/knowledge/redirect-handling.md rename to docs/knowledge/harvested/redirect-handling.md index d8c1e90..a676317 100644 --- a/docs/knowledge/redirect-handling.md +++ b/docs/knowledge/harvested/redirect-handling.md @@ -1,59 +1,59 @@ # redirect-handling ## Rules -- A redirect is attempted only for status codes 301, 302, 303, 307, and 308; any other status, including 2xx, 4xx, 5xx, and non-redirect 3xx, is returned verbatim without consulting redirect logic. +- A redirect is attempted only for status codes 301, 302, 303, 307, and 308; any other status, including 2xx, 4xx, 5xx, and non-redirect 3xx, is returned verbatim without consulting redirect logic (REDIR-1). spec · `docs/product-spec/10-redirect-handling.md:7` · high · sha:f2a0d207be56 -- Status codes 300, 304, and 305 MUST NOT be auto-followed even with a Location header, and 305 in particular must never redirect to a server-chosen proxy. +- Status codes 300, 304, and 305 MUST NOT be auto-followed even with a Location header, and 305 in particular must never redirect to a server-chosen proxy (REDIR-2). spec · `docs/product-spec/10-redirect-handling.md:7` · high · sha:f2a0d207be56 -- For 301 and 302, a redirect is followed only if the original request method is in the configured allowed-method set (default {GET, HEAD}), and when followed, the original method and body are preserved, with deliberately no automatic POST-to-GET rewrite. +- For 301 and 302, a redirect is followed only if the original request method is in the configured allowed-method set (default {GET, HEAD}), and when followed, the original method and body are preserved, with deliberately no automatic POST-to-GET rewrite (REDIR-3). spec · `docs/product-spec/10-redirect-handling.md:8` · high · sha:f2a0d207be56 -- 307 and 308 redirects preserve method and body and are followed only if the method is in the allowed-method set. +- 307 and 308 redirects preserve method and body and are followed only if the method is in the allowed-method set (REDIR-4). spec · `docs/product-spec/10-redirect-handling.md:8` · high · sha:f2a0d207be56 -- 303 is not followed by default; when opted in it is re-issued as a GET with the body dropped and every Content-* request header, matched case-insensitively, removed, regardless of the original method. +- 303 is not followed by default; when opted in it is re-issued as a GET with the body dropped and every Content-* request header, matched case-insensitively, removed, regardless of the original method (REDIR-5). spec · `docs/product-spec/10-redirect-handling.md:8` · high · sha:f2a0d207be56 -- Any followed method-preserving redirect (301/302/307/308) re-sends the original body, so the body MUST be replayable; if present and not replayable, the operation MUST fail with a clear error naming replayability rather than corrupting or truncating the re-send, and the redirect is not attempted (303 is exempt because it drops the body). +- Any followed method-preserving redirect (301/302/307/308) re-sends the original body, so the body MUST be replayable; if present and not replayable, the operation MUST fail with a clear error naming replayability rather than corrupting or truncating the re-send, and the redirect is not attempted (303 is exempt because it drops the body) (REDIR-6). spec · `docs/product-spec/10-redirect-handling.md:9` · high · sha:f2a0d207be56 -- The Authorization header MUST be stripped before every redirect re-issue, including same-origin and the 303 GET rebuild, because re-attaching a credential for a known origin is the auth layer's job. +- The Authorization header MUST be stripped before every redirect re-issue, including same-origin and the 303 GET rebuild, because re-attaching a credential for a known origin is the auth layer's job (REDIR-7). spec · `docs/product-spec/10-redirect-handling.md:13` · high · sha:f2a0d207be56 -- A redirect is cross-origin if and only if the resolved target differs from the original (seed) request origin in scheme, host (case-insensitive), or effective port (scheme default when omitted); the comparison MUST be against the seed origin, not the previous hop, so a same-origin sub-redirect on a foreign host cannot re-expose the credential. +- A redirect is cross-origin if and only if the resolved target differs from the original (seed) request origin in scheme, host (case-insensitive), or effective port (scheme default when omitted); the comparison MUST be against the seed origin, not the previous hop, so a same-origin sub-redirect on a foreign host cannot re-expose the credential (REDIR-8). spec · `docs/product-spec/10-redirect-handling.md:13` · high · sha:f2a0d207be56 -- On a cross-origin redirect, whether method-preserving or a 303 GET rebuild, the origin-scoped Cookie and Proxy-Authorization headers MUST also be stripped. +- On a cross-origin redirect, whether method-preserving or a 303 GET rebuild, the origin-scoped Cookie and Proxy-Authorization headers MUST also be stripped (REDIR-9). spec · `docs/product-spec/10-redirect-handling.md:13` · high · sha:f2a0d207be56 -- On a same-origin redirect the Cookie header SHOULD be retained, with only Authorization stripped same-origin; a more conservative port MAY strip all cookies. +- On a same-origin redirect the Cookie header SHOULD be retained, with only Authorization stripped same-origin; a more conservative port MAY strip all cookies (REDIR-10). spec · `docs/product-spec/10-redirect-handling.md:13` · high · sha:f2a0d207be56 -- Because the auth layer runs inside the redirect loop, a cross-origin re-issue MUST carry an out-of-band signal instructing the auth layer to skip credential stamping; this signal MUST be impossible for a server-supplied Location to forge into a leak, MUST only suppress stamping and never cause a credential to be sent, and MUST be removed by the credential-attaching layer before dispatch; a same-origin re-issue is not signaled and is re-stamped normally. +- Because the auth layer runs inside the redirect loop, a cross-origin re-issue MUST carry an out-of-band signal instructing the auth layer to skip credential stamping; this signal MUST be impossible for a server-supplied Location to forge into a leak, MUST only suppress stamping and never cause a credential to be sent, and MUST be removed by the credential-attaching layer before dispatch; a same-origin re-issue is not signaled and is re-stamped normally (REDIR-11). spec · `docs/product-spec/10-redirect-handling.md:14` · high · sha:f2a0d207be56 -- The redirect layer clears any inbound copy of the cross-origin marker on every re-issue before conditionally setting its own on a cross-origin hop, making it impossible for a server-supplied Location to forge the marker. +- The redirect layer clears any inbound copy of the cross-origin marker on every re-issue before conditionally setting its own on a cross-origin hop, making it impossible for a server-supplied Location to forge the marker (REDIR-11). spec · `docs/product-spec/10-redirect-handling.md:14` · high · sha:f2a0d207be56 -- The redirect follower MUST wrap the auth layer, with redirect outer and auth inside per hop, which is what necessitates the Authorization-stripping and cross-origin-signal requirements. +- The redirect follower MUST wrap the auth layer, with redirect outer and auth inside per hop, which is what necessitates the Authorization-stripping and cross-origin-signal requirements (REDIR-24). spec · `docs/product-spec/10-redirect-handling.md:14` · high · sha:f2a0d207be56 -- Userinfo in the Location target (user:pass@) MUST be dropped before re-issue, and server-supplied embedded credentials MUST never be used. +- Userinfo in the Location target (user:pass@) MUST be dropped before re-issue, and server-supplied embedded credentials MUST never be used (REDIR-12). spec · `docs/product-spec/10-redirect-handling.md:15` · high · sha:f2a0d207be56 -- Stripping userinfo and resolving the Location generally MUST preserve the wire-exact, already-percent-encoded path, query, and fragment, and MUST preserve bracketed IPv6 literal hosts and explicit ports; re-encoding that would decode %2F to / or %26 to & is forbidden. +- Stripping userinfo and resolving the Location generally MUST preserve the wire-exact, already-percent-encoded path, query, and fragment, and MUST preserve bracketed IPv6 literal hosts and explicit ports; re-encoding that would decode %2F to / or %26 to & is forbidden (REDIR-13). spec · `docs/product-spec/10-redirect-handling.md:15` · high · sha:f2a0d207be56 -- A relative Location MUST be resolved against the current hop's request URL per RFC 3986; absolute values are used as-is after userinfo stripping. +- A relative Location MUST be resolved against the current hop's request URL per RFC 3986; absolute values are used as-is after userinfo stripping (REDIR-14). spec · `docs/product-spec/10-redirect-handling.md:19` · high · sha:f2a0d207be56 -- An HTTPS-to-HTTP scheme downgrade across a single hop MUST be rejected by default, failing with a clear error, and permitted only via an opt-in that surfaces the downgrade observably; credential stripping applies regardless, and the check is evaluated per hop. +- An HTTPS-to-HTTP scheme downgrade across a single hop MUST be rejected by default, failing with a clear error, and permitted only via an opt-in that surfaces the downgrade observably; credential stripping applies regardless, and the check is evaluated per hop (REDIR-15). spec · `docs/product-spec/10-redirect-handling.md:19` · high · sha:f2a0d207be56 -- The redirect step MUST detect redirect loops by recording every visited absolute URI, seeded with the original request URI, and when a redirect would revisit a seen URI, MUST stop and return the current redirect response without throwing, leaving its body open for the caller. +- The redirect step MUST detect redirect loops by recording every visited absolute URI, seeded with the original request URI, and when a redirect would revisit a seen URI, MUST stop and return the current redirect response without throwing, leaving its body open for the caller (REDIR-16). spec · `docs/product-spec/10-redirect-handling.md:20` · high · sha:f2a0d207be56 -- The number of followed redirects MUST be capped by max-hops (default 3); on reaching the cap the last response is returned as-is even if itself a 3xx, without throwing, and max-hops 0 MUST disable redirect following entirely. +- The number of followed redirects MUST be capped by max-hops (default 3); on reaching the cap the last response is returned as-is even if itself a 3xx, without throwing, and max-hops 0 MUST disable redirect following entirely (REDIR-17). spec · `docs/product-spec/10-redirect-handling.md:20` · high · sha:f2a0d207be56 -- A malformed or unresolvable Location, such as an invalid URI, illegal characters, or an unsupported scheme, MUST NOT throw; the step logs it and returns the current redirect response unfollowed. +- A malformed or unresolvable Location, such as an invalid URI, illegal characters, or an unsupported scheme, MUST NOT throw; the step logs it and returns the current redirect response unfollowed (REDIR-18). spec · `docs/product-spec/10-redirect-handling.md:21` · high · sha:f2a0d207be56 -- A redirect response with a missing or empty Location MUST be returned unfollowed. +- A redirect response with a missing or empty Location MUST be returned unfollowed (REDIR-19). spec · `docs/product-spec/10-redirect-handling.md:21` · high · sha:f2a0d207be56 -- The redirect step MUST manage response-body lifecycle deterministically -- before issuing a follow-up the prior redirect response's body MUST be closed; if building the follow-up throws (non-replayable body, downgrade rejection) the current response MUST be closed before the error propagates; on any 'return current' outcome the returned response is left open for the caller. +- The redirect step MUST manage response-body lifecycle deterministically -- before issuing a follow-up the prior redirect response's body MUST be closed; if building the follow-up throws (non-replayable body, downgrade rejection) the current response MUST be closed before the error propagates; on any 'return current' outcome the returned response is left open for the caller (REDIR-22). spec · `docs/product-spec/10-redirect-handling.md:22` · high · sha:f2a0d207be56 -- Redirect following SHOULD be an iterative loop, not unbounded recursion, so it is stack-safe. +- Redirect following SHOULD be an iterative loop, not unbounded recursion, so it is stack-safe (REDIR-23). spec · `docs/product-spec/10-redirect-handling.md:22` · high · sha:f2a0d207be56 -- A configured redirect predicate MUST fully override the built-in decision and receive a read-only, defensively-copied condition snapshot containing the current response, the count of redirects already followed, and an insertion-ordered set of visited URIs including the current request's, so it cannot mutate the live cycle-detection state. +- A configured redirect predicate MUST fully override the built-in decision and receive a read-only, defensively-copied condition snapshot containing the current response, the count of redirects already followed, and an insertion-ordered set of visited URIs including the current request's, so it cannot mutate the live cycle-detection state (REDIR-20). spec · `docs/product-spec/10-redirect-handling.md:26` · high · sha:f2a0d207be56 -- On the non-redirect fast path, a status that is not a recognized redirect code, the implementation SHOULD short-circuit before allocating a condition snapshot and MUST NOT consult the predicate; but a recognized 3xx always allocates the snapshot and consults the predicate, even with no usable Location. +- On the non-redirect fast path, a status that is not a recognized redirect code, the implementation SHOULD short-circuit before allocating a condition snapshot and MUST NOT consult the predicate; but a recognized 3xx always allocates the snapshot and consults the predicate, even with no usable Location (REDIR-21). spec · `docs/product-spec/10-redirect-handling.md:26` · high · sha:f2a0d207be56 -- The configured allowed-method set MUST be stored as an immutable defensive copy so post-construction mutation of the caller's collection cannot change policy. +- The configured allowed-method set MUST be stored as an immutable defensive copy so post-construction mutation of the caller's collection cannot change policy (REDIR-26). spec · `docs/product-spec/10-redirect-handling.md:27` · high · sha:f2a0d207be56 -- Each followed hop, loop detection, and scheme-downgrade event SHOULD be emitted as structured records with URLs passed through a redactor, redaction failures degrading to a placeholder rather than crashing logging; the malformed-Location event is the exception, logging the raw Location string as received since it failed to parse and cannot be redacted. +- Each followed hop, loop detection, and scheme-downgrade event SHOULD be emitted as structured records with URLs passed through a redactor, redaction failures degrading to a placeholder rather than crashing logging; the malformed-Location event is the exception, logging the raw Location string as received since it failed to parse and cannot be redacted (REDIR-28). spec · `docs/product-spec/10-redirect-handling.md:27` · high · sha:f2a0d207be56 ## Constraints @@ -63,11 +63,11 @@ design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:49-53` · high · sha:b0e2bb42d809 ## Reference -- Redirect following is a synchronous pillar step coordinating with the auth pillar via an internal cross-origin marker; the async pipeline follows no redirects. +- Redirect following is a synchronous pillar step coordinating with the auth pillar via an internal cross-origin marker; the async pipeline follows no redirects (REDIR-25). spec · `docs/product-spec/10-redirect-handling.md:3` · high · sha:f2a0d207be56 -- Only the auth step strips the internal cross-origin marker in the reference implementation, so a pipeline with no auth step, including the sync standard-resilience preset, forwards the internal marker to the transport; a robust port should strip the signal independently of whether a credential layer runs. +- Only the auth step strips the internal cross-origin marker in the reference implementation, so a pipeline with no auth step, including the sync standard-resilience preset, forwards the internal marker to the transport; a robust port should strip the signal independently of whether a credential layer runs (REDIR-11). spec · `docs/product-spec/10-redirect-handling.md:14` · high · sha:f2a0d207be56 -- The header the redirect target is read from MAY be configurable, defaulting to Location. +- The header the redirect target is read from MAY be configurable, defaulting to Location (REDIR-27). spec · `docs/product-spec/10-redirect-handling.md:27` · high · sha:f2a0d207be56 - Cross-origin detection in the port compares `new URL(target).origin` against the seed origin after normalizing default ports. design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:53-55` · high · sha:b0e2bb42d809 diff --git a/docs/knowledge/resource-management.md b/docs/knowledge/harvested/resource-management.md similarity index 100% rename from docs/knowledge/resource-management.md rename to docs/knowledge/harvested/resource-management.md diff --git a/docs/knowledge/retry-and-resilience.md b/docs/knowledge/harvested/retry-and-resilience.md similarity index 89% rename from docs/knowledge/retry-and-resilience.md rename to docs/knowledge/harvested/retry-and-resilience.md index ae88bc0..cb8dd75 100644 --- a/docs/knowledge/retry-and-resilience.md +++ b/docs/knowledge/harvested/retry-and-resilience.md @@ -5,83 +5,83 @@ spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:18-18` · high · sha:c2bf15dc8a06 - On the retry path specifically, a body-less request's re-send eligibility MUST gate on method idempotency rather than replayability, so only idempotent methods are retried when there is no body, meaning a body-less non-idempotent POST is not retried (BODY-5). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:19-19` · high · sha:c2bf15dc8a06 -- The retryable-status classifier MUST be single-sourced and treat exactly 408, 429, and all of 500-599 except 501 and 505 as retryable; this is the single definition the response-carrying exception flag and the stage stack's default predicate derive from, while the recovery stack layers its own configurable status allow-list on top. +- The retryable-status classifier MUST be single-sourced and treat exactly 408, 429, and all of 500-599 except 501 and 505 as retryable; this is the single definition the response-carrying exception flag and the stage stack's default predicate derive from, while the recovery stack layers its own configurable status allow-list on top (RETRY-1). spec · `docs/product-spec/09-retry-and-resilience.md:9` · high · sha:9efbe276001e -- The retryable-throwable set MUST be defined in exactly one place -- any throwable that is, or has anywhere in its cause chain, an I/O error or a timeout error, found via an iterative, identity-tracking cause-chain walk that terminates on a cyclic chain. +- The retryable-throwable set MUST be defined in exactly one place -- any throwable that is, or has anywhere in its cause chain, an I/O error or a timeout error, found via an iterative, identity-tracking cause-chain walk that terminates on a cyclic chain (RETRY-2). spec · `docs/product-spec/09-retry-and-resilience.md:10` · high · sha:9efbe276001e -- A response-carrying exception MUST derive its own retryable flag from the single status classifier at construction, not a hardcoded per-subclass constant. +- A response-carrying exception MUST derive its own retryable flag from the single status classifier at construction, not a hardcoded per-subclass constant (RETRY-3). spec · `docs/product-spec/09-retry-and-resilience.md:10` · high · sha:9efbe276001e -- A transport-level failure that produced no complete response, such as connection refused, TLS/DNS failure, socket read timeout, or peer reset, MUST be classified retryable unconditionally at the condition level, with safety gated separately. +- A transport-level failure that produced no complete response, such as connection refused, TLS/DNS failure, socket read timeout, or peer reset, MUST be classified retryable unconditionally at the condition level, with safety gated separately (RETRY-4). spec · `docs/product-spec/09-retry-and-resilience.md:10` · high · sha:9efbe276001e -- A request is re-sendable if and only if it has no body and its method is idempotent, or it has a body and that body is replayable; both retry stacks MUST apply this identical rule. +- A request is re-sendable if and only if it has no body and its method is idempotent, or it has a body and that body is replayable; both retry stacks MUST apply this identical rule (RETRY-5 / RECOV-18). spec · `docs/product-spec/09-retry-and-resilience.md:11` · high · sha:9efbe276001e -- When a request is not re-sendable, the retry logic MUST perform exactly one attempt and MUST NOT retry, even when the condition is retryable and even when there is no body to physically re-send (a bare non-idempotent POST). +- When a request is not re-sendable, the retry logic MUST perform exactly one attempt and MUST NOT retry, even when the condition is retryable and even when there is no body to physically re-send (a bare non-idempotent POST) (RETRY-7). spec · `docs/product-spec/09-retry-and-resilience.md:12` · high · sha:9efbe276001e -- Retry eligibility MUST require both a retryable condition and a re-sendable request; neither implies the other. +- Retry eligibility MUST require both a retryable condition and a re-sendable request; neither implies the other (RETRY-8). spec · `docs/product-spec/09-retry-and-resilience.md:12` · high · sha:9efbe276001e -- The unjittered exponential delay MUST be initialDelay times multiplier raised to (attempt minus 1), with attempt 1-indexed such that attempt 1 is the wait before the first retry, clamped to a maximum delay cap. +- The unjittered exponential delay MUST be initialDelay times multiplier raised to (attempt minus 1), with attempt 1-indexed such that attempt 1 is the wait before the first retry, clamped to a maximum delay cap (RETRY-9 / RECOV-21). spec · `docs/product-spec/09-retry-and-resilience.md:16` · high · sha:9efbe276001e -- Symmetric jitter MUST draw the effective delay uniformly from [d*(1-j/2), d*(1+j/2)] with midpoint d, j=0 returning d, j constrained to [0,1], a degenerate sub-nanosecond range returning the base delay, and a negative sample floored to zero. +- Symmetric jitter MUST draw the effective delay uniformly from [d*(1-j/2), d*(1+j/2)] with midpoint d, j=0 returning d, j constrained to [0,1], a degenerate sub-nanosecond range returning the base delay, and a negative sample floored to zero (RETRY-10). spec · `docs/product-spec/09-retry-and-resilience.md:16` · high · sha:9efbe276001e -- Delay computation MUST be overflow-safe, saturating to the cap rather than throwing, and MUST reject an attempt value less than 1. +- Delay computation MUST be overflow-safe, saturating to the cap rather than throwing, and MUST reject an attempt value less than 1 (RETRY-11). spec · `docs/product-spec/09-retry-and-resilience.md:16` · high · sha:9efbe276001e -- The pacing-header parser MUST recognize Retry-After as delta-seconds (integer and fractional), Retry-After as an RFC 1123 HTTP-date tolerant of an informational weekday and single-digit day, retry-after-ms and x-ms-retry-after-ms as integer milliseconds, and X-RateLimit-Reset as Unix epoch seconds whose delta is positively jittered to [100%,120%]. +- The pacing-header parser MUST recognize Retry-After as delta-seconds (integer and fractional), Retry-After as an RFC 1123 HTTP-date tolerant of an informational weekday and single-digit day, retry-after-ms and x-ms-retry-after-ms as integer milliseconds, and X-RateLimit-Reset as Unix epoch seconds whose delta is positively jittered to [100%,120%] (RETRY-15 / RECOV-24 / RECOV-25). spec · `docs/product-spec/09-retry-and-resilience.md:17` · high · sha:9efbe276001e -- The pacing-header parser MUST be total and never throw; malformed, negative, or out-of-range values MUST map to no hint (null), not a zero delay, so the caller falls back to backoff rather than hammering the server. +- The pacing-header parser MUST be total and never throw; malformed, negative, or out-of-range values MUST map to no hint (null), not a zero delay, so the caller falls back to backoff rather than hammering the server (RETRY-16 / RECOV-23). spec · `docs/product-spec/09-retry-and-resilience.md:17` · high · sha:9efbe276001e -- A valid HTTP-date or epoch value already in the past MUST yield a zero delay (retry immediately), distinct from an unparseable value which yields no hint. +- A valid HTTP-date or epoch value already in the past MUST yield a zero delay (retry immediately), distinct from an unparseable value which yields no hint (RETRY-17). spec · `docs/product-spec/09-retry-and-resilience.md:17` · high · sha:9efbe276001e -- Any computed pacing delta MUST be clamped to a finite ceiling of 365 days before nanosecond conversion. +- Any computed pacing delta MUST be clamped to a finite ceiling of 365 days before nanosecond conversion (RETRY-18 / RECOV-26). spec · `docs/product-spec/09-retry-and-resilience.md:17` · high · sha:9efbe276001e -- Numeric Retry-After parsing MUST be screened by a strict decimal grammar before any float parse, rejecting type-suffixed, hex-float, NaN, and Infinity forms. +- Numeric Retry-After parsing MUST be screened by a strict decimal grammar before any float parse, rejecting type-suffixed, hex-float, NaN, and Infinity forms (RETRY-19). spec · `docs/product-spec/09-retry-and-resilience.md:17` · high · sha:9efbe276001e -- A present pacing hint MUST override (replace, not augment) the exponential schedule for that single decision; a literal Retry-After hint MUST NOT receive additional symmetric jitter, and where a total-timeout deadline applies the hint MUST still be clamped against it. +- A present pacing hint MUST override (replace, not augment) the exponential schedule for that single decision; a literal Retry-After hint MUST NOT receive additional symmetric jitter, and where a total-timeout deadline applies the hint MUST still be clamped against it (RETRY-20 / RECOV-22). spec · `docs/product-spec/09-retry-and-resilience.md:18` · high · sha:9efbe276001e -- Pacing resolution MUST honor a defined precedence and return the first parseable value -- the recovery stack scans the whole header map with fixed precedence Retry-After numeric then date, then retry-after-ms, then x-ms-retry-after-ms, then X-RateLimit-Reset, while the stage stack walks a caller-configurable ordered header list. +- Pacing resolution MUST honor a defined precedence and return the first parseable value -- the recovery stack scans the whole header map with fixed precedence Retry-After numeric then date, then retry-after-ms, then x-ms-retry-after-ms, then X-RateLimit-Reset, while the stage stack walks a caller-configurable ordered header list (RETRY-21). spec · `docs/product-spec/09-retry-and-resilience.md:18` · high · sha:9efbe276001e -- A failure while parsing a pacing header MUST NOT mask the real upstream failure; the loop falls back to exponential backoff and the original throwable remains the surfaced error. +- A failure while parsing a pacing header MUST NOT mask the real upstream failure; the loop falls back to exponential backoff and the original throwable remains the surfaced error (RETRY-22 / RECOV-29). spec · `docs/product-spec/09-retry-and-resilience.md:18` · high · sha:9efbe276001e -- Thread interruption/cancellation MUST never be treated as a retryable failure; on interrupt during a blocking backoff wait, the implementation MUST restore the cancellation flag, cancel any externally-scheduled wake, abort the retry loop, and surface an interrupted-I/O error, and a downstream interrupt surfaced as an interrupted-I/O error is treated as terminal cancellation, not retried. +- Thread interruption/cancellation MUST never be treated as a retryable failure; on interrupt during a blocking backoff wait, the implementation MUST restore the cancellation flag, cancel any externally-scheduled wake, abort the retry loop, and surface an interrupted-I/O error, and a downstream interrupt surfaced as an interrupted-I/O error is treated as terminal cancellation, not retried (RETRY-23). spec · `docs/product-spec/09-retry-and-resilience.md:22` · high · sha:9efbe276001e -- A read-timeout represented as a subtype of the interrupted-I/O error MUST NOT be mistaken for cancellation; it remains a retryable condition. +- A read-timeout represented as a subtype of the interrupted-I/O error MUST NOT be mistaken for cancellation; it remains a retryable condition (RETRY-24). spec · `docs/product-spec/09-retry-and-resilience.md:22` · high · sha:9efbe276001e -- Non-recoverable runtime errors such as out-of-memory and stack overflow MUST NOT be retried, classified retryable, or logged; they MUST be surfaced unchanged with no suppressed-trail attachment. +- Non-recoverable runtime errors such as out-of-memory and stack overflow MUST NOT be retried, classified retryable, or logged; they MUST be surfaced unchanged with no suppressed-trail attachment (RETRY-25). spec · `docs/product-spec/09-retry-and-resilience.md:22` · high · sha:9efbe276001e -- The inter-attempt wait MUST be cancellable/interruptible and MUST NOT pin an execution carrier for its duration; a naive uninterruptible sleep that cannot be cancelled is non-conforming. +- The inter-attempt wait MUST be cancellable/interruptible and MUST NOT pin an execution carrier for its duration; a naive uninterruptible sleep that cannot be cancelled is non-conforming (RETRY-26 / RECOV-27). spec · `docs/product-spec/09-retry-and-resilience.md:23` · high · sha:9efbe276001e -- The recovery stack MUST enforce an optional total-timeout budget with per-attempt deadline shrinking, aborting before each attempt if the attempt cap is reached, if elapsed time is at or beyond the budget, or if elapsed plus the next delay would exceed the budget, clamping the delay so it cannot overshoot, with a zero budget disabling the deadline. +- The recovery stack MUST enforce an optional total-timeout budget with per-attempt deadline shrinking, aborting before each attempt if the attempt cap is reached, if elapsed time is at or beyond the budget, or if elapsed plus the next delay would exceed the budget, clamping the delay so it cannot overshoot, with a zero budget disabling the deadline (RETRY-27 / RECOV-20). spec · `docs/product-spec/09-retry-and-resilience.md:27` · high · sha:9efbe276001e -- Both retry stacks MUST compute backoff via one shared calculator and shared constants, and their attempt budgets MUST denote the same number of total wire sends under equivalent defaults. +- Both retry stacks MUST compute backoff via one shared calculator and shared constants, and their attempt budgets MUST denote the same number of total wire sends under equivalent defaults (RETRY-13 / RETRY-14 / RECOV-30). spec · `docs/product-spec/09-retry-and-resilience.md:28` · high · sha:9efbe276001e -- In the recovery stack, for a failure carrying a received response, the configured retryable-status set MUST be authoritative, able to both widen and narrow relative to the built-in classifier, while a no-response transport failure falls back to its always-retryable flag. +- In the recovery stack, for a failure carrying a received response, the configured retryable-status set MUST be authoritative, able to both widen and narrow relative to the built-in classifier, while a no-response transport failure falls back to its always-retryable flag (RETRY-37 / RECOV-17). spec · `docs/product-spec/09-retry-and-resilience.md:29` · high · sha:9efbe276001e -- A re-sent response whose error status is in the configured retryable-status set MUST be re-mapped into a typed failure so the loop keeps evaluating the budget, e.g. a 503,503,200 sequence reaches the 200; all other re-sent responses pass through as Success. +- A re-sent response whose error status is in the configured retryable-status set MUST be re-mapped into a typed failure so the loop keeps evaluating the budget, e.g. a 503,503,200 sequence reaches the 200; all other re-sent responses pass through as Success (RETRY-36 / RECOV-19). spec · `docs/product-spec/09-retry-and-resilience.md:29` · high · sha:9efbe276001e -- A retryable response's body/connection MUST be released before the backoff wait so a socket is not pinned across the delay; the pacing delay is computed from the still-open response first, and if the retry decision or delay computation throws, the response MUST still be closed before propagating. +- A retryable response's body/connection MUST be released before the backoff wait so a socket is not pinned across the delay; the pacing delay is computed from the still-open response first, and if the retry decision or delay computation throws, the response MUST still be closed before propagating (RETRY-35). spec · `docs/product-spec/09-retry-and-resilience.md:30` · high · sha:9efbe276001e -- On terminal failure, every prior failed attempt's exception MUST be attached to the surfaced exception as suppressed, skipping the surfaced instance itself so a reused exception instance cannot trip a self-suppression error, and on eventual success the prior trail MUST be discarded. +- On terminal failure, every prior failed attempt's exception MUST be attached to the surfaced exception as suppressed, skipping the surfaced instance itself so a reused exception instance cannot trip a self-suppression error, and on eventual success the prior trail MUST be discarded (RETRY-34). spec · `docs/product-spec/09-retry-and-resilience.md:30` · high · sha:9efbe276001e -- The asynchronous retry loop MUST be driven by an iterative trampoline; N retries MUST NOT build an N-deep chain of future continuations or stack frames, and a completion warranting another attempt hands control to a single active pump via a re-arm flag rather than recursing. +- The asynchronous retry loop MUST be driven by an iterative trampoline; N retries MUST NOT build an N-deep chain of future continuations or stack frames, and a completion warranting another attempt hands control to a single active pump via a re-arm flag rather than recursing (RETRY-30). spec · `docs/product-spec/09-retry-and-resilience.md:34` · high · sha:9efbe276001e -- Async backoff delays MUST be scheduled non-blockingly, with a zero-length delay completing inline and re-arming the active pump. +- Async backoff delays MUST be scheduled non-blockingly, with a zero-length delay completing inline and re-arming the active pump (RETRY-31). spec · `docs/product-spec/09-retry-and-resilience.md:34` · high · sha:9efbe276001e -- If the caller has already completed or cancelled the returned async result, the driver MUST launch no further attempts, and any response arriving from an in-flight attempt MUST be closed rather than leaked. +- If the caller has already completed or cancelled the returned async result, the driver MUST launch no further attempts, and any response arriving from an in-flight attempt MUST be closed rather than leaked (RETRY-32). spec · `docs/product-spec/09-retry-and-resilience.md:34` · high · sha:9efbe276001e -- Every terminal path of the async retry loop MUST complete the returned future, with a throwing predicate, delay computation, log call, or synchronous scheduler rejection each completing it exceptionally, closing any open retryable response first. +- Every terminal path of the async retry loop MUST complete the returned future, with a throwing predicate, delay computation, log call, or synchronous scheduler rejection each completing it exceptionally, closing any open retryable response first (RETRY-33). spec · `docs/product-spec/09-retry-and-resilience.md:34` · high · sha:9efbe276001e -- The stage stack's delay resolution MUST follow the precedence caller delay-override, then server pacing headers (response path only), then fixed delay, then exponential backoff, with the exception path skipping the header step. +- The stage stack's delay resolution MUST follow the precedence caller delay-override, then server pacing headers (response path only), then fixed delay, then exponential backoff, with the exception path skipping the header step (RETRY-39). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- A throwing user delay-override SHOULD be non-fatal, logging and falling back, while a throwing should-retry predicate SHOULD abort the call as a well-typed error, with fatal errors rethrown unchanged in both cases. +- A throwing user delay-override SHOULD be non-fatal, logging and falling back, while a throwing should-retry predicate SHOULD abort the call as a well-typed error, with fatal errors rethrown unchanged in both cases (RETRY-40). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- The stage stack MUST resolve the effective retry count as present-override-wins (validated non-negative), else the configured value, with a negative configured value clamped to the default and zero meaning no retries. +- The stage stack MUST resolve the effective retry count as present-override-wins (validated non-negative), else the configured value, with a negative configured value clamped to the default and zero meaning no retries (RETRY-41). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- All retry policy components MUST be immutable and stateless after construction and safe for concurrent invocation, with every piece of per-call state on the per-call stack/driver, never the shared instance. +- All retry policy components MUST be immutable and stateless after construction and safe for concurrent invocation, with every piece of per-call state on the per-call stack/driver, never the shared instance (RETRY-42 / RECOV-28). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- Each retry attempt MUST re-execute the downstream chain with fresh per-attempt continuation state rather than reusing the prior attempt's in-flight chain, and upstream steps MUST NOT mutate the shared in-flight request between attempts. +- Each retry attempt MUST re-execute the downstream chain with fresh per-attempt continuation state rather than reusing the prior attempt's in-flight chain, and upstream steps MUST NOT mutate the shared in-flight request between attempts (RETRY-44). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- The retry engine MUST NOT shut down or close a caller-supplied scheduler, and a process-wide default scheduler, when used, is likewise never shut down by the SDK. +- The retry engine MUST NOT shut down or close a caller-supplied scheduler, and a process-wide default scheduler, when used, is likewise never shut down by the SDK (RETRY-45). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- Retry-safety must be decided at the retry step independently of retryability and applied uniformly to protocol and transport failures, so a body-less request is retry-safe only if its method is idempotent (a bare POST is never retried even on a transport error) and a body-bearing request is retry-safe only if its body is replayable (a single-use/streaming body is never re-sent). +- Retry-safety must be decided at the retry step independently of retryability and applied uniformly to protocol and transport failures, so a body-less request is retry-safe only if its method is idempotent (a bare POST is never retried even on a transport error) and a body-bearing request is retry-safe only if its body is replayable (a single-use/streaming body is never re-sent) (XCUT-10). spec · `docs/product-spec/19-cross-cutting-invariants-and-policies.md:24` · high · sha:d6123be82c9e - The backoff calculator must apply overflow-safe saturation to the delay cap rather than throwing. design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:13-14` · high · sha:b0e2bb42d809 @@ -89,7 +89,7 @@ design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:33-36` · high · sha:b0e2bb42d809 ## Constraints -- The stage-based retry stack MUST NOT impose a total-timeout budget; a port that unifies the stacks MUST make the total-timeout an explicitly opt-in feature rather than always-on. +- The stage-based retry stack MUST NOT impose a total-timeout budget; a port that unifies the stacks MUST make the total-timeout an explicitly opt-in feature rather than always-on (RETRY-28). spec · `docs/product-spec/09-retry-and-resilience.md:27` · high · sha:9efbe276001e ## Conclusions @@ -97,7 +97,7 @@ spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:18-18` · high · sha:c2bf15dc8a06 - The body-less idempotency gate is retry-specific; the redirect path re-sends body-less requests per redirect semantics and does not consult idempotency (BODY-5). spec · `docs/product-spec/06-request-and-response-body-lifecycle.md:19-19` · high · sha:c2bf15dc8a06 -- 501 and 505 are excluded from the retryable status set because they mean the server cannot fulfill the request regardless of retry. +- 501 and 505 are excluded from the retryable status set because they mean the server cannot fulfill the request regardless of retry (RETRY-1). spec · `docs/product-spec/09-retry-and-resilience.md:9` · high · sha:9efbe276001e - The port single-sources the idempotent-method set, retryable-status set, and shared backoff calculator in one ES module because ES modules are singletons by default, unlike JVM classloaders which can each load their own copy of a class. design · `docs/sdk-design-nodejs/06-retry-redirect-and-authentication.md:1-8` · high · sha:b0e2bb42d809 @@ -113,25 +113,25 @@ spec · `docs/product-spec/09-retry-and-resilience.md:3` · high · sha:9efbe276001e - Retry happens only when both a retryable condition and a re-sendable request hold; the two axes are orthogonal. spec · `docs/product-spec/09-retry-and-resilience.md:7` · high · sha:9efbe276001e -- The idempotent-method set MUST be single-sourced and equal to {GET, HEAD, OPTIONS, PUT, DELETE}; POST and PATCH are re-sendable only via the replayable-body path. +- The idempotent-method set MUST be single-sourced and equal to {GET, HEAD, OPTIONS, PUT, DELETE}; POST and PATCH are re-sendable only via the replayable-body path (RETRY-6). spec · `docs/product-spec/09-retry-and-resilience.md:11` · high · sha:9efbe276001e -- Default retry tuning SHOULD be an initial delay of 200 ms, a multiplier of 2.0, a max delay of 8 s, a jitter of 0.2, and a budget of 3 sends. +- Default retry tuning SHOULD be an initial delay of 200 ms, a multiplier of 2.0, a max delay of 8 s, a jitter of 0.2, and a budget of 3 sends (RETRY-12). spec · `docs/product-spec/09-retry-and-resilience.md:16` · high · sha:9efbe276001e -- The recovery stack schedules the wake on a shared scheduler and blocks on the resulting future; the stage-sync stack performs an interruptible sleep that unmounts a virtual-thread carrier; async implementations schedule the delay without blocking a thread. +- The recovery stack schedules the wake on a shared scheduler and blocks on the resulting future; the stage-sync stack performs an interruptible sleep that unmounts a virtual-thread carrier; async implementations schedule the delay without blocking a thread (RETRY-26). spec · `docs/product-spec/09-retry-and-resilience.md:23` · high · sha:9efbe276001e -- The recovery stack's max-attempts default of 3 equals the stage stack's default max-retries of 2 plus one initial send. +- The recovery stack's max-attempts default of 3 equals the stage stack's default max-retries of 2 plus one initial send (RETRY-14). spec · `docs/product-spec/09-retry-and-resilience.md:28` · high · sha:9efbe276001e -- The recovery stack's configured retryable-status set is authoritative-contains, not an intersection with the baked-in classifier flag; a port should follow that. +- The recovery stack's configured retryable-status set is authoritative-contains, not an intersection with the baked-in classifier flag; a port should follow that (RETRY-37). spec · `docs/product-spec/09-retry-and-resilience.md:29` · high · sha:9efbe276001e -- Only the async retry stack currently implements the skip-self suppression guard in the reference implementation; a port MUST apply it to both stacks. +- Only the async retry stack currently implements the skip-self suppression guard in the reference implementation; a port MUST apply it to both stacks (RETRY-34). spec · `docs/product-spec/09-retry-and-resilience.md:30` · high · sha:9efbe276001e -- A fixed-delay configuration MAY force a flat delay disabling backoff and jitter, making the backoff path unreachable. +- A fixed-delay configuration MAY force a flat delay disabling backoff and jitter, making the backoff path unreachable (RETRY-43). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- An opt-in server-driven override MAY let a response header force or suppress the retry classification, flipping only classification and remaining subject to the attempt cap and the re-send-safety gate. +- An opt-in server-driven override MAY let a response header force or suppress the retry classification, flipping only classification and remaining subject to the attempt cap and the re-send-safety gate (RETRY-29). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- An optional per-attempt request header MAY stamp the 1-based attempt ordinal on a fresh per-attempt copy, never mutating the captured template and preserving any idempotency key, allocating nothing when disabled. +- An optional per-attempt request header MAY stamp the 1-based attempt ordinal on a fresh per-attempt copy, never mutating the captured template and preserving any idempotency key, allocating nothing when disabled (RETRY-38 / RECOV-31). spec · `docs/product-spec/09-retry-and-resilience.md:35` · high · sha:9efbe276001e -- A shared retryability classifier should treat HTTP status 408, 429, and all 5xx except 501/505 as retryable, and should treat a throwable as retryable iff it or any cause in its chain is an IO/timeout error (cause-chain traversal cycle-safe); where implemented, this exact status set is a hard contract. +- A shared retryability classifier should treat HTTP status 408, 429, and all 5xx except 501/505 as retryable, and should treat a throwable as retryable iff it or any cause in its chain is an IO/timeout error (cause-chain traversal cycle-safe); where implemented, this exact status set is a hard contract (CFG-35). spec · `docs/product-spec/16-configuration.md:58-58` · high · sha:367e27ec6481 - An idempotent method is an HTTP method whose repetition has the same effect as a single invocation; the SDK's idempotent set is `{GET, HEAD, OPTIONS, PUT, DELETE}`, used as the retry-safety gate for body-less requests. spec · `docs/product-spec/appendix-a-glossary.md:31` · high · sha:f0b3d2058626 diff --git a/docs/knowledge/sdk-positioning.md b/docs/knowledge/harvested/sdk-positioning.md similarity index 100% rename from docs/knowledge/sdk-positioning.md rename to docs/knowledge/harvested/sdk-positioning.md diff --git a/docs/knowledge/seams-and-extensibility.md b/docs/knowledge/harvested/seams-and-extensibility.md similarity index 98% rename from docs/knowledge/seams-and-extensibility.md rename to docs/knowledge/harvested/seams-and-extensibility.md index 1f54a67..83f39d7 100644 --- a/docs/knowledge/seams-and-extensibility.md +++ b/docs/knowledge/harvested/seams-and-extensibility.md @@ -5,7 +5,7 @@ spec · `docs/product-spec/01-product-overview.md:9-9` · high · sha:4f786c44354d - The byte-stream provider MUST expose factory operations to create a new empty in-memory buffer, a buffered reader over a raw input stream, a buffered reader over a byte array, a buffered writer over a raw output stream, and wrappers that add the buffered surface to a primitive source/sink (SEAM-3). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:7-7` · high · sha:0adae2d6a47f -- A reader/writer created over a caller's raw stream takes ownership of that stream, so closing the reader/writer closes the underlying stream (SEAM-3). +- A reader/writer created over a caller's raw stream takes ownership of that stream, so closing the reader/writer closes the underlying stream (SEAM-3 / IO-6). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:7-7` · high · sha:0adae2d6a47f - The synchronous transport MUST be a single-operation contract — given one request, produce one response — and MUST NOT pre-buffer the response body, leaving the caller to own reading and closing it (SEAM-11). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:12-12` · high · sha:0adae2d6a47f @@ -29,15 +29,15 @@ spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:25-25` · high · sha:0adae2d6a47f - Parametric deserialization targets MUST be expressible through a full generic type capture (SEAM-21). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:25-25` · high · sha:0adae2d6a47f -- Provider resolution MUST follow a fixed precedence: an explicitly installed provider always wins; otherwise the runtime auto-discovers providers registered on the classpath/plugin registry (SEAM-5). +- Provider resolution MUST follow a fixed precedence: an explicitly installed provider always wins; otherwise the runtime auto-discovers providers registered on the classpath/plugin registry (SEAM-5 / IO-33). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:34-34` · high · sha:0adae2d6a47f - Resolution MUST throw a descriptive error naming the install hint when zero providers are discoverable, and a descriptive error listing all candidates when more than one distinct provider is discoverable; exactly one discoverable provider is selected silently (SEAM-5). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:34-34` · high · sha:0adae2d6a47f -- Explicit installation MUST be idempotent for the same instance and MUST reject installing a different provider when one is already installed, naming both in the error (SEAM-6). +- Explicit installation MUST be idempotent for the same instance and MUST reject installing a different provider when one is already installed, naming both in the error (SEAM-6 / IO-32). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:35-35` · high · sha:0adae2d6a47f -- A successful auto-resolution MUST be cached process-wide (SEAM-7). +- A successful auto-resolution MUST be cached process-wide (SEAM-7 / IO-34). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:36-36` · high · sha:0adae2d6a47f -- When an explicit install replaces a different provider that had already been auto-resolved and handed out, the runtime SHOULD emit a warning rather than fail, because objects may already exist against the previous provider (SEAM-8). +- When an explicit install replaces a different provider that had already been auto-resolved and handed out, the runtime SHOULD emit a warning rather than fail, because objects may already exist against the previous provider (SEAM-8 / IO-35). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:38-38` · high · sha:0adae2d6a47f - The registry SHOULD tolerate one logical provider seen through more than one loader without misreporting it as multiple, de-duplicating by concrete implementation identity, and SHOULD recognize a thin delegating shim as its canonical target (SEAM-10). spec · `docs/product-spec/03-pluggable-seams-and-extension-model.md:39-39` · high · sha:0adae2d6a47f diff --git a/docs/knowledge/serde.md b/docs/knowledge/harvested/serde.md similarity index 98% rename from docs/knowledge/serde.md rename to docs/knowledge/harvested/serde.md index b3fd959..b47ac6c 100644 --- a/docs/knowledge/serde.md +++ b/docs/knowledge/harvested/serde.md @@ -5,9 +5,9 @@ spec · `docs/product-spec/14-serialization-serde.md:7-7` · high · sha:c6bc7789c3a9 - A Serde MUST declare the wire media type it produces, that media type MUST be used as the default Content-Type when a request body is created from a value plus a Serde, and the media type MUST NOT be defaulted to a format-agnostic constant at the SPI level. spec · `docs/product-spec/14-serialization-serde.md:8-8` · high · sha:c6bc7789c3a9 -- When encoding into or decoding from a caller-supplied stream, the serializer/deserializer MUST read/write the payload fully to EOF on the read side but MUST NOT close or take ownership of the caller's stream, and the encode-into-buffer profile likewise touches only the target region without assuming ownership. +- When encoding into or decoding from a caller-supplied stream, the serializer/deserializer MUST read/write the payload fully to EOF on the read side but MUST NOT close or take ownership of the caller's stream, and the encode-into-buffer profile likewise touches only the target region without assuming ownership (SEAM-20). spec · `docs/product-spec/14-serialization-serde.md:12-12` · high · sha:c6bc7789c3a9 -- The encode-into-buffer serialization profile MUST return the number of bytes written, MUST honor a start offset, MUST throw a range/overflow error distinct from the serde exception type when the offset is out of range or the payload does not fit, and MUST leave bytes before the offset untouched. +- The encode-into-buffer serialization profile MUST return the number of bytes written, MUST honor a start offset, MUST throw a range/overflow error distinct from the serde exception type when the offset is out of range or the payload does not fit, and MUST leave bytes before the offset untouched (SEAM-20). spec · `docs/product-spec/14-serialization-serde.md:13-13` · high · sha:c6bc7789c3a9 - Every decode operation MUST take an explicit runtime type witness for the target type, and a decoder MUST NOT rely on erased compile-time generics because on an erasure-based runtime that silently yields an untyped map/list which detonates as a cast error on first field access. spec · `docs/product-spec/14-serialization-serde.md:17-17` · high · sha:c6bc7789c3a9 @@ -15,11 +15,11 @@ spec · `docs/product-spec/14-serialization-serde.md:18-18` · high · sha:c6bc7789c3a9 - An ergonomic reified/inline decode helper, where the host language offers one, MUST capture the full generic type and route through the generic carrier rather than forwarding only the raw class. spec · `docs/product-spec/14-serialization-serde.md:19-19` · high · sha:c6bc7789c3a9 -- The generic type carrier MUST capture a concrete, fully-resolved type at construction and MUST reject construction with no type argument or an unresolved type variable, failing fast with an actionable message. +- The generic type carrier MUST capture a concrete, fully-resolved type at construction and MUST reject construction with no type argument or an unresolved type variable, failing fast with an actionable message (SEAM-22). spec · `docs/product-spec/14-serialization-serde.md:20-20` · high · sha:c6bc7789c3a9 -- Encode/decode failures MUST surface as the SDK's stable serde exception type or a subtype; adapters MUST catch the backing codec's processing failures and rethrow as the serde type, MUST chain the original as the cause, and MUST NOT allow a backing-library exception type to escape the SPI. +- Encode/decode failures MUST surface as the SDK's stable serde exception type or a subtype; adapters MUST catch the backing codec's processing failures and rethrow as the serde type, MUST chain the original as the cause, and MUST NOT allow a backing-library exception type to escape the SPI (SEAM-23). spec · `docs/product-spec/14-serialization-serde.md:24-24` · high · sha:c6bc7789c3a9 -- Write-path serde failures MUST be a serialization-specific subtype and read-path failures a deserialization-specific subtype, both of a common root exception type, so callers can distinguish direction while catching one base type. +- Write-path serde failures MUST be a serialization-specific subtype and read-path failures a deserialization-specific subtype, both of a common root exception type, so callers can distinguish direction while catching one base type (SEAM-23). spec · `docs/product-spec/14-serialization-serde.md:25-25` · high · sha:c6bc7789c3a9 - A genuine stream I/O error raised while reading/writing a caller-owned stream MUST propagate unwrapped as an I/O error and MUST NOT be re-wrapped as a serde exception; only malformed-input, shape-mismatch, or unencodable-value failures are wrapped. spec · `docs/product-spec/14-serialization-serde.md:27-27` · high · sha:c6bc7789c3a9 diff --git a/docs/knowledge/sse-streaming.md b/docs/knowledge/harvested/sse-streaming.md similarity index 99% rename from docs/knowledge/sse-streaming.md rename to docs/knowledge/harvested/sse-streaming.md index 54df65b..d833c9c 100644 --- a/docs/knowledge/sse-streaming.md +++ b/docs/knowledge/harvested/sse-streaming.md @@ -41,9 +41,9 @@ spec · `docs/product-spec/13-server-sent-events-and-streaming.md:37-37` · high · sha:dd401a407f5d - The SSE streaming facade MUST own exactly one closeable resource and MUST close it exactly once across the stream's whole life regardless of termination path (clean end, explicit close, use-block exit, partial consume, or mid-stream failure). spec · `docs/product-spec/13-server-sent-events-and-streaming.md:45-45` · high · sha:dd401a407f5d -- On reader end-of-stream during iteration, the SSE facade MUST both terminate the iterator cleanly and release the resource, so a fully-consumed stream needs no explicit close. +- On reader end-of-stream during iteration, the SSE facade MUST both terminate the iterator cleanly and release the resource, so a fully-consumed stream needs no explicit close (SSE-24). spec · `docs/product-spec/13-server-sent-events-and-streaming.md:46-46` · high · sha:dd401a407f5d -- A partial consume of the SSE stream MUST NOT strand the resource; closing after reading only some events MUST release it. +- A partial consume of the SSE stream MUST NOT strand the resource; closing after reading only some events MUST release it (SSE-25). spec · `docs/product-spec/13-server-sent-events-and-streaming.md:47-47` · high · sha:dd401a407f5d - The SSE streaming facade MUST be single-pass such that obtaining an iterator succeeds at most once, and a second attempt MUST fail loudly. spec · `docs/product-spec/13-server-sent-events-and-streaming.md:48-48` · high · sha:dd401a407f5d diff --git a/docs/knowledge/styleguide-overview.md b/docs/knowledge/harvested/styleguide-overview.md similarity index 100% rename from docs/knowledge/styleguide-overview.md rename to docs/knowledge/harvested/styleguide-overview.md diff --git a/docs/knowledge/testing.md b/docs/knowledge/harvested/testing.md similarity index 100% rename from docs/knowledge/testing.md rename to docs/knowledge/harvested/testing.md diff --git a/docs/knowledge/tooling-and-quality-gates.md b/docs/knowledge/harvested/tooling-and-quality-gates.md similarity index 91% rename from docs/knowledge/tooling-and-quality-gates.md rename to docs/knowledge/harvested/tooling-and-quality-gates.md index f646ee1..2e3e011 100644 --- a/docs/knowledge/tooling-and-quality-gates.md +++ b/docs/knowledge/harvested/tooling-and-quality-gates.md @@ -117,11 +117,11 @@ design · `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:37-41` · high · sha:2d2fd9dcfee4 ## Conflicts -- **design vs styleguide: package manager and lockfile** — RESOLVED in favor of the styleguide (2026-07-25, confirmed 2026-07-28 Phase 9 audit). The scaffold implements Bun (`bun.lock`, `.bun-version`, `bun install --frozen-lockfile` as the CI gate) throughout; the design's pnpm/`catalog:` framing describes a toolchain this repository does not use. Decision recorded at `docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md:54`; the enforcement properties pnpm's layout gave for free (isolated linker, workspace catalogs) were restored separately — see the Bun workspace catalogs adopted in Phase 6a and the isolated linker set at the 2026-07-25 checkpoint. - design `docs/sdk-design-nodejs/02-package-and-workspace-layout.md:50-51` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/01-formatting-and-tooling.md:112-120` · resolved 2026-07-25, backported 2026-07-28 -- **design vs styleguide: test runner and whether coverage gates the build** — RESOLVED as a split (2026-07-25, confirmed 2026-07-28 Phase 9 audit). Runner: `bun test` with `bun:test` symbol imports (the styleguide's choice) — the design's `c8`/`vitest` framing is dead. Gating: `NFR-5`/`NFR-17` are spec conformance obligations that outrank the styleguide's general "coverage is a trend, never a pass/fail gate" default; `bunfig.toml`'s `coverageThreshold = 0.8` blocks the build, as the scaffold's own plan already implemented. - design `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:12` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/11-testing.md:47-48,210-213` · resolved 2026-07-25, backported 2026-07-28 -- **design vs styleguide: gts as the lint and format baseline** — RESOLVED in favor of the styleguide (2026-07-25, confirmed 2026-07-28 Phase 9 audit). The plans extend `gts` in `eslint.config.js` and layer `@typescript-eslint`'s `strict-type-checked`/`stylistic-type-checked` tiers on top as the single permitted overlay, satisfying the design's rule set as well; the design's table never mentioning `gts` describes a toolchain this repository does not use. - design `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:8-10` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/01-formatting-and-tooling.md:66-83,122-129` · resolved 2026-07-25, backported 2026-07-28 +- **design vs styleguide: package manager and lockfile** — the design puts every version and tooling coordinate in one place via pnpm's `catalog:` protocol (pnpm ≥9), the direct analog of `gradle/libs.versions.toml` (**NFR-14**), and leans on pnpm's dedupe to keep one `@dexpace/core` instance; the styleguide mandates Bun — a committed `bun.lock`, `bun install --frozen-lockfile` as the CI gate, and a committed `.bun-version` — and is explicit that Bun's default flat `node_modules` buys reproducibility, not pnpm-style isolation. + design `docs/sdk-design-nodejs/02-package-and-workspace-layout.md:50-51` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/01-formatting-and-tooling.md:112-120` · resolved 2026-07-25 +- **design vs styleguide: test runner and whether coverage gates the build** — the design's gate table maps Kover's 80% aggregate line-coverage floor onto `c8`/`@vitest/coverage-v8` with a `coverage.thresholds` aggregate floor wired into the default `test` script; the styleguide makes `bun test` the runner and not a per-project choice, with `bun:test` symbols imported explicitly, and reports coverage as a floor and a trend, never as a pass/fail target. + design `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:12` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/11-testing.md:47-48,210-213` · resolved 2026-07-25 +- **design vs styleguide: gts as the lint and format baseline** — the design's gate table names ESLint with `@typescript-eslint`'s `strict-type-checked` and `stylistic-type-checked` configs and never mentions `gts`; the styleguide makes `gts` the whole toolchain, bans a standalone Prettier or ESLint config, and takes gts's Prettier defaults as final. + design `docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:8-10` · styleguide `/home/mohammad/Projects/dexpace/styleguide/typescript/01-formatting-and-tooling.md:66-83,122-129` · resolved 2026-07-25 ## Superseded diff --git a/docs/knowledge/transport-adapter.md b/docs/knowledge/harvested/transport-adapter.md similarity index 100% rename from docs/knowledge/transport-adapter.md rename to docs/knowledge/harvested/transport-adapter.md diff --git a/docs/knowledge/type-system.md b/docs/knowledge/harvested/type-system.md similarity index 100% rename from docs/knowledge/type-system.md rename to docs/knowledge/harvested/type-system.md diff --git a/docs/knowledge/typescript-idioms.md b/docs/knowledge/harvested/typescript-idioms.md similarity index 100% rename from docs/knowledge/typescript-idioms.md rename to docs/knowledge/harvested/typescript-idioms.md diff --git a/docs/knowledge/url-and-query-encoding.md b/docs/knowledge/harvested/url-and-query-encoding.md similarity index 100% rename from docs/knowledge/url-and-query-encoding.md rename to docs/knowledge/harvested/url-and-query-encoding.md diff --git a/docs/knowledge/variables-and-declarations.md b/docs/knowledge/harvested/variables-and-declarations.md similarity index 100% rename from docs/knowledge/variables-and-declarations.md rename to docs/knowledge/harvested/variables-and-declarations.md diff --git a/docs/knowledge/notes/data-modeling.md b/docs/knowledge/notes/data-modeling.md new file mode 100644 index 0000000..f7513e1 --- /dev/null +++ b/docs/knowledge/notes/data-modeling.md @@ -0,0 +1,13 @@ +# data-modeling — notes + +Hand-written. `docs/knowledge/harvested/data-modeling.md` is what the styleguide says; this file +records how this repository bounded one of its rules, and it wins. Each entry names the harvested +entry it answers by that entry's stable key. + +## Conflicts +- **`#private`-vs-`private`: the runtime-privacy carve-out is scoped to `packages/core/src/http/`, and that scoping is the settled answer rather than an unfinished sweep.** Resolves `http-domain-model/d26b9192`, the conflict statement left `unresolved 2026-07-25`, and bounds `data-modeling/2765e3ba` and `data-modeling/7ea87f23` rather than overturning either. `CLAUDE.md`'s "Domain model construction pattern" opens "Every model in `packages/core/src/http/` follows one shape" and mandates "`#private` fields only. Not TS `private`." *inside that shape*; the justification it cites is the styleguide's own carve-out for library internals that must stay unreachable **reflectively**, which is a claim about what a *consumer* can reach. Outside that directory the styleguide's default stands. The blanket reading the conflict entry worried about — "blanket across every model class, not per-use" — is not what shipped. + + **Measured 2026-09-04: 15 TS `private`/`protected` members against 75 `#private` fields.** Reproduce over `packages/*/src`, excluding tests and constructor parameters, with `grep -rnE '^\s+(private|protected)\s+(readonly\s+)?[a-zA-Z_]' --include='*.ts' packages/*/src | grep -v '\.test\.' | grep -v constructor` and `grep -rnE '^\s+#[a-zA-Z_]' --include='*.ts' packages/*/src | grep -v '\.test\.'`. `packages/core/src/http/` holds **none** of the 15 — it is at 100% compliance. The 15 are `NonceCountStore.counts` (`packages/core/src/auth/digest.ts`), `BearerTokenCache`'s six including its `startPostEviction` and `refresh` methods (`packages/core/src/auth/bearer-cache.ts`), and `CollisionWarningGate.warned` plus `RealLogEvent`'s seven (`packages/core/src/observability/logger.ts`). The first two classes are exported `@internal`; the last two are module-local and exported from nothing. None reaches the public barrel or any `packages/*/etc/*.api.md`, so no consumer can hold one to reflect on — which is the condition the carve-out is granted for, and it is absent. + + **Sweeping the 15 to `#private` was considered and rejected 2026-09-04.** It would churn `bearer-cache.ts`'s single-flight logic for no consumer-visible gain and would apply the carve-out past the reasoning that earns it. The deferral this had sat under since the Phase 4b validation review (F7, 2026-07-28) closes on that reasoning, not on a sweep, and its own summary was wrong: it called the residue "cosmetic (no per-class comment stating the justification)" when the residue was in fact 15 members using the other style entirely. Recording it here rather than in the register is what makes it survive the row's removal. + review · `CLAUDE.md` · high · sha:manual-2026-09-04-private-field-scope diff --git a/docs/knowledge/notes/deliberate-deviations.md b/docs/knowledge/notes/deliberate-deviations.md new file mode 100644 index 0000000..a5dde3d --- /dev/null +++ b/docs/knowledge/notes/deliberate-deviations.md @@ -0,0 +1,7 @@ +# deliberate-deviations — notes + +Hand-written. There is no harvested counterpart to this file, deliberately: see the entry below. + +## Reference +- **The deviation register is not harvested. Read the register itself, at the start of every phase.** `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` is a numbered ledger that each phase appends to, and a harvest of a ledger is a snapshot of one revision that goes stale on the next append. The snapshot this corpus used to carry proved the point: 13 entries against a 17-item register, roughly a third of it, mis-anchored, two entries substantively false, and pinned to a sha three revisions old. A description of an approach is harvestable because it changes slowly; a register is not. `docs/deviations.md` is the as-built audit of that same ledger, and `docs/open-items.md` is the second register under the same rule — read them, do not expect `bun run knowledge` to know them. + review · `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` · high · sha:manual-register-pointer diff --git a/docs/knowledge/notes/function-design.md b/docs/knowledge/notes/function-design.md new file mode 100644 index 0000000..fb32dc0 --- /dev/null +++ b/docs/knowledge/notes/function-design.md @@ -0,0 +1,13 @@ +# function-design — notes + +Hand-written. `docs/knowledge/harvested/function-design.md` is what the styleguide says; this file +records how this repository resolved a conflict inside it, and it wins. Each entry names the +harvested entry it answers by that entry's stable key. + +## Conflicts +- **The parameter-count threshold this repository enforces is the lint threshold, not the prose one: three positional parameters are allowed, four are an error.** Bounds `function-design/45a4ddba` ("a function must take an options object when it has 3 or more parameters") with `function-design/27da9d1f` ("positional parameters are capped by ESLint `max-params: ['error', 3]`"). The two are one parameter apart — the prose bans three, the enforcement bans four — and nothing in the corpus's `--section conflicts` reconciles them, which is why the Phase 4b validation review filed a row against the prose on 2026-07-28 and left it unowned. + + **Measured 2026-09-04: the repository follows the lint threshold, and has throughout.** `eslint.config.js` sets `max-params` to 3, so a three-parameter function is legal, and three-parameter functions ship across every subsystem — `Transport.send(request, options?, signal?)`, `fold(outcome, onSuccess, onFailure)`, `Deserializer.deserializeFrom(source, schema, typeName?)`, `redactHeaderValue(name, value, policy?)`. Where a fourth was genuinely wanted the code carries a documented `eslint-disable-next-line max-params` with a stated reason, which is the gate being *enforced* rather than evaded — the model builders' private constructors are the standing example. + + **Why the lint threshold wins, rather than the prose.** The boolean half of the prose rule is unaffected and still binds: any boolean parameter forces an options object regardless of count, and the control-flag conclusion at `docs/knowledge/harvested/function-design.md:44` sharpens it further. What is bounded is only the numeric threshold, and there the enforceable rule is the one a reviewer and a gate can agree on. A prose rule one notch stricter than its own enforcement produces exactly this: a repository that complies with the gate, a corpus that reads as if it does not, and a review that has to re-derive the answer every time. `docs/knowledge/harvested/api-design.md:14` states the neighbouring rule for *optional* parameters — collect them into an options object past two — and that one is followed independently; `DecodeTarget` exists because of it. + review · `eslint.config.js` · high · sha:manual-2026-09-04-max-params-threshold diff --git a/docs/knowledge/notes/pagination.md b/docs/knowledge/notes/pagination.md new file mode 100644 index 0000000..41b9ce3 --- /dev/null +++ b/docs/knowledge/notes/pagination.md @@ -0,0 +1,11 @@ +# pagination — notes + +Hand-written. `docs/knowledge/harvested/pagination.md` is what the documents say; this file is what +the implementation found, and it wins. Each entry names the harvested entry it answers by that +entry's stable key. + +## Superseded +- **Item-view close ordering: `PAGE-11` governs, and the `sdk-design-nodejs/07` §7.1 snippet does not.** Supersedes `pagination/81881061` (the Reference entry describing the snippet) and resolves the conflict statement `pagination/d108714e`. The item-level view copies the page's items, closes the page, and only then yields — never the snippet's `yield*` inside a `try` with `close()` in the `finally`. The standing tie-breaker applies: a normative MUST beats an illustrative snippet, and the cost is zero because `PAGE-2` guarantees materialized items survive close. Phase 6c implements copy-items → close → yield and wrote the erratum into `sdk-design-nodejs/07` §7.1. + + The reason this needed recording rather than silently correcting: **the conformance test is weaker than the requirement.** Appendix B's `PAGE-11` check ("take one item from a multi-item first page and stop; assert the first page's response was closed") *passes* under the snippet's ordering, because an early `break` drives `.return()` and therefore the `finally`. Following the design doc would have shipped a MUST violation the checklist could not catch. Phase 6c's `lifecycle.test.ts` adds the assertion appendix B does not make — that the close is observed *before* the first item is yielded. + review · `docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md` · high · sha:manual-6c-erratum diff --git a/docs/knowledge/notes/pipeline.md b/docs/knowledge/notes/pipeline.md new file mode 100644 index 0000000..e62c51f --- /dev/null +++ b/docs/knowledge/notes/pipeline.md @@ -0,0 +1,13 @@ +# pipeline — notes + +Hand-written. `docs/knowledge/harvested/pipeline.md` is what the design chapter and the styleguide +say; this file records what the implementation settled, and it wins. Each entry names the harvested +entry it answers by that entry's stable key. + +## Conflicts +- **The `Stage` ordering is a frozen constant object, not an `enum`, and that is settled by the compiler rather than by preference.** Resolves `pipeline/e66ace13`, the conflict statement left `unresolved 2026-07-25`, which weighed an `enum` for the pipeline `Stage` ordering against a union of string literals. Nothing was weighed in the end: this package compiles with `erasableSyntaxOnly`, which bans `enum` outright along with namespaces and constructor parameter properties, so the design chapter's `enum` option was never reachable. The port ships `Stage` as a union of string literals with `STAGE_ORDER` and `PILLAR_STAGES` as frozen constant objects beside it (`packages/core/src/pipeline/stage.ts`), all three `@public` and on the barrel (`packages/core/src/index.ts`). + + **Why this is a note and not a re-harvest.** The marker sits inside a harvested entry, and a hand edit there changes no `` sha, so the next harvest would regenerate or duplicate it — `docs/knowledge/README.md` is the contract. It is recorded here for the same reason `notes/data-modeling.md` records the `#private` scoping: the resolution outlives the register row that carried it (`docs/work/mvp/2026-09-04-open-items-dissolution.md` N3, whose first marker closed 2026-09-04 and whose second is this one). + + **A note on N3's own grep.** Phase 9's plan asks for `grep -rn "unresolved 2026-07-25" docs/knowledge/` to return empty. It will not, and resolving the markers makes it worse rather than better: a note that resolves a marker has to quote the marker string to name what it resolves, so each resolution adds a match. The check that means something is `bun run knowledge --topic pipeline`, where the harvested entry now prints `[overridden by notes/pipeline.md]`. + review · `packages/core/src/pipeline/stage.ts` · high · sha:manual-2026-09-04-stage-ordering-erasable-syntax diff --git a/docs/knowledge/notes/tooling-and-quality-gates.md b/docs/knowledge/notes/tooling-and-quality-gates.md new file mode 100644 index 0000000..fe156e0 --- /dev/null +++ b/docs/knowledge/notes/tooling-and-quality-gates.md @@ -0,0 +1,13 @@ +# tooling-and-quality-gates — notes + +Hand-written. `docs/knowledge/harvested/tooling-and-quality-gates.md` states each design-vs-styleguide +contradiction; this file records how the repository resolved it, and it wins. Each entry names the +harvested statement it resolves by that statement's stable key. + +## Conflicts +- **Package manager and lockfile: the styleguide wins.** Resolves `tooling-and-quality-gates/4f1a46a5`. The scaffold implements Bun throughout — `bun.lock`, `.bun-version`, and `bun install --frozen-lockfile` as the CI gate; the design's pnpm/`catalog:` framing describes a toolchain this repository does not use. The two enforcement properties pnpm's layout gave for free were restored separately: Bun workspace catalogs in Phase 6a, and the isolated linker at the 2026-07-25 checkpoint. Decided 2026-07-25, confirmed by the 2026-07-28 Phase 9 audit. + review · `docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md:54` · high · sha:manual-2026-07-25-package-manager +- **Test runner and coverage gating: a split decision.** Resolves `tooling-and-quality-gates/99637a28`. Runner: `bun test` with `bun:test` symbol imports, the styleguide's choice — the design's `c8`/`vitest` framing is dead. Gating: the styleguide's general "coverage is a trend, never a pass/fail gate" default loses here, because `NFR-5`/`NFR-17` are spec conformance obligations that outrank a general style default; `bunfig.toml`'s `coverageThreshold = 0.8` blocks the build. Decided 2026-07-25, confirmed by the 2026-07-28 Phase 9 audit. + review · `bunfig.toml` · high · sha:manual-2026-07-25-test-runner +- **gts as the lint and format baseline: the styleguide wins.** Resolves `tooling-and-quality-gates/90367d73`. `eslint.config.js` extends `gts` and layers `@typescript-eslint`'s `strict-type-checked` and `stylistic-type-checked` tiers on top as the single permitted overlay, which satisfies the design's rule set as well; the design's table never mentioning `gts` describes a toolchain this repository does not use. The corollary is load-bearing and easy to undo by accident: no root Prettier config, so `eslint.config.js` sources `gts/.prettierrc.json` itself. Decided 2026-07-25, confirmed by the 2026-07-28 Phase 9 audit. + review · `eslint.config.js` · high · sha:manual-2026-07-25-gts-baseline diff --git a/docs/open-items.md b/docs/open-items.md deleted file mode 100644 index cc9fcc7..0000000 --- a/docs/open-items.md +++ /dev/null @@ -1,191 +0,0 @@ -# Open Items - -Running register of everything known to be unmet, unverified, misreported, or deliberately deferred across the -implemented portion of this project. Reviewed state: **scaffold milestone** (committed, `0ebdc79`) and -**Phase 1 — Core HTTP Domain Model** (branch `2-phase-1-core-http-domain-model`, uncommitted at time of -review). Last reviewed **2026-07-30**. - -A requirement absent from this file is either satisfied or belongs to a phase that has not started. The point -of the file is that nothing is unmet *silently* — every gap below is either scheduled against a named phase or -awaiting a decision. - -**Status vocabulary** - -| Status | Meaning | -|---|---| -| **DECIDE** | Blocked on a human decision. Two or more defensible answers; picking one is the work. | -| **ACT** | Decision already made or obvious; the work is simply not done. | -| **SCHEDULED** | Deliberately deferred to a named phase. No action now; listed so it cannot be lost. | -| **WATCH** | Not a defect today. Becomes one when a stated trigger fires. | - ---- - -## A. Requirements unmet or misreported - -### A1 — HTTP-24: `charset` does not return null for an unknown encoding — **DECIDE** - -`product-spec/04` §4.4 conformance text: "`charset=utf-8` → UTF-8; `charset=bogus` → **null**; no charset → -null." Actual behavior: - -```ts -MediaType.parse('text/plain;charset=bogus').charset // → 'bogus', not undefined -``` - -`packages/core/src/http/media-type.ts` returns the parameter verbatim. There is no registry of recognized -encodings to resolve against, so "unknown" is not a state the current design can detect — the reference -contract presumably assumed a `Charset` type whose lookup can fail. - -The Phase 1 checklist marks HTTP-24 ✅ with no note, so the project currently *claims* conformance it does not -have. That is the actual defect; the behavior itself may well be the right call. - -Two ways out, both acceptable, but one must be chosen: -1. Resolve against a known-encoding set (e.g. `TextDecoder` probing or an explicit allow-list) and return - `undefined` for anything unrecognized. -2. Record a deliberate deviation in - `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, on the grounds that the - TypeScript port models charset as an opaque string and has no failing lookup to model "unknown" with. - -Either way: correct the checklist row, and add a test pinning the chosen behavior. The getter's TSDoc already -documents the current behavior honestly. - -### A2 — HTTP-22: the checklist describes an implementation that does not exist — **ACT** - -Phase 1 checklist, HTTP-22 row: `✅ | Task 7, HeaderName.of()'s static cache`. - -No such cache exists. The plan deliberately dropped interning (Task 7's `HeaderName` comment: "No interning: -HTTP-22 makes it a MAY, and an intern map keyed by caller-supplied names is exactly the unbounded, -process-lived, caller-influenced map XCUT-14's drain-to-cap rule forbids"), and -`packages/core/src/http/headers.ts` has no static map on `HeaderName`. - -The decision is right and the requirement is a MAY, so nothing about the code needs to change. The checklist -row is simply false and should read ⏳/N/A with the XCUT-14 reasoning, not ✅. - -### A3 — HTTP-11: `Response` exposes no range classification of its own — **DECIDE** - -`product-spec/04` §4.3: "Status MUST classify by range … **and a response MUST expose these derived from its -status**." `Response` carries only `status`; callers reach classification one hop away via -`response.status.isSuccess`. - -Defensible as satisfied — the classification *is* reachable and single-sourced on `Status`, and mirroring six -getters onto `Response` is pure surface duplication. But no one recorded that reading, so it is currently an -accident rather than a decision. Either add the delegating getters or write the interpretation into the -checklist row. - -### A4 — SEAM-1 is enforced narrowly relative to its conformance text — **ACT** - -`scripts/verify-seam-1.mjs` asserts `packages/core/package.json`'s `dependencies` is `{}`. The spec's -conformance clause is broader: "a dependency audit of the core module finds only the standard library plus the -compile-scope logging facade; **no transport/codec/stream symbol is referenced from core**." - -Blind spots today: `peerDependencies`, `optionalDependencies`, and `bundleDependencies` are unchecked, and -nothing inspects what the source actually imports. Low risk while core imports nothing but `URL`, but the gate -reads as stronger than it is. Cheap hardening: assert the other three dependency keys are absent-or-empty, and -add an import scan over `packages/core/src` allowing only relative specifiers and `node:`-prefixed builtins. - ---- - -## B. Gates and tooling - -### B1 — NFR-10 / NFR-17: CI never runs on the declared minimum runtime — **ACT** (trigger has now fired) - -The scaffold checklist deferred this explicitly: *"recommend adding an `actions/setup-node@v4` step pinned to -`18.17` running `scripts/verify-dual-consumption.mjs` once real Node-API usage lands (Phase 1 onward), rather -than adding it now for a function that touches no runtime API."* - -**That trigger has fired.** Phase 1 uses the native `URL` class, `Object.freeze`, class `static {}` blocks, and -`#private` fields — all real runtime surface. `verify:runtime-floor` checks that `engines.node` and the -compiled language level *agree*, but nothing ever executes the artifact on Node 18.17; CI runs whatever the -GitHub Actions runner defaults to. The half of NFR-10 that catches "we shipped syntax the declared floor cannot -run" is still missing. - -### B2 — NFR-13: SPDX headers missing on scaffold-era files — **ACT** - -Phase 1 established the convention ("every new source file opens with `// SPDX-License-Identifier: MIT` on line -1") and every file under `packages/core/src/http/` complies. Three files predating it do not: - -- `scripts/verify-runtime-floor.mjs` -- `scripts/verify-seam-1.mjs` -- `eslint.config.js` - -`scripts/verify-dual-consumption.mjs` gained one during Phase 1, which is what makes the omission of its two -siblings look accidental rather than scoped. NFR-13 is a review convention, not a mechanical gate, so this is a -one-line-per-file cleanup. - -### B3 — NFR-12: reproducible builds asserted, never proven — **WATCH** - -`bun install --frozen-lockfile` plus plain `tsc` are deterministic by construction, but nothing demonstrates -it. Becomes real at first publish (~Phase 10): build twice, diff artifact digests. - -### B4 — NFR-14: `expect-type` breaks the single-source-of-versions convention — **WATCH** - -Every other devDependency is centralized at the workspace root; Phase 1 added `expect-type` to -`packages/core/package.json`'s own `devDependencies`. Harmless with one package — it is exactly the restatement -NFR-14 warns about once a second package exists (Phase 8). Either hoist it to the root now or fold it into the -NFR-14 decision at Phase 8. - ---- - -## C. Documentation defects - -### C1 — Phase 1's scope statement contradicts its own plan — **ACT** - -`docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md` says the scope is "Full -`product-spec/04-core-http-domain-model.md` (HTTP-3 through HTTP-53, both MUST and SHOULD level) in one phase." - -The plan's own Self-Review then amends that: *"The Phase 1 spec's scope statement should be read — and amended -— as HTTP-3..35, 46..50, 53"*, with the body-lifecycle cluster deferred to Phase 3b. The amendment was never -applied to the design doc, so read literally the two documents disagree about what Phase 1 owed. Correct the -design doc's scope line to match the plan. - -### C2 — The structural-typing bypass deviation is not yet recorded — **SCHEDULED** (Phase 10) - -The Phase 1 design doc acknowledges that `#private` fields close the *accidental* structural-typing bypass but -not deliberate reflection abuse (`Object.create(Request.prototype)`), and states this is "to be listed in -`sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` when that phase is reached." Listed -here so the promise survives until then. - ---- - -## D. Scheduled deferrals - -No action now. Each is already owned by a named phase; this table exists so none can quietly lapse. - -| Item | Requirement | Owner phase | Note | -|---|---|---|---| -| Body lifecycle: write/replayability, single-use, close, charset | HTTP-36 – HTTP-43 | 3b | `Request`/`Response` `body` is typed `unknown` as an explicit placeholder | -| Lazy `TypedResponse` with parse-once memoization | HTTP-44, HTTP-45 | 3b | | -| `MultipartBody` — the one builder-based model HTTP-3 lists that Phase 1 did not build | HTTP-51 | 3b | Depends on body-lifecycle contracts | -| 1 MiB error-body buffering cap | HTTP-52 | 3b | | -| `Request.equals` compares body by reference, not by value | HTTP-46 (body clause) | 3b | Blocked on a real `Body` model supplying value equality | -| `RequestConditions.applyTo` cannot emit an obs-text ETag | HTTP-18 vs HTTP-48/50 | 10 | Spec text in scope does not resolve the tension; strict outbound path kept rather than guessed. Documented in `applyTo`'s TSDoc | -| Seam contracts (byte-stream, transport, codec, projection) | SEAM-2 – SEAM-30 | 2–8 | | -| Adapter packages, peer-dependency dedup | NFR-2 | 8 | | -| Shrink-survival regression guard | NFR-9 | 9 | | -| Concurrency-model agnosticism check | NFR-11 | 4 | No async code exists yet | -| Self-identifying version metadata (real `User-Agent`) | NFR-15 | 7/8 | | -| Publish + provenance CI job | NFR-16 | release | `prepublishOnly` wired; nothing published yet | -| NFR-8 re-confirmed as a documented non-applicability | NFR-8 | 10 | No reflection-driven discovery surface exists by design | - ---- - -## E. Process - -### E1 — Phase 1 has no commits — **DECIDE** - -`git log main..HEAD` shows only the scaffold commit. The Phase 1 plan specifies a commit after each of its 15 -tasks (`feat(core): add Status value type (HTTP-10/11/12)`, and so on); all ~40 files currently sit in the -index and working tree as one undifferentiated change. - -Not a correctness problem — every gate passes. But the per-task history the plan describes cannot be -reconstructed after the fact, and a single 3,300-line commit is materially harder to review or bisect. Decide -whether to reconstruct the task-by-task sequence before merging or to accept one squashed commit and note the -departure. - ---- - -## Maintaining this file - -Add an entry the moment a gap is found, not when it is fixed — the failure mode this file prevents is a -checklist row marked ✅ against code that does not implement it (A1, A2 are both instances). Remove an entry -only when the underlying requirement is genuinely satisfied *and* its checklist row agrees. When a phase -closes, re-scan its checklist against the code rather than trusting the marks. diff --git a/docs/sdk-design-nodejs/02-package-and-workspace-layout.md b/docs/sdk-design-nodejs/02-package-and-workspace-layout.md index f3bc234..bcaf7f7 100644 --- a/docs/sdk-design-nodejs/02-package-and-workspace-layout.md +++ b/docs/sdk-design-nodejs/02-package-and-workspace-layout.md @@ -7,10 +7,13 @@ of Gradle's multi-module build graph. | Package | Purpose | Runtime floor | Dependencies | |---|---|---|---| -| `@dexpace/core` | Domain model, I/O contracts (built directly on Web Streams, not pluggable — see §3.1), execution context, both pipeline layers, retry/redirect/auth, pagination, SSE parsing, the serde SPI + `Tristate`, the instrumentation SPI, configuration. | Any runtime with Web Streams, `fetch`-shaped `AbortSignal`, and `globalThis.crypto.subtle` (Node ≥18.17, current evergreen browsers, Deno, Bun, Cloudflare Workers). | none | +| `@dexpace/core` | Domain model, I/O contracts (built directly on Web Streams, not pluggable — see §3.1), execution context, both pipeline layers, retry/redirect/auth, pagination, SSE parsing, the serde SPI + `Tristate`, the instrumentation SPI, configuration. | Any runtime with Web Streams, `fetch`-shaped `AbortSignal`, and `globalThis.crypto` (Node ≥20.3, current evergreen browsers, Deno, Bun, Cloudflare Workers). **Node ≥18.17 was the claim until 2026-08-26 and it was wrong twice over:** Node exposes `globalThis.crypto` unflagged only from 19.0.0 and never to an ES module on any 18.x release, and `AbortSignal.any()` reached the 20.x line in 20.3.0. | none | | `@dexpace/codec-json` | Reference wire codec: `JSON.parse`/`JSON.stringify` plus `Tristate` wiring and Standard-Schema decode glue (§7.3). | same as core | none beyond a `@dexpace/core` peer | -| `@dexpace/transport-fetch` | Minimal transport built on the global `fetch`. The zero-dependency, built-into-the-runtime option — the Node analog of `sdk-transport-jdkhttp`'s "no extra library, but less low-level control" trade-off. | same as core | none beyond a `@dexpace/core` peer | -| `@dexpace/transport-undici` | Full-featured transport built on `undici`'s `Client`/`Pool`/`request()` API: connection-pool tuning, trailers, explicit socket-level cancellation. The Node analog of `sdk-transport-okhttp`'s "richer, but pulls in a real library" trade-off. | Node only | `undici` | +| `@dexpace/transport-fetch` | Minimal transport built on the global `fetch`. The zero-dependency, built-into-the-runtime option — the Node analog of `sdk-transport-jdkhttp`'s "no extra library, but less low-level control" trade-off. | Node/Bun. Its dependency list would run anywhere, but `redirect: 'manual'` — **TRANSPORT-1**'s mechanism — returns the raw 3xx only on an `undici`-backed runtime; a browser returns an opaque-redirect response (status `0`, no headers) the pipeline cannot redirect with. | none beyond a `@dexpace/core` peer | +| `@dexpace/transport-undici` | Full-featured transport built on `undici`'s `Dispatcher`/`request()` API: connection-pool tuning, proxy routing, ownership-aware close, explicit socket-level cancellation. The Node analog of `sdk-transport-okhttp`'s "richer, but pulls in a real library" trade-off. The owned dispatcher is an `Agent`, not a `Pool` — a `Pool` is bound to one origin at construction, and a general-purpose transport must reach whatever origin each `Request` names (Phase 8a design §4). | Node only | `undici` | +| `@dexpace/body-file` | The concrete `fileBody()` factory: a file-backed request `Body` with fail-fast `node:fs` construction validation. Cannot live in core (zero-`node:`-import invariant); is **not** an upstream of either transport, which recognize it structurally through `@dexpace/core`'s type-only `FileBodyDescriptor` and a `body.kind === 'file'` check (Phase 8a design §5). | Node only | none beyond a `@dexpace/core` peer | +| `@dexpace/transport-shared` | Internal plumbing both transports need identically — header drop/degrade, drop-log dedup, abort→SDK-error mapping, request-body pumping, and the delivery-detached signal fork. `@internal` exports only; published because `NFR-4` snapshots every published unit and because a transport's `dependencies` must resolve for consumers. Exists so neither transport has to depend on its sibling (Phase 8a design §7). | same as core | none beyond a `@dexpace/core` peer | +| `@dexpace/transport-conformance` | Unpublished. The one `TRANSPORT-N` conformance suite plus its `node:http` fixture server, run once per transport package so the two adapters cannot drift (Phase 8a design §8). | — | dev-only | | `@dexpace/logging-pino` | Bridges the core `Logger` seam to a caller-supplied `pino` instance. | Node/any pino-compatible runtime | `pino` (peer) | | `@dexpace/logging-debug` | Bridges the core `Logger` seam to the ubiquitous zero-config `debug` package, for consumers who want a logger with no configuration story at all. | any | `debug` (peer) | | `@dexpace/rx` | Thin optional sugar exposing pagination and SSE as RxJS `Observable`s for teams already standardized on RxJS (notably Angular shops). Not a bridge for the request/response pivot itself — see §3.2. | any | `rxjs` (peer) | diff --git a/docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md b/docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md index df0fb6a..7fd7b41 100644 --- a/docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md +++ b/docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md @@ -27,6 +27,10 @@ async function* items(): AsyncGenerator { } ``` +> [!NOTE] +> **Erratum on close ordering (PAGE-11 vs illustrative snippet):** The snippet above illustrates JavaScript's automatic `.return()`-on-abandon via `finally`, but closes *after* yielding items. **PAGE-11** (MUST) mandates closing *before* yielding any items on the page (`const items = page.items; await page.close(); yield* items;`). Materialized items survive close (**PAGE-2**), so closing before yielding releases the underlying response immediately and ensures an abandoned item iteration cannot strand an open response. + + and an early `break` out of the consumer's `for await` loop drives the `finally` — and therefore `page.close()` — automatically, with no wrapper type and no documented "must remember to close" convention required from callers. The page-level view's two-outstanding-pages buffering (**PAGE-12**: a `hasNext()` probe eagerly runs the next diff --git a/docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md b/docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md index f18ba82..06a46fc 100644 --- a/docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md +++ b/docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md @@ -46,7 +46,7 @@ than the artifact's declared floor, producing a symbol reference that link-check fails at call time on an older runtime (`NoSuchMethodError` on the JVM; a plain `TypeError: X is not a function` in Node). The TypeScript-specific version of this trap is a `tsconfig.json` `lib` setting newer than the package's declared `engines.node` floor — for instance, `lib: ["ES2023"]` type-checks a call to -`Array.prototype.toSorted` cleanly while `engines.node: ">=18.17"` promises a runtime that does not have it, +`Array.prototype.toSorted` cleanly while `engines.node: ">=18.17"` promised a runtime that does not have it, producing exactly the same class of silent, deferred-to-call-time failure the JVM side already learned to guard against. Each package's `tsconfig` `lib`/`target` must be pinned to match its own declared `engines.node` floor, not inherited loosely from whatever the workspace root happens to use for editor tooling, and CI should run the built diff --git a/docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md b/docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md index cb4580f..3def142 100644 --- a/docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md +++ b/docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md @@ -7,6 +7,23 @@ MUST-level correctness guarantee; each is a case where the JVM-specific mechanis does not exist in Node, and an equivalent, differently-shaped mechanism is substituted instead. Reconciled by Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md`), 2026-07-28. +**Three files carry "deviations" in their name. They are not interchangeable** (cross-reference added +2026-08-30, after the first of them was corrected against source without the other two being touched): + +| File | What it is | Numbering | +|---|---|---| +| **This section (§10)** | The **normative ledger** of deliberate deviations — the canonical, as-built list. Every item's number is the one the other files cite. | Owns items 1-17 | +| `docs/deviations.md` | The **as-built audit** of this ledger, performed against source rather than against the phase specs that produced it. Carries the `file:line` evidence for each item, and the record of which items this ledger got wrong. Restates §10's item numbers; it does not assign its own. | Follows §10's | +| `docs/knowledge/notes/deliberate-deviations.md` | Neither. A one-entry **pointer** in the knowledge corpus saying that this register is not harvested and must be read here. The harvested copy that used to sit at `docs/knowledge/deliberate-deviations.md` was dropped on 2026-08-31: it held about a third of this section, at a three-revision-old sha, with two entries substantively false. A register accumulates rows; a harvest of one is a snapshot that goes stale on the next append. | None | + +**Renumbering this section renumbers `docs/deviations.md`.** Its section headings and its two summary tables are +keyed to the numbers above, with no independent identity to fall back on; change one and the other must change in +the same commit. + +**Where a *new* deviation is recorded:** in the owning phase spec's own `## Deviation Ledger (for Phase 10)` +section, never here directly. This section is Phase 10's **output** — the consolidation of those per-phase +ledgers — not their intake. + 1. **Single execution model eliminates every thread/CAS/interrupt-flag primitive, and collapses the sync/async transport seam into one.** **SEAM-11** describes a synchronous, blocking transport contract as distinct from **SEAM-16**'s asynchronous one; Node has no blocking-I/O execution model to give that distinction meaning, so @@ -47,9 +64,15 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de 4. **True runtime encapsulation of domain models is not fully achievable.** ECMAScript `#private` fields close the "official construction path" hole **HTTP-2**/**SEAM-29** care about, but TypeScript's structural typing means a hand-built object literal can still impersonate a public interface type and bypass builder validation entirely. - This is an acknowledged, language-level limitation, not an oversight; the mitigation — exporting only concrete - classes, never bare structural interfaces, from each package's public entry point — narrows but does not - eliminate the gap (Phase 1). + This is an acknowledged, language-level limitation, not an oversight; the mitigation — exporting the + **`http/` wire-model types** as concrete classes rather than bare structural interfaces — narrows but does not + eliminate the gap (Phase 1). *Corrected 2026-08-29: the mitigation previously read "exporting only concrete + classes, never bare structural interfaces, from each package's public entry point", which the API report + contradicts — `packages/core/etc/core.api.md` exports 61 interfaces against 58 classes. Most are seams + (`Transport`, `Serde`, `Logger`) or options records, where structural typing is the point and no builder + validation is being bypassed. At least one is not: `Configuration` is a builder-built, frozen type exported as + a bare interface and accepted structurally by `setGlobalConfiguration()` and `resolveProxyOptions()`. The + mitigation is real for the domain models it was written about; it is not a package-wide property.* 5. **Schema-as-witness replaces reflective generic-type capture, and the codec-configuration surface it would have carried does not exist.** **SERDE-5**-**SERDE-8**'s mechanism (a reflectively-reconstructed type token) has no TypeScript equivalent — TypeScript erases types more completely than JVM generics erasure, leaving no raw class @@ -63,9 +86,18 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de portable across non-Node runtimes deliberately excludes MD5. The port vendors a small, dependency-free MD5 implementation for RFC 7616 interoperability and uses `crypto.subtle` directly for SHA-256/SHA-256-sess (Phase 5c). -7. **Configuration layering has three tiers, not four.** **CFG-1**'s override → environment → system-property → - default chain loses its system-property tier outright; Node has no ambient key/value store distinct from - environment variables to fill that slot, and the port does not fabricate one (Phase 7a). +7. **Configuration keeps all four layering tiers, but the platform supplies nothing to bind the third to.** + **CFG-1**'s override → environment → system-property → default chain is implemented in full: `getString` + resolves an exact-key override, then the environment source under the exact key, then the *property* source + under **CFG-3**'s normalized key (lower-cased, `_` → `.`), then the caller's default. The property layer is a + first-class, caller-supplyable `SourceFn` seam — `ConfigurationBuilder.withPropertySource()` and + `getRawProperty()` are both public API — so a host that *does* have an ambient key/value store can bind it. + What deviates is only the **default production wiring**: `defaultConfiguration()` binds a property source that + always returns `undefined`, because Node has no ambient store distinct from `process.env`, and routing a + synthetic "system property" back through `process.env` under a different key would invent a layer the platform + does not have (Phase 7a). *Corrected 2026-08-29: this entry previously read "three tiers, not four — the + system-property tier is lost outright", which understated the as-built code. The tier exists and is + substitutable; only its default binding is empty.* 8. **Cancellation is `AbortController`/`AbortSignal` end-to-end, not "interrupt-and-restore-a-flag."** Every cancellable operation in the port — the transport call, the retry backoff wait, a derived per-call timeout — composes the same signal type. `Promise` has no public `cancel()` unlike `CompletableFuture`; cancellation is @@ -86,13 +118,32 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de keep/retain configuration) is not applicable by design, full stop — this port has no reflection-driven discovery surface to keep-configure at all, the same discovery machinery Item 2 above already retired. This closes the item permanently rather than leaving it re-flagged for a future phase. -11. **`Symbol.asyncDispose` is adopted opportunistically, not uniformly, and this is deliberate, not drift.** - Internal `io/` primitives ship `close()` only — the symbol postdates the package's declared `>=18.17` Node - floor, and these types are `@internal` and never surface to a consumer who'd use the ergonomic disposal syntax - (Phase 3a). Public, consumer-facing disposable resources added in later phases — `Body`/`Response` (Phase 3b), - `SseStream` (Phase 6b), `Page` (Phase 6c) — each add `[Symbol.asyncDispose]` as optional and runtime-guarded - rather than declaring `implements AsyncDisposable`, so the type works whether or not the running Node version - supports the symbol, without raising the package's declared floor. Confirmed consistent across all four sites. +11. **`Symbol.asyncDispose` is adopted opportunistically, not uniformly, and every install is runtime-guarded.** + The symbol postdates the packages' declared Node floor (`>=20.3` since 2026-08-26; on the 20.x line the + symbol arrives in 20.4.0), so nothing may declare it as a plain class member: on the floor the computed key + evaluates to `undefined` and binds the method to the string key `"undefined"`, leaving a junk prototype entry + and no working disposal, while the emitted `.d.ts` promises `AsyncDisposable` unconditionally. The port + therefore runs a **two-tier policy**, verified against the code 2026-08-29: + - **`close()` only, no disposal member at all.** Internal `io/` primitives — `@internal`, never surfaced to a + consumer who would use the ergonomic syntax (Phase 3a). Also `Body`/`Response` (Phase 3b), which are + *public* but deliberately teardown-by-`close()`; `http/response.test.ts` and + `body/response-body-logging.test.ts` each pin the **absence** of the `"undefined"` key, and are the origin + of the rule the other tier follows. + - **Guarded runtime install.** `SseStream` (Phase 6b), `Page` (Phase 6c), and both transports — + `FetchTransport` and `UndiciTransport` (Phase 8a) — install `[Symbol.asyncDispose]` via + `Object.defineProperty` behind `typeof Symbol.asyncDispose === 'symbol'`. None declares + `implements AsyncDisposable` and none emits the member into its `.d.ts`, so nothing promises a consumer on + the floor a method that is not there. + + **This entry previously misdescribed the code on three counts and is corrected here rather than restated.** + It claimed `Body`/`Response` add the member (they never have, and two tests assert they do not); it claimed + all sites were "optional and runtime-guarded" (only `SseStream` was — `Page`, `FetchTransport`, and + `UndiciTransport` each declared a plain class member *and* `implements AsyncDisposable`, and both transport + factories publicly returned `Transport & AsyncDisposable`); and it omitted the two transport sites entirely + while asserting consistency "across all four sites". The three unguarded sites were repaired on 2026-08-29 — + see the changeset `2026-08-29-guard-symbol-asyncdispose-installs.md`. The type-system cost of keeping the + floor at `>=20.3` is that `await using` does not type-check against these types; `close()` is the supported + teardown path, and raising the floor to `>=20.4` in a later release would restore the declaration honestly. 12. **The redirect/auth cross-origin marker is a real header, not a `WeakSet`, and its two interpretive questions are now settled by Phase 10 directly, not by a Phase 9 conformance sweep that was never going to run them.** An earlier `WeakSet` design was rejected mid-draft: it breaks once retry's attempt-stamping sits @@ -128,15 +179,35 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de path. Neither transport retries a partial send internally (**TRANSPORT-18**); the SDK's own retry layer handles it via the replayability gate instead. `Response.protocol` is a hardcoded `HTTP_1_1` best-effort default because neither `fetch`'s `Response` nor undici's `ResponseData` surface the negotiated protocol - version (all: Phase 8a). -14. **Reproducible builds and publish provenance stay open, unblocking only at first real release.** **NFR-12** - (byte-identical builds from identical source) and **NFR-16** (publish provenance enforced on the release path) - are soft gaps: `bun install --frozen-lockfile` and plain `tsc` are deterministic by construction, and - `prepublishOnly` + `npm publish --provenance` are scripted (Phase 0 Task 3), but neither has been exercised — - no build artifact or real publish exists yet. Phase 10 does not manufacture a false close here: **NFR-12** - unblocks when the workspace is built twice and the output digests diffed identical; **NFR-16** unblocks when - the scripted publish path actually runs against a real registry. Both remain open, target "first real - release." + version (all: Phase 8a). Phase 8a's implementation added one more, found only by building it: **a custom + proxy `challengeHandler` cannot be dispatched by `transport-undici` either** — undici's `ProxyAgent` takes + its credential solely from its own constructor and rejects any per-request `Proxy-Authorization` with + `InvalidArgumentError` (a deliberate security fix on their side), and the constructor runs before any + challenge has been seen, so no handler-minted credential can reach the exchange that provoked it. This is + the case **TRANSPORT-30**'s own text anticipates: the handler is surfaced with a WARN at construction and + again on the first real `407`, proxy auth falls back to Basic (`ProxyOptions.credentials`, which *is* + passed to the `ProxyAgent` constructor), the `407` reaches the caller untouched, and a per-request + `Proxy-Authorization` is dropped from the outbound pass — logged by name like any other drop — rather than + turning every proxied send into a hard failure. The Phase 8a *plan* had specified a retry-with-stamped- + credential flow instead; that flow is not implementable on this platform (Phase 8a). +14. **`NFR-12` is closed on evidence; `NFR-16` alone stays open until first real release.** These two were + recorded together as soft gaps that "cannot be verified without a real artifact" — true while the repository + was docs-only, and no longer true for the first of them once Phases 1-9 shipped code. They are now separated: + - **NFR-12 (byte-identical builds) — closed 2026-08-29, verified.** Two clean builds of an identical source + tree (every `dist/` and `*.tsbuildinfo` swept between them) produce **644 emitted files, byte-identical**; + `npm pack` of `@dexpace/core` twice produces an identical tarball digest. The check is now a blocking CI + step rather than an assertion — `bun run verify:reproducible-build` + (`scripts/verify-reproducible-build.mjs`), which sweeps, builds twice, and diffs a SHA-256 per emitted + file. It was negative-tested by injecting a `Date.now()` into the one build-time codegen step + (`packages/core/scripts/gen-version.mjs`) and confirming it fails naming the offending file. + - **NFR-16 (publish provenance) — still open, target "first real release."** Its conformance test is + behavioral ("a CI/release build fails an unsigned publication; a local build without keys still publishes + unsigned") and needs a real registry and a real OIDC token. *Corrected 2026-08-29: this entry previously + claimed `npm publish --provenance` was "scripted (Phase 0 Task 3)". It is not — the string appears in no + `package.json`, no workflow, and no `.npmrc`; there is no `.npmrc` and no release workflow at all + (`.github/workflows/` holds `ci.yml` only). Only `prepublishOnly` is wired, exactly as + `docs/open-items.md`'s own row has always said. Authoring the release workflow with `--provenance` and + `id-token: write` is actionable **now** and is the unblocking work; only exercising it needs the registry.* 15. **A server-issued ETag containing obs-text does not round-trip through a conditional request, by deliberate choice.** `RequestConditions.applyTo` writes entity tags through `Headers`' outbound `set`, which enforces **HTTP-18**'s MUST-level restriction (HTAB plus printable ASCII 0x20-0x7E only, rejecting any byte ≥ 0x80). @@ -155,3 +226,13 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de different data shape — push-based `Observable`s — not plumbing for the request/response pivot; its `sseEvents$`/`typedSse$` are single-subscription, not standard cold/repeatable Observables, because `SseStream` wraps an already-consumed-once HTTP response body (Phase 8b). +17. **`TransportFailureError` adds a third level to an error tree the styleguide caps at two.** The + styleguide holds custom error hierarchies to two levels deep, and Phase 3a flattened this very tree to + obey it — the four I/O leaves extend `DexpaceError` directly, and `isIoError` exists to group them + without reintroducing a middle tier (`packages/core/src/io/errors.ts`). Phase 8a's **TRANSPORT-20** + reintroduces one: `TransportFailureError extends IoError extends DexpaceError`. The subtyping *is* the + requirement rather than an accident of modelling — `classify.ts`'s cause-walk returns `true` for every + `IoError`, so extending it is what makes a no-response failure retryable with no edit to the retry + layer, and a flat sibling would have to be named there by hand and again for every transport added + later. One level of depth buys the canonical-subtype clause. Held at exactly three: a fourth level is + not sanctioned by this row (Phase 8a). diff --git a/docs/sdk-documentation/architecture.md b/docs/sdk-documentation/architecture.md new file mode 100644 index 0000000..7b923d2 --- /dev/null +++ b/docs/sdk-documentation/architecture.md @@ -0,0 +1,194 @@ +# Architecture + +**Start here.** This tree documents the code that exists, package by package and seam by seam. It is +the front door for the other ten files: + +| File | Covers | +|---|---| +| [`http.md`](./http.md) | The domain model: `Request`, `Response`, `Headers`, `Status`, `QueryParams`, and their kin | +| [`bodies.md`](./bodies.md) | Request bodies as producers, response bodies as owned resources | +| [`pipelines.md`](./pipelines.md) | Stages, steps, the four pillars, `standardResilience()` | +| [`auth.md`](./auth.md) | Tiers, credentials, schemes, challenges, and the HTTPS guard | +| [`errors.md`](./errors.md) | The error tree, and which failure means what | +| [`quality-gates.md`](./quality-gates.md) | Every blocking gate, what it protects, how to run it | +| [`write-a-transport.md`](./write-a-transport.md) | Implementing `Transport`, and proving it | +| [`write-a-serde.md`](./write-a-serde.md) | Implementing `Serde` | +| [`write-a-paging-strategy.md`](./write-a-paging-strategy.md) | Implementing `PaginationStrategy` | +| [`write-a-response-handler.md`](./write-a-response-handler.md) | Turning a `Response` into your model | + +## What this tree is not + +It is not the API reference. Every exported symbol's signature is in +`packages/*/etc/*.api.md`, regenerated by `api:local` and diffed in CI by `bun run api`; what each +symbol *means*, including every `@throws`, is in the TSDoc, which ships in the emitted `.d.ts` and +appears on hover. Both are generated and gate-verified. A third hand-written copy would drift, and +the harvested styleguide's rule is one authoritative place per fact +(`docs/knowledge/harvested/documentation.md:32`). + +What those two cannot express is what this tree holds: **how the packages compose, which one to +install for which job, and worked examples that cross a package boundary.** + +It is also not the specification. `docs/product-spec/` is normative and numbered; every `HTTP-N`, +`SEAM-N`, `RETRY-N`, `PAGE-N` identifier here is an entry there. Where the port deliberately differs, +`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` is the ledger and +`docs/deviations.md` the as-built audit of it. + +## The shape of the thing + +This is a toolkit for building HTTP client libraries, not an HTTP client. `@dexpace/core` defines the +models, the pipeline, and the seams; it never opens a socket. The networking arrives through a +transport package you choose. + +``` + your library + │ + ▼ + Runtime ─── a Pipeline of Steps, itself a Transport + │ + │ PRE_REDIRECT · REDIRECT · POST_REDIRECT + │ PRE_RETRY · RETRY · POST_RETRY + │ PRE_AUTH · AUTH · POST_AUTH + │ PRE_LOGGING · LOGGING · POST_LOGGING + │ PRE_SERDE · SERDE · POST_SERDE + ▼ SEND + Transport ── @dexpace/transport-fetch │ @dexpace/transport-undici │ yours + │ + ▼ + wire +``` + +A `Runtime` implements `Transport`, so a pipeline is substitutable wherever a transport is — which is +what makes `PipelineBuilder.seedFrom()` and nested pipelines work at all. + +## The eleven packages + +Nine are published; two are private and exist only to serve the build. + +| Package | Provides | Third-party dependencies | +|---|---|---| +| `@dexpace/core` | Models, pipeline, seams, retry/redirect/auth/logging pillars, SSE, pagination, configuration, observability | **none** | +| `@dexpace/transport-fetch` | `fetchTransport()` over the runtime's global `fetch` | none | +| `@dexpace/transport-undici` | `undiciTransport()` — pools, proxies, real `close()` | `undici` | +| `@dexpace/transport-shared` | `@internal` plumbing both transports need identically | none | +| `@dexpace/codec-json` | `jsonSerde()` — the reference wire codec, with PATCH tri-state | none | +| `@dexpace/body-file` | `fileBody()` — a file-backed request body over `node:fs` | none | +| `@dexpace/logging-pino` | `createPinoLogger()` | `pino` (optional peer) | +| `@dexpace/logging-debug` | `createDebugLogger()` | `debug` (optional peer) | +| `@dexpace/rx` | `Observable` views of SSE and pagination | `rxjs` (peer) | +| `@dexpace/shrink-test` | *private.* Proves the published bundles survive minify + tree-shake | — | +| `@dexpace/transport-conformance` | *private.* The shared `TRANSPORT-N` suite both transports run | — | + +### Two rules the layout enforces mechanically + +**Zero runtime dependencies, everywhere.** `SEAM-1` says core takes none. `bun run verify:seam-1` +asserts it for **every** package under `packages/`, not core alone, and `NFR-2` is the reason: each +optional capability is a separately installable unit taking core plus at most one external library. +`transport-undici` spends its one on `undici`; `transport-fetch` spends none. Reaching for a small +date or URL utility is exactly the reflex that gate exists to catch. + +**`@dexpace/core` is always a peer, never a dependency.** Two copies of core in one install would +defeat the branded symbols and identity checks the seams rely on — the dual-package hazard. The same +gate checks this, and `verify:dual-consumption` then imports each built package from plain `node` and +exercises it end to end. + +## The seams + +A seam is an interface core defines and does not implement. There are four that matter, and each is +small enough to quote in full. + +```typescript +interface Transport { + send(request: Request, options?: RequestOptions, signal?: AbortSignal): Promise; + close(): Promise; +} + +interface Serde { + readonly serializer: Serializer; + readonly deserializer: Deserializer; + readonly mediaType: string; +} + +interface PaginationStrategy { + parse(response: Response, template: Request): Promise>; +} + +interface Logger { + atLevel(level: LogLevel): LogEvent; + withContext(fields: Readonly>): Logger; +} +``` + +There is **no registration step and no discovery mechanism**. A conforming object is a valid +implementation; you pass it in. `SEAM-5`–`SEAM-10` describe a classpath-style plugin registry with +conflict resolution, and this port will never build it — a permanent simplification, recorded as an entry +that is explicitly *not* a deferral in the `SEAM-5`–`SEAM-10` row of +[`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md), +where the dissolved deferral register's rows went. `Tracer`, `Span`, `Meter`, +`Counter`, `Histogram` and `Clock` are the same shape: duck-typed, so an OpenTelemetry object +satisfies them with no adapter. + +## A request, end to end + +```typescript +import { + ApiKeyCredential, + Request, + createAuthDescriptor, + createAuthRequirement, + standardResilience, + toHttpError, +} from '@dexpace/core'; +import {undiciTransport} from '@dexpace/transport-undici'; + +const transport = undiciTransport({agentOptions: {connections: 32}}); + +const client = standardResilience(transport, { + retry: {settings: {maxAttempts: 4}}, + redirect: {maxHops: 3}, + auth: { + credentials: {apiKey: {credential: new ApiKeyCredential('k'), headerName: 'X-Api-Key'}}, + tiers: {client: createAuthDescriptor([createAuthRequirement('API_KEY')])}, + }, +}); + +const response = await client.send( + Request.newBuilder().url('https://api.example.com/v1/things').build(), +); + +const failure = await toHttpError(response); // drains and closes on a 4xx/5xx +if (failure !== null) throw failure; + +try { + console.log(await response.text()); +} finally { + await response.close(); + await transport.close(); // the pipeline never owns the transport (PIPE-27) +} +``` + +Reading that stack outward from the wire: + +1. **Bodies.** A request `Body` is a *producer*: `writeTo(sink)` emits bytes on demand, and + `replayable` decides whether a retry may re-send it. A response body is a + `ReadableStream` the **caller** owns and must `close()`. See [`bodies.md`](./bodies.md). +2. **Models.** `Request`, `Response`, `Headers`, `QueryParams`, `RequestOptions` and + `RequestConditions` are frozen at construction and reachable only through a builder, so + case-insensitivity, multi-value ordering, header-injection defenses and method/body legality are + fixed once and behave identically under every transport. See [`http.md`](./http.md). +3. **Context.** `DispatchContext` promotes to `RequestContext` and then `ExchangeContext`, carrying an + `InstrumentationBundle` throughout. A step reads `ctx.context.kind` to know which promotion it is + in. +4. **Pipeline.** Sixteen ordered stages; five of them pillars admitting one step each. See + [`pipelines.md`](./pipelines.md). +5. **Transport.** Two methods. See [`write-a-transport.md`](./write-a-transport.md). + +## Runtime floor and module format + +ESM only, `NodeNext` resolution, `engines.node >= 20.3`. The floor is derived rather than chosen: +`scripts/verify-runtime-floor.mjs` pairs the TypeScript language level with the Node version whose +built-ins the SDK actually calls — `globalThis.crypto` is absent from ESM on every Node 18, and +`AbortSignal.any()` landed in 20.3.0. Moving it is a reviewed decision about supported runtimes, never +a mechanical bump, and one such request has already been refused: `Symbol.asyncDispose` arrived in +20.4, so `await using` is **not** offered on `Page`, `fetchTransport()` or `undiciTransport()`, and +`close()` is the teardown on every runtime — see `open-items.md`'s Section D row +[`await using` support](../work/mvp/2026-09-04-open-items-dissolution.md#d-nfr-10-await-using). diff --git a/docs/sdk-documentation/auth.md b/docs/sdk-documentation/auth.md new file mode 100644 index 0000000..8873c4e --- /dev/null +++ b/docs/sdk-documentation/auth.md @@ -0,0 +1,198 @@ +# Authentication + +The auth pillar resolves *which* credential a call needs, then *stamps* it — once per attempt, per +redirect hop, per retry. It runs inside redirect and inside retry (`AUTH-27`), which is why a retried +request never replays a stale token and a redirected one is re-stamped against the new hop. + +## Tiers + +```typescript +interface AuthTiers { + readonly perCall?: AuthDescriptor; // highest precedence + readonly operation?: AuthDescriptor; + readonly client?: AuthDescriptor; // lowest +} +``` + +The most specific tier that is set wins (`AUTH-4`). `perCall` comes from +`RequestOptions.newBuilder().auth(descriptor)`, which is how one call opts out of, or into, something +different from the client default. **`operation` comes from `RequestOptions.operationAuth`**, set with +`RequestOptions.newBuilder().operationAuth(descriptor)` — both per-call slots ride on the same +`RequestOptions`, and the AUTH pillar step reads them into the two tiers +(`packages/core/src/auth/auth-step.ts:252,756`; `packages/core/src/auth/resolve.ts:19` names the mapping). `client` is the +configured default. Only the *selection* is tiered; whether a selected requirement can be satisfied at +all is `AUTH-6`, and failing it is an `AuthResolutionError`. + +An `AuthDescriptor` is a list of `AuthRequirement`s, each a scheme plus optional scopes and +parameters. Both are built by factory, never as an object literal: + +```typescript +import {createAuthDescriptor, createAuthRequirement} from '@dexpace/core'; + +const clientTier = createAuthDescriptor([ + createAuthRequirement('OAUTH2', ['read:things']), + createAuthRequirement('NO_AUTH'), // allowsAnonymous becomes true +]); +``` + +The factories validate and freeze (`AUTH-3`). A descriptor containing a `NO_AUTH` requirement reports +`allowsAnonymous`, which is how "authentication is optional here" is expressed. + +## Credentials + +```typescript +interface AuthCredentialSet { + readonly bearer?: {provider: TokenProvider; marginMs?: number}; + readonly basic?: BasicCredential; + readonly digest?: DigestCredential; + readonly apiKey?: {credential: ApiKeyCredential | NameKeyCredential; headerName?: string; prefix?: string}; +} +``` + +The five schemes are `OAUTH2`, `API_KEY`, `BASIC`, `DIGEST` and `NO_AUTH`. A requirement names a +scheme; the credential set supplies the material for it. A requirement with no matching credential is +an `AuthResolutionError` at send time, not a silent unauthenticated request. + +**Every credential type is nominal, not structural.** `ApiKeyCredential`, `NameKeyCredential`, +`BearerToken`, `BasicCredential` and `DigestCredential` each carry a `#private` field, so no +caller-side object literal is assignable to them and the validation behind each cannot be routed +around. All five override `toString()` and Node's inspect symbol, so a credential cannot leak into a +log line or a stack trace by accident. + +```typescript +import { + ApiKeyCredential, + BasicCredential, + createBearerToken, + DigestCredential, +} from '@dexpace/core'; + +const credentials = { + basic: new BasicCredential('alice', 'super-secret'), + digest: new DigestCredential('bob', 'super-secret', ['SHA-256', 'MD5']), + apiKey: {credential: new ApiKeyCredential('super-secret'), headerName: 'X-Api-Key'}, +}; + +String(new ApiKeyCredential('super-secret')); // 'ApiKeyCredential{key=***}' +String(createBearerToken('t', 1)); // 'BearerToken{token=***, expiresAt=1}' — the expiry survives +String(credentials.basic); // 'BasicCredential{username=alice, password=***}' +``` + +There is no `password` property to read back, by design: `BasicCredential` and `DigestCredential` +shipped as plain `{username, password}` records until 2026-09-04, and a plain property is reachable +through `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect` — so +`util.inspect(credentials)` printed the password in clear beside `ApiKeyCredential{key=***}`. The +username and the Digest algorithm preference stay visible; `AUTH-8` permits non-secret fields to. + +The inspect symbol matters as much as `toString`: `console.log(credential)` and `util.inspect` do not +route an object argument through `toString`, so both are overridden (`AUTH-8`). + +## A worked client + +```typescript +import { + createAuthDescriptor, + createAuthRequirement, + createBearerToken, + standardResilience, +} from '@dexpace/core'; +import {undiciTransport} from '@dexpace/transport-undici'; + +declare function mintToken(): Promise; + +const client = standardResilience(undiciTransport(), { + auth: { + credentials: { + bearer: { + provider: async () => createBearerToken(await mintToken(), Date.now() + 3_600_000), + marginMs: 60_000, + }, + }, + tiers: {client: createAuthDescriptor([createAuthRequirement('OAUTH2')])}, + }, +}); +``` + +`TokenProvider` is `() => Promise`. The cache refreshes a token `marginMs` before its +`expiresAt`, and **concurrent refreshes are serialized** — a burst of calls arriving at expiry mints +one token, not one per call (`XCUT-12`). A provider returning a null or already-expired token is an +`AuthResolutionError` (`AUTH-35`), not a request sent with a dead token. + +## The HTTPS guard + +A credentialed scheme meeting a non-HTTPS URL is a `PlaintextCredentialError`, raised before the +request is dispatched (`AUTH-28`). There is no option to disable it. `NO_AUTH` never trips it, which +is why an auth-less `standardResilience()` installs a `NO_AUTH`-only step rather than no step at all — +the pillar slot stays filled and the behaviour stays uniform. + +**Once guarded, always guarded.** A `challengeHook` may return any request it likes, including one +whose URL has been downgraded to `http://`. When the outbound pass ran the guard, the replay is +guarded too — unconditionally, without inspecting a single header name, because +`ApiKeyCredentialConfig.headerName` lets this step stamp a header no fixed list would contain. On a +`NO_AUTH` hop, which is never guarded outbound, a replacement carrying `Authorization` or +`Proxy-Authorization` still trips the guard. A genuinely credential-free re-issue over `http://` is +what `XCUT-16` explicitly permits, and it still proceeds. + +## Challenges + +```typescript +type ChallengeHook = ( + response: Response, + request: Request, + options?: {signal?: AbortSignal}, +) => Promise; +``` + +A `challengeHook` sees a `401` and may return a replacement request; returning `undefined` means "I +cannot satisfy this", and the `401` surfaces to the caller unchanged. The built-in Basic and Digest +handlers — including the RFC 7235 `WWW-Authenticate` parser and RFC 7616 Digest with MD5, MD5-sess, +SHA-256 and SHA-256-sess — are internal and drive themselves; the hook is for schemes this SDK does +not implement. + +There is deliberately **no** way to append a handler to the built-in list. A `handlers` field existed +and was cut at review: it forced three types onto the public barrel and could not compose with the +internal handlers, so it was replace-semantics masquerading as extension. The shape to ship, if a +caller ever needs it, is an append field plus public `basicHandler`/`digestHandler` factories +(the *caller-supplied `ChallengeHandler` list on `AuthStepSettings`* row, still live and unscheduled +until a second auth scheme needs one — archived under *Live deferrals* in +[`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md) +when the deferral register was dissolved on 2026-09-04). + +**Basic and Digest never stamp preemptively.** They react to a challenge. That is an interpretation of +`§11` rather than a stated requirement, and it is ledgered as one. + +**Every challenge the response offered is considered, not just the first.** A server may send +`WWW-Authenticate` (or `Proxy-Authenticate`) once with several comma-separated challenges, or once per +challenge — RFC 9110 §5.3 permits both, and RFC 7616 §3.3 recommends the repeated form for Digest +algorithm discovery. Which of the two shapes reaches the step is partly the transport's accident: +`@dexpace/transport-fetch` comma-joins repeated values because WHATWG `Headers` does, +`@dexpace/transport-undici` keeps them apart. The step reads *every* value and parses each on its own, +so the offer is the same list either way, and `@dexpace/transport-conformance` carries a row asserting +that. A challenge the SDK cannot answer — an unsupported algorithm, an `auth-int`-only `qop`, a realm +this client cannot echo back, an empty `realm` or `nonce` — is declined and the next one is tried; a +`401` offering nothing answerable surfaces unchanged and unclosed. + +**Digest `-sess` sends `cnonce` whether or not `qop` was negotiated.** `MD5-sess` and `SHA-256-sess` +fold the client nonce into HA1 (RFC 7616 §3.4.2), so a server handed a `-sess` response without one +cannot verify it. `nc` and `qop` stay conditional on a negotiated `qop`. `AUTH-22`'s literal wording +says all three are conditional; the departure is a row in +[`deviations.md`](../deviations.md). + +## Redirects and credentials + +Credentials are attached at an origin and must not follow a request to a different one. The redirect +pillar marks a cross-origin hop with an internal header; the auth step is that marker's consumer and +first stripper, and a `POST_AUTH` guard strips it again as an idempotent backstop, so the marker can +never reach the wire (`REDIR-11`, `AUTH-29`). + +That guard is installed by `standardResilience()`, and as of 2026-09-02 a hand-built pipeline can +install it too: `withRedirect(builder)` seats the pillar and its guard together, and +`stripCrossOriginMarkerStep()` is the guard on its own. Reaching for bare `redirectStep()` without +one of them is what forwards the marker — see [`pipelines.md`](./pipelines.md). + +## Proxy credentials are a separate axis + +`ProxyOptions.credentials` answers a proxy's `407`; the auth pillar answers an origin's `401`. They +never cross: proxy credentials are never sent in answer to a `401`, and a per-request +`Proxy-Authorization` header is dropped from the outbound pass whenever a proxy is configured. See +`@dexpace/transport-undici`'s README — it is the only transport that can route a proxy at all. diff --git a/docs/sdk-documentation/bodies.md b/docs/sdk-documentation/bodies.md new file mode 100644 index 0000000..a5b3d04 --- /dev/null +++ b/docs/sdk-documentation/bodies.md @@ -0,0 +1,203 @@ +# Bodies + +There are two body concepts and they are not symmetric. A **request** body is a producer you hand to +the SDK. A **response** body is a resource the SDK hands to you, and you own it. + +## Request bodies are producers, not buffers + +```typescript +interface Body { + readonly kind: 'byte-array' | 'string' | 'stream' | 'form-urlencoded' | 'multipart' | 'file'; + readonly mediaType: string | undefined; + readonly contentLength: number; + readonly replayable: boolean; + writeTo(sink: WritableStream): Promise; +} +``` + +`writeTo` emits bytes on demand into a sink the transport supplies. Nothing is buffered until +something asks for it, which is what lets a file body of any size cost a constant amount of heap. + +**The concrete classes are exported as types only.** `ByteArrayBody`, `StringBody`, +`FormUrlEncodedBody`, `MultipartBody` and `StreamBody` are `export type`, never values, because +exporting the class would publish `new ByteArrayBody(...)` as a field-wise constructor — which +`HTTP-2` forbids and which duplicates the factory for no stated need. Construct through the factory +and annotate with the type. + +| Factory | Signature | `replayable` | +|---|---|---| +| `byteArrayBody` | `(bytes, mediaType?)` | `true` | +| `stringBody` | `(text, mediaType?)` | `true` | +| `formUrlEncodedBody` | `(input)` — a `QueryParams`, `Map`, plain object, or entry list | `true` | +| `multipartBody` | `(parts, boundary?)` | as its least-replayable part | +| `streamBody` | `(stream, mediaType?, contentLength?)` | `false` | +| `serdeBody` | `(value, serde, mediaType?)` | `true` | +| `fileBody` (`@dexpace/body-file`) | `(path, {start?, count?})` | `true` | + +```typescript +import {Request, multipartBody, stringBody} from '@dexpace/core'; +import {fileBody} from '@dexpace/body-file'; + +const upload = multipartBody([ + {name: 'metadata', body: stringBody('{"kind":"photo"}', 'application/json')}, + {name: 'file', filename: 'cat.jpg', body: fileBody('./cat.jpg')}, +]); + +const request = Request.newBuilder() + .method('POST') + .url('https://api.example.com/v1/uploads') + .body(upload) + .build(); +``` + +`multipartBody` generates a boundary when you do not supply one, and validates a supplied one against +the boundary grammar — a bad boundary is a `MultipartBoundaryError` at construction, not a corrupt +request on the wire. + +## Replayability, and what retry does about it + +`replayable` answers one question: can this body be sent a second time? A `ReadableStream` is +single-use by construction, so `streamBody(...).replayable` is `false` and a retry of a request +carrying one cannot re-send it. + +`materialize(body)` is the escape hatch — it drains the body once, into memory, and returns an +equivalent replayable one: + +```typescript +import {materialize, streamBody} from '@dexpace/core'; + +declare const someStream: ReadableStream; + +const once = streamBody(someStream, 'application/octet-stream'); +const many = await materialize(once); // now replayable; the original is consumed + +console.log(once.replayable, many.replayable); // false true +``` + +That is a deliberate cost, taken deliberately: buffering an arbitrarily large upload to make it +retryable is a decision for the caller who knows how large it is, not for the retry engine. A +retryable body arrives at the retry pillar already retryable. + +**A body is single-use even when `replayable` is `true`** in one sense that matters: `writeTo` may be +called again, but the *sink* may not be reused. Each call needs its own sink, and the transport +supplies one per attempt. + +## Response bodies belong to the caller + +`Response.body` is a `ReadableStream | null`. `Response` also offers `bytes()` and +`text()`, which drain it. + +**You close it. Always. On every path.** `BODY-15` puts ownership with the caller, and nothing in the +pipeline closes a response it hands you — not the retry pillar, not the redirect pillar, not +`Runtime.send()`. + +```typescript +import type {Response} from '@dexpace/core'; + +declare const response: Response; + +async function read(): Promise { + try { + return await response.text(); + } finally { + await response.close(); + } +} +``` + +`close()` is idempotent, and idempotent in the strict sense: the promise is **memoized**, not +flag-guarded, so a release that *fails* propagates that failure to every caller rather than the +second call reporting success over a connection that was never released. + +`bytes()` and `text()` close the response themselves, whether the read succeeds or not (`BODY-16`) — +including when an external consumer already holds the reader lock and `getReader()` throws. Calling +`close()` afterwards is still correct and costs nothing. + +**On the request side, a single-use body written twice raises `ConsumedBodyError`** (`BODY-3`). That +is a different error from anything on the response side; `isBodyError(e)` narrows to it and its two +siblings. + +### The one place the SDK closes a response for you + +`toHttpError(response)` does, because it must: + +```typescript +import {toHttpError, type Response} from '@dexpace/core'; + +declare const response: Response; + +const failure = await toHttpError(response); +if (failure !== null) throw failure; // response is already drained and closed +console.log(await response.text()); // only reachable on 2xx/3xx +``` + +On an error status it drains the body up to a 1 MiB cap (`BODY-30`/`HTTP-52`), keeps the preview on +the returned `HttpStatusError`, and closes the response — the connection is released even for a +20 GB error body, because the drain keeps reading past the cap and discards. On a non-error status it +returns `null` and leaves the response untouched and unread. + +`HttpStatusError` therefore carries the status, the headers, and a bounded body preview. The full +body is irrecoverably gone; that is the trade, and it is deliberate. The cap itself is required — +`BODY-30`/`HTTP-52` — and the decision to size it once for every consumer rather than make it +configurable is recorded as a **closed deferral** — the "Every buffering cap" row of +[`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md), +where the dissolved deferral register's rows went — not as a deviation: +nothing here departs from the reference contract. `errors.md` states the same fact the same way. + +### Reading a response as a model + +`TypedResponse` pairs a `Response` with a parse function and exposes `value()`: + +```typescript +import {TypedResponse, type Response} from '@dexpace/core'; + +declare const response: Response; + +const typed = new TypedResponse(response, async r => JSON.parse(await r.text()) as {id: number}); +const {id} = await typed.value(); +``` + +Status, headers, protocol and request stay reachable without consuming anything. For the +schema-driven form, see [`write-a-response-handler.md`](./write-a-response-handler.md). + +## File bodies, and why they are a separate package + +```typescript +import {fileBody} from '@dexpace/body-file'; + +const body = fileBody('./upload.bin', {start: 1024, count: 4096}); +``` + +`@dexpace/core` cannot import `node:fs`; its zero-`node:`-import invariant is hard, and that is the +whole reason `@dexpace/body-file` exists as its own unit. + +The file is stat'd at **construction**, not at send time (`HTTP-40`, `BODY-11`), and all four ways +the range can be wrong are rejected there: the path must exist and be a regular file, `start >= 0`, +`start <= size`, `count >= 0`, and `start + count <= size`. The `start <= size` check earns its place +independently — `count` defaults to `size - start`, which goes negative past end-of-file and then +*satisfies* the sum check, silently producing a zero-byte upload. + +`writeTo()` opens a **fresh** handle per call, so a retry re-sends the same bytes. It does not close +the sink it was handed (`BODY-8` — closing belongs to whoever created it) but aborts it on failure, +so a consumer sees the error rather than a silently truncated stream. A short read raises rather than +reporting success (`BODY-13`). + +**Transports recognize a file body structurally, on `body.kind === 'file'`, never by `instanceof`.** +That is what lets `@dexpace/transport-undici` dispatch straight off the file — honoring `start`/`count`, +one fewer userspace copy — while depending on neither `@dexpace/body-file` nor anything it exports. +`FileBodyDescriptor` in core is the structural contract both sides agree on. + +## Serde bodies + +`serdeBody(value, serde, mediaType?)` serializes through a `Serde` and takes the serde's own media +type unless you override it: + +```typescript +import {serdeBody} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; + +const body = serdeBody({name: 'ada'}, jsonSerde()); // Content-Type: application/json +``` + +See [`write-a-serde.md`](./write-a-serde.md) for the seam itself, including the PATCH tri-state +problem that makes `{}` and `{"x": null}` different messages. diff --git a/docs/sdk-documentation/errors.md b/docs/sdk-documentation/errors.md new file mode 100644 index 0000000..fd62950 --- /dev/null +++ b/docs/sdk-documentation/errors.md @@ -0,0 +1,221 @@ +# Errors + +Every error this SDK raises **as a condition a caller might handle** descends from `DexpaceError`, +which descends from `Error`. There are no bare `throw new Error(...)` sites, and wrap-and-rethrow +always passes `{cause}`, so the original is always reachable. + +The one deliberate exception is `InvariantViolation`, which extends `Error` directly because it +signals a bug rather than a condition — see the end of this file. + +```typescript +import {DexpaceError, type Request, type Transport} from '@dexpace/core'; + +declare const client: Transport; +declare const request: Request; + +try { + await client.send(request); +} catch (error) { + if (error instanceof DexpaceError) { + // ours: name, message, and a cause chain + } + throw error; +} +``` + +Each class sets `this.name = new.target.name` in its constructor, so `error.name` is the class name +even after minification changes the function's own `name`. + +## The tree + +Two levels by rule, with exactly one sanctioned third. + +``` +Error +└── DexpaceError + │ + │ (the ten HTTP domain-model errors — `isDomainModelError` matches this whole group) + ├── RequiredFieldError a builder was missing a required field (HTTP-4) + ├── HeaderValidationError a header name or value broke the grammar + ├── UrlConstructionError the URL could not be built + ├── MediaTypeParseError malformed media type + ├── EtagParseError malformed ETag + ├── ProtocolParseError unrecognized protocol token + ├── HttpRangeValidationError malformed or impossible Range + ├── RequestOptionsValidationError timeoutMs / maxRetries out of range + ├── RequestConditionsValidationError contradictory if-match / if-none-match state + ├── RequestBodyNotAllowedError a body on a method that forbids one + │ + ├── IoError — a byte-level failure + │ └── TransportFailureError the one third level, see below + ├── HttpStatusError — the server answered 4xx/5xx (see `toHttpError`) + ├── CancellationError — the caller aborted; terminal, never retried + ├── ConsumedBodyError — a single-use request body was written twice (BODY-3) + ├── MultipartBoundaryError — a supplied multipart boundary broke the grammar + ├── FormBodyValidationError — form-encoded input was not encodable + ├── AuthResolutionError — no configured credential satisfies the resolved tier + ├── PlaintextCredentialError — a credentialed scheme met a non-HTTPS URL (AUTH-28) + ├── SerializationError — a value could not be serialized + ├── DeserializationError — wire bytes did not satisfy the schema + ├── SseStreamError — the SSE stream failed + ├── SseLineTooLongError — a line exceeded the bounded buffer + ├── PaginationError — engine misuse or a precondition violation + └── OperationAssemblyError — an OperationDescriptor could not build a Request +``` + +**`TransportFailureError extends IoError` is the deliberate third level.** It is ledgered +(`docs/deviations.md` item 17): flattening it would force every consumer to discriminate on a string +tag, and `catch (e) { if (e instanceof IoError) }` still catches a transport failure. Held at exactly +three; a fourth is not sanctioned. + +**`DomainModelError` was a second such tier, and it is gone.** It sat between `DexpaceError` and the +ten model leaves above as an empty marker class, and nothing in the SDK ever narrowed on it. Those +ten now extend `DexpaceError` directly, and `isDomainModelError(e)` replaces +`e instanceof DomainModelError` — same union, no inheritance level. That is a breaking change to a +barrel export, taken while `@dexpace/core` is still `0.0.0` and it is cheap to take. It does **not** +make the tree uniformly two-level: `TransportFailureError` above is required to be a third level by +`TRANSPORT-20`. + +## The distinction that matters most: cancel versus timeout + +```typescript +import { + CancellationError, + TransportFailureError, + type Request, + type RequestOptions, + type Transport, +} from '@dexpace/core'; + +declare const client: Transport; +declare const request: Request; +declare const options: RequestOptions; +declare const signal: AbortSignal; + +export async function call(): Promise { + try { + await client.send(request, options, signal); + } catch (error) { + if (error instanceof CancellationError) return; // the caller asked to stop + if (error instanceof TransportFailureError) throw error; // retryable: the network failed + throw error; + } +} +``` + +- **`CancellationError` is terminal.** The caller aborted. The retry engine will not retry it, and + nothing further will be attempted. A raw `DOMException` from `AbortSignal` is never surfaced; both + shipped transports map it (`TRANSPORT-3`/`TRANSPORT-4`). +- **`TransportFailureError` is retryable.** A timeout, a connection reset, a DNS failure. + +`composeSignal(userSignal, timeoutMs)` builds the combined signal, and `isTimeoutSignal(signal)` tells +the two apart at the point of abort — which is exactly how a transport decides which of the two +errors to raise. + +**Neither narrowing changes when a retry pillar is installed.** What `retryStep` throws once it gives +up is the final attempt's own error, unwrapped, so the two `instanceof` checks above read the same at +`maxAttempts: 1` and at `maxAttempts: 3`. The earlier attempts ride beside it and are read with +`retryAttempts(error)` — see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars). + +## Narrowing helpers + +Three predicates exist for the cases where `instanceof` on a union is tedious: + +```typescript +import {isBodyError, isDomainModelError, isSerdeError} from '@dexpace/core'; + +declare const e: unknown; + +isBodyError(e); // ConsumedBodyError | MultipartBoundaryError | FormBodyValidationError +isSerdeError(e); // SerializationError | DeserializationError +isDomainModelError(e); // the ten domain-model leaves; replaces `e instanceof DomainModelError` +``` + +## HTTP status failures + +A 4xx or 5xx is **not** an exception. `send()` resolves with the response, because the response is +often the useful part. `toHttpError` is the opt-in conversion: + +```typescript +import {toHttpError, type Request, type Transport} from '@dexpace/core'; + +declare const client: Transport; +declare const request: Request; + +const response = await client.send(request); +const failure = await toHttpError(response); +if (failure !== null) throw failure; +``` + +On an error status it drains the body to a 1 MiB cap (`BODY-30`/`HTTP-52`), keeps that preview on the +error, and **closes the response**. On any other status it returns `null` and leaves the response +untouched. The full error body is irrecoverable after the call; that is the documented trade. + +## Errors raised by the pipeline and the redirect pillar + +Eight classes that a `@throws` tag named but no package exported were promoted to the barrel on +2026-09-02, so `instanceof` now works for all of them (`docs/work/mvp/2026-09-04-open-items-dissolution.md` U9): + +| Error | Raised by | +|---|---| +| `SchemeDowngradeError` | the redirect pillar, on a rejected HTTPS→HTTP hop | +| `NonReplayableBodyError` | the redirect pillar, when a hop needs a body resend it cannot do | +| `PillarCollisionError` | `PipelineBuilder`, on a second step in one pillar stage | +| `AnchorNotFoundError` | `insertBefore`/`insertAfter`/`replace`, on an unknown `type` symbol | +| `CrossStageEditError` | an insert or replace whose incoming step declares a different stage than its anchor | +| `ReservedStageError` | any attempt to install a user step onto the terminal `SEND` stage | +| `CursorAlreadyAdvancedError` | a step reusing an already-invoked `next()`/`fork()` continuation | +| `EndOfStreamError` | a `BufferedSource` read that required more bytes than the source delivered | + +**The two redirect errors carry redacted messages and raw properties, and the split is deliberate.** +`SchemeDowngradeError.message` and `NonReplayableBodyError.message` name their URLs in the redacted +form `OBS-11`/`OBS-12` define — userinfo as `***:***@`, every non-allow-listed query value as `***` — +because a message is rendered by every logger, every `cause` chain and every consumer `console.error`, +and `http.redirect.rejected` hands the error straight to `LogEvent.cause()`. The unredacted URLs stay +on `fromUrl` / `toUrl` / `targetUrl`, which is where program code reads them: + +```ts +try { + await client.send(request); +} catch (error) { + if (error instanceof SchemeDowngradeError) { + console.error(error.message); // https://***:***@api.example.com/v1?token=*** + retargetTo(error.toUrl); // the real URL, for code rather than for a log + } +} +``` + +Two more joined them on the same date, both from `XCUT-8`'s "never fabricate a successful exception": + +| Error | Raised by | +|---|---| +| `HttpStatusValidationError` | `new HttpStatusError(status, …)` when `status` is not an integer in 400–599. The constructor validated nothing before, so a consumer could build an `HttpStatusError` claiming a `200`. `toHttpError` is the total form — it returns `null` instead of throwing | +| `RetryDiscardedResponseError` | the retry engine's prior-attempt trail (`retryAttempts()`), for a response it discarded whose status is outside 400–599. Reachable only if you widen `RetrySettings.retryableStatuses` to include a non-error code; the trail previously said `HttpStatusError` for it, which claimed an HTTP failure that had not happened | + +All of them descend from `DexpaceError`, so the broad catch works too. + +**`InvariantViolation` is the one that stays unreachable, deliberately.** It extends `Error` +directly, not `DexpaceError`, and it is not exported. It signals a broken internal precondition — a +bug in this SDK or in a seam implementation you supplied, never a condition to handle — so the +`@throws` tags that used to name it now read as prose ("an assertion failure (a caller bug, not a +catchable condition)"), because a `@throws` tag is supposed to name the type *and what the caller +should do about it* (`docs/knowledge/harvested/documentation.md:24`), and "catch this" is the wrong +answer here. `standardResilience()` raises it synchronously for invalid pillar settings, such as a +non-finite bearer refresh margin or a non-header-safe Digest username, which is the one case a +consumer will see it: at wiring time, loudly, before any request is sent. + +`DuplicateContextKeyError` is likewise unexported, because the only thing that raises it — +`ContextStore` — is itself `@internal` and appears in no `.d.ts` a consumer reads. + +## What does not throw + +Worth stating, because each looks like it should: + +- **Exceeding the redirect hop cap.** `maxHops` returns the current 3xx response, unfollowed + (`REDIR-17`, `decide.ts:205`). `maxHops: 0` is how "do not follow redirects" is spelled, and it is + the same code path. +- **A retry budget running out.** The last response is returned live and unread; ownership transfers + to you (`docs/work/mvp/2026-09-04-open-items-dissolution.md` P7). +- **An unrecognized status code.** `Status.of(599)` is a valid `Status`; see [`http.md`](./http.md). +- **A malformed ETag.** `ETag.parse` returns `undefined`. `EtagParseError` is for the construction + paths that cannot degrade. diff --git a/docs/sdk-documentation/http.md b/docs/sdk-documentation/http.md new file mode 100644 index 0000000..4ef7dc6 --- /dev/null +++ b/docs/sdk-documentation/http.md @@ -0,0 +1,172 @@ +# The HTTP domain model + +Everything in `packages/core/src/http/` follows one shape, and the shape is the point: a model is +frozen at construction and reachable only through a builder or a static factory, so +case-insensitivity, multi-value semantics, ordering, header-injection defenses, method/body legality +and total status handling are decided **once** and behave identically under every transport. + +## Building and deriving + +```typescript +import {Request} from '@dexpace/core'; + +const request = Request.newBuilder() + .method('POST') + .url('https://api.example.com/v1/things') + .headers( + Request.newBuilder().build().headers.newBuilder() + .set('Accept', 'application/json') + .add('X-Tag', 'a') + .add('X-Tag', 'b') + .build(), + ) + .build(); +``` + +`Request.newBuilder()` (static) starts empty. `request.newBuilder()` (instance) returns a builder +**pre-filled from that instance, deep-copying every collection** — so deriving never aliases the +source (`HTTP-3`), and mutating the derived request's headers cannot reach back into the original. + +There is no public constructor on any of them (`HTTP-2`). The emitted `.d.ts` declares the +constructor `private`, so a consumer cannot construct around `build()`'s validation. A missing +required field is a `` `${name} is required` `` error from one shared helper (`HTTP-4`), never a +bespoke message per field. + +**One place still leaks mutability, deliberately.** `request.url` returns a *clone* of the native +`URL` on every access, because `URL` is mutable and freezing the model cannot cascade into it +(`HTTP-5`). Reading it in a loop allocates; hoist it. + +## Headers + +```typescript +headers.get('content-type'); // first value, case-insensitive +headers.getAll('set-cookie'); // every value, in insertion order +headers.names(); // the names as first written +headers.entries(); // [name, value] per value, not per name +``` + +`HeadersBuilder` has four mutators, and the split is not cosmetic: + +| Method | For | +|---|---| +| `set(name, value)` | Outbound. Replaces every existing value. `null` removes the name | +| `add(name, value)` | Outbound. Appends, preserving order | +| `setInbound` / `addInbound` | The **lenient** pair, for values a server sent | + +Outbound values are validated against the strict field-value grammar: a CR, LF or NUL in a header +value is a `HeaderValidationError`, because that is header injection. Inbound values are accepted +leniently — obs-text bytes and all — because rejecting what a server actually sent would make the +client unable to read real responses. `HTTP-18`/`HTTP-48`/`HTTP-50`'s tension is exactly this, and +`docs/deviations.md` item 15 records the one case it cannot resolve: a server-issued `ETag` +containing obs-text does not round-trip, because replaying it outbound would have to pass the strict +grammar. + +`HeaderName.of(raw)` is the validated name type; every accessor takes `string | HeaderName`. + +## Status + +`Status` is **total**. Any integer is a `Status`: + +```typescript +Status.of(200).name // 'OK' +Status.of(200).isSuccess // true +Status.of(599).name // undefined +Status.of(599).isRecognized // false +Status.of(599).isServerError // true +Status.recognized(599) // undefined +``` + +An unrecognized code is never an error — a server is free to invent one — but `recognized()` lets a +caller tell a vendor code from a registered one when that matters. The class predicates +(`isInformational`, `isSuccess`, `isRedirect`, `isClientError`, `isServerError`, `isError`) are +range checks and work on unrecognized codes too. + +## The value types + +These have no builder; a static factory is the whole surface. + +| Type | Factories | Notes | +|---|---|---| +| `Status` | `of`, `recognized` | above | +| `Protocol` | `HTTP_1_1`, `HTTP_2`, `parse` | Both shipped transports always report `HTTP_1_1`: neither `fetch`'s `Response` nor undici's `ResponseData` exposes the negotiated version. A ledgered deviation, not a silent gap | +| `MediaType` | `of`, `parse` | `parse('text/plain;charset=utf-8').charset` → `'utf-8'`. `matches(pattern)` does wildcard subtype matching. **`charset` is resolved against the runtime's WHATWG encoding registry**, so an unrecognized label answers `undefined` — `parse('text/plain;charset=bogus').charset` is `undefined` (`HTTP-24`) while `parameter('charset')` still returns `'bogus'` verbatim and `render()` round-trips it (`HTTP-25`) | +| `ETag` | `parse`, `ANY` | `parse` returns `undefined` on a malformed tag rather than throwing. `isWeak`, `opaque`, `raw` | +| `HttpRange` | `bounded`, `open`, `suffix`, `parse` | `kind` discriminates the three | + +## Per-call options + +`RequestOptions` carries what belongs to *this* call rather than to the request: + +```typescript +import {RequestOptions} from '@dexpace/core'; + +const options = RequestOptions.newBuilder() + .timeoutMs(5_000) + .maxRetries(3) + .tags(new Map([['operation', 'listThings']])) + .build(); +``` + +`RequestOptions.EMPTY` is the shared no-op instance. A step reads it as `ctx.options`, and a +transport receives it as `send()`'s second argument. Both range checks are the **full** range, not +only the lower bound: `maxRetries` rejects anything that is not a non-negative integer, and +`timeoutMs` rejects zero, negatives, `Infinity`, `NaN`, a fractional value and anything above +`2**32 - 1` (`HTTP-35`) — the range is `AbortSignal.timeout()`'s, the only one a transport can +honor. + +`auth` on the builder is the **per-call** auth tier, the highest-precedence one; see +[`auth.md`](./auth.md). + +## Conditional requests + +`RequestConditions` is a builder over the four conditional headers, and it applies itself: + +```typescript +import {ETag, RequestConditions, type Request} from '@dexpace/core'; + +declare const request: Request; + +const conditions = RequestConditions.newBuilder() + .ifNoneMatch(ETag.parse('"abc"') ?? ETag.ANY) + .ifModifiedSince(new Date(0)) + .build(); + +const conditioned = request.newBuilder().headers(conditions.applyTo(request.headers)).build(); +``` + +`applyTo` returns a **new** `Headers` — it never mutates the one it is given. + +## Query parameters + +`QueryParams` is the one URL-manipulation surface. It is not `URLSearchParams`, and the difference is +deliberate: `URLSearchParams` re-serializes a whole query string, reorders parameters, and re-encodes +what was already encoded. `QueryParams` preserves insertion order and encodes exactly once +(`docs/work/mvp/2026-09-04-open-items-dissolution.md` J4), with RFC 3986 component encoding rather than +`application/x-www-form-urlencoded`'s — so a space becomes `%20`, not `+`, and a literal `+` becomes +`%2B`. + +```typescript +import {QueryParams} from '@dexpace/core'; + +const params = QueryParams.newBuilder() + .add('q', 'a b') + .add('plus', 'c+d') + .add('flag', null) // HTTP-28: a value-less parameter, stored as the empty string + .build(); + +params.encode(); // 'q=a%20b&plus=c%2Bd&flag=' +``` + +`add(name, null)` records a value-less parameter as a single empty string, never the text `"null"`. +A name whose value list ends up empty is dropped at `build()` so it cannot leave a phantom entry that +`has()` reports and `encode()` never emits (`HTTP-30`). + +The pagination engine's query splice and the `Link`-header tokenizer are deliberately **not** +exported: publishing them would put a second URL-manipulation surface next to this one. + +## Operations + +`buildRequest(baseUrl, operation)` assembles a `Request` from an `OperationDescriptor` — a declarative +path template with its parameters — for callers generating clients from a service description rather +than writing builders by hand. It raises `OperationAssemblyError` on a template a parameter set +cannot satisfy. diff --git a/docs/sdk-documentation/pipelines.md b/docs/sdk-documentation/pipelines.md new file mode 100644 index 0000000..93fb6e7 --- /dev/null +++ b/docs/sdk-documentation/pipelines.md @@ -0,0 +1,334 @@ +# Pipelines + +A pipeline is an ordered list of steps ending at a transport. Building one is the main thing a client +library does with this SDK. + +## The three types + +```typescript +type Next = (request?: Request) => Promise; +type Step = (request: Request, ctx: StepContext) => Promise; + +interface StepDescriptor { + readonly type: symbol; // stable identity, for anchoring and removal + readonly stage: Stage; // where in the order it sits + readonly fn: Step; // the behaviour +} +``` + +A step receives the request, may rewrite it, calls `ctx.next(maybeRewritten)` to invoke everything +below, and may post-process the response on the way back up. `ctx.next()` with no argument passes the +request through unchanged. + +```typescript +interface StepContext { + readonly next: Next; + readonly context: ExecutionContext; // DispatchContext | RequestContext | ExchangeContext + readonly options?: RequestOptions; // the per-call options + readonly signal?: AbortSignal; // the caller's signal + readonly fork?: () => Next; // a fresh chain, for steps that re-drive +} +``` + +`fork` is what separates a re-driving step from an ordinary one. `next` may be called once; a step +that retries or follows a redirect calls `ctx.fork()` to obtain a fresh downstream chain per attempt. +Retry, redirect and auth all use it. An ordinary step does not need it and should not take it. + +## The sixteen stages + +``` +PRE_REDIRECT REDIRECT POST_REDIRECT +PRE_RETRY RETRY POST_RETRY +PRE_AUTH AUTH POST_AUTH +PRE_LOGGING LOGGING POST_LOGGING +PRE_SERDE SERDE POST_SERDE +SEND +``` + +`STAGE_ORDER` is that array. `PILLAR_STAGES` is the set `{REDIRECT, RETRY, AUTH, LOGGING, SERDE}` — +each admits **exactly one** step and raises on a second. The `PRE_`/`POST_` stages around them stack +with append/prepend semantics, and are the user-extensible slots. + +Order is not arbitrary. Redirect wraps retry wraps auth (`AUTH-27`), so a retry attempt re-resolves +credentials and a redirect hop re-stamps them. Getting that backwards means replaying a stale token +or leaking a credential across an origin. + +`SERDE` is reserved and ships no behaviour anywhere in this roadmap's scope. It is a pillar so that a +future serde step cannot be installed twice. + +## The preset + +```typescript +import {standardResilience} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const client = standardResilience(fetchTransport(), { + retry: {settings: {maxAttempts: 4, totalTimeoutMs: 30_000}}, + redirect: {maxHops: 3, allowSchemeDowngrade: false}, + logging: {granularity: 'headers'}, + // auth: omitted -> a NO_AUTH-only step that stamps nothing +}); +``` + +Every slot is optional and every omitted one takes that pillar's own defaults. +`PIPE-24`'s "installs into empty pillar slots only" holds **by construction**: the function always +starts from a fresh `PipelineBuilder`, so no slot can be occupied and no runtime check is needed. + +`standardResilience()` also installs the redirect pillar through `withRedirect()`, which seats a +second, `POST_AUTH` step alongside it — the guard that strips the SDK's internal cross-origin marker +header before dispatch (`REDIR-11(c)`). **Both are public as of 2026-09-02** +(`docs/work/mvp/2026-09-04-open-items-dissolution.md` U7): call `withRedirect(builder)` to get the pillar and its guard together, or +`stripCrossOriginMarkerStep()` to install the guard yourself. A pipeline that installs bare +`redirectStep()` and neither of them forwards the marker to the wire. + +## Extending the preset + +```typescript +import {PipelineBuilder, standardResilience, type Step} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const requestId: Step = async (request, ctx) => + ctx.next( + request + .newBuilder() + .headers(request.headers.newBuilder().set('X-Request-Id', crypto.randomUUID()).build()) + .build(), + ); + +const runtime = PipelineBuilder.seedFrom(standardResilience(fetchTransport()), 'flatten') + .append({type: Symbol('x-request-id'), stage: 'PRE_SERDE', fn: requestId}) + .build(); +``` + +`seedFrom(runtime, mode)` takes a **built** runtime and returns a builder seeded from it: + +- **`'flatten'`** unpacks the runtime's steps into the new builder, so the result is one flat chain + and the new step sits in true stage order among the old ones. +- **`'nest'`** installs the whole runtime as a single terminal unit, so the old pipeline runs as an + opaque inner chain. Use this when the inner pipeline's ordering must be preserved exactly. + +The rest of the builder API operates by descriptor `type` symbol: `insertBefore`, `insertAfter`, +`replace`, `remove`, `reload`. Anchoring on a symbol rather than a position is what keeps an edit +correct when the surrounding pipeline changes. + +## Runtime + +```typescript +class Runtime implements Transport { + send(request: Request, options?: RequestOptions, signal?: AbortSignal): Promise; + close(): Promise; + get steps(): readonly StepDescriptor[]; + get transport(): Transport; +} +``` + +Because `Runtime` **is** a `Transport`, a pipeline is substitutable for the transport it wraps — which +is what makes `'nest'` mode and `Paginator`'s `transport` field work on a full pipeline. + +`Runtime.close()` is a documented no-op. The pipeline never owns the transport it was given +(`PIPE-27`); closing it is the caller's job, in the `finally` that also closes the response. + +## Execution context + +A call promotes through three context shapes, and a step reads `ctx.context.kind` to know which it is +in: + +| `kind` | Shape adds | Meaning | +|---|---|---| +| `'dispatch'` | `key`, `instrumentation` | Before a request exists | +| `'request'` | `request`, `operationName` | A request has been assembled | +| `'exchange'` | the response side | A response has arrived | + +All three carry the same `InstrumentationBundle`, so trace and span identity survive the promotions. +`activateSpan(span)` returns a `Scope`; `getActiveSpan()` reads the current one. Propagation is +`AsyncLocalStorage`-based, which is why `@dexpace/rx` installs no RxJS scheduler — adding +`observeOn`/`subscribeOn` downstream makes reinstating the context the caller's job. + +**What a call leaves behind is nothing.** `send()` scopes both async-scoped stores with +`AsyncLocalStorage.run`, so the moment it settles — resolved or rejected — the active span and the +diagnostic fields are what they were before it. That is a guarantee about *your* context, not only the +pipeline's: an application log emitted after `await client.send(...)` carries no `trace.id` from it. +Inside the call the usual rules apply, and a step that pushes fields with the handle form +(`activateSpanForCorrelation`) should still close its scope in the same continuation that opened it. + +## Instrumenting a pipeline + +Both constructors take an options bag that the built pipeline carries into every call: + +```typescript +import { + PipelineBuilder, + createInstrumentationBundle, + standardResilience, + type PipelineOptions, + type Runtime, + type Tracer, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +declare const otelTracer: Tracer; // whatever your tracing library hands you + +const instrumented: PipelineOptions = { + instrumentation: createInstrumentationBundle(() => otelTracer), + operationName: 'GetUser', +}; + +// `StandardResilienceOptions` extends `PipelineOptions`, so the preset takes the same two fields +// beside its per-pillar ones. +export const preset: Runtime = standardResilience(fetchTransport(), instrumented); +export const handBuilt: Runtime = new PipelineBuilder( + fetchTransport(), + instrumented, +).build(); +``` + +- **`instrumentation`** is the bundle every context of the call carries, and its `tracerFactory` is + what opens spans. `send()` asks it for `'http.client.operation'` once per call and opens **one** + span there (`OBS-29`), outside every pillar — a retry attempt and a redirect hop stay inside it. The + LOGGING pillar asks again per transmission, and those spans are its children. Omitted, the pipeline + carries the no-op bundle (`CTX-15`) and opens no span at all. +- **`operationName`** is `CTX-16`'s advisory label. It is carried unchanged across every promotion, + readable from a custom step as `ctx.context.operationName`, and used to name the tracer the LOGGING + pillar asks for. It never influences the request, the dispatch decision, or the store key. + +Both are per-pipeline, not per-call: build a second pipeline for a second operation name. +`PipelineBuilder.seedFrom(runtime, 'flatten')` carries them over, since the flattened builder replaces +the runtime it seeded from; `'nest'` does not need to, because the seeded runtime is still there, +driving its own contexts as the terminal transport. + +## The four shipped pillars + +| Pillar | Factory | Key settings | +|---|---|---| +| Retry | `retryStep(options?)` | `maxAttempts`, `retryableStatuses`, `totalTimeoutMs`, `attemptHeaderName`, backoff (`initialDelayMs`, `multiplier`, `maxDelayMs`, `jitter`, `fixedDelayMs`), injectable `clock`/`random` | +| Redirect | `redirectStep(overrides?)` | `maxHops`, `allowedMethods`, `allow303`, `allowSchemeDowngrade`, `locationHeader`, `predicate` | +| Auth | `authStep(settings)` | `credentials`, `tiers`, `challengeHook`, `bearerMarginMs` — see [`auth.md`](./auth.md) | +| Logging | `loggingStep(settings?)` | `granularity`, `configKey`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | + +Retry and redirect are worth a few notes each, because both surprise people: + +- **What retry throws is the last attempt's own error.** The class you catch does not depend on how + many attempts ran: a refused connection is a `TransportFailureError` whether `maxAttempts` was 1 or + 3, and an abort that lands during a backoff wait is a `CancellationError` (`XCUT-1`). The earlier + attempts are not thrown away — `retryAttempts(caught)` returns them, oldest first, with the error + you passed in excluded from its own trail (`RETRY-34`): + + ```typescript + import { + retryAttempts, + TransportFailureError, + type Request, + type Runtime, + } from '@dexpace/core'; + + declare const runtime: Runtime; + declare const request: Request; + + export async function send(): Promise { + try { + await runtime.send(request); + } catch (error) { + if (error instanceof TransportFailureError) { + for (const prior of retryAttempts(error)) { + console.error('an earlier attempt failed:', prior); + } + } + throw error; + } + } + ``` + + One entry per attempt that failed *before* the one you caught — which is not an attempt count, so + resist writing `length + 1`. The surfaced error is an attempt's own only when it came from one, and + sometimes it did not: a cancellation or timeout the engine observes between attempts is synthesized + at that gate, and so is a failure from stamping the attempt header, which runs before the request + goes out. On those paths the trail already covers every send. Narrowing the catch does not help — + a timeout signal is mapped to `TransportFailureError`, the same class a real send failure raises. + + The trail is a side table keyed by the error, not a property on it, so nothing is added to an + object you may not own; an error that never went through a retry loop answers with an empty list. + Until 2026-09-05 the pillar wrapped its terminal failure in a `SuppressedError` instead, which made + the surfaced class a function of the attempt budget. +- **Retry pacing honours the server.** `Retry-After`, `X-RateLimit-Reset` and friends are parsed in a + fixed precedence and win over computed backoff. Every computed delta is clamped to a 365-day + ceiling that `RETRY-18` mandates — so a server that sends `X-RateLimit-Reset` in milliseconds + instead of epoch seconds parks the retry for a year, which is indistinguishable from a hang. Set + `totalTimeoutMs` if that matters to you; it is opt-in and `undefined` by default + (`docs/work/mvp/2026-09-04-open-items-dissolution.md` P4). +- **Retry hands back a live response.** Any response the engine *discards* is closed. The response + that ends the loop — attempt cap reached, budget spent, status not retryable — is returned live and + unread. Ownership transfers to you (`docs/work/mvp/2026-09-04-open-items-dissolution.md` P7). +- **Redirects are never followed by the transport.** Both shipped transports pin redirects off, so + the pipeline is the single redirect authority (`TRANSPORT-1`/`TRANSPORT-2`). +- **A non-replayable body ends a redirect.** `PIPE-40` and `REDIR-22` disagree about what should + happen; this port closes the response and throws (`docs/work/mvp/2026-09-04-open-items-dissolution.md` G1). + +### Turning logging on from the environment + +`loggingStep()` with no `granularity` resolves one from layered configuration (`OBS-35`), and the +default is `'none'`. What it reads is the process-wide configuration slot — **which starts empty**. +`CFG-13` makes that slot last-write-wins with no ambient default, so nothing in this SDK reads +`process.env` until you say so, and `DEXPACE_LOG_LEVEL=headers` on its own changes nothing: + +```typescript +import { + defaultConfiguration, + loggingStep, + setGlobalConfiguration, + standardResilience, + type Runtime, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +// Once, at start-up. `defaultConfiguration()` is the production wiring: its environment seam reads +// the live `process.env` on every lookup, and its property seam finds nothing (Node has no ambient +// key/value store distinct from the environment). +setGlobalConfiguration(defaultConfiguration()); + +// Now `DEXPACE_LOG_LEVEL=headers` reaches the step. `configKey` renames the variable it reads — +// `OBS-35` says an SDK must not bake a key name in, and `DEXPACE_LOG_LEVEL` is only the default. +export const client: Runtime = standardResilience(fetchTransport(), { + logging: {configKey: 'ACME_HTTP_LOG_LEVEL'}, +}); +export const step = loggingStep({configKey: 'ACME_HTTP_LOG_LEVEL'}); +``` + +Parsing is tolerant either way: `' Headers '` and `'HEADERS'` both resolve to `'headers'`, and an +absent, empty or unrecognised value falls back to `'none'`. An explicit `granularity` in the settings +wins over both, and is the right choice for a library that does not want to read a host application's +environment at all. + +Body-preview capture is best-effort and says so when it fails: a request body that cannot be probed +or a response body that errors mid-drain emits `http.instrumentation.bodyCaptureFailed`, carrying the +direction and the cause, and the request still completes (`OBS-20`). Like every +`http.instrumentation.*` diagnostic it emits at `verbose`, so a logger that filters that level will +not show it. + +## Testing a pipeline + +Nothing here needs a socket. A `Transport` is two methods, so the test double is a literal: + +```typescript +import {Protocol, Response, Status, type Transport} from '@dexpace/core'; + +const alwaysOk: Transport = { + send: async request => + Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(null) + .build(), + close: async () => undefined, +}; +``` + +Timing is testable without waiting: `retryStep({clock, random})` takes both seams, so a test drives +the backoff schedule deterministically. `loggingStep({clock, logger, meter})` takes the same shape. + +Core does carry a scripted `FakeTransport` under `src/testing/`, but it is `@internal` and **not** +exported from the barrel — it exists for core's own multi-attempt tests. A consumer writes the +five-line literal above. + +`@dexpace/transport-conformance` is the other half of this, for transport authors rather than +pipeline authors: see [`write-a-transport.md`](./write-a-transport.md). diff --git a/docs/sdk-documentation/quality-gates.md b/docs/sdk-documentation/quality-gates.md new file mode 100644 index 0000000..c178ad9 --- /dev/null +++ b/docs/sdk-documentation/quality-gates.md @@ -0,0 +1,108 @@ +# Quality gates + +Twenty named steps across two CI jobs, every one of them blocking +([`.github/workflows/ci.yml`](../../.github/workflows/ci.yml)). Seventeen in the `ci` job, three in a +`node-conformance` matrix that runs after it. `bun run test` passing is not evidence that the work is +done; the whole set is. + +## Running them + +```bash +node .claude/skills/ci-preflight/run-ci.mjs --clean +``` + +That is the command. `--clean` sweeps every `dist/` and `*.tsbuildinfo` first, so the run starts from +the tree CI checks out rather than a warm one, and it pins every step to `.bun-version`'s Bun via +mise. Both matter: Phase 8a's transport rows passed on Bun 1.4.0 and failed three ways on the pinned +1.3.14, and a missing `build:deps` entry is invisible against a warm `dist/`. + +## The `ci` job, in order + +| Step | Command | What it protects | +|---|---|---| +| Install | `bun install --frozen-lockfile` | The lockfile is authoritative | +| Knowledge-corpus structure | `verify:knowledge-structure` | `docs/knowledge/`'s two trees stay separate. First, because it is pure Node over Markdown — a corpus mistake reports in seconds | +| Typecheck | `typecheck` | `tsc --noEmit` per package, over twelve projects | +| Lint | `lint` | `gts lint` — formatting **and** type-aware rules, both fatal | +| Build | `build` | Every package's `dist/` | +| Test | `test --coverage` | Both Bun test trees, one coverage report, 80% line floor | +| Gate self-tests | `test:scripts` | The gates' own logic, on `node --test` | +| API surface | `api` | All 9 committed `etc/*.api.md` reports match | +| Package health | `lint:publish` | `publint` + `attw` over every built package | +| Dual consumption | `verify:dual-consumption` | Plain `node` imports each built package and exercises it | +| Consumer types | `verify:consumer-types` | The built `.d.ts` compiles on the declared `lib` with `types: []` | +| SEAM-1 | `verify:seam-1` | Zero runtime dependencies in **every** package, plus the `@dexpace/core` peer rule | +| SSE-37/38 | `verify:sse-37` | No serde dependency and no reconnect path in core SSE | +| Runtime floor | `verify:runtime-floor` | `tsconfig` target and `engines.node` agree | +| Test partition | `verify:test-partition` | The five files that keep the two `tests/` suites apart | +| Reproducible build | `verify:reproducible-build` | Two clean builds of one tree agree, `dist/` and tarball (`NFR-12`) | +| Dependency audit | `audit` | `bun audit --audit-level=high --prod` | + +Three of those deserve their reasons stated, because each exists because something silently broke. + +**`test:scripts` tests the gates themselves.** A gate whose own logic degrades — a bad glob, a +swallowed assertion — still exits 0, so nothing else in the run would notice. It became blocking in +Phase 10, and the proof it should have been is that `knowledge.test.mjs` had been failing on `main` +since `36c3f96` with nobody noticing. + +**`verify:reproducible-build` runs after every step that needs `dist/`, deliberately.** It sweeps +every `dist/` and rebuilds the workspace twice, so it would otherwise pull the rug from under any +step above that resolves a workspace package by name. It is not the last step — `Dependency audit` +follows it (`.github/workflows/ci.yml:93` then `:96`), and can, because `bun audit` reads manifests +rather than build output. + +**`verify:seam-1` covers every package, not core alone.** `NFR-2` is the reason: each optional +capability is core plus at most one external library, and the gate is what makes reaching for a small +utility a red build rather than a code-review argument. + +## The `node-conformance` job + +Install, build, then `bun run test:node` under real Node — as a matrix over `engines.node`'s declared +floor (`20.3.0`) and `lts/*`, with `fail-fast: false`, because "broken on the floor" and "broken on +LTS" are different diagnoses. + +It exists because Bun's Web Streams, `AbortSignal` and `Uint8Array` are an independent implementation +of Node's, and `packages/core/src/io/` is where they diverge. **A change to a runtime-divergent +surface adds a case there, not only to `bun run test`.** + +## Two test trees, and the rule between them + +``` +packages/*/src/*.test.ts colocated unit tests bun +tests/conformance/xcut/ cross-cutting conformance bun +tests/node-conformance/ runtime conformance node --test, against dist/ +``` + +`bun run test` is the **only** command that runs both Bun trees; it passes `./packages ./tests` +explicitly. A bare `bun test` silently runs only the first, because `bunfig.toml`'s +`[test] root = "packages"` governs discovery — and reports green over a suite it never opened, with no +"0 files matched" to notice. + +`tests/node-conformance/` must never run on Bun; that is the only reason the tree exists. One key +holds the line — `bunfig.toml`'s `[test] pathIgnorePatterns` — and Bun accepts an unknown `[test]` key +without complaint, so a typo gives no warning and no failure. Measured on pinned Bun 1.3.14: with the +key, 164 files; without it, 178, of which thirteen pass silently and the fourteenth trips an unrelated +timer assertion. Treat that exit code as an accident, not a control. `verify:test-partition` checks +the key's name and the four other files that must agree with it. + +## Gates that are not in CI, on purpose + +| Command | Why not | +|---|---| +| `bun run knowledge:drift` | 16 of the 47 corpus sources are a sibling styleguide repository no CI checkout has. Drift is normal and the fix is a re-harvest, not a red build | +| `bun run shrink-test` | Runs inside the default build via `@dexpace/shrink-test`, not as its own step | +| The `housekeeping` skill's probe | A hand-run maintenance tool, like `test:scripts` was before Phase 10 promoted it | + +## Per-package obligations + +- **A changeset** for any consumer-facing change: `bun run changeset`, never `bunx changeset`. The + wrapper renames the generated file to `YYYY-MM-DD-.md`. +- **A regenerated API report** after changing a package's exports: `cd packages/ && bun run + api:local`, then commit it. `bun run api` is what CI diffs. +- **A TSDoc block with `@public`** on anything the barrel exports, plus `@throws` naming each + catchable error class. `api-extractor` records an undocumented export as `(undocumented)` in the + committed report, which makes the omission a reviewable diff. +- **`// SPDX-License-Identifier: MIT` on line 1** of every source file (`NFR-13`). +- **A requirement-ID citation** in every test file's header comment. +- **A reason on every `eslint-disable`** (`eslint-comments/require-description`, wired for `NFR-7`). + Suppressing a rule without a stated reason fails lint. diff --git a/docs/sdk-documentation/write-a-paging-strategy.md b/docs/sdk-documentation/write-a-paging-strategy.md new file mode 100644 index 0000000..5913cbf --- /dev/null +++ b/docs/sdk-documentation/write-a-paging-strategy.md @@ -0,0 +1,192 @@ +# Write a paging strategy + +A strategy is one method: + +```typescript +interface PaginationStrategy { + parse(response: Response, template: Request): Promise>; +} + +interface PageInfo { + readonly items: readonly T[]; + readonly nextRequest: Request | undefined; // undefined ends the walk +} +``` + +Given the page that just arrived and the request that fetched it, produce this page's items and the +request that fetches the next one. `undefined` for `nextRequest` is how a walk ends — there is no +separate "done" flag to keep consistent with it. + +**`template` is not the request the walk started from.** The glossary calls it "the original request +template", but the engine passes the request it sent for *this* page and then makes your `nextRequest` +the following hop's template — it advances with the walk +(`packages/core/src/pagination/paginator.ts:165,213`, and the contract on +`PaginationStrategy.parse` at `strategy.ts:10-15`). Read the parameter as "the request to derive the +next one from". Where you specifically want the URL the response actually came from — after a redirect +or a step's rewrite — use `response.request.url`, which is what `pageNumberStrategy` reads its current +page number from. + +## Three ship already + +Reach for a custom strategy only when none of these fits. + +```typescript +import {cursorStrategy, linkHeaderStrategy, pageNumberStrategy} from '@dexpace/core'; + +interface Thing { + readonly id: string; +} +declare function parseItems(payload: string): readonly Thing[]; + +// ?cursor=, taken from the payload +cursorStrategy({ + extract: async r => { + const {items, next} = JSON.parse(await r.text()) as { + items: readonly Thing[]; + next: string | null; + }; + return {items, cursor: next}; + }, + parameterName: 'cursor', // default +}); + +// RFC 8288 Link: <...>; rel="next" +linkHeaderStrategy({extract: async r => parseItems(await r.text()), headerName: 'Link'}); + +// ?page=1,2,3… +pageNumberStrategy({extract: async r => parseItems(await r.text()), startPage: 1}); +``` + +`extract` is handed the live response. `Response` has `text()` and `bytes()`, not `json()` — this is +the SDK's own model, not the WHATWG one. + +Each takes an `extract` that reads the payload and lets the strategy own the URL manipulation. + +## Driving one + +```typescript +import { + Paginator, + Request, + type PaginationStrategy, + type Transport, +} from '@dexpace/core'; + +interface Thing { + readonly id: string; +} +declare const client: Transport; +declare const strategy: PaginationStrategy; +declare const signal: AbortSignal; + +const paginator = new Paginator({ + transport: client, // a Runtime is a Transport, so a full pipeline works here + initialRequest: Request.newBuilder().url('https://api.example.com/v1/things').build(), + strategy, + maxPages: 50, + signal, +}); + +for await (const thing of paginator.items()) { /* item by item */ } +for await (const page of paginator.pages()) { /* page by page */ } +``` + +`items()` and `pages()` each build a **fresh** generator per call (`PAGE-8`), so two iterations are +independent walks, not two views of one. That is also why `@dexpace/rx`'s `pageItems$`/`pages$` are +cold and repeatable while its SSE observables are not. + +## The five rules + +**1. Take everything you need from the response before your promise settles** (`PAGE-5`). The +response you are handed is live and single-use; the engine may close it the moment `parse` resolves. +Read the items and the cursor first, retain nothing past the call, and never hand the `Response` +itself to a caller. + +**`parse` returns a `Promise`, and that is deliberate — do not "fix" it toward the literal +requirement.** `PAGE-5` says the strategy reads what it needs *synchronously* inside `parse`. Node has +no synchronous body read, so the literal form is unimplementable here and the discipline the clause +protects — single use, nothing retained — is what the async signature preserves. Recorded in the `PAGE-5` +"synchronously inside parse" row of +[`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md), +where the dissolved deferral register's rows went — precisely so an async signature does not later read as an oversight. Every shipped strategy's `extract` above is `async` for the same reason. + +**2. Build `nextRequest` from the template, not from scratch.** The template carries the headers, auth +tier and options the walk was started with, and it is the previous hop's request rather than page one's, +so deriving from it accumulates the walk's state instead of re-deriving it. A next request built from +scratch loses all of that. + +```typescript +const next = template + .newBuilder() + .url(withQueryParam(template.url, 'cursor', cursor)) + .build(); +return pageInfo(items, next); +``` + +`pageInfo(items, nextRequest?)` is the `PageInfo` factory. Use `QueryParams` for the URL work — see +[`http.md`](./http.md) for why not `URLSearchParams`. + +**3. A page is closed before its items are yielded** (`PAGE-11`). The engine does this for you. It is +worth knowing because it means your `extract` is the **only** place the response body is readable, and +because `sdk-design-nodejs/07` §7.1's illustrative snippet has it backwards — closing after yielding — +which is an erratum recorded in `docs/knowledge/notes/pagination.md` and `docs/work/mvp/2026-09-04-open-items-dissolution.md` J1. + +**4. Terminate.** Returning a `nextRequest` equal to the one just fetched is an infinite walk. +`maxPages` on `PaginatorInit` is the backstop, not the design. Loop detection is not the paginator's +job. + +**5. Always return a well-formed `PageInfo`** (`PAGE-4`). `items` must be an array — an empty one is +fine and is a perfectly valid non-terminal page — and `nextRequest === undefined` is the **single, +exclusive** end-of-stream signal. A `PageInfo` that is itself `null` or `undefined`, or whose `items` +is either, is a programmer error and the engine treats it as one: it closes the response and throws +an assertion naming the invariant you broke. It does **not** end the walk quietly, because "the +strategy forgot to `return`" and "the server ran out of pages" must not look the same from the +outside. Use `pageInfo(items, next?)` and this cannot happen; the check exists because `parse` +crosses a seam, where an `any`-typed decode or a trusted server field can produce a shape the types +say is impossible. + +Terminating and failing are different acts. To *end* the walk, return `pageInfo(items)` with no next +request. To *fail* it, throw — the engine closes the response and your error reaches the consumer +unwrapped (`PAGE-13`, `PAGE-28`). + +`PaginationError` is reserved for engine misuse and precondition violations — not for "the server +returned a page I did not understand", which is your `extract`'s error to raise. + +## The fetcher form + +When the API is already wrapped in functions rather than reachable as requests, skip +`PaginationStrategy` entirely: + +```typescript +import {paginateWithFetchers, type FetcherPage, type PagingOptions} from '@dexpace/core'; + +interface Thing { + readonly id: string; +} +declare function firstPage(options: PagingOptions): Promise | undefined>; +declare function nextPage(key: string, options: PagingOptions): Promise | undefined>; + +for await (const page of paginateWithFetchers({ + first: async options => firstPage(options), + next: async (key, options) => nextPage(key, options), + maxPages: 20, +})) { + console.log(page.items); +} +``` + +`first` and `next` return a `FetcherPage` — a `Page` plus either a `continuationToken` or a +`nextLink` — or `undefined` to end the walk. This is the adapter for a generated client whose +pagination is already a pair of methods. + +## Disposal + +`Page` has a `close()`. It does **not** support `await using`: `Symbol.asyncDispose` arrived in Node +20.4 and this project's floor is 20.3, so the disposal member is installed only when the symbol +exists and is never declared in the `.d.ts`. Declaring it anyway would be a type that lies on the +supported runtime, which `NFR-10` forbids. `close()` is the teardown on every runtime, and +`open-items.md`'s Section D row [`await using` support](../work/mvp/2026-09-04-open-items-dissolution.md#d-nfr-10-await-using) +records the decision with the four reasons the floor does not move instead. + +Within a `Paginator` walk the engine closes each page for you; `close()` matters when you hold a +`Page` yourself. diff --git a/docs/sdk-documentation/write-a-response-handler.md b/docs/sdk-documentation/write-a-response-handler.md new file mode 100644 index 0000000..7611147 --- /dev/null +++ b/docs/sdk-documentation/write-a-response-handler.md @@ -0,0 +1,160 @@ +# Write a response handler + +A response handler turns a `Response` into your model. Core ships three shapes; write your own when +none fits. + +## The three shipped shapes + +All three come from `@dexpace/core`. + +| Shape | Use when | +|---|---| +| `decodeSuccessResponse(response, deserializer, target)` | The common case: decode 2xx, raise on 4xx/5xx | +| `decodeResponse(response, deserializer, target)` | You want the error body decoded too — an RFC 7807 problem document, say | +| `new TypedResponse(response, parse)` | You need the status, headers and request *alongside* the value, decoded lazily | + +`target` is `{schema, typeName?}`: the runtime type witness plus an optional label that names it in +an error message. It travels as one object because a schema and its label describe one thing, and +because `(response, deserializer, schema, typeName)` is four parameters. + +```typescript +import {decodeSuccessResponse, type Response, type Schema} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; + +interface Thing { + readonly id: string; +} +declare const ThingSchema: Schema; // any object with parse(input: unknown): Thing + +const serde = jsonSerde(); + +export async function readThing(response: Response): Promise { + return decodeSuccessResponse(response, serde.deserializer, { + schema: ThingSchema, + typeName: 'Thing', + }); +} +``` + +**Import `Response` from `@dexpace/core`, always.** A bare `Response` in a signature resolves to the +DOM global under a `lib` that includes `"DOM"`, and the two are unrelated types — this SDK's +`Response` has `status: Status`, `close()` and `request`, and no `json()`. The mistake typechecks +until you try to pass one. + +`decodeSuccessResponse` delegates to `decodeResponse` on a 2xx and to `toHttpError` on a **4xx or 5xx**, +so a `404` becomes an `HttpStatusError` carrying `status`, a replayable `body()` and a non-consuming +`preview()` — a bounded in-memory copy at the 1 MiB `HTTP-52`/`BODY-30` cap, taken before the response +was closed. There is no headers accessor on it; read anything else you need off the response before you +hand it over. Any *other* non-2xx — a `1xx`, or an unfollowed `3xx` such as a `304` — is neither decoded +nor mapped: the response is closed and a `DeserializationError` is raised whose message leads with the +status code and which carries `ETag` and `Location` as fields, so conditional and redirect context +survives the close (`SERDE-28`). + +## What the shipped handlers guarantee, and what you must reproduce + +**1. The response is closed on every path.** Success, missing body, codec failure, stream failure. A +handler that returns early on one branch strands a connection. + +**2. A close failure never displaces the real failure.** This is the part that looks like a +one-liner and is not: + +```typescript +// WRONG: when close() rejects while an error is already in flight, the finally's +// rejection REPLACES it — the caller is told their connection dropped when in fact +// their payload was malformed. +try { + return await work(); +} finally { + await response.close(); +} +``` + +The rule the shipped handlers follow (`RECOV-12`): + +| Work | Close | Result | +|---|---|---| +| threw | ok | the work error propagates | +| threw | threw | the work error stays **primary**; the close error attaches as `suppressed` | +| ok | threw | the close error propagates — it is the only failure there is | + +The suppression wrapper's `name` is `'SuppressedError'`, `.error` is the primary and `.suppressed` the +release failure. **`instanceof SuppressedError` is not a valid test**: the class is absent on this +project's declared Node floor and a structurally identical stand-in is built there instead. Test the +shape, or read `.error` unconditionally. + +**A failed release is the only thing in this SDK that builds one.** Every site that constructs the +pairing — the response chain, the redirect and auth and retry pillars, the serde handlers, the SSE +stream, the paginator — is doing this one job: a `close()` that threw while an error was already in +flight. It is a genuine two-value shape, which is why it fits. + +The retry pillar used to build one for a *second* reason, folding N prior attempt errors into a +nested chain of pairs so that what you caught after three attempts was a wrapper rather than the +failure. It no longer does: it surfaces the last attempt's error unwrapped and records the earlier +ones beside it, read back with `retryAttempts()` (see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars)). +So if you catch a `'SuppressedError'` from this SDK, `.suppressed` is a teardown failure, never an +earlier attempt. + +**3. Only payload failures are re-typed.** `SERDE-12`: a malformed body or a shape mismatch becomes a +`DeserializationError` with the original chained; a genuine stream failure propagates untouched, +because re-wrapping it would tell a caller their payload was malformed when their socket dropped. + +**4. `isSerdeError(e)` is the supported discriminator**, not `instanceof` against a stream-error class +— that class is not public surface. + +### The limit of that discriminator, stated plainly + +Every error already in this SDK's typed tree passes through untouched, so a stream failure raised by +core's own I/O layer is always recognizable. A **foreign** one is not. `decodeResponse` hands the live +stream to the codec and never reads it, so at the catch a transport's raw error is indistinguishable +from a non-conforming codec leaking one — and since `SERDE-27` requires a codec failure to surface as +a serde exception, the untyped case is wrapped. + +In practice that means a `fetch`/undici `TypeError('terminated')`, a hand-built `ReadableStream` +errored with a bare `Error`, or an aborted body (`DOMException` named `'AbortError'`) is reported as a +`DeserializationError`. Read `isSerdeError(e) === true` as "payload **or** foreign stream", not as +proof of a payload failure. Fixing it needs the transport to tag its stream errors. + +## Writing one + +```typescript +import {DeserializationError, type Response} from '@dexpace/core'; + +export async function readNdjson( + response: Response, + parseLine: (line: string) => T, +): Promise { + try { + const text = await response.text(); // text() closes the response itself (BODY-16) + return text + .split('\n') + .filter(line => line.length > 0) + .map(parseLine); + } catch (cause) { + throw new DeserializationError('could not decode the NDJSON payload', {cause}); + } +} +``` + +`response.text()` and `.bytes()` close the response whether the read succeeds or not, which is why +this handler needs no `finally`. A handler that reads `response.body` directly does, and then owes +rule 2's ordering. + +**`decodeResponse` never buffers.** It hands the live body stream to `Deserializer.deserializeFrom`, +which reads it to EOF. Whether the codec buffers is the codec's business — `@dexpace/codec-json` +must, because `JSON.parse` has no incremental form, and that is ledgered. A handler that needs to act +on the payload *before* it ends reads `response.body` itself, as above. + +## Two more things a response can be + +**Server-Sent Events.** `sseStreamFrom(response)` yields `SseEvent`s; `typedSseStream(stream, mapper)` +decodes them into your models. The mapper returns `mapperValue(v)`, `MAPPER_SKIP` or `MAPPER_DONE`. +Core's SSE parser has **no** serde dependency and no reconnect path, and +`bun run verify:sse-37` is a blocking CI step that proves both. + +**A page.** See [`write-a-paging-strategy.md`](./write-a-paging-strategy.md). + +## Ownership, once more + +The pipeline never closes a response it hands you. `toHttpError` and the two `decode*` handlers do, +because they read it. A handler you write must decide which it is and say so in its own TSDoc — that +is the single fact a caller cannot recover from the signature. diff --git a/docs/sdk-documentation/write-a-serde.md b/docs/sdk-documentation/write-a-serde.md new file mode 100644 index 0000000..5893836 --- /dev/null +++ b/docs/sdk-documentation/write-a-serde.md @@ -0,0 +1,273 @@ +# Write a serde + +A serde is a wire codec behind three interfaces. It is bigger than it first looks: the encode half +has **four allocation profiles** and the decode half **two**, and an implementor owes all six +(`SEAM-20`, `SERDE-3`/`SERDE-4`/`SERDE-5`/`SERDE-6`). + +```typescript +interface Serde { + readonly serializer: Serializer; + readonly deserializer: Deserializer; + readonly mediaType: string; +} + +interface Serializer { + serialize(value: unknown): Uint8Array; // fresh buffer + serializeToString(value: unknown): string; // fresh string + serializeInto(value: unknown, target: Uint8Array, offset?: number): number; // caller's buffer + serializeTo(value: unknown, sink: WritableStream, + options?: {signal?: AbortSignal}): Promise; // caller's sink +} + +interface Deserializer { + deserialize(data: Uint8Array, target: DecodeTarget): T; + deserializeFrom(source: ReadableStream, target: DecodeTarget, + options?: {signal?: AbortSignal}): Promise; +} + +interface Schema { + parse(input: unknown): T; +} +``` + +No encode method takes a `Schema` — encoding has the value in hand and needs no witness. + +`@dexpace/core` ships **no** codec. `@dexpace/codec-json` is the reference implementation and a peer +of core, never a dependency of it — which is what forced the seam to be public in the first place: a +separate package can only reach core through its published entry point. + +## Schema is the type witness + +`Schema` is one method, `parse(input: unknown): T`. Zod, Valibot, ArkType, a hand-written +predicate — anything with a `parse` satisfies it, and nothing registers. This is `SEAM-21`'s type +witness: TypeScript erases generics, so a deserializer cannot reflect on `T`; the schema **is** the +runtime carrier of the type, and it is also the source of the static one, so there is no separate type +argument to keep in sync. + +`typeName` is diagnostics only. It never selects behaviour; it makes a `DeserializationError` +message name the thing that failed to parse. + +## The minimum + +All six methods, no shortcuts. This is the shape, not a sketch: + +```typescript +import { + DeserializationError, + SerializationError, + type DecodeTarget, + type Serde, +} from '@dexpace/core'; + +const TEXT = new TextEncoder(); + +/** + * Settle when `operation` settles, or as soon as `signal` aborts — whichever comes first, with the + * caller's own `reason` surfaced verbatim. `throwIfAborted()` alone cannot interrupt a `read()` or + * `write()` that never resolves, which is the case that leaves a caller's stream locked forever. + */ +const raceAbort = async ( + operation: Promise, + signal: AbortSignal | undefined, +): Promise => { + if (signal === undefined) return operation; + signal.throwIfAborted(); + let onAbort = (): void => undefined; + const aborted = new Promise((_resolve, reject) => { + onAbort = (): void => { + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + try { + // The loser of the race keeps `Promise.race`'s own handler, so a `read()` that rejects after + // the lock is released never becomes an unhandled rejection. + return await Promise.race([operation, aborted]); + } finally { + signal.removeEventListener('abort', onAbort); + } +}; + +export function csvSerde(): Serde { + const encode = (value: unknown): string => { + if (!Array.isArray(value)) { + throw new SerializationError('the csv serializer takes an array of rows'); + } + return value.map(row => String(row)).join('\n'); + }; + + const decode = (text: string, target: DecodeTarget): T => { + const rows: unknown = text.split('\n'); + // SERDE-13: reject a wire null before the schema runs, unless the target says it admits one. + if (rows === null && target.admitsNull !== true) { + throw new DeserializationError( + `wire null cannot be decoded into the non-null target ${target.typeName ?? 'the target type'}`, + ); + } + try { + return target.schema.parse(rows); + } catch (cause) { + throw new DeserializationError( + `could not decode ${target.typeName ?? 'the target type'}`, + {cause}, + ); + } + }; + + return { + mediaType: 'text/csv', + serializer: { + serialize: value => TEXT.encode(encode(value)), + serializeToString: encode, + serializeInto(value, target, offset = 0) { + const bytes = TEXT.encode(encode(value)); + // A plain RangeError, deliberately: SERDE-4 says a buffer that does not fit is the + // caller's arithmetic error, not an encoding failure, so it is NOT a SerializationError. + if (offset < 0 || offset + bytes.length > target.length) { + throw new RangeError('the encoded payload does not fit at that offset'); + } + target.set(bytes, offset); + return bytes.length; + }, + async serializeTo(value, sink, options) { + options?.signal?.throwIfAborted(); // before the lock: an aborted call leaves the sink free + const writer = sink.getWriter(); // TypeError if contended — a programmer error, not re-typed + try { + // Raced, not just checked: a slow sink parks this write, and the abort must reach it. + await raceAbort(writer.write(TEXT.encode(encode(value))), options?.signal); + } finally { + writer.releaseLock(); // never close or abort: the caller owns the sink (SERDE-3) + } + }, + }, + deserializer: { + deserialize: (data, target) => decode(new TextDecoder().decode(data), target), + async deserializeFrom(source, target, options) { + options?.signal?.throwIfAborted(); // before the lock: an aborted call never takes one + const reader = source.getReader(); + const chunks: Uint8Array[] = []; + try { + for (;;) { + // Raced, not checked between chunks: a source that stalls mid-body parks the loop + // inside `read()`, where a between-chunks check never runs again. + const {done, value} = await raceAbort(reader.read(), options?.signal); + if (done) break; + chunks.push(value); + } + } finally { + reader.releaseLock(); // never cancel: the caller owns the source (SERDE-3) + } + const joined = new Uint8Array(chunks.reduce((n, c) => n + c.length, 0)); + let at = 0; + for (const chunk of chunks) { + joined.set(chunk, at); + at += chunk.length; + } + return decode(new TextDecoder().decode(joined), target); + }, + }, + }; +} +``` + +Seven rules, all visible above: + +1. **Raise `SerializationError` / `DeserializationError`, never a raw error** — with one stated + exception: `serializeInto`'s out-of-range or does-not-fit case is a plain `RangeError` with no + chained cause (`SERDE-4`). Both serde errors descend from `DexpaceError`, and `isSerdeError(e)` + narrows the union. +2. **Always pass `{cause}`.** The underlying parser's message is what a caller actually debugs with. +3. **Never take ownership of a caller's stream** (`SERDE-3`). `serializeTo` does not close or abort + the sink; `deserializeFrom` does not cancel the source. Release your lock and leave the resource + to whoever opened it — including on the failure path. +4. **A contended stream is a plain `TypeError`**, not re-typed. Two writers on one sink is a + programmer error, not an encoding failure. +5. **A wire failure propagates unwrapped** (`SERDE-12`). Re-wrapping a write or read failure as a + serde exception tells a caller their payload was malformed when their socket dropped. The rule is + direction-agnostic: it applies to `serializeTo` as much as to `deserializeFrom`. +6. **A wire `null` decoded into a non-null target MUST throw** `DeserializationError` naming that + target, on **every** entry point (`SERDE-13`), never return a `null` that detonates at a later + field access. The fallback label is the literal `'the target type'`; each codec repeats it, + because `SEAM-1` leaves core with no exported constant to share. +7. **An abort must race the pending operation, not sit between two of them** (`SERDE-3`). The seam + promises that "an aborted call never leaves the caller's source locked", and a + `throwIfAborted()` between chunks cannot keep it: a source that stalls mid-body parks the drain + inside `read()`, so the call never settles and the lock is never released. Race each pending + `read()`/`write()` against the signal, remove the listener in a `finally`, then release the lock + as usual. Releasing a reader with a read still outstanding is legal on every supported runtime + and does unlock the stream — the outstanding read rejects, differently per runtime + (`AbortError` on Bun 1.3.14, `TypeError: Invalid state: Releasing reader` on Node 20.3 and 26, + measured 2026-09-05), which is why the caller must see the signal's `reason` instead. + `@dexpace/codec-json` holds **one** listener for the whole drive rather than one per chunk; the + example above takes the simpler per-operation form. + +`mediaType` is the default `Content-Type` — `serdeBody(value, serde)` reads it, and a caller may +override per body. + +## Using one + +```typescript +import {serdeBody} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; + +const serde = jsonSerde(); +const body = serdeBody({name: 'ada'}, serde); // Content-Type: application/json +``` + +See [`write-a-response-handler.md`](./write-a-response-handler.md) for the decode side. + +## The tri-state problem + +This is the part a JSON codec gets wrong by default, and the reason `@dexpace/codec-json` exists as a +reference rather than as a five-line `JSON.parse` wrapper. + +A PATCH request has **three** meanings for a field, and JavaScript gives you two: + +| Intent | Wire | JavaScript | +|---|---|---| +| Leave it alone | key absent | `undefined` | +| Clear it | `"x": null` | `null` | +| Set it | `"x": 1` | `1` | + +`JSON.stringify` drops `undefined` keys, so the first two collapse the moment anything round-trips +through an optional property. `Tristate` is core's answer — a branded discriminated union of +`'absent' | 'null' | 'present'`: + +```typescript +import {absent, foldTristate, nullValue, present} from '@dexpace/core'; + +const patch = { + name: present('ada'), + nickname: nullValue(), // emits "nickname": null + bio: absent(), // omits the key entirely +}; + +foldTristate(patch.name, { + onAbsent: () => 'unchanged', + onNull: () => 'cleared', + onPresent: value => `set to ${value}`, +}); +``` + +`isPresent`, `isNull`, `isAbsent` and `valueOrNull` are the narrowing helpers; `ofNullable` lifts a +`T | null | undefined`. The `TRISTATE_BRAND` symbol is what makes `isTristate` reliable against a +caller-shaped object literal. + +`jsonSerde()` wires the encoding side **on by default** — `jsonSerde({tristate: false})` is the only +way out — and exports `tristate(schema)` and `tristateObject(shape)` to lift your schemas, plus +`tristateReplacer` for use with a bare `JSON.stringify`. + +If your format has its own three-state encoding, map `Tristate` onto it. If it genuinely has only +two, say so in the README rather than silently collapsing absent into null. + +## What core's serde seam does not do + +- **No default codec, and no fallback to JSON.** A pipeline with no serde configured serializes + nothing. +- **No content negotiation.** `mediaType` is a default, not a negotiation. +- **No incremental decode.** `deserializeFrom` reads its source to EOF before returning; it is + streaming *input*, not streaming *output*. `@dexpace/codec-json` must buffer, because `JSON.parse` + has no incremental form, and that limitation is ledgered. +- **No SSE coupling.** Core's SSE parser has no serde dependency at all, and + `bun run verify:sse-37` is a blocking CI step that proves it. `typedSseStream(stream, mapper)` is + where a caller plugs decoding in. diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md new file mode 100644 index 0000000..27c2576 --- /dev/null +++ b/docs/sdk-documentation/write-a-transport.md @@ -0,0 +1,230 @@ +# Write a transport + +A transport is two methods: + +```typescript +interface Transport { + send(request: Request, options?: RequestOptions, signal?: AbortSignal): Promise; + close(): Promise; +} +``` + +There is no registration step. A conforming object is a valid transport; you pass it to +`standardResilience()` or `new PipelineBuilder(...)`. Write one when the two shipped adapters do not +fit — a different HTTP client, a mock service, an in-process loopback, an instrumented wrapper. + +## The smallest useful one + +```typescript +import { + Headers, + Protocol, + Response, + Status, + type Request, + type Transport, +} from '@dexpace/core'; + +export function echoTransport(): Transport { + return { + async send(request: Request): Promise { + return Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .headers(Headers.newBuilder().setInbound('content-type', 'text/plain').build()) + .body(new Blob([request.url.href]).stream()) + .build(); + }, + async close(): Promise {}, + }; +} +``` + +Note `setInbound`, not `set`: values a server sent are accepted leniently. Using the strict setter on +a real server's headers means a response with an obs-text byte in it becomes unreadable. + +## Thirteen rules a real transport must follow + +The full contract is `docs/product-spec/17-transport-adapter-conformance-contract.md`, thirty +`TRANSPORT-N` clauses. These are the ones that are easy to get wrong. + +**1. Never follow redirects** (`TRANSPORT-1`/`TRANSPORT-2`). Pin them off at the client — `fetch`'s +`redirect: 'manual'`, undici's `maxRedirections: 0` — and pin them off even behind a caller-supplied +dispatcher that may carry a redirect interceptor. The pipeline is the single redirect authority, and +a transport that follows a hop silently defeats loop detection, the hop cap, credential stripping and +the downgrade guard all at once. + +**2. Drop the framing headers, and log every drop by name** +(`TRANSPORT-10`–`TRANSPORT-13`). `Content-Length`, `Host` and `Transfer-Encoding` are computed by the +client, so forwarding a caller's copy corrupts framing. `Connection` is in the drop set for a +`fetch`-class transport and not for an undici-class one — §17 says so explicitly. Log the **name**, +never the value, and dedupe per name by default. + +**3. Whatever your native client refuses, drop that header — never the request** (`TRANSPORT-12`). +This is the half that is easy to miss, because the refusal happens somewhere you are not looking. +`@dexpace/core` admits every printable ASCII byte in a header name (`http/ascii-validation.ts`), so +`X Custom` is a model-valid name no HTTP client on this platform will carry. Both shipped clients +also refuse `Expect`, `Keep-Alive` and `Upgrade` outright, and undici refuses `Connection` with any +value but `close`/`keep-alive`. Find out *where* your client decides: WHATWG `Headers.append` throws +at construction, which a `try`/`catch` degrades for free; undici validates inside `dispatch`, so +that transport has to ask the question itself before handing the array over. Getting this wrong +does not look like a bug in your transport — it looks like a retryable network failure that burns +the caller's whole retry budget re-proving a permanent misconfiguration. The shared suite has a row +per name. + +**4. Classify a native rejection with the shared table, not by hand** (`TRANSPORT-20`, `RETRY-2`). +There are two kinds, and they are not the same kind of thing. An *exchange* that failed — connection +refused, DNS, TLS, peer reset, read timeout — is the retryable `TransportFailureError`, which +`TRANSPORT-20` makes a MUST. A *request* the client refused before dispatching — an unsupported +scheme, a forbidden method, an argument its own validation rejects — can never succeed on a retry, +and `retry/classify.ts` is an allow-list over `IoError`, so reporting it as anything outside that +tree makes it non-retryable for free. Both shipped transports report it as a bare `TypeError` with +the native error as `cause`, matching the `TypeError` they already raise for a misconfiguration +caught at construction. + +Telling the two apart is runtime-specific enough that you should not: call +`toDispatchFailure(error, fallbackMessage)` from `@dexpace/transport-shared`. Node's global `fetch` +reports an unsupported scheme as `TypeError: fetch failed` with an `unknown scheme` *cause* — the +same top-level shape as a DNS failure — while Bun 1.3.14 reports it as +`TypeError [ERR_INVALID_ARG_VALUE]` with no cause, and undici's dispatcher as +`UND_ERR_INVALID_ARG`. The two shipped transports disagreed about `ftp://` until audit #67 / #82 for +exactly that reason. The default is retryable, so a shape the table does not recognize keeps +`TRANSPORT-20`'s MUST. + +**5. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the +retryable `TransportFailureError`; a caller abort is the terminal `CancellationError`. A raw +`DOMException` must never surface. `isTimeoutSignal(signal)` is how you tell them apart. + +**6. Dispatch over a fork of the caller's signal — and keep the fork even when there is no signal** +(`SEAM-16`, `TRANSPORT-9`). Both native clients tie a response body's lifetime to the signal they +were given, so a caller who aborts a moment after `send()` resolves would find the body they already +own torn out from under them. Fork the signal, forward the caller's abort through it, and detach at +delivery. + +The fork runs the other way too, and that half is easy to miss. When a streaming request-body +producer fails while the native call is still pending, your `send()` rejects and nothing is left +awaiting that call: a response arriving afterwards is dropped with its body neither read nor +released, which is the leak `TRANSPORT-9` names. Abort the fork before you rethrow. That is why +`forkSignal()` hands back a live signal even when the caller supplied none and no timeout was +composed — a send with no signal at all is precisely the case where nothing could cancel it. Read +whether the *caller* aborted before you pull the fork yourself, or every producer failure surfaces +as a `CancellationError`; and let `detach()` latch the abort, so the second direction cannot become +the `SEAM-16` violation the first one exists to prevent. + +**7. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do +not close it. + +**8. A response that can carry no body must report `body === null`** (`TRANSPORT-24`, +`TRANSPORT-25`). The WHATWG null-body statuses — `101`, `103`, `204`, `205`, `304` — plus every +`HEAD` response and a 2xx `CONNECT`. Do not forward whatever your native client produced: three of +the four combinations the two shipped adapters meet disagree. undici's dispatcher always hands back +a `BodyReadable`; Node's `fetch` returns `null`; Bun 1.3.14's `fetch` returns a live +`ReadableStream`. `hasNoResponseBody(method, status)` in `@dexpace/transport-shared` is the rule, so +that a consumer can branch on `null` instead of reading to discover there is nothing there. + +Whatever handle you then decline to expose is yours to release — `cancel()` it, `dump()` it — before +you return. `Response.close()` is a no-op on a null body, so nobody else will, and an undrained +`BodyReadable` holds a pooled connection open until the dispatcher times it out. + +A `Content-Length` on a body-less response is not a lie to correct: on a `HEAD` it describes the +body a `GET` would have returned, and it must survive verbatim. Only the body is absent. + +**9. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is +never touched by your `close()`. One you constructed is yours to close. Make that decision once, at +construction, and make supplying both a caller-owned client *and* an option that would build one a +construction-time `TypeError` rather than a silent win for one of them. + +**10. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). +No unbounded await — a graceful drain would stall teardown for as long as one in-flight send against +a slow peer takes. Destroying is the sanctioned choice; in-flight sends then reject with +`CancellationError`, and so does a `send()` issued after `close()`, because it cannot succeed over a +dispatcher that no longer exists and so is not a retryable failure. Declare your post-close mode +(`SEAM-15`) either way: `@dexpace/transport-fetch`'s `close()` is a documented no-op over a runtime +global it does not own, and `send()` keeps working after it. + +**11. Recognize a file body structurally, and still write it through `writeTo`** +(`TRANSPORT-28`, `BODY-13`). `body.kind === 'file'` widens the body to `FileBodyDescriptor` — +`path`, `start`, `count`. Never `instanceof` against `@dexpace/body-file`: a transport must not +depend on it. + +Reading `path` yourself is the trap. It is a shorter path to the wire, and it skips the +descriptor's own `writeTo`, which is where `BODY-13`'s `transferred === count` check lives — the +only thing that can notice a file truncated between `stat` and `send`, because `Content-Length` is +dropped outbound (rule 2) so the framing cannot. `@dexpace/transport-undici` did exactly this until +2026-09-05 and uploaded short files with a 200. Unless your client has a genuine kernel `sendfile` +path, treat a file body as an ordinary `Body` and let `writeTo` produce the bytes; `TRANSPORT-28`'s +zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the +wire — are the descriptor's to keep, not yours. + +**12. Refuse at construction what you cannot honour** (`TRANSPORT-30`, `HTTP-35`). `ProxyType` +admits `socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand +you a proxy your client cannot build. Reject it in the factory with a typed error that names the +type, before you allocate anything — not on the first send, where it arrives as whatever the native +client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a +misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. + +A transport-wide default timeout is the same shape of decision. It ends up in +`AbortSignal.timeout()`, whose range is an integer in `1 .. 2**32 - 1`, and nothing downstream will +check it for you: `RequestOptions.timeoutMs` is validated at its setter, so an unchecked +`defaultTimeoutMs` is the last path by which `1.5` or `2**32` reaches a deadline — where Node throws +a `RangeError` on the first send and Bun 1.3.14 quietly accepts it. Call +`requireValidDefaultTimeoutMs(value)` from `@dexpace/transport-shared` first thing in your +constructor, before anything is allocated. + +**13. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. + +## Prove it + +Do not hand-roll the assertions. `@dexpace/transport-conformance` is the suite both shipped adapters +run, which is what keeps them from drifting apart: + +```typescript +import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; +import {myTransport} from '../src/index.js'; + +runTransportConformanceSuite('my-transport', () => myTransport(), { + supportsInternalCancel: false, // TRANSPORT-8: a cancel path distinct from a caller abort + supportsProxy: false, // TRANSPORT-30 + dropsConnectionHeader: true, // TRANSPORT-11: is `Connection` in your drop set? + // HTTP-35, required: the rows hand this values `AbortSignal.timeout()` refuses and expect your + // factory to refuse them too, rather than deferring the failure to the first send. + buildWithDefaultTimeoutMs: value => myTransport({defaultTimeoutMs: value}), + // TRANSPORT-30, optional: a proxy type your configuration can express and your client cannot + // honour. Omit it and the row asserts `supportsProxy` is false, rather than skipping. + // unsupportedProxy: {type: 'socks5', build: () => myTransport({proxy: socks5Proxy})}, +}); +``` + +Those capability entries are the clauses §17 scopes to a subset of transports, plus the one builder +the suite needs to construct a deliberately misconfigured transport; everything else runs +unconditionally. The suite starts its own fixture server, and a second one on a separate origin +for the rows that deliberately leave a connection unusable — a client that reuses a poisoned +connection otherwise fails thirty rows downstream, which is a debugging problem of a different order. + +The package is `private` and its `exports` name `./src/index.ts`, so it resolves unbuilt and is a +`devDependency`. + +## Reuse the plumbing + +`@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are +`@internal` and it is not a package to install directly, but reading it is the fastest way to see +what a correct implementation of rules 2, 3, 4, 5, 6, 8, 9 and 12 looks like: + +| Module | Concern | +|---|---| +| `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | +| `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | +| `default-timeout.ts` | Rule 12: the range check a transport-wide default timeout has to pass | +| `dispatch-classification.ts` | Rule 4: the one table deciding permanent-versus-retryable for a native rejection | +| `body-less.ts` | Rule 8: which method/status pairs can carry no response body at all | +| `abort-mapping.ts` | Rule 5's single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | +| `body-pump.ts` | Turning a `Body` into a request stream the transport owns, plus idempotent teardown for an abandoned producer | +| `signal-fork.ts` | Rule 6's fork-and-detach | + +## Package it + +`@dexpace/core` goes in `peerDependencies`, never `dependencies` — two copies of core defeat the +identity checks the seams rely on, and `verify:seam-1` enforces it. Take at most one external HTTP +library (`NFR-2`). Declare `engines.node` honestly; `verify:runtime-floor` checks it against your +`tsconfig` target. diff --git a/docs/superpowers/README.md b/docs/superpowers/README.md new file mode 100644 index 0000000..c4ed020 --- /dev/null +++ b/docs/superpowers/README.md @@ -0,0 +1,35 @@ +# `docs/superpowers/` — the inbox, not the archive + +New phase documents land here. They do not stay here. + +The Superpowers `brainstorming` and `writing-plans` skills write to hard-coded paths — +`docs/superpowers/specs/YYYY-MM-DD--design.md` (`brainstorming/SKILL.md:100`, restated at `:206`) +and `docs/superpowers/plans/YYYY-MM-DD-.md` (`writing-plans/SKILL.md:18`, restated at +`:157`). Those skills are installed globally, shared across projects, and this repository cannot change +them. So the path stays, and this directory is the drop point it writes into. + +The archive is [`docs/work/`](../work/). Every finished phase's design, plan and checklist lives under +`docs/work/mvp/phaseN/` — one directory per phase, sub-phases nested inside it, each file keeping its +`YYYY-MM-DD-` prefix. The 62 documents that were here on 2026-08-31 moved there in a single `git mv` +commit, so `git log --follow` still resolves each one across the move. + +## What to do with a file that appears here + +Run the `housekeeping` skill (`.claude/skills/housekeeping/`). Its probe stage lists every file sitting +in `specs/` or `plans/` — staged or not — and its apply stage works out which `docs/work/mvp/phaseN/` +directory each belongs in and moves it with `git mv`. + +**It does not repoint the references.** That is deliberate and the tool says so when it finishes: a +maintenance tool that rewrites prose to make its own check pass produces documentation that is true and +useless at the same time. After `--write`, run the probe's `links` and `citations` checks and fix what +they report, in the same commit. Doing the whole thing by hand is fine too; the rules are in +[`docs/README.md`](../README.md) and the layout is visible in `docs/work/mvp/`. + +A file left here is not lost — it is just not filed. The probe reports it every run until it is. + +## What must not happen here + +Do not point a citation at `docs/superpowers/`. It is a staging path, and anything written here is +scheduled to move. Cite `docs/work/mvp/phaseN/` — the path the document will have for the rest of +its life. The one deliberate exception is a document describing the *skills'* write behavior, such as +this file and the roadmap's "How Phases Get Executed" section. diff --git a/docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md b/docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md deleted file mode 100644 index 1c9b16a..0000000 --- a/docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md +++ /dev/null @@ -1,401 +0,0 @@ -# Node.js SDK — v1 Roadmap - -**Status:** Draft, approved for planning. - -**Purpose:** High-level, ordered phase list from empty repo to a spec-conformant v1 of the `nodejs-sdk`. This is -an index, not an implementation plan — each phase gets its own brainstorm → spec → plan cycle when its turn -comes. Do not add implementation detail to this document as phases complete; instead link to the phase's own -spec file. - -**Governing documents:** - -- `docs/product-spec.md` (+ `docs/product-spec/*`) — the language-agnostic, normative contract. Requirement IDs - (`SEAM-*`, `HTTP-*`, `IO-*`, `BODY-*`, `CTX-*`, `PIPE-*`, `RECOV-*`, `RETRY-*`, `REDIR-*`, `AUTH-*`, `PAGE-*`, - `SSE-*`, `SERDE-*`, `OBS-*`, `CFG-*`, `TRANSPORT-*`, `ASYNC-*`, `XCUT-*`, `NFR-*`) are the vocabulary every - phase below cites against. -- `docs/sdk-design-nodejs.md` (+ `docs/sdk-design-nodejs/*`) — the Node/TS port design, already broken into the - seams this roadmap sequences. -- `/home/mohammad/Projects/dexpace/styleguide/typescript/` (core rules) and - `/home/mohammad/Projects/dexpace/styleguide/typescript-bun/` (toolchain/runtime rules) — binding, in force from - Phase 0 onward, for every phase without exception. - -## Cross-Cutting Constraints (apply to every phase, not their own phase) - -- **Styleguide enforcement is continuous**, not a one-time gate. Every phase's code is written and reviewed - against `styleguide/typescript`'s 15 chapters (Tiger Style overlay on Google's TS guide) from the moment the - toolchain exists (Phase 0). -- **Package manager and test runner: Bun, not pnpm.** `sdk-design-nodejs/02` specifies a pnpm workspace; the - styleguide mandates Bun (`bun install`, `bun.lock`, `.bun-version`, `bun test`) as binding for all dexpace - projects. Resolved 2026-07-23 in favor of the styleguide — see the - [scaffold milestone design](./2026-07-23-scaffold-milestone-design.md) for the reconciled shape. The - multi-package workspace *layout* from `sdk-design-nodejs/02` (package map, project references, peer-dependency - discipline) still holds; only the pnpm-specific mechanics are replaced. Library packages still build with - plain `tsc` (never `Bun.build`, which is reserved for services), per `typescript-bun/08-build-and-distribution.md`. -- **Dual JS/TS consumption.** TypeScript is the source of truth; the SDK must serve both TS and plain-JS - consumers. `tsc` compiles to ESM JS + `.d.ts`; no TS-only runtime syntax leaks into shipped output (the - styleguide's erasable-syntax stance already helps here — no enums, no decorators, no constructor parameter - properties). Verified per-package as each package is built, not only once at the end. -- **Requirement-ID traceability.** Each phase's deliverable should be traceable back to the product-spec - requirement IDs it satisfies, feeding `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md` - and the Phase 9 conformance pass. - -## Phase List - -| Phase | Name | Package(s) | Product-spec refs | sdk-design refs | -|---|---|---|---|---| -| 0 | Toolchain & Style Gate | workspace root, `@dexpace/core` (stub) | — | §2, §9 (see [scaffold milestone design](./2026-07-23-scaffold-milestone-design.md)) | -| 1 | Core HTTP Domain Model | `@dexpace/core` | §4 | §4 | -| 2 | Seam Foundations | `@dexpace/core` | §3 | §3 | -| 3a | I/O Contracts | `@dexpace/core` | §5 | §3.1 (Web Streams direct, no pluggable provider) — see [Phase 3a design](./2026-07-24-phase3a-io-contracts-design.md) | -| 3b | Body Lifecycle | `@dexpace/core` | §6 | §3.1 — see [Phase 3b design](./2026-07-25-phase3b-body-lifecycle-design.md) | -| 4a | Execution Context | `@dexpace/core` | §7 | §5 — see [Phase 4a design](./2026-07-25-phase4a-execution-context-design.md) | -| 4b | Recovery-Chain Primitives | `@dexpace/core` | §8.2 | §5 — see [Phase 4b design](./2026-07-25-phase4b-recovery-chain-design.md) | -| 4c | Stage-Based Pipeline | `@dexpace/core` | §8.1 | §5 — see [Phase 4c design](./2026-07-25-phase4c-stage-pipeline-design.md) | -| 5a | Resilience — Retry | `@dexpace/core` | §9, appendix C `RECOV-17`–`RECOV-34` | §6 — see [Phase 5a design](./2026-07-26-phase5a-retry-design.md) | -| 5b | Resilience — Redirect | `@dexpace/core` | §10 | §6 — see [Phase 5b design](./2026-07-26-phase5b-redirect-design.md) | -| 5c | Resilience — Auth | `@dexpace/core` | §11 | §6 — see [Phase 5c design](./2026-07-26-phase5c-auth-design.md). Both 5b and 5c were drafted solo/concurrently (user away from keyboard); 5c's own doc records reconciling with 5b's cross-origin-marker design after finding it mid-draft — see its "Alignment with 5b's shipped design" sections | -| 6a | Serde | `@dexpace/core`, `@dexpace/codec-json` | §14 | §7.3 — see [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) | -| 6b | SSE | `@dexpace/core` | §13 | §7.2 — see [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) | -| 6c | Pagination | `@dexpace/core` | §12 | §7.1 — see [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) | -| 7a | Configuration & Platform Primitives | `@dexpace/core` | §16, appendix C `RECOV-33` | §8 — see [Phase 7 segmentation design](./2026-07-28-phase7-segmentation-design.md) and [Phase 7a design](./2026-07-28-phase7a-configuration-design.md) | -| 7b | Instrumentation & Observability | `@dexpace/core`, `@dexpace/logging-pino`, `@dexpace/logging-debug` | §15 | §8 — see [Phase 7 segmentation design](./2026-07-28-phase7-segmentation-design.md) and [Phase 7b design](./2026-07-28-phase7b-observability-design.md) | -| 8a | Transport Adapters | `@dexpace/transport-fetch`, `@dexpace/transport-undici`, `@dexpace/body-file`, `@dexpace/transport-shared` | §17 | §3.2 (single `Promise` primitive collapses JVM's SEAM-11/SEAM-16 fragmentation) — see [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) and [Phase 8a design](./2026-07-28-phase8a-transport-design.md) | -| 8b | Async-Runtime Bridge | `@dexpace/rx` | §18 | §3.2 (RxJS `Observable` is the only Node-worthwhile async adapter) — see [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) and [Phase 8b design](./2026-07-28-phase8b-async-runtime-design.md) | -| 9 | Cross-Cutting Invariants & Conformance | all packages, `@dexpace/shrink-test` | §19, §20, appendix B | — see [Phase 9 design](./2026-07-28-phase9-cross-cutting-conformance-design.md) and [Phase 9 plan](../plans/2026-07-28-phase9-cross-cutting-conformance.md) | -| 10 | Deviation Reconciliation | — (review only) | — | §10 | - -**Status note (2026-07-27).** Phases 5a/5b/5c have a design **and** a written implementation plan; none of the -three has been executed — no `src/retry/`, `src/redirect/`, or `src/auth/` exists yet. 5b's and 5c's plans were -reviewed against the knowledge corpus and against each other's declared APIs before execution; the corrections -that outlive their own phase are logged below (see the `cross-origin.ts`, `AuthTiers`, preemptive-stamp, and -`DigestChallengeUnsupportedError` rows). Everything else stayed inside the two plans' own Deviation Ledgers. - -**Status note (2026-07-28).** A cross-phase deferral review swept this log against every written design/plan. -Two real gaps were found and folded into the unexecuted plans: `StepContext` never exposed the caller's per-call -`RequestOptions` (`PIPE-17`'s "readable by any step" MUST — extended 5a Task 1's amendment to two fields), which -in turn left `RETRY-41`'s per-call retry-count override (`RequestOptions.maxRetries`, `HTTP-35`) wired to -nothing (now read by 5a Task 9) and left `AUTH-4`'s `perCall` tier with no per-call source (now -`RequestOptions.auth?: AuthDescriptor`, amended in 5c Task 14). Bookkeeping: the rows targeting Phase 2 and -Phase 3b below were marked resolved-at-design/plan level, and `NFR-13`'s SPDX convention was written into -Phase 1's plan. No executed code exists yet, so every change was a document edit, not a retrofit. - -**Status note (2026-07-28, later same day).** Phase 6 was brainstormed and split into 6a (Serde) / 6b (SSE) / -6c (Pagination) — see the [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md). The split -review produced three findings recorded in the log below that outlive the sizing question: three Phase-0 deferrals -(`NFR-2`, `NFR-14`, peer-dependency dedup) become live in 6a rather than Phase 8, because `@dexpace/codec-json` — -not a transport adapter — is the workspace's first second package; `sdk-design-nodejs/07`'s item-view snippet -contradicts `PAGE-11`'s close-before-yield MUST in a way appendix B's own conformance test does not catch; and -`PAGE-5`'s "synchronously inside parse" needs an explicit re-expression for a runtime with no synchronous body -read. - -**Status note (2026-07-28, end of day).** All three sub-phases now have **both** a design and a written -implementation plan (`specs/2026-07-28-phase6{a,b,c}-*-design.md`, `plans/2026-07-28-phase6{a,b,c}-*.md`); none -has been executed — no `src/serde/`, `src/sse/`, `src/pagination/`, or `packages/codec-json/` exists yet. The -three plans were then reviewed against each other and against the knowledge corpus, the same pass 5b/5c got. The -corrections that outlive their own sub-phase are logged below (the `Symbol.asyncDispose` row, whose stated -premise 6b/6c invalidate, and the `PAGE-11` erratum row, which needed carrying into `docs/knowledge/` and not -only into `sdk-design-nodejs/07`). Everything else stayed inside the three plans' own task lists and Deviation -Ledgers. One process note worth keeping: the segmentation design declares the three sub-phases order-free, but -each plan's **Prerequisite** section had been written as a linear chain (6b "Phases 0 through 6a", 6c "0 through -6b"), which would have silently re-imposed the dependency the split exists to avoid. All three now state -"Phases 0 through 5c" plus an explicit note naming what — if anything — a sibling sub-phase adds. - -**Ordering rationale:** toolchain first (Phase 0) so every subsequent phase is written under the style/quality -gates from line one. From there, bottom-up by dependency: domain model before the seams that operate on it, -seams before the pipelines built on top of them, pipelines before the resilience layer wrapping them, and -pagination/SSE/serde/instrumentation as the outer layers consuming everything underneath. Transport and -async-runtime adapters (Phase 8) come late because they are the most Node-specific judgment calls (per -sdk-design's §3 framing) and benefit from every other seam already being stable. Conformance (Phase 9) and -deviation reconciliation (Phase 10) close the roadmap by construction — they audit what phases 0-8 built rather -than building anything new. - -## How Phases Get Executed - -Each phase, when its turn comes: - -1. Its own brainstorming session — scoped to that phase alone, referencing this roadmap for context. -2. A spec file at `docs/superpowers/specs/YYYY-MM-DD--design.md`. -3. Its own implementation plan (via the writing-plans skill), executed independently. - -This document is updated only to mark a phase's status (not-started / in-progress / done) and link to its spec -once written — it does not absorb implementation detail from completed phases. **Exception:** the Deferred Items -Log below. Every phase's brainstorming session should check this log for entries targeting it before starting, -and append any new deferral it produces before that phase is considered done — this is how a decision made in -Phase 0 ("we'll handle NFR-2 properly once adapter packages exist") doesn't silently evaporate by Phase 8. - -## Deferred Items Log - -Every item a phase's design or checklist explicitly pushed to a later phase, consolidated here so it isn't lost -between a phase's own spec/checklist files and this index. One row that is *not* a deferral, included anyway -because it's easy to mistake for one: `SEAM-5`–`SEAM-10` will **never** be built in this port — that's a -permanent simplification, not a postponement. - -| Item | Originated in | Target phase | Note | -|---|---|---|---| -| `NFR-2` — each optional capability a separately installable unit (core + ≤1 external lib) | Phase 0 | **Phase 6a** (codec half; transport half stays **Phase 8a**) — retargeted 2026-07-28 | Originally "Phase 8, no adapter packages exist yet." The Phase 6 segmentation review found the premise false one phase early: `@dexpace/codec-json` is the workspace's first separately installable unit and takes **zero** external libraries — the cleanest instance of the requirement in the whole roadmap. 6a disposes of the codec half; `transport-fetch`/`transport-undici` close the rest in 8a — `transport-fetch` trivially (zero external libs), `transport-undici` with exactly one (`undici`). See the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) | -| `NFR-9` — automated shrink-survival regression guard | Phase 0 | **Resolved in Phase 9 (design)** | Explicitly out of scope per the scaffold design's own "Out of scope" list. Phase 9's design ships `@dexpace/shrink-test` (private, unpublished devDependency): an esbuild bundle/minify/tree-shake step, a dual-package-hazard fixture app, and a child-process round-trip guard wired into the default build as `bun run shrink-test`. Lands when Phase 9's plan executes | -| `NFR-11` — concurrency-model agnosticism, no async-framework type leak | Phase 0 | **Resolved in Phase 4c** | 4c's `Step`/`Next`/`Runtime` public surface is `Promise`-only — no RxJS, no generator, no framework-specific async type appears anywhere in the pipeline layer. Deferral closed | -| `NFR-12` — reproducible, byte-identical builds | Phase 0 | Phase 10 / first real release | Still open — Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 14) records the intended verification (double-build the workspace, diff output digests) but cannot execute it without a real build artifact. Unblocks at first real release | -| `NFR-13` — SPDX license header per source file | Phase 0 | Phase 1 onward — **written into Phase 1's plan (2026-07-28)** | Soft gap; the spec itself calls this "a review convention, not a mechanical gate". A 2026-07-28 plans review found no phase plan actually carried the convention, so Phase 1's plan now states it in its Global Constraints (`// SPDX-License-Identifier: MIT`, line 1 of every new file, all phases onward) — enforcement stays review-level | -| `NFR-14` — single source of truth for dependency/tool versions (Bun `catalog:`-equivalent) | Phase 0 | **Phase 6a** — retargeted 2026-07-28 | Trivially true today (one package, zero deps); the row's own text said it "becomes a real decision the moment a second package with its own dependencies exists." That moment is 6a scaffolding `@dexpace/codec-json`, not Phase 8. 6a picks the Bun equivalent of the pnpm `catalog:` protocol `sdk-design-nodejs/02` specifies, confirmed against `styleguide/typescript-bun/` | -| `NFR-15` — self-identifying version metadata (real `User-Agent`, never a placeholder) | Phase 0 | **Resolved in Phase 7a (design)** / **Phase 8a** | 7a's design ships `CFG-36`'s build/runtime descriptor (version via build-time codegen, never a runtime placeholder) and `RECOV-33`'s client-identity step that stamps it into `User-Agent`. Node-transport wiring (the header actually reaching the wire) still waits for 8a's concrete transports — a conformance test confirming `TRANSPORT-11`'s header-drop pass leaves it untouched, not new stamping logic. See the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) | -| `NFR-16` — publish provenance enforced on the release path | Phase 0 | Phase 10 / first real release | Still open — Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 14) records the intended verification (run the scripted `prepublishOnly` + `npm publish --provenance` path for real) but cannot execute it without a real publish. Unblocks at first real release | -| `NFR-8` — shrinker keep/retain configuration | Phase 0 | Phase 10 (Deviation Reconciliation) — closed 2026-07-28 | Re-confirmed as not applicable by design in Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 10) — this port has no reflection-driven discovery surface to keep-configure. Closed 2026-07-28 | -| Peer-dependency dedup for `@dexpace/core` (dual-package-hazard guard) | Phase 0 | **Phase 6a** — retargeted 2026-07-28 | Mechanism specified in `sdk-design-nodejs/02` §2. `@dexpace/codec-json` is the first package to declare the `@dexpace/core` peer, so the guard installs in 6a. Not theoretical for this package specifically: `sdk-design-nodejs/02` names the `Tristate` discriminant and the `Outcome` sum type as exactly the branded-symbol checks two non-identical copies of core would break — and `Tristate` is 6a's own deliverable | -| `NFR-10`/`NFR-17` residual — CI running the built artifact against the *declared minimum* Node version (18.17), not just whatever the runner defaults to | Phase 0 | **Resolved in Phase 2 (plan)** (pulled forward from Phase 3) | Low-risk while the only export was a trivial `ping()`. Phase 2 is where it stops being trivial: `composeSignal()` calls `AbortSignal.any()`, which landed in **exactly** Node 18.17.0 — the declared floor to the patch version. Phase 2's plan Task 7 ships the `node-floor-conformance` CI job (`actions/setup-node` pinned to 18.17.0 running `scripts/verify-node-floor.mjs`, which forces the `AbortSignal.any()` branch); its checklist marks the row ✅. Lands when Phase 2's plan executes | -| `MultipartBody` model (one of HTTP-3's "each builder-based model" list) | Phase 1 | **Resolved in Phase 3b (design)** | Retargeted from "Phase 3" when Phase 3 split. 3b's design ships `MultipartBody` in full — composite replayability (`BODY-2`), one shared framing routine driving both declared length and written bytes, RFC-2046 boundary generation/validation (`MultipartBoundaryError`), part-header quoting/escaping (`HTTP-51`). Lands when 3b's plan executes | -| `Request`/`Response` real body type (currently `unknown` placeholder) | Phase 1 | **Resolved in Phase 3b (design)** | 3b's design replaces both placeholders — `Request` carries the §6 `Body` model (replayability, consume-once), `Response.body` is a single-use `ReadableStream \| null` (`BODY-14`). Lands when 3b's plan executes | -| `Logger`/`LogEvent` seam | Phase 2 | **Resolved in Phase 7b (design)** | `sdk-design-nodejs/03` §3.5 discusses it inside the seam-mapping doc, but it carries no `SEAM-N` ID — it's an `OBS-*` concern. 7b's design ships the facade, the process-wide global logger slot, and the two bridge packages (`@dexpace/logging-pino`, `@dexpace/logging-debug`). Lands when 7b's plan executes | -| `FakeTransport` test double | Phase 2 | **Resolved in Phase 5a** | Deliberately not built speculatively — 4a and 4b both used file-local stubs instead, and 4c's own brainstorm chose to keep doing so rather than build a shared double for PIPE-9's empty-pipeline case alone. 5a is the phase that finally needs one: scripted multi-response sequences (`503,503,200`), wire-send counting, and per-response close observation. Ships at `packages/core/src/testing/fake-transport.ts` (`@internal`) alongside `countingResponse()`, whose `ReadableStream` `cancel()` hook is the **only** sanctioned way to observe `Response.close()` — instances are `Object.freeze`d, so a spy assignment throws. 5b and 5c consume it unchanged. Deferral closed | -| Phase 4 split into 4a (Execution Context, `§7`) / 4b (recovery-chain primitives, `§8.2`) / 4c (stage-based pipeline, `§8.1`) | Phase 4 brainstorm | — | ~76 combined normative IDs, comparable to Phase 3's ~79 that forced its own 3a/3b split; each sub-phase gets its own brainstorm→spec→plan cycle. Dependency order: 4a first (contexts are the pipeline's own per-call correlation state), then 4b and 4c | -| Phase 5 split into 5a (Retry, `§9`) / 5b (Redirect, `§10`) / 5c (Auth, `§11`) | Phase 5 brainstorm | — | 111 combined normative IDs — the largest single phase in the roadmap, well past the ~76–79 that already forced the Phase 3 and Phase 4 splits. Build order is forced by coupling, not just size: retry is independent of the other two; redirect owns the cross-origin marker `REDIR-11` defines and `AUTH-29` reads, so it must precede auth; the standard-resilience preset needs all three steps installed, so it closes 5c. Each sub-phase gets its own brainstorm→spec→plan cycle | -| Phase 6 split into 6a (Serde, `§14`) / 6b (SSE, `§13`) / 6c (Pagination, `§12`) | Phase 6 brainstorm (2026-07-28) | — | 107 combined normative IDs (`PAGE` 36, `SSE` 41, `SERDE` 30), between the ~76–79 that forced the Phase 3 and Phase 4 splits and Phase 5's 111. Cut along the spec's own section boundaries because **the spec forbids the couplings that would cross them**: `SSE-37` (MUST) bars any serde dependency from core SSE, and `§12`'s preamble declares pagination serde-agnostic — so the cross-segment contract surface is empty by mandate, which is exactly the property whose absence caused the 5b/5c drift below. **No segment depends on another; the 6a→6b→6c order is convenience, not dependency**, and any sub-phase may execute out of order. 6a leads only because it scaffolds the workspace's second package and is the one segment that reshapes an already-published seam (`SEAM-21`); 6c trails because it is the most coupled to *earlier* phases (4c's `Runtime`, 5a's `StepContext.options`, 3b's `Response` body). Full rationale, per-segment ownership, and the collapsed-ID clusters in the [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) | -| Collapsed-requirement disposition tables for Phase 6 — `PAGE-25`–`PAGE-33` (§12.9's async engine: this port has one async model, so the async generator *is* the engine), `SSE-18`/`SSE-31` (threading re-expressed against the event loop), `SERDE-8`/`SERDE-21`/`SERDE-22`/`SERDE-25`/`SERDE-26` (codec-engine configuration with no configurable engine to configure) | Phase 6 brainstorm | Each owning sub-phase's design (6c, 6b, 6a) | Same service 5a's `RECOV-17`–`RECOV-34` table performs: without a row-by-row disposition, a naive appendix-B sweep reads ~18 collapsed requirements as uncovered. The segmentation design identifies the clusters and what does **not** collapse inside each — notably `PAGE-26`/`PAGE-27`/`PAGE-32`'s close-exactly-once obligations (re-expressed as `finally`-block obligations on the single generator) and `SSE-31`'s close-during-in-flight-read branch, both of which stay real, testable work. **Note (Phase 9 design, 2026-07-28):** Phase 9's actual design scopes to `§19`/`§20` (`XCUT`/`NFR`) only — it does not re-verify `PAGE`/`SSE`/`SERDE` disposition, which stays each owning sub-phase's own responsibility as this row already states (6c, 6b, 6a respectively) | -| `sdk-design-nodejs/07` §7.1's item-view snippet closes the page *after* yielding its items; `PAGE-11` (MUST) requires closing *before* | Phase 6 brainstorm | **Phase 6c** (erratum against `sdk-design-nodejs/07` **and** `docs/knowledge/pagination.md`) | The 2026-07-28 plans review found the erratum was being written into `sdk-design-nodejs/07` only, while `docs/knowledge/pagination.md` carries the *same* wrong ordering in its Reference section directly beside the correct MUST in its Rules section. The knowledge corpus is the standing tie-breaker every later phase consults, so an erratum that skips it leaves the contradiction live; 6c's plan now amends both. Recorded because **the conformance test is weaker than the requirement**: the snippet's `finally` still passes `PAGE-11`'s stated check (an early `break` drives `.return()`, hence the close), so following the design doc ships a violation the appendix-B checklist would not catch. Resolution per the standing tie-breaker (normative spec + knowledge corpus win over an illustrative snippet): `PAGE-11` governs — copy items, close, *then* yield. Costs nothing, since materialized items survive close per `PAGE-2`. The snippet remains correct about the thing §7.1 is actually arguing (JavaScript's automatic `.return()`-on-abandon), just not about close ordering | -| `PAGE-5`'s "strategy MUST read everything it needs from the response **synchronously** inside parse" | Phase 6 brainstorm | **Phase 6c** (design must state the re-expression) | Node has no synchronous body read, so the literal reading is unimplementable and `parse` returns a promise. Every part of the requirement's actual intent survives: single-use-body discipline, no retention of the response or its body past the call, no close, no mutation. Flagged so an async signature does not later read as an oversight or get "fixed" back toward a literal reading | -| `SSE-41` — reactive SSE adapter (backpressure-honoring `Observable` view, fatal/non-fatal split, source-ownership documentation) | Phase 6 brainstorm | **Phase 8b** (`@dexpace/rx`) | `MAY`. 6b ships the pull-based `AsyncGenerator` surface `SSE-39` mandates; the reactive view is a bridge package, and the roadmap scopes `§18`'s async-runtime adapters to 8b specifically (not 8a's transports) as of the 2026-07-28 [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md). `sdk-design-nodejs/02` identifies RxJS's push-based `Observable` as the one async shape in the Node ecosystem worth bridging at all. Is `ASYNC-21` restated — the segmentation design's §5.2 names it 8b's marquee deliverable | -| Appendix C `RECOV-17`–`RECOV-34` reconciliation (18 rows filed under "Recovery-chain pipeline primitives" that `§8.2`'s prose never defines — it stops at `RECOV-16`) | Phase 4 sizing review | **Resolved in Phase 5a** | They are retry-engine requirements stated a second time for the reference's second retry stack. Since this port collapses both stacks into one engine (`RETRY-28`, `sdk-design/06`), 16 of the 18 collapse onto the same implementation as their `§9` twin (e.g. `RECOV-21` restates `RETRY-9`/`10`/`11`'s backoff formula verbatim); `RECOV-34`'s settings-object validation is partially new; `RECOV-32` and `RECOV-33` have **no** `§9` twin and are genuinely new work. The full row-by-row mapping table lives in the [Phase 5a design](./2026-07-26-phase5a-retry-design.md) — a naive appendix-B sweep should read it rather than re-deriving it, or it will read 18 requirements as uncovered. **Note (Phase 9 design, 2026-07-28):** `RECOV-*` is outside Phase 9's actual `XCUT`/`NFR`-scoped design; its disposition stays 5a's own responsibility per the table this row already points to | -| Real W3C Trace Context generation (trace-id/span-id byte generation, hex encoding, `traceparent`/`tracestate` parsing) — `InstrumentationBundle`'s actual tracing backend | Phase 4a | **Resolved in Phase 7b (design)** | 4a ships only `CTX-14`'s bundle shape and `CTX-15`'s no-op default. 7b's design generates real W3C/Datadog/no-op trace and span ids via `globalThis.crypto.getRandomValues` and lets a caller-supplied `tracerFactory` flow into `InstrumentationBundle` at pipeline-build time, without changing its already-frozen shape. Lands when 7b's plan executes | -| `contextsEqual()` value-equality utility for `ExecutionContext` | Phase 4a | Not scheduled — build only if 4b or 4c turns out to need one | `CTX-6` describes a consequence of key uniqueness, not a mandate for a new equality API; no consumer identified yet, so not built speculatively (same discipline as the original `FakeTransport` deferral) | -| `PIPE-35` — FLATTEN-vs-NEST seeding of a builder from an existing pipeline | Phase 4c | **Resolved in Phase 5c (design)** | Placed under `§8.1`'s "Bridges." heading but **not** bridge machinery — a builder capability independent of the sync/async collapse that disposes `PIPE-31`–`PIPE-34`. Deferred because 4c is the phase that first makes a pipeline constructible at all, so no caller yet holds one to seed from; the MUST clause ("make the choice explicit, never accidental") is vacuously satisfied while no seeding path exists. 5c's design ships `PipelineBuilder.seedFrom(runtime, 'flatten' \| 'nest')`, an explicit, non-defaulted mode argument. Deferral closed at design level; implementation lands when 5c's plan executes | -| `PIPE-2`'s redirect/retry conformance clause and `PIPE-40`'s 2-hop-redirect conformance clause | Phase 4c | `PIPE-40` → **Resolved in Phase 5b (design)**; `PIPE-2` → **Resolved in Phase 5c (design)** | 4c ships pipeline plumbing and zero pillar steps, so neither clause is testable there. `PIPE-40` is a contract on wrapping steps, closed by 5b's own two-hop `FakeTransport` test (wire-send count, per-hop close, final-response-open). `PIPE-2`'s stage-ordering half *is* covered in 4c; only the "auth step re-runs per redirect hop" half needed both a redirect step and an auth step — 5c's design specifies the per-hop re-run and adds the joint conformance test (auth step, "Closing `PIPE-2`'s remaining half and `AUTH-29`, jointly with 5b") | -| `PIPE-24`/`PIPE-39` — the standard-resilience preset (and `PIPE-24`'s "installs into empty slots only" clause) | Phase 4c | **Resolved in Phase 5c (design)** | 4c dispositioned both as "no preset shipped, revisit when one exists." A preset needs all three pillar steps installed, so it cannot land before auth. 5c's design ships `standardResilience()`, installing exactly the three pillars that exist by then (redirect, retry, auth) — `LOGGING` stays empty until Phase 7b ships a real logging step (**resolved in Phase 7b's design**, which amends `standardResilience()` to install it), a documented scope boundary, not a re-deferral | -| `PIPE-36` — a shipped pillar family locks its stage assignment | Phase 4c | **Resolved in Phase 5a** | 4c deferred it to "whichever future phase ships the first real pillar step family." That is 5a, and it is satisfied structurally: `retryStep()` is a factory returning a `StepDescriptor` with `stage: 'RETRY'` baked in — steps are functions carrying a descriptor, not classes with a subclassable stage assignment, so there is nothing to relocate. Deferral closed | -| Public-barrel promotion of the pillar-step authoring surface (`retryStep`, `StepDescriptor`, `Stage`, `PipelineBuilder`, `Runtime`) | Phase 4c, re-confirmed in Phase 5a | **Resolved in Phase 5c (design)** | 4c left "whether SDK callers ever author custom steps against a public surface" to "whichever phase first ships a pillar step." 5a answers: not yet. A caller cannot assemble a working pipeline until 5c's preset exists, and publishing `retryStep` alone would freeze shapes 5c may still reshape. 5c's design promotes `Stage`/`STAGE_ORDER`/`PILLAR_STAGES`/`StepDescriptor`/`StepContext`/`Next`/`PipelineBuilder`/`Runtime`/`retryStep`/`redirectStep`/`authStep`/`standardResilience`; everything else under `auth/` stays `@internal`. `packages/core/etc/core.api.md`'s diff at 5c's plan-writing time is the mechanical proof | -| `RETRY-29` — opt-in server-driven retry-classification override header | Phase 5a brainstorm | Not scheduled | `MAY`. Lets a response header force or suppress the retry classification. Widens the classifier's input surface to server-controlled values, which is a trust decision deserving its own deliberation rather than a default. No caller identified | -| `RECOV-33` — client-identity header step (Append/Replace token composition, blank-line suppression) | Phase 5a brainstorm | **Resolved in Phase 7a (design)** | One of only two appendix-C `RECOV-17`–`RECOV-34` rows with no `§9` `RETRY-*` twin (the other, `RECOV-32`'s idempotency key, shipped in 5a because retry preserves it per `RETRY-38`). Pure configuration-driven header composition with zero retry coupling, so it travels with `CFG-*` in 7a, ships as `clientIdentityStep()` consuming `CFG-36`'s build/runtime descriptor, and closes `NFR-15` alongside it. Lands when 7a's plan executes | -| `StepContext.signal` **and** `StepContext.options` — exposing the call's `AbortSignal` and per-call `RequestOptions` to steps | Phase 5a brainstorm (`signal`); 2026-07-28 plans review (`options`) | **Phase 5a, Task 1** | Found during 5a's spec self-review: 4c's `Cursor` accepts and threads a `signal` but `StepContext` never exposed it, so no step could observe cancellation — `RETRY-26`'s cancellable wait and `RETRY-32`'s "no attempts after cancellation" were both unimplementable. A 2026-07-28 review found the identical gap for `options`: `Cursor` threads them to terminal dispatch but `PIPE-17`'s "readable by any step" MUST was unsatisfied, and with it `RETRY-41`'s per-call override (`RequestOptions.maxRetries`, `HTTP-35`'s "0 disables retries for this call") had no wire — Phase 1 designed the knob, nothing read it. Both fields land as one additive amendment in 5a Task 1; 5a Task 9 wires the retry override, 5c Task 14 wires the per-call auth descriptor. **2026-07-29:** 4c's own design and plan now record the `PIPE-17` half as a deferral naming 5a Task 1, so the MUST is no longer deferred silently (4c validation review, F1); 4c's plan also forbids adding the two fields early, since their shape belongs to their first reader | -| `SEAM-30` cleanup (cancel an orphaned response on the completion race) | Phase 2 | **Phase 8a** | Documented as a TSDoc contract obligation on `Transport.send()` in Phase 2; only a real Transport implementation has a response to actually cancel. Collapses onto `TRANSPORT-9` (and `ASYNC-5`, which collapses onto the same thing) per the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) §5.1 — closes as part of 8a's conformance suite, not separate work | -| Byte-stream provider implementation (`ByteQueue`, `BufferedSource`/`Sink`, `TeeSink`) | Discussed in Phase 2 (`sdk-design/03` §3.1), built in | **Phase 3a** | `sdk-design-nodejs/03` covers this in the same document as Phase 2's other seams — the roadmap's phase split puts the *contract* in Phase 2 and the *implementation* in Phase 3a; don't conflate the two | -| Every buffering **cap** — `BODY-19`'s configurable tap cap, `BODY-30`/`HTTP-52`'s 1 MiB error-body cap, `BODY-34`'s shared preview-size configuration | Phase 3a | **Resolved in Phase 3b (design)** | Deliberate placement, not an omission — §5 bounds nothing; every spec-mandated cap sits in §6, and 3b's design wires all three: the `withRequestLogging` tee's `tapCapBytes` (`BODY-19`), `toHttpError()`'s fixed 1 MiB error-body cap (`BODY-30`/`HTTP-52`), and one shared preview-size parameter threaded through both logging tees and `toHttpError` (`BODY-34`). The rejected `maxRetainedBytes`-on-`BufferedSource` reasoning stands — don't re-litigate. Lands when 3b's plan executes | -| Promotion of any §5 type into the published `@dexpace/core` barrel | Phase 3a | **Resolved in Phase 3b (design)** — never promoted | 3b decided: `Body.writeTo` takes the platform's `WritableStream`, not `BufferedSink`, so no §5 type ever surfaces — all of `src/io/` stays `@internal` permanently. `api-extractor`'s report staying byte-identical across 3a was the mechanical proof the freeze held until the decision | -| `MAX_BYTE_ARRAY_LENGTH` constant value (`IO-9`) | Phase 3a | Phase 3a plan time | Core is runtime-agnostic, so `node:buffer`'s constant is off-limits; V8 and JavaScriptCore disagree and both have moved theirs; 12.6 forbids an import-time probe. Design fixes the *mechanism* (conservative constant + `RangeError` backstop); the number itself is confirmed when the plan is written | -| `Symbol.asyncDispose` on §5 resources (styleguide 13.1/13.2) | Phase 3a | **Re-scoped 2026-07-28 — the premise expired in Phase 6** | Declined in 3a for the same reason Phase 2 declined it on `Transport`: `Symbol.asyncDispose` postdates the `>=18.17` floor, and TypeScript does not polyfill it for a library *declaring* the method — the computed key silently becomes the string `"undefined"` at run time. The row's own escape clause was **"costs nothing today since no §5 type is public,"** and that stopped being true in Phase 6: 6b publishes `SseStream` and 6c publishes `Page`, both resource-owning classes whose primary teardown is a public `close()` — exactly the shape `styleguide/typescript/13` §13.1 forbids, with §13.2 prescribing `[Symbol.asyncDispose]` delegating to the legacy `close()`. It also has a second consumer now: `PAGE-12` (MUST) requires consumers of the page-level view to be *told* to wrap it in a scoped/auto-close construct, and `await using` is that construct. Both sub-phases therefore ship a **runtime-guarded, optionally-typed** `[Symbol.asyncDispose]`: installed via `Object.defineProperty` only when the well-known symbol exists (so the `"undefined"`-key hazard cannot occur on the declared floor), typed optional (so it never promises `await using` support the pinned 18.17.0 runtime cannot honor), and delegating to `close()`, which stays the supported teardown on every runtime. Requires `esnext.disposable` on the TypeScript `lib` list — a types-only change that does not move `engines.node`. Promotion to an unconditional `implements AsyncDisposable` is a one-line change still gated on the floor passing 18.18; **that** is the residue this row now tracks, not the expired "no public resource type" premise. See 6b's and 6c's designs, "Disposal" | -| `SEAM-5`–`SEAM-10` (discovery/registration/conflict-resolution machinery) | Phase 2 | **Never** — not deferred | Node has no pluggable byte-stream factory or fragmented async ecosystem to discover across; a permanent, documented simplification vs. the JVM reference, recorded in Phase 10's reconciled deviation ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 2), not "TODO'd" anywhere. Closed 2026-07-28 | -| Concrete `Serde` implementation (`@dexpace/codec-json`) | Phase 2 | **Phase 6a** | Phase 2 ships the `Serde` interface only. Narrowed from "Phase 6" by the 2026-07-28 segmentation review | -| Concrete `Transport` implementations (`@dexpace/transport-fetch`, `-undici`) | Phase 2 | **Phase 8a** | Phase 2 ships the `Transport` interface only. Narrowed from "Phase 8" by the 2026-07-28 [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) | -| `SEAM-21` — explicit runtime type token for deserialization (the type-witness mechanism) | Phase 2 | **Phase 6a** | `sdk-design-nodejs/03` §3.3 defers to §7.3. Phase 2's `Serde.deserialize(data: unknown): T` is the erased/inferred generic SEAM-21 forbids, so the interface **will** change shape — which is why Phase 2 keeps `Serde` out of the package barrel and marks it `@internal`, so the rework is not a breaking change to a published API. Narrowed from "Phase 6" by the 2026-07-28 segmentation review, which also made this the reason 6a leads the phase: reshaping a seam belongs before, not after, other work built on the same barrel. 6a additionally decides whether the reshaped seam is finally promoted to the public barrel, and whether `Serde` stays generic in `T` at all once the schema carries `T` | -| `SEAM-14` — close *behavior* (idempotent, ownership-aware, releases only self-created resources) | Phase 2 | **Phase 8a** | The `close(): Promise` **signature is locked in Phase 2** — adding a required method to a published seam later is a breaking change. Only the behavior waits, until a transport owns a pool worth releasing. Asymmetric across 8a's two packages: `transport-fetch` owns no persistent resource (a sanctioned no-op close); `transport-undici` owns a real `Pool`/`Client`/`Agent` | -| `SEAM-12` — concurrent-call conformance test | Phase 2 | **Phase 8a** | Stated as a TSDoc contract obligation on `Transport.send()` in Phase 2; "fire many concurrent requests and assert no cross-talk" needs a real transport to fire through. Collapses onto `TRANSPORT-29` (and `ASYNC-22`, its twin) per the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) §5.1 | -| `SEAM-18` (sync↔async bridges) | Phase 2 | **Never** — not deferred | Same class as `SEAM-5`–`SEAM-10`: a bridge connects two transport seams and this port has one. Every obligation SEAM-18 names presupposes a blocking transport Node cannot idiomatically have. Its one non-bridge clause ("per-call options MUST be threaded through, not dropped") survives as a `Transport.send()` obligation. Recorded in Phase 10's reconciled deviation ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 2). Closed 2026-07-28 | -| `HTTP-18`/`HTTP-48`/`HTTP-50` — outbound header strictness vs. ETag obs-text permission, discovered replaying a server-issued ETag with obs-text bytes through a conditional request | Phase 1 | **Resolved in Phase 10** | `RequestConditions.applyTo`'s strict outbound path is kept; `HTTP-18`'s MUST-level splitting defense (reinforced by `XCUT-18`) outranks `HTTP-48`'s SHOULD-level obs-text permission. See Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 15). Closed 2026-07-28 | -| `FileBody` (`BODY-11`/`BODY-12`/`BODY-13`/`BODY-36`) — file-backed request body | Phase 3b brainstorm | **Resolved in Phase 8a (design)** | Needs `node:fs`, which conflicts with `@dexpace/core`'s zero-`node:`-import invariant. Resolved as a **structural, not nominal, recognition contract**: `@dexpace/core`'s `Body.kind` union gains a `'file'` member and a type-only `FileBodyDescriptor` interface (zero runtime cost — types erase), retrofitted into Phase 3b's plan; the concrete `fileBody()` factory needing real `node:fs` validation ships in a new fourth Phase 8a package, `@dexpace/body-file`, which both transports depend on and recognize via `body.kind === 'file'` structural narrowing, never a cross-package `instanceof`. Separately, 8a's design confirms (not merely flags) that true kernel-level zero-copy dispatch (`TRANSPORT-28`'s SHOULD) **has no Node analogue** — neither `fetch` nor `undici` expose a `sendfile`-shaped API for outbound bodies — recorded as a `PAGE-29`-shaped collapse in 8a's Deviation Ledger, not chased further. See [Phase 8a design](./2026-07-28-phase8a-transport-design.md) §5 | -| `packages/core/src/redirect/cross-origin.ts` (the `REDIR-11`/`AUTH-29` shared signal — a real header, `CROSS_ORIGIN_MARKER_HEADER`, plus `hasCrossOriginMarker()`/`clearCrossOriginMarker()`) | Phase 5b brainstorm | **Resolved in Phase 5b (design)** | 5b ships and owns this module; 5c's own design (drafted concurrently, before either doc knew of the other) originally guessed an incompatible `WeakSet`-keyed shape against `REDIR-11`'s prose directly, then corrected itself against 5b's actual design once found mid-draft — see 5c's "How this doc was produced" / "Alignment with 5b's shipped design" sections. Recorded here as a caution: two solo brainstorms sharing a cross-phase contract, run without coordinating with each other, is exactly the scenario this kind of drift comes from — re-check for it explicitly if this ever happens again rather than assuming file-discovery mid-draft will always catch it. **The caution earned itself twice.** Catching the marker's *shape* mid-draft did not catch its *scope*: 5c's design consumed the marker on the outbound pass but still answered a `401`/`WWW-Authenticate` challenge on a marked hop, which would have stamped exactly the credential the marker exists to suppress — onto the server-chosen foreign host, over a URL whose HTTPS guard was deliberately skipped. Found in a plan review before any code existed and fixed in both 5c's plan and design (the marker now suppresses the whole hop, not just the outbound pass), but a cross-phase contract review needs to cover every place the consuming phase *acts on* the contract, not just where it reads it | -| `standardResilience()` gains a `LOGGING` pillar step | Phase 5c brainstorm | **Resolved in Phase 7b (design)** | 5c's preset installs only the three pillars that exist by then (redirect, retry, auth); a real logging step doesn't exist until Phase 7b. 7b's design amends `standardResilience()` to install `loggingStep()` (inert by default at `granularity: 'none'`) into the previously-empty slot. Lands when 7b's plan executes | -| `DigestChallengeUnsupportedError` — confirm a real caller-facing API needs to distinguish "unsatisfiable challenge" from "no replacement" before shipping it | Phase 5c brainstorm | **Resolved in Phase 10 — 2026-07-28** | `authStep()` itself never surfaces this distinction (both cases just leave the 401 unchanged); the leaf was sketched for a lower-level API 5c's design did not otherwise build. 5c's plan **kept** it rather than cutting — as an `@internal` leaf for a caller composing `composingHandler`/`digestHandler` directly, bypassing `authStep()`. **Resolved:** kept, permanently — no forced usage-sweep will ever run (Phase 9 is `XCUT`/`NFR`-scoped, no phase's code exists yet for one regardless), and an `@internal`-tier leaf costs nothing sitting unused; it can be removed later without a breaking change if it genuinely proves dead weight once real callers exist | -| Basic/Digest never stamp preemptively — an *interpretation*, not a stated requirement | Phase 5c brainstorm | **Resolved in Phase 10 — 2026-07-28** | `§11` phrases `AUTH-14` and `AUTH-15`–`AUTH-22` entirely as reactions to a parsed challenge, and never describes a preemptive-Basic path the way it separately describes Bearer's preemptive cached-token stamp; Digest structurally cannot stamp before seeing `realm`/`nonce`. 5c treats both uniformly as challenge-only. **Resolved:** confirmed correct as designed — the spec's asymmetry (describing Bearer's preemptive path, staying silent on Basic/Digest) reads as deliberate, and staying reactive matches this port's conservative-by-default posture elsewhere (credential-stripping by default, downgrade-deny by default). See Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 12) | -| True per-call / per-operation `AuthTiers`, resolved per call rather than fixed at step construction | Phase 5c plan | `perCall` tier: **Resolved in Phase 5c (design, 2026-07-28 revision)**. `operation` tier: unscoped | Originally fully unscoped because no phase shipped a per-call lookup source. The 2026-07-28 plans review closed the `perCall` half: the vehicle is `RequestOptions` (per-call operational overrides are exactly its Phase 1 charter), not `ExecutionContext` — `RequestOptions` gains `auth?: AuthDescriptor` (type-only, cycle-free import; amended in 5c Task 14 alongside `pipeline/builder.ts`'s existing amendment precedent), steps read it via `StepContext.options` (5a Task 1, `PIPE-17`), and `authStep` resolves `{...settings.tiers, perCall: ctx.options.auth}` when present. The `operation` tier still has no distinct source — nothing in this roadmap ships a per-operation layer (no codegen/client surface), so `operation` and `client` both remain construction-time configuration; that residue is a plumbing gap, not a conformance one (`AUTH-4`–`AUTH-7` are mechanically satisfied), and stays open here | -| Redirect predicate's scope over safety mechanics (credential stripping, downgrade, replayability, loop/cap) — 5b reads `REDIR-20`'s "MUST fully override" as scoped to code/method eligibility only, not these | Phase 5b brainstorm | **Resolved in Phase 10 — 2026-07-28** | A judgment call made without the user present; 5b's own design flagged it as narrow and mechanical to reverse if wrong. **Resolved:** confirmed correct as designed — `REDIR-20`'s snapshot (response, redirect count, visited URIs) carries nothing about credentials, and safety mechanics are separately governed by `XCUT-17`'s own universal, non-overridable framing; a predicate opting out of them would be a security regression, not a convenience. See Phase 10's reconciled ledger (`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, Item 12) | -| Redirect structured logging (`SHOULD`-level hop/loop/downgrade events) | Phase 5b brainstorm | **Partially resolved in Phase 7b (plan, 2026-07-28)** | Same disposition as 5a's equivalent gap for retry. 7b's amendment to 5b's `redirect-step.ts` ships the hop event and a rejection event (distinguishing `SchemeDowngradeError`) via `getGlobalLogger()`, no change to `StepContext`'s shape. **Not fully closed:** `decide()`'s `Decision` type carries no reason discriminant on `'return-current'`, so a genuine loop-vs-hop-cap-vs-normal-termination distinction is out of scope for this retrofit — would need `Decision` reshaped, touching every assertion in `decide.test.ts`. 5a's equivalent (attempt-failed, retries-exhausted) closes cleanly with no such gap, since `Outcome.kind` already discriminates success/failure. Both land when their respective plans execute | -| 5a's `RetryConfig.clock`/`random` retyped against 7a's real `Clock` seam, replacing its ad hoc injection point | Phase 7a brainstorm | 7a (doc amendment to 5a's design/plan) | Single-sources the injectable-determinism seam 5a's own design already noted it was pre-empting ("the same injectable-determinism seam `CFG-15` wants for the clock") | -| 5a's private RFC 1123 parser in `pacing.ts` re-sourced from 7a's shared `config/http-date.ts` | Phase 7a brainstorm | 7a (doc amendment to 5a's design/plan) | 7a's module is a superset (adds the formatter 5a never needed); 5a's parser becomes an import, not a second implementation | -| 5a's private `RETRYABLE_STATUSES`/`isRetryableStatus` in `classify.ts` re-sourced from 7a's `config/retryable.ts` | Phase 7a brainstorm | 7a (doc amendment to 5a's design/plan) | `CFG-35` mandates one shared retryability definition; 7a Task 3 ships the identical set (408, 429, 5xx except 501/505) and 5a's `classify.ts` re-exports it unchanged, so `RETRY-1` and `CFG-35` cannot drift apart | -| `challengeHandler` slot on `ProxyOptions` has no protocol behind it | Phase 7a brainstorm | **Resolved in Phase 8a (design)** | The type carries the slot per `CFG-22`'s field list. Resolved as `transport-undici`-only: undici ships `ProxyAgent`/proxy-407 dispatch; `transport-fetch` ships no `proxy` option on `FetchTransportOptions` at all (an absent option, not a silently-ignored one) and documents no proxy support, since honoring `TRANSPORT-30` there would require depending on `undici` internally anyway, undercutting `transport-fetch`'s zero-added-dependency purpose. `§17`'s own preamble licenses this single-transport scoping. See [Phase 8a design](./2026-07-28-phase8a-transport-design.md) §6 | -| Whether `clientIdentityStep` should be added to `standardResilience()`'s default install list | Phase 7a brainstorm | **Resolved in Phase 10 — 2026-07-28** | Not installed by default — no requirement mandates it (`RECOV-33` governs the step's own internal composition, not whether a preset installs it; `NFR-15` only requires that *when* a `User-Agent` is emitted it's real, not that every call carry one), and 5c's preset already closed its own scope for the pillars that exist. **Resolved:** stays out, permanently — adding it would be unrequested preset scope creep; a caller who wants it installs it explicitly, already possible via the public authoring surface | -| Retry/redirect structured-logging event names/fields | Phase 7b brainstorm | Phase 7b plan time | No spec-fixed vocabulary exists for these `SHOULD`-level events; naming is a plan-time detail, not a design-level decision | -| Whether `standardResilience()` should also accept a `tracerFactory`/`meter` pass-through convenience | Phase 7b brainstorm | **Resolved in Phase 9 (design)** — no friction found | No requirement mandates preset-level convenience wiring beyond installing the `LOGGING` step itself. Phase 9's `tests/conformance/xcut/fixtures/composed-pipeline.ts` configures logging/tracing/metrics the same way 7b's own tests do — a `LoggingStepSettings` object passed to `standardResilience()`'s existing `logging` option, plus `setGlobalLogger()` for a spy `Logger` — with no need for a separate `tracerFactory`/`meter` preset-level parameter. Closed, not just deferred again | -| A real `@opentelemetry/sdk-metrics`-backed `Meter` adapter package | Phase 7b brainstorm | Not scheduled | `OBS-31` only requires the no-op default and that core not depend on a metrics runtime; no package in the roadmap's phase table ships a concrete metrics backend, unlike tracing's duck-typed zero-adapter path | -| Phase 8 split into 8a (Transport Adapters, `§17`) / 8b (Async-Runtime Bridge, `§18`) | Phase 8 brainstorm (2026-07-28) | — | 52 nominal combined IDs (`TRANSPORT` 30, `ASYNC` 22) — well under the ~76–79 that forced the Phase 3/4 splits — but §17 is paid twice (two full `Transport` implementations, `transport-fetch` and `transport-undici`) and nine Deferred Items Log rows land here, pushing effective weight to Phase-7-before-its-split territory. Cut along the package boundary the roadmap table already implied, verified empty by the same test Phase 6 applied: `@dexpace/rx` depends only on Phase 6's `Page`/`SseStream`, never on `Transport`, and nothing in `Transport`'s collapsed `Promise`-returning contract (`sdk-design-nodejs/03` §3.2) references RxJS or any `ASYNC-*` id. **No segment depends on the other; the 8a→8b order is convenience, not dependency** (8a leads only because it is the larger, riskier half). A large share of `§18`'s `ASYNC-*` IDs collapse onto their `§17` `TRANSPORT-*` twin (the SEAM-11/SEAM-16 collapse restated at the async-adapter layer) or are inapplicable outright — Node has no blocking-transport/worker-thread-pool model for `ASYNC-3`/`4`/`7`/`14` to bite on, the same premise that already closed `SEAM-18` as "Never." Full rationale, per-segment ownership, the collapsed-ID disposition tables, and open items (notably `FileBody`'s package placement and whether Node's HTTP stack has any zero-copy dispatch path at all) in the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) | - -**Status note (2026-07-28, Phase 7).** Phase 7 was brainstormed and split into 7a (Configuration & Platform -Primitives, `§16`) / 7b (Instrumentation & Observability, `§15`) — see the -[Phase 7 segmentation design](./2026-07-28-phase7-segmentation-design.md). Unlike Phase 6's three segments, this -split has one real (if soft) cross-segment dependency — `OBS-35`'s log-level resolution wants 7a's `Configuration` -— so 7a leads and 7b trails deliberately, rather than "order is convenience only." Both sub-phases got full -designs in this same session (not just a segmentation note): [7a](./2026-07-28-phase7a-configuration-design.md) -and [7b](./2026-07-28-phase7b-observability-design.md). All six Deferred Items Log rows that previously targeted -bare "Phase 7" are updated above to point at 7a or 7b specifically, each marked resolved-at-design-level. Three -new retrofits to 5a's already-written (still unexecuted) design/plan came out of 7a's brainstorm (`Clock`, RFC -1123 parser, and `RETRY-1`/`CFG-35` retryable-status single-sourcing); two more amendments — to 5a's and 5b's -steps for structured logging, and to 5c's preset for the `LOGGING` slot — came out of 7b's. No executed code -exists yet for any phase, so every change listed here is a document edit, not a retrofit to shipped code. - -**Execution order is no longer the numeric order for Phase 5.** These five retrofits do not merely annotate 5a/5b/5c -— they make Phase 7 a *prerequisite* of Phase 5's execution, in both directions the amendment banners record: -7a's `config/{clock,http-date,retryable}.ts` must exist before 5a's plan runs (its Task 8 consumes `Clock`), and -7b's `observability/{logger,redaction,logging-step}.ts` must exist before 5b's Task 6 and 5c's Task 16 run. The -**Ordering rationale** above ("resilience layer... instrumentation as the outer layers consuming everything -underneath") describes the dependency direction as originally designed; it holds for everything except these -named modules, which invert it. Anyone executing plans in roadmap order must run 7a (and, for 5b/5c, 7b) first, -or execute 5a/5b/5c against the pre-amendment text and accept a duplicate-implementation deviation. Each affected -plan's own **Prerequisite** section states this; this note exists so the roadmap does not read as contradicting -them. - -**Status note (2026-07-28, Phase 8).** Phase 8 was brainstormed solo (user away from keyboard, `docs/knowledge/` -as standing tie-breaker per standing instruction) and split into 8a (Transport Adapters, `§17`) / 8b -(Async-Runtime Bridge, `§18`) — see the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md). -Only a segmentation document was produced this session, not full per-sub-phase designs (unlike Phase 7, which got -both in one sitting) — 8a and 8b each still need their own brainstorm → spec → plan cycle. Nine Deferred Items -Log rows that previously targeted bare "Phase 8" or "first concrete Transport" are updated above to point at 8a -or 8b specifically; none is resolved-at-design-level yet, only re-targeted and, where the segmentation review's -own analysis showed it, pre-dispositioned as collapsed/not-applicable (recorded in the segmentation design's §5, -carried forward into 8a's/8b's own row-by-row tables when those designs are written, not re-derived). Two package -column changes: Phase 8's roadmap-table row splits into 8a/8b, and the segmentation design flags a **possible -fourth package** (`FileBody`'s home, e.g. `@dexpace/body-node`) that 8a's own design must confirm or reject -before the roadmap table can be updated further — not decided by this pass. No executed code exists yet for any -phase, so every change here is a document edit. - -**Status note (2026-07-28, Phase 8, continued).** Both sub-phases got full designs and written implementation -plans in a follow-up pass this same day: [8a design](./2026-07-28-phase8a-transport-design.md) / -[8a plan](../plans/2026-07-28-phase8a-transport.md) and [8b design](./2026-07-28-phase8b-async-runtime-design.md) / -[8b plan](../plans/2026-07-28-phase8b-async-runtime.md). Neither plan has been executed — no `packages/` -directory exists in this repository as of this pass. The "possible fourth package" question above is settled: -8a's design confirms `@dexpace/body-file` (a fourth Phase 8a package, `FileBody`'s concrete factory) plus a fifth, -`@dexpace/transport-shared` (header-mapping helpers both transports need identically, found necessary only once -the plan reached implementation-level detail — the segmentation design and 8a's own design doc did not anticipate -this fifth package; it surfaced from "don't duplicate the same algorithm in two sibling packages" rather than -from any `TRANSPORT-N` requirement directly). The roadmap table's 8a row above is updated to list all four -published packages. `challengeHandler`'s protocol and the zero-copy-dispatch question are both resolved (not -merely flagged) in 8a's design — see the updated Deferred Items Log rows above. 8b's design resolved `ASYNC-18` -as inapplicable to the whole port, not merely out of 8b's scope — a correction to the segmentation design's -framing, recorded in 8b's design §3 and not requiring a Deferred Items Log row of its own since nothing was ever -targeted at a phase to begin with. - -**Status note (2026-07-28, Phase 9).** Phase 9 was brainstormed solo (user away from keyboard, `docs/knowledge/` -as standing tie-breaker per standing precedent) and got a full design **and** a written implementation plan in -one session: [design](./2026-07-28-phase9-cross-cutting-conformance-design.md) / -[plan](../plans/2026-07-28-phase9-cross-cutting-conformance.md). Neither has been executed — no `packages/` -directory exists in this repository as of this pass. Per the roadmap's own framing ("audits what phases 0-8 built -rather than building anything new"), Phase 9's scope is deliberately narrow: a per-ID disposition table for all -24 `XCUT` IDs and all 17 `NFR` IDs (the grep across every prior spec/plan turned up exactly two incidental -`XCUT-N` citations before this pass, confirming this is the first systematic tabulation of that family), one new -package (`@dexpace/shrink-test`, closing `NFR-9`), and one new top-level `tests/conformance/xcut/` integration -suite driving 5c/7b's `standardResilience()` composed pipeline — not a general re-litigation of every open -judgment call that happened to say "Phase 9" in this log. Three consequences of that narrower scope: - -- `NFR-9` closes here (design-level) — see the updated row above. -- One deferred item closes here too: whether `standardResilience()` needs a `tracerFactory`/`meter` pass-through - convenience — resolved no, the composed-pipeline fixture needed no such convenience (see the updated row above). -- Four deferred items that targeted "Phase 9 conformance sweep" turned out to be `AUTH-*`/`REDIR-*` interpretive - judgment calls or preset-shape questions, not `XCUT`/`NFR` conformance checks, and are retargeted above to - Phase 10 (Deviation Reconciliation) — the roadmap's other audit-only phase and the one that already carries - this class of write-up. This retargeting is a document edit only; it does not touch Phase 10's own design or - plan files. - -Also closed as part of this pass: three `unresolved 2026-07-25` markers in `docs/knowledge/tooling-and-quality-gates.md` -(package manager/lockfile, test-runner/coverage-gating, `gts` baseline) that a 2026-07-25 cross-phase checkpoint -had already decided but never back-ported into the corpus itself — directly relevant here since `NFR-5`/`NFR-6`/ -`NFR-7` are exactly the rows those stale markers left unconfirmed. - -## Open Findings — Phase 3b Validation Review (2026-07-28) - -A validation pass over `specs/2026-07-25-phase3b-body-lifecycle-design.md` and -`plans/2026-07-25-phase3b-body-lifecycle.md` (`docs/validation-prompts/phase3b-body-lifecycle-validation-prompt.md`) -returned **BLOCKED** on two runtime defects and a cluster of overclaimed disposition rows. **All findings except -D1 and D2 below are applied** to both documents. Recorded here rather than in the Deferred Items Log because -these are review findings against an unexecuted phase, not deferrals of work. - -The two blockers, both now fixed, are worth naming since they generalize: (1) `ReadableStream.cancel()` rejects -with `TypeError` on a locked stream and reading to `{done: true}` does **not** release the reader's lock, so -`Response.bytes()`, `toHttpError()` and the response-logging wrapper each had a `finally`-scoped close that -replaced a successful read with a `TypeError` — a `reader.releaseLock()`-before-cancel constraint now sits in the -plan's Global Constraints, and **every later phase that takes a reader and later closes the stream inherits it**; -(2) `HTTP-39`/`BODY-10`'s exact-length copy was dispositioned as "reuses Phase 3a's `writeAll`" while the plan's -own global constraint forbids importing `BufferedSink`, leaving a declared `contentLength` unverified and a short -stream sending a truncated body silently. - -**Cross-phase note for 4b.** 4b's preamble relies on `Response.close()` latching `#closed` before awaiting -`body.cancel()` so a close rejection propagates exactly once. That still holds: the latch is unchanged and the -only rejection now swallowed is the `TypeError` a still-locked external reader produces, which `BODY-15` requires -close to tolerate. Every other close failure propagates as before. - -| # | Sev | Finding | Where | Resolution | -|---|---|---|---|---| -| D1 | major — **OPEN, needs a decision** | Task 13 Step 6 specifies a **minor** changeset on the reasoning that `Request.body`'s move from `unknown` to `Body \| undefined` is "not breaking for any real caller, since `unknown` accepted nothing usable before." That premise is false — `unknown` accepted *everything*, which is exactly why Task 7 Step 1 has to rewrite every `.body('x')` call site in the existing suite. `api-design.md:72` classes a narrowed parameter type as breaking, requiring MAJOR. `ResponseBuilder.body` narrows the same way | PLAN Task 13 Step 6; `api-design.md:72` | **Undecided.** Either (a) ship it as **major**, which is what the corpus rule says and what the plan now instructs by default, or (b) if `@dexpace/core` is still pre-1.0 and the repo's release policy treats 0.x breaks as minor, keep minor and record the policy pointer. The plan carries both branches with the false justification deleted; pick one before Task 13 runs. Settle once — Phases 4a/4b/5 narrow Phase-1 placeholder types the same way | -| D2 | major — **OPEN, blocked on unwritten code** | Three Phase-1/3a symbols the 3b plan now calls could not be verified: `MAX_ARRAY_BYTES` (assumed exported from `io/byte-queue.ts`, backing `AllocationLimitError`'s `limit` argument — used by both logging tees' `BODY-32` cap clamp), `Status.isError` (used by `toHttpError`'s `BODY-31` gate, replacing a `code < 400` that wrongly swept non-standard 6xx into the error path), and `Protocol.token` (used by `TypedResponse`). `packages/` does not exist on the planning branch, so none could be checked | PLAN Task 10, 11 (`MAX_ARRAY_BYTES`), Task 12 (`Status.isError`), Task 9 (`Protocol.token`) | **Not a design decision — a verification the executing agent must do first.** Task 11's Interfaces block carries a "Verify before writing" note. If a name differs, use the real one; do **not** add a second constant or a local `isError` helper. If `Status` genuinely has no `isError`, `HTTP-11`'s classification is itself a Phase-1 gap and the gate becomes `code >= 400 && code <= 599` pending that fix | - -**Applied without needing a decision** (recorded so the reasoning survives): `BODY-34`'s "one shared cap" -contradiction resolved in the plan's favour — the shared preview cap covers the two logging tees, and -`toHttpError`'s 1 MiB cap is separate because `HTTP-52` *fixes* its value and a spec-fixed value cannot be the -configurable one; `BODY-26`/`BODY-29` built (`LoggedResponseBody` gained a non-draining `error()` and a -regime-dependent `contentLength`); `BODY-25` ledgered as structurally inapplicable — `ReadableStreamDefaultReader` -takes no requested count, so "zero bytes for a positive count" has no analog; `BODY-32`'s negative-cap rejection -added to both tees, which previously accepted a negative cap and silently mirrored nothing; `HTTP-3`'s -`MultipartBodyBuilder` added (`HTTP-3` names "the multipart body" explicitly and Phase 1 could not satisfy it); -`HTTP-2` honored by exporting the concrete body classes from the public barrel as **types only**; the `@internal` -tags removed from the three errors Task 13 promotes, which would have made `api-extractor` either fail or -silently omit them; `withResponseLogging` decomposed under the 70-line cap and made pull-driven, since its -`start()`-loop tail stream eagerly materialized the whole remainder of exactly the oversized bodies the cap -exists to keep off the heap. - -**Correction to 4b's F2 below.** That row states "Phases 1/2/3b/4a ship zero" assertions. **3b no longer does** — -`invariant` pre/postconditions now sit on both tees' caps, `materialize`'s byte accounting, `MultipartBody`'s -framing length, `StreamBody`'s `contentLength`, `drainOnce`'s cap, and `toHttpError`'s buffer loop. Phases 1, 2 -and 4a still ship zero, so 4b's F2 remains open as a project-level question for Phase 10 — 3b is now a second -data point alongside 4c that the rule is applicable, not just aspirational. - -## Open Findings — Phase 4b Validation Review (2026-07-28) - -A validation pass over `specs/2026-07-25-phase4b-recovery-chain-design.md` and -`plans/2026-07-25-phase4b-recovery-chain.md` (`docs/validation-prompts/phase4b-recovery-chain-validation-prompt.md`) -returned **BLOCKED**. The `RECOV-1`–`RECOV-16` mapping itself is sound and every cross-phase reference 4b consumes -checks out against the earlier phase plans — `toHttpError(): Promise` (3b), `RequestOptions.EMPTY` -(Phase 1), `Transport.send(request, options?, signal?)` + `CancellationError` (Phase 2), and `Response.close()` latching -`#closed` *before* awaiting `body.cancel()` so it propagates a close rejection exactly once (3b). Nothing below is a -defect in that mapping. Recorded here rather than in the Deferred Items Log because these are review findings against -an unexecuted phase, not deferrals of work. - -**Status (2026-07-28): F3–F10 are applied** to `specs/2026-07-25-phase4b-recovery-chain-design.md` and -`plans/2026-07-25-phase4b-recovery-chain.md`. **F1 and F2 remain open — they need decisions**, and both documents now -carry a blocking notice pointing here. The rows below keep the full finding text so the reasoning survives; the -Resolution column records what was done. - -**F1 is cross-phase and blocks four phases, not one.** Phases 5a, 6b and 6c all reach for native `SuppressedError` on -the same false premise, so whichever resolution lands has to land in all four at once. - -| # | Sev | Finding | Where | Resolution | -|---|---|---|---|---| -| F1 | **blocker** — OPEN | `SuppressedError` does not exist on the declared runtime floor. `engines.node` is `">=18.17"`, raised at most to `18.18.0` at the 2026-07-25 checkpoint (which exposes `Symbol.dispose`/`Symbol.asyncDispose` only — Node backported those two symbols; `SuppressedError` is a V8 global from the full Explicit Resource Management proposal). `esnext.disposable` in `lib` supplies its *type*, so `new SuppressedError(...)` type-checks and then throws `ReferenceError` at call time — the exact `NFR-10` trap `tooling-and-quality-gates.md:60-61` describes. `bun test` passes locally; the `node-floor-conformance` job pinned to `18.17.0`, `verify:node-floor` and `test:node` all fail | PLAN:19-20 (Tech Stack, claims it is "already available since Phase 3b's checkpoint lib bump" — false), PLAN:804, SPEC:124; also 5a plan:36, 6b design:163, 6c design:192 | **Undecided.** Either (a) raise `engines.node` past the first release shipping Explicit Resource Management — a consumer-visible breaking change, and the checkpoint at plan:57 forbids unsanctioned floor moves — or (b) a runtime-guarded `suppress(primary, secondary)` helper in `packages/core/src/` using native `SuppressedError` when `globalThis.SuppressedError` exists and attaching a `suppressed` property otherwise, matching the guarded shape already sanctioned for `Symbol.asyncDispose`. Confirm the first supporting Node release before choosing (a). **Partially applied 2026-07-28:** the false Tech Stack claim is deleted and replaced with a blocking notice at the top of the plan stating the real constraint; the mechanism itself is untouched pending the (a)/(b) call | -| F2 | major — OPEN | Zero assertions across the whole `recovery/` module — a dozen functions, no `invariant()` call, against `assertions.md:6-7`'s 2-per-function module average (and `styleguide-overview.md:22-23` Rule 8). Neither document acknowledges the rule or argues an exemption. Concretely: no `apply()` checks that a step returned a value at all, so a step returning `undefined` poisons the fold silently. Project-wide inconsistency, not 4b's alone — Phases 1/2/3b/4a ship zero, 4c ships fifteen | PLAN:463-479, 818-859, 964-966, 1352-1370 | **Undecided.** Either postcondition assertions at the fold sites, or a Deviation Ledger row. Worth settling at the project level (Phase 10) rather than per-phase | -| F3 | major — ✅ applied | SPEC:270 still says "the only new failure surface is `wrapCancellation()`'s `invariant()` crash" — stale text from a superseded draft. SPEC:194-204, SPEC:279 and PLAN:63-74 all state the opposite. An agent executing from the File Layout section would restore the `invariant()`, and because the helper runs inside `dispatchWithRecovery`'s own `catch`, that throw bypasses the response and recovery chains — the one failure mode `RECOV-2` exists to prevent | SPEC:270-271 | Replace with `assertNever`'s `InvariantViolation` crash, matching the already-correct PLAN:89-90 | -| F4 | minor — ✅ applied | Spec never designs the `assertNever` addition Task 1 builds. PLAN modifies `packages/core/src/invariant.ts` (new exported symbol, two tests, its own commit); SPEC's File Layout lists only `recovery/` | SPEC:258-268 vs PLAN:102-103, 124-197 | Add the `invariant.ts` line to the spec's File Layout with a one-line note that `fold()` is the codebase's first discriminated-union `switch` | -| F5 | minor — ✅ applied | `RECOV-14`'s second normative sentence (steps safe for concurrent invocation; per-request state never on the step instance) is claimed but neither designed nor tested — both documents cite `RECOV-14` for the defensive copy only. The design does satisfy it (all per-call state is local), but nothing records or guards that | SPEC:141-144, PLAN:49-51 | One sentence in the design + one plan test interleaving two `apply()` calls on one chain | -| F6 | minor — ✅ applied | `RECOV-32`/`RECOV-33` read as silent drops. 4b's deferral sentence covers "backoff, budget, pacing headers → Phase 5"; neither an idempotency-key header injector nor `User-Agent` composition is any of those. Both *are* built — `RECOV-32` in Phase 5a Task 11, `RECOV-33` in Phase 7a Task 9 — but 4b names neither, and 7a is not "Phase 5" | SPEC:18-20 | Extend the Scope sentence to name `RECOV-17`–`RECOV-31`/`RECOV-34` → 5a, `RECOV-32` → 5a, `RECOV-33` → 7a | -| F7 | minor — ✅ applied | `#private` fields with no justifying comment, against `data-modeling.md:20-23` (`private` is the default; `#private` needs a stated runtime-privacy requirement). Neither chain class needs it — unlike 3b's `Response`, whose `#closed` genuinely must survive `Object.freeze(this)`. Inherited pattern: 4a's `ContextStore` does the same | SPEC:64, 78-79; PLAN:464, 819-820, 833, 847 | Ledger row recording `#private` as the package-wide field style with no runtime-privacy claim; project-wide reconciliation is Phase 10's | -| F8 | minor — ✅ applied | Plan's `ResponseRecoveryChain` property test drops half of what the spec specifies. SPEC promises the property also proves the response-step phase never runs on a `Failure` input (`RECOV-4`); the plan's generator emits recovery steps only and never seeds a `Failure`, asserting only that `apply()` settles | SPEC:293-295 vs PLAN:754-773 | **Applied 2026-07-28 — generator extended**, not spec narrowed: the property now generates response *and* recovery steps over a seed that is arbitrarily `Success` or `Failure`, and asserts `responseStepRuns === 0` on every `Failure` seed. Task 3's expected test count moves 12 → 13 | -| F9 | minor — ✅ applied | `fold(outcome, onSuccess, onFailure)` takes three positional parameters, tripping `function-design.md:22-23` ("options object at 3 or more"), which is one stricter than the lint gate (`max-params: ['error', 3]` errors at four). Passes CI while violating the corpus. Phase 2's shipped `Transport.send(request, options?, signal?)` is the same shape | SPEC:36, PLAN:320 | Ledger row recording it as deliberate (matching `Transport.send`), or `fold(outcome, {onSuccess, onFailure})`. See the corpus conflict below | -| F10 | minor — ✅ applied | `statusMappingStep` is a module-level `const` arrow, against `function-design.md:18-21` ("top-level named `function` declarations… arrows are reserved for inline callbacks"). `func-style`'s `allowArrowFunctions: true` will not catch it, and named declarations survive in stack traces — which matters for a function whose whole job is to `throw` | SPEC:227, PLAN:1081 | `export async function statusMappingStep(...)` plus `statusMappingStep satisfies ResponseStep` to keep the conformance check | - -**Corpus conflict surfaced, not a finding.** `function-design.md:22-23` requires an options object at 3+ parameters -while `function-design.md:40-41` sets `max-params: ['error', 3]`, which errors only at four — the prose is one -parameter stricter than its own stated enforcement. F9 is filed against the prose; if the lint threshold is the -authority, F9 dissolves. Worth settling in the corpus rather than per-phase. - -A second conflict the 4b documents met and resolved correctly, recorded so a later reader does not re-litigate it: -`resource-management.md:4-5,72` mandates `using`/`await using` and documents that native disposal builds a -`SuppressedError` with the *disposal* failure primary, while `RECOV-12` requires the opposite priority. 4b picks -`RECOV-12` and argues it at SPEC:107-113 / PLAN:55-59. Correct call, already justified in-document. - -## Open Findings — Phase 4c Validation Review (2026-07-29) - -A validation pass over `specs/2026-07-25-phase4c-stage-pipeline-design.md` and -`plans/2026-07-25-phase4c-stage-pipeline.md` -(`docs/validation-prompts/phase4c-stage-pipeline-validation-prompt.md`) returned **NEEDS WORK — no blockers.** -The `PIPE-1`–`PIPE-40` mapping is sound and every cross-phase reference 4c consumes checks out against the earlier -phase plans: `Transport.send(request, options?, signal?)` + `close()` (Phase 2), `DexpaceError` as the taxonomy -root under `http/errors.ts` (Phase 2's retrofit), `RequestOptions.EMPTY` (Phase 1), `Status.of`/`Protocol.HTTP_1_1` -(Phase 1), and 4a's `createRequestContext(request, init?)`, `promoteToRequest`/`promoteToExchange`, -`ContextStore.install/get/close/clear/size` with the `kind`/`key`/`request`/`instrumentation`/`operationName` -context shape. Nothing below is a defect in that mapping. - -**Status: F1–F8 are applied** to both 4c documents. **F9 remains open — it needs a decision.** - -| # | Sev | Finding | Where | Resolution | -|---|---|---|---|---| -| F9 | major — **OPEN, needs a decision** | `Cursor` accepts the caller's `AbortSignal`, threads it to the terminal transport, and never checks it between steps. `concurrency-and-async.md:46` requires `signal.throwIfAborted()` "at the top of each loop iteration or before each expensive step"; the step walk (and, worse, a pillar step's fork-driven re-drives) is exactly that. An aborted call keeps walking steps and keeps re-driving until the transport hop finally rejects | PLAN `cursor.ts` `#dispatch`; SPEC "Cursor and fork" | **Undecided**, because the fix is not one line: a raw `signal.throwIfAborted()` surfaces a `DOMException` the SDK taxonomy does not own, against Phase 2's `CancellationError` and `XCUT-1`'s "cancellation is terminal, non-retryable, flag preserved" — and `RECOV-11`/4b's `wrapCancellation` already has a shape for this. Either (a) check in `#dispatch` and map to `CancellationError`, or (b) leave the cursor signal-blind and let 5a's `ctx.signal` + `RETRY-32` carry cancellation, recording (b) as a Deviation Ledger row. Settle before 5a Task 1 lands, since 5a is what makes the signal reachable from a step | -| F1 | major — ✅ applied | `PIPE-17`'s "options MUST be readable by any step" was claimed satisfied while `StepContext` exposes only `next`/`fork`/`context`. A MUST silently unmet is a blocker; it is a legitimate deferral only if the document names the phase that takes it — neither did. (The work itself is already scheduled: 5a Task 1, per the Deferred Items Log row below) | SPEC "Steps", PLAN Self-Review `PIPE-17` row | Both documents now record the partial deferral by name — `StepContext.options`/`.signal` land in **Phase 5a Task 1**; the plan's Global Constraints forbid adding them early, since their shape belongs to their first reader | -| F2 | major — ✅ applied | Spec listed `replace` among the operations that raise `PillarCollisionError` on an occupied pillar; the plan's `replace()` deliberately runs no pillar check. `PIPE-5` exempts replace by name ("it swaps a single occupant within its own stage 1:1") and the collision error points the caller *at* replace — an agent following the spec would have made replacing a pillar step impossible, since the incoming type is distinct by definition | SPEC:285 vs PLAN `replace()` | `replace` removed from the collision bullet, `prependAll` added to it, and the exemption spelled out with `PIPE-5`'s own wording | -| F3 | major — ✅ applied | `afterEach(() => contextStore.clear())` in `runtime.test.ts` and `builder.test.ts`. 4a's plan forbids this by name — it wipes entries a sibling test file installed in the same `bun test` process (`testing.md:50,52`), and 4a's own store tests avoid the singleton for exactly this reason. Not needed either: `Runtime.send()` evicts its own entry in a `finally` on both paths | PLAN runtime.test.ts, builder.test.ts | Both hooks deleted (and the now-unused `afterEach`/`contextStore` imports), replaced by a comment recording why. The one surviving `contextStore.size` read is a before/after **delta** inside a single test, which the 2026-07-26 review already sanctioned | -| F4 | major — ✅ applied | `NFR-13`'s SPDX header was absent from all eleven code listings and from Global Constraints, against "written into Phase 1's plan… line 1 of every new file, all phases onward" (Deferred Items Log) and 4a's precedent | PLAN, every code block | Global Constraints bullet added, `// SPDX-License-Identifier: MIT` prepended to every listing, and Task 6 gains Step 3b's grep — 4a's gate, copied. **Project-wide drift, not 4c's alone:** the 4b, 5a, 5b, 5c, 6b and 6c plans carry no SPDX header either; Phase 9's `NFR-13` sweep is where that gets closed | -| F5 | major — ✅ applied | The design's "**Property tests:**" heading and the Phase 4 checklist's "Property tests where invariants exist ✅ … 4c (edit-order independence, batch ordering)" row both claimed properties the plan never shipped — `builder.test.ts` had no `fast-check` import and two hand-picked examples. `testing.md:29` puts an invariant-bearing assembler like `build()` squarely in property-test territory | SPEC "Testing" vs PLAN builder.test.ts | Three real `fc.assert` properties added (edit-sequence-equals-from-scratch for `PIPE-22`; batch order preserved / reversed for `PIPE-38`), generated over the non-pillar stages so cases exercise ordering rather than `PIPE-5`'s collision. Task 5's expected count 19 → 22; Tech Stack names `fast-check`. The spec's "arbitrary sequence" now says `append`/`prepend`, matching what the generator emits — the anchored edits need a generated anchor that exists, which makes the model larger than the property it proves, so they stay example-tested | -| F6 | minor — ✅ applied | `PillarCollisionError` and `AnchorNotFoundError` carried their symbols as fields but never rendered them into the message, while `PIPE-5` asks the error to "name both step types", `PIPE-21` to identify "the missing type", both 4c documents claimed exactly that, and `error-handling.md:40` requires identifying inputs in the message — a bare `symbol` field is invisible in a stack trace or log line | PLAN errors.ts | Both messages interpolate `String(type)` (`Symbol(retry)`), matching 4a's `DuplicateContextKeyError`; the fields stay for `error-handling.md:44`, and `errors.test.ts` now asserts the message names them | -| F7 | minor — ✅ applied | `StepContext.fork?: () => Next` spelled bare, against the plan's own `exactOptionalPropertyTypes` constraint ("optional properties are spelled `?: T \| undefined`, never bare `?: T`") — the same shape 5a Task 1's added fields will use | SPEC:135, PLAN step.ts | `fork?: (() => Next) \| undefined` in both documents | -| F8 | minor — ✅ applied | Spec's `PipelineBuilder` listing tagged `insertBefore` with `PIPE-19` and `replace` with "PIPE-18/19"; `PIPE-18` covers both inserts and `PIPE-19` covers replace. Also `#exchangeSource` in prose for what is a module-level exported function, not a private field | SPEC:274-275, SPEC:386 | IDs corrected; the prose names `exchangeSource` and says it is the module-level helper | - -**Not findings, recorded so they are not re-raised.** Assertion density (`assertions.md:6-7`) is already open -project-wide as 4b's F2 — 4c is the phase that *satisfies* it, not one that violates it. `STAGE_ORDER` and -`PILLAR_STAGES` in `CONSTANT_CASE` sit against `naming-conventions.md:14`, whose worked example is literally a -module-level `new Set(...)` staying `lowerCamelCase` because its contents can mutate; a `ReadonlySet` type does -not make the underlying `Set` deeply immutable and `Object.freeze` cannot fix a `Set`. Left alone because the -casing question is project-wide (Phase 1's `Protocol`/`Status` statics, 4b's constants) and renaming one phase's -two constants would fork the convention rather than settle it — Phase 10's reconciliation owns it. diff --git a/docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md b/docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md new file mode 100644 index 0000000..9fcf4ff --- /dev/null +++ b/docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md @@ -0,0 +1,294 @@ +# Node.js SDK — v1 Roadmap + +**Status:** Draft, approved for planning. + +**Purpose:** High-level, ordered phase list from empty repo to a spec-conformant v1 of the `nodejs-sdk`. This is +an index, not an implementation plan — each phase gets its own brainstorm → spec → plan cycle when its turn +comes. Do not add implementation detail to this document as phases complete; instead link to the phase's own +spec file. + +**Governing documents:** + +- `docs/product-spec.md` (+ `docs/product-spec/*`) — the language-agnostic, normative contract. Requirement IDs + (`SEAM-*`, `HTTP-*`, `IO-*`, `BODY-*`, `CTX-*`, `PIPE-*`, `RECOV-*`, `RETRY-*`, `REDIR-*`, `AUTH-*`, `PAGE-*`, + `SSE-*`, `SERDE-*`, `OBS-*`, `CFG-*`, `TRANSPORT-*`, `ASYNC-*`, `XCUT-*`, `NFR-*`) are the vocabulary every + phase below cites against. +- `docs/sdk-design-nodejs.md` (+ `docs/sdk-design-nodejs/*`) — the Node/TS port design, already broken into the + seams this roadmap sequences. +- `/home/mohammad/Projects/dexpace/styleguide/typescript/` (core rules) and + `/home/mohammad/Projects/dexpace/styleguide/typescript-bun/` (toolchain/runtime rules) — binding, in force from + Phase 0 onward, for every phase without exception. + +## Cross-Cutting Constraints (apply to every phase, not their own phase) + +- **Styleguide enforcement is continuous**, not a one-time gate. Every phase's code is written and reviewed + against `styleguide/typescript`'s 15 chapters (Tiger Style overlay on Google's TS guide) from the moment the + toolchain exists (Phase 0). +- **Package manager and test runner: Bun, not pnpm.** `sdk-design-nodejs/02` specifies a pnpm workspace; the + styleguide mandates Bun (`bun install`, `bun.lock`, `.bun-version`, `bun test`) as binding for all dexpace + projects. Resolved 2026-07-23 in favor of the styleguide — see the + [scaffold milestone design](./scaffold/2026-07-23-scaffold-milestone-design.md) for the reconciled shape. The + multi-package workspace *layout* from `sdk-design-nodejs/02` (package map, project references, peer-dependency + discipline) still holds; only the pnpm-specific mechanics are replaced. Library packages still build with + plain `tsc` (never `Bun.build`, which is reserved for services), per `typescript-bun/08-build-and-distribution.md`. +- **Dual JS/TS consumption.** TypeScript is the source of truth; the SDK must serve both TS and plain-JS + consumers. `tsc` compiles to ESM JS + `.d.ts`; no TS-only runtime syntax leaks into shipped output (the + styleguide's erasable-syntax stance already helps here — no enums, no decorators, no constructor parameter + properties). Verified per-package as each package is built, not only once at the end. +- **Requirement-ID traceability.** Each phase's deliverable should be traceable back to the product-spec + requirement IDs it satisfies, feeding `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md` + and the Phase 9 conformance pass. + +## Phase List + +| Phase | Name | Package(s) | Product-spec refs | sdk-design refs | +|---|---|---|---|---| +| 0 | Toolchain & Style Gate | workspace root, `@dexpace/core` (stub) | — | §2, §9 (see [scaffold milestone design](./scaffold/2026-07-23-scaffold-milestone-design.md)) | +| 1 | Core HTTP Domain Model | `@dexpace/core` | §4 | §4 | +| 2 | Seam Foundations | `@dexpace/core` | §3 | §3 | +| 3a | I/O Contracts | `@dexpace/core` | §5 | §3.1 (Web Streams direct, no pluggable provider) — see [Phase 3a design](./phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md) | +| 3b | Body Lifecycle | `@dexpace/core` | §6 | §3.1 — see [Phase 3b design](./phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md) | +| 4a | Execution Context | `@dexpace/core` | §7 | §5 — see [Phase 4a design](./phase4/phase4a/2026-07-25-phase4a-execution-context-design.md) | +| 4b | Recovery-Chain Primitives | `@dexpace/core` | §8.2 | §5 — see [Phase 4b design](./phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md) | +| 4c | Stage-Based Pipeline | `@dexpace/core` | §8.1 | §5 — see [Phase 4c design](./phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md) | +| 5a | Resilience — Retry | `@dexpace/core` | §9, appendix C `RECOV-17`–`RECOV-34` | §6 — see [Phase 5a design](./phase5/phase5a/2026-07-26-phase5a-retry-design.md) | +| 5b | Resilience — Redirect | `@dexpace/core` | §10 | §6 — see [Phase 5b design](./phase5/phase5b/2026-07-26-phase5b-redirect-design.md) | +| 5c | Resilience — Auth | `@dexpace/core` | §11 | §6 — see [Phase 5c design](./phase5/phase5c/2026-07-26-phase5c-auth-design.md). Both 5b and 5c were drafted solo/concurrently (user away from keyboard); 5c's own doc records reconciling with 5b's cross-origin-marker design after finding it mid-draft — see its "Alignment with 5b's shipped design" sections | +| 6a | Serde | `@dexpace/core`, `@dexpace/codec-json` | §14 | §7.3 — see [Phase 6 segmentation design](./phase6/2026-07-28-phase6-segmentation-design.md) | +| 6b | SSE | `@dexpace/core` | §13 | §7.2 — see [Phase 6 segmentation design](./phase6/2026-07-28-phase6-segmentation-design.md) | +| 6c | Pagination | `@dexpace/core` | §12 | §7.1 — see [Phase 6 segmentation design](./phase6/2026-07-28-phase6-segmentation-design.md) | +| 7a | Configuration & Platform Primitives | `@dexpace/core` | §16, appendix C `RECOV-33` | §8 — see [Phase 7 segmentation design](./phase7/2026-07-28-phase7-segmentation-design.md) and [Phase 7a design](./phase7/phase7a/2026-07-28-phase7a-configuration-design.md) | +| 7b | Instrumentation & Observability | `@dexpace/core`, `@dexpace/logging-pino`, `@dexpace/logging-debug` | §15 | §8 — see [Phase 7 segmentation design](./phase7/2026-07-28-phase7-segmentation-design.md) and [Phase 7b design](./phase7/phase7b/2026-07-28-phase7b-observability-design.md) | +| 8a | Transport Adapters | `@dexpace/transport-fetch`, `@dexpace/transport-undici`, `@dexpace/body-file`, `@dexpace/transport-shared` | §17 | §3.2 (single `Promise` primitive collapses JVM's SEAM-11/SEAM-16 fragmentation) — see [Phase 8 segmentation design](./phase8/2026-07-28-phase8-segmentation-design.md) and [Phase 8a design](./phase8/phase8a/2026-07-28-phase8a-transport-design.md) | +| 8b | Async-Runtime Bridge | `@dexpace/rx` | §18 | §3.2 (RxJS `Observable` is the only Node-worthwhile async adapter) — see [Phase 8 segmentation design](./phase8/2026-07-28-phase8-segmentation-design.md) and [Phase 8b design](./phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md) | +| 9 | Cross-Cutting Invariants & Conformance | all packages, `@dexpace/shrink-test` | §19, §20, appendix B | — see [Phase 9 design](./phase9/2026-07-28-phase9-cross-cutting-conformance-design.md) and [Phase 9 plan](./phase9/2026-07-28-phase9-cross-cutting-conformance.md) | +| 10 | Deviation Reconciliation | `@dexpace/core`, `@dexpace/transport-fetch`, `@dexpace/transport-undici`, `@dexpace/shrink-test` — **corrected 2026-08-30**; this cell read `— (review only)` and the phase shipped code. See the Phase 10 status note below | — | §10 | + +**Status note (2026-07-27).** Phases 5a/5b/5c have a design **and** a written implementation plan; none of the +three has been executed — no `src/retry/`, `src/redirect/`, or `src/auth/` exists yet. 5b's and 5c's plans were +reviewed against the knowledge corpus and against each other's declared APIs before execution; the corrections +that outlive their own phase are logged below (see the `cross-origin.ts`, `AuthTiers`, preemptive-stamp, and +`DigestChallengeUnsupportedError` rows). Everything else stayed inside the two plans' own Deviation Ledgers. + +**Status note (2026-07-28).** A cross-phase deferral review swept this log against every written design/plan. +Two real gaps were found and folded into the unexecuted plans: `StepContext` never exposed the caller's per-call +`RequestOptions` (`PIPE-17`'s "readable by any step" MUST — extended 5a Task 1's amendment to two fields), which +in turn left `RETRY-41`'s per-call retry-count override (`RequestOptions.maxRetries`, `HTTP-35`) wired to +nothing (now read by 5a Task 9) and left `AUTH-4`'s `perCall` tier with no per-call source (now +`RequestOptions.auth?: AuthDescriptor`, amended in 5c Task 14). Bookkeeping: the rows targeting Phase 2 and +Phase 3b below were marked resolved-at-design/plan level, and `NFR-13`'s SPDX convention was written into +Phase 1's plan. No executed code exists yet, so every change was a document edit, not a retrofit. + +**Status note (2026-07-28, later same day).** Phase 6 was brainstormed and split into 6a (Serde) / 6b (SSE) / +6c (Pagination) — see the [Phase 6 segmentation design](./phase6/2026-07-28-phase6-segmentation-design.md). The split +review produced three findings recorded in the log below that outlive the sizing question: three Phase-0 deferrals +(`NFR-2`, `NFR-14`, peer-dependency dedup) become live in 6a rather than Phase 8, because `@dexpace/codec-json` — +not a transport adapter — is the workspace's first second package; `sdk-design-nodejs/07`'s item-view snippet +contradicts `PAGE-11`'s close-before-yield MUST in a way appendix B's own conformance test does not catch; and +`PAGE-5`'s "synchronously inside parse" needs an explicit re-expression for a runtime with no synchronous body +read. + +**Status note (2026-07-28, end of day).** All three sub-phases now have **both** a design and a written +implementation plan (`specs/2026-07-28-phase6{a,b,c}-*-design.md`, `plans/2026-07-28-phase6{a,b,c}-*.md`); none +has been executed — no `src/serde/`, `src/sse/`, `src/pagination/`, or `packages/codec-json/` exists yet. The +three plans were then reviewed against each other and against the knowledge corpus, the same pass 5b/5c got. The +corrections that outlive their own sub-phase are logged below (the `Symbol.asyncDispose` row, whose stated +premise 6b/6c invalidate, and the `PAGE-11` erratum row, which needed carrying into `docs/knowledge/` and not +only into `sdk-design-nodejs/07`). Everything else stayed inside the three plans' own task lists and Deviation +Ledgers. One process note worth keeping: the segmentation design declares the three sub-phases order-free, but +each plan's **Prerequisite** section had been written as a linear chain (6b "Phases 0 through 6a", 6c "0 through +6b"), which would have silently re-imposed the dependency the split exists to avoid. All three now state +"Phases 0 through 5c" plus an explicit note naming what — if anything — a sibling sub-phase adds. + +**Ordering rationale:** toolchain first (Phase 0) so every subsequent phase is written under the style/quality +gates from line one. From there, bottom-up by dependency: domain model before the seams that operate on it, +seams before the pipelines built on top of them, pipelines before the resilience layer wrapping them, and +pagination/SSE/serde/instrumentation as the outer layers consuming everything underneath. Transport and +async-runtime adapters (Phase 8) come late because they are the most Node-specific judgment calls (per +sdk-design's §3 framing) and benefit from every other seam already being stable. Conformance (Phase 9) and +deviation reconciliation (Phase 10) close the roadmap by construction — they audit what phases 0-8 built rather +than building anything new. + +## How Phases Get Executed + +Each phase, when its turn comes: + +1. Its own brainstorming session — scoped to that phase alone, referencing this roadmap for context. +2. A spec file. +3. Its own implementation plan (via the writing-plans skill), executed independently. + +Both land in `docs/superpowers/` first — the `brainstorming` and `writing-plans` skills hard-code that +path — and are collected from there into `docs/work/mvp/phaseN/`, which is where a phase's design, plan +and checklist live once the phase is done. The `housekeeping` skill does the collecting. + +This document is updated only to mark a phase's status (not-started / in-progress / done) and link to its spec +once written — it does not absorb implementation detail from completed phases. It carried one exception until +2026-08-31, the Deferred Items Log, which is now `docs/deferred-items.md`. Every +phase's brainstorming session should check that register for entries targeting it before starting, and append +any new deferral it produces before that phase is considered done — this is how a decision made in Phase 0 +("we'll handle NFR-2 properly once adapter packages exist") doesn't silently evaporate by Phase 8. + +## Deferred Items Log + +**Moved out on 2026-08-31.** The aggregate log — 74 rows — is now +`docs/deferred-items.md`, a register at the `docs/` root beside `open-items.md` +and `deviations.md`. + +It was here because there was nowhere else to put it, and this document's own rule (["How Phases Get +Executed"](#how-phases-get-executed)) had to carve out an exception for it: the roadmap records phase +*status*, "**Exception:** the Deferred Items Log below." The exception is gone with the log. A phase's +brainstorm still checks the register before starting and appends to it before the phase is done — at the +new path. + +## Phase Status Notes + +**Reading these.** Each note below is dated and is not retro-edited. Written when the log sat in this file, +they say "the row above" and "the rows above"; every such reference now means a row of +`docs/deferred-items.md`, and the ones that name a specific row have been +repointed in place. The four `## Open Findings` review sections that used to follow them are +[`docs/work/mvp/2026-09-04-open-items-dissolution.md`](./2026-09-04-open-items-dissolution.md) Sections Q, R, S and T. + +**Status note (2026-07-28, Phase 7).** Phase 7 was brainstormed and split into 7a (Configuration & Platform +Primitives, `§16`) / 7b (Instrumentation & Observability, `§15`) — see the +[Phase 7 segmentation design](./phase7/2026-07-28-phase7-segmentation-design.md). Unlike Phase 6's three segments, this +split has one real (if soft) cross-segment dependency — `OBS-35`'s log-level resolution wants 7a's `Configuration` +— so 7a leads and 7b trails deliberately, rather than "order is convenience only." Both sub-phases got full +designs in this same session (not just a segmentation note): [7a](./phase7/phase7a/2026-07-28-phase7a-configuration-design.md) +and [7b](./phase7/phase7b/2026-07-28-phase7b-observability-design.md). All six Deferred Items Log rows that previously targeted +bare "Phase 7" are updated in `docs/deferred-items.md` to point at 7a or 7b specifically, each marked resolved-at-design-level. Three +new retrofits to 5a's already-written (still unexecuted) design/plan came out of 7a's brainstorm (`Clock`, RFC +1123 parser, and `RETRY-1`/`CFG-35` retryable-status single-sourcing); two more amendments — to 5a's and 5b's +steps for structured logging, and to 5c's preset for the `LOGGING` slot — came out of 7b's. No executed code +exists yet for any phase, so every change listed here is a document edit, not a retrofit to shipped code. + +**Execution order is no longer the numeric order for Phase 5.** These five retrofits do not merely annotate 5a/5b/5c +— they make Phase 7 a *prerequisite* of Phase 5's execution, in both directions the amendment banners record: +7a's `config/{clock,http-date,retryable}.ts` must exist before 5a's plan runs (its Task 8 consumes `Clock`), and +7b's `observability/{logger,redaction,logging-step}.ts` must exist before 5b's Task 6 and 5c's Task 16 run. The +**Ordering rationale** above ("resilience layer... instrumentation as the outer layers consuming everything +underneath") describes the dependency direction as originally designed; it holds for everything except these +named modules, which invert it. Anyone executing plans in roadmap order must run 7a (and, for 5b/5c, 7b) first, +or execute 5a/5b/5c against the pre-amendment text and accept a duplicate-implementation deviation. Each affected +plan's own **Prerequisite** section states this; this note exists so the roadmap does not read as contradicting +them. + +**Status note (2026-07-28, Phase 8).** Phase 8 was brainstormed solo (user away from keyboard, `docs/knowledge/` +as standing tie-breaker per standing instruction) and split into 8a (Transport Adapters, `§17`) / 8b +(Async-Runtime Bridge, `§18`) — see the [Phase 8 segmentation design](./phase8/2026-07-28-phase8-segmentation-design.md). +Only a segmentation document was produced this session, not full per-sub-phase designs (unlike Phase 7, which got +both in one sitting) — 8a and 8b each still need their own brainstorm → spec → plan cycle. Nine Deferred Items +Log rows that previously targeted bare "Phase 8" or "first concrete Transport" are updated there to point at 8a +or 8b specifically; none is resolved-at-design-level yet, only re-targeted and, where the segmentation review's +own analysis showed it, pre-dispositioned as collapsed/not-applicable (recorded in the segmentation design's §5, +carried forward into 8a's/8b's own row-by-row tables when those designs are written, not re-derived). Two package +column changes: Phase 8's roadmap-table row splits into 8a/8b, and the segmentation design flags a **possible +fourth package** (`FileBody`'s home, e.g. `@dexpace/body-node`) that 8a's own design must confirm or reject +before the roadmap table can be updated further — not decided by this pass. No executed code exists yet for any +phase, so every change here is a document edit. + +**Status note (2026-07-28, Phase 8, continued).** Both sub-phases got full designs and written implementation +plans in a follow-up pass this same day: [8a design](./phase8/phase8a/2026-07-28-phase8a-transport-design.md) / +[8a plan](./phase8/phase8a/2026-07-28-phase8a-transport.md) and [8b design](./phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md) / +[8b plan](./phase8/phase8b/2026-07-28-phase8b-async-runtime.md). Neither plan has been executed — no `packages/` +directory exists in this repository as of this pass. The "possible fourth package" question above is settled: +8a's design confirms `@dexpace/body-file` (a fourth Phase 8a package, `FileBody`'s concrete factory) plus a fifth, +`@dexpace/transport-shared` (header-mapping helpers both transports need identically, found necessary only once +the plan reached implementation-level detail — the segmentation design and 8a's own design doc did not anticipate +this fifth package; it surfaced from "don't duplicate the same algorithm in two sibling packages" rather than +from any `TRANSPORT-N` requirement directly). The roadmap table's 8a row above is updated to list all four +published packages. `challengeHandler`'s protocol and the zero-copy-dispatch question are both resolved (not +merely flagged) in 8a's design — see the updated Deferred Items Log rows in `docs/deferred-items.md`. 8b's design resolved `ASYNC-18` +as inapplicable to the whole port, not merely out of 8b's scope — a correction to the segmentation design's +framing, recorded in 8b's design §3 and not requiring a Deferred Items Log row of its own since nothing was ever +targeted at a phase to begin with. + +**Status note (2026-08-26, Phase 5a EXECUTED).** Phase 5a is implemented and green across the full gate +sequence — the first phase to run out of numeric order, per the execution-order note above. Closed by this +execution: `PIPE-36`, `PIPE-17`'s "readable by any step" MUST (via `StepContext.options`), +`StepContext.signal`, the `FakeTransport` double, `RECOV-32`, and the `RECOV-17`-`RECOV-34` reconciliation — +each row there already anticipated 5a and is now satisfied in code rather than only at design level. Two new +rows were added: the Phase 7b log-event deferral, and the record that 7a's Tasks 1-3 were executed early as +5a's prerequisite. Still deferred out of 5a: `RETRY-29` (not scheduled), `RECOV-33` (7a Task 9), and +public-barrel promotion of the step-authoring surface (5c) — `packages/core/etc/core.api.md` is byte-identical +across the phase, which is that decision's mechanical proof. Per-requirement disposition: +[2026-07-26-phase5a-retry-checklist.md](./phase5/phase5a/2026-07-26-phase5a-retry-checklist.md). + +**Status note (2026-07-28, Phase 9).** Phase 9 was brainstormed solo (user away from keyboard, `docs/knowledge/` +as standing tie-breaker per standing precedent) and got a full design **and** a written implementation plan in +one session: [design](./phase9/2026-07-28-phase9-cross-cutting-conformance-design.md) / +[plan](./phase9/2026-07-28-phase9-cross-cutting-conformance.md). Neither has been executed — no `packages/` +directory exists in this repository as of this pass. Per the roadmap's own framing ("audits what phases 0-8 built +rather than building anything new"), Phase 9's scope is deliberately narrow: a per-ID disposition table for all +24 `XCUT` IDs and all 17 `NFR` IDs (the grep across every prior spec/plan turned up exactly two incidental +`XCUT-N` citations before this pass, confirming this is the first systematic tabulation of that family), one new +package (`@dexpace/shrink-test`, closing `NFR-9`), and one new top-level `tests/conformance/xcut/` integration +suite driving 5c/7b's `standardResilience()` composed pipeline — not a general re-litigation of every open +judgment call that happened to say "Phase 9" in this log. Three consequences of that narrower scope: + +- `NFR-9` closes here (design-level) — see the updated row in `docs/deferred-items.md`. +- One deferred item closes here too: whether `standardResilience()` needs a `tracerFactory`/`meter` pass-through + convenience — resolved no, the composed-pipeline fixture needed no such convenience (see the updated row there). +- Four deferred items that targeted "Phase 9 conformance sweep" turned out to be `AUTH-*`/`REDIR-*` interpretive + judgment calls or preset-shape questions, not `XCUT`/`NFR` conformance checks, and are retargeted there to + Phase 10 (Deviation Reconciliation) — the roadmap's other audit-only phase and the one that already carries + this class of write-up. This retargeting is a document edit only; it does not touch Phase 10's own design or + plan files. + +Also closed as part of this pass: three `unresolved 2026-07-25` markers in `docs/knowledge/tooling-and-quality-gates.md` +(package manager/lockfile, test-runner/coverage-gating, `gts` baseline) that a 2026-07-25 cross-phase checkpoint +had already decided but never back-ported into the corpus itself — directly relevant here since `NFR-5`/`NFR-6`/ +`NFR-7` are exactly the rows those stale markers left unconfirmed. + +**Status note (2026-08-30, Phase 10 EXECUTED — scope corrected).** Phase 10 is executed, and it **shipped code**. +The phase-table row above and this phase's own design (`2026-07-28-phase10-deviation-reconciliation-design.md:15`, +"Phase 10 ships no package") both said the opposite; both are corrected in place rather than overwritten, because +an unrecorded scope change is the exact failure mode this phase spent its audit correcting elsewhere. What +actually landed, on `25-phase-10-deviation-reconciliation`: + +- **A live defect, found by auditing the ledger against source rather than against the specs that produced it.** + `Page`, `FetchTransport` and `UndiciTransport` each declared `[Symbol.asyncDispose]` as a plain computed class + member. The symbol arrived in Node 20.4 and every package declares `engines.node ">=20.3"`, so on the declared + floor the computed key evaluated to `undefined` and the method bound to the string key `"undefined"` — junk on + the prototype, no disposal, and a `.d.ts` promising `AsyncDisposable` regardless (`NFR-10`). All three now + install it through a guarded module-scope `Object.defineProperty`, matching `SseStream` + (`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:314`, + `packages/transport-undici/src/undici-transport.ts:566`, `packages/core/src/sse/stream.ts:209`). +- **A breaking type change across three packages,** with two changesets: `Page` no longer declares `implements + AsyncDisposable` and the two transport factories no longer return `Transport & AsyncDisposable`, so `await + using` stops type-checking. Pre-1.0, so `minor` per the same initial-development carve-out the earlier `Body` + narrowing used. +- **A new blocking CI step** closing `NFR-12` on evidence — `bun run verify:reproducible-build` + (`scripts/verify-reproducible-build.mjs`), see the `NFR-12` row in `docs/deferred-items.md`. +- **Three further defects, from three subsequent review passes:** a `verify-dual-consumption` assertion that + passed on the floor only *because* of the junk prototype key, a dispatcher leak in `UndiciTransport.close()` + where the first rejecting `destroy()` aborted the reverse walk and stranded the `ProxyAgent` holding the pooled + connections, and a stranded body producer in `send()` from evaluating `prepareBody()` before header mapping. +- **An extended shrink guard** — `packages/shrink-test/` now asserts the disposal installs survive a real esbuild + `bundle + minify + treeShaking` pass, which is the standing evidence for keeping `"sideEffects": false` on the + three packages carrying one. + +**Why the "review only" scope was right to break, and where that judgment is recorded.** The audit's method — +re-derive every ledger claim from as-built source — is what surfaced the defect; a documents-only phase would +have copied the wrong claim forward. Fixing a live correctness defect found *by* the audit is inside the phase's +purpose, and leaving it recorded-but-unfixed would have shipped a `.d.ts` that lies on the declared floor. The +project-wide **convention sweeps** that also named Phase 10 were held to the original scope and re-deferred +instead — see the three rows added to `docs/deferred-items.md` and the dated dispositions on 4b's F2/F7 and +4c's `CONSTANT_CASE` note, now `docs/work/mvp/2026-09-04-open-items-dissolution.md` Sections S and T. Per-item evidence: `docs/deviations.md` (the as-built audit). + +## Open Findings + +**Moved out on 2026-08-31.** The four review sections that used to close this document are now +[`docs/work/mvp/2026-09-04-open-items-dissolution.md`](./2026-09-04-open-items-dissolution.md): + +| Was | Now | +|---|---| +| `## Open Findings — Phase 3b Validation Review (2026-07-28)` | Section Q | +| `## Open Findings — Phase 3b Execution (2026-08-25, expanded 2026-08-26)` | Section R | +| `## Open Findings — Phase 4b Validation Review (2026-07-28)` | Section S | +| `## Open Findings — Phase 4c Validation Review (2026-07-29)` | Section T | + +They are review findings against phase documents, which is the running register's subject, not the roadmap's. +Each moved verbatim, with a relocation banner naming its origin. **Their row IDs did not change and are not +this register's item IDs:** Sections S and T each number their rows `F1`–`F10` and `F1`–`F9`, the reviews' +own numbering, which collides with Section F's items. A citation has to name the section — "Section S's F2", +never a bare "F2". diff --git a/docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md b/docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md similarity index 98% rename from docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md rename to docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md index 775fa29..32e4926 100644 --- a/docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md +++ b/docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md @@ -2,10 +2,10 @@ **Purpose:** a single gate to run before starting Phase 3b (or Phase 4, if 3b is folded elsewhere) that verifies the four already-planned phases — -[scaffold](./2026-07-23-scaffold-milestone.md), -[Phase 1 (core HTTP domain model)](./2026-07-23-phase1-core-http-domain-model.md), -[Phase 2 (seam foundations)](./2026-07-23-phase2-seam-foundations.md), -[Phase 3a (I/O contracts)](./2026-07-24-phase3a-io-contracts.md) +[scaffold](./scaffold/2026-07-23-scaffold-milestone.md), +[Phase 1 (core HTTP domain model)](./phase1/2026-07-23-phase1-core-http-domain-model.md), +[Phase 2 (seam foundations)](./phase2/2026-07-23-phase2-seam-foundations.md), +[Phase 3a (I/O contracts)](./phase3/phase3a/2026-07-24-phase3a-io-contracts.md) — are not just individually self-reviewed but hold together as one artifact, and that nothing they trade off against `docs/knowledge` (the styleguide + spec + design-doc corpus) went unrecorded. @@ -107,7 +107,7 @@ stop reading any single phase's file. | Bun workspace catalogs as the `NFR-14` mechanism (§5.8) | This checkpoint | Phase 8 — do not add a catalog block while `@dexpace/core` is the only package; there is nothing to deduplicate and it adds indirection with no payoff | | Node-runtime conformance suite `test:node`, matrixed floor + LTS (§5.9) | This checkpoint | Seed now with `composeSignal` + Phase 3a `io/`; **every later phase touching a runtime-divergent surface (Phase 3b bodies, Phase 4 pipelines, Phase 8 transports) must add to it, not just to `bun test`** | -- [ ] Each row's target phase still exists in the current roadmap (spot-check against `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` if it's been revised since these plans were written). +- [ ] Each row's target phase still exists in the current roadmap (spot-check against `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` if it's been revised since these plans were written). ## 5. Knowledge-base validation findings diff --git a/docs/work/mvp/2026-09-04-open-items-dissolution.md b/docs/work/mvp/2026-09-04-open-items-dissolution.md new file mode 100644 index 0000000..cd34b16 --- /dev/null +++ b/docs/work/mvp/2026-09-04-open-items-dissolution.md @@ -0,0 +1,1905 @@ +# Open items — the register, dissolved 2026-09-04 + +**This file is an archive of record. Nothing is appended to it.** + +`docs/open-items.md` was the running register of everything known to be unmet, unverified, misreported +or surprising across the implemented portion of this project. On 2026-09-04 a maintainer pass decided +every open question in it rather than re-triaging them, and what remained no longer earned a register +at the `docs/` root. The file was moved here, whole, on that day. + +It is the third and last register to be dissolved. `deferred-items.md` went on the same date +([the purge note](./2026-09-04-register-retirement-purge.md)), its `NFR-16` row becoming +[`first-release.md`](../../first-release.md). Two registers remain and neither is a successor to this +one: + +| Register | Holds | +|---|---| +| [`deviations.md`](../../deviations.md) | Where this port deliberately differs from the reference contract, including deviations found outside a phase | +| [`first-release.md`](../../first-release.md) | Release readiness, the blockers before a first publish, and the decisions owed before the first version bump | + +**Where a finding goes now.** A deviation goes to `deviations.md`. A release blocker or a +before-the-bump decision goes to `first-release.md`. Everything else goes where it is enforced: a +gate, a test, or a TSDoc comment on the thing it concerns. That is the change this dissolution makes +— a concern that only a register remembered was a concern nothing acted on, which is how twenty items +came to name a phase that had shipped without doing the work. + +**Item IDs stay reserved and still resolve.** They are cited from source comments (`docs/open-items.md +K16` in `packages/core/src/index.ts`, `K18` in `config/build-info.ts`, and so on), and those citations +were rewritten to name this file on 2026-09-04. No ID is ever renumbered or reused, here or anywhere. +The 102 IDs retired before that date are in [the purge note](./2026-09-04-register-retirement-purge.md). + +**What was decided on the day this closed**, beyond the items already marked below: `K1`, `K12`, +`H19`, `N3`, `V11`, `W1`, `V2`, `H9`, `H10`, `H15`, `N4`, `G13`, `O2`, `F4`, `X1`, `X2` and `X3` were +fixed in code, gates or notes; `K11`, `K19`, `H11` and `K13` were closed on a reading; `G1`'s erratum +was drafted into `deviations.md` for the specification owner to apply. The `WATCH` and `RECORDED` +rows below were neither: a `WATCH` is not a defect and a `RECORDED` row is a note about a decision +already taken, so both survive as reasoning rather than as work. Their triggers live on the code they +concern. + +--- + +## Section index + +Each section is a review. Its letter is permanent: source comments cite items as `docs/open-items.md K11`. +**A letter is never reused and an item is never renumbered.** A new review appends the next letter. + +How many such citations exist is not written here. One command derives it, from the same regex and file +set the check uses. Three documents once stated three different, all-wrong counts of it, and this +paragraph was one of them until 2026-09-02: + +```bash +node .claude/skills/housekeeping/probe.mjs --only=citations +``` + +**A citation into Section R is written with a section qualifier** — `open-items.md R.E3`, never a bare +`E3` — because that section's rows carry the 3b execution review's own numbering rather than this register's +item IDs, and the three sibling reviews relocated beside it number their rows in `D` and `F` namespaces that +collide with Section F's. The probe's citation check resolves the qualified and the bare form alike. + +**The "Item IDs" column names what each section still holds, and nothing else.** A letter missing from the +table below is spent, not free: its items are all closed, and it is never reused. + +| Section | Subject | Item IDs | +|---|---|---| +| A, C | Phase 1, re-verified at every review since | `A2`, `A4`, `A6`; `C1`, `C3` | +| D | Scheduled deferrals, Phase 1 onward | **none.** A bare table; its rows are cited by the anchors on them, or by row title, not by an item ID | +| F | Phase 4b — recovery-chain primitives | `F1`, `F2`, `F4`, `F7`, `F9` | +| G | Phase 5b — redirect | `G1`, `G5`, `G6`, `G8`, `G9`, `G13` | +| H | Phase 6a — serde | `H4`, `H7`–`H11`, `H15`–`H20`. `H10` and `H15` are **MOVED**: the headings are held here, the bodies are in [`first-release.md`](../../first-release.md) | +| I | Phase 6b — Server-Sent Events | `I2`, `I3`, `I4` | +| K | Phase 7a — configuration and platform primitives | `K1`, `K3`, `K6`–`K8`, `K11`–`K13`, `K16`, `K18`–`K20`. Closed 2026-09-04: `K1` and `K12` **FIXED**, `K11` and `K19` **CLOSED**. Their headings are held here — `K11` is cited from `packages/core/src/config/build-info.ts:37` and from `CLAUDE.md` | +| L | Phase 7b — instrumentation and observability | `L1`, `L4`. `L1` is a SPLIT: its `OBS-19` and `OBS-28` halves are closed, `OBS-29` is live as `V2` | +| M | Phase 8b — async-runtime bridge | `M1` | +| N | Phase 9 — cross-cutting invariants and conformance | `N3`, `N4` | +| O | Knowledge-corpus split | `O1`, `O2` | +| P | Phase 5a — retry (merged from the repository-root register, 2026-08-31) | `P3`–`P8` | +| R | Phase 3b execution, relocated from the roadmap, 2026-08-31 | **table rows, not `###` items:** `E3`, `E4`, the review's own numbering. Cite one qualified — `R.E3` | +| U | Documentation restructure | `U4`, `U5` | +| V | Register audit, 2026-09-02 | `V2`, `V11` | +| W | Register dispositions taken 2026-09-04 | `W1` | +| X | Holes found while closing the 2026-09-04 decision pass | `X1`–`X4` | + +**Reviewed state.** Scaffold milestone (`0ebdc79`); Phase 1 (branch `2-phase-1-core-http-domain-model`, +uncommitted at time of review); Phases 3a/3b; Phase 4a (`7-phase-4a-execution-context`, three passes); +Phase 4b (`8-phase-4b-recovery-chain-primitives`); Phase 5b (`12-phase-5b-resilience-redirect`, three +passes); Phase 5a (three passes, now Section P, re-verified against source 2026-08-31); Phases 6a/6b/6c, +7a/7b, 8b, 9. Register-wide audit of every section against the tree, 2026-09-02 (Section V). Last +reviewed **2026-09-02**. + +**Phase 4c is still not registered here.** It is merged and has an executed checklist, but never ran the scan +this file's maintenance rule asks for, so its absence means "not reviewed", not "nothing found". The 4b and +4c validation reviews read those phases' *documents* before either was executed; neither is a review of the +shipped code. Phase 5a's gap closed on 2026-08-31 with Section P. + +**Status vocabulary** + +| Status | Meaning | +|---|---| +| **DECIDE** | Blocked on a human decision. Two or more defensible answers; picking one is the work. | +| **ACT** | Decision already made or obvious; the work is simply not done. | +| **SCHEDULED** | Deliberately deferred to a named phase. No action now; listed so it cannot be lost. | +| **WATCH** | Not a defect today. Becomes one when a stated trigger fires. | +| **UNSCHEDULED** | Real, unowned, and deliberately not scheduled. The roadmap's phase table ends at Phase 10 and every phase has shipped, so there is no phase to name; the row carries `trigger: …` instead, and no phase is invented to hold it. Added 2026-09-02, when the audit found twenty items naming a phase that had closed without doing the work. | +| **BLOCKED** | Real, understood, and stopped on a decision that is the owner's to make — not merely unowned. Distinct from `DECIDE` in that the analysis is finished and the blocking reason is named. | +| **FIXED** | Closed by work done for this register, with `file:line` evidence in the item's own dated note. | +| **CLOSED** | Closed by a reading rather than by work: the premise was false, the requirement is satisfied by delegation, or the decision is won't-fix. The reasoning is in the item. | +| **MOVED** | Still live, but its body now lives in another `docs/` register and is maintained there. The heading and the ID stay here, reserved and resolving, with a pointer in place of the body. Added 2026-09-04, when the two items whose only trigger was the first release went to [`first-release.md`](../../first-release.md). | + +--- + +## Section A — Requirements unmet or misreported + +### A2 — HTTP-22: the checklist describes an implementation that does not exist — **ACT** + +Phase 1 checklist, HTTP-22 row: `✅ | Task 7, HeaderName.of()'s static cache`. + +No such cache exists. The plan deliberately dropped interning (Task 7's `HeaderName` comment: "No interning: +HTTP-22 makes it a MAY, and an intern map keyed by caller-supplied names is exactly the unbounded, +process-lived, caller-influenced map XCUT-14's drain-to-cap rule forbids"), and +`packages/core/src/http/headers.ts` has no static map on `HeaderName`. + +The decision is right and the requirement is a MAY, so nothing about the code needs to change. The checklist +row is simply false and should read ⏳/N/A with the XCUT-14 reasoning, not ✅. + +### A4 — SEAM-1 is enforced narrowly relative to its conformance text — **ACT** + +`scripts/verify-seam-1.mjs` asserts `packages/core/package.json`'s `dependencies` is `{}`. The spec's +conformance clause is broader: "a dependency audit of the core module finds only the standard library plus the +compile-scope logging facade; **no transport/codec/stream symbol is referenced from core**." + +Blind spots today: `peerDependencies`, `optionalDependencies`, and `bundleDependencies` are unchecked, and +nothing inspects what the source actually imports. Low risk while core imports nothing but `URL`, but the gate +reads as stronger than it is. Cheap hardening: assert the other three dependency keys are absent-or-empty, and +add an import scan over `packages/core/src` allowing only relative specifiers and `node:`-prefixed builtins. + +--- + +### A6 — CTX-12 / XCUT-14: the drain **loop**'s shape is unverifiable, and untested — **WATCH** + +`ContextStore.#drain` is a post-insert loop, as CTX-12 (SHOULD) and XCUT-14 (MUST) require. No test proves it +is a loop, and none can: `install` and `installIfAbsent` each set exactly one key before draining, so the map +is never more than one over the cap at drain entry and a second pass is unreachable. Replacing the loop body +with a single check-then-evict breaks nothing — confirmed by mutation testing across the module (that mutant is +the only meaningful survivor of 23). + +The Phase 4a plan's Self-Review claims CTX-12 is covered by "a property-style burst test [that] asserts the +size never overshoots after any single insert". That test is real and passing, but it pins the **bound**, not +the drain's shape. + +Not a defect today: the loop is present, the bound holds, and on a single-threaded runtime the two shapes are +behaviorally identical. `#drain` and the drain `describe` block both now carry a note saying so, so the loop is +not "simplified" away by a later reader. + +**Trigger:** a runtime where inserts can stack more than one overshoot before a drain runs (worker threads, a +future concurrent store), or any change that lets the map exceed `cap + 1`. At that point the shape becomes +observable and owes a real test. + +--- + +## Section C — Documentation defects + +### C1 — Phase 1's scope statement contradicts its own plan — **ACT** + +`docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md` says the scope is "Full +`product-spec/04-core-http-domain-model.md` (HTTP-3 through HTTP-53, both MUST and SHOULD level) in one phase." + +The plan's own Self-Review then amends that: *"The Phase 1 spec's scope statement should be read — and amended +— as HTTP-3..35, 46..50, 53"*, with the body-lifecycle cluster deferred to Phase 3b. The amendment was never +applied to the design doc, so read literally the two documents disagree about what Phase 1 owed. Correct the +design doc's scope line to match the plan. + +### C3 — The Phase 4 checklist under-reports Phase 4a — **ACT** + +`docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md` still carries its +banner: "the plans are reviewed and corrected as of 2026-07-26 but **not yet executed**. Every ✅ means 'the +plan builds and tests it,' not 'it is on `main`.'" Phase 4a's rows are now built, tested, and committed on +`7-phase-4a-execution-context`, so the banner understates them while 4b and 4c remain unbuilt. + +The same checklist maps only `CTX-*`. It has no `XCUT-14` row, even though +`docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md:66` names "4a's context registry" +as an XCUT-14 site and appendix B's only conformance row that `ContextStore` satisfies is B.8's +"Caller/server-keyed maps bounded with drain-to-cap loop (XCUT-14)" — appendix B has no CTX section at all. The +ID is now cited in `store.ts` and `store.test.ts`; the checklist is the remaining gap. + +Split the banner per sub-phase, and add an `XCUT-14` row pointing at 4a Task 4 (qualified by A6 above). + +--- + +## Section D — Scheduled deferrals + +> **Historical, as of 2026-09-02, and doubly so now.** `docs/deferred-items.md` was the authoritative +> deferral register when this section was written; it was dissolved on 2026-09-04 (Section W), so this +> section's rows are the surviving long-form reasoning rather than a pointer to a shorter aggregate. The +> `NFR-16` provenance row's live half is [`first-release.md`](../../first-release.md). The rows that stay do so +> because other documents link to them — the +> `await using` row and the `NFR-16` provenance row each carry an HTML anchor — and because a row's +> reasoning is often longer here than in the aggregate. Cite a row by that anchor or by its title, +> never by line. The **status marks below were re-derived against the tree on 2026-09-02**; the rows +> themselves are not re-litigated here. + +No action now. Each is already owned by a named phase; this table exists so none can quietly lapse. + +| Item | Requirement | Owner phase | Note | +|---|---|---|---| +| `Request.equals` compares body by reference, not by value | HTTP-46 (body clause) | 3b | Blocked on a real `Body` model supplying value equality. **Still open 2026-09-02** — `Body` ships but exposes no value equality, so the blocker stands | +| `RequestConditions.applyTo` cannot emit an obs-text ETag | HTTP-18 vs HTTP-48/50 | 10 | Spec text in scope does not resolve the tension; strict outbound path kept rather than guessed. Documented in `applyTo`'s TSDoc | +| `contextsEqual()`, value equality over `ExecutionContext` | CTX-5 (equality framing) | none | Built only if 4b or 4c needs one. `CTX-5`'s operative half — pinning an explicit shared key — ships via `ContextInit.key` | +| Publish + provenance CI job | NFR-16 | release | `prepublishOnly` wired; nothing published yet. **Sharpened 2026-08-29:** there is no release workflow at all and `--provenance` appears in no manifest, workflow, or `.npmrc`. §10's ledger claimed the flag "is scripted"; it never was. Authoring the workflow is actionable **now** — only running it against a real registry is blocked. **2026-09-02:** `.github/workflows/release.yml` is authored (push to `main`, `changesets/action@v1`, `id-token: write`, `NPM_CONFIG_PROVENANCE: 'true'`), so "there is no release workflow at all" is no longer true. Inert until an `NPM_TOKEN` secret exists. Of the two prerequisites that still blocked the first publish, one is now fixed — no manifest carried a `repository` field, and all nine publishable manifests now do — and one is a maintainer call: `.changeset/config.json`'s `"access": "restricted"` conflicts with provenance's public transparency log | +| `await using` support on `Page`, `fetchTransport()`, `undiciTransport()` | NFR-10 | **none — decided against 2026-08-30** | These three declared `[Symbol.asyncDispose]` as a plain class member; on the `>=20.3` floor the computed key is `undefined`, so the method bound to the string key `"undefined"` — junk on the prototype, no disposal, and a `.d.ts` promising `AsyncDisposable` regardless. Fixed 2026-08-29 to `SseStream`'s guarded install, which costs the type-level `await using` affordance (`close()` is unaffected). **This row previously read "raising the floor to `>=20.4` restores the declaration honestly and lets all four sites drop the guard." That is now a rejected option, not a pending one — the floor stays `>=20.3` and all four guarded installs stay.** Four reasons, in the order that decides it. (1) `NFR-10` is **MUST**-level and requires that "the emitted-artifact target and the visible-API level must agree" (`docs/product-spec/20-non-functional-requirements-and-quality-bar.md:29`); the unguarded class member violated it directly, and the guarded install *is* the repair — not a workaround waiting to be undone. (2) The same requirement's next clause: "A capability that genuinely requires a newer runtime MUST be isolated into its own unit that declares the higher floor explicitly; that unit MUST NOT be a hard dependency of the general-purpose core." Raising core's floor to recover `await using` is the exact inverse — it drags every consumer onto a higher runtime for one syntactic affordance. (3) **The floor is derived, not chosen.** `scripts/verify-runtime-floor.mjs:33` pairs language level `es2023` with `>=20.3`, and its own banner comment (`:22-29`) says the floor is "set by the runtime built-ins the SDK calls rather than by the syntax it emits" and that "adding or moving a row here is a reviewed choice about what runtimes the SDK supports, never a mechanical bump." `>=20.3` is the *minimum* Node that runs what this project emits — `globalThis.crypto` is absent from ESM on every Node 18, and `AbortSignal.any()` landed in 20.3.0. Moving it to satisfy a type-level convenience inverts what the gate is for. (4) **There is a decided precedent.** `docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md:208` rejected raising the floor for `SuppressedError` on the same reasoning and shipped a guarded shim instead — `packages/core/src/suppress.ts`. `close()` remains the supported teardown on every runtime; a consumer who has raised *their own* floor to 20.4+ can still reach the installed member through a cast. See §10 ledger item 11 and I3 below; `Page` carries the identical guarded install, decided the same way | +| Erratum for the `PIPE-40` / `REDIR-22` contradiction | PIPE-40 vs REDIR-22 | 10 | **Still open 2026-09-02, and now unowned:** Phase 10 shipped without writing the erratum, and `docs/product-spec/` is frozen. Behavior is chosen and tested; one of the two spec sentences still needs correcting. See G1 | + +--- + +## Section F — Phase 4b (Recovery-Chain Primitives) + +Three review passes ran over this phase; everything they found is either fixed in the branch or listed here. +Nothing below blocks the phase — the `RECOV-1`–`RECOV-16` mapping is satisfied and every CI step is green. + +### F1 — `ResponseRecoveryChain.apply()` still trusts its *seed* outcome — **WATCH** + +`RECOV-8` is absolute: "the response recovery chain's apply operation MUST NOT throw under any input." Pass 2 +found and closed the reachable half — a *step* returning a non-outcome used to raise +`TypeError: undefined is not an object` out of `apply()`, because `toFailureClosingSuccess` read `.kind` outside +its `try`. That function is now total, and three regression tests pin it. + +What is not guarded is the seed: `apply(garbage)` with at least one response step installed throws on the +`current.kind !== 'success'` read at the loop head. Left alone deliberately — `current` is only ever the +caller's argument at that point, and the sole caller is `dispatchWithRecovery`, which constructs it with +`success()` or `wrapCancellation()`. Guarding it needs either a cast plus an optional chain (which +`no-unnecessary-condition` rejects on a typed value) or the postcondition assertions `recovery/` +deliberately does without, assertion density being a won't-fix project-wide. + +**Trigger:** `recovery/` gaining a public export, or any JavaScript caller reaching `apply()` directly. Either +makes the seed a third-party value and this a real defect. + +### F2 — A step returning a non-outcome poisons the fold silently when nothing downstream reads it — **WATCH** + +The mirror of F1 on the value side. A response step returning `undefined` yields `success(undefined)`; if no +later step touches it, `apply()` resolves with a malformed Success and `dispatchWithRecovery` hands `undefined` +back as the response. Nothing throws, so `RECOV-8` holds — the failure surfaces layers away, in the caller. + +This is the concrete cost named in the roadmap's finding F2 (assertion density), and it is why that finding is +recorded as a Deviation Ledger row rather than as "no assertions needed." **Trigger:** the same as F1, or +Phase 5's retry step being the first real third-party-shaped consumer. + +### F4 — The chains are classes where `data-modeling.md:10` asks for free functions — **UNSCHEDULED** (2026-09-02) + +`RequestRecoveryChain` / `ResponseRecoveryChain` own no lifecycle and hold no mutable state, so the corpus +would have them be plain data plus free functions. Kept as classes because `RECOV-14`'s text is written about +the chain and step *instances*, and because the defensive copy wants a construction boundary. Ledgered in the +phase design. + +**2026-09-02: the owner is spent.** Phase 10 shipped and did not touch it; the roadmap ends there, so +no phase is named in its place. The shape is stable and the reasoning above still holds. +**Trigger:** the first change to `recovery/`'s own surface that would rewrite these constructors +anyway — at that point free functions cost nothing extra, and until then the churn buys only +conformity. + +### F7 — `suppress()`'s branch selection is only ever half-covered on any single runtime — **WATCH** + +`suppress()` returns the native `SuppressedError` where the runtime has one and `FallbackSuppressedError` where +it does not. No test forces the other branch by deleting the global — that cannot survive parallel execution +(`docs/knowledge/harvested/testing.md:50`). Coverage comes from the `test:node` matrix instead: `lts/*` exercises the +native branch, the pinned `20.3.0` exercises the fallback. **Trigger:** if the matrix ever collapses to one +runtime, or the floor rises past Node 24 (where the fallback becomes dead code to be deleted, not guarded). + +### F9 — The Phase 4 checklist's ✅ marks for 4a and 4c are still plan-level — **UNSCHEDULED** (2026-09-02) + +`docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md` now says so +explicitly in its Status line, but the §7.x and §8.1 tables read identically to the §8.2 ones that +are now real. Re-scan both when their phases execute, per this file's own maintenance rule. + +**Path corrected 2026-09-02.** This row cited `plans/2026-07-26-…`, a directory that has not existed +since the 2026-08-31 restructure; the file is under `docs/work/mvp/phase4/`. The citation checker +does not catch this, because it matches `docs/open-items.md ` back-references, not file paths. + +**Status 2026-09-02.** 4a and 4c both executed, so the trigger this WATCH named has fired and the +re-scan is owed. `docs/work/` is never retro-edited, so the outcome is a *note*, not a rewrite of the +checklist. **Trigger: the next review that reads the Phase 4 checklist as evidence** — until then no +document depends on those marks, and Sections F, S and T each carry the findings a re-scan would +produce. + +--- + +## Section G — Phase 5b (Redirect) + +Three review passes ran over this phase. Everything they found is either fixed in the branch or listed here. +Nothing below blocks the phase — `REDIR-1`–`REDIR-27` are satisfied, `PIPE-40` is closed, and every CI step is +green. `REDIR-28` is the one requirement in the chapter that ships unimplemented, and it is scheduled. + +### G1 — `PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path — **UNSCHEDULED** (2026-09-02) + +Two `MUST`s naming the same trigger and prescribing opposite dispositions. + +`product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, +**non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." + +`product-spec/10-redirect-handling.md` (`REDIR-22`): "if building the follow-up throws (**non-replayable +body**, downgrade rejection) the current response MUST be closed before the error propagates." + +5b implements `REDIR-22` — closes, then throws — on three grounds: `REDIR-6` independently fixes the control +flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response +never *returned* cannot be "returned unclosed"; specific governs general, since `§10` owns the redirect step's +lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no +caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return +unclosed as it requires. + +Not a code decision left open — the behavior is chosen, tested, and reasoned. What is open is that **one of the +two spec sentences needs an erratum**, which is Phase 10's to write. Recorded in the 5b design's Deviation +Ledger and asserted with the reasoning inline in `redirect-step.test.ts`. + +**2026-09-02: Phase 10 shipped and did not write it, so the owner is spent.** No phase is invented to +replace it. `docs/product-spec/` is a frozen tree, so the erratum is a deliberate hand edit by +whoever owns the specification, not a maintenance action. Nothing in the code waits on it. +~~**Trigger: the next deliberate amendment of `docs/product-spec/08-execution-pipelines.md` or +`docs/product-spec/10-redirect-handling.md`** — the erratum rides along with it.~~ + +**2026-09-04: the erratum is drafted, so what is left is applying it rather than deciding it.** +[`deviations.md`](../../deviations.md) now carries both the deviation row and a *Proposed erratum for +`PIPE-40`* section with the replacement sentence written out. It edits `PIPE-40` only: that +requirement is the general rule and needs to stop naming a trigger `REDIR-22` has already claimed, +while `REDIR-22` is correct as written. **UNSCHEDULED — trigger: the specification owner applying +it.** A hand edit to a frozen tree is theirs; nothing in the code waits on it, and the behaviour is +unaffected either way. + +### G5 — The marker-stripping guard is not the last step before `SEND` — **WATCH** + +`REDIR-11`(c) requires the internal cross-origin marker be removed before dispatch. `stripCrossOriginMarkerStep()` +occupies `POST_AUTH`, but `STAGE_ORDER` runs six more stages after it — `PRE_LOGGING`, `LOGGING`, +`POST_LOGGING`, `PRE_SERDE`, `SERDE`, `POST_SERDE` — before `SEND`. A step installed in any of them runs +*closer to the wire than the guard* and could put the marker back. + +Not a defect today: no step exists in any of those stages, so the guard is effectively last. It is also not a +plausible accident — nothing would write that header by name. + +**Trigger:** a step installed after `POST_AUTH` that copies or synthesizes request headers wholesale rather +than setting named ones. 7b's `loggingStep` and 6a's serde step are the first two occupants of those stages; +neither should touch it, but neither has been read yet. + +### G6 — Loop detection keys on `href`, so a fragment-only difference is a distinct URI — **WATCH** + +`REDIR-16` says "recording every visited absolute URI". `visited` stores `URL.href`, which includes the +fragment — so `https://h/a` → `https://h/a#x` → `https://h/a#y` is three distinct entries, not a loop. + +Correct by the letter (a fragment is part of the URI) and harmless in practice, because `REDIR-17`'s hop cap +bounds the chain regardless — the default budget of 3 stops it. Worth recording only because the reasoning is +non-obvious and the alternative (stripping the fragment before the visited check) would be a silent behavior +change if someone "fixed" it later. + +Verified in the same pass that the *dangerous* normalizations do collapse: `HTTPS://EXAMPLE.COM/a` and +`https://example.com:443/a` both normalize to a href already in the set, so case and default-port variation +cannot be used to spin past the cap. Both are pinned by tests, in `bun test` and on Node's own URL parser. + +### G8 — The 5b design doc's process note claims it is uncommitted — **ACT** + +`docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md:23` ends: "Not committed — left for the user to +review and commit if it holds up." It was committed in `c6603aa` ("Planning (#26)") and has since been amended +twice. The sentence is stale and should be dropped or rewritten; the rest of the process note (that the design +was authored autonomously and every judgment call is re-listed in the Deviation Ledger for challenge) is still +accurate and worth keeping. Left as-is rather than rewritten unilaterally, because it is the author's own +process note about their own delegation. + +### G9 — `retry/engine.ts` was edited by a phase that does not own it — **WATCH** + +5b's review pass 1 found both of its close-before-throw paths replacing the error they were meant to +propagate, because `Response.close()` rethrows whatever cancelling the body raised. The fix needed +`releaseQuietly`/`withReleaseFailure`, which existed as module-private helpers inside 5a's `retry/engine.ts`. +Rather than ship a second copy of a helper whose identity guard is load-bearing, they were extracted to +`packages/core/src/recovery/release.ts` and both call sites now import them. + +The move is behavior-neutral — the diff is one import added and the two functions removed verbatim, and 5a's +suite passes untouched — and the new module has its own tests at 100% coverage. Recorded because a file +belonging to a merged phase changed outside that phase's plan, which is exactly the kind of edit a later +conformance sweep should be able to find an explanation for. + +**Trigger:** none expected. Re-verify at Phase 9 that 5a's checklist rows for `RECOV-12`/`RETRY-22` still point +at code that exists where they say it does. + +--- + +### G13 — two pre-existing cleanups Phase 5c's Reader pass found and deliberately did not take — **UNSCHEDULED** (2026-09-02) + +Both predate 5c, sit in files Passes 1 and 2 declared settled, and were left alone rather than widening a +review pass into a refactor of earlier phases. + +1. **`hasForbiddenOutboundByte` breaks its own family's naming.** `packages/core/src/http/ascii-validation.ts` + exports `hasForbiddenNameByte`, `hasForbiddenInboundValueByte`, and `hasForbiddenOutboundByte` — the + outbound *value* predicate is the only one that omits `Value`. At + `packages/core/src/auth/digest.ts:229-231` and `:426` a reader cannot tell from the call whether the + name rule or the value rule is being applied, and the two differ (HTAB is excepted by one and not the + other). `hasForbiddenOutboundValueByte` restores the symmetry; **nine call sites outside the module, + re-measured 2026-09-02** — `http/headers.ts`, `http/media-type.ts`, `body/media-type-safety.ts` + (two) and `auth/digest.ts` (four). + + **Line numbers corrected 2026-09-02.** This entry cited `digest.ts:213` and `digest.ts:408`; both + hold unrelated prose. +2. **`PipelineBuilder`'s duplicated bucket lookup.** `insertAfter`, `insertBefore`, and `replace` each repeat + the same three lines — `const bucket = this.#buckets.get(anchor.stage);` plus an `invariant` whose message + is identical in all three. One `#requireBucket(stage)` collapses them. + +**Trigger:** the next edit to `ascii-validation.ts` (1) or `pipeline/builder.ts` (2) — stated as a +file rather than as a phase since 2026-09-02, because the roadmap ends at Phase 10 and every phase +has shipped. Both files were re-read on that date and both cleanups are still owed, unchanged; +neither file was touched by the register audit, so the trigger has not fired. + +--- + +## Section H — Phase 6a (Serde) + +Recorded at implementation time, before the three review passes. Everything here is either a deliberate +deviation from the phase plan, a requirement clause satisfied by delegation rather than by code, or work the +phase surfaced and deliberately left out of scope. + +### H4 — plan deviations taken during implementation — **RECORDED** + +Six places where the shipped code departs from +`docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde.md` as written (path corrected 2026-09-02; +the `plans/` directory has not existed since the 2026-08-31 restructure): + +1. **`packages/codec-json` declares `engines.node: ">=20.3"`, not the plan's `">=18.17"`.** The plan also told + Task 8 to copy `target`/`lib` verbatim from core (ES2023) and to stop if the two disagreed. They did: + `scripts/verify-runtime-floor.mjs` pairs `es2023` with `>=20.3`, and a package peer-depending on + `@dexpace/core` cannot honestly declare a floor below core's own. Raised to match. +2. **`decodeResponse`'s `closingAfter` is built on Phase 4b's `releaseQuietly`/`withReleaseFailure`, not on a + fresh `SuppressedError` construction.** The plan's blocking notice resolved to the guarded `suppress()` + helper; 4b already wraps that helper in a pair that also carries the identity guard `Response.close()`'s + memoized rejection needs. Reusing it keeps one suppression mechanism across retry, redirect, auth, and serde. +3. **The codec's `SERDE-12` test asserts with a plain sentinel `Error`, not `IoError`.** The plan's Task 10 test + imports `IoError` from `@dexpace/core`; that class is deliberately package-private (Phase 3b froze `io/` as + unexported), so the import does not resolve. A sentinel proves the stronger property anyway: the codec + re-wraps *nothing* coming off the stream. +4. **No workspace-root `tsconfig.json` solution file was created.** The plan's Task 8 said to add a project + reference to it; this repo has never had one — `typecheck` and `build` name each package's tsconfig + directly, and those two root scripts were extended instead. ~~`packages/codec-json/tsconfig.json` still + carries the `references: [{"path": "../core"}]` entry, which is what lets it typecheck against core's + *source* before core's `dist/` exists.~~ + + **Corrected 2026-09-02.** That last sentence has been false since H18, four items below in this + same section, removed the `references` entry — `packages/codec-json/tsconfig.json` has no + `references` key today, and H18 states why: with `dist/` guaranteed present by `build:core`, the + package typechecks against the **published** declarations rather than being redirected back to + core's source. Section H asserted both things at once, four items apart. +5. **`MISSING` is module-private, not a package export, and is a plain `Symbol`.** An earlier implementation + put it on `@dexpace/codec-json`'s barrel. The plan's Task 12 "Produces" block names only `tristate` and + `tristateObject`, and its own test declares the sentinel locally — so the promotion was drift, not a + decision. No caller has to construct one: `tristate()` also accepts plain `undefined` for Absent, and + `tristateObject` feeds the sentinel itself. `Symbol.for` was likewise dropped for a plain `Symbol`, because + unlike `TRISTATE_BRAND` nothing crosses a package boundary on this identity, so the cross-realm registry + bought nothing. Corrected in review. +6. **`SERDE-30` ships as the `tristateToString()` free function, not as a `toString()` on the sentinels.** The + design's Requirement Coverage row says "`Absent`/`Null` sentinels carry a stable `toString()`"; the + implementation exports a free function over the whole union instead. `SERDE-30` is a MAY and is satisfied + either way — this is a design-table mismatch, not a requirement gap. Deliberately **not** changed in review: + giving `absent()` and `nullValue()` a `toString` that `present()`'s result did not have would make the + discriminated union structurally inconsistent, and a free function is what the + discriminated-union-over-classes pattern asks for. + +**Trigger:** none — these are settled. Listed so a Phase 9 sweep reading the plan against the code does not +read them as drift. + +### H7 — coverage now excludes `**/dist/**` — **RECORDED** + +`bunfig.toml` gained `coveragePathIgnorePatterns = ["**/dist/**"]`. From this phase on, `@dexpace/codec-json`'s +tests reach core through its public entry point, which Bun resolves to `packages/core/dist/index.js` — so +without the exclusion the suite instruments core twice and the reported figure halves without a line of real +coverage changing. The 80% floor is a statement about `packages/*/src`, and now says so. + +**`bun test` is build-dependent from this phase on**, which is the same fact seen from the other side. Bun +resolves `@dexpace/core` through the workspace symlink and that package's `exports` map, i.e. to +`packages/core/dist/index.js` — verified: `import.meta.resolve('@dexpace/core')` returns exactly that. So on a +fresh clone `bun test` cannot resolve core for the codec's six test files until `bun run build` has run, and +against a **stale** `dist/` the codec suite reports green over yesterday's core. CI is safe (its Build step +precedes its Test step); local runs are not. The root `test` script was deliberately **not** changed to build +first — that would slow the inner loop and change a documented command's meaning — so `CLAUDE.md`'s Commands +section now marks `bun test` build-dependent instead. + +**Trigger:** none. + +### H8 — `SERDE-12`'s discrimination: one bug fixed, one residual limit — **PARTLY RESOLVED / OPEN** (promotions FIXED 2026-09-04; the foreign-stream residual is what stays open) + +*Rewritten after the Phase 6a adversarial review (G1/G2). The previous text asserted "`decodeResponse` +implements `SERDE-12` correctly" and framed the whole gap as nominal-vs-structural. That was wrong on the +behaviour, and the correction is larger than the original entry.* + +**What was actually broken.** The guard read `e instanceof IoError || e instanceof DeserializationError`. +`packages/core/src/io/errors.ts` is a **flat** tree — `EndOfStreamError`, `SourceContractViolationError`, +`ClosedResourceError` and `AllocationLimitError` all extend `DexpaceError` *directly*, not `IoError` — so the +guard whitelisted one of five I/O classes and re-stamped the other four, plus every foreign stream error, as +`DeserializationError`. Reproduced against the built artifacts on both Bun and Node: + +| body stream errored with | caller received (before) | `isSerdeError(e)` | +|---|---|---| +| `IoError` | `IoError`, identity preserved | `false` ✅ | +| `EndOfStreamError` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | +| `ClosedResourceError` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | +| `AllocationLimitError` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | +| `SourceContractViolationError` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | +| plain `Error('ECONNRESET')` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | +| `TypeError('terminated')` (undici) | `DeserializationError` | **`true`** ❌ | +| `DOMException` `'AbortError'` | `SuppressedError{.error: DeserializationError}` | `false` ❌ | + +The `SuppressedError` rows are a second-order effect worth recording: `Response.close()` cancels the body, and +`cancel()` on an *already-errored* stream replays that stream's stored error. `withReleaseFailure`'s identity +guard normally collapses that — but once the primary had been replaced by a fresh `DeserializationError` the +two objects differed, so the pair was suppressed together and a `SuppressedError` became the top-level +throwable. The abort case is the sharp one: a caller writing `if (e.name === 'AbortError')` saw +`'SuppressedError'`. + +**Fixed.** The guard is now a single `e instanceof DexpaceError` pass-through: anything already in this SDK's +typed tree is never re-typed. That subsumes all five I/O leaves, `DeserializationError`, and `HttpStatusError` +in one check, and it collapses the `SuppressedError` rows too, because the primary is once again the same +object `cancel()` replays. One test per leaf, plus an `HttpStatusError` case, in +`packages/core/src/serde/response-handlers.test.ts`. + +**The residual, which is irreducible here.** A *foreign* stream error — a `fetch`/undici body's +`TypeError('terminated')`, a hand-built `ReadableStream` errored with a bare `Error`, an aborted body's +`DOMException` — is still wrapped as `DeserializationError`. Core hands the live stream to the codec and never +reads it, so at the point of the catch a transport's raw error and a non-conforming codec leaking one are the +same shape, and `SERDE-27` requires the codec case be surfaced as a serde exception. Removing the wrap would +breach `SERDE-27`; keeping it mis-types foreign transport errors. Resolving it needs the transport to **tag** +its stream errors (a wrapping `TransformStream` at the transport seam), which is new machinery and was out of +scope for a review pass. Documented on `decodeResponse`'s TSDoc naming the affected transports, and pinned by +a test that asserts the limitation rather than the ideal — so when tagging lands, that test is the one that +changes. + +**Two open promotion questions, both deliberately not taken here.** + +1. ~~`IoError`/`isIoError` are still not on `@dexpace/core`'s public barrel~~, so the nominal + discriminator a caller would `instanceof` does not exist. The original entry's reasoning stands: the + 6a design's "Public Barrel" section does not list them, and promoting them reopens a Phase 3b + decision. + + **Half-corrected 2026-09-02.** `IoError` **is** exported — `packages/core/src/index.ts:34`, + promoted in `a0d734d` alongside `TransportFailureError` — so the nominal discriminator does exist + and has since Phase 8a. `isIoError` is still unexported (`packages/core/src/io/errors.ts:102`), and + `EndOfStreamError` was promoted in this pass, one of eight a sweep of the `@throws` tags found + naming an unreachable class. What remains of this sub-item is the guard + function alone, which matters only to a caller who wants to catch the four flat leaves as one + category without naming them. **UNSCHEDULED — trigger: a consumer that needs the category catch; + `e instanceof DexpaceError` plus a `name` check covers it today.** +2. `SuppressedError`/`SuppressedErrorLike` are likewise unexported, and one can still reach a caller (a decode + failure whose release *also* fails — the 304 case has a passing test). Both handlers' `@throws` now + document the shape: `name` is `'SuppressedError'`, `.error` is primary, `.suppressed` rides along, and + `instanceof SuppressedError` is **not** a valid test because the class is absent on the declared + `engines.node >=20.3` floor. Exporting a *type* for it is the narrowest possible fix. + +~~**Trigger:** Phase 9 or Phase 10, whichever next audits the public barrel, for both promotion questions.~~ +The foreign-stream-error residual triggers on the phase that builds the transport adapter, which is the only +layer that can tag a stream error at its source. + +**Both promotion questions FIXED 2026-09-04; the residual stays open.** + +Sub-item 1 cost more than "export the guard", and that is why it had not been taken: `isIoError` +narrows to five classes and only two of them — `IoError` and `EndOfStreamError` — were public, so +exporting the guard alone named three forgotten exports and api-extractor would have rejected it. +All four are promoted together: `SourceContractViolationError`, `ClosedResourceError` and +`AllocationLimitError` are now `@public` alongside `isIoError` +(`packages/core/src/io/errors.ts:49,65,80,103`), all five exported from +`packages/core/src/index.ts`. + +**What decided it, beyond symmetry.** These are not hypothetical classes a caller might one day +meet. This item's own *Fixed* note above changed `decodeResponse`'s guard to a single +`e instanceof DexpaceError` pass-through, which means a caller genuinely receives a +`ClosedResourceError` or an `AllocationLimitError` today, with its identity preserved — and until +now had no name to catch it by. The three fields that landed `(undocumented)` in the first +regeneration carry TSDoc now. + +Sub-item 2 is the narrowest fix it named: `SuppressedErrorLike` is exported as a **type** +(`packages/core/src/suppress.ts:8`, `export type` at the barrel). The class stays unexported — +`FallbackSuppressedError` is an implementation detail and `instanceof SuppressedError` is invalid on +the `>=20.3` floor either way, which is precisely why a structural type is the right shape. + +The precedent this followed is one day old: `3675b55` flattened `DomainModelError` and replaced it +with a `@public isDomainModelError` guard, making "flat tree plus an exported guard" the settled +taxonomy. `CLAUDE.md` was already listing `isIoError` beside `isBodyError` and `isDomainModelError` +as though it were exported; it now is. + +### H9 — every decode target is treated as non-null — **UNSCHEDULED** (2026-09-02) + +`SERDE-13` says a wire `null` decoded into a **non-null** target must fail. An implementation sees a schema +*value*, which carries no nullability it could read, so `@dexpace/codec-json`'s `decodeText` rejects a +top-level wire `null` unconditionally, before the schema runs — and the `Deserializer` TSDoc raises that to a +contract obligation on every implementor, since no implementor can do better. + +Two consequences, named so a later phase does not rediscover them as bugs: + +1. A `200` whose entire body is the literal `null` does not decode. A legitimately nullable top-level target is + outside this contract. +2. `tristate(inner)` cannot serve as a **top-level** decode target for the explicit-null case `SERDE-16` + describes. It works exactly as documented as a *field* combinator inside `tristateObject`, which is the + documented use and what every test exercises. + +The check is deliberate and should not be casually relaxed: moving it after the schema would let a permissive +schema such as `{parse: (i) => i}` return the wire `null` as a non-null `T`, which is precisely the heap +pollution `SERDE-5` and `SERDE-13` exist to prevent. Both consequences are now stated on `Deserializer`'s and +`jsonSerde`'s TSDoc. Raised in the Phase 6a shape review as F3. + +**Trigger (2026-09-02, restated without a phase):** the first **consumer** — a generated client, most +likely — that declares a nullable top-level response body. No phase can be named: the roadmap ends at +Phase 10 and all ten have shipped. Deliberately **no** opt-in flag was invented here — new public +surface needs design sign-off, not a review pass. The open question is unchanged: whether +`DecodeTarget` should carry an explicit "this target admits null" opt-in. + +### H10 — one concept, two spellings across the seam and the handler layer — **MOVED** (2026-09-04) + +The seam spells one concept positionally — `Deserializer.deserialize(data, schema, typeName?)` — while +`decodeResponse`/`decodeSuccessResponse` bundle the identical pair as `DecodeTarget`; the direction was +decided on 2026-09-02 (unify on the object form) and only the timing was ever open. + +**Moved to [`first-release.md`](../../first-release.md) on 2026-09-04**, under *Decisions owed before the first +version bump*, because its only stated trigger — the pre-publish breaking-change batch, before the first +non-`0.0.0` release — is a release decision rather than a discovery made after the work. + +The ID stays reserved and still resolves: the heading above is what +`docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-checklist.md` cites. + +### H11 — `tristate()`/`tristateObject()` are format-agnostic but ship in a format-specific package — **UNSCHEDULED** (2026-09-02) + +`packages/codec-json/src/tristate-schema.ts` imports nothing but `@dexpace/core` and operates on already-parsed +JavaScript values. Nothing in it is JSON-specific: the same combinators would work unchanged behind a CBOR or +msgpack codec, because "a missing key surfaces as `undefined` on the parsed object" is true of every one of +them. + +The tension runs both ways and neither direction is free: + +- **Against the current home:** when a second codec lands, it either duplicates this module or takes a + dependency on `@dexpace/codec-json` — and adapter-to-adapter dependencies are exactly what + `sdk-design-nodejs/02` §2's peer rule exists to prevent. Moving a public export between published packages + later is a breaking change for both. +- **Against moving it to core:** the 6a design's Scope section says "Not in scope: a schema library", and + `tristate()`/`tristateObject()` *are* schema constructors. Core defines `Schema` as a witness the caller + supplies and deliberately owns no surface for building one. Moving them in contradicts that boundary. + +No code was moved. Recorded so the second codec's phase makes this deliberately rather than discovering it +mid-implementation. Raised in the Phase 6a shape review as F11. + +**2026-09-02: left in `codec-json`, deliberately.** Neither argument above got stronger and no second +codec is on the roadmap, so moving a public export between published packages now would pay a +breaking change for a duplication that does not yet exist. **UNSCHEDULED — trigger: a second wire +codec.** No phase is named: the roadmap ends at Phase 10 and none is planned. Whoever writes that +codec meets this row before writing an import of `@dexpace/codec-json`. + +### H20 — the coverage floor measures only the Bun run — **RECORDED, no gate** (2026-08-31) + +`bunfig.toml`'s `coverageThreshold = 0.8` is enforced by `bun test` alone. `bun run test:node` contributes +nothing to it: `node --test` collects no coverage here, and the two runs do not share a report. So a line +reached only by `tests/node-conformance/` counts as uncovered, and a line covered only there cannot lift the +number. + +Surfaced by the issue-55 audit, which named three gaps in the pre-Phase-10 arrangement. The naming gap was +closed by the tree move. The static-checks gap (`.mjs` gets the gts/format baseline only, and `tsc` never +opens the subtree) is recorded in `tests/tsconfig.json`'s own comment with its compensating control — CI runs +that suite on two Node versions. This is the third, and it was the one left unrecorded. + +Not obviously a defect. The floor is a statement about `packages/*/src`, and the Node suite is deliberately +thin and additive rather than a second unit suite (`tests/node-conformance/README.md`), so its lines are +mostly re-assertions of behaviour `bun test` already covers. Merging the two reports would also mean +producing coverage from `node --test` over the BUILT `dist/`, which maps back to `src/` only through source +maps. Recorded so that "the floor covers everything" is never assumed. + +**Trigger:** a requirement whose ONLY test is a `tests/node-conformance/` case — at that point the floor is +actively misreporting, and the case needs either a `bun test` counterpart or an explicit note in its phase +checklist. + +### H15 — no `AbortSignal` on any long-running async API in this phase — **MOVED** (2026-09-04) + +Two stream-driving SPI methods — `Deserializer.deserializeFrom` and `Serializer.serializeTo` — drive a +stream they did not open and accept no `{signal}`, which the project-wide position decided 2026-09-02 says +they owe; abort is honored transitively today, so what is owed is the parameter, not the behaviour. + +**Moved to [`first-release.md`](../../first-release.md) on 2026-09-04**, under *Decisions owed before the first +version bump*, because its only stated trigger — the pre-publish breaking-change batch, H10's batch, same +file and same break — is a release decision rather than a discovery made after the work. + +The ID stays reserved and still resolves: `packages/core/src/seams/serde.ts:99,170` cite this item from +TSDoc `@remarks`, and so does +`docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-checklist.md`. + +### H17 — `SERDE-20`'s array-element half is the platform's, not this codec's — **RECORDED** + +Found by the Phase 6a reader pass, by mutation: deleting `tristateReplacer`'s `!Array.isArray(this)` conjunct +left all 100 codec-json tests green, and `tristate-replacer.ts` reports 100% line coverage either way. + +The branch was dead. `JSON.stringify`'s own `SerializeJSONArray` step appends the literal `null` for any array +element whose replacer returned `undefined` — so an Absent in an array position degrades to a wire `null` +without a line of code here. The conjunct, the `this: unknown` parameter it needed, and the two comments +narrating the mechanism were removed; one why-comment now records where the behaviour actually comes from, and +the array-position tests were kept as characterization of the platform behaviour the requirement rides on. + +Consequence for the published surface: `tristateReplacer`'s signature lost its `this` parameter, so +`etc/codec-json.api.md` was regenerated. A caller's `JSON.stringify(v, tristateReplacer)` is unaffected — +`JSON.stringify` passed `this` and the function simply no longer reads it. + +**Trigger:** none. Recorded so a later phase does not "restore" the branch on the reasonable-looking grounds +that SERDE-20 names two positions and only one has code. + +### H16 — deep-nesting encode diverges between Bun and Node — **RECORDED, no gate** + +A ~20k-deep object encodes successfully under Bun (whose `JSON.stringify` is iterative) and raises a +stack-overflow `RangeError` under Node, which `encodeToText` correctly wraps as `SerializationError`. **Both +outcomes are correct** — one succeeds, the other reports an unencodable value through the stable serde type — +so no `tests/node-conformance/` case was added: a test asserting "either encodes or throws `SerializationError`" +asserts nothing a reader can act on. Recorded only so a future reader who trips over the difference does not +file it as a bug. + +**Trigger:** none. + +--- + +### H18 — every type-aware command now builds core first — **RECORDED** + +`packages/codec-json` imports `@dexpace/core` by package name, so `tsc` resolves it through core's +`package.json` `types` field to `packages/core/dist/index.d.ts`. That file does not exist on a fresh clone, and +TypeScript's project-reference source redirect does not help: module resolution fails before the redirect is +ever consulted. CI runs `typecheck` before `build`, so the first CI run of this branch failed with 30 +`TS2307: Cannot find module '@dexpace/core'` errors across all nine codec-json files. + +The fix is a `build:core` script (`tsc -b packages/core/tsconfig.build.json`, incremental) that `typecheck`, +`lint`, `fix`, and `build` each run first. `packages/codec-json/tsconfig.json`'s `references` entry was removed +at the same time: with `dist/` guaranteed present, the package now typechecks against the **published** +declarations rather than being redirected back to core's source, so a symbol that `stripInternal` removes +cannot typecheck green here and fail at build. + +Verified by deleting every `dist/` and `.tsbuildinfo` and running both CI jobs in their exact order. + +**Trigger:** none. Recorded because the failure mode is invisible on a developer machine that has ever run +`bun run build`, and the obvious "simplification" — dropping the `build:core` prefix — reintroduces it. + +--- + +### H19 — `fast-uri` pinned by a root `overrides` entry; two dev-only advisories left open — **FIXED** (2026-09-04) + +`bun run audit` (`--audit-level=high --prod`) failed in CI on `GHSA-7p8r-x3mc-p8w7`: `fast-uri <3.1.5` +mistakes a backslash for an authority introducer, so a crafted URI resolves to an unintended host. It reaches +this tree only through dev tooling — `eslint -> @eslint/eslintrc -> ajv -> fast-uri`, and +`@microsoft/api-extractor`. + +**Why the branch surfaced it and `main` does not.** The package is in `main`'s lockfile too, so the exposure +predates this work. What changed is the *path*: Phase 6a gives `packages/codec-json` its own +`devDependencies`, and the pinned CI Bun (`.bun-version` 1.3.14) does not apply `--prod` to a **workspace +member's** dev dependencies the way it does to the root's. Local Bun 1.4.0 reports "checked 0 packages" for +the same command. So the gate's behaviour depends on the Bun version, and the version that is pinned is the +stricter one. + +Fixed by a root `overrides: {"fast-uri": "^3.1.5"}`, which resolves to 3.1.6 — a patch release inside the +range every consumer of it already accepts. Pinned rather than suppressed: there was nothing to weigh. + +**Still open, deliberately:** `bun audit --audit-level=high` (without `--prod`) reports two more high +advisories, both dev-only and both present on `main` — `js-yaml` via `@changesets/cli` +(GHSA-5p4m-2wfm-xmqj) and `tmp` via `gts -> inquirer -> external-editor` (GHSA-ph9p-34f9-6g65). Neither fails +the gate today, because both reach the tree only through **root** dev dependencies, which 1.3.14 does filter. +They are left alone as out of scope for a serde phase — but the filtering asymmetry above is what stands +between them and a red CI run, so they should be pinned the same way rather than waited on. + +~~**Trigger:** the next phase that touches root tooling, or the first CI run that reports either of them.~~ + +**FIXED 2026-09-04.** Both are pinned the same way `fast-uri` was, in the same root block: + +```json +"overrides": {"fast-uri": "^3.1.5", "js-yaml": "^4.3.1", "tmp": "^0.2.6"} +``` + +`bun audit --audit-level=high` now reports `No vulnerabilities found (checked 365 packages)`, where +before it reported three across two advisories — `js-yaml` reached the tree on **two** paths at two +major versions (`eslint > @eslint/eslintrc > js-yaml` at 4.3.0 and +`@changesets/cli > @manypkg/get-packages > read-yaml-file > js-yaml` at 3.15.0), and the 4.x pin +resolves both because 3.x's own consumers accept it. `tmp` moved `0.0.33 -> ^0.2.6`, which is a +major bump for `gts -> inquirer -> external-editor`; `bun run lint` was re-run and passes, which is +the only path in this repository that reaches it. + +The trigger had in fact fired without being noticed: `3675b55` added three `.changeset/` files, +which is root tooling, and took neither pin. That is the shape this register exists to catch, and it +is why the fix is taken here rather than deferred to the next such commit. + +--- + +## Section I — Phase 6b (Server-Sent Events) + +### I2 — Hand-rolled `SseLineReader` vs `BufferedSource.readUtf8Line()` — **RECORDED** + +`BufferedSource.readUtf8Line()` (`IO-14`) treats `\n` and `\r\n` as terminators but keeps a lone `\r` as line +content. `SSE-2` requires the opposite: a lone `\r` terminates an SSE line by itself. Both contracts are +normative for their respective subsystems, so SSE frames its own lines in `src/sse/line-reader.ts` rather than +reshaping a frozen Phase 3a surface. Recorded so Phase 10's deviation review does not read the duplication as +accidental. + +### I3 — `[Symbol.asyncDispose]` runtime-guarded and omitted from `.d.ts` — **WATCH** + +Node 20.3 (the pinned floor verified by `verify:runtime-floor` and CI `node-conformance`) predates +`Symbol.asyncDispose` (which landed in Node 20.4). TypeScript does not polyfill the well-known symbol for a +library that declares the member, so declaring it on the interface would cause `.d.ts` compilation failures for +consumers on standard `ES2023` lib without `esnext.disposable`. `SseStream` therefore installs +`[Symbol.asyncDispose]` at run time only when the symbol exists. `Response` (HTTP-38) goes further and ships +no disposal member at all — `close()` is its whole teardown surface, and `http/response.test.ts` pins the +absence of the `"undefined"` prototype key an unguarded declaration would leave behind. ~~Becomes an +unconditional `implements AsyncDisposable` when `engines.node` moves past Node 20.4.~~ + +**Widened 2026-08-30 (Phase 10).** `Page`, `FetchTransport`, and `UndiciTransport` were the three sites that +had declared the member unguarded; all four now share `SseStream`'s shape. See §10 ledger item 11. The +deferred-items row "`await using` support on `Page`, `fetchTransport()`, `undiciTransport()`" this +paragraph used to name was discharged with the rest of that table; it is reproduced in +[the purge note](./2026-09-04-register-retirement-purge.md), which `J3` and `R.E1` both cite. + +**Corrected 2026-08-30 (Phase 10): the "becomes unconditional when the floor moves" sentence is struck, not +merely deferred.** Raising `engines.node` to `>=20.4` to recover the declaration is **decided against** — +`NFR-10` (MUST) both requires the emitted target and the visible API level to agree *and* forbids making a +higher-floor capability a hard requirement of the general-purpose core, and the floor is derived from the +runtime built-ins the SDK calls (`scripts/verify-runtime-floor.mjs:22-29,33`), not chosen. The guarded install +is the permanent shape here, matching the `SuppressedError` precedent (`packages/core/src/suppress.ts`). Full +reasoning and citations in the archived deferral named above. This item stays **WATCH** only for the narrower +thing it was always about: if a *future* TypeScript or `lib` change makes an optionally-typed declaration +honest on the floor, revisit the typing — never the floor. + +### I4 — `SSE-21` hash equality is N/A in JavaScript — **RECORDED** + +`SSE-21` mentions value equality and hash. JavaScript does not have language-level hash maps keyed by object +value equality (`hashCode`); value equality is provided via `sseEventsEqual()` (`SSE-21`). + +--- + +## Section K — Phase 7a (Configuration & Platform Primitives) + +### K1 — `clientIdentityStep` is not reachable from the public barrel — **FIXED** (2026-09-04) + +`RECOV-33`'s step is implemented and tested (`config/client-identity-step.ts`), but it is **not** exported +from `packages/core/src/index.ts`. It returns a `StepDescriptor`, and the whole of `pipeline/` — +`StepDescriptor`, `Stage`, `Step`, `StepContext` — is `@internal` and absent from the barrel; +api-extractor rejects a `@public` export whose return type is a forgotten export. The phase design assumed +5c had already promoted the pipeline authoring surface, which has not run. Promoting it is a decision about +4c/5c's surface, not 7a's, so it was left alone rather than widened here. **Trigger:** when the phase that +publishes `StepDescriptor` lands, add `clientIdentityStep`/`ClientIdentitySettings` to the barrel, retag both +`@public`, and regenerate the API report. Until then `NFR-15`'s stamping step is in-package only. + +**2026-09-02: "the whole of `pipeline/` is `@internal` and absent from the barrel" is false, and has +been since Phase 5c.** `packages/core/src/index.ts:82-84` exports `Stage`, `Next`, `Step`, +`StepContext` and `StepDescriptor`, and the next line exports `PipelineBuilder` — 5c's Task 16 +promotion. So the trigger this row named **has fired**: the phase that +publishes `StepDescriptor` landed, and `clientIdentityStep` was simply not added alongside it. + +The same false statement was duplicated in source at `packages/core/src/index.ts:252-253`, 168 lines +below the exports that refute it. Both are corrected. + +What is left is a decision, not a blocker: adding `clientIdentityStep` and `ClientIdentitySettings` +to the barrel widens the public surface, which needs sign-off rather than a maintenance pass. +~~**UNSCHEDULED — trigger: the next deliberate public-surface addition**, which should settle this and +K11's folder question together, as K11 asks.~~ + +**FIXED 2026-09-04.** Both symbols are `@public` and on the barrel: +`packages/core/src/config/client-identity-step.ts:11,107` carry the tags, +`packages/core/src/index.ts` exports them in the Phase 7a block, and they land in the report at +`packages/core/etc/core.api.md:187,194` as `clientIdentityStep(settings?: ClientIdentitySettings): +StepDescriptor`. The regeneration is ten added lines and no deleted ones — purely additive, so no +pre-publish batch was needed. + +**What decided it.** The step is unreachable *and* uninstalled: `grep -rn clientIdentityStep` over +`packages/`, `tests/` and `examples/` returns its own definition, its own test and one comment, +so `standardResilience` does not install it either. Its own TSDoc +(`packages/core/src/config/client-identity-step.ts:100`) says "a caller adds it to their own +pipeline" — which no caller could do. Every other step factory was already public: `authStep`, +`retryStep`, `redirectStep`, `loggingStep`, `stripCrossOriginMarkerStep`. `RECOV-33`'s stamping step +was the sole exception, and `NFR-15` had no reachable implementation. + +**The barrel question was NOT settled with it, and deliberately.** K1 and K11 were paired on the +premise that one sign-off answers both; the pairing dissolves once the export lands, because a +`@public` symbol named against its own module path makes its folder invisible. See K11, closed the +same day, and `X1`/`X2` for the two holes this pass found and did not take. + +### K3 — `CFG-12` is documented, not enforced — **WATCH** + +"Builders SHOULD be usable single-threaded only" is a JVM statement about publication safety. A +`ConfigurationBuilder` has no cross-thread reachability in this runtime to guard, so the requirement is +carried as a doc comment on the class and nothing more. There is no test, because there is no observable +behavior to assert. **Trigger:** if a worker-thread story ever puts a builder behind `postMessage`. + +### K6 — Six defects in the phase plan's own implementation sketches — **WATCH** + +The plan doc at `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration.md` still contains the sketches +below. They were corrected in the shipped code; the plan was not rewritten, so a future reader following it +verbatim would reintroduce them. The plan now opens with a banner saying so and pointing here and at the +checklist as the as-built record, which is the mitigation — the six sketches are deliberately left in place +rather than rewritten, because a completed phase's plan is a historical artifact, not a maintained document. +**Trigger:** if this plan is ever used as an execution input again. + +1. **Task 5's hash-consistency test asserts a false claim.** `deepEqual([1, {x: 2}, [3, 4]], [1, {x: 2}, [3, + 4]])` is asserted `true`, but `CFG-33` makes non-arrays fall back to ordinary equality, so two distinct + object literals are not equal and the sketched test fails against a correct implementation. +2. **Task 7's `CFG-22` test tests nothing.** It builds an object literal whose `toString` returns a + hard-coded masked string and asserts that string is masked. `ProxyOptions` in the same sketch has no + `toString` at all, despite the design doc's interface declaring one. +3. **Task 6's `getInt` uses `Number.parseInt`**, which resolves `"12abc"` to `12` — `CFG-5` says an + unparseable value returns the default. +4. **Task 2's parser uses `Date.UTC(year, ...)`**, which maps a four-digit year below 100 onto 1900-1999, so + `0026` silently parses as 1926; and it range-checks fields individually without rejecting a rolled-over + calendar date such as `31 Feb`. +5. **Task 7 omits `CFG-26` entirely** — no backslash escape, no property/environment precedence, and a + trim-then-filter order that is the reverse of the one the requirement specifies. +6. **Task 8 Step 6 rewrites `build` to `tsc -b`**, which would replace the working + `tsc -p tsconfig.build.json`. Only the `prebuild` line was added. + +### K7 — `eslint.config.js`'s Node-globals block was widened — **WATCH** + +`packages/core/scripts/gen-version.mjs` is the first build script living under a package rather than at the +repo root, and the config's globals block listed `scripts/*.mjs` only, so its `console` call tripped +`no-undef`. `packages/*/scripts/*.mjs` was added to the same list. **Trigger:** if package-level scripts ever +need a different tier than the root ones. + +### K8 — `src/generated/version.ts` is committed and can be stale — **WATCH** + +The generated version constant is committed deliberately, so an unbuilt `bun test` reports a real version +rather than a placeholder. That makes it possible for the file to disagree with `package.json` between a +version bump and the next build. `prebuild` regenerates it on both the package and root `build` scripts, and +release goes through `prepublishOnly`'s build, so a published artifact cannot carry a stale value — but a +working tree can, and nothing fails if the regenerated file is left uncommitted. **Trigger:** if CI ever +needs to assert the committed file matches `package.json`, add a `git diff --exit-code` after `prebuild`. + +### K11 — `client-identity-step.ts`'s folder placement is provisional — **CLOSED** (2026-09-04) + +`RECOV-33`'s step lives at `packages/core/src/config/client-identity-step.ts` because the phase design doc's +File Layout names that path. Two arguments say it is not its long-term home, both recorded here rather than +acted on, because relocating a file into a neighbouring phase's folder on 7a's authority is the same +surface-widening the phase declined to do for K1: + +1. **It is the sole outbound `config/ → pipeline/` edge.** Three of its four imports leave the folder — + `../http/headers.js`, `../http/request.js`, `../pipeline/step.js` — and only `./build-info.js` stays. + Every other module under `config/` imports nothing beyond `../invariant.js` and `../generated/version.js`. + The file is grouped by which phase built it, not by feature (`docs/knowledge/harvested/module-organization.md:12`). + There is no cycle today; the risk is that 5a's `RetryConfig.clock` and 7b's logging step both create the + return edge, and nothing in CI would catch the loop (see K12). +2. **Its `RECOV-32` sibling would land elsewhere.** The idempotency-key step — the adjacent requirement, the + same kind of object — is planned for `packages/core/src/recovery/idempotency-key.ts` + (`docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md:157,2529`). Two adjacent `RECOV-3x` steps in two + unrelated folders. + +Also noted here, since it is the same class of question: the barrel comment at `packages/core/src/index.ts` +explains the absence of a `config/index.ts`. This repo carries both patterns — `http/`, `body/`, `io/`, and +`seams/` have internal barrels; `pipeline/`, `context/`, and `config/` do not — and so does the knowledge +corpus, where `docs/knowledge/harvested/module-organization.md:18` bans internal barrels outright while +`docs/knowledge/harvested/api-design.md:8` endorses one per feature folder, with nothing in the corpus's +`--section conflicts` reconciling them. 7a followed its design doc, which rules a `config/index.ts` out by +name. ~~**Trigger:** the same phase as K1 — whichever one promotes the pipeline authoring surface.~~ Settle the +file's folder and the barrel question together there, and record the corpus tension at that point. + +**2026-09-02: that trigger fired in Phase 5c and nothing happened.** The pipeline authoring surface +was promoted then (`packages/core/src/index.ts:82-86`) — see K1, whose own statement of the blocker +was stale for the same reason. Neither the folder nor the barrel question was settled, and no phase +remains to hand them to. + +Nothing has decayed in the meantime: the `config/ → pipeline/` edge is still the only outbound one, +K12's import-cycle gate still does not exist, and `RECOV-32`'s sibling is still planned for +`recovery/`. ~~**UNSCHEDULED — trigger: the same one K1 now carries**, the next deliberate +public-surface addition, which is when `clientIdentityStep`'s home and its export are one decision +rather than two.~~ + +**CLOSED 2026-09-04 — won't-fix, and the sentence above is wrong twice.** Re-measured when K1's +export landed: + +1. **"the `config/ → pipeline/` edge is still the only outbound one" is false**, and had been since + `d64a107`, the commit before this one. Three further outbound edges leave the folder: + `packages/core/src/config/clock.ts:3` → `../cancellation.js`, + `packages/core/src/config/configuration.ts:4` and + `packages/core/src/config/proxy.ts:9` → `../observability/logger.js`. Only the `→ pipeline/` edge + is unique *in kind*; "the sole outbound edge", which is how argument 1 was stated, is not why the + file stands out. +2. **"`RECOV-32`'s sibling … the same kind of object" is false.** It shipped, and it is a different + kind of object. `idempotencyKeyStep` returns `RequestStep` + (`packages/core/src/recovery/request-chain.ts:11`, `(request: Request) => Promise` — a + bare function in a recovery fold); `clientIdentityStep` returns `StepDescriptor`, a pipeline-stage + descriptor carrying a `stage` and a `type` symbol + (`packages/core/src/config/client-identity-step.ts:111`). Two adjacent `RECOV-3x` *requirements*, + not two instances of one thing that got separated. + +With both arguments gone, the move has no case left and a cost: relocating to `recovery/` trades the +one outbound `→ pipeline/` edge for a new `→ config/` one, because the step reads +`./build-info.js`. And it buys nothing a consumer can observe — since 2026-09-04 the symbol is +`@public` and named in the barrel against its own module path, so its folder is invisible outside the +package. The corpus tension the row also carried +(`docs/knowledge/harvested/module-organization.md:18` bans internal barrels, +`docs/knowledge/harvested/api-design.md:8` endorses one per feature folder, nothing in +`--section conflicts` reconciles them) is unreconciled and stays recorded in the barrel comment; it is +not a reason to move a file. `packages/core/src/config/build-info.ts:37` still cites this item for the +outbound-edge concern, which remains accurate — that is K12's subject, not this row's. + +### K12 — No import-cycle gate exists in CI — **FIXED** (2026-09-04) + +`docs/knowledge/harvested/module-organization.md:20` treats any import cycle as a bug rather than a style nit, and +`:22` requires it be gated in CI with `madge --circular src` or `eslint-plugin-import/no-cycle` as a required +check. Neither string appears anywhere in `package.json`, `eslint.config.js`, or +`.github/workflows/ci.yml` — verified 2026-08-27. Every other rule in that topic is either enforced or +deliberately deviated from; this one is simply absent, so the repo's twelve-plus source folders rely on +review alone. Deliberately **not** added by Phase 7a: a new blocking CI step is repo tooling, outside a +feature phase's scope, and it belongs with whoever owns `.github/workflows/ci.yml`'s gate list. +~~**Trigger:** the next phase that touches the CI gate list, or the first observed cycle — K11's +`config/ → pipeline/` edge being the nearest candidate.~~ + +**FIXED 2026-09-04.** `scripts/verify-import-cycles.mjs`, wired as `bun run verify:import-cycles` +and as a blocking CI step ("Import-cycle check") between the test-partition and reproducible-build +steps. It walks every relative specifier under each package's `src/`, resolves `.js` back to `.ts` +per NodeNext, and reports the first cycle it closes as the full list of files on it. + +**Hand-written rather than `madge` or `eslint-plugin-import/no-cycle`, which is a deviation from +`docs/knowledge/harvested/module-organization.md:22`'s named tools and is deliberate.** Every other +`verify:*` gate here is dependency-free `.mjs`, `verify:seam-1` asserts zero runtime dependencies per +package, and the graph to walk is small enough that the whole traversal is a depth-first search with +a colour map. The gate's own suite is `scripts/verify-import-cycles.test.mjs`, seven cases run by the +blocking `test:scripts` step — including the two a naive implementation gets wrong, a self-import and +a diamond, the latter being re-convergence rather than recursion. + +**Type-only edges count**, deliberately: an erased `import type` cannot deadlock module +initialization, but a type cycle is still the design smell the requirement is about. + +**First run: no cycles across 171 source files.** So K11's feared `config/ → pipeline/` loop does not +exist, and neither does any other. The gate is installed against the next one, not this one. + +### K13 — A `CFG-7`-valid duration can exceed what any timer can honor — **UNSCHEDULED** (2026-09-02; the trigger named a phase that could not fire it) + +`Clock.sleep` now rejects any `ms` above `2 ** 31 - 1` with an `InvariantViolation` naming the ceiling +(`packages/core/src/config/clock.ts`), because `setTimeout` silently clamps a larger delay to `1` — `sleep(2 ** +31)` returned in 7ms instead of waiting 24.8 days, so an overflowed retry backoff became *no* backoff and a hot +loop against the upstream. That closes the silent half. + +The residue is on the other side of the seam. `Configuration.getDuration` still returns +`8.64e26` for `9999999999999999999d` and `8.64e27` for `P100000000000000000000D`, and it is **right** to: +`CFG-7` defines the grammar and those parse correctly under it, so returning the caller's `fallback` there +would invent a rejection rule the requirement does not state. Deliberately **not** bounded by 7a for that +reason. The consequence today is contained — the value is `CFG-7`-valid, and the first thing that tries to +*wait* it raises a named error at the point of use rather than looping — but it means a config typo is caught +one layer later than it could be. **Trigger:** the first phase that wires a configured duration into a timer, +which is Phase 5a's retry engine. Bound it there, at the configuration boundary, following `RECOV-34`'s +precedent — that requirement already bounds every retry duration at "representable in nanoseconds (~292-year +ceiling)" and rejects at construction rather than at use. + +**2026-09-02: "which is Phase 5a's retry engine" is wrong, and nothing else can fire this either.** +`Configuration.getDuration` has **zero non-test consumers** — +`packages/core/src/config/duration.ts:69` says so in a comment of its own — so Phase 5a's retry +engine never wired a configured duration into a timer, and neither has anything since. +`retrySettings` takes numbers from a caller, not from `Configuration`. This row has been waiting on a +hand-off no phase was ever going to make. + +The residue is unchanged and still correct to leave: `getDuration` returns `8.64e26` for +`9999999999999999999d` because `CFG-7`'s grammar accepts it, and inventing a rejection rule the +requirement does not state would be the deviation. **UNSCHEDULED — trigger: the first real consumer +of `getDuration`**, which is the layer that knows what it will do with the value. + +**What the advice above applied to was found anyway, one layer in** — and then answered from the +other direction. The defect: a configured retry delay was validated only from below, so an +unwaitable one failed inside the retry loop rather than at the call that supplied it. The fix is not +the `RECOV-34`-style bound this row recommends — `Clock.sleep` was made to chain timers and honor any +finite duration, so no unwaitable duration is left to reject. The general lesson stated above (bound +at the configuration boundary, not at the point of use) is sound and simply had no defect left to +apply to. + +### K16 — `deepEqual` / `deepHash` require acyclic, bounded-depth input — **WATCH** + +Both helpers in `packages/core/src/config/equality.ts` recurse with no cycle guard and no depth cap, so a +self-referential array (`const a = []; a.push(a)`), a mutually recursive pair, or ~100k levels of nesting +raises `RangeError: Maximum call stack size exceeded` rather than terminating. `CFG-33` says nothing either +way; the precondition is simply undocumented. Pinned by two tests in `equality.test.ts` so the constraint is +discoverable at review time rather than in production. + +Deliberately **not** fixed: the module is not exported from the package barrel (`packages/core/src/index.ts` +records that decision) and has **zero callers** as of 2026-08-27, so a `WeakSet` cycle guard and a depth cap +would be cost paid by nothing. **Trigger:** the first consumer. It either supplies the acyclic, +bounded-depth guarantee itself — which every `CFG-33` use the spec describes does, since it compares +byte arrays and header value lists — or adds the guard and the cap at that point, and this entry closes. + +### K18 — `isHeaderSafe` duplicates `http/ascii-validation.ts`'s outbound byte predicate — **WATCH** (split out of K11, 2026-08-27) + +`packages/core/src/config/build-info.ts` carries its own four-line printable-ASCII-plus-HTAB predicate rather +than importing `hasForbiddenOutboundByte` from `packages/core/src/http/ascii-validation.ts`. The two encode +the same character class for the same reason — an ambient value that RECOV-33 puts straight into an outbound +header. + +The duplication is **deliberate** and the trade is stated at the call site: `config/`'s outbound edges are +already a live concern (K11 tracks the `config/ → pipeline/` one), and adding a second one to reuse four +lines is the wrong side of that trade while K12's import-cycle gate does not exist. + +Recorded separately from K11 because K11's own resolution — settle `client-identity-step.ts`'s folder and the +`config/index.ts` barrel question — would not touch this predicate, so a reader following the old pointer +found nothing that owned it. **Trigger:** whichever phase consolidates the ASCII predicates, or the first +third caller of the same character class; `build-info.ts`'s `isHeaderSafe` is one of the call sites it folds +in. Until then the risk is one-way drift — a fix to the `http/` predicate that this copy does not receive. +Cheap to bound: both are exercised by tests that assert the same class (`build-info.test.ts`'s header-safety +cases and `http/`'s own), so a divergence surfaces as a test failure rather than as silent behavior. + +### K19 — No `fast-check` property test logs its seed — **CLOSED** (2026-09-04; the premise is false) + +`docs/knowledge/harvested/testing.md:44` requires the seed of a failing seeded `fast-check` property test to reach CI +output, "or the shrunk counterexample that found the bug is lost". ~~No `fc.assert` call anywhere under +`packages/core/src` passes a `seed`, `numRuns`, or a `reporter` — verified 2026-08-27 across all 20 +`fc.assert` sites, of which Phase 7a contributes 12.~~ A property failure in CI today reports the shrunk +counterexample for that run only; re-running does not reproduce it. + +**Both halves of that sentence were wrong (re-measured 2026-09-02).** There are **64** `fc.assert` +sites under `packages/core/src`, not 20 — the count was taken before four phases landed and never +retaken — and **five of them are already seeded** with `{seed: 0x3b}`, added in `e3ba885`: two in +`body/multipart-body.test.ts` and one each in `body/materialize.test.ts`, +`body/request-body-logging.test.ts` and `body/response-body-logging.test.ts`. So the "no call +anywhere" claim was already false when it was written. + +That makes the finding *stronger*, not weaker: the repo is in exactly the half-migrated state this +row argues against — five seeded, fifty-nine not — which is the shape +`docs/knowledge/harvested/styleguide-overview.md:32-33` forbids and which the row's own reasoning +("a seeding convention that covers half the suite is worse than none") names as the thing to avoid. + +**CLOSED 2026-09-04: the requirement is already met, and was met before the row was written.** +`docs/knowledge/harvested/testing.md:44` asks that the seed of a failing seeded property test *reach +CI output*. `fast-check` 3.23.2 puts it there unconditionally, for every `fc.assert` site, seeded or +not — `node_modules/fast-check/lib/check/runner/utils/RunDetailsFormatter.js:62` formats every +property failure as `` `Property failed after ${out.numRuns} tests\n{ seed: ${out.seed}, path: +"${out.counterexamplePath}", endOnFailure: true }` ``, and the two neighbouring formatters at `:45` +and `:77` do the same for the too-many-preconditions and interrupted cases. A failing property in CI +therefore already prints the seed and the shrink path needed to reproduce it locally. + +That dissolves the row as stated. The counts are unchanged and correct — re-measured 2026-09-04, 64 +`fc.assert` sites and 5 seeded — but "five seeded, fifty-nine not" is not a half-migrated *seeding +convention*, because there was never a convention to migrate to: the reporting the rule asks for is +the library's default. The five `{seed: 0x3b}` sites are determinism pins on body-layer property +tests, which is a different thing from a reproducibility convention and is left alone. + +No repo-wide decision is owed and `bunfig.toml` needs no `preload`. The original reasoning below is +kept because it is what a reader will otherwise re-derive: + +Deliberately **not** fixed by Phase 7a, and deliberately not fixed in this phase's tests alone: a seeding +convention that covers half the suite is worse than none, because the half without it looks deliberate. This +is one repo-wide decision — a shared `fc.configureGlobal({seed, verbose})` in a test preload, or a documented +`FC_SEED` environment convention — and it belongs with whoever owns `bunfig.toml`'s test configuration. +**UNSCHEDULED (2026-09-02) — trigger: the first property failure in CI that cannot be reproduced +locally**, or any deliberate change to `bunfig.toml`'s test configuration. No phase is named: the +roadmap ends at Phase 10 and every phase has shipped. + +### K20 — `Retry-After: ` changed disposition from no-hint to retry-immediately — **RECORDED** (2026-08-27) + +Surfaced by rebasing 7a onto the branch that already carried 5a. Before 7a, `retry/pacing.ts` used a private +RFC 1123 parser that **rejected** a four-digit year in `[0,99]`, so `Retry-After: Thu, 01 Jan 0026 00:00:00 +GMT` produced no hint and the caller fell back to backoff. 7a deletes that private copy in favour of the +shared `config/http-date.ts`, which reads the year literally (never `Date.UTC`, whose legacy mapping would +turn `0026` into 1926) and therefore accepts it as a well-formed instant in the past. + +Kept 7a's reading. `RETRY-16`'s no-hint rule is about values that are *malformed, negative, or out of +range*; a literal year 26 CE is none of those, and `RETRY-17` is explicit that "a valid HTTP-date ... already +in the past MUST yield a zero delay (retry immediately), distinct from an unparseable value which yields no +hint". RFC 1123's `date1` year is `4DIGIT` with no lower bound, so rejecting `0026` was the stricter-than-spec +half of the two implementations. `packages/core/src/retry/pacing.test.ts` now asserts `0` for that input and +records why. + +The behavioral cost is real but narrow: a server sending an absurdly old `Retry-After` gets an immediate +retry rather than a backed-off one. No such header exists in practice, and `RETRY-17` prescribes exactly this +for every other past instant, so a plausibility floor would be a new deviation rather than a fix. + +**Trigger:** a real server observed to send a pre-1970 `Retry-After`, or a spec erratum giving `RETRY-16` a +lower bound on the year. + +--- + +## Section L — Phase 7b (Instrumentation & Observability) + +Recorded at implementation time. Verified against `docs/product-spec/15-instrumentation-and-observability.md` (`OBS-1`..`OBS-18`, `OBS-20`..`OBS-27`, `OBS-30`..`OBS-40` executed; `OBS-19`, `OBS-28`, `OBS-29` deferred per design). + +### L1 — Deferred HTTP-Tracer Vocabulary and Transport Policies — **SPLIT 2026-09-02** (OBS-19 **FIXED**; OBS-28 **CLOSED**; OBS-29 **UNSCHEDULED**) + +- `OBS-19` (dropped-header verbosity policy): Deferred to Phase 8a alongside the concrete `fetch` transport that first detects unencodable caller-set headers. +- `OBS-28` (richer HTTP-tracer vocabulary with per-attempt and transport milestones) and `OBS-29` (HTTP-tracer lifecycle ordering contract): Deferred to Phase 8a (interface + transport milestones) and Phase 9 (ordering verification). Phase 7b ships operation/attempt-level `startSpan`/`end` (`OBS-21`..`OBS-25`). + +**Trigger:** Phase 8a transport adapters and Phase 9 conformance sweep. + +**2026-09-02: both named phases shipped without taking any of the three.** A grep for `OBS-29` +across `docs/work/mvp/phase8/` and `docs/work/mvp/phase9/` returns nothing. One row deferring three +requirements to two closed phases is the exact unowned-MUST shape this register exists to prevent, so +the row is split and each requirement is dispositioned on its own. + +**`OBS-19` — FIXED (2026-09-02).** Phase 8a's `createDropLogger` had three modes and one level; all +three now emit at the level each requirement names — `'all'` and `'first-per-name'` warn, `'quiet'` +stays silent (`packages/transport-shared/src/drop-log.ts`). The same pass settled the `TRANSPORT-13` +checklist contradiction that sat beside it. + +**`OBS-28` — CLOSED (2026-09-02), satisfied-by-level.** A SHOULD whose own text asks only that every +method default to a no-op so adding an event is non-breaking; the port ships that mechanism. + +**`OBS-29` — UNSCHEDULED, and it is a MUST.** See V2, which carries the full finding. + +### L4 — Attempt-Level vs. Operation-Level Span and Metric Scope (PIPE-2) — **RECORDED** (2026-08-28) + +`PIPE-2` fixes the `LOGGING` pillar step inside `RETRY` and `REDIRECT` pipelines. Consequently, `startSpan('http.client.request')` and metric increments (`http.client.request.count`, `http.client.request.duration`) execute per HTTP transmission attempt/hop. The higher-level logical operation span and HTTP-tracer lifecycle are owned by Phase 8a / `OBS-29`. + + +## Section M — Phase 8b (Async-Runtime Bridge, `@dexpace/rx`) + +Recorded at implementation time. Verified against `docs/product-spec/18-asynchronous-runtime-adapter-contract.md` +(`ASYNC-1`..`ASYNC-22`) and `SSE-41`. + +### M1 — The `AsyncIterable`→`Observable` Bridge Is Hand-Written, Not `rxjs`'s `from()` — **RECORDED** (2026-08-28) + +8b's design (§1) and plan (Global Constraints) both instructed: do not hand-write the pull loop, use RxJS's own +`from(asyncIterable)`, and prove it satisfies `ASYNC-6`/`ASYNC-13`/`ASYNC-21` rather than assuming it. The proof +failed on one clause. `rxjs@7.8.2`'s async-iterable path tests `subscriber.closed` only *after* a pull resolves, +so unsubscribing while a pull is suspended never reaches the source. For pagination that is invisible; for SSE it +is the common case — an idle event stream is permanently suspended, so `unsubscribe()` would leave the response +body unreleased and the connection open until the server next sent something. + +Resolved through the fallback both documents pre-authorized, scoped to that clause alone: +`packages/rx/src/from-async-iterable.ts` (`@internal`) adds a teardown that releases the caller-supplied source +and drives `iterator.return()`, release first so a suspended pull settles before the queued generator return. +No scheduler, no error re-wrapping, no buffering. + +Full rationale in the plan's Self-Review. This is a deviation from the *plan's* implementation instruction, not +from the product spec — `ASYNC-6`/`ASYNC-21`/`SSE-41` are satisfied as written, and +`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`'s Phase 8b rows are unaffected. + +**Trigger:** an RxJS release that closes the gap. `from-async-iterable.conformance.test.ts`'s last case asserts +the defect (`returns === 0` after an idle unsubscribe) and fails when it is fixed; at that point delete the +module and go back to `from()`. + +## Section N — Phase 9 (Cross-Cutting Invariants & Conformance) + +Findings from the first systematic `XCUT-1`–`XCUT-24` / `NFR-1`–`NFR-17` pass. Every row here was found by +driving the **composed** pipeline (`standardResilience()` over a real `fetchTransport()` against a local +`node:http` fixture), which is the shape no earlier phase's own unit tests exercise. + +### N3 — Phase 9's plan asks for a `docs/knowledge/` grep that cannot return empty — **UNSCHEDULED** (2026-09-02) + +The plan's Task 11 Step 4 runs `grep -rn "unresolved 2026-07-25" docs/knowledge/` and expects no output. It +cannot pass as written. The design scoped §4 to the **three** markers in `tooling-and-quality-gates.md`; the +grep is repo-wide and two further markers live elsewhere: + +| Marker | File | State in the code | +|---|---|---| +| `#private` fields as the default for model state | `http-domain-model.md:131` | Settled in practice — `#private` throughout `src/http/`, documented as the pattern in CLAUDE.md | +| `enum` for the pipeline `Stage` ordering | `pipeline.md:179` | Settled in practice — `erasableSyntaxOnly` bans `enum`; the port ships `STAGE_ORDER`/`PILLAR_STAGES` frozen constant objects | + +Both are resolved *by the implementation* but never marked resolved *in the corpus*, which is exactly the +silent-gap shape this register exists to prevent. Deliberately not marked here: writing a resolution into +`docs/knowledge/` is a decision record, the checkpoint's rule only obliges markers its own §5 touched, and +neither is an `XCUT`/`NFR` question — Phase 9's scope. Phase 10 owns deviation reconciliation and is the +right place. The three markers Phase 9 *was* scoped to were already backported at planning time (`c6603aa`) +and were confirmed still correct, not re-made. + +**2026-09-02: Phase 10 did not mark them, and `docs/knowledge/` is now a frozen tree.** Recording a +resolution there is a hand edit to `notes/`, not a maintenance action — and by the corpus's own rule +it belongs in `docs/knowledge/notes/`, never in `harvested/`, since a hand edit inside a harvested +entry changes no `` sha and the next harvest regenerates or duplicates it. Both markers remain +resolved *by the implementation* and unmarked *in the corpus*. ~~**UNSCHEDULED — trigger: the next +deliberate edit to `docs/knowledge/notes/`, or the next re-harvest**, which is the moment the note +can be written without a lone edit to a frozen tree.~~ + +**FIXED 2026-09-04, both markers.** The `#private` marker closed earlier the same day with +`docs/knowledge/notes/data-modeling.md`, which names `http-domain-model/d26b9192`. The `enum`/`Stage` +marker closes here with `docs/knowledge/notes/pipeline.md`, naming `pipeline/e66ace13`; both +harvested entries now print `[overridden by notes/…]` in every query result, and +`bun run knowledge:drift` reports 12 note citations resolving and 0 not. + +**The grep in Phase 9's plan still cannot return empty, and resolving the markers made it worse — +3 matches where there were 2.** That is structural, not a regression: a note has to quote the marker +string to name what it resolves. The check that means something is +`bun run knowledge --topic pipeline --section conflicts`, where the resolution is visible as an +override. The plan step is the defect; it is a dated record under `docs/work/` and is not +retro-edited, so this note is the correction. + +### N4 — `rxjs` version restated in three places against `NFR-14` — **UNSCHEDULED** (2026-09-02) + +`NFR-14` asks that dependency and tool versions live in a single source of truth so a bump is one edit. The +root `workspaces.catalog` holds `@microsoft/api-extractor`, `expect-type`, `fast-check` and `typescript`. +`rxjs@^7.8.0` is stated three times instead: root `devDependencies`, `packages/rx` `devDependencies`, and +`packages/rx` `peerDependencies`. A bump is three edits, two of which are easy to miss. + +The peer range is legitimately per-package — it is part of what `@dexpace/rx` publishes, not a build +coordinate. The two `devDependencies` restatements are the defect; a `rxjs` catalog entry collapses them. + +`debug >=4.0.0` and `pino >=8.0.0` are **not** defects for the same reason: each appears once, as a published +peer range. `undici ^6.21.1` likewise appears once, as `transport-undici`'s own runtime dependency. + +Not fixed in Phase 9: it edits another package's manifest and changes the lockfile, which is 8b's surface. +Owner: Phase 10, alongside its own dependency pass. + +**2026-09-02: Phase 10 shipped without the dependency pass, so the owner is spent.** The finding is +unchanged and the fix is still one catalog entry collapsing the two `devDependencies` restatements. +Not taken in this pass because it changes the lockfile, and a lockfile change wants its own install +and its own verification rather than riding along with a register audit. **UNSCHEDULED — trigger: the +next `rxjs` bump**, which is the moment the three-places cost is actually paid. + + +## Section O — Knowledge-corpus split (2026-08-31) + +### O1 — The `knowledge-harvest` skill's default `--corpus` still points at the tree no query reads — **WATCH** + +`docs/knowledge/` is now two trees, `harvested/` and `notes/`, and `bun run knowledge` reads only those two. +The producing skill lives outside this repository (`~/.claude/skills/knowledge-harvest/`, user-global, shared +across projects) and its documented default is `/docs/knowledge/`, with its canonical stored-run command +naming `--corpus docs/knowledge`. A run that forgets `--corpus docs/knowledge/harvested` therefore writes a +third copy of the corpus at the root, which no query reads. + +Separately, `merge.py` emits a `## Superseded` heading unconditionally and offers `supersede` as one of its +four conflict resolutions. A harvest that writes one into `harvested/` produces an entry the structure gate +rejects, correctly — the resolution has to be hand-moved to `notes/`. + +Neither can be fixed from inside this repository. The compensating controls are all here and all blocking: +`verify:knowledge-structure` rejects a `.md` stranded at the root of `docs/knowledge/` and rejects a +`Superseded` entry under `harvested/`, and the invocation is stated in `CLAUDE.md`, `docs/knowledge/README.md` +and the `knowledge-lookup` skill. + +**Trigger:** anyone changing the global skill — make `harvested/` its default when the two-tree layout is +present, and stop emitting `Superseded` into a harvest target that has a sibling `notes/`. + +### O2 — A note's key citation is checked by a report, not by a gate — **ACT** (2026-09-02; mechanism chosen) + +A note names the harvested rule it overrides by that rule's stable key (`/<8 hex>`, digested from the +entry's text). `bun run knowledge:drift` reports a citation that no entry carries any more, `--key` resolves +one on demand, and a harvested entry that a note overrides prints `[overridden by notes/…]`. None of that is +blocking: a re-harvest that rewords a rule silently breaks every note citing it, and only a hand-run report +says so. + +The issue that introduced the split specified three structural rules and this is not among them, so it was not +added unilaterally. Two candidate mechanisms were reviewed: + +1. **Fail `verify:knowledge-structure` on an unresolvable key.** Ten lines, no new file, and it makes a + re-harvest that orphans a note a red build rather than a silent rot. It also means a legitimate re-harvest + cannot land until the notes it invalidates are updated in the same commit — which is arguably the point. +2. **Commit a key manifest** (`harvested/KEYS.md`) regenerated only by a harvest, and fail when the live key + set diverges without a matching `SOURCES.md` sha change. This catches strictly more: it detects *any* hand + edit to harvested text, including one that keeps the role, the section and the source and so passes all + four current rules. Cost is a new generated artifact and a coupling to a skill this repo does not own. + +**2026-09-02: mechanism 1 is chosen; implementing it is not part of this pass.** Extend +`scripts/verify-knowledge-structure.mjs` to fail on a note carrying a backticked `/<8 hex>` +key that no harvested entry carries. Ten lines, no new file, no new artifact, and it uses the parser +that gate already loads. Mechanism 2's key manifest was rejected on cost: it commits a generated +artifact and couples this repository to a skill it does not own, to catch a strictly larger class +(any hand edit to harvested text) that `verify:knowledge-structure`'s four existing rules and the +`harvested/`-is-never-hand-edited convention already discourage. + +The consequence mechanism 1 carries is the point rather than an objection: a legitimate re-harvest +cannot land until the notes it invalidates are updated in the same commit. + +**Trigger:** the next re-harvest, or the next deliberate change to `scripts/verify-knowledge-structure.mjs`. +Until then `bun run knowledge:drift` is the check, and it is named in the phase-start section of the +`knowledge-lookup` skill. + +## Section P — Phase 5a (Retry) + +> **Merged 2026-08-31 from the repository-root `open-items.md`.** Phase 5a's code review (passes 1–3, +> 2026-08-26) wrote its findings to a second register at the repository root, created in `cba4721` and never +> folded in — which is exactly the gap this file's own preamble named ("Two phases are shipped but were never +> registered here: 4c and 5a"). The root file is deleted; its nine findings are below, numbered `P1`–`P9`, +> text unchanged apart from the heading form and the status word. Its own status legend was +> 🔴 defect, owner named — 🟡 accepted limitation — 🟢 correct, documented to stop a future "fix" — +> 📄 documentation drift; each is restated in this register's vocabulary in the heading, with the original +> `**Owner:**` line kept intact. +> +> **Every item was re-verified against as-built source on 2026-08-31** before being merged, per this file's +> maintenance rule. Eight still held; the ninth had already been closed by Phase 7b. +> +> Findings that *were* fixed in 5a are not listed — they are in the code and its tests. + +### P3 — `RetrySettings.retryableStatuses` is immutable by type, not at runtime — **WATCH** + +**Where:** `packages/core/src/retry/settings.ts` + +`retrySettings()` returns `Object.freeze({...})`, but freeze is shallow and does not seal a `Set`'s +internal slots: anyone holding the settings object can still call `.add()` on the status set and +change policy for every later call. + +`RECOV-34`'s actual requirement — a *defensive copy* so a caller mutating **their own** source +collection cannot alter policy — is satisfied and tested. What is not achievable is `RETRY-42`'s +"immutable after construction" as a runtime guarantee. + +This is a deliberate house position, not an oversight: `config/retryable.ts` records it — *"`Object.freeze` +does not seal a `Set`'s internal slots, so a frozen `Set` would be a misleading no-op — typed +`ReadonlySet` instead, same treatment as Phase 1's `IDEMPOTENT_METHODS`."* A genuine runtime guarantee +would need a wrapper object with no mutators, which changes the shape every consumer reads. + +**Owner:** none. Recorded so the gap between the type-level and runtime guarantee is not rediscovered +as a bug. + +**Re-verified 2026-08-31:** unchanged. `packages/core/src/retry/settings.ts:104-106` still returns +`Object.freeze({… retryableStatuses: new Set(merged.retryableStatuses)})`, and freeze does not seal a `Set`'s +internal slots. + + +### P4 — `RETRY-18`'s 365-day pacing ceiling is spec-mandated and operationally hazardous — **UNSCHEDULED** (2026-09-02; and the described behaviour is not what happens) + +**Where:** `packages/core/src/retry/pacing.ts` + +A server that sends `X-RateLimit-Reset` in **milliseconds** instead of epoch seconds — a common +server-side mistake — produces a delta of roughly 56,000 years. `RETRY-18`/`RECOV-26` require +clamping to a 365-day ceiling, so the parser returns exactly that: a retry parked for a year, which +is indistinguishable from a hang. + +Nothing shortens it by default. `totalTimeoutMs` would, but `RETRY-28` makes it explicitly opt-in and +it is `undefined` by default. The caller's own `AbortSignal` is the only other exit. + +Implementing a tighter ceiling would be a deviation from a MUST, so the port complies. Recorded +because "spec-compliant" and "safe by default" diverge here, and the mitigation (set +`totalTimeoutMs`) is a caller decision that needs documenting when the retry surface is finally +published in Phase 5c. + +**Owner:** Phase 5c, as a documentation obligation on the public retry surface. + +**Re-verified 2026-08-31: the Phase 5c documentation obligation was not discharged.** `MAX_PACING_MS` is +`packages/core/src/retry/pacing.ts:13-14`, commented as the `RETRY-18`/`RECOV-26` ceiling. +`RetrySettings.totalTimeoutMs`'s TSDoc (`settings.ts:22-27`) documents the opt-in and `RETRY-28`'s reasoning +but says nothing about the year-long park it mitigates, and neither does `retryStep`. Owner is now unassigned +— Phase 5c is closed. **The work is a TSDoc paragraph on `totalTimeoutMs` naming the failure mode.** + +**2026-09-02: "a retry parked for a year" is not what happens, and the truth is a separate defect.** +`MAX_PACING_MS` is 31,536,000,000 ms; `Clock.sleep`'s `MAX_SLEEP_MS` is 2,147,483,647 ms — the +clamped hint is **fourteen times** the longest delay the platform can honor, so `waitFor` reaches +`sleep`, `sleep` rejects with an `InvariantViolation`, `waitFor` re-throws it (the signal is not +aborted), and the loop's `catch` returns `failure(InvariantViolation)`. The operator gets an +assertion failure naming the ceiling, not a hang. Measured 2026-09-02 against +`packages/core/src/retry/pacing.ts:14,25` and `packages/core/src/config/clock.ts:65,75`. The clock +defect itself is fixed; this row is the documentation obligation that outlived it. + +**UNSCHEDULED — trigger: the next deliberate edit to `RetrySettings`'s TSDoc.** The owed paragraph +changed twice in one day. It is not "this can park for a year"; it was briefly "a hint above ~24.8 +days fails the call"; and since `Clock.sleep` was made to chain timers it is back to the original +hazard, now real rather than theoretical: **a server-sent pacing hint clamped to `RETRY-18`'s +365-day ceiling is genuinely waited**, and `totalTimeoutMs` is the only thing that shortens it. + + +### P5 — `parsePacingHint` reads only the first value of a repeated header — **WATCH** + +**Where:** `packages/core/src/retry/pacing.ts` + +`Headers.get()` returns the first value. Given `Retry-After: garbage` followed by `Retry-After: 5`, +the parser tries `garbage`, fails, falls through the remaining header names, and returns `null` — no +hint, fall back to backoff — rather than trying the second value. + +Safe (`RETRY-16`'s fallback is the conservative answer) and arguably correct, since a repeated +`Retry-After` is malformed to begin with. `RETRY-21`'s precedence is defined across header *names*, +not across duplicate values of one name, so nothing requires the second value to be tried. + +**Owner:** none. Recorded because "first usable value wins" reads, on a fast skim of `RETRY-21`, like +it should scan duplicates too. + +**Re-verified 2026-08-31:** unchanged. + + +### P6 — A fixed delay is deliberately not clamped to `maxDelayMs` — **WATCH** + +**Where:** `packages/core/src/retry/backoff.ts` + +`computeDelay` returns `fixedDelayMs` before the cap is applied, so `fixedDelayMs: 3_600_000` with +`maxDelayMs: 8000` waits an hour. This looks like a missed clamp and is not: `RETRY-43` describes the +mode as *"zeroing the base and cap so only the fixed delay applies"* — the cap is part of the schedule +this mode replaces, not a bound that outlives it. + +Documented in the field's own TSDoc. Listed here so a future reviewer reaches the reasoning before +"fixing" it. + +**Re-verified 2026-08-31:** unchanged. `packages/core/src/retry/backoff.ts:73` still returns +`settings.fixedDelayMs` before the cap is applied. + + +### P7 — A response that ends the retry loop is handed over live, not closed — **WATCH** + +**Where:** `packages/core/src/retry/engine.ts` + +`RETRY-32` says *"any response that arrives from an already-in-flight attempt MUST be closed rather +than leaked."* The engine closes every response it **discards**. A response that survives the gates — +attempt cap reached, budget spent, status not retryable — is returned **live and unread**, even when +the caller has already aborted. + +That is not a leak: ownership transfers to the caller, which is the only reader that could close it, +and a `Promise` always resolves to its awaiter, so this port has no "value that can never be +delivered" case for the reference's orphan rule to bite on. Both halves are asserted. + +The narrowing is inseparable from `RETRY-36`'s disposition (`toHttpError` drains the body and drops +the headers irreversibly, and 4c's pillar signature must return a `Response`), which the phase design +already ledgers. + +**Re-verified 2026-08-31:** unchanged. + + +### P8 — The Phase 5a design doc overstates the `RETRY-32` guarantee — **UNSCHEDULED** (2026-09-02) + +**Where:** `docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md`, "The wait" + +> `RETRY-32`: once the caller's signal is aborted the driver launches no further attempts, and any +> response arriving from an in-flight attempt is closed rather than leaked. + +The second clause describes only responses the engine discards — see the item above. The +implementation checklist carries the corrected wording; the design doc still carries the blanket +claim, and was left alone because it is a phase design of record, not a working document. + +**Owner:** Phase 9 (cross-cutting conformance), which reads these documents as its source. + +**Re-verified 2026-08-31:** unchanged. The blanket claim is still at +`docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md:357`. The design is a dated record and is not +retro-edited, which is why this item exists instead of an edit; what is owed is a correction *note*, not a +rewrite. Phase 9 is closed, so the owner is now unassigned. + +**2026-09-02: the correction note is this item**, and that is the whole of what is owed. A reader who +reaches the design doc's blanket claim and does not reach this register is the failure mode, and the +only fix for it that does not retro-edit a dated record is a pointer from the checklist — which +already carries the corrected wording. **UNSCHEDULED — trigger: any future reader citing +`docs/work/mvp/phase5/phase5a/…-design.md:357` as evidence of `RETRY-32`'s scope.** P7 carries the +accurate statement. + + +## Section R — Phase 3b execution (2026-08-25, expanded 2026-08-26) + +> **Relocated.** What Phase 3b's execution found once the code existed. Relocated verbatim on 2026-08-31 from the roadmap's +`## Open Findings — Phase 3b Execution` section. Its rows are labelled `E1`–`E7` — the review's own +numbering, not this register's item IDs. + +Findings that surfaced only once Phase 3b's plan was actually executed, across three review passes. Nearly all +are **checkpoint-owned**, not 3b-owned: the 3b design took the checkpoint +(`plans/2026-07-25-checkpoint-scaffold-through-phase3a.md`) as a signed-off prerequisite, and it has not run. +Every box in that document is unchecked and no commit implements it. + +### Why nobody noticed: the checkpoint was cherry-picked, not skipped + +The more useful framing than "the checkpoint did not run" is that **parts of it did**, which is exactly what made +the 3b plan's prerequisite claim plausible to whoever wrote it. Measured status of every `§5` item as of +2026-08-26: + +| § | Item | Status | +|---|---|---| +| 5.1 | Coverage floor as a *blocking* gate | **Done** — `bunfig.toml` carries `coverage = true`, `coverageThreshold = 0.8` | +| 5.2 | Flatten the `DomainModelError` tier | **Done 2026-09-04** — the ten leaves reparented onto `DexpaceError`, the empty marker class deleted, `isDomainModelError` published in its place. E2 retired | +| 5.3 | Error leaves carry identifying `readonly` fields | **Partial** — 2 of 10; E3 below | +| 5.4 | `Symbol.asyncDispose` + floor bump + `lib` entry | **Open** | +| 5.5 | Bounded collections vs `RetentionWindow`/tap | **No action needed** — confirmatory in the checkpoint itself | +| 5.6 | `AbortSignal.any` composition | **No action needed** — confirmatory | +| 5.7 | Flat hoisting lets a package resolve an undeclared dependency | **Open** — E4 below | +| 5.8 | `NFR-14`'s stale "no direct Bun equivalent" reason | **Resolved in Phase 6a (2026-08-27)** | +| 5.9 | `bun test` proves nothing about the Node runtime | **Done 2026-08-26** | +| 5.10 | Per-class `#private` justification comments | **Open** | +| 5.11 | Phase 4 pre-commitment: `Stage` must not be an `enum` | Not yet due (Phase 4) | +| 5.12 | Tooling conflicts already resolved by the plans | Recorded only | + +Partial application is worse here than none at all. `§5.1` is visible in `bunfig.toml` and half of `§5.3` is +visible in `errors.ts`, so a reader checking whether the checkpoint had landed would have found evidence that it +had. **Verify a prerequisite against the artifact it was supposed to produce, not against a spot check.** + +| # | Sev | Finding | Where | Resolution | +|---|---|---|---|---| +| E3 | major — **OPEN, checkpoint §5.3** | §5.3 requires every error subclass to carry its identifying inputs as sanitized `readonly` fields, because `JSON.stringify(error)` and structured-log field enumeration bypass `.message` entirely. It was applied to **two** leaves and stopped: `RequiredFieldError` carries `fieldName`, `HeaderValidationError` carries `kind` + `escapedName`. The other **eight** carry nothing — their identifying data exists only interpolated into the message string, which is precisely the shape the rule forbids. Not raised by any of Phase 3b's three review passes either; found only when the checkpoint was audited item by item | `packages/core/src/http/errors.ts` | **Open.** Same file and the same ten classes as the §5.2 flatten, which shipped on 2026-09-04 without §5.3's fields, so the "one pass rather than two" saving that pairing offered is spent. §5.3 also specifies the sanitization shape per leaf: the offending *name* control-character-escaped, the offending *value* never stored raw (a `valueLength`, a masked minimum fragment, or no field at all), and for `MediaTypeParseError` the failing token/offset rather than the full input. It further asks for a file comment on `errors.ts` recording *why* fields are sanitized at construction — that comment is what stops a later contributor "restoring" the raw value | +| E4 | major — **OPEN, checkpoint §5.7** | No isolated linker is configured. `bunfig.toml` carries only a `[test]` block and there is no `.npmrc` at all, so the install is flat-hoisted by default. Under flat hoisting `@dexpace/core` can import a package it never declared and still pass every gate — including `verify:seam-1`, which reads the `dependencies` map rather than what the code actually resolves. That is the one phantom-dependency failure mode `SEAM-1`'s gate structurally cannot see | `bunfig.toml` (no linker key); no `.npmrc`; `scripts/verify-seam-1.mjs` | **Open.** §5.7 requires confirming the exact linker option against the pinned Bun version before writing it. Low effort, and it strengthens a `SEAM-1` guarantee the project treats as foundational | + +### Phase-3-owned residuals + +Distinct from the checkpoint items above: these belong to Phase 3 itself and are recorded in its ledger and +checklist rather than being anyone else's to close. **Marks re-derived against the tree 2026-09-02.** + +| Item | Level | Disposition | +|---|---|---| +| Multipart boundary **non-appearance** in part content | `HTTP-51`, ⚠️ partial | RFC 2046 puts two duties on the sender; only the `bchars` grammar half is checkable here, because a `StreamBody` part's bytes do not exist until the write and a partial scan would read as a complete guarantee. Mitigated by generating a 32-character Web Crypto boundary by default and documenting the obligation on both caller-supplied entry points. Revisit only if demand for caller-chosen boundaries appears | +| `StreamBody` always single-use, no mark/reset | `BODY-9` (SHOULD), bounded | Node's `ReadableStream` has no generic mark/reset. Closes only if the platform gains one | + +Also worth carrying forward, since three separate defects in 3b traced to the same root: **a `Body`/sink decorator +must forward BOTH teardown paths.** A `WritableStream` adapter that declares `write` and `close` but no `abort` +silently swallows the delegate's abort — the default abort algorithm is a no-op — leaving the real sink open and +locked and letting a truncated body be committed downstream as a complete one. Likewise `pipeTo`'s default +`preventCancel: false` cancels the *source* when the destination fails, which takes cancellation ownership away +from the caller (`BODY-8`). Phase 4c's stage pipeline and Phase 8a's transports both wrap sinks; both inherit this. + + +## Section U — Documentation restructure (2026-08-31) + +Found while giving `docs/` a stated structure: three frozen trees, a `work/` tree of process records, an +as-built `sdk-documentation/` tree, and three registers at the root. Everything below is a consequence of +that pass, not of a phase. + +### U4 — `docs/deviations.md` is keyed to a file inside a frozen tree — **WATCH** + +`docs/deviations.md` states its own coupling: "§10 is the owner of the item numbers… **If §10 renumbers, +this file must be renumbered in the same commit.**" §10 is +`docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, which now sits in a tree +the housekeeping skill refuses to write to. + +That is the right arrangement — the normative ledger should not be edited by a maintenance tool — but it +means the two halves of one numbering scheme are now on opposite sides of a freeze boundary, and only one of +them can be repaired by the tool that notices the drift. Nothing checks that the two agree. + +**Trigger:** the next deviation added to §10, which is a hand edit by definition, and must carry the matching +`docs/deviations.md` edit in the same commit. + +### U5 — `CLAUDE.md` and `README.md` have no gate, and had drifted for nine phases — **WATCH** + +Measured on `f93ccd9`, before this pass: `CLAUDE.md` claimed "two published packages today" against 9 +publishable and 2 private; its API section named 2 committed reports against 9; its gate list omitted +`verify:sse-37`, a blocking CI step; and its documentation-hierarchy table omitted `docs/open-items.md`, the +largest file in the tree. `README.md` was two lines and misspelled "platform". + +Every one of those is checkable against the repository in a few lines of script, and the `housekeeping` +skill's probe stage now does check them (`.claude/skills/housekeeping/probe.mjs`). It is deliberately **not** +a CI step — it is a hand-run tool, like `bun run test:scripts` was before Phase 10 promoted it. + +**Trigger:** the same drift recurring after a phase lands. The probe existing is not the same as the probe +being run; if it recurs, the answer is a blocking CI step, and the precedent for promoting one is +`test:scripts`, made blocking on 2026-08-31 after running in no CI job at all. + +**The skill's own tests are not run by any CI step either.** How many is not written here — the rule +that a count belongs in a command rather than in a sentence, applied: +`node --test .claude/skills/housekeeping/*.test.mjs` reports it. It read **78** on 2026-09-02 before +this pass added four, and **82** after; this row said 77 and a second row said 75, so two rows of one +register carried two different wrong counts of one thing. `package.json`'s `test:scripts` +globs +`scripts/*.test.mjs`, and these live in `.claude/skills/housekeeping/`. Promoting them is a one-line glob +change; the argument for it is `test:scripts`'s, exactly — a gate whose own logic degrades still exits 0, so nothing +else in the run notices. Not done here because wiring this skill into CI was explicitly out of scope for the +change that added it. Run them by hand with `node --test .claude/skills/housekeeping/*.test.mjs`. + +## Section V — Register audit (2026-09-02) + +A pass over all 21 preceding sections against the working tree at `b040968`, verifying every claim +rather than trusting the item's own text. Most of what it found belongs in the items themselves and +is recorded there as a dated note. What is below is what had **no item to belong to**: findings the +earlier reviews stated as prose without an ID, defects nobody had registered at all, and two places +where the register contradicted itself. + +**Why a new letter rather than edits in place.** An item ID is permanent and is cited from source, so +a finding without one cannot be cited, cannot be closed, and does not appear in the section index. +Six of the fifteen below existed only as prose in the four relocated reviews, Sections Q, R, S and T, +carrying no ID of their own; each is numbered here. + +The audit's own driving observation, which is not a numbered item because it is a pattern rather than +a defect: **a phase closes, and the items naming it as owner keep the name.** Twenty items named a +phase that had shipped without doing the work. The roadmap's phase table ends at Phase 10 and Phase +10 is executed, so there is no phase to hand them to and none is invented — hence the `UNSCHEDULED` +status this section adds to the vocabulary above, which carries a `trigger:` instead of an owner. + +--- + +### V2 — `OBS-29` is an unimplemented MUST with no owning row — **UNSCHEDULED** (2026-09-02) + +The one finding in this audit that is a MUST rather than a mismatch. + +`docs/product-spec/15-instrumentation-and-observability.md:54` requires an HTTP-tracer lifecycle: +`operationStarted` once at the start; `operationSucceeded` and `operationFailed` mutually exclusive +and each once at the end; attempt events any number of times; retries-exhausted immediately followed +by `operationFailed` with the same throwable; and **one tracer instance per logical operation**. + +`packages/core/src/observability/tracing.ts:40-42` declares `Tracer` with exactly one method, +`startSpan(name: string): Span`. None of that vocabulary exists under those names. + +**L1 deferred it to Phase 8a and Phase 9. Both shipped; neither took it.** A grep for `OBS-29` across +`docs/work/mvp/phase8/` and `docs/work/mvp/phase9/` returns nothing. An unimplemented MUST sitting +behind a marker that points at two closed phases is precisely the failure this register exists to +prevent, which is why it gets its own letter rather than staying inside L1. + +**What the port does have.** A span is started once (`observability/logging-step.ts:427`) and ended +once, on exactly one of two paths: `span.end()` for success (`:398`), or +`span.recordException(error)` then `span.end()` for failure (`:414-415`). That is start / succeeded / +failed, in order, mutually exclusive, once each — the ordering half of `OBS-29`, under different +names. + +**What it does not have, and this is the part that decides the question.** `PIPE-2` fixes the +`LOGGING` pillar step *inside* the `RETRY` and `REDIRECT` pipelines, so `startSpan` runs **per +transmission attempt and per redirect hop**, not once per logical operation. L4 records exactly that. +`OBS-29`'s "one tracer instance corresponds 1:1 to a single logical operation" is therefore not +satisfied by the span-per-attempt shape, and the per-attempt and retries-exhausted events have no +place to attach. + +**One mitigating datum, which the appendix supplies and the chapter does not.** Appendix C's own +entry for `OBS-29` (`appendix-c-consolidated-normative-requirement-index.md:509`) ends: "This is a +documented emission contract; pipeline/transport wiring to emit it is a follow-up, so it is not yet +runtime-enforced." The spec anticipates the wiring lagging the contract. + +**Decided 2026-09-02: BOTH halves, separately** — the shape is recorded as a deviation, and the +missing wiring stays open. They are different claims and collapsing them into one row is what let +this sit behind two closed phases in the first place. + +**The shape is a `deviations.md` row**, under "Deviations recorded outside a phase": span-shaped +tracing carries `OBS-29`'s ordered started/succeeded/failed lifecycle, and the row states plainly +that the **1:1 tracer-to-logical-operation binding is not met**, because `PIPE-2` puts the span +inside the RETRY/REDIRECT pillars. A reader who finds only that row must not come away thinking the +requirement is covered, which is why the row says which half it is not claiming. + +**The wiring stays here. UNSCHEDULED — trigger: the follow-up wiring appendix C `:509` names — an +outermost per-operation span opened by the client entry point, outside the pillars.** That is the +only shape that satisfies the 1:1 binding without moving the `LOGGING` step out of the pillars, which +`PIPE-2` fixes. Not implemented in this pass: it is new observability surface, not a repair. + +--- + +### V11 — the Phase 4b validation review's corpus conflict declines to be a finding, and has sat unsettled since 2026-07-28 — **FIXED** (2026-09-04) + +`docs/knowledge/harvested/function-design.md:22-23` requires an options object at three or more +parameters; `:40-41` sets `max-params: ['error', 3]`, which errors only at four. The prose is one +parameter stricter than its own stated enforcement, and the Phase 4b validation review filed its row +`F9` against the prose. That review says the conflict is "worth settling in the corpus rather than +per-phase" and stops there — no ID, no owner, no trigger. + +**What the repository actually does, measured 2026-09-02: it follows the lint threshold.** +`eslint.config.js` sets `max-params` to 3 (errors at four), and three-parameter functions ship +throughout — `Transport.send(request, options?, signal?)`, `fold(outcome, onSuccess, onFailure)`, +`Deserializer.deserializeFrom(source, schema, typeName?)`, `redactHeaderValue(name, value, policy?)`. +Several carry a documented `eslint-disable-next-line max-params` for a fourth, which is the gate +being enforced rather than evaded. So the conflict is settled in practice and unsettled on paper, and +that row, filed against the prose, dissolves under the reading the code takes. + +**Not settled *in the corpus* here.** `docs/knowledge/harvested/` is frozen and is never hand-edited; +the mechanism for recording a reading against a harvested rule is a note under +`docs/knowledge/notes/`, and writing one is a deliberate edit to a frozen tree rather than a +maintenance action. ~~**UNSCHEDULED — trigger: the next deliberate edit to +`docs/knowledge/notes/`**, which should carry this alongside N3's two markers and the two `` +paths the documentation restructure left pointing at files it had moved. One visit, three fixes.~~ + +**FIXED 2026-09-04.** `docs/knowledge/notes/function-design.md` records the reading: the threshold +this repository enforces is the lint one — three positional parameters legal, four an error — and it +bounds `function-design/45a4ddba` (the prose, "3 or more") with `function-design/27da9d1f` (the +enforcement, `max-params: ['error', 3]`). Both harvested entries print +`[overridden by notes/function-design.md:8]`. The boolean half of the prose rule is explicitly left +binding; only the numeric threshold is bounded. + +The visit carried N3's remaining marker too (`notes/pipeline.md`), so two of the three fixes it named +are done. **The two stale `` paths are not**, and cannot be taken this way: they are inside +`docs/knowledge/harvested/`, which is frozen and regenerated, so correcting them is a re-harvest of +those sources rather than an edit. Registered as `X4` rather than left inside this closed row. + +--- + +## Section W — register dispositions taken 2026-09-04 + +A maintainer pass over `docs/deferred-items.md`, row by row, deciding each rather than re-deferring +it. Four rows left the table: `challengeHandler` (kept, reasoning moved onto the field itself), +`DomainModelError` (flattened — Section R's `E2` and Section V's `V3` retired in the same change), +the `operation` `AuthTier`, and `#private`-vs-`private` (closed as correctly scoped, recorded as +`docs/knowledge/notes/data-modeling.md`). Three of the four closed cleanly. The fourth did not, and +is `W1`. + +**The pass ended by dissolving the register itself.** With four rows decided, what remained did not +earn a file: one row with live actionable content, and five unscheduled deferrals each carrying a +trigger and nothing to act on. So `docs/deferred-items.md` was deleted. The `NFR-16` row became +[`first-release.md`](../../first-release.md); the five others are archived under *Live deferrals* in +[`work/mvp/2026-09-04-register-retirement-purge.md`](./2026-09-04-register-retirement-purge.md). +**A new deferral is an open item in this file from here on**, stating the trigger that would discharge +it — there is no second register to send one to. + +--- + +### W1 — the `operation` `AuthTier` row was closed on a premise the petstore spike had already falsified — **FIXED** (2026-09-04) + +The row read `BLOCKED — no source layer exists on this roadmap`, and justified carrying no trigger on +the grounds that "there is no trigger to state because nothing on the roadmap can fire one." It was +removed from `docs/deferred-items.md` on that basis. + +**The premise was already false when the row was read.** `7b26c1c` (2026-09-03, PR #65) added +`examples/petstore/` as the witness for the codegen target surface — which is precisely the +per-operation configuration layer the row said nothing would ship — and its findings document +measured the gap rather than merely noting it +([`examples/petstore/FINDINGS.md`](../../../examples/petstore/FINDINGS.md), §4): + +1. **`AUTH-4`'s precedence chain is reimplemented outside core.** The spike's executor folds the + operation's descriptor into the per-call slot — `const auth = call.auth ?? operation?.auth` — so + the top two-thirds of the tier chain lives in consumer code. Every generated SDK would carry it. +2. **Core cannot tell the two tiers apart once they are folded.** A caller's genuine per-call + override and an operation's declared requirement arrive in the same slot, the collision is + resolved before core sees it, and core can therefore neither audit nor log which tier won. +3. `AuthTiers.operation` has no writer anywhere in the workspace. + +**No requirement is unmet.** `AUTH-4`–`AUTH-7` are mechanically satisfied — presence-selects-the-tier +works, and the spike's canary asserts all three outcomes including an unsatisfiable `OAUTH2` +requirement raising `AuthResolutionError` with `transport.calls` still empty. What is defective is +the *shape*: core publishes a tier it gives consumers no way to fill, so the consumer reimplements +core's own precedence rule. + +**The fix is already specified**, by the spike rather than by this item: either `RequestOptions` +gains `operationAuth?: AuthDescriptor`, which `effectiveTiers()` folds into `AuthTiers.operation`, or +`StepContext.options` carries the operation descriptor separately. Either makes the consumer-side +fold disappear. Both are additive to a `0.0.0` package. + +**Why an open item rather than a restored deferral.** The register boundary is *when* the item was +created: a deferral is a decision taken before the work, an open item a discovery made after. Phase +5c's decision not to build the tier stands and is genuinely settled — what changed is that a later +spike found the consequence. That is a discovery, so it lands here. The closure of the deferral is +not reversed. + +~~**Owner: whoever lands the codegen surface.** Not scheduled against a phase, because the roadmap's +phase table ends at Phase 10 and Phase 10 is executed.~~ + +**FIXED 2026-09-04, by the first of the two options the spike named.** `RequestOptions` gains +`operationAuth?: AuthDescriptor`, a second per-call slot alongside `auth`, and `effectiveTiers()` +folds it into `AuthTiers.operation`: + +- `packages/core/src/http/request-options.ts` — the `#operationAuth` field, the getter, the + `RequestOptionsBuilder.operationAuth()` setter, and `newBuilder()` carrying it forward (HTTP-3). +- `packages/core/src/auth/auth-step.ts` — `effectiveTiers(configured, perCall, operation)` now + applies **each slot only when present**, so a configured tier is never overwritten with + `undefined`; `{...configured, perCall: undefined}` would have erased a `perCall` the step was + constructed with. +- `packages/core/etc/core.api.md` — additive, 39 inserted lines across this and the same day's other + promotions, none deleted. + +**Why this option and not the other.** `StepContext.options` already travels from `Runtime.send` to +every step across every retry attempt and redirect hop, and it is the carrier `authStep` reads today +(`ctx.options?.auth`). Carrying the operation descriptor as a *separate* `StepContext` field would +have added a second parallel carrier for the same lifetime and made the pipeline's plumbing wider +for one consumer. The slot is generator-facing rather than caller-facing, and precedence protects it +either way: a hand-written caller that fills `operationAuth` is still outranked by `auth`. + +**Verified end to end by removing the fold it existed to eliminate.** +`examples/petstore/src/service-core.ts`'s `requestOptions()` no longer computes +`call.auth ?? operation?.auth`; it fills `.auth(call.auth)` and `.operationAuth(operation?.auth)` and +lets core resolve `perCall ?? operation ?? client` itself. The spike's canary passes unchanged, 15 +tests — including the unsatisfiable-`OAUTH2` case that must raise `AuthResolutionError` with +`transport.calls` still empty. All three tiers are now distinguishable inside core, which was +consequence 2 of the finding. + +New tests: four in `packages/core/src/http/request-options.test.ts` (round-trip, `newBuilder` +carry-forward, slot independence, `EMPTY`) and two in `packages/core/src/auth/auth-step.test.ts` +(the operation tier beating the client tier; a per-call descriptor still beating an operation one). + +`AuthTiers.operation`'s TSDoc no longer says nothing writes it. + +--- + +## Section X — holes found while closing the 2026-09-04 decision pass + +A maintainer pass that decided the register's open questions rather than re-triaging them, closing +`K1`, `K11`, `K12`, `K19`, `H19`, `N3`, `V11` and `W1`, promoting `H8`'s two barrel questions, and +drafting `G1`'s erratum into [`deviations.md`](../../deviations.md). Each closure was taken by reading +the tree rather than the item's own text, and four holes turned up in that reading that belonged to +no existing row. + +They are here rather than inside the rows that found them because an item without an ID cannot be +cited, cannot be closed, and does not appear in the section index — the same reason Section V exists. + +### X1 — `idempotencyKeyStep` is unreachable, and blocked by the forgotten-export rule K1 escaped — **UNSCHEDULED** (2026-09-04) + +`RECOV-32`'s step is implemented and tested at `packages/core/src/recovery/idempotency-key.ts:37`, +is tagged `@internal` (`:35`), is absent from `packages/core/src/index.ts`, and has **zero +consumers** — `grep -rn idempotencyKeyStep` over `packages/`, `tests/` and `examples/` returns its +own definition, its own test and one TSDoc reference. Exactly K1's shape, one requirement over. + +**It is not K1's fix repeated, because the blocker K1 shed is still live here.** K1 was unblocked +when Phase 5c promoted `StepDescriptor` (`packages/core/src/index.ts:82-86`, `@public` at +`packages/core/etc/core.api.md:1314`), so exporting `clientIdentityStep` named no forgotten export. +`idempotencyKeyStep` returns `RequestStep` (`packages/core/src/recovery/request-chain.ts:11`), which +is **not** exported — see `X2`. api-extractor rejects a `@public` export whose return type is a +forgotten export, so this one cannot be promoted on its own. + +**Trigger: `X2`.** Promoting the return type is the decision; this row follows it mechanically. + +### X2 — the whole of `recovery/` is absent from the public barrel — **UNSCHEDULED** (2026-09-04) + +`grep -n "recovery/" packages/core/src/index.ts` returns nothing, and no recovery-chain symbol +appears in `packages/core/etc/core.api.md`. `RequestStep`, `ResponseStep`, +`RequestRecoveryChain` and `ResponseRecoveryChain` are all in-package only, so the `RECOV-*` chain +surface is unreachable from a published entry point in the same way `pipeline/`'s authoring surface +was before Phase 5c promoted it. + +Recorded rather than taken, for the reason K1 sat unclosed for two phases: promoting a seam to +unblock one step factory is the decision made backwards. `F4` is the same surface's other open +question — whether the chains stay classes or become plain data plus free functions +(`docs/knowledge/harvested/data-modeling.md:10`) — and a promotion that ships the class shape +forecloses it. **Trigger: whichever comes first, a decision on `F4` or a consumer that needs to +build a recovery chain**; the two should be settled in one pass, and `X1` rides along with it. + +### X3 — `examples/petstore/`'s canary runs in no CI step — **UNSCHEDULED** (2026-09-04) + +`7b26c1c` added the petstore spike as the witness for the codegen target surface, and its +`canary.test.ts` and `regen.test.ts` are 15 real assertions over the public API — including the +`AuthResolutionError`-with-empty-`transport.calls` case that is the sharpest thing anyone has written +about `AUTH-6`. Nothing runs them. `grep -n "petstore\|examples" package.json .github/workflows/ci.yml` +returns nothing; the root test script is `bun test ./packages ./tests`, and `examples/` is neither. +`bun test ./examples/petstore` passes today only because it was run by hand. + +`gts lint .` **does** reach `examples/` (the spike's own finding 7), so the tree is type- and +lint-checked and merely never executed. That is the worse half of the two: a witness that compiles +and is never run degrades silently, which is exactly what `W1`'s fix would have had no way to detect +had the canary not been run deliberately. + +Not fixed here because it is a change to the CI gate list and to the root test script's scope, and +the scoping question is real — `examples/` is not a workspace member, its coverage would land in the +80% floor's denominator, and the honest fix may be a separate `test:examples` step rather than +widening `bun run test`. **Trigger: the next change to the CI gate list**, which `K12` just made and +this row deliberately did not ride along with. + +### X4 — two harvested `` paths still point at files the documentation restructure moved — **UNSCHEDULED** (2026-09-04) + +Named by `V11` as the third of its "one visit, three fixes", and the one a visit to +`docs/knowledge/notes/` cannot take: the paths are inside `docs/knowledge/harvested/`, which is +frozen and regenerated. A hand edit there changes no `` sha, so the next harvest reproduces the +stale path — `docs/knowledge/README.md` is the contract. Correcting them means re-harvesting those +sources with `--corpus docs/knowledge/harvested`, not editing them. + +`bun run knowledge:drift` is where they surface, alongside the 3 `DRIFT` rows of 47 sources it +reports today. **Trigger: the next re-harvest.** Split out of `V11` so that row could close on the +part that was actually takeable. + +--- + +## Maintaining this file + +Add an entry the moment a gap is found, not when it is fixed — the failure mode this file prevents is a +checklist row marked ✅ against code that does not implement it (A2 is one such instance). +**Never delete a live entry.** When the underlying requirement is genuinely satisfied *and* its checklist +row agrees, remove the body and drop the ID from the Section index; nothing is kept in its place. The ID is +never renumbered and never reused. When a phase closes, re-scan its checklist against the code rather than +trusting the marks. + +**A new review is a new section, with the next letter.** Never renumber an existing item and never reuse a +letter — including a letter the Section index no longer lists, whose items are all closed: item IDs are +cited from source comments, which no gate updates. `node scripts/knowledge.mjs` has +nothing to do with this file; the check that every citation resolves lives in the `housekeeping` skill's probe +(`.claude/skills/housekeeping/probe.mjs`), which found six mis-cited IDs the first time it ran. + +**Heading form.** `## Section — ` for a section, `### — — **STATUS**` +for an item. Sections A–G used `## A.` until 2026-08-31; the letters did not change, only the form. + +**Do not open a second register.** One was opened at the repository root in `cba4721` and sat unmerged for +five days across four phases (now Section P). A finding that is not in this file is not registered, wherever +else it is written down. + diff --git a/docs/work/mvp/2026-09-04-register-retirement-purge.md b/docs/work/mvp/2026-09-04-register-retirement-purge.md new file mode 100644 index 0000000..a0ac9fd --- /dev/null +++ b/docs/work/mvp/2026-09-04-register-retirement-purge.md @@ -0,0 +1,451 @@ +# Register retirement purge — the audit trail the two registers no longer carry + +**2026-09-04.** `docs/work/mvp/2026-09-04-open-items-dissolution.md` and `docs/deferred-items.md` each carried a retirement table: one +compact row per item that had been resolved, or per deferral that had been discharged. Both tables were +deleted on this date, by decision of the repository owner, together with the prose in each register that +described them. This note is where their contents went. + +**Why the note exists at all.** The registers' own rule was that a retired ID is never released: a source +comment citing `K10` or `T.F9` still had to resolve, and the `housekeeping` probe's citation check read the +retirement table as a second namespace of resolvable IDs alongside the live `### <ID>` headings. Deleting the +tables took that namespace with it and left 25 citations pointing at nothing. Rather than rewrite the dated +records that carry them — `docs/work/` is never retro-edited, and `.changeset/` is published release history — +the namespace moved here. `.claude/skills/housekeeping/probe.mjs` reads the **Purged item IDs** table below as +the second source, so every one of those citations resolves again, to a row that says what the item was and +how it closed. + +**What this note is not.** It is not a register. Nothing is appended here as work proceeds; it is a dated +record of two deletions, both made on 2026-09-04 — first the two retirement tables described above, then, +later the same day, the whole of `docs/deferred-items.md`. A new finding still goes to +`docs/work/mvp/2026-09-04-open-items-dissolution.md`. **A new deferral goes there too now**, as an open item stating the trigger that would +discharge it, because there is no deferral register left to send it to: `docs/work/mvp/2026-09-04-open-items-dissolution.md` and +`docs/deviations.md` are the two that remain. The five deferrals that were still live when the register was +deleted are archived under *Live deferrals* below — an archive of record, not an intake, and not appended to +either. + +**Counts, at the moment of deletion.** 102 retired item IDs, 25 retired rows that never carried an ID, and 67 +discharged deferrals. `docs/work/mvp/2026-09-04-open-items-dissolution.md` went from 1933 lines to 1767, `docs/deferred-items.md` from 199 to +104. Every table below is reproduced verbatim from `git show HEAD:` of the pre-deletion files, commit +`853c349`; the rows' own `file:line` evidence is unmodified and may itself have drifted since the date each +row carries. + +**The register's own deletion, later the same day.** `docs/deferred-items.md` did not stay at 104 lines. A +maintainer pass decided four of its ten remaining rows rather than re-deferring them (`docs/work/mvp/2026-09-04-open-items-dissolution.md` +Section W), and the file — 99 lines by then — was deleted outright: the `NFR-16` row became +[`docs/first-release.md`](../../first-release.md), the other five moved to *Live deferrals* below, and every +live reference to the path across the tree was repointed. `docs/work/` was not, because it is never +retro-edited; its ~20 mentions of the register are correct records of what was true when they were written. + +**Citations still in flight.** The bare-ID references inside test titles and inline comment shorthand — +`(T.F9/V15)`, `(H14/P1, RECOV-12)`, "the very collision V13 closed", 44 of them across 20 files — were +deliberately left in place. They read as part of the code, not as register pointers, and the citation check +never matched them (it requires the `open-items.md` path beside the ID). + +--- + +## Purged item IDs + +The 102 IDs the deleted `## Retired items` table reserved. **The IDs are still not released** — never +renumbered, never reused. This table is what the probe's citation check resolves them against. + +| ID | Title | Resolution | Date | Evidence | +|---|---|---|---|---| +| `A1` | HTTP-24: `charset` did not return null for an unknown encoding | resolved against the runtime's WHATWG encoding registry, returns `undefined` | 2026-09-02 | `packages/core/src/http/media-type.ts` — the `charset` getter and `isKnownEncoding` | +| `A3` | HTTP-11: `Response` exposes no range classification of its own | closed on the delegation reading — `response.status.isSuccess` is one hop | 2026-09-02 | deviations.md row: "`HTTP-11`'s range classifications are on `Status` only" | +| `A5` | CTX-8: the duplicate-key error's message did not identify the key | default keys gained a serial description | 2026-09-02 | `packages/core/src/context/context.ts` — `defaultKey()` | +| `B1` | NFR-10/NFR-17: CI never runs on the declared minimum runtime | closed by the `node-conformance` job, a matrix over the declared floor and current LTS | 2026-08-26 | `.github/workflows/ci.yml`; `tests/node-conformance/` | +| `B2` | NFR-13: SPDX headers missing on scaffold-era files | all three files carry the line-1 header | 2026-09-02 | `eslint.config.js:1`, added in `d8217af` | +| `B3` | NFR-12: reproducible builds asserted, never proven | gated — two clean builds agree on every emitted file and every tarball | 2026-08-29, widened 2026-08-30 | `scripts/verify-reproducible-build.mjs`, a blocking CI step | +| `B4` | NFR-14: `expect-type` breaks the single-source-of-versions convention | premise inverted — nine manifests all read `"catalog:"` | 2026-09-02 | root `package.json:12` | +| `C2` | The structural-typing bypass deviation is not yet recorded | Phase 10 recorded it as §10 ledger item 4 | 2026-09-02 | `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:64-75` | +| `E1` | Phase 1 has no commits | decided by what happened — one squashed commit, accepted | 2026-09-02 | commit `8051364` | +| `F3` | Zero `invariant()` assertions across `recovery/` | won't fix — density is not a target, decided project-wide | 2026-09-02 | deviations.md row: "`invariant()` density is not a target, project-wide"; `docs/deferred-items.md`'s *Assertion-density rule applied project-wide* row (retired) | +| `F5` | `#private` fields carry no per-use justification | closed — `CLAUDE.md` states the convention once, with the styleguide 6.7 carve-out | 2026-09-02 | `CLAUDE.md`, "Domain model construction pattern"; `docs/deferred-items.md`'s *`#private`-vs-`private`* row, still deferred | +| `F6` | `RECOV-11` is a no-op in this port | ledgered as promised | 2026-09-02 | `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md:36-37` | +| `F8` | 4b did not depend on 4a | premise corrected — 4b's only outside imports are `http/`, `body/`, `seams/`, `invariant`, `suppress`; the sequencing half closed when both merged into the 4c branch | 2026-08-26 | `packages/core/src/recovery/` | +| `G2` | `REDIR-28`/`REDIR-15` observability clause ships unimplemented | all four event families now emit | 2026-09-02 | `packages/core/src/redirect/redirect-step.ts` | +| `G3` | `Decision` carries no reason on `'return-current'` | `RedirectStopReason` shipped; the two blocked events emit | 2026-09-02 | `packages/core/src/redirect/decide.ts` | +| `G4` | `REDIR-20`'s "fully override" read as scoped to eligibility | the scoped reading is confirmed and recorded | 2026-09-02 | deviations.md row: "`REDIR-20`'s 'fully override' is read as scoped" | +| `G7` | `XCUT-17`(b)'s foreign-host half needs an auth layer | delivered by Phase 5c — the auth step is the marker's consumer | 2026-09-02 | `packages/core/src/auth/auth-step.ts`, `planOutbound` | +| `G10` | Phase 5c publishes `Step`, the context family, and a PROVISIONAL `InstrumentationBundle` | accepted — `activeSpan`/`tracerFactory` are documented as provisional in the emitted `.d.ts`, on the interface and on each member, which is the honest form of the trade | Phase 5c | `packages/core/src/context/instrumentation.ts`; `packages/core/etc/core.api.md` | +| `G11` | `DigestChallengeUnsupportedError` was speculative | cut during Phase 5c's own shape review, before it shipped | Phase 5c | `packages/core/src/auth/` — no such class | +| `G12` | `AUTH-37`'s failed background refresh is swallowed silently | emits `http.auth.bearerRefreshFailed` at `warning`, then continues | 2026-09-02 | `packages/core/src/auth/bearer-cache.ts`, `warnRefreshFailed` | +| `H1` | `@dexpace/codec-json` buffers the whole decoded body before parsing | accepted deviation — `JSON.parse` has no incremental form, so this is a property of the format, not of the seam; `decodeResponse` itself never buffers | Phase 6a | `packages/codec-json/src/json-serde.ts`; `packages/core/src/serde/response-handlers.ts` | +| `H2` | `SERDE-23` (ignore unknown fields) is satisfied by delegation, not enforcement | accepted deviation — stripping or rejecting an extra wire key is the caller's schema's property, and core cannot override it without defeating caller-supplied schemas | Phase 6a | `jsonSerde`'s TSDoc, `packages/codec-json/src/json-serde.ts` | +| `H3` | No serde-specific error base class | accepted deviation — two flat leaves under `DexpaceError` plus `isSerdeError`, because checkpoint §5.2 caps the tier at two levels | Phase 6a | `packages/core/src/serde/errors.ts`; `packages/core/etc/core.api.md` | +| `H5` | `NFR-8`/`NFR-9` shrinker keep-configuration | Phase 9 answered it the other way — `NFR-8` not applicable, `NFR-9` shipped | 2026-09-02 | `packages/shrink-test/`; deviations.md §10 | +| `H6` | Assertion density in 6a | won't fix — the same decision F3 records | 2026-09-02 | F3, retired; `docs/deferred-items.md`'s *Assertion-density rule applied project-wide* row (retired) | +| `H12` | `seams/index.ts` is an unimported internal barrel | deleted, after a three-way proof that it was dead | 2026-09-02 | `packages/core/src/seams/` carries no `index.ts` | +| `H13` | `test:scripts` runs in no CI job | closed by the `Gate self-tests (scripts/*.test.mjs)` step, mirrored in the preflight | 2026-08-31 | `.github/workflows/ci.yml`; `.claude/skills/ci-preflight/run-ci.mjs` | +| `H14` | `decodeSuccessResponse`'s 4xx/5xx branch is unprotected against a teardown failure | fixed at `toHttpError` with `releaseQuietly`/`withReleaseFailure` | 2026-09-02 | `packages/core/src/body/http-status-error.ts`; three cases in its test | +| `I1` | `SSE-41` reactive adapter deferred to Phase 8b | delivered by Phase 8b | 2026-09-02 | `packages/rx/src/sse.ts:34,50` | +| `J1` | `PAGE-11` close-before-yield vs §7.1's illustrative snippet | resolved with an erratum — `PAGE-11` governs; materialized items survive close, so closing first releases the response immediately | Phase 6c | `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md` §7.1; `docs/knowledge/notes/pagination.md` | +| `J2` | `PAGE-5`/`PAGE-29` asynchronous `PaginationStrategy.parse` signature | resolved with a spec clarification — bodies arrive as async streams, so `parse` returns `Promise<PageInfo<T>>` with the isolated non-mutating semantics intact | Phase 6c | `packages/core/src/pagination/strategy.ts` | +| `J3` | `Page<T>` disposal is a runtime-guarded install, not `implements AsyncDisposable` | resolved — guarded `[Symbol.asyncDispose]` install with `close()` as the supported teardown; the `>=20.4` floor bump is decided against | 2026-08-30 | `packages/core/src/pagination/page.ts`; Section D's `await using` row; `I3` | +| `J4` | WHATWG encode-set boundary and verbatim query splice | resolved by design — hand-rolled tokenization over the raw query substring, so untargeted parameters survive byte-for-byte (`PAGE-21`/`PAGE-22`) | Phase 6c | `packages/core/src/pagination/query-splice.ts` | +| `J5` | Transport-direct pagination without an internal resilience loop | resolved by design — resilience composes externally at the pipeline layer (`PIPE-9`), keeping the engine transport-agnostic | Phase 6c | `packages/core/src/pagination/paginator.ts` | +| `J6` | `items()` vs `pages()` single-use asymmetry | resolved by design — `items()` re-walks and closes each page before yielding; `pages()` hands out live connection ownership and so is single-use (`PAGE-8`/`PAGE-14`) | Phase 6c | `packages/core/src/pagination/paginator.ts` | +| `J7` | Iterative generator drive without a trampoline | resolved by design — `PAGE-31` sanctions native loops; `#walk` and `driveFetchers` are `async function*` loops in constant stack space | Phase 6c | `packages/core/src/pagination/paginator.ts`, `fetchers.ts` | +| `J8` | Error unwrapping and root-cause propagation | resolved by design — `PaginationError` is reserved for engine misuse; transport, parse and network failures propagate unwrapped with their causes (`PAGE-28`) | Phase 6c | `packages/core/src/pagination/errors.ts` | +| `K2` | Proxy resolution implements the property tier the design ledgered as collapsed | resolved — the ledger wording was narrowed to say the *production sources* collapse, not the resolution logic; without the tier `CFG-24`/`CFG-26` would have been silent gaps | 2026-08-27 | `packages/core/src/config/proxy.ts`; the 7a design doc's ledger row | +| `K4` | `CFG-28`'s global-configuration convenience resolver is not built | closed — the clause is a MAY, and practice threads a `Configuration` | 2026-09-02 | `packages/transport-undici/` | +| `K5` | `CFG-35`'s throwable axis is not in this phase | already delivered by Phase 5a when the row was written | 2026-09-02 | `packages/core/src/retry/classify.ts` | +| `K9` | `Configuration.default()` ships as a free `defaultConfiguration()` | resolved — `Configuration` is an `interface` in this port and cannot carry a static; the plan's own alternative placement | 2026-08-27 | `packages/core/src/config/configuration.ts:354` | +| `K10` | `CFG-24`'s warning half is not emitted | emits `http.proxy.configRejected` on every rejection path | 2026-09-02 | `packages/core/src/config/proxy.ts` | +| `K14` | A configuration seam that fails is silently invisible | `readLayer` emits `config.sourceFailed`, carrying layer and key | 2026-09-02 | `packages/core/src/config/configuration.ts` | +| `K15` | `HTTP-17`: `hasForbiddenNameByte` permits a space in a header name | decided — the predicate matches the frozen requirement exactly, no deviation filed | 2026-09-02 | `packages/core/src/http/ascii-validation.ts`; `docs/product-spec/04-core-http-domain-model.md:32` | +| `K17` | `formatProxyOptions` re-brackets an IPv6 host stored bare | resolved — bracketing lives in the formatter, keyed off a colon in the host; `host` stays bare as the stored representation | 2026-08-27 | `packages/core/src/config/proxy.ts:222` | +| `L2` | G2's deferred emissions | resolved — `REDIR-28` hop and rejection logging and `REDIR-15` downgrade logging are active through `getGlobalLogger()`; see `G2` | 2026-08-28 | `packages/core/src/redirect/redirect-step.ts` | +| `L3` | G12, K10, K14 config and auth logger retrofit | all three sites emit, taken as one change | 2026-09-02 | `auth/bearer-cache.ts`, `config/proxy.ts`, `config/configuration.ts` | +| `M2` | `ASYNC-*` IDs marked 🚫 are not satisfied anywhere | Phase 8a landed both transports and the shared conformance suite | 2026-09-02 | `a0d734d`; `packages/transport-conformance/` | +| `M3` | `ASYNC-18` confirmed a full-port collapse at implementation time | resolved — `@dexpace/rx` contains no timer, scheduler or backoff; already reflected in §10 ledger item 1 | 2026-08-28 | `packages/rx/`; `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` | +| `N1` | Cancellation surfaces two different types depending on the layer | a core-side `abortToSdkError` maps both core sites, timeout still distinct | 2026-09-02 | `packages/core/src/cancellation.ts` | +| `N2` | `HttpStatusError`'s constructor fabricates a "successful exception" | constructor enforces 400-599; `retire()` got its own trail-entry leaf | 2026-09-02 | `packages/core/src/body/errors.ts`, `packages/core/src/retry/errors.ts` | +| `O3` | The phase records still carry pre-split corpus paths | won't fix — `docs/work/` files are dated records of what a phase planned, are never retro-edited, and `CLAUDE.md` states the split so it does not read as an oversight | 2026-09-02 | `CLAUDE.md`; `grep -rhoE "docs/knowledge/[a-z0-9-]+\.md" docs/work \| wc -l` | +| `P1` | `toHttpError`'s `finally` can mask the drain failure | merged into H14, which is canonical and carries the fix | 2026-09-02 | H14, retired | +| `P2` | `RequestOptionsBuilder.maxRetries` — the pattern deserves a sweep | sweep run over every public numeric setter; four holes fixed to the full range | 2026-09-02 | `http/request-options.ts`, `retry/settings.ts` | +| `P9` | Phase 7b still owes `engine.ts` two log events | both ship, plus a third the finding did not anticipate | 2026-08-31 | `packages/core/src/retry/engine.ts` | +| `U1` | Five citations point at paths the restructure moved | closed as a finding — four corrections handed to the frozen trees, the `.changeset/` one left permanently | 2026-09-02 | `docs/knowledge/notes/`, `docs/sdk-design-nodejs/10-…` (see HANDOFF) | +| `U2` | Three phase deferrals never reached the aggregate log | recovered into the aggregate | 2026-08-31 | `docs/deferred-items.md` | +| `U3` | Three `F` namespaces coexist; a bare citation is ambiguous | options 1 and 3 together — a section qualifier, taught to the citation check | 2026-09-02 | `.claude/skills/housekeeping/probe.mjs`; this file's Section index | +| `U6` | Six citations named the wrong section, four resolved to nothing | five corrected in source; the `.changeset/` one left as frozen history | 2026-08-31 | `packages/core/src/config/` | +| `U7` | `redirectStep()` is public; the guard that makes it safe is not | option 1 — `withRedirect` and `stripCrossOriginMarkerStep` promoted | 2026-09-02 | `packages/core/etc/core.api.md` | +| `U8` | Two published READMEs shipped a sample that does not compile | both show `close()` in a `finally`, and say why | 2026-08-31 | `packages/transport-fetch/README.md`, `packages/transport-undici/README.md` | +| `U9` | A `@throws` named a class that does not exist, ten more unreachable | option 3 — eight promoted, the two bug-signalling tags rewritten | 2026-09-02 | `packages/core/etc/core.api.md`; `docs/sdk-documentation/errors.md` | +| `U10` | Three documents stated three different, all-wrong citation counts | replaced by one derivation, a command rather than a sentence | 2026-09-01 | `node .claude/skills/housekeeping/probe.mjs --only=citations` | +| `U11` | The count checker could not read the counts it was written for | `parseNumeral` plus a subject-anchored, required claim table | 2026-09-01 | `.claude/skills/housekeeping/probe.mjs`, `probe.test.mjs` | +| `V1` | `OBS-19`/`TRANSPORT-13`: three modes, one level | `'all'` and `'first-per-name'` now warn; `'quiet'` stays silent | 2026-09-02 | `packages/transport-shared/src/drop-log.ts` | +| `V4` | `retrySettings` accepted a delay no timer can honor | closed from the other side — V13's chunked clock waits any finite duration | 2026-09-02 | `packages/core/src/retry/settings.ts`; `packages/core/src/config/clock.ts` | +| `V5` | One defect, two register letters, two contradicting statuses | resolved by merge — H14 canonical, P1 points at it | 2026-09-02 | H14 and P1, both retired | +| `V6` | The register hard-coded its own citation count | number replaced by the derivation command U10 prescribes | 2026-09-02 | this file's Section index | +| `V7` | Section Q's `Response.close()` latch claim is stale | Q's superseded paragraph is removed by this retirement pass | 2026-09-02 | `packages/core/src/http/response.ts:200-207` | +| `V8` | Section Q's assertion-count correction is wrong about Phase 4a | Q's superseded paragraph is removed by this retirement pass | 2026-09-02 | `packages/core/src/context/store.ts:35,49,67` | +| `V9` | Section R's "Suggested order" sequences work before Phase 4 | closed as advice; the block is removed by this retirement pass | 2026-09-02 | Section R | +| `V10` | Section R's Phase-3 residuals show four shipped rows as pending | marks corrected, then the four rows retired with this pass | 2026-09-02 | the four `R — residual` rows below | +| `V12` | Section H asserts and denies the same fact, four items apart | `H4`'s stale sentence struck and pointed at `H18` | 2026-09-02 | `packages/codec-json/tsconfig.json` carries no `references` key | +| `V13` | A `RETRY-18`-clamped pacing hint exceeds any timer | `Clock.sleep` chains timers, so a 365-day hint is waitable and no deviation is owed | 2026-09-02 | `packages/core/src/config/clock.ts`, `sleepInChunks` | +| `V14` | `N2`'s premise is false: core constructs the forbidden exception | `RetryDiscardedResponseError` gave `retire()` an honest trail entry | 2026-09-02 | `packages/core/src/retry/errors.ts` | +| `V15` | Section T's `F9` deadline passed; `Cursor` never checks the signal | `Cursor.#dispatch` checks at every step boundary, mapping through N1's mapper | 2026-09-02 | `packages/core/src/pipeline/cursor.ts` | +| `Q.D1` | Changeset level for `Request.body`'s narrowing to `Body \| undefined` | resolved to minor under semver's initial-development carve-out, pointer in the changeset | 3b execution | `.changeset/2026-08-25-body-lifecycle.md` | +| `Q.D2` | Three Phase-1/3a symbols the 3b plan called could not be verified | verified against the real code; the real names were used, no duplicates added | 3b execution | `packages/core/src/io/limits.ts` (`MAX_BYTE_ARRAY_LENGTH`), `http/status.ts`, `http/protocol.ts` | +| `R.E1` | `[Symbol.asyncDispose]` declared ahead of the declared floor | 3b reverted to `close()`-only; the §5.4 reopening closed when the `>=20.4` bump was rejected | 2026-09-02 | Section D's `await using` row; `I3` | +| `R.E5` | `bun test` proves nothing about the Node runtime | §5.9's own prescription implemented — a `node --test` tree plus a two-version CI matrix | 2026-08-26 | `tests/node-conformance/`; `.github/workflows/ci.yml` | +| `R.E6` | No per-class `#private` justification comment | closed on `F5`'s reading — the convention is stated once, project-wide | 2026-09-02 | `CLAUDE.md`, "Domain model construction pattern" | +| `R.E7` | `NFR-14`'s stale "no direct Bun equivalent" reason | moot — Phase 6a adopted workspace catalogs | 2026-08-27 | root `package.json`, `workspaces.catalog` | +| `R.E8` | `crypto` is absent from ESM on every Node 18 | floor raised to `>=20.3`, with `lib`/`target` moved to ES2023 | 2026-08-26 | `scripts/verify-runtime-floor.mjs`; `packages/*/package.json` | +| `S.F1` | `SuppressedError` does not exist on the declared runtime floor | branch (b) — a runtime-guarded `suppress()` helper, not a `>=24` floor | 2026-08-26 | `packages/core/src/suppress.ts` | +| `S.F2` | Zero assertions across the whole `recovery/` module | ledgered in 4b, then settled project-wide as won't-fix at `F3` | 2026-09-02 | `F3`, retired; `docs/deferred-items.md`'s *Assertion-density rule applied project-wide* row (retired) | +| `S.F3` | Stale `wrapCancellation()` `invariant()` sentence in the 4b spec | replaced with `assertNever`'s `InvariantViolation`, matching the plan | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` design doc | +| `S.F4` | The spec never designs the `assertNever` addition the plan builds | `invariant.ts` added to the spec's File Layout | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` design doc | +| `S.F5` | `RECOV-14`'s concurrent-invocation clause claimed but untested | one design sentence plus an interleaved-`apply()` test | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` plan | +| `S.F6` | `RECOV-32`/`RECOV-33` read as silent drops | the Scope sentence now names 5a and 7a by requirement | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` design doc | +| `S.F7` | `#private` fields with no justifying comment | closed on `F5`'s reading; the cosmetic remainder is logged, not owned | 2026-08-30 | `CLAUDE.md`; `docs/deferred-items.md`'s *`#private`-vs-`private`* row, still deferred | +| `S.F8` | The chain property test drops half of what the spec promises | generator extended to seed `Failure` and assert `RECOV-4` | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` plan | +| `S.F9` | `fold()`'s three positional parameters trip the corpus prose | dissolves under the lint threshold the repository actually follows — see `V11` | 2026-09-02 | `eslint.config.js` `max-params`; `V11`, live | +| `S.F10` | `statusMappingStep` is a module-level `const` arrow | changed to a named `function` declaration with a `satisfies` check | 2026-07-28 | `docs/work/mvp/phase4/phase4b/` plan | +| `T.F1` | `PIPE-17`'s "options readable by any step" claimed while unmet | both documents record the partial deferral by name — 5a Task 1 | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` documents | +| `T.F2` | Spec lists `replace` among the pillar-collision raisers | `replace` removed, `prependAll` added, `PIPE-5`'s exemption spelled out | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` design doc | +| `T.F3` | `contextStore.clear()` in `afterEach` wipes sibling test state | both hooks deleted, with a comment recording why | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` plan | +| `T.F4` | `NFR-13`'s SPDX header absent from every 4c listing | Global Constraints bullet, every listing, and Task 6's grep | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` plan | +| `T.F5` | Claimed property tests the plan never shipped | three real `fc.assert` properties added for `PIPE-22`/`PIPE-38` | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` plan | +| `T.F6` | Pipeline errors carry symbols as fields but never render them | both messages interpolate `String(type)` | 2026-07-29 | `packages/core/src/pipeline/errors.ts` | +| `T.F7` | `StepContext.fork?: () => Next` spelled bare | spelled `?: (() => Next) \| undefined` in both documents | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` documents | +| `T.F8` | `PIPE-18`/`PIPE-19` tags swapped on the builder listing | IDs corrected; the prose names the module-level helper | 2026-07-29 | `docs/work/mvp/phase4/phase4c/` design doc | +| `T.F9` | `Cursor` never checks the signal between steps | option (a) — check in `#dispatch`, map through `abortToSdkError`; see `V15` | 2026-09-02 | `packages/core/src/pipeline/cursor.ts` | + +--- + +## Purged rows without an item ID + +The 25 rows of the deleted `## Retired rows without an item ID` table. These reserve no ID — they are +Section D's deferral rows, Section R's Phase-3-owned residuals, and `L1`'s two resolved halves, cited by +section and row title rather than by ID. The probe does not read this table. + +| Row | Subject | Resolution | Date | Evidence | +|---|---|---|---|---| +| L1 — `OBS-19` | Dropped-header verbosity policy deferred to Phase 8a | fixed — the policy now has three levels, not one; recorded as V1 | 2026-09-02 | `packages/transport-shared/src/drop-log.ts` | +| L1 — `OBS-28` | Richer HTTP-tracer vocabulary deferred to Phase 8a | closed as satisfied-by-level — a SHOULD whose no-op extension mechanism ships | 2026-09-02 | `packages/core/src/observability/tracing.ts:40-74` | +| R — residual: `BODY-34`'s shared preview-cap value | one cap value for both logging tees | shipped in 7b — one configured value threaded through both | 2026-09-02 | `packages/core/src/observability/logging-step.ts` | +| R — residual: `BODY-4`/`BODY-5` replayability consultation | resilience must read `body.replayable` | shipped in 5a and 5b | 2026-09-02 | `retry/classify.ts` `isResendable`; `redirect/decide.ts` | +| R — residual: `FileBody` | `HTTP-40`/`BODY-11`/`12`/`13`/`36` | shipped in 8a as a package, on the structural `Body.kind === 'file'` contract | 2026-09-02 | `packages/body-file/` | +| R — residual: logging tees unwired to any `Logger` | both tees need a driver | shipped in 7b — both are driven from the logging step | 2026-09-02 | `packages/core/src/observability/logging-step.ts` | +| D — Body lifecycle | `HTTP-36`–`HTTP-43` | shipped in 3b | 2026-08-26 | `packages/core/src/body/` | +| D — Lazy `TypedResponse<T>` | `HTTP-44`, `HTTP-45` | shipped in 3b | 2026-08-26 | `packages/core/src/body/typed-response.ts` | +| D — `MultipartBody` | `HTTP-51` | shipped in 3b; the non-appearance clause stays open in Section R's residuals | 2026-08-26 | `packages/core/src/body/multipart-body.ts` | +| D — 1 MiB error-body buffering cap | `HTTP-52` | shipped in 3b; `RECOV-16` reuses it unchanged | 2026-08-26 | `packages/core/src/body/http-status-error.ts` | +| D — Seam contracts | `SEAM-2`–`SEAM-30` | verified shipped across Phases 2 and 6a; §10 item 2 records the byte-stream removal | 2026-09-02 | `packages/core/src/seams/` | +| D — Adapter packages, peer-dependency dedup | `NFR-2` | nine publishable packages, core a peer of every one | 2026-09-02 | `bun run verify:seam-1` | +| D — Shrink-survival regression guard | `NFR-9` | `@dexpace/shrink-test`, private, in the CI step list | 2026-09-02 | `packages/shrink-test/` | +| D — Concurrency-model agnosticism check | `NFR-11` | 4c executed; §10 item 1 carries the full-port collapse | 2026-09-02 | `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` | +| D — `CTX-17`'s positive half | `CTX-17` | `Runtime.send()` installs and evicts its own store entry | 2026-09-02 | `packages/core/src/pipeline/runtime.ts` | +| D — Real W3C Trace Context generation | `CTX-14`, `CTX-15` | `generateTraceId`/`generateSpanId` ship with the all-zero sentinel guard | 2026-09-02 | `packages/core/src/observability/tracing.ts` | +| D — `FakeTransport` test double | — | ships and is used across the retry, redirect, auth and observability suites | 2026-09-02 | `packages/core/src/testing/fake-transport.ts` | +| D — Self-identifying version metadata | `NFR-15` | shipped; the missing barrel export is `K1`, not this row | 2026-09-02 | `packages/core/src/config/build-info.ts`, `client-identity-step.ts` | +| D — `NFR-8` re-confirmed as a documented non-applicability | `NFR-8` | recorded as a deviation — no reflection-driven discovery surface exists | 2026-09-02 | deviations.md §10 | +| D — Redirect structured logging | `REDIR-28`, `REDIR-15`, `XCUT-17`(d) | all three families emit; see `G2` | 2026-09-02 | `packages/core/src/redirect/redirect-step.ts` | +| D — Redirect's loop-detected and malformed-Location events | `REDIR-28` | both emit behind the stop-reason discriminant; see `G3` | 2026-09-02 | `packages/core/src/redirect/decide.ts` | +| D — The cross-origin marker's consumption side | `REDIR-11`(b/c), `XCUT-17`(b), `AUTH-29` | `planOutbound` reads the marker and clears it; see `G7` | 2026-09-02 | `packages/core/src/auth/auth-step.ts` | +| D — Auth re-runs per redirect hop | `PIPE-2` | `standardResilience()` seats `authStep()` inside the redirect pillar | 2026-09-02 | `packages/core/src/auth/preset.ts` | +| D — Public-barrel promotion of `redirectStep`/`withRedirect` | — | the half 5c left is closed; see `U7` | 2026-09-02 | `packages/core/etc/core.api.md` | +| D — Re-confirm the redirect predicate's scope | `REDIR-20` | confirmed and recorded as a deviations row; see `G4` | 2026-09-02 | deviations.md, "Deviations recorded outside a phase" | + +--- + +## Purged deferral rows + +The 67 rows of `docs/deferred-items.md`'s deleted `## Delivered and retired` table: every deferral since +delivered, closed, or settled as won't-fix, with the phase and commit that discharged it. Two of them +(`SEAM-5`–`SEAM-10` and `SEAM-18`) were never deferrals at all — permanent simplifications kept there because +they are easy to mistake for one. Rows are keyed by requirement ID or topic, never by line number. + +| Row key (requirement ID or topic) | Origin phase | Delivered / closed by (phase, commit) | Evidence (`file:line`) | Date retired | +|---|---|---|---|---| +| `NFR-2` — each optional capability a separately installable unit | Phase 0 | Phase 6a (`743f316`) + Phase 8a (`a0d734d`) | `packages/codec-json/package.json` (`dependencies: {}`); `packages/transport-fetch/package.json:22-24` + `scripts/verify-seam-1.mjs:32-35` — allow-list-qualified: the gate asserts zero *unlisted* runtime deps, not zero deps | 2026-09-02 | +| `NFR-9` — shrink-survival regression guard | Phase 0 | Phase 9 (`d8217af`) | `packages/shrink-test/src/{bundle,fixture-app,run-shrink-guard}.ts` — runs via `bun run test` (`package.json:54`), **not** the `build` script and not a named CI step | 2026-09-02 | +| `NFR-11` — concurrency-model agnosticism | Phase 0 | Phase 4c (`63ed1b7`) | `packages/core/src/pipeline/step.ts` — `Step`/`Next`/`Runtime` are `Promise`-only; no framework async type leaks | 2026-09-02 | +| `NFR-12` — reproducible byte-identical builds | Phase 0 | closed 2026-08-29 | `scripts/verify-reproducible-build.mjs`, a blocking CI step over `dist/` and every `npm pack` tarball | 2026-09-02 | +| `NFR-13` — SPDX header per source file | Phase 0 | Phase 1 plan (2026-07-28) | Phase 1's Global Constraints; enforcement stays review-level by the spec's own wording, with no mechanical gate | 2026-09-02 | +| `NFR-14` — one source of truth for tool versions | Phase 0 | Phase 6a, closed 2026-08-27 | root `package.json` `workspaces.catalog`; members reference `"catalog:"` | 2026-09-02 | +| `NFR-15` — real `User-Agent`, never a placeholder | Phase 0 | Phase 7a (`bd37a08`) + Phase 8a (`a0d734d`) | `packages/core/src/config/client-identity-step.ts:109`; conformance at `packages/transport-conformance/src/run-suite.ts:517-525` | 2026-09-02 | +| `NFR-8` — shrinker keep/retain configuration | Phase 0 | Phase 10, closed 2026-07-28 | not applicable by design — this port has no reflection-driven discovery surface to keep-configure (§10 Item 10) | 2026-09-02 | +| Peer-dependency dedup for `@dexpace/core` (dual-package hazard) | Phase 0 | Phase 6a, closed 2026-08-27 | `scripts/verify-seam-1.mjs` asserts the peer + `peerDependenciesMeta` for every non-core package; `packages/codec-json/src/cross-package.test.ts` proves the consequence | 2026-09-02 | +| `NFR-10`/`NFR-17` — CI against the declared minimum Node | Phase 0 | Phase 2 (`8e55792`), replaced wholesale in Phase 3 (`e3ba885`) | `.github/workflows/ci.yml:99-130` — job `node-conformance`, matrix `['20.3.0','lts/*']` at `:110`, `bun run test:node` at `:129`. `scripts/verify-node-floor.mjs` shipped in Phase 2 and was **deleted** in `e3ba885` | 2026-09-02 | +| `MultipartBody` model (HTTP-3, HTTP-51, BODY-2) | Phase 1 | Phase 3b (`e3ba885`) | `packages/core/src/body/multipart-body.ts` | 2026-09-02 | +| `Request`/`Response` real body type | Phase 1 | Phase 3b (`e3ba885`) | `packages/core/src/http/request.ts:118`; `packages/core/src/http/response.ts:126` (`BODY-14`) | 2026-09-02 | +| `Logger`/`LogEvent` seam | Phase 2 | Phase 7b (`bd37a08`) | `packages/core/src/observability/logger.ts:274` + the global slot; bridges `@dexpace/logging-pino`, `@dexpace/logging-debug` | 2026-09-02 | +| `FakeTransport` test double | Phase 2 | Phase 5a (`cba4721`) | `packages/core/src/testing/fake-transport.ts` (`@internal`), with `countingResponse()` | 2026-09-02 | +| Phase 4 split into 4a / 4b / 4c | Phase 4 brainstorm | executed, Phase 4 (`63ed1b7`) | ~76 combined normative IDs; 4a first, then 4b and 4c | 2026-09-02 | +| Phase 5 split into 5a / 5b / 5c | Phase 5 brainstorm | executed, Phase 5 (`cba4721`) | 111 combined IDs; retry → redirect → auth, an order forced by coupling not size | 2026-09-02 | +| Phase 6 split into 6a / 6b / 6c | Phase 6 brainstorm (2026-07-28) | executed, Phase 6 (`743f316`) | 107 combined IDs; no segment depends on another — `SSE-37` and §12's preamble make the cross-segment surface empty by mandate | 2026-09-02 | +| Collapsed-requirement disposition tables for Phase 6 | Phase 6 brainstorm | Phase 6a/6b/6c (`743f316`) | 6a design `:339`; 6c design `:352` (`PAGE-25`–`PAGE-28` at `:360-363`); 6b design | 2026-09-02 | +| 3a's `readUtf8Line()` unusable for SSE (`IO-14` vs `SSE-2`) | Phase 6b | Phase 6b, closed 2026-08-27 | `packages/core/src/sse/line-reader.ts` — 6b owns its own reader rather than reshaping a frozen 3a surface | 2026-09-02 | +| `sdk-design-nodejs/07` §7.1 closes the page after yielding; `PAGE-11` requires before | Phase 6 brainstorm | Phase 6c, closed 2026-08-27 | `PAGE-11` governs — copy items, close, then yield; both the design and the knowledge corpus amended | 2026-09-02 | +| `PAGE-5`'s "synchronously inside parse" literal reading | Phase 6 brainstorm | Phase 6c, closed 2026-08-27 | Node has no synchronous body read, so `parse` returns a promise; every part of the requirement's intent survives | 2026-09-02 | +| `SSE-41` — reactive SSE adapter | Phase 6 brainstorm | Phase 8b (`a0d734d`) | `packages/rx/src/sse.ts:13` and `:39` | 2026-09-02 | +| Appendix C `RECOV-17`–`RECOV-34` reconciliation (18 rows) | Phase 4 sizing review | Phase 5a (`cba4721`) | row-by-row mapping table in the Phase 5a design; 16 collapse onto their §9 twin, `RECOV-32`/`RECOV-33` are genuinely new | 2026-09-02 | +| Real W3C Trace Context generation | Phase 4a | Phase 7b (`bd37a08`) | `packages/core/src/config/identifiers.ts:25-30`; `packages/core/src/observability/tracing.ts:206-223` | 2026-09-02 | +| `contextsEqual()` value-equality utility for `ExecutionContext` | Phase 4a | won't-fix, 2026-09-02 | 4b and 4c both shipped in `63ed1b7` without needing it, so the row's own trigger can never fire | 2026-09-02 | +| `PIPE-35` — FLATTEN-vs-NEST pipeline seeding | Phase 4c | Phase 5c (`cba4721`) | `packages/core/src/pipeline/builder.ts:238` — `seedFrom(runtime, 'flatten' \| 'nest')`, non-defaulted | 2026-09-02 | +| `PIPE-2` / `PIPE-40` conformance clauses | Phase 4c | Phase 5b + Phase 5c (`cba4721`) | 5b's two-hop `FakeTransport` test; 5c's per-hop auth re-run closing `PIPE-2`'s remaining half with `AUTH-29` | 2026-09-02 | +| `PIPE-24`/`PIPE-39` — the standard-resilience preset | Phase 4c | Phase 5c (`cba4721`) | `packages/core/src/auth/preset.ts:103` — `standardResilience()` | 2026-09-02 | +| `PIPE-36` — a shipped pillar family locks its stage | Phase 4c | Phase 5a (`cba4721`) | satisfied structurally: `retryStep()` returns a `StepDescriptor` with `stage: 'RETRY'` baked in, so there is nothing to relocate | 2026-09-02 | +| Public-barrel promotion of the pillar-step authoring surface | Phase 4c, re-confirmed in Phase 5a | Phase 5c (`cba4721`) | `packages/core/etc/core.api.md` — `Stage`/`STAGE_ORDER`/`PILLAR_STAGES`/`StepDescriptor`/`PipelineBuilder`/`Runtime`/the three pillar factories/`standardResilience` | 2026-09-02 | +| `RECOV-33` — client-identity header step | Phase 5a brainstorm | Phase 7a (`bd37a08`) | `packages/core/src/config/client-identity-step.ts:109` | 2026-09-02 | +| `StepContext.signal` and `StepContext.options` (`PIPE-17`) | Phase 5a brainstorm (`signal`); 2026-07-28 plans review (`options`) | Phase 5a Task 1 (`cba4721`) | `packages/core/src/pipeline/step.ts:58` and `:64` | 2026-09-02 | +| `RequestOptionsBuilder.maxRetries` accepts `Infinity`/`NaN`/fractions | Phase 5a code review (2026-08-26) | Phase 5's merge `cba4721` (2026-08-27) | `packages/core/src/http/request-options.ts:178`; `.changeset/2026-08-26-max-retries-range-check.md`. **Phase 10 was never its owner** and this row named it as such until 2026-08-30 | 2026-09-02 | +| The two structured retry log events + `RETRY-40`'s diagnostic half | Phase 5a execution (2026-08-26) | Phase 7b Task 9 (`bd37a08`) | `packages/core/src/retry/engine.ts:323` and `:396`, plus a third the plan never named, `http.retry.delayOverrideFailed` | 2026-09-02 | +| Phase 7a Tasks 1-3 executed early as 5a's prerequisite | Phase 5a execution (2026-08-26) | executed inside Phase 5a's window (`cba4721`) | the row's prescription ("7a's plan should mark Tasks 1-3 done") was **struck 2026-09-02**: `docs/work/` is a dated record and is never retro-edited, so 7a's plan still correctly reads `- Create:` at `:122`, `:265`, `:429` | 2026-09-02 | +| `SEAM-30` — orphan-response cleanup on the completion race | Phase 2 | Phase 8a (`a0d734d`) | `packages/transport-fetch/src/fetch-transport.ts:241`; `packages/transport-undici/src/undici-transport.ts:454` | 2026-09-02 | +| Byte-stream provider (`ByteQueue`, `BufferedSource`/`Sink`, `TeeSink`) | Discussed in Phase 2 (`sdk-design/03` §3.1), built in | Phase 3a (`e3ba885`) | `packages/core/src/io/{byte-queue,buffered-source,buffered-sink,tee-sink}.ts`, behind an internal barrel | 2026-09-02 | +| Every buffering cap — `BODY-19`, `BODY-30`/`HTTP-52`, `BODY-34` | Phase 3a | Phase 3b (`e3ba885`) for two; `BODY-34` in Phase 7b (`bd37a08`) | `packages/core/src/body/request-body-logging.ts:80-88`; `http-status-error.ts:18`; `observability/logging-step.ts:64-65`. **`BODY-34` is NOT threaded through `toHttpError`** — the code says so at `http-status-error.ts:16-17`, and this row claimed otherwise until 2026-09-02 | 2026-09-02 | +| Promotion of any §5 type into the published barrel | Phase 3a | Phase 3b (`e3ba885`) — never promoted; error leaves promoted in Phase 8a (`a0d734d`) | `packages/core/src/index.ts:35` exports `IoError`/`TransportFailureError` (`core.api.md:503`, `:1301`); no provider type is promoted. `packages/core/src/io/index.ts:5-7`'s "NOTHING here is re-exported" comment is stale and is a source edit nobody has made | 2026-09-02 | +| `MAX_BYTE_ARRAY_LENGTH` constant value (`IO-9`) | Phase 3a | Phase 3a plan time (`e3ba885`) | `packages/core/src/io/limits.ts:28` — `2 ** 31 - 1`, with the `AllocationLimitError` backstop at `:39-40` | 2026-09-02 | +| `Symbol.asyncDispose` on §5 resources | Phase 3a | closed 2026-08-30 | runtime-guarded, optionally-typed install in 6b/6c delegating to `close()`. Promotion to `implements AsyncDisposable` is **rejected, not pending** — `open-items.md` Section D. This row claimed a live residue until 2026-09-01; there is none | 2026-09-02 | +| `SEAM-5`–`SEAM-10` — discovery/registration/conflict-resolution machinery | Phase 2 | **Never** — permanent simplification, closed 2026-07-28 | §10 Item 2. Node has no pluggable byte-stream factory or fragmented async ecosystem to discover across. Never a deferral; kept here because it is easy to mistake for one | 2026-09-02 | +| Concrete `Serde` implementation (`@dexpace/codec-json`) | Phase 2 | Phase 6a, closed 2026-08-27 | `packages/codec-json/etc/codec-json.api.md` — `jsonSerde()`, the Tristate replacer, the decode combinators | 2026-09-02 | +| Concrete `Transport` implementations | Phase 2 | Phase 8a (`a0d734d`) | `packages/transport-fetch/src/fetch-transport.ts`; `packages/transport-undici/src/undici-transport.ts` | 2026-09-02 | +| `SEAM-21` — explicit runtime type token for deserialization | Phase 2 | Phase 6a, closed 2026-08-27 | every decode entry point takes a caller-supplied `Schema<T>`; `Serde` dropped its type parameter and the reshaped seam is public, forced by the package split | 2026-09-02 | +| `SEAM-14` — close *behavior* | Phase 2 | Phase 8a (`a0d734d`) | `fetch-transport.ts:292-300` (sanctioned no-op); `undici-transport.ts:511-532` (`destroy()`, reverse order, idempotent, memoized); test `undici-transport.test.ts:234` | 2026-09-02 | +| `SEAM-12` — concurrent-call conformance test | Phase 2 | Phase 8a (`a0d734d`) | `packages/transport-conformance/src/run-suite.ts:432` (group) and `:440` (many concurrent sends) | 2026-09-02 | +| `SEAM-18` — sync↔async bridges | Phase 2 | **Never** — permanent simplification, closed 2026-07-28 | §10 Item 2. Its one non-bridge clause survives as a `Transport.send()` obligation. Never a deferral | 2026-09-02 | +| `HTTP-18`/`HTTP-48`/`HTTP-50` — outbound header strictness vs ETag obs-text | Phase 1 | Phase 10, closed 2026-07-28 | strict outbound path kept; `HTTP-18`'s MUST outranks `HTTP-48`'s SHOULD (§10 Item 15) | 2026-09-02 | +| `FileBody` (`BODY-11`/`12`/`13`/`36`) | Phase 3b brainstorm | Phase 8a (`a0d734d`) | `@dexpace/body-file`'s `fileBody()`; core carries a type-only `FileBodyDescriptor` and `body.kind === 'file'` structural narrowing, never a cross-package `instanceof` | 2026-09-02 | +| `redirect/cross-origin.ts` — the `REDIR-11`/`AUTH-29` shared signal | Phase 5b brainstorm | Phase 5b (`cba4721`) | `packages/core/src/redirect/cross-origin.ts`. Kept as a standing caution: two solo brainstorms sharing a cross-phase contract drifted twice, on the marker's *shape* and again on its *scope* | 2026-09-02 | +| `standardResilience()` gains a `LOGGING` pillar step | Phase 5c brainstorm | Phase 7b (`bd37a08`) | `packages/core/src/auth/preset.ts:111` — `.append(loggingStep(options.logging))`, inert at `granularity: 'none'` | 2026-09-02 | +| `DigestChallengeUnsupportedError` | Phase 5c brainstorm | **cut before shipping, in Phase 5c** | absent from `packages/` entirely. 5c checklist `:229`; `open-items.md` G11. **This row asserted the opposite — "kept, permanently" — until 2026-09-02**, sending a reader after a symbol that does not exist | 2026-09-02 | +| Basic/Digest never stamp preemptively (an interpretation) | Phase 5c brainstorm | Phase 10, closed 2026-07-28 | confirmed correct as designed; the spec's asymmetry reads as deliberate (§10 Item 12) | 2026-09-02 | +| Redirect predicate's scope over safety mechanics (`REDIR-20`) | Phase 5b brainstorm | Phase 10, closed 2026-07-28 | confirmed correct as designed — safety mechanics are governed by `XCUT-17`'s universal framing (§10 Item 12) | 2026-09-02 | +| Redirect structured logging — `REDIR-28`'s four event families | Phase 5b brainstorm | Phase 7b (`bd37a08`); the reason discriminant and the last two events 2026-09-02 (uncommitted at time of writing) | `packages/core/src/redirect/decide.ts:44-58` (`RedirectStopReason`) and `:72`; `redirect-step.ts:126` hop, `:70` loop, `:118` downgrade, `:81` malformed-`Location` (raw by `REDIR-28`'s own carve-out). Closed jointly with `open-items.md` G3 | 2026-09-02 | +| 5a's `RetryConfig.clock`/`random` retyped against 7a's `Clock` | Phase 7a brainstorm | Phase 7a (`bd37a08`) | `packages/core/src/retry/retry-step.ts:3`, `:35`, `:42`, `:124`; `engine.ts:46`, `:48` | 2026-09-02 | +| 5a's private RFC 1123 parser re-sourced from `config/http-date.ts` | Phase 7a brainstorm | Phase 7a (`bd37a08`) | `packages/core/src/retry/pacing.ts:10` — one parser in the codebase, not two | 2026-09-02 | +| 5a's private `RETRYABLE_STATUSES` re-sourced from `config/retryable.ts` | Phase 7a brainstorm | Phase 7a (`bd37a08`) | `packages/core/src/retry/classify.ts:13` re-exports it, so `RETRY-1` and `CFG-35` cannot drift | 2026-09-02 | +| Whether `clientIdentityStep` joins `standardResilience()` | Phase 7a brainstorm | Phase 10, closed 2026-07-28 | stays out, permanently — no requirement mandates it; a caller installs it explicitly | 2026-09-02 | +| Retry/redirect structured-logging event names and fields | Phase 7b brainstorm | Phase 7b plan time (`bd37a08`) | `retry/engine.ts:18`, `:323`, `:396`; `redirect/redirect-step.ts:50`, `:73`, `:81` — all six under one `http.` prefix | 2026-09-02 | +| Whether the preset accepts a `tracerFactory`/`meter` pass-through | Phase 7b brainstorm | Phase 9 (`d8217af`) | `tests/conformance/xcut/fixtures/composed-pipeline.ts` configures all three through the existing `logging` option — no friction found, closed rather than re-deferred | 2026-09-02 | +| Phase 8 split into 8a / 8b | Phase 8 brainstorm (2026-07-28) | executed, Phase 8 (`a0d734d`) | 52 nominal combined IDs, but §17 is paid twice (two full `Transport` implementations) and nine log rows landed there | 2026-09-02 | +| Assertion-density rule applied project-wide | Phase 4b validation review F2 (2026-07-28) | won't-fix, 2026-09-02 | 43 non-test modules call `invariant()`; `http/`, `seams/`, `generated/` and `recovery/` are at zero, which is the correct shape. `open-items.md` F3/H6; `deviations.md`, "Deviations recorded outside a phase". **This row said thirteen modules and named `recovery/` as the lone holdout** until 2026-09-02 | 2026-09-02 | +| `CONSTANT_CASE` vs `lowerCamelCase` for module-level immutable collections | Phase 4c validation review (2026-07-29) | resolved by practice, 2026-09-02 | 24 module-level collections swept across all eleven packages, every one `CONSTANT_CASE`, zero `lowerCamelCase`. Declarations at `packages/core/src/pipeline/stage.ts:38`, `:58`. **This row's trigger had already fired six times** and it cited the import line, not the declaration | 2026-09-02 | +| `BODY-34`'s single shared preview-cap **value** | Phase 3b checklist | Phase 7b (`bd37a08`) | `packages/core/src/observability/logging-step.ts:64-65`, resolved once at `:465-466`, threaded to both tees at `:327` and `:379`. Recovered by the 2026-09-02 audit; had never reached this register | 2026-09-02 | +| Wiring either logging tee to a real `Logger` | Phase 3b checklist | Phase 7b (`bd37a08`) | `packages/core/src/observability/logging-step.ts:492` — `settings.logger ?? getGlobalLogger()`. Recovered by the 2026-09-02 audit; had never reached this register | 2026-09-02 | + +--- + +## Live deferrals — the five rows that did NOT close + +**Every other table in this note records something that ended.** This one does not. These five rows were +still live when `docs/deferred-items.md` was deleted later on 2026-09-04: nothing below has been built, +nothing below was withdrawn, and each one still carries a trigger a reader could observe firing. They are +reproduced here because the register that held them is gone and this note is now their archive of record — +the only place in the repository where they exist. Read them as outstanding work, not as history. + +The sixth survivor, the `NFR-16` publish-provenance row, is not in this table. It was the one row with live, +actionable content of its own, and it became [`docs/first-release.md`](../../first-release.md) on the same +day rather than being archived here. + +**This still does not make the note a register.** Nothing is appended to this table as work proceeds. When +one of these five triggers fires the work is done and the row is simply no longer true; a *new* deferral +goes to `docs/work/mvp/2026-09-04-open-items-dissolution.md` as an open item carrying its trigger, per the preamble above. Rows are keyed +by requirement ID or topic, never by line number — the convention the register itself used. + +Reproduced verbatim from `docs/deferred-items.md`'s `## Still deferred` table as it stood at deletion, +including each row's own dated mid-sentence corrections. The `file:line` evidence inside them is unmodified +and carries the same drift caveat as every other table here. **One mechanical exception:** four relative +Markdown link targets inside these rows were written from the `docs/` root and are re-anchored to this +file's directory, so they still resolve. No word of the rows changed. + +| Item | Originated in | Target phase | Note | +|---|---|---|---| +| `RETRY-29` — opt-in server-driven retry-classification override header | Phase 5a brainstorm | **UNSCHEDULED — trigger: a consumer needs server-driven backoff beyond `Retry-After`** | `MAY` — the spec's own level, at `docs/product-spec/09-retry-and-resilience.md:35` and appendix C: "an opt-in server-driven override **MAY** let a response header force or suppress the retry classification … The override MUST flip only classification and MUST remain subject to the attempt cap and the re-send-safety gate." **What is built.** Server *pacing* headers are honored in full: `packages/core/src/retry/pacing.ts:62-90` reads `Retry-After` (numeric form, then HTTP-date), then `retry-after-ms`, then `x-ms-retry-after-ms`, then `X-RateLimit-Reset`, first parseable value winning (`RETRY-21`/`RECOV-24`), with a strict decimal grammar screening the value before any float parse (`:16-21`, `RETRY-19`), a one-year clamp (`:14`, `RETRY-18`/`RECOV-26`), and no jitter applied to a literal `Retry-After` (`:56`, `RETRY-20`). **What is not built.** Nothing lets a response header change the retry *classification*: `packages/core/src/retry/classify.ts` decides retryability from its allow-list and the shared `RETRYABLE_STATUSES` alone (shared meaning re-exported out of `config/retryable.ts` at `packages/core/src/retry/classify.ts:13`, so `RETRY-1` and `CFG-35` cannot drift). The two surfaces answer different questions — pacing answers "how long," `RETRY-29` answers "whether" — so honoring `Retry-After` is not partial credit toward this row. Not scheduled because it widens the classifier's input surface to server-controlled values, a trust decision deserving its own deliberation rather than a default. No caller identified. **Trigger added 2026-09-02**; the register table above requires an unscheduled deferral to carry one, and this row had none | +| A real `@opentelemetry/sdk-metrics`-backed `Meter` adapter package | Phase 7b brainstorm | **UNSCHEDULED — trigger: the first consumer that needs a metrics backend, or a post-mvp delivery** | `OBS-31` only requires the no-op default and that core not depend on a metrics runtime, and both hold: the default is `NOOP_METER` (`packages/core/src/observability/metrics.ts:56`, over the frozen `NOOP_COUNTER`/`NOOP_HISTOGRAM` at `:40,45`), `@dexpace/core` declares `dependencies: {}`, and `verify:seam-1` enforces it. Re-confirmed 2026-09-02: eleven packages exist and none is a metrics backend — unlike tracing, which has a duck-typed zero-adapter path (a caller-supplied `tracerFactory`, over the trace- and span-id generators Phase 7b shipped at `packages/core/src/observability/tracing.ts:206-223`) and so needs no package at all. **Trigger added 2026-09-02.** The register table above requires an unscheduled row to carry a trigger; this row had none, having read only "Not scheduled" since Phase 7b | +| RFC 7616 §4 `username*` (RFC 5987) extended notation for a non-ASCII Digest username | Phase 5c checklist | **UNSCHEDULED — trigger: a Digest server observed requiring `username*` or `userhash`** | **Recovered by the 2026-08-31 register audit; never reached this log.** `digestHandler()` rejects a non-header-safe username at construction today — a loud refusal, not a silent mangle, so the failure mode is safe. Implementing `username*` would let such a username be sent correctly rather than refused, and is the standard's own answer. Not scheduled: no server has been observed requiring it, and the refusal is a correct (if narrow) implementation until one is. **Trigger added 2026-09-02** per the register table above; the row had read only "Unscoped". Source: [5c's checklist](./phase5/phase5c/2026-07-26-phase5c-auth-checklist.md):227 — corrected 2026-09-02 from `:220`, which is the `## Deferred Items` heading, not the row | +| A caller-supplied `ChallengeHandler` list on `AuthStepSettings` | Phase 5c checklist | **UNSCHEDULED — trigger: a second auth scheme needing a challenge handler beyond Digest** | **Recovered by the 2026-08-31 register audit; never reached this log.** `handlers` was removed at review: it forced three types onto the public barrel and could not compose with the built-in handlers, which stay internal. If a caller ever needs to ADD a handler rather than replace the whole reaction, the shape to ship is an append-semantics field plus public `basicHandler`/`digestHandler` factories — not the replace-semantics field that was cut. **Trigger added 2026-09-02** per the register table above; the row had read only "Unscoped". Source: [5c's checklist](./phase5/phase5c/2026-07-26-phase5c-auth-checklist.md):230 — corrected 2026-09-02 from `:220`, which is the `## Deferred Items` heading, not the row | +| Zero-copy file dispatch for `@dexpace/body-file` — both a read-only memory-mapped `fileBody()` view (`BODY-36`, MAY) **and** a real `sendfile(2)`-equivalent path if Node's HTTP stack ever exposes one (`TRANSPORT-28`, SHOULD) | Phase 8a brainstorm (both) | **UNSCHEDULED — trigger: Node exposes a `sendfile(2)`/mmap path for the HTTP stack** | **Two 8a rows merged here 2026-09-02, because they share one trigger.** The 2026-08-31 register audit recovered the `BODY-36` half and missed the `sendfile(2)` half, so a standing revisit trigger lived only in a phase document — the exact failure the Maintenance rule above forbids. `BODY-36` is a MAY for local hashing/signing without heap copying; no caller identified in this roadmap's scope, same "don't build speculatively" discipline as `FakeTransport`'s original deferral. `TRANSPORT-28`'s kernel-transfer SHOULD has no Node analogue today — neither `fetch` nor `undici` exposes a `sendfile`-shaped API for outbound bodies — which 8a's design **confirmed** rather than merely flagged, recording it as a `PAGE-29`-shaped collapse in its own Deviation Ledger and again in [`deviations.md`](../../deviations.md), the transport-impossibility list ("Zero-copy `sendfile(2)` (`TRANSPORT-28`, SHOULD)"). Revisit `transport-undici`'s file-body dispatch if a future Node/undici release adds a genuine kernel-transfer API, not before. Sources: [8a's design](./phase8/phase8a/2026-07-28-phase8a-transport-design.md):585 (the `sendfile(2)` revisit trigger) and `:586` (the memory-mapped view) — the memory-mapped citation is corrected 2026-09-02 from `:581`, which is the `## Deferred Items` heading. `@dexpace/body-file` shipped in 8a without either path and the collapse is settled; what is *not* settled, and is why this row stays open, is the revisit trigger | + +--- + +## Retired review sections + +Three of the four reviews the roadmap once carried in its own `## Open Findings` headings were relocated +into `docs/work/mvp/2026-09-04-open-items-dissolution.md` on 2026-08-31 as Sections Q, S and T: a validation pass over Phase 3b's design and +plan **before** either was executed, and two more over Phase 4b's and Phase 4c's. Every row of all three +reached a resolved disposition on 2026-09-02, and on 2026-09-04 the prose that framed them followed the rows +out of the register — a validation pass over an unexecuted phase is a dated record of what was true that +week, not an item anyone can still act on. + +It is kept here because the prose is worth more than the rows were. Each section names the blockers it found +and the constraint each one generalizes into, the corpus conflicts it surfaced without ruling on, and a long +*applied without needing a decision* inventory — corrections made to the phase documents themselves, which +therefore left no trace anywhere else. That inventory is the part a later phase would otherwise re-derive. +Section R, the fourth relocated review, stays in the register: its `E2`–`E4` are open. + +Reproduced verbatim, headings demoted one level. The cross-references inside them stand as written: `V11` is +a live register item, and `V15` and the row IDs resolve against the **Purged item IDs** table above. + + +### Section Q — Phase 3b validation review (2026-07-28) + +> **Relocated.** A validation pass over Phase 3b's design and plan **before** either was executed. Relocated verbatim on +2026-08-31 from the roadmap's `## Open Findings — Phase 3b Validation Review (2026-07-28)` section. Its rows +are labelled `D1`, `D2` — the review's own numbering, not this register's item IDs. + +A validation pass over `specs/2026-07-25-phase3b-body-lifecycle-design.md` and +`plans/2026-07-25-phase3b-body-lifecycle.md` (`docs/validation-prompts/phase3b-body-lifecycle-validation-prompt.md`) +returned **BLOCKED** on two runtime defects and a cluster of overclaimed disposition rows. **All findings except +D1 and D2 below are applied** to both documents. Recorded here rather than in `docs/deferred-items.md` because +these are review findings against an unexecuted phase, not deferrals of work. + +The two blockers, both now fixed, are worth naming since they generalize: (1) `ReadableStream.cancel()` rejects +with `TypeError` on a locked stream and reading to `{done: true}` does **not** release the reader's lock, so +`Response.bytes()`, `toHttpError()` and the response-logging wrapper each had a `finally`-scoped close that +replaced a successful read with a `TypeError` — a `reader.releaseLock()`-before-cancel constraint now sits in the +plan's Global Constraints, and **every later phase that takes a reader and later closes the stream inherits it**; +(2) `HTTP-39`/`BODY-10`'s exact-length copy was dispositioned as "reuses Phase 3a's `writeAll`" while the plan's +own global constraint forbids importing `BufferedSink`, leaving a declared `contentLength` unverified and a short +stream sending a truncated body silently. + +All rows retired 2026-09-02. Row IDs `Q.D1` and `Q.D2` remain reserved. + +**Applied without needing a decision** (recorded so the reasoning survives): `BODY-34`'s "one shared cap" +contradiction resolved in the plan's favour — the shared preview cap covers the two logging tees, and +`toHttpError`'s 1 MiB cap is separate because `HTTP-52` *fixes* its value and a spec-fixed value cannot be the +configurable one; `BODY-26`/`BODY-29` built (`LoggedResponseBody` gained a non-draining `error()` and a +regime-dependent `contentLength`); `BODY-25` ledgered as structurally inapplicable — `ReadableStreamDefaultReader` +takes no requested count, so "zero bytes for a positive count" has no analog; `BODY-32`'s negative-cap rejection +added to both tees, which previously accepted a negative cap and silently mirrored nothing; `HTTP-3`'s +`MultipartBodyBuilder` added (`HTTP-3` names "the multipart body" explicitly and Phase 1 could not satisfy it); +`HTTP-2` honored by exporting the concrete body classes from the public barrel as **types only**; the `@internal` +tags removed from the three errors Task 13 promotes, which would have made `api-extractor` either fail or +silently omit them; `withResponseLogging` decomposed under the 70-line cap and made pull-driven, since its +`start()`-loop tail stream eagerly materialized the whole remainder of exactly the oversized bodies the cap +exists to keep off the heap. + + +### Section S — Phase 4b validation review (2026-07-28) + +> **Relocated.** A validation pass over Phase 4b's design and plan before execution. Relocated verbatim on 2026-08-31 from +the roadmap's `## Open Findings — Phase 4b Validation Review (2026-07-28)` section. + +**Its rows are `F1`–`F10`, the review's own numbering.** Section F above numbers *its* items `F1`–`F9`, and +Section T below numbers a different review's rows `F1`–`F9` again. Three `F` namespaces, no overlap in +meaning. A citation must name the section — "Section S's F2", never a bare "F2". Renumbering was rejected: +the roadmap's own status notes cite "4b's F2/F7" by these numbers, and a dated record that changes its row +IDs stops matching the documents that quote it. + +A validation pass over `specs/2026-07-25-phase4b-recovery-chain-design.md` and +`plans/2026-07-25-phase4b-recovery-chain.md` (`docs/validation-prompts/phase4b-recovery-chain-validation-prompt.md`) +returned **BLOCKED**. The `RECOV-1`–`RECOV-16` mapping itself is sound and every cross-phase reference 4b consumes +checks out against the earlier phase plans — `toHttpError(): Promise<HttpStatusError | null>` (3b), `RequestOptions.EMPTY` +(Phase 1), `Transport.send(request, options?, signal?)` + `CancellationError` (Phase 2), and `Response.close()` latching +`#closed` *before* awaiting `body.cancel()` so it propagates a close rejection exactly once (3b). Nothing below is a +defect in that mapping. Recorded here rather than in `docs/deferred-items.md` because these are review findings against +an unexecuted phase, not deferrals of work. + +**All ten rows are retired 2026-09-02** — `F1` and `F2` closed, `F3`–`F10` applied to the 4b documents. +Row IDs `S.F1`–`S.F10` remain reserved. `F1`'s resolution is the runtime-guarded `suppress()` helper +(`packages/core/src/suppress.ts`); read it before designing against `SuppressedError` anywhere, because +`esnext.disposable` in `lib` supplies `Symbol.asyncDispose`'s *type* and not `SuppressedError`'s *runtime*. + +**Corpus conflict surfaced, not a finding.** `function-design.md:22-23` requires an options object at 3+ parameters +while `function-design.md:40-41` sets `max-params: ['error', 3]`, which errors only at four — the prose is one +parameter stricter than its own stated enforcement. F9 is filed against the prose; if the lint threshold is the +authority, F9 dissolves. Worth settling in the corpus rather than per-phase. +→ **numbered 2026-09-02 as V11**, and given a disposition. "Worth settling" with no owner and no +trigger is how a finding sits unsettled for five weeks; V11 states which of the two the repository +actually follows and what would change it. + +A second conflict the 4b documents met and resolved correctly, recorded so a later reader does not re-litigate it: +`resource-management.md:4-5,72` mandates `using`/`await using` and documents that native disposal builds a +`SuppressedError` with the *disposal* failure primary, while `RECOV-12` requires the opposite priority. 4b picks +`RECOV-12` and argues it at SPEC:107-113 / PLAN:55-59. Correct call, already justified in-document. + + +### Section T — Phase 4c validation review (2026-07-29) + +> **Relocated.** A validation pass over Phase 4c's design and plan before execution. Relocated verbatim on 2026-08-31 from +the roadmap's `## Open Findings — Phase 4c Validation Review (2026-07-29)` section. **Its rows are `F1`–`F9`, +the review's own numbering** — see the namespace note on Section S. + +A validation pass over `specs/2026-07-25-phase4c-stage-pipeline-design.md` and +`plans/2026-07-25-phase4c-stage-pipeline.md` +(`docs/validation-prompts/phase4c-stage-pipeline-validation-prompt.md`) returned **NEEDS WORK — no blockers.** +The `PIPE-1`–`PIPE-40` mapping is sound and every cross-phase reference 4c consumes checks out against the earlier +phase plans: `Transport.send(request, options?, signal?)` + `close()` (Phase 2), `DexpaceError` as the taxonomy +root under `http/errors.ts` (Phase 2's retrofit), `RequestOptions.EMPTY` (Phase 1), `Status.of`/`Protocol.HTTP_1_1` +(Phase 1), and 4a's `createRequestContext(request, init?)`, `promoteToRequest`/`promoteToExchange`, +`ContextStore.install/get/close/clear/size` with the `kind`/`key`/`request`/`instrumentation`/`operationName` +context shape. Nothing below is a defect in that mapping. + +**All nine rows are retired 2026-09-02** — `F1`–`F8` applied to the 4c documents, `F9` fixed to its own +option (a); see `V15`. Row IDs `T.F1`–`T.F9` remain reserved. + +**Not findings, recorded so they are not re-raised.** Assertion density (`assertions.md:6-7`) is already open +project-wide as 4b's F2 — 4c is the phase that *satisfies* it, not one that violates it. `STAGE_ORDER` and +`PILLAR_STAGES` in `CONSTANT_CASE` sit against `naming-conventions.md:14`, whose worked example is literally a +module-level `new Set(...)` staying `lowerCamelCase` because its contents can mutate; a `ReadonlySet` type does +not make the underlying `Set` deeply immutable and `Object.freeze` cannot fix a `Set`. Left alone because the +casing question is project-wide (Phase 1's `Protocol`/`Status` statics, 4b's constants) and renaming one phase's +two constants would fork the convention rather than settle it — Phase 10's reconciliation owns it. + +**Re-deferred 2026-08-30 (Phase 10): NOT SCHEDULED.** Phase 10 does not own it and did not settle it. The +`CONSTANT_CASE`-vs-`lowerCamelCase` question for module-level immutable collections is a naming-convention +call, not a deviation from the reference contract, so it is outside a reconciliation phase's scope; Phase 10 is +also the last row of the phase table, so there is no later phase to hand it to and none is invented here. The +state is unchanged and still consistent within itself — `STAGE_ORDER` and `PILLAR_STAGES` remain `CONSTANT_CASE` +and remain the pipeline's only such pair (`packages/core/src/pipeline/builder.ts:12`, `:179`, `:248`, `:269`). +**Trigger:** the next module-level immutable collection added outside `pipeline/`, which would make the fork +visible in a third place and force the choice — or a naming-convention sweep commissioned as its own phase. +Logged in `docs/deferred-items.md` so it is tracked rather than silent. + + +--- + +## The retirement rule, as the register stated it + +The register's status vocabulary carried one row more than the eight it keeps, and that row went out with the +tables. It is the definition the 102 IDs above were retired under, so it is reproduced here rather than lost: +**RETIRED** was never a status an item carried — it was what happened to one that reached a resolved +disposition, whether `FIXED`, `RESOLVED`, `CLOSED`, "merged into", or a closed decision such as won't-fix, an +accepted deviation or an accepted risk. The body was removed and nothing kept in its place; the ID was never +released, never renumbered, never reused. It entered the vocabulary on 2026-09-02, when 76 items and 49 table +rows had accumulated in one file with nothing left to act on — the same pressure that produced this note two +days later. diff --git a/docs/superpowers/plans/2026-07-23-phase1-core-http-domain-model-checklist.md b/docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-checklist.md similarity index 100% rename from docs/superpowers/plans/2026-07-23-phase1-core-http-domain-model-checklist.md rename to docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-checklist.md diff --git a/docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md b/docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md similarity index 98% rename from docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md rename to docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md index a8a86ff..67c63f4 100644 --- a/docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md +++ b/docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md @@ -5,8 +5,8 @@ **Purpose:** Implement the immutable, transport-agnostic HTTP domain model — Request, Response, Headers, Status, MediaType, Protocol, QueryParams, RequestOptions, and the conditional-request helpers (ETag, HttpRange, RequestConditions) — as the first piece of real domain code in `@dexpace/core`. This is Phase 1 of the -[v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on the toolchain the -[scaffold milestone](./2026-07-23-scaffold-milestone-design.md) established. +[v1 roadmap](../2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on the toolchain the +[scaffold milestone](../scaffold/2026-07-23-scaffold-milestone-design.md) established. **Scope:** Full `product-spec/04-core-http-domain-model.md` (HTTP-3 through HTTP-53, both MUST and SHOULD level) in one phase, including the conditional-request helpers (HTTP-48/49/50) — they're small and self-contained, and diff --git a/docs/superpowers/plans/2026-07-23-phase1-core-http-domain-model.md b/docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model.md similarity index 99% rename from docs/superpowers/plans/2026-07-23-phase1-core-http-domain-model.md rename to docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model.md index fc58662..da3a98c 100644 --- a/docs/superpowers/plans/2026-07-23-phase1-core-http-domain-model.md +++ b/docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model.md @@ -3507,7 +3507,7 @@ test file in one shot. ## Self-Review **Spec coverage** (every `HTTP-N`/`SEAM-N` ID cited in -`docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md`, mapped to the task that implements +`docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md`, mapped to the task that implements it): HTTP-3/4/5 (construction/immutability/derivation) → every task, pattern established in Task 1 and repeated throughout. HTTP-6/7/8/9 → Task 9 (Request) + Task 2 (Method). HTTP-46/47 → Task 9. HTTP-10/11/12 → Task 3. HTTP-13..22 → Tasks 6–7. HTTP-23..27, HTTP-53 → Task 5. HTTP-28..32 → Task 8. HTTP-33 → Task 4. HTTP-34/35 → Task diff --git a/docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md b/docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation-design.md similarity index 88% rename from docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md rename to docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation-design.md index cbc6622..0d1e626 100644 --- a/docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md +++ b/docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation-design.md @@ -4,7 +4,7 @@ Judgment calls made without a live back-and-forth are called out explicitly in their own section below rather than folded silently into the ledger; flag any of them for revision on review. -**Purpose:** Phase 10 is the last-but-one phase in the [v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md). +**Purpose:** Phase 10 is the last-but-one phase in the [v1 roadmap](../2026-07-23-nodejs-sdk-v1-roadmap-design.md). It audits every deliberate deviation from the JVM reference contract that Phases 0–8 introduced while building `@dexpace/core` and its satellite packages, reconciles them against the pre-implementation prediction in `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` (§10), and produces the @@ -25,6 +25,21 @@ draft's revision voids: Phase 9 has since shipped its own design and plan (2026- conformance only, and will never produce that evidence. Phase 10 decides those four directly instead (§Group L); nothing in this phase's own scope is left waiting on Phase 9. +> **Corrected 2026-08-30 — the scope above is what was planned, not what shipped.** Phase 10 shipped code, in +> three published packages. Two premises this paragraph rests on were false by the time it executed. The first +> is "docs-only repository state": Phases 1-9 had all shipped by then, so `NFR-12` needed a *build*, not a +> *release*, and it closed on evidence rather than staying open (see the correction on Group N below). The +> second is "consolidation and cross-referencing, not new investigation": the audit was performed against +> as-built source rather than against the phase specs that produced the ledger, and that method found a live +> defect — `Page`, `FetchTransport` and `UndiciTransport` declared `[Symbol.asyncDispose]` as a plain computed +> class member, which on the declared `engines.node ">=20.3"` floor bound the method to the string key +> `"undefined"` (`NFR-10`; the symbol landed in Node 20.4). Fixing it is a breaking type change with two +> changesets, and three later review passes found three more defects behind it. The full inventory, with the +> reasoning for breaking this scope and for holding the line on the project-wide convention sweeps that also +> named this phase, is the roadmap's **Status note (2026-08-30, Phase 10 EXECUTED — scope corrected)**; the +> per-item as-built evidence is `docs/deviations.md`. This paragraph is left standing rather than rewritten so +> the planned-versus-actual gap stays legible. + **Governing documents:** `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` (the document this phase rewrites), the roadmap's own Deferred Items Log, and every Phase 2, 3a–8b, and 9 spec's Deviation Ledger section plus the handful of plan-level "Deviation Ledger Additions" sections (5c, 6a, 6b, 6c) @@ -163,7 +178,7 @@ with 5a's attempt-stamping producing fresh `Request` copies; 5c's marker also su hook on a marked hop, not just the outbound stamp (a bug the 5c design caught before shipping). Two items were originally left open pending Phase 9's conformance-test execution against reference fixtures. **That premise is now void**: Phase 9 was brainstormed and planned (2026-07-28) after this design's first draft, and its actual -scope is `XCUT`/`NFR` conformance only (`docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md`) +scope is `XCUT`/`NFR` conformance only (`docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md`) — it does not re-audit `AUTH-*`/`REDIR-*` interpretive calls, and no phase's code exists yet for a fixture-based sweep to run against regardless. The roadmap's own Deferred Items Log was updated to retarget both rows here; Phase 10 decides them rather than deferring a second time: @@ -203,6 +218,19 @@ the already-scripted `prepublishOnly` + `npm publish --provenance` path for real open with "first real release" as the unblock trigger, exactly as the roadmap's Deferred Items Log already states — Phase 10 doesn't manufacture a false close here. +> **Corrected 2026-08-30 — `NFR-12`'s half of this group was wrong, and it closed.** The premise "needs a real +> build artifact… neither exists in this docs-only repo state" expired the moment Phases 1-9 shipped code. +> `NFR-12` needed a *build*, not a *publish*, and was verifiable from Phase 1 onward; it sat open two phases +> longer than it had to. It is now closed on evidence and kept closed by a gate: +> `scripts/verify-reproducible-build.mjs` sweeps every `dist/` and `*.tsbuildinfo`, builds twice, and compares a +> SHA-256 per emitted file **and** per `npm pack` tarball across all nine publishable packages — 644 emitted +> files and 9 tarballs byte-identical. It is a blocking CI step and a `ci-preflight` step, and was +> negative-tested by injecting a `Date.now()` into `packages/core/scripts/gen-version.mjs`, the one build-time +> codegen step. **`NFR-16` is unaffected and this group's disposition still holds for it:** its conformance test +> is behavioral and needs a real registry and a real OIDC token. One sub-claim about `NFR-16` was also wrong and +> is corrected in §10 itself — `npm publish --provenance` was never scripted; only `prepublishOnly` is. See +> `docs/deviations.md` §14 and the roadmap's `NFR-12` / `NFR-16` rows. + **Group O — HTTP-18 vs. HTTP-48/50 ETag obs-text replay tension (decision made now).** Not in the original 12; flagged by Phase 1's plan as unresolved and explicitly targeted at Phase 10. `RequestConditions.applyTo` writes entity tags through `Headers`' outbound `set`, which enforces `HTTP-18`'s **MUST**-level restriction (HTAB + diff --git a/docs/superpowers/plans/2026-07-28-phase10-deviation-reconciliation.md b/docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation.md similarity index 94% rename from docs/superpowers/plans/2026-07-28-phase10-deviation-reconciliation.md rename to docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation.md index fb8d55b..226c11d 100644 --- a/docs/superpowers/plans/2026-07-28-phase10-deviation-reconciliation.md +++ b/docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation.md @@ -4,7 +4,7 @@ **Goal:** Rewrite `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` in place with the as-built, reconciled deviation ledger from Phases 0-8, per -`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md`, and update the roadmap's Deferred +`docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation-design.md`, and update the roadmap's Deferred Items Log rows that name Phase 10 by name. **Architecture:** No code, no package. Two document edits: (1) a full-content replacement of §10's twelve @@ -42,7 +42,7 @@ not modify, every Phase 2 and 3a-8b spec and the 5c/6a/6b/6c plans' Deviation Le ``` docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md # full rewrite (Task 1, 2) -docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md # Deferred Items Log # (Task 3) +docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md # Deferred Items Log # (Task 3) rows + status note ``` @@ -76,7 +76,7 @@ This section is the as-built reconciliation of every place the port's Node-idiom pre-implementation prediction now that Phases 0-8 have each shipped a design and plan. None of these narrow a MUST-level correctness guarantee; each is a case where the JVM-specific mechanism a requirement was worded around does not exist in Node, and an equivalent, differently-shaped mechanism is substituted instead. Reconciled by -Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-design.md`), 2026-07-28. +Phase 10 (`docs/work/mvp/phase10/2026-07-28-phase10-deviation-reconciliation-design.md`), 2026-07-28. 1. **Single execution model eliminates every thread/CAS/interrupt-flag primitive, and collapses the sync/async transport seam into one.** **SEAM-11** describes a synchronous, blocking transport contract as distinct from @@ -172,7 +172,7 @@ Phase 10 (`docs/superpowers/specs/2026-07-28-phase10-deviation-reconciliation-de challenge-reaction hook is suppressed on a marked hop too, not only the outbound stamp — a leak the Phase 5c design caught before shipping (Phase 5c). Two items from this area were originally left open pending Phase 9's conformance sweep against real fixtures; Phase 9's actual design scoped itself to `XCUT`/`NFR` - conformance only (`docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md`) and will + conformance only (`docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md`) and will never produce that evidence, so Phase 10 decides both directly instead of leaving them open indefinitely: - **Redirect predicate scope over safety mechanics — confirmed, 5b's reading is correct.** `REDIR-20`'s "fully override the built-in decision" scopes to the follow/no-follow determination the predicate is @@ -282,7 +282,7 @@ tokens — visually confirm that range string is present with a second check: `g ### Task 3: Update the roadmap's Deferred Items Log **Files:** -- Modify: `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (Deferred Items Log table, currently +- Modify: `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (Deferred Items Log table, currently the table starting after the `## Deferred Items Log` heading) **Interfaces:** @@ -292,7 +292,7 @@ tokens — visually confirm that range string is present with a second check: `g the deferral inline. - [ ] **Step 1: Read the current table** from - `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (the `## Deferred Items Log` section) to + `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (the `## Deferred Items Log` section) to get exact current row text before editing — table rows shift line numbers as earlier edits land, so match by row content (`| NFR-8 |`, `| NFR-12 |`, etc.), not by line number. @@ -386,7 +386,7 @@ installs it explicitly, already possible via the public authoring surface. - [ ] **Step 12: Verify the table still parses as Markdown** — every row (old and new) has the same number of `|`-delimited columns as the table's header row. -Run: `awk -F'|' '/^\|.*(NFR-8|NFR-12|NFR-16|SEAM-5|HTTP-18|DigestChallengeUnsupportedError|Basic\/Digest never stamp|Redirect predicate|Whether `clientIdentityStep`)/{print NF}' docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` +Run: `awk -F'|' '/^\|.*(NFR-8|NFR-12|NFR-16|SEAM-5|HTTP-18|DigestChallengeUnsupportedError|Basic\/Digest never stamp|Redirect predicate|Whether `clientIdentityStep`)/{print NF}' docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` Expected: every printed number identical to the header row's own column count (check the header row's count first with the same `awk -F'|'` pattern against `| Item | Originated in |`). **Use `^\|.*(...)`, not `^\| \`(...)`** — three of these nine rows (`Basic/Digest never stamp preemptively`, `Redirect predicate's scope...`, `Whether \`clientIdentityStep\`...`) open with plain prose, not a backtick-quoted term, so an anchor requiring a backtick immediately after `| ` @@ -408,7 +408,7 @@ silently skips them and under-verifies. - [ ] **Step 1: Re-open every one of these 17 files and confirm every *row* of each one's ledger table has a corresponding sentence in the rewritten §10** (this list is exhaustive — every phase from 2 through 9 that has a Deviation Ledger section; regenerate it with - `grep -rln '^## Deviation Ledger (for Phase 10)' docs/superpowers/specs/` rather than trusting this transcription): + `grep -rln '^## Deviation Ledger (for Phase 10)' docs/work/mvp/` rather than trusting this transcription): Check rows, not phases. A phase-label grep is not sufficient evidence here: several phases are cited by many items, so `Phase 2` (or `Phase 5a`, or `Phase 8a`) appearing in §10 proves only that *something* from that @@ -417,23 +417,23 @@ silently skips them and under-verifies. validation. Walk each table row by row. ``` -2 docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md -3a docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md -3b docs/superpowers/specs/2026-07-25-phase3b-body-lifecycle-design.md -4a docs/superpowers/specs/2026-07-25-phase4a-execution-context-design.md -4b docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md -4c docs/superpowers/specs/2026-07-25-phase4c-stage-pipeline-design.md -5a docs/superpowers/specs/2026-07-26-phase5a-retry-design.md -5b docs/superpowers/specs/2026-07-26-phase5b-redirect-design.md -5c docs/superpowers/specs/2026-07-26-phase5c-auth-design.md + docs/superpowers/plans/2026-07-26-phase5c-auth.md -6a docs/superpowers/specs/2026-07-28-phase6a-serde-design.md + docs/superpowers/plans/2026-07-28-phase6a-serde.md -6b docs/superpowers/specs/2026-07-28-phase6b-sse-design.md + docs/superpowers/plans/2026-07-28-phase6b-sse.md -6c docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md + docs/superpowers/plans/2026-07-28-phase6c-pagination.md -7a docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md -7b docs/superpowers/specs/2026-07-28-phase7b-observability-design.md -8a docs/superpowers/specs/2026-07-28-phase8a-transport-design.md -8b docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md -9 docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md +2 docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md +3a docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md +3b docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md +4a docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context-design.md +4b docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md +4c docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md +5a docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md +5b docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md +5c docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md + docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth.md +6a docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md + docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde.md +6b docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md + docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse.md +6c docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md + docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination.md +7a docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md +7b docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md +8a docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-design.md +8b docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md +9 docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md ``` Expected, primary: every ledger table row across those 16 files is either represented by a sentence in the diff --git a/docs/superpowers/plans/2026-07-23-phase2-seam-foundations-checklist.md b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-checklist.md similarity index 97% rename from docs/superpowers/plans/2026-07-23-phase2-seam-foundations-checklist.md rename to docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-checklist.md index b0ab1f1..fde7ada 100644 --- a/docs/superpowers/plans/2026-07-23-phase2-seam-foundations-checklist.md +++ b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-checklist.md @@ -1,7 +1,7 @@ # Phase 2 — Seam Foundations Implementation Plan — Checklist Verification of [2026-07-23-phase2-seam-foundations.md](./2026-07-23-phase2-seam-foundations.md) against every -requirement ID cited in `docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md`'s disposition table +requirement ID cited in `docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md`'s disposition table (`docs/product-spec/03-pluggable-seams-and-extension-model.md`), plus the HTTP-29 retrofit (`docs/product-spec/04-core-http-domain-model.md`) and the NFR-10/NFR-17 residual pulled forward from Phase 3. diff --git a/docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md rename to docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md index 37dcbd1..87ed88d 100644 --- a/docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md +++ b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md @@ -4,7 +4,7 @@ **Purpose:** Build the seam *contracts* — `Transport`, `Serde<T>`, and the operation-input projection (`buildRequest()`) — that later phases' pipelines, resilience layer, and concrete adapters build on. This is -Phase 2 of the [v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on Phase 1's domain model. +Phase 2 of the [v1 roadmap](../2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on Phase 1's domain model. **Scope is narrower than the JVM reference's "seams" concept.** Per `sdk-design-nodejs/03`, Node collapses most of what the JVM reference needs multiple seams and a discovery mechanism for: there is one `Transport` shape (not a diff --git a/docs/superpowers/plans/2026-07-23-phase2-seam-foundations.md b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations.md similarity index 99% rename from docs/superpowers/plans/2026-07-23-phase2-seam-foundations.md rename to docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations.md index 35d46c5..beffdff 100644 --- a/docs/superpowers/plans/2026-07-23-phase2-seam-foundations.md +++ b/docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations.md @@ -4,7 +4,7 @@ **Goal:** Ship the seam *contracts* in `@dexpace/core` — `Transport`, `Serde<T>`, and `buildRequest()` / `OperationDescriptor` — that later phases' pipelines, resilience layer, and concrete adapters build on, per -`docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md`. +`docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md`. **Architecture:** Three new interfaces/functions in a new `packages/core/src/seams/` folder, plus two small retrofits to Phase 1's `src/http/` folder (`encodeRfc3986Component` extraction, a new `DexpaceError` taxonomy @@ -1145,7 +1145,7 @@ git commit -m "feat(core): wire Phase 2 public barrel, Node-floor CI conformance ## Self-Review -**Spec coverage** (every requirement ID in `docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md`'s +**Spec coverage** (every requirement ID in `docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md`'s disposition table, mapped to the task implementing it): - SEAM-11/SEAM-16 (collapsed) → Task 4, `Transport.send(): Promise<Response>` structurally covers the diff --git a/docs/superpowers/plans/2026-07-24-phase3a-io-contracts-checklist.md b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-checklist.md similarity index 80% rename from docs/superpowers/plans/2026-07-24-phase3a-io-contracts-checklist.md rename to docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-checklist.md index ff74490..4c96e01 100644 --- a/docs/superpowers/plans/2026-07-24-phase3a-io-contracts-checklist.md +++ b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-checklist.md @@ -2,7 +2,7 @@ Verification of [2026-07-24-phase3a-io-contracts.md](./2026-07-24-phase3a-io-contracts.md) against every requirement ID in `docs/product-spec/05-i-o-contracts.md`, as dispositioned by -`docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md`. +`docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md`. **Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred (named target phase) — N/A Not applicable in this port. @@ -13,7 +13,7 @@ requirement ID in `docs/product-spec/05-i-o-contracts.md`, as dispositioned by |---|---|---|---|---| | IO-1 | MUST | Tail-append, transferred count, ≥1 when non-exhausted, 0 for count 0, −1 at end, never over-deliver | ✅ | Task 2 (`ByteQueue.read`), Task 6 (`BufferedSource.read`); partial-then-EOF asserted at both | | IO-2 | MUST | A 0-count read returns 0 and never reports end-of-stream | ✅ | Tasks 2, 6 — checked **before** exhaustion, commented as load-bearing at both sites, asserted on a fresh and an exhausted source | -| IO-3 | MUST | Negative count rejected as an argument error before any I/O | ✅ | Tasks 2, 6, 9 via `assertCount`/`invariant`; asserted to leave both source and destination untouched | +| IO-3 | MUST | Negative count rejected as an argument error before any I/O | ✅ | Tasks 2, 6, 9, 10 via `assertCount`, single-sourced in `limits.ts`; asserted to leave both source and destination untouched. **Corrected during Phase 3b:** the guard shipped as three byte-for-byte copies and `TeeSink` — the fourth size-taking surface — had none, so a negative count reached it and was rejected only indirectly by whichever `ByteQueue` call ran first, and not at all on its `count === 0` and short-source early returns | | IO-4 | MUST | Sink write removes exactly N from the source HEAD; fails rather than writing partially | ✅ | Task 2 (`ByteQueue.write`), Task 9 (`BufferedSink.write`); nothing reaches the wire on the short-source path | | IO-5 | MUST | Sink exposes flush; source and sink both closeable | ✅ | Task 9 | | IO-18 | SHOULD | `emit` (cheap handoff) distinguished from `flush` (full force-out); in-memory may no-op returning self | ✅ | Task 9 | @@ -35,7 +35,7 @@ requirement ID in `docs/product-spec/05-i-o-contracts.md`, as dispositioned by |---|---|---|---|---| | IO-11 | MUST | `exhausted()`, single-byte read, count-less read of all remaining (empty when exhausted) | ✅ | Task 6 | | IO-12 | MUST | Exact-count read returns exactly N or fails; never short | ✅ | Task 6, asserted across chunk boundaries and on the short path | -| IO-13 | MUST | UTF-8 and explicit-charset reads, with symmetric write-side encodings | ✅ (read) / ⚠️ (write, bounded) | Task 7 (read: any `TextDecoder` label, ISO-8859-1 round-trip per the requirement's own conformance note), Task 9 (write: **UTF-8 and ISO-8859-1 only**). `TextEncoder` is UTF-8-only and `SEAM-1` forbids an encoding dependency, so full symmetry is unreachable; any other label throws rather than silently re-encoding. Ledgered deviation | +| IO-13 | MUST | UTF-8 and explicit-charset reads, with symmetric write-side encodings | ✅ (read) / ⚠️ (write, bounded) | Task 7 (read: any `TextDecoder` label; `text-codec.ts`'s `decodeText` implements true ISO-8859-1 and sets `ignoreBOM`, and is deliberately **not** interchangeable with `http/charset.ts`'s whole-body `decodeBodyText` — the names were disambiguated in Phase 3b), Task 9 (write: **UTF-8 and ISO-8859-1 only**), plus two `fast-check` round-trip property tests in `buffered-sink.test.ts` — sink-out/source-back through UTF-8, and through ISO-8859-1 asserting one byte per code point, which is what distinguishes an honored charset from a silent UTF-8 re-encoding. `TeeSink`'s own `writeUtf8`/`writeString` are asserted to mirror the primary's exact encoded bytes and to refuse an unsupported label identically. `TextEncoder` is UTF-8-only and `SEAM-1` forbids an encoding dependency, so full symmetry is unreachable; any other label throws rather than silently re-encoding. Ledgered deviation | | IO-14 | MUST | Line read consumes the terminator; `\n` and `\r\n` both terminate; lone `\r` is content; final unterminated line as-is; absent when exhausted first | ✅ | Task 7, including a `fast-check` property test with **adversarially generated chunk boundaries**, so a terminator straddling two stream chunks is covered — the case the requirement's rationale names and hand-picked examples miss | | IO-15 | MUST | Skip advances exactly N, fails if fewer remain; `skip(0)` a no-op even at/after EOF | ✅ | Task 6 | | IO-16 | SHOULD | Read-only host-native byte-stream bridge; symmetric writable bridge; closing the bridge closes the owner | ✅ | Task 12. Host-native means `ReadableStream`/`WritableStream` for this port, per `sdk-design/03` §3.1 — no `node:` import; Task 13 Step 9 greps to enforce that | @@ -56,12 +56,12 @@ requirement ID in `docs/product-spec/05-i-o-contracts.md`, as dispositioned by | ID | Level | Requirement gist | Status | Where | |---|---|---|---|---| -| IO-17 | MUST | Write-all pumps to exhaustion, terminates only on −1; a foreign source's zero-read for a positive request is an I/O error, never EOF and never spun on | ✅ | Task 11 (`writeAll`); the violation is raised in Task 5's `#pullOnce` and driven by `protocolViolatingStream` | +| IO-17 | MUST | Write-all pumps to exhaustion, terminates only on −1; a foreign source's zero-read for a positive request is an I/O error, never EOF and never spun on | ✅ | Task 11 (`writeAll`); the violation is raised in Task 5's `#pullOnce` and driven by `protocolViolatingStream`. **Corrected during Phase 3b:** a primitive source that *over*-reported its transferred count was left to `ByteQueue.takeBytes` and surfaced as `EndOfStreamError: delivered 2 of 99 bytes` — a foreign source's broken accounting reported as an exhausted stream, the exact confusion this requirement forbids. Both misreport directions now raise `SourceContractViolationError` and are asserted | | IO-25 | MUST | Tee mirrors into the tap AND forwards the full untruncated payload; the wire body is never reduced | ✅ | Task 10, plus **the most important property test in §5**: for arbitrary write sequences and arbitrary tap caps, the primary receives the exact concatenation of every written byte | | IO-26 | MUST | Tap capacity limit; default effectively unbounded; a limit of 0 mirrors nothing while forwarding everything | ✅ | Task 10 (`Number.POSITIVE_INFINITY` default, spelled as a value rather than a magic number); all three cases asserted | | IO-27 | MUST | Mirror BEFORE forwarding; clear staging even on a failed write so no stale bytes prepend | ✅ | Task 10, both clauses asserted; staging cleared in a `finally` so it holds on the throwing path | | IO-28 | MUST | No direct backing-buffer handle; attempting it fails, directing callers at the typed writes | ✅ | Task 10 (`get buffer(): never`) | -| IO-29 | MUST | Tee's own flush/close/emit forward to the PRIMARY only, leaving the tap intact | ✅ | Task 10, with snapshot-after-close asserted | +| IO-29 | MUST | Tee's own flush/close/emit forward to the PRIMARY only, leaving the tap intact | ✅ | Task 10. All three asserted: `close` with snapshot-after-close, and `flush`/`emit` both by returning the tee with the tap intact and — the observable proof they are not swallowed by the decorator — by rejecting with `ClosedResourceError` once the primary is closed, which only the primary can raise | ## 5.6 Provider factories, timeouts, and thread-safety @@ -80,10 +80,11 @@ requirement ID in `docs/product-spec/05-i-o-contracts.md`, as dispositioned by | Nothing enters the published API surface | Design decision (styleguide 10.3, Phase 2's `Serde<T>` precedent) | ✅ | Task 13 Step 8 — `git diff --exit-code packages/core/etc/core.api.md` must produce no output. Mechanical proof, not a review promise | | No runtime dependency added | `SEAM-1` | ✅ | Task 13 Step 7 runs `verify:seam-1`; `mitata` is a root devDependency only | | No `node:` import in core | `sdk-design/03` §3.1, runtime-agnosticism | ✅ | Task 13 Step 9 greps `packages/core/src/` and fails on any match | -| Property tests where invariants exist | styleguide 11.5 | ✅ | Task 4 (`ByteQueue` ×4), Task 7 (`readUtf8Line`), Task 8 (views ×2), Task 10 (`TeeSink` wire payload) | +| Property tests where invariants exist | styleguide 11.5 | ✅ | Task 4 (`ByteQueue` ×4), Task 7 (`readUtf8Line`), Task 8 (views ×2), Task 9 (charset round-trips ×2), Task 10 (`TeeSink` wire payload) | +| Rejection assertions are awaited and attributable | styleguide 11.9 | ✅ | `test-support/rejection.ts`. bun types `.rejects.toThrow()` as `void`, so the plan's `await expect(…).rejects` form fails `@typescript-eslint/await-thenable`; the helper awaits the promise and returns the reason instead, with no `eslint-disable`. Ledgered | | Negative-space and cleanup assertions | styleguide 11.9, 13.9 | ✅ | Idempotent close (Tasks 4, 5, 6, 9), both IO-42 directions (Tasks 4, 6), parent-close invalidation (Task 8), failed-write tap capture (Task 10) | | Determinism — no fake clocks needed | styleguide 11.8 | ✅ | IO-40 means this layer owns no timer; every stream under test is built from an in-memory array | -| Fakes over `mock.module` | styleguide 11.3 | ✅ | Task 5's `test-support/fake-stream.ts`, excluded from the build via `tsconfig.build.json` | +| Fakes over `mock.module` | styleguide 11.3 | ✅ | Task 5's `test-support/fake-stream.ts` and `test-support/rejection.ts`, both excluded from the build via `tsconfig.build.json`'s `src/io/test-support/**` | | No type-level tests | styleguide 11.6 | ✅ (correctly absent) | 11.6 requires them for public generics and conditional types; this phase publishes neither. Stated rather than manufactured | | Committed baseline bench | styleguide 15.6 | ✅ | Task 13, `byte-queue.bench.ts`. Baseline only — no optimization applied, no 15.10 ledger notes, per 15.1/15.6's "do not tune ahead of a profile" | | 80% aggregate coverage floor | `NFR-5` | ✅ | Task 13 Step 7 | diff --git a/docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md similarity index 91% rename from docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md rename to docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md index a61994c..f21a9d5 100644 --- a/docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md +++ b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md @@ -5,7 +5,7 @@ **Purpose:** Implement the byte-streaming primitives — `ByteQueue`, `BufferedSource`/`BufferedSink`, the non-consuming peek/slice views, `TeeSink`, the pump, and the provider factories — that Phase 3b's bodies, Phase 6's SSE and serde, and Phase 8's transports all read and write through. This is the first half of Phase 3 of the -[v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on Phase 2's seam contracts. +[v1 roadmap](../../2026-07-23-nodejs-sdk-v1-roadmap-design.md), building on Phase 2's seam contracts. **Scope:** every requirement in `docs/product-spec/05-i-o-contracts.md` — `IO-1` through `IO-42`, both MUST and SHOULD level — is dispositioned here. Most are implemented; three groups are deliberately not built and one is not @@ -14,7 +14,7 @@ full." **Governing documents:** `docs/product-spec/05-i-o-contracts.md` (normative, cited by ID throughout), `docs/sdk-design-nodejs/03-seam-by-seam-idiomatic-mapping.md` §3.1 (the Web Streams mapping this design follows), -and `docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md` (the `DexpaceError` root and the +and `docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md` (the `DexpaceError` root and the barrel-as-enforcement-point precedent this phase reuses). Styleguide: `styleguide/typescript/` chapters 05, 06, 08, 09, 10, 11, 12, 13, 15. @@ -399,6 +399,10 @@ layer where the temptation to dump the offending bytes into the message is stron | `TeeSink` as a sink decorator | `sdk-design/03` §3.1 phrasing | `TransformStream` queueing muddies `IO-27`'s mirror-before-forward ordering; §3.1's substantive point is untouched | | `IO-30` resolution half, `IO-39` not built | product-spec §5.6 | No registry exists — same class as `SEAM-5`–`SEAM-10` | | `IO-38` not applicable | product-spec §5.4 | The requirement is about a close on one thread invalidating a slice being read on another, so it presupposes an instance can reach a second thread. None can. **Class instances are not structured-cloneable at all** — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: a `ReadableStreamDefaultReader` is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject | +| `"DOM.AsyncIterable"` added to `tsconfig.base.json`'s `lib` | Phase 2's `lib: ["ES2022", "DOM"]` baseline | `IO-16`'s `toReadableStream()` returns a `ReadableStream`, and asserting it with `for await (const chunk of …)` needs the async-iteration declarations, which TypeScript ships in a separate `lib` entry from `DOM`. Workspace-wide because the `lib` array is; no runtime effect and no new dependency (`SEAM-1` untouched), and the API report is unchanged. The alternative — driving the bridge test through `getReader()` — was rejected because async iteration is how a consumer will actually use the bridge, so the test should exercise that path | +| `packages/core/src/invariant.ts` created in this phase | The plan's prerequisite, which lists `invariant` as existing from Phase 1 | Phase 1 shipped `requireField` for HTTP-4's required-field message, not a general assertion primitive, so `invariant` (styleguide 5.6, 8.7) did not exist. `IO-3`, `IO-10`, and `IO-21` all need it, so it was added here as an `@internal` module with `InvariantViolation` as its own class. Nothing in `src/http/` was changed to route through it — Phase 1's `requireField` still owns HTTP-4's message | +| `ByteQueue.takeBytes` checks `MAX_BYTE_ARRAY_LENGTH` *before* the short-source check | The plan's Task 3 code, which checked size first | With the plan's ordering, an over-limit request on a short queue raised `EndOfStreamError`, hiding the real problem, and the plan's own `IO-9` test (`takeBytes(MAX + 1)` expects `AllocationLimitError`) could not pass. `IO-9`'s actionable-refusal requirement wins over reporting a size mismatch that is a consequence of the over-limit ask | +| Rejection assertions go through a `rejection()` test helper, not `await expect(…).rejects.toThrow(…)` | The plan's test code, which used `await expect(…).rejects` throughout | bun types `rejects` as `Matchers<unknown>` whose `toThrow()` returns `void`, even though at run time it returns a promise. So the plan's form fails this repo's type-aware `@typescript-eslint/await-thenable` gate, and dropping the `await` to satisfy lint leaves the assertion racing test teardown — bun still fails the run, but the failure can attribute to a later test. `test-support/rejection.ts` awaits the promise, returns the rejection reason, and fails loudly if the promise resolves, so every assertion is awaited and attributable with no `eslint-disable` | | Write-side charsets limited to UTF-8 and ISO-8859-1 | `IO-13`'s "symmetric write-side encodings" | `TextEncoder` is UTF-8-only and `SEAM-1` forbids an encoding dependency. Read side stays fully general via `TextDecoder`; the write side covers the two encodings HTTP needs, and `IO-13`'s own conformance note names ISO-8859-1 as the non-UTF-8 case. Any other label throws rather than silently corrupting bytes. The `writeUtf8(begin, end)` substring-range overload is subsumed by `String.prototype.slice` at the call site | ## Testing @@ -416,7 +420,11 @@ invariant-bearing functions, and §5 is almost nothing else: covered; `IO-14`'s own rationale calls out surviving slice-window boundaries, and that is exactly the case hand-picked examples miss. - **`readString`/`writeString`** — round-trip through UTF-8 and through ISO-8859-1 (`IO-13`, whose conformance note - names a non-UTF-8 charset explicitly). + names a non-UTF-8 charset explicitly). The ISO-8859-1 generator excludes code points `0x80`–`0x9F`: the WHATWG + Encoding Standard maps the label `iso-8859-1` onto windows-1252, so `TextDecoder` returns U+20AC for `0x80` + rather than U+0080. That is the platform's asymmetry, not the sink's — the write side is a straight + code-point-to-byte map — and HTTP needs none of those C1 controls. Recorded here so Phase 9 does not read the + excluded band as an untested gap. - **View independence** — N views at arbitrary offsets and counts each read the same bytes a direct read at that window would, and no view's read advances another's cursor (`IO-19`, `IO-20`, `IO-23`). - **`TeeSink`** — for arbitrary write sequences and arbitrary tap caps, the primary receives the exact concatenation diff --git a/docs/superpowers/plans/2026-07-24-phase3a-io-contracts.md b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts.md similarity index 99% rename from docs/superpowers/plans/2026-07-24-phase3a-io-contracts.md rename to docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts.md index 827eec6..038bcf5 100644 --- a/docs/superpowers/plans/2026-07-24-phase3a-io-contracts.md +++ b/docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts.md @@ -5,7 +5,7 @@ **Goal:** Ship the byte-streaming primitives in `@dexpace/core` — `ByteQueue`, `RetentionWindow`, `BufferedSource` (with peek/slice views), `BufferedSink`, `TeeSink`, `writeAll`, and the `IO-30` factories — satisfying `product-spec/05-i-o-contracts.md` (`IO-1`–`IO-42`), per -`docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md`. +`docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md`. **Architecture:** A new `packages/core/src/io/` folder, layered strictly one way: `limits`/`errors` → `byte-queue` → `retention-window` → `buffered-source`/`buffered-sink` → `tee-sink`/`pump` → diff --git a/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-checklist.md b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-checklist.md new file mode 100644 index 0000000..433bf46 --- /dev/null +++ b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-checklist.md @@ -0,0 +1,100 @@ +# Phase 3b — Body Lifecycle Implementation Plan — Checklist + +Verification of [2026-07-25-phase3b-body-lifecycle.md](./2026-07-25-phase3b-body-lifecycle.md) against every +requirement ID in `docs/product-spec/06-request-and-response-body-lifecycle.md`, as dispositioned by +`docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md`. + +**Legend:** ✅ Implemented and tested — 📄 Contract-obligation-only (this phase guarantees the property; a later +phase consults it) — ⏳ Deferred (named target phase) — 🚫 Not built (permanent simplification, named reason). + +## 6.1 The body model + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| HTTP-36 / BODY-1 | MUST | Body produces bytes via a single write-to-sink operation; reports media type, content length (-1 unknown), replayability | ✅ | Task 2 (`Body` interface), Tasks 3/4/6 (five concrete variants). `writeTo` takes the platform `WritableStream<Uint8Array>`, which is what keeps all of `src/io/` `@internal` | +| BODY-2 | MUST | Composite replayability; declared length collapses to unknown if any part's is | ✅ | Task 6; both directions asserted, plus the `-1` collapse | +| HTTP-51 | SHOULD | One shared framing routine drives both the declared length and the emitted bytes | ✅ | Task 6 (`renderPartHeader` called by `computeContentLength` **and** `writeTo`), plus a `fast-check` property that declared length equals bytes written for arbitrary part sets. Header parameter values are quoted/escaped and CR/LF stripped; a second property asserts an arbitrary part name never injects extra framing CRLFs. The shared routine alone is **not** sufficient: it takes each part's own `contentLength` on trust, and `MultipartPart.body` is the `@public` `Body` interface, so a caller implementation can report one length and write another. `writeTo` therefore also verifies its total against the declared value — refusing an overrunning chunk before it reaches the sink, and raising inside the writer scope on a short total so the sink is aborted rather than cleanly closed. Both directions asserted | +| HTTP-51 (boundary) | SHOULD | Caller-supplied boundary validated | ✅ (grammar) / ⚠️ (non-appearance) | Task 6. RFC 2046 puts two duties on the sender: a `bchars`-valid delimiter, and one appearing in no part. The first is enforced (`MultipartBoundaryError`); the second **cannot** be checked here, because a `StreamBody` part's bytes do not exist until the write, and a partial scan would read as a complete guarantee. Mitigated by generating the boundary by default — 32 random characters from Web Crypto — and by documenting the obligation on both entry points. Ledgered | +| BODY-3 / HTTP-37 | MUST | Materialize-once; the consumed-once guard is checked before the first suspension point | ✅ | Task 4 (`StreamBody.#consumed`, set before the first `await`), Task 5 (`materialize` holds no state of its own — `writeTo` can be called directly, and the guard must cover that path too). Property test over N concurrent callers: exactly one drains, every other observes `ConsumedBodyError` | +| BODY-8 | MUST | Stream ownership stated per variant | ✅ | Task 4. `StreamBody.writeTo` never cancels the caller's stream — **on either path**. The declared-length path only `releaseLock()`s, and the unknown-length path passes `preventCancel: true`, because `pipeTo`'s default cancels the source when the destination fails. Both asserted with a `cancel` spy | +| BODY-9 | SHOULD | Mark/reset replay on a stream body | ✅ (bounded) | Task 4 — `StreamBody` is always single-use. Node's `ReadableStream` has no generic mark/reset; a caller wanting replay calls `materialize` or uses `byteArrayBody`. Ledgered | +| HTTP-39 / BODY-10 | MUST | Declared length verified; a stream that disagrees fails rather than sending a truncated or overrunning body | ✅ | Task 4's `#writeExactly`. The overrun check runs **before** the write, not after the loop: once a transport has stamped `Content-Length`, an extra byte is already on the socket and no thrown error can recall it. A short stream raises `EndOfStreamError(delivered, declared)` from inside the writer scope, so the sink is aborted rather than cleanly closed | +| HTTP-38 / BODY-35 | MUST | Replayability classified by source; form-urlencoded always replayable, `+` for space | ✅ | Task 3. Encoding routes through Phase 1's `QueryParams`, so the RFC 3986 rules are single-sourced and only the `%20`→`+` swap is local; a postcondition asserts no literal space survives | +| BODY-4, BODY-5 | MUST | Replayability and idempotency gate a re-send | 📄 | Contract-obligation-only. This phase guarantees `replayable` is correct; Phase 5's retry/redirect/auth steps consult it. No task builds consultation | +| HTTP-40 / BODY-11, BODY-12, BODY-13, BODY-36 | MUST | File-backed body | ⏳ Phase 8a | Needs `node:fs`, against core's zero-`node:`-import invariant. Resolved in Phase 8a's design as a structural recognition contract plus a `@dexpace/body-file` package. In the roadmap's Deferred Items Log | +| HTTP-2 / HTTP-3 | MUST | Builder-based models expose a pre-populated `newBuilder()`; never a public field-wise constructor | ✅ | Task 6 adds `MultipartBodyBuilder` with static and instance `newBuilder()`, copying the parts list so the builder never aliases the source. `HTTP-2` holds two ways: concrete body classes are exported **as types only** (never as values, so `new ByteArrayBody(...)` is unreachable), and `Response` — which *is* a value export — keeps its `private constructor` plus the `createResponse` friend hook. The committed API report is the mechanical gate for both | +| HTTP-1 / XCUT-15 | MUST | A constructed model cannot drift from what it emits | ✅ | Tasks 3/4/6 — every variant calls `freezeBody(this)` last, so `contentLength` cannot be reassigned after construction; `MultipartBody` additionally deep-copies its parts array. Asserted across all five variants | + +## 6.2 Response body + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| HTTP-41 / BODY-14 | MUST | Single-use body; repeat access returns the same reference, not a replay | ✅ | Task 8 — `Response.body` is a plain getter over a `#private` field, so this needs no separate guard | +| HTTP-41 / BODY-15, HTTP-43 | MUST | Idempotent close; releases the connection whether or not the body was read | ✅ | Task 8. Memoized on the close *promise*, not a boolean: a flag set before the `await` reports a failed release as success to every later caller. Tolerates a body locked by an external consumer, since `BODY-15` forbids assuming the body was read | +| HTTP-41 / BODY-16 | MUST | Convenience readers close in a finally-style guarantee | ✅ | Task 8. Two ordering constraints, both load-bearing and both asserted. `reader.releaseLock()` is the first statement of the `finally`, before the close: `cancel()` rejects with `TypeError` on a locked stream and reading to `{done: true}` does not release the lock, so the reverse order turns every successful read into a rejection. And the reader is acquired **inside** the try: `getReader()` itself throws when an external consumer already holds the lock — which `BODY-15` forbids assuming away — so acquiring it above the try skipped the close on exactly the path the guarantee most needs to cover | +| HTTP-42 | MUST | `text()` uses the declared charset, falling back to UTF-8 | ✅ | Task 8 via `http/charset.ts`. Falls back for an absent, unparseable, **and** unrecognized label; all three asserted | +| HTTP-44, HTTP-45 | MUST | Raw fields without touching the body; parse-once memoized including failure; concurrent first callers serialized | ✅ | Task 9. The promise is cached before the first `await`, and the parse is wrapped in an `async` IIFE so a parser that throws *synchronously* is memoized too — a bare `??=` would re-run the handler against a body whose bytes are already gone | + +## 6.3 Request-body logging + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| BODY-17 | MUST | Mirror into the tap AND forward the full untruncated payload | ✅ | Task 10, plus the property test that carries this phase's most important invariant: for arbitrary payloads and arbitrary caps, the primary receives the exact concatenation of every written byte. The adapter stream forwards **both** teardown paths — `close` and `abort` — so a delegate failure reaches the caller's sink rather than stopping at the decorator, and `writeTo`'s own `catch` releases the writer when a delegate refuses before touching the adapter at all | +| BODY-18 | MUST | The tap clears at the start of every write | ✅ | Task 10 — asserted across two writes of a replayable delegate, which is what a Phase 7 retry loop does | +| BODY-19 | MUST | Tap capacity cap; the full payload is unaffected by it | ✅ | Task 10; cap of 0, cap below payload, and cap above payload all asserted | +| BODY-20 | SHOULD | A partial failure still yields the bytes mirrored up to that point | ✅ | Task 10 — mirror-before-forward, so the chunk that failed is captured | +| BODY-21 | MUST | The materialized form stays wrapped and keeps its tap cap | ✅ | Task 10. `materialize()` is a member, not the free function, because the return type must stay `LoggedBody`. Each wrapper gets its **own** tap buffer rather than sharing one: two live wrappers over a single `ByteQueue` means the materialized wrapper's `BODY-18` clear-on-write silently rewrites the preview the pre-materialization wrapper still holds | +| BODY-37 | MUST | No writable-buffer escape hatch; `snapshot()` is the only read path | ✅ | Task 10 — the tap is closure-scoped, asserted absent from the wrapper's own keys. Restates `IO-28` at this layer | + +## 6.4 Response-body logging + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| BODY-22 | MUST | Lazy; the delegate is drained exactly once | ✅ | Task 11 — the drain is memoized on one in-flight promise. `snapshot()` is a trigger alongside `read()`, and being synchronous it starts the drain and returns what has been captured so far rather than awaiting it, which is what lets `BODY-26`'s "snapshot returns the partial bytes without throwing" hold | +| BODY-23 | MUST | Fits-cap: full capture, every later read a fresh non-consuming view | ✅ | Task 11 | +| BODY-24 | MUST | Exceeds-cap: prefix then live tail, exactly once; a second read fails | ✅ | Task 11. The tail is pull-driven, one chunk per `pull()` — an eager `start()` loop would materialize the whole remainder of exactly the oversized bodies the cap exists to keep off the heap | +| BODY-25 | MUST | A zero-byte delivery is never treated as end-of-stream | ✅ | Task 11. `ReadableStreamDefaultReader.read()` carries no requested count, so the clause has no *literal* analog — but the tolerant reading made the same upstream succeed or fail depending only on which wrapper it passed through, since `RetentionWindow` raises on the same input under the identically-worded `IO-17`. Enforced on **both** read paths, the drain and the exceeds-cap tail | +| BODY-26 | MUST | A drain failure is cached, not propagated-and-forgotten; partial capture is never presented as complete | ✅ | Task 11 — `read()` re-throws on every call, `snapshot()` returns the partial bytes without throwing, `error()` surfaces it **without triggering a drain**. All three asserted | +| BODY-27 | MUST | The delegate is closed at most once across every close path | ✅ | Task 11 — one close-once guard shared by the wrapper's own close and the one-shot tail's completion and cancel. The reader's lock is released before the cancel, same trap as `Response.bytes` | +| BODY-28 | MUST | The captured buffer survives close | ✅ | Task 11 — depends on `IO-42`'s explicit in-memory carve-out, which is why Phase 3a's `ByteQueue.close()` deliberately leaves its read surface usable | +| BODY-29 | SHOULD | Reported length is the captured size only when the whole body fit | ✅ | Task 11 — the delegate's declared length otherwise, since the capture is only a bounded prefix | + +## 6.5 Error bodies and caps + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| HTTP-52 / BODY-30 | MUST | 4xx/5xx buffered up to a fixed 1 MiB and re-served replayably; buffering inside the close-guaranteeing scope | ✅ | Task 12. The loop keeps draining past the cap so the connection is still released, and drops the excess | +| BODY-31 | MUST | Error statuses only; a non-error response is handed back with its body intact | ✅ | Task 12 — gated on `Status.isError` (`HTTP-11`'s 400–599 band), **not** a bare `code < 400`, which would sweep a non-standard 6xx that `HTTP-10` requires `Status.of` to accept into the error path and consume a body `BODY-31` says must be returned intact | +| BODY-32 | MUST | Every byte-capped capture operation validates its cap | ✅ | Tasks 10 and 11 — both `tapCapBytes` and `capBytes` reject a negative value and clamp to the platform max. An unvalidated negative cap makes `size < cap` permanently false and silently mirrors nothing. The capless-snapshot clause is inherited: every `snapshot()` here delegates to `ByteQueue.snapshot()`, which already raises `AllocationLimitError` over the platform max (`IO-9`) | +| BODY-33 | SHOULD | Non-consuming error-body preview | ✅ | Task 12 — served from the buffered copy, so it is repeatable; `null` for no body. Decodes with the response's declared charset falling back to UTF-8, and never raises a `RangeError` out of a method on an error object | +| BODY-34 | MUST | One shared preview-size cap | ✅ (parameter) / ⏳ Phase 7 (value) | Tasks 10 and 11 each take it as a parameter; Phase 7 supplies the single config value that feeds both when it wires a real `Logger`. `toHttpError`'s 1 MiB cap is explicitly **not** this cap — `HTTP-52` fixes its value, so it cannot be the configurable one | + +## Cross-cutting plan obligations + +| Obligation | Source | Status | Where | +|---|---|---|---| +| No runtime dependency added | `SEAM-1` | ✅ | `verify:seam-1`; `dependencies` stays `{}` | +| No `node:` import in core | `sdk-design/03` §3.1 | ✅ | `grep -rn "from 'node:" packages/core/src/` is empty. Web Crypto, `TextEncoder`/`TextDecoder` and Web Streams are platform globals | +| The published `.d.ts` compiles for a consumer | `NFR-10`; new `verify:consumer-types` gate | ✅ | Compiles a throwaway consumer against the built declarations using the `lib`/`target` read from `tsconfig.base.json`, with `types: []`. Added because a real defect cleared every other gate: `typecheck` passes on dev-only ambient globals, `build` emits regardless, `api` only compares a report, `lint:publish` checks resolution and export shape, and `verify:dual-consumption` runs `node`, not `tsc`. Verified to fail on the reintroduced defect | +| Nothing from `src/io/` enters the public surface | Phase 3a's open promotion question | ✅ | Answered by `writeTo` taking the platform `WritableStream`: `io/` stays `@internal` indefinitely. The API report carries no `ByteQueue`/`BufferedSource`/`BufferedSink`/`TeeSink`/`IoError`, and neither logging tee | +| `http/` does not import `io/` | Plan Global Constraints | ✅ | `body/` is `io/`'s only new consumer. Of the *stream-shaped* types the constraint names, it takes `ByteQueue` alone — never `BufferedSource`/`BufferedSink`/`TeeSink`, whose reader/writer-bound, `ByteQueue`-and-count-shaped signatures do not compose with `writeTo`'s chunk-shaped sink. It additionally imports two `io/` error leaves and `MAX_BYTE_ARRAY_LENGTH`, which the constraint does not restrict and which exist precisely to be reused rather than duplicated. `http/request.ts`'s `Body` import is type-only and erases | +| Every public symbol documented | CLAUDE.md; `api-extractor` | ✅ | `packages/core/etc/core.api.md` contains **zero** `(undocumented)` markers. This regressed to 62 during implementation — `Response`'s wholesale rewrite also dropped 11 of Phase 1's own TSDoc blocks — and is now mechanically clean | +| Property tests where invariants exist | styleguide 11.5 | ✅ | Task 5 (`materialize` concurrency), Task 6 (framing length ×1, header injection ×1), Task 10 (tap independence), Task 11 (two-regime completeness) | +| Assertion density | styleguide ch05 §5.7 | ✅ | `invariant` preconditions on both caps and on `contentLength`; postconditions in `materialize`, `computeContentLength`, `drainOnce`, `toHttpError`'s buffer loop, and the form encoder | +| Negative space and cleanup | styleguide 11.9, 13.9 | ✅ | Double-close on `Response` and the response wrapper; write-after-consumed; read-after-tail-consumed; a delegate failure aborting rather than closing the primary sink; a caller stream that must not be cancelled | +| Every test file cites its requirement IDs | Phase 1 convention, for Phase 9 | ✅ | Top-of-file comment in all ten `body/` test files, both modified `http/` ones, and the `io/` files this phase touched | +| SPDX header on line 1 | `NFR-13` | ✅ | Every file under `packages/core/src/`. `http/response.test.ts` lost it in Task 8's wholesale rewrite and has it back | +| 80% aggregate coverage floor | `NFR-5` | ✅ | Well above; `bun test` runs coverage by default, and the threshold is enforced rather than merely reported — raising it to `0.999` makes the identical suite exit 1 | +| The body surface runs on Node, not only Bun | checkpoint §5.9; roadmap E5 | ✅ | `test/node-conformance/body-lifecycle.test.mjs` exercises `Response.bytes`/`text`/`close`'s reader-lock discipline, `StreamBody`'s `preventCancel` ownership, multipart framing including the Web Crypto boundary, and `toHttpError` buffering against the **built** artifact under `node --test`. Added when E5 was closed; this phase's surface is a founding member of that suite because §6 is where Web Streams semantics first reach a consumer | +| Changeset committed | Consumer-facing change | ✅ | `RequestBuilder.body` and `ResponseBuilder.body` both narrow from `unknown`, which `api-design.md` classes as breaking. Released as **minor** under semver's 0.x initial-development carve-out, with the pointer recorded — this is D1's decision, taken | + +## Deferred out of this phase + +| Item | Target | Note | +|---|---|---| +| `FileBody` (`HTTP-40`/`BODY-11`/`BODY-12`/`BODY-13`/`BODY-36`) | Phase 8a | Needs `node:fs`. Already in the roadmap's Deferred Items Log | +| `BODY-34`'s single shared preview-cap **value** | Phase 7 | This phase ships both tees' parameters; Phase 7 owns the `Logger`/config surface that threads one value through them | +| `BODY-4`/`BODY-5` replayability **consultation** | Phase 5 | Retry, redirect, and auth read the property this phase guarantees | +| Wiring either logging tee to a real `Logger` | Phase 7 | Mechanism ships now because the IDs are §6; nothing constructs one yet. Matches Phase 2 shipping `Serde<T>`'s interface with no implementation | +| `[Symbol.asyncDispose]` on `Response`, `LoggedResponseBody`, `Transport`, and Phase 3a's four resource owners | Checkpoint §5.4 | Blocked on the `engines.node` floor bump. Must land on all seven at once — see the ledger row | +| Removing `DomainModelError` as a class tier | Checkpoint §5.2 | A breaking change to a barrel-exported class. `io/`'s leaves are already flat; the Phase 1 tier is the residual | diff --git a/docs/superpowers/specs/2026-07-25-phase3b-body-lifecycle-design.md b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md similarity index 67% rename from docs/superpowers/specs/2026-07-25-phase3b-body-lifecycle-design.md rename to docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md index 333f673..b464859 100644 --- a/docs/superpowers/specs/2026-07-25-phase3b-body-lifecycle-design.md +++ b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md @@ -7,16 +7,16 @@ planning branch), both open in the roadmap's "Open Findings — Phase 3b Validat **Purpose:** Implement the request/response body lifecycle — body production and replayability, materialize-once, response single-use/close, the lazy parsed-response wrapper, request/response body-logging tees, and bounded error-body buffering — on top of a tested and frozen Phase 3a. This is the second half of Phase 3 of the -[v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md). +[v1 roadmap](../../2026-07-23-nodejs-sdk-v1-roadmap-design.md). **Scope:** every requirement in `docs/product-spec/06-request-and-response-body-lifecycle.md` — `BODY-1` through `BODY-37`, `HTTP-36` through `HTTP-52` — is dispositioned here, except the file-backed-body cluster (`HTTP-40`/`BODY-11`/`BODY-12`/`BODY-13`/`BODY-36`), deferred to Phase 8 (see "Explicitly Out of Scope"). **Governing documents:** `docs/product-spec/06-request-and-response-body-lifecycle.md` (normative, cited by ID -throughout), `docs/sdk-design-nodejs/03-seam-by-seam-idiomatic-mapping.md` §3.1, `docs/superpowers/specs/2026-07-24-phase3a-io-contracts-design.md` -(the frozen surface this phase builds on and, in one place, retrofits), `docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md` -(the two-level error-hierarchy rule this phase's error tree follows), and `docs/superpowers/specs/2026-07-23-phase1-core-http-domain-model-design.md` +throughout), `docs/sdk-design-nodejs/03-seam-by-seam-idiomatic-mapping.md` §3.1, `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md` +(the frozen surface this phase builds on and, in one place, retrofits), `docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md` +(the two-level error-hierarchy rule this phase's error tree follows), and `docs/work/mvp/phase1/2026-07-23-phase1-core-http-domain-model-design.md` (the `Request`/`Response` classes whose `unknown` body placeholder this phase replaces). Styleguide: `styleguide/typescript/` chapters 05, 06, 08, 09, 10, 11, 12, 13, 15. @@ -52,7 +52,7 @@ Everything else in `§6` ships in this phase. | BODY-17, BODY-18, BODY-19, BODY-21, BODY-37 | MUST | `withRequestLogging` tee decorator over `Body` — a self-contained tee reusing only `ByteQueue`, **not** Phase 3a's `TeeSink` class (whose `ByteQueue`-and-count signature does not compose with `writeTo`'s chunk-shaped sink; see the section below) | | BODY-20 | SHOULD | Partial-failure snapshot returns bytes mirrored up to the failure | | BODY-22, BODY-23, BODY-24, BODY-27, BODY-28 | MUST | Response-body logging wrapper, two regimes (fits-cap capture vs. exceeds-cap prefix+tail), shared close-once guard | -| BODY-25 | MUST | **Structurally inapplicable on Node** — the wrapper reads through a `ReadableStreamDefaultReader`, which has no requested-count parameter, so "returns zero for a positive requested count" has no analog; a zero-length chunk is not an EOS signal and is captured as-is, with EOS signalled only by `{done: true}`. Ledgered | +| BODY-25 | MUST | Implemented: a zero-length delegate chunk raises `SourceContractViolationError`, matching `RetentionWindow` under the identically-worded `IO-17`. `ReadableStreamDefaultReader.read()` carries no requested count, so the clause has no *literal* analog — but a response body reaches both this tee and `BufferedSource`, and the tolerant reading made the same upstream succeed or fail depending only on which wrapper it passed through. Ledger entry withdrawn (review finding, 2026-08-24) | | BODY-26 | MUST | `LoggedResponseBody.error(): Error \| null` — the drain failure is cached in the wrapper's closure; `read()` re-throws it on every call, `snapshot()` returns the partial bytes without throwing, and `error()` surfaces it **without triggering a drain** | | BODY-29 | SHOULD | `LoggedResponseBody.contentLength` — the captured size in the fits-cap regime, the delegate's declared length otherwise (the capture is only a bounded prefix) | | HTTP-52 / BODY-30, BODY-31 | MUST | `toHttpError(response)` — 1 MiB fixed cap, 4xx/5xx only, buffering inside the response's own close-guaranteeing scope | @@ -147,17 +147,30 @@ class Response { readonly body: ReadableStream<Uint8Array> | null; // single-use (BODY-14) text(): Promise<string>; // BODY-16, HTTP-42 bytes(): Promise<Uint8Array>; // BODY-16 - close(): Promise<void>; // BODY-15, HTTP-43 - [Symbol.asyncDispose](): Promise<void>; // delegates to close() — checkpoint §5.4's now-bumped floor applies + close(): Promise<void>; // BODY-15, HTTP-43 — the only teardown interface; see below } ``` -`Response` owns a resource (the body's connection) it releases via `close()`, so the checkpoint's §5.4 fix — already a -prerequisite of this phase, floor bumped and `lib` extended before 3b starts — applies here too: -`[Symbol.asyncDispose]` is the primary teardown interface, `close()` a retained delegate, so `await using response = -...` works. The response-body logging wrapper (below) gets the same treatment for the same reason; `Body` itself -does not, since no variant in this phase owns a closeable resource it must release (`StreamBody` is explicitly -caller-owned, per `BODY-8`). +**Teardown is `close()` only — no `[Symbol.asyncDispose]`.** This design was written assuming the checkpoint's §5.4 +fix (floor bump to the first Node release exposing `Symbol.dispose`/`Symbol.asyncDispose`, plus the matching `lib` +entry) had already landed as a prerequisite. It had not: at implementation time `engines.node` was still `">=18.17"` +and `tsconfig.base.json`'s `lib` was `["ES2022", "DOM", "DOM.AsyncIterable"]`, with none of Phase 2's `Transport` or +Phase 3a's `ByteQueue`/`BufferedSource`/`BufferedSink`/`RetentionWindow` carrying the symbol. Shipping it on +`Response` alone would have meant: + +- **A run-time trap on the declared floor.** `Symbol.asyncDispose` evaluates to `undefined` below Node 18.18, so the + computed key binds the method to the string `"undefined"` — precisely the failure Phase 3a's design named when it + declined the symbol, and it fails silently at run time, not at build time. +- **A broken published `.d.ts`.** The symbol's *type* reaches this package only through a dev-only global. A consumer + compiling against the built package on the same `lib` this repo declares gets + `TS2550: Property 'asyncDispose' does not exist on type 'SymbolConstructor'`. No gate covers this — + `verify:dual-consumption` runs `node`, not `tsc`. +- **An inconsistent taxonomy.** Two of seven resource-owning classes would have it and five would not. + +So this phase keeps Phase 3a's shipped decision, and both `Response` and the response-body logging wrapper assert the +absence rather than leaving it implicit. Adding it back is checkpoint §5.4's job, in one pass across all seven owners, +once the floor actually moves. Ledgered below. `Body` itself never needed it: no variant in this phase owns a +closeable resource (`StreamBody` is explicitly caller-owned, per `BODY-8`). Mirrors the `writeTo` decision: the public surface is the platform `ReadableStream`, not an internal wrapper. `text()`/`bytes()` drain the reader with a plain manual chunk-accumulate-and-concatenate loop — **no `io/` @@ -250,7 +263,7 @@ way to read the tap. **Response-body logging wrapper (`BODY-22`–`29`):** ```typescript -interface LoggedResponseBody extends AsyncDisposable { +interface LoggedResponseBody { // close() only — same reason as Response, above read(): Promise<ReadableStream<Uint8Array>>; snapshot(): Uint8Array; // non-consuming; partial bytes even after a failed drain (BODY-26) error(): Error | null; // the cached drain failure, WITHOUT triggering a drain (BODY-26) @@ -318,7 +331,8 @@ third instance of the same violation: ``` DexpaceError (Phase 2 root) -├── RequiredFieldError, HeaderValidationError, … (Phase 1, flattened per checkpoint §5.2) +├── DomainModelError (Phase 1 — checkpoint §5.2 has NOT run; still a +│ └── RequiredFieldError, HeaderValidationError, … class tier. See the note below the diagram) ├── CancellationError, OperationAssemblyError (Phase 2, already flat) ├── IoError (Phase 3a — unchanged; already a flat leaf, used bare │ at 4 sites in buffered-source/-sink.ts, tee-sink.ts @@ -332,6 +346,19 @@ DexpaceError (Phase 2 root) └── HttpStatusError (3b, new — BODY-30/31) ``` +**The Phase 1 tier is still three deep.** This section assumed checkpoint §5.2 had already removed +`DomainModelError` as a class tier. It has not run, so after this phase's retrofit the taxonomy is *mixed*: +`DexpaceError → EndOfStreamError` is two levels while `DexpaceError → DomainModelError → RequiredFieldError` is +still three. That is strictly better than before — the `io/` leaves no longer add a *second* independent violation — +but it is a real residual, and it is deliberately **not** fixed here: removing `DomainModelError` deletes a class +exported from the public barrel that consumers can `instanceof`, which is a breaking API change belonging to +checkpoint §5.2, not to a body-lifecycle phase. Ledgered below and owned by the checkpoint (roadmap finding E2). + +A second checkpoint item lives in the same file and should be done in the same pass: §5.3 requires every error +leaf to carry its identifying inputs as sanitized `readonly` fields, and it was applied to two of the ten Phase 1 +leaves and stopped (roadmap finding E3). Whoever opens `http/errors.ts` for the flattening is already touching +every class E3 names. + Grouping is restored the way the checkpoint prescribed, and lands on the lighter of its two sanctioned options: an exported type-guard union per category (`isIoError(e): e is IoError | EndOfStreamError | ...`, `isBodyError(e): e is ConsumedBodyError | MultipartBoundaryError`) rather than a `kind` discriminant field on `DexpaceError` — the @@ -348,10 +375,19 @@ Phase 3a's design doc left open whether `§5` would be promoted to the public ba What *does* go public, for the first time since Phase 2: the `Body` interface, the *types* of its concrete variants and their factory functions, `MultipartPart`, `MultipartBodyBuilder`, `materialize`, `TypedResponse`, -`HttpStatusError`/`toHttpError`, and the two new error leaves a caller can actually trigger and -needs to catch — `ConsumedBodyError` (double-write on a single-use body) and `MultipartBoundaryError` (an invalid -caller-supplied boundary) — matching Phase 1's precedent that domain-model validation errors are public, not -internal. The logging tees and `toHttpError`'s internals stay `@internal` (unwired until Phase 7). +`HttpStatusError`/`toHttpError`, and the error leaves a caller can actually trigger and +needs to catch — `ConsumedBodyError` (double-write on a single-use body), `MultipartBoundaryError` (an invalid +caller-supplied boundary), and `FormBodyValidationError` (a form field that cannot be rendered) — matching Phase 1's +precedent that domain-model validation errors are public, not internal. `formUrlEncodedBody`'s input widened past +this design's `ReadonlyMap<string, string>` during implementation, which brings `FormUrlEncodedInput` and +`FormUrlEncodedValue` public alongside it; both are ledgered below. + +**Every public symbol carries a TSDoc block**, including each member of each exported class and interface. +`api-extractor` records an undocumented reachable member as `(undocumented)` in the committed report, so the +enforcement point is mechanical: `packages/core/etc/core.api.md` must contain zero occurrences of that marker, and +`bun run api` fails CI on any drift. It is also the gate for `HTTP-2`: a `constructor(...)` line appearing under a +class the barrel exports **as a value** means a public field-wise constructor escaped the builder, and the private +constructor plus its `createX` friend hook is what keeps it out. The logging tees and `toHttpError`'s internals stay `@internal` (unwired until Phase 7). **Two consequences a barrel edit alone won't enforce.** First, the concrete body *classes* are exported as types only, never as values: exporting the class exposes `new ByteArrayBody(...)` as a public field-wise constructor, @@ -379,14 +415,24 @@ packages/core/src/body/ request-body-logging.ts # withRequestLogging tee decorator (@internal) response-body-logging.ts # response-body logging wrapper, two regimes (@internal) http-status-error.ts # HttpStatusError, toHttpError() - errors.ts # ConsumedBodyError, MultipartBoundaryError + errors.ts # ConsumedBodyError, MultipartBoundaryError, FormBodyValidationError + write-body.ts # withBodyWriter — the one writer scope every variant shares + media-type-safety.ts # header-safety validation for a Body's media type + freeze-body.ts # freezeBody — HTTP-1's freeze, single-sourced index.ts # barrel — Body/variants/factories/TypedResponse/HttpStatusError/ConsumedBodyError/ - # MultipartBoundaryError re-exported from src/index.ts; logging tees stay internal-only + # MultipartBoundaryError/FormBodyValidationError re-exported from src/index.ts; + # logging tees stay internal-only ``` +The last three files are not in this design's original plan; each earned its place during implementation and is +ledgered below. They exist as named modules rather than inlined code for the same reason `io/limits.ts`'s +`assertAllocatable` does: each encodes a rule applied at four or five call sites, and a rule applied in five shapes +is a rule that drifts. + Also modifies (not creates): `packages/core/src/http/request.ts` and `response.ts` (real body types replace -Phase 1's `unknown` placeholder; `Response` gains `text()`/`bytes()`/`close()`), and `packages/core/src/io/errors.ts` -(the flattening retrofit above). +Phase 1's `unknown` placeholder; `Response` gains `text()`/`bytes()`/`close()`), a new +`packages/core/src/http/charset.ts` (`HTTP-42`'s charset resolution, shared by `Response.text()` and +`HttpStatusError.preview()`), and `packages/core/src/io/errors.ts` (the flattening retrofit above). ## Deviation Ledger (for Phase 10) @@ -398,9 +444,26 @@ Phase 1's `unknown` placeholder; `Response` gains `text()`/`bytes()`/`close()`), | Both logging tees are new, self-contained implementations, not built on Phase 3a's `TeeSink`/`BufferedSource` | none — forced by the `writeTo` decision above | `TeeSink`/`BufferedSource`/`BufferedSink` are reader/writer-bound with `ByteQueue`-and-count-shaped signatures; `Body.writeTo`'s chunk-shaped `WritableStream<Uint8Array>` doesn't compose with them without rewriting Phase 3a's frozen surface. Only `ByteQueue` (pure in-memory, unbound to a stream shape) is reused | | Phase 3a's `IoError` tier flattened in this phase, not in 3a itself | phase-boundary discipline (each phase's own frozen surface) | The checkpoint's `§5.2` fix for `DomainModelError` missed the identically-shaped `IoError` tier; carrying the inconsistency forward into a fourth phase was judged worse than a scoped retrofit here | | Logging tees and `toHttpError`'s preview machinery shipped `@internal`, unwired to any `Logger` | none — matches Phase 2's `Serde<T>` precedent | No `Logger`/config surface exists until Phase 7 | -| `BODY-25`'s zero-byte-read-for-a-positive-count clause not implemented | `BODY-25` (MUST) | Structurally inapplicable: `ReadableStreamDefaultReader.read()` takes no requested count, so the failure mode has no analog. EOF is signalled only by `{done: true}`, which is what the drain loop keys on, so the silent truncation `BODY-25` guards against cannot arise | | `BODY-34`'s shared preview cap covers the two logging tees only, not `toHttpError` | `BODY-34` (MUST), read literally as "all three" | `HTTP-52` *fixes* the error-body cap at 1 MiB, so it cannot also be the configurable shared value. The two capture sites `BODY-34` actually names — request-side tee and response-side drain — do share one cap | | Concrete `Body` classes exported from the public barrel as types only, never as values | none — required by `HTTP-2` | Exporting the class as a value publishes a field-wise constructor, which `HTTP-2` forbids; the factory functions are the sanctioned construction path and the classes remain usable as type annotations | +| `close()` only, no `[Symbol.asyncDispose]`, on `Response` and `LoggedResponseBody` | this design's own §"Response Body", which assumed checkpoint §5.4 had landed | The checkpoint has not run: `engines.node` is still `">=18.17"` and `lib` carries no `esnext.disposable`. Below Node 18.18 the computed key evaluates to `undefined` and binds the method to the string `"undefined"`; and the symbol's type reaches this package only through a dev-only global, so a consumer compiling against the published `.d.ts` on this repo's own declared `lib` fails with `TS2550`. Two of seven resource owners would have carried it. Matches Phase 3a's shipped decision; **owned by checkpoint §5.4**, to be added to all seven at once | +| Phase 1's `DomainModelError` tier left three-deep while `io/`'s leaves are flattened | checkpoint §5.2, which this design's error-tree diagram assumed had already run | Removing `DomainModelError` deletes a barrel-exported class consumers can `instanceof` — a breaking API change that belongs to the checkpoint, not to a body phase. The residual is a *mixed* taxonomy, strictly better than the two independent violations that preceded it. **Owned by checkpoint §5.2** | +| `write-body.ts` — one shared `withBodyWriter` scope for all five variants | this design's File Layout, which had each variant close its own sink | The naive `try { … } finally { await writer.close() }` is wrong twice: closing an already-errored writer rejects with `TypeError`, and a throwing `finally` *replaces* the in-flight exception, destroying the real cause (`RECOV-12`) — which also declassifies the failure for `RETRY-2`'s cause-chain walk. Aborting rather than closing on failure additionally tells the transport the message is broken, where a clean close would signal a complete body that was never written | +| `media-type-safety.ts` — `Body.mediaType` validated as header-safe at construction | none — closes a `HTTP-51` header-injection hole this design did not anticipate | `mediaType` is interpolated verbatim into a multipart part header, so a CR/LF in it can append arbitrary headers, close the header block, and forge a closing boundary — while the shared framing routine keeps the declared length consistent with the corrupted bytes. Uses the same predicate as outbound header-value validation (`HTTP-26`). `headerSafeMediaType` is the inbound counterpart: `HTTP-19` lets a received `content-type` carry obs-text that `HTTP-18` forbids outbound, so `HttpStatusError.body()` drops it rather than raising from an accessor on an error object | +| `freeze-body.ts` — every `Body` variant frozen at construction | this design, which specified no freeze | `readonly` is erased at run time, so a caller could reassign `contentLength` after construction and desynchronize the value a transport stamps into `Content-Length` from the bytes `writeTo` emits. That is the same drift `HTTP-51` makes `MultipartBody` share one framing routine to prevent and `HTTP-1`/`XCUT-15` make it defensively copy its parts for — left open one level up. Matches the `Object.freeze(this)` step every `packages/core/src/http/` model already performs | +| `http/charset.ts` — `HTTP-42`'s charset resolution extracted and shared | this design, which put the charset logic inside `Response.text()` | `HttpStatusError.preview()` needs the identical resolve-then-fall-back-to-UTF-8 rule (`BODY-33`), and two copies of a fallback chain drift. Named `decodeBodyText`, **not** `decodeText`: `io/text-codec.ts`'s `decodeText` deliberately implements true ISO-8859-1 and sets `ignoreBOM` for per-fragment decoding (`IO-13`, `SSE-12`), while this one delegates to `TextDecoder`'s windows-1252 mapping and consumes a leading BOM, which is right for a whole message body. The two are not interchangeable and the names now say so | +| `FormBodyValidationError` public, and `formUrlEncodedBody` accepts `FormUrlEncodedInput`/`FormUrlEncodedValue` | this design's `formUrlEncodedBody(params: ReadonlyMap<string, string>)` | A field value that is neither a primitive nor `null` cannot be rendered; dropping it silently puts an incomplete body on the wire, so it is raised naming the key. Widening the input to `QueryParams`, a map, a record, or entry pairs reuses Phase 1's `QueryParams` encoder rather than a second hand-rolled one, which is also what makes the `+`-for-space rule single-sourced (`HTTP-38`/`BODY-35`) | +| `StringBody.mediaType` defaults to `text/plain; charset=utf-8`; `StringBody.text` and `FormUrlEncodedBody.params` are public fields | this design, which defaulted `mediaType` to `undefined` and exposed neither field | The default states the encoding `writeTo` actually emits instead of leaving a text body with no declared type. The two readable fields are the non-destructive way to inspect a body a caller already holds — the alternative is draining it, which for a `Body` is the one thing an inspection must not do | +| `Response.close()` memoized on a promise rather than a boolean flag | this design's `#closed = false` sketch | A flag set before the `await` reports a FAILED release as success to every later caller, over a connection that was never released. Handing every caller the same promise propagates the failure on every path while still cancelling at most once — the shape `BufferedSink.close()` already settled on for `IO-5`/`IO-41` | +| `TypedResponse`'s raw fields are getters, not constructor-assigned `readonly` fields | this design's class sketch | Same observable surface; delegating to the wrapped `Response` keeps the two from being able to disagree. `value()` additionally wraps the parse in an `async` IIFE so a parser that throws *synchronously* is still memoized — a bare `??=` never completes the assignment in that case and re-runs the handler against a single-use body whose bytes are gone, which `HTTP-44` forbids | +| `assertCount` hoisted into `io/limits.ts`, and `TeeSink.write` gained it | Phase 3a's frozen surface | `IO-3`'s guard existed as three byte-for-byte copies (`byte-queue.ts`, `buffered-source.ts`, `buffered-sink.ts`) and `TeeSink` — the fourth size-taking surface — had none, so a negative count reached it and was rejected only indirectly, by whichever `ByteQueue` call happened to run first, and not at all on its `count === 0` and short-source early returns. Behavior-preserving for the three that had it | +| `BODY-25`'s zero-chunk check applied on the exceeds-cap tail path too, not only in `drainOnce` | the first implementation of this design's two-regime wrapper | A rule enforced in one regime and not the other makes the same violating upstream pass or fail depending only on how big the body happened to be | +| `MultipartBody.writeTo` verifies the bytes it writes against its own declared `contentLength` | this design, which treated the shared framing routine as sufficient | The shared routine keeps the *framing* consistent but takes each part's own `contentLength` on trust — and `MultipartPart.body` is the `@public` `Body` interface, so a caller-supplied implementation can report one length and write another. Measured drift on a one-part body: declared 59, written 63. A bounded writer now refuses a chunk that would carry the message past the declared total (early, for the reason `StreamBody.#writeExactly` checks early) and a short total raises inside the writer scope so the sink is aborted rather than cleanly closed | +| A caller-supplied multipart boundary is validated for grammar only, not for non-appearance in part content | `HTTP-51`'s "caller-boundary validation", read as covering RFC 2046's full sender obligation | RFC 2046 puts two duties on the sender: a `bchars`-valid delimiter, and one that appears in no part. Only the first is checkable here — a `StreamBody` part's bytes do not exist until the write, so a scan would be a *partial* check that reads as complete, which is worse than a stated limitation. Mitigated where it matters: the default boundary is 32 random characters from Web Crypto and is generated unless the caller opts out, and both entry points now document the obligation a caller-supplied delimiter carries. A caller who supplies a guessable boundary alongside attacker-influenced part content can have that content forge a closing delimiter | +| `Response.bytes()`/`text()` and `toHttpError` acquire the body reader *inside* the try | the first implementation, which acquired it above | `getReader()` itself throws a `TypeError` when an external consumer already holds the lock, and `BODY-15` forbids assuming the body was never touched — so the one failure `BODY-16`'s close guarantee most needs to cover was the one that skipped the close entirely and held the connection | +| The request-logging tee closes the primary sink when a delegate resolves without closing the adapter | none — closes a hole in this design's own decorator | `Body.writeTo`'s contract is that the body closes the sink it was given, and this wrapper is the only place that takes a writer on behalf of someone else's `Body`. A delegate that just resolves would strand the caller's sink open and locked with nothing thrown to notice it by | +| A foreign primitive source that over-reports its transferred count raises `SourceContractViolationError`, not `EndOfStreamError` | Phase 3a's `factories.ts`, which left the over-report direction to `ByteQueue.takeBytes` | It surfaced as `end of stream: delivered 2 of 99 bytes` — reporting a foreign source's broken accounting as an exhausted stream, which is the exact confusion `IO-17` forbids. The under-report direction already had `assertDrained`; the file's own comment claimed both were covered | +| New blocking gate `verify:consumer-types` | none — this is the gate whose absence let the `Symbol.asyncDispose` defect ship | It compiles a throwaway consumer against the built `.d.ts` using the `lib`/`target` read from `tsconfig.base.json`, with `types: []`. `typecheck` passes on dev-only ambient globals, `build` emits regardless, `api` only compares a report, `lint:publish` checks resolution and export shape rather than whether declarations resolve, and `verify:dual-consumption` runs `node`, not `tsc`. Verified to fail on the reintroduced defect and pass once reverted | ## Testing diff --git a/docs/superpowers/plans/2026-07-25-phase3b-body-lifecycle.md b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle.md similarity index 99% rename from docs/superpowers/plans/2026-07-25-phase3b-body-lifecycle.md rename to docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle.md index 7aa09ea..aa73152 100644 --- a/docs/superpowers/plans/2026-07-25-phase3b-body-lifecycle.md +++ b/docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle.md @@ -3,7 +3,7 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. > **⚠ Two open items before this plan is fully executable** — see "Open Findings — Phase 3b Validation Review -> (2026-07-28)" in [the roadmap](../specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md). +> (2026-07-28)" in [the roadmap](../../2026-07-23-nodejs-sdk-v1-roadmap-design.md). > > - **D1 (Task 13 Step 6):** the changeset bump is **major** by default, because narrowing `RequestBuilder.body` > from `unknown` to `Body | undefined` is a breaking parameter change. If the repo's release policy treats 0.x @@ -17,7 +17,7 @@ `Request`/`Response`'s real body types, `TypedResponse<T>`, request/response body-logging tees, and bounded error-body buffering — satisfying `product-spec/06-request-and-response-body-lifecycle.md` (`BODY-1`–`BODY-37`, `HTTP-36`–`HTTP-52`, minus the file-backed-body cluster deferred to Phase 8), per -`docs/superpowers/specs/2026-07-25-phase3b-body-lifecycle-design.md`. +`docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md`. **Architecture:** A new `packages/core/src/body/` folder plus surgical retrofits to two already-shipped files: `packages/core/src/io/errors.ts` (flattening a leftover 3-tier error shape) and `packages/core/src/http/request.ts` @@ -32,7 +32,7 @@ already relied on elsewhere per `sdk-design-nodejs/10`'s SHA-256-via-`crypto.sub new runtime dependencies — `SEAM-1` untouched. **Prerequisite:** This plan assumes Phases 0, 1, 2, and 3a are already implemented exactly as their own plans -specify, **and** the checkpoint (`docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md`) has +specify, **and** the checkpoint (`docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md`) has signed off — concretely: `packages/core/src/http/*`, `seams/*`, `io/*` exist; `DexpaceError` is the flat taxonomy root with `DomainModelError` already removed as a class tier (checkpoint §5.2); `engines.node` and `tsconfig.base.json` `lib` are already bumped for `Symbol.dispose`/`Symbol.asyncDispose` (checkpoint §5.4), and `Transport` plus every @@ -2989,12 +2989,19 @@ bun test --coverage bun run api bun run lint:publish bun run verify:dual-consumption +bun run verify:consumer-types bun run verify:seam-1 -bun run verify:node-floor +bun run verify:runtime-floor bun run test:node bun run audit ``` +> **Corrected 2026-08-26.** This sequence originally called `bun run test:node`, which did not exist — the +> step could not be executed as written (roadmap finding E5). The script now exists, and +> `bun run verify:node-floor` has been removed from the list because checkpoint §5.9 folded that script's two +> `AbortSignal.any` assertions into the conformance suite rather than keeping two parallel Node entry points. +> `verify:consumer-types` and `verify:runtime-floor` are added because both are blocking CI steps. + Expected: all exit 0. Coverage at or above the 80% aggregate floor (`NFR-5`). - [ ] **Step 4: Verify no `node:` import crept in** diff --git a/docs/superpowers/plans/2026-07-26-phase4-execution-context-and-pipelines-checklist.md b/docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md similarity index 86% rename from docs/superpowers/plans/2026-07-26-phase4-execution-context-and-pipelines-checklist.md rename to docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md index c4d49d1..cae3949 100644 --- a/docs/superpowers/plans/2026-07-26-phase4-execution-context-and-pipelines-checklist.md +++ b/docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md @@ -1,17 +1,18 @@ # Phase 4 (4a + 4b + 4c) — Execution Context & Pipelines — Checklist Verification of the three Phase 4 implementation plans — -[4a Execution Context](./2026-07-25-phase4a-execution-context.md), -[4b Recovery-Chain Primitives](./2026-07-25-phase4b-recovery-chain.md), -[4c Stage-Based Pipeline](./2026-07-25-phase4c-stage-pipeline.md) — against every requirement ID in +[4a Execution Context](./phase4a/2026-07-25-phase4a-execution-context.md), +[4b Recovery-Chain Primitives](./phase4b/2026-07-25-phase4b-recovery-chain.md), +[4c Stage-Based Pipeline](./phase4c/2026-07-25-phase4c-stage-pipeline.md) — against every requirement ID in `docs/product-spec/07-execution-context-model.md` (`CTX-*`) and `docs/product-spec/08-execution-pipelines.md` (`PIPE-*`, `RECOV-*`), as dispositioned by their design docs. **Legend:** ✅ Planned, implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred (named target phase) — N/A Not applicable in this port. -**Status:** the plans are reviewed and corrected as of 2026-07-26 (see *Review findings applied*, below) but -**not yet executed**. Every ✅ means "the plan builds and tests it," not "it is on `main`." +**Status:** the plans are reviewed and corrected as of 2026-07-26 (see *Review findings applied*, below). **4b +is executed as of 2026-08-26**; 4a and 4c are not. For the §8.2 table a ✅ now means "built, tested and on the +branch"; everywhere else it still means "the plan builds and tests it." --- @@ -77,7 +78,7 @@ Verification of the three Phase 4 implementation plans — | RECOV-9 | SHOULD | Recovery steps should return a Failure rather than throw | ✅ | Satisfied structurally — both shapes are handled identically, documented rather than enforced | | RECOV-10 | MUST | Unwrap: Success returns the response; Failure rethrows the throwable **unchanged** | ✅ | 4b Task 6 — asserted with `rejects.toBe(typedError)`, identity not message | | RECOV-11 | MUST | Wrapping a cancellation throwable re-asserts the cancellation signal | ✅ (reframed) | 4b Task 4. An `AbortSignal` is durable once fired and the SDK never holds the caller's `AbortController`, so there is nothing to re-assert; the helper is `failure(error)` and **never throws**, which is what keeps RECOV-2 absolute. Ledgered | -| RECOV-12 | MUST | A step throwing while holding a Success closes that response exactly once, close error `suppressed`, original primary | ✅ | 4b Task 3 (`toFailureClosingSuccess`, hand-built `SuppressedError` — never `using`, whose auto-generated one inverts the priority). Close observed via the body stream's `cancel()` hook, since `Response` is frozen | +| RECOV-12 | MUST | A step throwing while holding a Success closes that response exactly once, close error `suppressed`, original primary | ✅ | 4b Task 3 (`toFailureClosingSuccess`) over Task 1b's guarded `suppress()` — never `using`, whose auto-generated `SuppressedError` inverts the priority, and never `new SuppressedError(...)`, which is absent on the declared floor. Close observed via the body stream's `cancel()` hook, since `Response` is frozen. Re-forced from real Node in `test/node-conformance/recovery-chain.test.mjs` | | RECOV-13 | MUST | A deliberately *returned* different outcome is never auto-closed | ✅ | 4b Task 3 — only a caught throw reaches the close path; asserted for both a substitute Failure and a substitute Success | | RECOV-14 | MUST | Step lists immutable; response chain copies both | ✅ | 4b Tasks 2 and 3 — the request chain is copied too, which the reference does not do and the requirement's own text recommends. Ledgered | | RECOV-15 | MUST | Only 400..599 map to the typed exception; every other status passes through | ✅ | 4b Task 5, delegating to Phase 3b's unchanged `toHttpError()` | @@ -155,6 +156,7 @@ Verification of the three Phase 4 implementation plans — | Negative-space assertions | styleguide 11.9 | ✅ | Duplicate-key install, no-op closes, cross-stage edits, missing anchors, reserved SEND, continuation reuse, transport `close()` never called | | Options object over positional params | `max-params: 3` | ✅ | `ContextInit` (4a), `DispatchConfig` (4b), `CursorInit` (4c). No `eslint-disable` anywhere in Phase 4 | | Fakes over mocks; no owned interface mocked | styleguide 11.3 | ✅ | File-local `Transport` stubs throughout; no `FakeTransport`, no `mock.module`, and (as of the 2026-07-26 review) no patched `Response` method and no patched `contextStore` singleton | +| A runtime-divergent surface gets a `test/node-conformance/` case | `test/node-conformance/README.md` membership rule | ✅ | 4b: `recovery-chain.test.mjs` — `SuppressedError`'s presence is the divergence (Bun and current Node have it, the 20.3 floor does not), plus `RECOV-12`'s release-exactly-once over Node's own Web Streams | | Every test file cites its requirement IDs | Phase 1 convention, for Phase 9 | ✅ | Top-of-file comment in every test file across all three plans | | 80% aggregate coverage floor | `NFR-5` | ✅ | Each phase's gate task | @@ -197,6 +199,34 @@ stay accurate except where noted here. Full text in the roadmap's *Open Findings --- +## Phase 4b execution (2026-08-26) + +Both of 4b's open decisions closed before execution; neither changed a `RECOV-*` disposition above. + +| Item | Resolution | +|---|---| +| **F1 (blocker, cross-phase):** `SuppressedError` absent on the declared floor | Branch (b) — `packages/core/src/suppress.ts` ships `suppress(error, suppressed, message)`, native class where the runtime has one and a shape-compatible stand-in where it does not, global read per call. Branch (a) was disqualified on evidence: `SuppressedError` reached Node in **24.0.0**, so raising the floor means dropping Node 18, 20 and 22 for one error class, against a floor of `>=20.3` set by `AbortSignal.any()`. The helper discharges the obligation for 5a, 6a, 6b and 6c too — they substitute the call when they execute | +| **F2:** zero `invariant()` assertions across `recovery/` | Deviation Ledger row in 4b's design, naming the concrete cost (a step returning `undefined` poisons the fold silently). Project-wide inconsistency — 1/2/3b/4a ship zero, 4c ships fifteen — so Phase 10 settles the density rule once rather than 4b becoming the one module that differs | +| Merge residue | The phase-3 merge left `bunfig.toml` with a duplicated `[test] root` key, which TOML rejects — `bun test` failed to load bunfig at all on this branch. Fixed in its own commit before any 4b work | + +**Gate evidence (all exit 0), every step both CI jobs run, in order:** `bun install --frozen-lockfile`, +`typecheck`, `lint`, `build`, `bun test --coverage` (588 tests across 50 files; 98.68% funcs / 99.73% lines +against the 80% floor), `api` with `packages/core/etc/core.api.md` byte-identical, `lint:publish`, +`verify:dual-consumption`, `verify:consumer-types`, `verify:seam-1`, `verify:runtime-floor`, `audit`, and the +`node-conformance` job's `test:node` (36 cases, 35 before this phase). Also `test:scripts` (named `test:knowledge` at the time), which CI does not +run. Structural: no `node:` import, no `enum`, no `recovery/index.ts`, SPDX on line 1 of all 15 new files, no +import cycle anywhere under `packages/core/src`. + +**Three review passes ran before this was called done.** Pass 1 (corpus-driven) found a dead `satisfies` +statement reaching the published `dist/`, two test files that could not survive parallel execution, a missing +type-level test for the exported generic `Outcome<T>`, an untranscribed `RECOV-15` conformance clause, and two +step-down-rule violations. Pass 2 (normative-text-driven) found a **`RECOV-8` violation**: `apply()` could +throw a `TypeError` when a step returned a non-outcome, against "MUST NOT throw under any input" — closed by +making `toFailureClosingSuccess` total; plus an unguarded `String()` in `assertNever`'s default message. Pass 3 +re-ran every CI step and swept the structure. What survives is in `docs/work/mvp/2026-09-04-open-items-dissolution.md` under Phase 4b. + +--- + ## Deferred out of Phase 4 | Item | Target | Note | diff --git a/docs/superpowers/specs/2026-07-25-phase4a-execution-context-design.md b/docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context-design.md similarity index 100% rename from docs/superpowers/specs/2026-07-25-phase4a-execution-context-design.md rename to docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context-design.md diff --git a/docs/superpowers/plans/2026-07-25-phase4a-execution-context.md b/docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context.md similarity index 99% rename from docs/superpowers/plans/2026-07-25-phase4a-execution-context.md rename to docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context.md index 7b35814..84ff6cd 100644 --- a/docs/superpowers/plans/2026-07-25-phase4a-execution-context.md +++ b/docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context.md @@ -5,7 +5,7 @@ **Goal:** Ship the execution context promotion chain and its bounded process-wide store in `@dexpace/core` — `DispatchContext`/`RequestContext`/`ExchangeContext`, the `InstrumentationBundle` shape, and `ContextStore` — satisfying `product-spec/07-execution-context-model.md` (`CTX-1`–`CTX-20`), per -`docs/superpowers/specs/2026-07-25-phase4a-execution-context-design.md`. +`docs/work/mvp/phase4/phase4a/2026-07-25-phase4a-execution-context-design.md`. **Architecture:** A new `packages/core/src/context/` folder, layered `instrumentation` → `errors` → `context` → `store` (there is deliberately **no `index.ts`** — see Global Constraints). The three context flavors are plain frozen interfaces plus free functions — no class, since diff --git a/docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md b/docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md similarity index 82% rename from docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md rename to docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md index 00202d7..6068689 100644 --- a/docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md +++ b/docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md @@ -1,17 +1,19 @@ # Phase 4b — Recovery-Chain Primitives — Design -**Status:** Draft, approved for planning. **⛔ Two open decisions block execution** — see the blocking notice at -the top of `docs/superpowers/plans/2026-07-25-phase4b-recovery-chain.md`. In short: `RECOV-12`'s -`SuppressedError` does not exist on the declared `engines.node` floor (a cross-phase problem shared with 5a, 6b -and 6c), and this phase's zero-assertion module contradicts the corpus's 2-per-function average. Both are -tracked in the roadmap's "Open Findings — Phase 4b Validation Review (2026-07-28)" section. The rest of that -review's findings are applied to this document. +**Status:** Implemented 2026-08-26. **Both open decisions are closed.** `RECOV-12`'s `SuppressedError` is +reached through a runtime-guarded `suppress()` helper — branch (b) of the cross-phase F1 decision, which the +roadmap resolved on the verified version facts (`SuppressedError` reached Node only in 24.0.0; branch (a) would +mean `>=24`, dropping Node 18/20/22 outright). F2 (this phase's zero `invariant()` assertions) is recorded as a +Deviation Ledger row for Phase 10's project-wide pass, the disposition F2 itself named as the alternative to +fold-site postconditions. Both are tracked in the roadmap's "Open Findings — Phase 4b Validation Review +(2026-07-28)" section. **Purpose:** Implement the recovery-chain primitives — `Outcome<T>`, the request and response recovery chains, the unified dispatch orchestrator, the cancellation-wrapping helper, and the status→typed-exception mapping step — satisfying `docs/product-spec/08-execution-pipelines.md` §8.2 (`RECOV-1`–`RECOV-16`). This is the second of three sub-phases the roadmap's Phase 4 ("Execution Context & Pipelines") splits into: 4a (execution context, -done), **4b** (this document, `§8.2`), 4c (stage-based pipeline, `§8.1`, built on 4a+4b). +**not yet implemented** — 4b turned out not to depend on it; see `docs/work/mvp/2026-09-04-open-items-dissolution.md` F8), **4b** (this +document, `§8.2`), 4c (stage-based pipeline, `§8.1`, which does depend on both 4a and 4b). **Governing documents:** `docs/product-spec/08-execution-pipelines.md` §8.2/§8.3 (normative, cited by ID throughout), `docs/sdk-design-nodejs/05-pipeline-architecture.md` (Node-port mapping for both pipeline layers), @@ -113,8 +115,16 @@ declared order within each group. than throw; both are handled identically by `apply()`. - **`RECOV-12`:** when a step throws while the current outcome is a `Success` holding a response, `apply()` closes that response (`response.close()`, from Phase 3b) before wrapping the throwable into a `Failure`, - attaching any close error as `suppressed` via a manually-constructed `SuppressedError` so a close failure never - masks the primary throwable. The response is released exactly once. + attaching any close error as `suppressed` through the `suppress()` helper so a close failure never masks the + primary throwable. The response is released exactly once. + + **`SuppressedError` is not a global on the declared floor.** It belongs to the full Explicit Resource + Management proposal, which reached Node only in **24.0.0**; `engines.node` is `>=20.3` and this package's + `lib` (`ES2023`, `DOM`, `DOM.AsyncIterable`) does not even supply the *type*. `packages/core/src/suppress.ts` + wraps that gap: `suppress(error, suppressed, message)` constructs the native class when + `globalThis.SuppressedError` exists and a shape-compatible stand-in (`name`, `error`, `suppressed`) when it + does not, reading the global per call rather than at module load. Callers never branch on which one they got, + and assertions are written against the shape, never `instanceof SuppressedError`. **This must be a hand-written `try`/`catch` around the `close()` call, not `using`/`await using`.** Native disposal's own `SuppressedError` construction puts the *later* error first — when a body already threw and @@ -133,7 +143,7 @@ declared order within each group. await current.value.close(); // Response.close() is Promise<void> (3b) -- must be awaited to be catchable } catch (closeError) { return failure( - new SuppressedError(originalError, closeError, 'response close failed while handling step error'), + suppress(originalError, closeError, 'response close failed while handling a step error'), ); } } @@ -287,6 +297,7 @@ the violation — retrofit it before Phase 3b is executed, or record it in Phase ``` packages/core/src/invariant.ts # MODIFY: add assertNever() +packages/core/src/suppress.ts # NEW: suppress(), the runtime-guarded SuppressedError (F1 branch (b)) packages/core/src/recovery/ outcome.ts # Outcome<T>, success(), failure(), fold() @@ -297,7 +308,7 @@ packages/core/src/recovery/ status-mapping.ts # statusMappingStep() ``` -`invariant.ts` is the one file outside `recovery/` this sub-phase touches. `docs/knowledge/data-modeling.md` +`invariant.ts` and `suppress.ts` are the two files outside `recovery/` this sub-phase touches. `docs/knowledge/data-modeling.md` requires every discriminated-union `switch` to close with `default: return assertNever(x)`, "defined once and imported everywhere," and no prior phase plan actually adds it — `fold()` is the codebase's first such `switch`, so `assertNever` lands here as a small addition alongside the `invariant()`/`InvariantViolation` that module @@ -317,6 +328,9 @@ see its section above for why an `invariant()` there would violate `RECOV-2`. | No new per-status typed-exception hierarchy for `RECOV-15` | `RECOV-15`'s "matching typed exception" (which some ports read as a per-status class family) | Phase 3b's flat `HttpStatusError` (carrying `status` + buffered body) already satisfies this, and the corpus caps custom error hierarchies at two levels; a per-status class family would violate that cap | | No default/preset recovery chain shipped in 4b | none — scope decision | Matches 4a's "primitives only" discipline; Phase 5 (retry) is the first real consumer and decides its own composition | | `#private` fields and methods on both chain classes (`#steps`, `#responseSteps`, `#recoverySteps`, `#runResponsePhase`, `#runRecoveryPhase`) | `docs/knowledge/data-modeling.md:20-23` — `private` is the default; `#private` requires a comment justifying a genuine runtime-privacy requirement | **No runtime-privacy claim is made.** These classes are unfrozen holders of a readonly array, unlike 3b's `Response`, whose `#closed` genuinely must survive `Object.freeze(this)`. `#private` is the established package-wide field style (Phase 1, 3b, and 4a's `ContextStore` all use it), so switching 4b alone would fragment the package and trip the corpus's own "never mix two styles within a module/package" rule. Recorded as a project-wide deviation for Phase 10 to reconcile in one pass, not fixed here | +| `RECOV-12`'s suppressed-error pairing goes through a runtime-guarded `suppress()` helper rather than `new SuppressedError(...)` | none — a runtime-floor constraint, not a spec deviation. Listed so Phase 10 sees the shape | `SuppressedError` reached Node in 24.0.0; `engines.node` is `>=20.3` and `lib` does not supply the type. Raising the floor to reach one error class would drop Node 18, 20 and 22. The helper returns the native class where it exists and a shape-compatible stand-in where it does not, so nothing downstream branches. Phases 5a, 6a, 6b and 6c share the helper | +| `RequestRecoveryChain` / `ResponseRecoveryChain` are classes holding an immutable step array | `docs/knowledge/data-modeling.md:10` — classes are reserved for things that own a lifecycle or hold mutable runtime state behind an invariant; everything else is plain data transformed by free functions | Neither chain owns a lifecycle or mutable state — a free `applyRequestChain(steps, request)` would satisfy the corpus directly. Kept as classes because `RECOV-14`'s second clause is written about the *step instance* and the chain instance ("per-request state never on the step instance"), and because the defensive copy has to happen once at a construction boundary rather than on every call. Recorded rather than corrected: the shape is what `§8.2` describes and what Phase 5's retry step will compose against | +| Zero `invariant()` assertions across `recovery/` | `docs/knowledge/assertions.md:6-7`'s 2-per-function module average (Rule 8) | F2, deliberately not closed here. The concrete cost is named: no `apply()` postcondition checks that a step returned a value at all, so a step returning `undefined` poisons the fold silently and surfaces layers away. It is a project-wide inconsistency rather than 4b's — Phases 1/2/3b/4a ship zero, 4c ships fifteen — so adding assertions to 4b alone would deepen the split rather than close it. Phase 10 settles the density rule once and applies it everywhere | | `fold(outcome, onSuccess, onFailure)` takes three positional parameters | `docs/knowledge/function-design.md:22-23` — "an options object when it has 3 or more parameters" | The prose rule is one parameter stricter than its own stated enforcement (`max-params: ['error', 3]` errors at four), so this passes lint while violating the corpus text — flagged as a corpus conflict in the roadmap, not silently ignored. Three positional parameters match Phase 2's already-shipped `Transport.send(request, options?, signal?)`; `fold(outcome, {onSuccess, onFailure})` would make 4b the only module in the package reading differently for a canonical two-branch fold | ## Testing @@ -350,6 +364,19 @@ trigger an auto-close of the original (`RECOV-13`); `dispatchWithRecovery` rethr byte-for-byte unchanged, no wrapping (`RECOV-10`); a transport-raised `CancellationError` with no caller signal still reaches the recovery steps rather than escaping the orchestrator (`RECOV-2`/`RECOV-11`). +**Type-level tests.** `Outcome<T>` is an exported generic type, so it ships `expectTypeOf` assertions +(styleguide 11.6): the `kind` union is closed, each variant's payload is reachable only after narrowing, and two +`@ts-expect-error` lines prove the negative — a narrowed `Success` has no `error` and a narrowed `Failure` has +no `value`. `statusMappingStep`'s conformance to `ResponseStep` is asserted the same way, in the test file +rather than as a module-level `satisfies` statement: `satisfies` erases to its operand, not to nothing, so the +module-level form leaves a dead `statusMappingStep;` expression statement in the published `dist/`. + +**Node-runtime conformance.** `SuppressedError`'s presence is exactly the kind of runtime divergence +`test/node-conformance/`'s membership rule exists for — Bun and current Node ship it, the declared floor does +not — so `recovery-chain.test.mjs` forces the guarded branch from real Node and re-runs `RECOV-12`'s +release-exactly-once over Node's own Web Streams. `bun test` alone would only ever exercise whichever branch +Bun's runtime happens to take. + **A `Response` is frozen** (Phase 1's `Object.freeze(this)`, preserved by 3b's retrofit), so no test may patch `response.close` — the assignment throws `TypeError` under an ES module's strict mode. `RECOV-12`/`RECOV-13`'s close assertions observe the body stream's `cancel()` hook instead, the way 3b's own `response.test.ts` does. diff --git a/docs/superpowers/plans/2026-07-25-phase4b-recovery-chain.md b/docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain.md similarity index 89% rename from docs/superpowers/plans/2026-07-25-phase4b-recovery-chain.md rename to docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain.md index 0316f97..1853569 100644 --- a/docs/superpowers/plans/2026-07-25-phase4b-recovery-chain.md +++ b/docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain.md @@ -5,7 +5,7 @@ **Goal:** Ship `Outcome<T>`, the request/response recovery chains, the unified dispatch orchestrator, the cancellation-wrapping helper, and the status→typed-exception mapping step in `@dexpace/core`, satisfying `product-spec/08-execution-pipelines.md` §8.2 (`RECOV-1`–`RECOV-16`), per -`docs/superpowers/specs/2026-07-25-phase4b-recovery-chain-design.md`. +`docs/work/mvp/phase4/phase4b/2026-07-25-phase4b-recovery-chain-design.md`. **Architecture:** A new `packages/core/src/recovery/` folder, six independent files with no folder-level barrel (`docs/knowledge/module-organization.md`'s "never create internal barrels" applies — a future consumer imports @@ -16,46 +16,38 @@ reuse Phase 3b's already-async `toHttpError()` directly. **Nothing in this phase barrel** — `recovery/` is resilience-layer plumbing; `api-extractor`'s committed report must come back byte-identical. -**Tech Stack:** TypeScript 5.8+, `fast-check` for the invariant-bearing-function property tests. No new runtime +**Tech Stack:** TypeScript 5.8+, `fast-check` for the invariant-bearing-function property tests, and the +runtime-guarded `suppress()` helper this phase adds for `RECOV-12` (see the notice below). No new runtime dependencies — `SEAM-1` untouched. -> ### ⛔ BLOCKED — do not execute this plan yet +> ### ✅ UNBLOCKED — executed 2026-08-26 > -> **`RECOV-12`'s `SuppressedError` is not available on the declared runtime floor.** An earlier draft of this -> plan claimed it was "already available since Phase 3b's checkpoint lib bump" — that is false and has been -> removed. The checkpoint raised `engines.node` only to the first release exposing `Symbol.dispose`/ -> `Symbol.asyncDispose` (`plans/2026-07-25-checkpoint-scaffold-through-phase3a.md:57`, believed `18.18.0`, -> which also forbids any further floor movement as "unreviewed drift"). Node backported those two symbols on -> its own; `SuppressedError` is a V8 global from the full Explicit Resource Management proposal and is absent -> on every 18.x runtime. +> **F1 (`SuppressedError`) resolved to branch (b): a runtime-guarded `suppress()` helper.** The roadmap's +> "F1 resolution — the verified version facts" settled the choice on evidence: `SuppressedError` belongs to the +> full Explicit Resource Management proposal, which reached Node only in **24.0.0**. Branch (a) is therefore not +> a patch bump — it means `engines.node >= 24`, dropping Node 18, 20 and 22 outright for one error class. The +> floor stayed at `>=20.3` (set by `AbortSignal.any()`), and this package's `lib` is `["ES2023", "DOM", +> "DOM.AsyncIterable"]`, which does not supply `SuppressedError`'s type either. > -> Adding `esnext.disposable` to `lib` supplies the *type* only. So `new SuppressedError(...)` at Task 3 type- -> checks, passes `bun test` locally, and then throws `ReferenceError: SuppressedError is not defined` at call -> time — precisely the `NFR-10` trap `docs/knowledge/tooling-and-quality-gates.md:60-61` describes. Task 7's -> `bun run verify:node-floor`, `bun run test:node`, and the `node-floor-conformance` job pinned to `18.17.0` -> would all fail. +> `packages/core/src/suppress.ts` ships `suppress(error, suppressed, message)`: it constructs the native class +> when `globalThis.SuppressedError` exists and a shape-compatible stand-in (`name`, `error`, `suppressed`) when +> it does not, reading the global per call rather than capturing it at module load. Task 3 calls it instead of +> `new SuppressedError(...)`, and every assertion is written against the shape rather than +> `toBeInstanceOf(SuppressedError)` — the `instanceof` form would silently assert nothing on the floor runtime. +> Both branches are forced in `suppress.test.ts`, and `test/node-conformance/recovery-chain.test.mjs` re-forces +> the guarded branch from real Node, since `bun test` alone only ever exercises whichever branch Bun takes. > -> **Needs a decision, and it is cross-phase** — Phases 5a, 6a, 6b and 6c reach for `SuppressedError` on the same -> premise (`plans/2026-07-26-phase5a-retry.md:36`, `plans/2026-07-28-phase6a-serde.md`'s `closingAfter` helper, -> `specs/2026-07-28-phase6b-sse-design.md:163`, `specs/2026-07-28-phase6c-pagination-design.md:192`), so -> whichever option lands must land in all five: +> **The same helper is the fix for Phases 5a, 6a, 6b and 6c**, which reach for `SuppressedError` on the same +> premise. Their plans now point here; each replaces `new SuppressedError(...)` with `suppress(...)` when it +> executes. > -> - **(a) Raise `engines.node`** past the first release shipping Explicit Resource Management. Consumer-visible -> breaking change, and the checkpoint forbids unsanctioned floor moves. Confirm the exact release first. -> - **(b) A runtime-guarded `suppress(primary, secondary)` helper** in `packages/core/src/`, using native -> `SuppressedError` when `globalThis.SuppressedError` exists and attaching a `suppressed` property otherwise — -> the same guarded shape the roadmap already sanctioned for `Symbol.asyncDispose`. Changes Task 3's -> `expect(wrapped).toBeInstanceOf(SuppressedError)` assertion. -> -> **Second open decision, non-blocking: assertion density.** This phase ships zero `invariant()` calls across -> roughly a dozen functions, against `docs/knowledge/assertions.md:6-7`'s 2-per-function module average. The -> concrete cost: no `apply()` checks that a step returned a value at all, so a step returning `undefined` -> poisons the fold silently and surfaces layers away. Project-wide inconsistency rather than 4b's alone — -> Phases 1/2/3b/4a ship zero, 4c ships fifteen. Resolve as either postcondition assertions at the fold sites -> or a Deviation Ledger row, ideally project-wide at Phase 10. -> -> Both items are tracked in the roadmap's "Open Findings — Phase 4b Validation Review (2026-07-28)" section. -> Everything else that review raised (F3–F10) is already applied to this plan and its design. +> **F2 (assertion density) resolved to a Deviation Ledger row**, the alternative F2 itself named. This phase +> ships zero `invariant()` calls across roughly a dozen functions, against `docs/knowledge/assertions.md:6-7`'s +> 2-per-function module average, with a named cost: no `apply()` postcondition checks that a step returned a +> value at all, so a step returning `undefined` poisons the fold silently. It is a project-wide inconsistency +> rather than 4b's — Phases 1/2/3b/4a ship zero, 4c ships fifteen — so assertions added to 4b alone would deepen +> the split rather than close it. The design's ledger carries the row; Phase 10 settles the density rule once, +> project-wide. **Prerequisite:** This plan assumes Phases 0, 1, 2, 3a, 3b, and 4a are already implemented exactly as their own plans specify. Concretely: `packages/core/src/http/*` exports `DexpaceError`, `Request`, `Response`, @@ -92,8 +84,11 @@ addition alongside the existing `invariant()`/`InvariantViolation` it already ex - **`RECOV-12`'s close-on-throw is a hand-written `try`/`catch`, never `using`/`await using`.** Native disposal's auto-generated `SuppressedError` puts the *later* error (the disposal failure) first, making it primary and the original body error `.suppressed` — the opposite of what `RECOV-12` wants (the step's original - throwable stays primary; a close failure rides along as `.suppressed`). Construct - `new SuppressedError(originalError, closeError, message)` by hand — original first. + throwable stays primary; a close failure rides along as `.suppressed`). Build it with + `suppress(originalError, closeError, message)` from `../suppress.js` — original first. **Never + `new SuppressedError(...)`:** it is absent on the declared floor (Node 24.0.0 and up only) and absent from + this package's `lib`, so the direct form neither type-checks nor runs there. Assert its shape (`name`, + `error`, `suppressed`), never `toBeInstanceOf(SuppressedError)`. - **`RECOV-13`: a step that deliberately *returns* a different outcome (no throw) is never auto-closed.** Only a caught throw triggers the close-and-wrap path. Do not add a "close whenever the outcome changes" check — that would violate `RECOV-13` by closing a response a step meant to keep alive or already closed itself. @@ -151,6 +146,8 @@ addition alongside the existing `invariant()`/`InvariantViolation` it already ex ``` packages/core/src/invariant.ts # MODIFY: add assertNever() (Task 1) packages/core/src/invariant.test.ts # MODIFY: add assertNever coverage +packages/core/src/suppress.ts # NEW: suppress(), the guarded SuppressedError (F1 (b)) (Task 1b) +packages/core/src/suppress.test.ts # NEW: both branches of the guard, forced packages/core/src/recovery/ outcome.ts # Outcome<T>, success(), failure(), fold() (Task 1) @@ -186,7 +183,7 @@ No `recovery/index.ts` (see Global Constraints). Task 7 runs the full gate seque onFailure): R` (from `recovery/outcome.ts`). Every later task in this plan imports `Outcome`/`success`/`failure` from `outcome.js`. -- [ ] **Step 1: Write the failing test for `assertNever`** +- [x] **Step 1: Write the failing test for `assertNever`** ```typescript // packages/core/src/invariant.test.ts @@ -212,12 +209,12 @@ describe('assertNever', () => { }); ``` -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/invariant.test.ts` Expected: FAIL — `assertNever is not a function` (or similar export error). -- [ ] **Step 3: Add `assertNever` to `invariant.ts`** +- [x] **Step 3: Add `assertNever` to `invariant.ts`** Append to the existing file (do not touch the existing `invariant()`/`InvariantViolation` exports): @@ -234,19 +231,19 @@ export function assertNever(value: never, message = `unreachable case: ${String( } ``` -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/invariant.test.ts` Expected: PASS, including the 2 new tests. -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/invariant.ts packages/core/src/invariant.test.ts git commit -m "feat(core): add assertNever exhaustiveness helper" ``` -- [ ] **Step 6: Write the failing test for `Outcome<T>`** +- [x] **Step 6: Write the failing test for `Outcome<T>`** ```typescript // packages/core/src/recovery/outcome.test.ts @@ -329,12 +326,12 @@ describe('fold identity law (RECOV-1)', () => { }); ``` -- [ ] **Step 7: Run and confirm it fails** +- [x] **Step 7: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/outcome.test.ts` Expected: FAIL — `Cannot find module './outcome.js'`. -- [ ] **Step 8: Write `outcome.ts`** +- [x] **Step 8: Write `outcome.ts`** ```typescript // packages/core/src/recovery/outcome.ts @@ -379,12 +376,12 @@ export function fold<T, R>(outcome: Outcome<T>, onSuccess: (value: T) => R, onFa } ``` -- [ ] **Step 9: Run and confirm it passes** +- [x] **Step 9: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/outcome.test.ts` Expected: PASS, 7 tests. -- [ ] **Step 10: Commit** +- [x] **Step 10: Commit** ```bash git add packages/core/src/recovery/outcome.ts packages/core/src/recovery/outcome.test.ts @@ -393,6 +390,43 @@ git commit -m "feat(core): add Outcome<T>, success/failure/fold (RECOV-1)" --- +### Task 1b: `suppress()` — the runtime-guarded `SuppressedError` (F1 branch (b)) + +**Files:** +- Create: `packages/core/src/suppress.ts` +- Create: `packages/core/src/suppress.test.ts` + +**Interfaces:** +- Consumes: nothing. +- Produces: `interface SuppressedErrorLike`, `suppress(error: unknown, suppressed: unknown, message: string): + SuppressedErrorLike`. Task 3's `toFailureClosingSuccess` is its first call site; Phases 5a, 6a, 6b and 6c are + the next ones. + +- [x] **Step 1: Write the failing test** — both branches of the guard, forced. The native branch is skipped when + the runtime has no `SuppressedError`; the fallback branch is reached by deleting the global inside a + `try`/`finally` that restores the original property descriptor, with a following test asserting the restore + actually happened. Without that, whichever branch the test runtime happens to take is the only one ever + covered, and the floor runtime takes the *other* one. + +- [x] **Step 2: Run and confirm it fails** — `Cannot find module './suppress.js'`. + +- [x] **Step 3: Write `suppress.ts`** — read `globalThis.SuppressedError` **per call**, not at module load, via + an intersection cast (`globalThis as typeof globalThis & {SuppressedError?: SuppressedErrorConstructor}`); a + cast to a bare optional-property type trips TS's weak-type check. Fall back to a module-private class + extending `Error` that sets `name = 'SuppressedError'` and assigns `error`/`suppressed` in the constructor + body — no parameter properties (`erasableSyntaxOnly`). + +- [x] **Step 4: Run and confirm it passes** — 5 tests. + +- [x] **Step 5: Commit** + +```bash +git add packages/core/src/suppress.ts packages/core/src/suppress.test.ts +git commit -m "feat(core): add a runtime-guarded suppress() helper for RECOV-12" +``` + +--- + ### Task 2: `recovery/request-chain.ts` **Files:** @@ -405,7 +439,7 @@ git commit -m "feat(core): add Outcome<T>, success/failure/fold (RECOV-1)" - Produces: `type RequestStep = (request: Request) => Promise<Request>`, `class RequestRecoveryChain`. Task 6 (`orchestrator.ts`) imports `RequestRecoveryChain`. -- [ ] **Step 1: Write the failing test** +- [x] **Step 1: Write the failing test** ```typescript // packages/core/src/recovery/request-chain.test.ts @@ -489,12 +523,12 @@ describe('RequestRecoveryChain.apply fold law', () => { }); ``` -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/request-chain.test.ts` Expected: FAIL — `Cannot find module './request-chain.js'`. -- [ ] **Step 3: Write `request-chain.ts`** +- [x] **Step 3: Write `request-chain.ts`** ```typescript // packages/core/src/recovery/request-chain.ts @@ -529,12 +563,12 @@ export class RequestRecoveryChain { } ``` -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/request-chain.test.ts` Expected: PASS, 5 tests (including the fast-check property, which itself runs 100 cases by default). -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/recovery/request-chain.ts packages/core/src/recovery/request-chain.test.ts @@ -556,7 +590,7 @@ git commit -m "feat(core): add RequestRecoveryChain (RECOV-3, RECOV-14)" Outcome<Response>) => Promise<Outcome<Response>>`, `class ResponseRecoveryChain`. Task 6 imports `ResponseRecoveryChain`; Task 5's `statusMappingStep` is typed as a `ResponseStep`. -- [ ] **Step 1: Write the failing test** +- [x] **Step 1: Write the failing test** ```typescript // packages/core/src/recovery/response-chain.test.ts @@ -756,9 +790,12 @@ describe('RECOV-12: close-on-throw while holding a Success', () => { expect(result.kind).toBe('failure'); const wrapped = result.kind === 'failure' ? result.error : undefined; - expect(wrapped).toBeInstanceOf(SuppressedError); - expect((wrapped as SuppressedError).error).toBe(originalError); - expect((wrapped as SuppressedError).suppressed).toBe(closeError); + // Shape, not `toBeInstanceOf(SuppressedError)`: the native class does not exist on the floor + // runtime, so the instanceof form would assert nothing there. + expect(wrapped).toBeInstanceOf(Error); + expect((wrapped as SuppressedErrorShape).name).toBe('SuppressedError'); + expect((wrapped as SuppressedErrorShape).error).toBe(originalError); + expect((wrapped as SuppressedErrorShape).suppressed).toBe(closeError); }); }); @@ -867,16 +904,17 @@ describe('RECOV-14: steps are safe for concurrent invocation', () => { }); ``` -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/response-chain.test.ts` Expected: FAIL — `Cannot find module './response-chain.js'`. -- [ ] **Step 3: Write `response-chain.ts`** +- [x] **Step 3: Write `response-chain.ts`** ```typescript // packages/core/src/recovery/response-chain.ts import type {Response} from '../http/response.js'; +import {suppress} from '../suppress.js'; import {failure, success, type Outcome} from './outcome.js'; /** @internal */ @@ -887,15 +925,17 @@ export type RecoveryStep = (outcome: Outcome<Response>) => Promise<Outcome<Respo /** * Shared close-on-throw handling for both phases (RECOV-12): if the outcome held at the moment of the * throw was a Success, its response is closed before the throwable is wrapped into a Failure. A close - * failure is attached as `suppressed` on the ORIGINAL throwable -- constructed by hand, original first -- - * never via `using`/`await using`, whose auto-generated SuppressedError would invert that priority. + * failure is attached as `suppressed` on the ORIGINAL throwable -- built through `suppress()`, original first + * -- never via `using`/`await using`, whose auto-generated SuppressedError would invert that priority. */ async function toFailureClosingSuccess(thrownError: unknown, current: Outcome<Response>): Promise<Outcome<Response>> { if (current.kind === 'success') { try { await current.value.close(); } catch (closeError) { - return failure(new SuppressedError(thrownError, closeError, 'response close failed while handling step error')); + return failure( + suppress(thrownError, closeError, 'response close failed while handling a step error'), + ); } } return failure(thrownError); @@ -953,12 +993,12 @@ export class ResponseRecoveryChain { } ``` -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/response-chain.test.ts` Expected: PASS, 13 tests (including the fast-check property and the RECOV-14 concurrency test). -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/recovery/response-chain.ts packages/core/src/recovery/response-chain.test.ts @@ -980,7 +1020,7 @@ git commit -m "feat(core): add ResponseRecoveryChain (RECOV-4..RECOV-9, RECOV-12 disposition lives — if it is still a pure pass-through once Phase 5 lands, inline it there and move the disposition wholly into Phase 10's deviation ledger rather than keeping an abstraction with no behavior. -- [ ] **Step 1: Write the failing test** +- [x] **Step 1: Write the failing test** ```typescript // packages/core/src/recovery/cancellation.test.ts @@ -1026,12 +1066,12 @@ describe('wrapCancellation (RECOV-11)', () => { }); ``` -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/cancellation.test.ts` Expected: FAIL — `Cannot find module './cancellation.js'`. -- [ ] **Step 3: Write `cancellation.ts`** +- [x] **Step 3: Write `cancellation.ts`** ```typescript // packages/core/src/recovery/cancellation.ts @@ -1060,7 +1100,7 @@ export function wrapCancellation(error: unknown): Outcome<never> { } ``` -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/cancellation.test.ts` Expected: PASS, 4 tests. @@ -1068,7 +1108,7 @@ Expected: PASS, 4 tests. `CancellationError` is imported by the *test* only (to prove a classified cancellation gets no special treatment); `cancellation.ts` itself no longer needs it, so do not add the import back to the module. -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/recovery/cancellation.ts packages/core/src/recovery/cancellation.test.ts @@ -1090,7 +1130,7 @@ git commit -m "feat(core): add wrapCancellation (RECOV-11)" task in this plan -- a future consumer (Phase 5 or 4c) installs it into a `ResponseRecoveryChain`'s response-step list. -- [ ] **Step 1: Write the failing test** +- [x] **Step 1: Write the failing test** ```typescript // packages/core/src/recovery/status-mapping.test.ts @@ -1149,12 +1189,12 @@ describe('statusMappingStep (RECOV-15)', () => { }); ``` -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/status-mapping.test.ts` Expected: FAIL — `Cannot find module './status-mapping.js'`. -- [ ] **Step 3: Write `status-mapping.ts`** +- [x] **Step 3: Write `status-mapping.ts`** ```typescript // packages/core/src/recovery/status-mapping.ts @@ -1191,12 +1231,12 @@ statusMappingStep satisfies ResponseStep; `Response` is now imported (type-only) because the explicit parameter and return annotations a `function` declaration needs replace the inference the `: ResponseStep` annotation was supplying. -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/status-mapping.test.ts` Expected: PASS, 4 tests. -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/recovery/status-mapping.ts packages/core/src/recovery/status-mapping.test.ts @@ -1219,7 +1259,7 @@ git commit -m "feat(core): add statusMappingStep, wiring 3b's toHttpError into t - Produces: `interface DispatchConfig`, `dispatchWithRecovery(request: Request, config: DispatchConfig): Promise<Response>`. Terminal task of this plan -- no later task consumes this file. -- [ ] **Step 1: Write the failing test** +- [x] **Step 1: Write the failing test** ```typescript // packages/core/src/recovery/orchestrator.test.ts @@ -1409,12 +1449,12 @@ describe('RECOV-11: the catch routes every throwable through wrapCancellation', `CancellationError` is imported from `../seams/transport.js` (Phase 2) alongside the `Transport` type. -- [ ] **Step 2: Run and confirm it fails** +- [x] **Step 2: Run and confirm it fails** Run: `cd packages/core && bun test src/recovery/orchestrator.test.ts` Expected: FAIL — `Cannot find module './orchestrator.js'`. -- [ ] **Step 3: Write `orchestrator.ts`** +- [x] **Step 3: Write `orchestrator.ts`** ```typescript // packages/core/src/recovery/orchestrator.ts @@ -1476,12 +1516,12 @@ export async function dispatchWithRecovery(request: Request, config: DispatchCon } ``` -- [ ] **Step 4: Run and confirm it passes** +- [x] **Step 4: Run and confirm it passes** Run: `cd packages/core && bun test src/recovery/orchestrator.test.ts` Expected: PASS, 7 tests. -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add packages/core/src/recovery/orchestrator.ts packages/core/src/recovery/orchestrator.test.ts @@ -1498,7 +1538,7 @@ git commit -m "feat(core): add dispatchWithRecovery orchestrator (RECOV-2, RECOV - Consumes: every preceding task. - Produces: nothing new; verifies the whole phase is green and the public surface did not move. -- [ ] **Step 1: Run the full gate sequence** +- [x] **Step 1: Run the full gate sequence** ```bash cd /home/mohammad/Projects/dexpace/nodejs-sdk @@ -1517,7 +1557,17 @@ bun run audit Expected: all exit 0. Coverage at or above the 80% aggregate floor (`NFR-5`). -- [ ] **Step 2: Verify no `node:` import crept in** +- [x] **Step 1b: Add the Node-runtime conformance case** + +`test/node-conformance/README.md`'s membership rule: a phase touching a runtime-divergent surface adds a case +there, not only to `bun test`. `SuppressedError`'s presence is exactly that divergence — Bun and current Node +ship it, the declared 20.3 floor does not — so `test/node-conformance/recovery-chain.test.mjs` forces the +guarded branch from real Node (including with the global deleted) and re-runs `RECOV-12`'s +release-exactly-once over Node's own Web Streams implementation, whose `cancel()` and reader-lock timing are +independent of Bun's. `suppress` and `recovery/` are `@internal` with no public subpath, so it imports them by +direct `dist/` path, the way `io-byte-stream.test.mjs` does. + +- [x] **Step 2: Verify no `node:` import crept in** ```bash ! grep -rn "from 'node:" packages/core/src/recovery/ @@ -1525,7 +1575,7 @@ Expected: all exit 0. Coverage at or above the 80% aggregate floor (`NFR-5`). Expected: exit 0, no matches. -- [ ] **Step 3: Verify the public API surface did not move** +- [x] **Step 3: Verify the public API surface did not move** Step 1 already regenerated the report via `bun run api`; this only inspects the result. Run from the repo root: @@ -1538,7 +1588,7 @@ Expected: **no output, exit 0.** Nothing from `src/recovery/` reached the publis 3a's/4a's gate. If this fails, remove whatever export leaked into `packages/core/src/index.ts` rather than accepting the report change. -- [ ] **Step 4: Add a changeset** +- [x] **Step 4: Add a changeset** Because nothing enters the public API, this is a patch-level, no-consumer-impact change: @@ -1549,7 +1599,7 @@ bun run changeset Select `@dexpace/core`, choose **patch**, summary: `Internal: recovery-chain primitives for product-spec §8.2 (RECOV-1..16). No public API change.` -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add .changeset/ diff --git a/docs/superpowers/specs/2026-07-25-phase4c-stage-pipeline-design.md b/docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md similarity index 93% rename from docs/superpowers/specs/2026-07-25-phase4c-stage-pipeline-design.md rename to docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md index 774dd37..e4b7df7 100644 --- a/docs/superpowers/specs/2026-07-25-phase4c-stage-pipeline-design.md +++ b/docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md @@ -3,7 +3,7 @@ **Status:** Draft, approved for planning. **One open finding (2026-07-29 validation review, F9):** whether `Cursor` should observe the caller's `AbortSignal` between steps, and as which error type — tracked in the roadmap's "Open Findings — Phase 4c Validation Review (2026-07-29)" section -(`docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`). Decide before Phase 5a Task 1 lands +(`docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`). Decide before Phase 5a Task 1 lands `StepContext.signal`. Everything else from that review is applied. **Purpose:** Implement the stage-based pipeline — the fixed-stage step composition runtime, its builder with @@ -456,6 +456,7 @@ first ships a pillar step. | `PIPE-26`'s "delegate execute/execute-async to its own send/send-async" satisfied by one `send()` method | `PIPE-26`'s literal two-method framing | Follows directly from the row above: `Transport` has one method, so there is nothing to delegate to beyond it | | Steps are functions wrapped in a `StepDescriptor`, not classes implementing an interface | Reference's class-based step modeling (`PIPE-36`'s subclass-locking framing) | `sdk-design-nodejs/05`'s existing precedent; `StepDescriptor.type` (a `symbol`) carries the identity/anchor-matching role a class hierarchy would otherwise provide | | `Stage` is a string-literal union plus an explicit `STAGE_ORDER` array, not numeric enum values with gaps | `PIPE-3`'s "sparse numeric order keys" (SHOULD, naming a mechanism) | The styleguide bars TS `enum` outright (erasable-syntax rule, binding since Phase 0); a string union + ordered array satisfies the same underlying goal (inserting a stage never touches existing stages' identities) without a numeric type at all | +| Two `eslint-disable` directives ship in this phase | The plan's lint-gate pre-check, which asserted "No `eslint-disable` anywhere in this phase" | Added during implementation (2026-08-26). (1) `PillarCollisionError(stage, existingType, incomingType, options?)` is four parameters and `max-params` counts them — the plan's audit covered the builder's methods and `Cursor`/`Runtime`'s constructors but not the error leaves. `PIPE-5` fixes the first three and `DexpaceError`'s contract fixes the trailing `options?: ErrorOptions`, so the shape is not reducible; `HttpStatusError` (Phase 3) established the same exemption for the same reason. (2) `runtime.test.ts` disables `@typescript-eslint/require-await` on a step that throws before its first `await` — that shape *is* the case under test (`PIPE-29`/`PIPE-30`'s structural claim), and rewriting it as `Promise.reject` would exercise something else. Both carry the `-- reason` that `eslint-comments/require-description` demands (`NFR-7`) | ## Deferred Items @@ -514,6 +515,24 @@ ships (plumbing, no pillar steps — a test that needs redirect or retry *behavi conformance clause describes the handle silently resuming past already-visited steps — the behavior the one-shot guard makes unreachable, so that clause is not transcribable as written. +**Added during the 2026-08-26 implementation review**, all testable with plumbing alone and none of them +covered by the list above as written: + +- `PIPE-12`'s two remaining clauses, each its own case: a step that short-circuits (returns without calling + `next`) never reaches the terminal transport, and a step may substitute the outbound response on the way + back out. +- `PIPE-26`'s "a configured pipeline can stand in wherever a transport is expected... and options survive the + indirection": a `Runtime` nested as another `Runtime`'s transport — step order across both hops, and the + caller's `options`/`signal` reaching the terminal transport by reference. +- `PIPE-14`'s stickiness *across forks* (the design's "visible to every subsequent fork" claim, distinct from + the downstream-within-one-drive case): a substitution made inside one fork is what the next fork dispatches. +- `PIPE-10`/`PIPE-11`'s concurrency claim: two interleaved `send()` calls on one `Runtime` each reach the + transport with their own in-flight request, which is what a shared mutable `#request` would break. +- `PIPE-25`/`PIPE-10`'s immutability, structurally rather than by equality: `Runtime.steps` is frozen, and + mutating the array handed to the constructor afterwards cannot reach the built runtime. +- `PIPE-23`'s all-or-nothing on the `SEND` rejection path, not only on a pillar collision; and `PIPE-20`/ + `PIPE-5`'s interaction — a pillar emptied by `remove` accepts a step of a different type. + `PIPE-40`'s response-release discipline is a contract on wrapping steps, not on `Cursor`/`Runtime` (see "Cursor and fork"), and its conformance clause is a 2-hop redirect — untestable without a redirect step. It moves to the phase that ships one, alongside `PIPE-2`'s second half. diff --git a/docs/superpowers/plans/2026-07-25-phase4c-stage-pipeline.md b/docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline.md similarity index 99% rename from docs/superpowers/plans/2026-07-25-phase4c-stage-pipeline.md rename to docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline.md index 088474c..bf1b11a 100644 --- a/docs/superpowers/plans/2026-07-25-phase4c-stage-pipeline.md +++ b/docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline.md @@ -6,13 +6,13 @@ > terminal transport but never checks it between steps, against `docs/knowledge/concurrency-and-async.md:46`. > The fix is not mechanical — a raw `throwIfAborted()` raises a `DOMException` the SDK taxonomy does not own — > so it is recorded, undecided, in the roadmap's "Open Findings — Phase 4c Validation Review (2026-07-29)" -> section (`docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`). Build this plan as written; do +> section (`docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`). Build this plan as written; do > not improvise a cancellation check. Every other finding from that review is already applied below. **Goal:** Ship the stage-based pipeline in `@dexpace/core` — the fixed-stage step composition runtime, its builder with surgical edit operations, the per-call cursor/fork mechanism, and the execution-context-store wiring — satisfying `product-spec/08-execution-pipelines.md` §8.1 (`PIPE-1`–`PIPE-40`), per -`docs/superpowers/specs/2026-07-25-phase4c-stage-pipeline-design.md`. +`docs/work/mvp/phase4/phase4c/2026-07-25-phase4c-stage-pipeline-design.md`. **Architecture:** A new `packages/core/src/pipeline/` folder, six files with no folder-level barrel (`docs/knowledge/module-organization.md`'s "never create internal barrels" — matching Phase 4b's actual diff --git a/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md new file mode 100644 index 0000000..2464cb5 --- /dev/null +++ b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md @@ -0,0 +1,154 @@ +# Phase 5a — Retry Implementation Plan — Checklist + +Verification of [2026-07-26-phase5a-retry.md](./2026-07-26-phase5a-retry.md) against every requirement ID in +`docs/product-spec/09-retry-and-resilience.md` and appendix C's `RECOV-17`–`RECOV-34`, as dispositioned by +`docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md`. + +**Status: EXECUTED (2026-08-26).** Every task below is implemented, tested, and green across the full gate +sequence (`typecheck`, `lint`, `build`, `bun test` with coverage, `api:ci`, `lint:publish`, +`verify:dual-consumption`, `verify:consumer-types`, `verify:seam-1`, `verify:runtime-floor`, `test:node`, +`audit`). `packages/core/etc/core.api.md` and `packages/core/src/index.ts` are byte-identical to `main` — +nothing in this phase reaches the public barrel. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +## Executed out of numeric order: the Phase 7a prerequisite slice + +This plan's Prerequisite section requires Phase 7a's `config/` module to exist first — its Task 8 consumes the +`Clock` seam, its Task 4 imports the shared RFC 1123 parser, and its Task 2 re-exports the shared +retryable-status classifier. `packages/core/src/config/` did not exist. Rather than ship the private copies the +plan's Global Constraints ban, the three files 7a's plan specifies were built first, verbatim from +[2026-07-28-phase7a-configuration.md](../../phase7/phase7a/2026-07-28-phase7a-configuration.md) Tasks 1–3, with their tests: + +| File | Requirements | From | +|---|---|---| +| `packages/core/src/config/clock.ts` | `CFG-15`, `CFG-16`, `CFG-17` | 7a Task 1 | +| `packages/core/src/config/http-date.ts` | `CFG-29`, `CFG-30`, `CFG-31` | 7a Task 2 | +| `packages/core/src/config/retryable.ts` | `CFG-35` | 7a Task 3 | + +Phase 7a's own execution should mark these three tasks done rather than rebuild them; its Tasks 4–10 +(`identifiers`, `equality`, `configuration`, `proxy`, build-info, `client-identity-step`, barrel promotion) are +untouched here. These three are **not** promoted to the public barrel by this phase — 7a's Task 10 owns that +decision. + +## 9.1 The two independent axes + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| RETRY-1 | MUST | Single-sourced retryable status set — 408, 429, 5xx except 501/505 | ✅ | Task 2, re-exported from `config/retryable.ts` (`CFG-35`) rather than defined twice | +| RETRY-2 | MUST | Retryable-throwable classification walks the cause chain | ✅ | Task 2 — iterative, identity-tracking walk; a cyclic `cause` chain terminates instead of spinning, asserted directly | +| RETRY-3 | MUST | Retryability derived from the carried status, not a stored per-subclass flag | ✅ | Task 2 — `HttpStatusError.status` is consulted at classification time; there is no constant to get wrong | +| RETRY-4 | MUST | Transport-level failure (refused, TLS/DNS, socket read timeout, peer reset) retryable at the condition level | ✅ | Task 2 — such failures surface as `IoError` subclasses, which the allow-list admits unconditionally | +| RETRY-5 | MUST | Body-bearing request re-sendable iff its body is replayable | ✅ | Task 2 (`isResendable`), over Phase 3b's `Body.replayable` | +| RETRY-6 | MUST | Idempotent method set is `{GET, HEAD, OPTIONS, PUT, DELETE}`, single-sourced | ✅ | Task 2 imports Phase 1's `http/method.ts` `isIdempotent` (`HTTP-9`); nothing is restated | +| RETRY-7 | MUST | A bare non-idempotent POST is not re-sendable even with nothing to re-send | ✅ | Task 2, asserted; re-asserted end-to-end on both entry points (Tasks 8, 10) | +| RETRY-8 | MUST | BOTH axes must hold before a retry | ✅ | Task 2 (the two predicates), Task 8 (`decideRetry` gates them in order) | +| RETRY-37 | MUST | For a failure carrying a response the CONFIGURED status set is authoritative alone — widens and narrows | ✅ | Task 2 — `isRetryableFailure`'s second parameter; both directions asserted, and the built-in flag is not AND-ed in | + +## 9.2 Backoff and pacing + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| RETRY-9 | MUST | `initialDelay × multiplier^(attempt−1)`, 1-indexed, clamped to the cap | ✅ | Task 3, plus a `fast-check` monotonicity/cap property | +| RETRY-10 | MUST | Symmetric jitter over `[d(1−j/2), d(1+j/2)]`, midpoint `d`, `j=0` the identity | ✅ | Task 3 — window asserted exactly at both ends and by property test; a negative sample floors to zero | +| RETRY-11 | MUST | `attempt < 1` rejected; overflow saturates rather than throwing | ✅ | Task 3 — `invariant()` for the programmer error; `Math.min` absorbs `Infinity` into the cap | +| RETRY-12 | MUST | Defaults: 200 ms, ×2, 8 s cap, 20% jitter, 3 attempts | ✅ | Task 5 (`DEFAULT_RETRY_SETTINGS`) | +| RETRY-43 | MAY | Fixed-delay mode short-circuits backoff AND jitter | ✅ | Task 3 — a `MAY`, but `RETRY-39`'s MUST precedence chain names it as a step, so it is load-bearing | +| RETRY-15 | MUST | Recognized pacing forms: `Retry-After` seconds, `Retry-After` HTTP-date, `retry-after-ms`, `x-ms-retry-after-ms`, `X-RateLimit-Reset` | ✅ | Task 4 | +| RETRY-16 | MUST | The parser is TOTAL: never throws, every failure path returns "no hint" (`null`), never `0` | ✅ | Task 4 — asserted by `fast-check` over arbitrary strings, and separately that the result is `null` or a finite non-negative number | +| RETRY-17 | MUST | A validly-parsed instant already in the past yields `0` | ✅ | Task 4 (both the HTTP-date and `X-RateLimit-Reset` forms) | +| RETRY-18 | MUST | Every computed delta clamps to a 365-day ceiling | ✅ | Task 4 | +| RETRY-19 | MUST | Strict decimal grammar screens the numeric form before any float parse | ✅ | Task 4 — `30d`, `0x1p3`, `NaN`, `Infinity`, `1e3`, `+30`, and surrounding whitespace all rejected | +| RETRY-20 | MUST | A hint REPLACES the schedule for that one decision, unjittered, still budget-clamped | ✅ | Task 8 (`resolveDelay`), asserted end-to-end | +| RETRY-21 | MUST | Fixed precedence, first usable value wins | ✅ | Task 4 — including the fall-through from an unparseable `Retry-After` to `retry-after-ms` | +| RETRY-22 | MUST | A pacing-parse failure never masks the upstream failure | ✅ | Structural — the parser is total, so the original throwable is what the trail carries regardless; asserted in Task 8 | +| RETRY-13 | MUST | One backoff/classifier definition, no second copy | ✅ | Structural under ES modules — one `computeDelay`, one `parsePacingHint`, one status set; both adapters call the same `runWithRetry` | +| RETRY-14 | MUST | Both stacks' budgets denote the same number of sends | ✅ | Structural — there is one budget (`RetrySettings.maxAttempts`), so there is nothing to reconcile. `runWithRetry` asserts it is finite and `>= 1` once per call: a non-finite budget does not fail loudly on its own, it makes the attempt gate permanently false and the loop simply never stops, and this is the single choke point both adapters pass through | + +## 9.3 Cancellation, timeout, and the wait + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| RETRY-23 | MUST | Caller cancellation is never retryable | ✅ | Task 2 — keyed off the abort reason's `name`; `CancellationError` is outside the allow-list, asserted so a future re-parenting under `IoError` breaks loudly. The reference's "restore the interruption flag" half is N/A: `AbortSignal` is latched and observable by every later reader without re-assertion | +| RETRY-24 | MUST | A read timeout represented as an interrupted-I/O subtype stays retryable | ✅ | Task 2 — `AbortSignal.timeout()` aborts with a `DOMException` named `TimeoutError`; asserted bare and wrapped as a `cause`, and again under Node in `test/node-conformance/retry.test.mjs` | +| RETRY-25 | MUST | Never retry fatal errors (`OutOfMemoryError`, `StackOverflowError`) | N/A | Vacuous by construction — the classifier is an allow-list, so an unlisted throwable was never opted in. V8 has no catchable OOM class. Asserted anyway for a stack-overflow `RangeError` and a bare string throw | +| RETRY-26 | MUST | Cancellable inter-attempt wait that does not pin an execution carrier | ✅ | Task 8 (`waitFor`) — delegates to Phase 7a's `Clock.sleep` (`CFG-17`) rather than hand-rolling a second timer-versus-signal race, so the wait sits behind the injected seam and the unit suite stays deterministic. Node has no carriers to pin, so the substance is prompt cancellability (`XCUT-3`): asserted for an abort raised before the wait, for one raised while the wait is pending, and — against a REAL timer — in `test/node-conformance/retry.test.mjs` | +| RETRY-31 | MUST | The wait is non-blocking; a zero delay does not schedule a timer | ✅ | Task 8 — `await` on a timer yields the event loop; `delayMs <= 0` continues inline without reaching the clock at all, asserted by counting `sleep` invocations. The same guard keeps a caller `delayOverride` returning a negative number away from `Clock.sleep`'s negative-duration rejection | +| RETRY-32 | MUST | No further attempts once the caller has cancelled; a response arriving from an already-in-flight attempt closed rather than leaked | ✅ | Task 8 (the loop's first statement), Task 9 (asserted through the pipeline: zero wire sends). Second clause asserted both ways: a response arriving after the abort that the engine **discards** is released (`cancelCount` 1), while one that **ends the loop** is handed to the caller live and unclosed — ownership transfers rather than leaking, since the caller is the only reader that could close it. The design doc's blanket "any response arriving from an in-flight attempt is closed" describes only the first case | +| RETRY-33 | MUST | Every terminal path returns an outcome | ✅ | Task 8 — honored literally, not merely as a rejected promise. `stampAttempt`'s header build, `toHttpError`'s body drain, and a misbehaving injected clock's `sleep` can each throw; all three are folded into a failure outcome **carrying the trail**, because letting one escape as a bare rejection would silently discard every prior attempt `RETRY-34` requires to ride along. Asserted | +| RETRY-45 | MUST | Never shut down a caller-supplied scheduler | N/A | No scheduler object exists to own. The intent survives as `clearTimeout` hygiene on both wait exits, so no dangling timer keeps the event loop alive | + +## 9.4 Budgets, reconciliation, and the discard path + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| RETRY-27 | MUST | Total-timeout budget spanning attempts and delays; three independent abort conditions | ✅ | Task 8 — `budgetExhausted` (elapsed ≥ budget), `overshootsBudget` (elapsed + next delay > budget, which SURFACES rather than merely clamping), and `clampToBudget`. Both the abort and the clamp ship; `0` and `undefined` disable | +| RETRY-28 | MUST | A port that unifies the stacks makes the total timeout explicitly opt-in | ✅ | Task 5 — `totalTimeoutMs` is optional and undefined by default | +| RETRY-34 | MUST | On terminal failure every prior attempt's error rides along as suppressed; discarded on success; skip-self guard | ✅ | Task 8 (`withTrail`) — built through Phase 4b's `suppress()`, never `new SuppressedError(...)` (the native class reached Node only in 24.0.0; the floor is `>=20.3`). A reused instance never suppresses itself, asserted; the ≥3-attempt nested fold is asserted oldest-innermost | +| RETRY-35 | MUST | A discarded response's body is released, including when the retry decision throws | ✅ | Task 8 — the `finally` in `retireAndSchedule`; observed through `countingResponse`'s stream, never a spy on a frozen `Response` | +| RETRY-36 | MUST | A re-sent retryable-status response is remapped into a typed failure so the loop keeps evaluating the budget | ✅ (narrowed, ledgered) | Task 8 — the remap applies **only to responses the engine is discarding**. Gates run first; a response that survives them is returned live and unread. `toHttpError()` drains the body and drops the headers irreversibly, and 4c's pillar signature must return a `Response`. Full reasoning in the design doc; recorded in its deviation ledger | +| RETRY-30 | MUST | N retries must not build an N-deep continuation or stack chain | N/A | An `await` loop is already iterative — each iteration's frame is released before the next begins. No trampoline, re-arm flag, or pump is built. Same disposition class as 4c's `PIPE-29`/`PIPE-30` | + +## 9.5 Knobs, stamping, and re-drive + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| RETRY-38 | SHOULD | Stamp the 1-based attempt ordinal on a fresh per-attempt copy, preserving every other header | ✅ | Task 7 — disabled by default, in which case the original instance is returned and nothing is allocated | +| RETRY-39 | MUST | Delay precedence: caller override → server pacing → fixed delay → exponential backoff | ✅ | Task 8 (`resolveDelay`); the exception path skips the header step, having no headers | +| RETRY-40 | MUST | A throwing user delay-override is non-fatal; a throwing should-retry predicate aborts the call | ✅ (override half) / N/A (predicate half) | Task 8 — the override throw is caught and the schedule used instead, asserted. The predicate half is unreachable: 5a exposes no user should-retry predicate, so there is no caller code on that path to throw. The "log it" clause is Phase 7b's Task 9 (see Cross-phase below). Ledgered | +| RETRY-41 | MUST | Effective retry count is present-override-wins; zero means no retries; a negative configured value is clamped to the default | ✅ (override) / 🚫 (clamp — rejected instead, ledgered) | Task 9 reads `ctx.options?.maxRetries` and runs the engine with `maxAttempts = maxRetries + 1`, asserted both narrowing (`0` → one send) and widening (`2` → three sends). The per-call value is **revalidated** at the step: `RequestOptionsBuilder` rejects only a negative, which is weaker than `retrySettings()`'s `Number.isFinite(...) && >= 1`, so `Infinity`/`NaN` would otherwise reach `maxAttempts` and make the engine's attempt gate permanently false — an unbounded retry loop reachable from the public options API. The clamp collides head-on with `HTTP-35` (also MUST), which REJECTS a negative max-retries at construction precisely so it cannot be silently reinterpreted as "use default". The port takes `HTTP-35`'s line on both surfaces — the builder rejects the option, `retrySettings()` trips `invariant()` on a negative `maxAttempts` | +| RETRY-42 | MUST | Settings and every policy component immutable, stateless, and safe for concurrent invocation | ✅ | Task 5 (frozen settings, defensively copied status set), Task 8 (attempt count and start instant are locals, asserted by two concurrent `runWithRetry` calls over one settings object), Task 9 (the per-call derivation re-freezes rather than handing back a bare spread of a frozen source) | +| RETRY-44 | MUST | Re-execute the downstream chain with FRESH per-attempt continuation state | ✅ | Task 9 — `ctx.fork()` once per attempt, 4c's mechanism's first consumer. Task 10 is the recovery-side mirror: each attempt re-runs the whole chain, asserted by counting request-chain applications. The second clause (upstream steps must not mutate the shared in-flight request) is free — `Request` is immutable and frozen | +| RETRY-29 | MAY | Opt-in server-driven retry-classification override header | ⏳ | Not scheduled. Widens the classifier's input surface to server-controlled values; wants an explicit trust decision, not a default. Deferred Items Log | + +## Appendix C — `RECOV-17`–`RECOV-34` + +Appendix C files eighteen `RECOV-*` rows under "Recovery-chain pipeline primitives" that +`08-execution-pipelines.md` §8.2 never defines in prose — they are retry-engine requirements stated a second +time for the reference's second retry stack. This port collapses both stacks into one engine (`RETRY-28`), so +they collapse onto the same implementation. Phase 9's conformance sweep should read this table rather than +re-deriving it. + +| Appendix C | `§9` equivalent | Status | Where | +|---|---|---|---| +| RECOV-17 | `RETRY-1`, `RETRY-4`, `RETRY-8`, `RETRY-37` | ✅ | Task 2 (`classify.ts`); reached on this entry point by Task 10 | +| RECOV-18 | `RETRY-5`, `RETRY-6`, `RETRY-7` | ✅ | Task 2 | +| RECOV-19 | `RETRY-36` | ✅ (narrowed as above) | Task 8 | +| RECOV-20 | `RETRY-27` | ✅ | Task 8 — both the abort and the clamp | +| RECOV-21 | `RETRY-9`, `RETRY-10`, `RETRY-11` | ✅ | Task 3 — the same formula verbatim | +| RECOV-22 | `RETRY-20` | ✅ | Task 8 | +| RECOV-23 | `RETRY-16`, `RETRY-17` | ✅ | Task 4 — totality is the property test | +| RECOV-24 | `RETRY-15`, `RETRY-19`, `RETRY-21` | ✅ | Task 4 | +| RECOV-25 | `RETRY-15` (`X-RateLimit-Reset` clause) | ✅ | Task 4 — positive jitter bounded to `[100%, 120%]` INSIDE the parser, so many clients released at one reset instant do not stampede. A literal `Retry-After` receives no additional jitter (`RETRY-20`) | +| RECOV-26 | `RETRY-11`, `RETRY-18` | ✅ | Tasks 3, 4 | +| RECOV-27 | `RETRY-23`, `RETRY-26` | ✅ | Task 8 | +| RECOV-28 | `RETRY-42` | ✅ | Task 8 — per-call locals, asserted concurrently | +| RECOV-29 | `RETRY-22` | ✅ | Structural (total parser), asserted in Task 8 | +| RECOV-30 | `RETRY-13`, `RETRY-14` | ✅ | Structural here — one engine, no second stack to drift from. Both adapters (Tasks 9, 10) call the same `runWithRetry` | +| RECOV-31 | `RETRY-38` | ✅ | Task 7 | +| RECOV-32 | **none** (net-new) | ✅ | Task 11 — `recovery/idempotency-key.ts`. Method-gated (default `{POST, PUT, PATCH}`, defensively copied), respect-existing by default with the strategy **not** invoked in that case, strategy invoked at most once per applicable request, never mutating the input | +| RECOV-33 | **none** (net-new) | ⏳ | Phase 7a Task 9. Client-identity header stamping has no retry coupling; it is configuration-driven, so it travels with `CFG-*` | +| RECOV-34 | partial (`RETRY-11`, `RETRY-41`) | ✅ (settings validation) / 🚫 (configurable retryable-method set) | Task 5 — construction validation rejects negative or non-finite durations, `multiplier < 1.0`, `maxAttempts < 1`, and `jitter` outside `[0,1]`; the status set is a defensive copy. **No configurable retryable-METHOD set ships**: `RETRY-6`/`HTTP-9` fix the idempotent set and make Phase 1's `method.ts` its single source, so there is nothing per-instance to copy and no requirement obliges configurability. Ledgered | + +## Cross-phase obligations + +| Obligation | Status | Where | +|---|---|---| +| `PIPE-36` — a shipped pillar family locks its stage assignment | ✅ | Task 9 — satisfied structurally: `retryStep()` is a factory returning a descriptor with `stage: 'RETRY'` baked in. There is no class to subclass and no way for a caller to relocate it. Deferred out of 4c to "whichever future phase ships the first real pillar step family" — that is this one | +| `PIPE-17` — the caller's per-call options readable by any step | ✅ | Task 1 — `StepContext.options`, populated from the cursor's existing field, shared by reference across every fork. Previously threaded only into the terminal dispatch, leaving the clause unsatisfied outright | +| `StepContext.signal` — the 4c amendment | ✅ | Task 1 — additive and optional; no behavior change for any step that ignores it. `RETRY-26`'s cancellable wait and `RETRY-32`'s no-further-attempts rule are both unimplementable without it, and 5b/5c need the same access | +| 2026-07-28 Phase 7a retrofit (`Clock`, RFC 1123 parser, retryable-status single-sourcing) | ✅ | Applied — the three `config/` files exist and are imported, not duplicated. See the prerequisite-slice table above | +| 2026-07-28 Phase 7b retrofit (two `SHOULD`-level structured log events in `engine.ts`) | ⏳ Phase 7b Task 9 | **Deliberately NOT applied here**, per this plan's own 2026-07-29 correction: 5a executes before 7b, so an `observability/logger.js` import would not resolve, and 7b needs 5a's `FakeTransport`, so the dependency cannot run the other way. `engine.ts` carries a comment at its head marking both emission points and naming 7b's Task 9 as their owner. `RETRY-40`'s "log and fall back" is the same row — the fall-back half ships here, the log half there | +| `FakeTransport` — the twice-punted shared double | ✅ | Task 6 — `packages/core/src/testing/fake-transport.ts`, `@internal`. Scripted response sequences (last entry repeats), wire-send counting, and `countingResponse()`, the only sanctioned way to observe `Response.close()`: `Response` is frozen, so a spy over `close` throws. The counter observes release by BOTH routes — `cancel()` for an abandoned response, `pull()`-to-EOF for one the engine retired through `toHttpError()`'s bounded drain — because a helper counting `cancel()` alone would read zero on exactly the `RETRY-35` path it exists to prove | +| Node-runtime conformance (`CLAUDE.md`'s membership rule) | ✅ | `test/node-conformance/retry.test.mjs` — the `TimeoutError`-name classification that `RETRY-24` keys off (asserted against a real `AbortSignal.timeout()` with a ref'd deadline), the suppressed-trail shape across the native/fallback split, and release-on-discard over Node's own Web Streams | +| Public barrel unchanged | ✅ | Task 12 — `git diff --exit-code` on `core.api.md` and `index.ts` is empty. 4c left "do we publish a step-authoring surface" to the first phase shipping a pillar step; this phase answers **not yet**, because a caller cannot assemble a working pipeline until 5c ships the standard-resilience preset, and publishing `retryStep` alone would freeze `StepDescriptor`/`Stage`/`PipelineBuilder` shapes 5c may still reshape | + +## Deferred out of Phase 5a + +| Item | Target | Reason | +|---|---|---| +| `RETRY-29` — opt-in server-driven retry-classification override | Not scheduled | `MAY`. Widens the classifier's input surface to server-controlled values; wants an explicit trust decision, not a default | +| `RECOV-33` — client-identity header step | Phase 7a (Task 9) | Configuration-driven header composition with no retry coupling; belongs with `CFG-*` | +| Public-barrel promotion of `retryStep` and the step-authoring surface | Phase 5c | Needs the standard-resilience preset (`PIPE-24`, `PIPE-39`) and `PIPE-35`'s `seedFrom`, which need all three pillar steps installed | +| The two structured retry log events (`retry.attemptFailed`, `retry.exhausted`) | Phase 7b (Task 9) | Cycle-breaking: 5a cannot import `observability/`, 7b needs 5a's `FakeTransport` | diff --git a/docs/superpowers/specs/2026-07-26-phase5a-retry-design.md b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-26-phase5a-retry-design.md rename to docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md index 3eabc69..d690617 100644 --- a/docs/superpowers/specs/2026-07-26-phase5a-retry-design.md +++ b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md @@ -15,8 +15,8 @@ Retry/Redirect/Auth") splits into: **5a** (this document, retry), 5b (redirect, > this document) is re-sourced from Phase 7a's shared `config/http-date.ts` rather than staying a private > copy; and `classify.ts`'s `RETRYABLE_STATUSES`/`isRetryableStatus` (`RETRY-1`) are re-exported from Phase 7a's > `config/retryable.ts` (`CFG-35`) rather than defined here a second time. See -> `docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md`'s Scope section for the rationale; see -> the amended `docs/superpowers/plans/2026-07-26-phase5a-retry.md` for the concrete diffs. +> `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md`'s Scope section for the rationale; see +> the amended `docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md` for the concrete diffs. **Governing documents:** `docs/product-spec/09-retry-and-resilience.md` (normative, cited by ID throughout), `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md` (`RECOV-17`–`RECOV-34`), diff --git a/docs/superpowers/plans/2026-07-26-phase5a-retry.md b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md similarity index 98% rename from docs/superpowers/plans/2026-07-26-phase5a-retry.md rename to docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md index 0be4314..910f116 100644 --- a/docs/superpowers/plans/2026-07-26-phase5a-retry.md +++ b/docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md @@ -6,14 +6,14 @@ pacing-header parser, validated settings, the attempt loop, per-attempt stamping, the idempotency-key recovery step, and the two thin adapters binding the engine to the stage pipeline (4c) and the recovery chain (4b) — satisfying `product-spec/09-retry-and-resilience.md` (`RETRY-1`–`RETRY-45`) and appendix C's -`RECOV-17`–`RECOV-34`, per `docs/superpowers/specs/2026-07-26-phase5a-retry-design.md`. +`RECOV-17`–`RECOV-34`, per `docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-design.md`. > **Amended 2026-07-28 (Phase 7a retrofit):** `RetryConfig.now`/`RetryStepOptions.now` are retyped to > `clock: Clock`, consuming Phase 7a's `config/clock.ts` seam instead of an ad hoc `() => number`; > `pacing.ts`'s private RFC 1123 parser is replaced by an import from Phase 7a's `config/http-date.ts`; and > `classify.ts`'s private `RETRYABLE_STATUSES`/`isRetryableStatus` are replaced by a re-export from Phase 7a's > `config/retryable.ts` (CFG-35). All three are single-sourcing corrections, not behavior changes — see -> `docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md`'s Scope section. This plan's execution now +> `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md`'s Scope section. This plan's execution now > depends on Phase 7a's `config/` module existing first (see Prerequisite below); every other task is > unaffected. > @@ -28,7 +28,7 @@ satisfying `product-spec/09-retry-and-resilience.md` (`RETRY-1`–`RETRY-45`) an > neither side could break. **An agent executing this plan must skip the Phase 7b retrofit blocks in Task 8** > and build `engine.ts` without any `observability/` import; Phase 7b's plan Task 9 adds the two emission > sites afterwards. The blocks stay here as the specification of what Task 9 will write. See -> `docs/superpowers/plans/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. +> `docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. **Architecture:** A new `packages/core/src/retry/` folder of eight independent files with no folder-level barrel, plus one file in `src/recovery/` and one in a new `src/testing/`. The engine core is pure functions — @@ -38,14 +38,24 @@ classification, backoff math, and pacing parsing take no I/O and no clock — dr engine, not two stacks** — `RETRY-28` explicitly instructs a unifying port to make the total-timeout opt-in, which `RetrySettings.totalTimeoutMs` does. -**Tech Stack:** TypeScript 5.8+, native `SuppressedError`, `fast-check` for the four invariant-bearing pure +**Tech Stack:** TypeScript 5.8+, Phase 4b's guarded `suppress()` helper (never native `SuppressedError` — see below), `fast-check` for the four invariant-bearing pure functions, `bun test`. No new runtime dependencies — `SEAM-1` untouched. No `node:` imports — core's zero-`node:` invariant, mechanically enforced since the scaffold, still holds (the RFC 1123 parser and the timer are both platform-neutral). +> ### ✅ F1 CLOSED — use `suppress()`, not `new SuppressedError(...)` +> +> Resolved 2026-08-26 in Phase 4b as branch (b): `packages/core/src/suppress.ts` ships +> `suppress(error, suppressed, message)` — native `SuppressedError` when `globalThis.SuppressedError` exists, a +> shape-compatible stand-in (`name`, `error`, `suppressed`) when it does not. The native class reached Node only +> in **24.0.0** and `engines.node` is `>=20.3`, so the direct form neither type-checks (not in this package's +> `lib`) nor runs on the floor. Every `new SuppressedError(...)` below becomes `suppress(...)`, and every +> `toBeInstanceOf(SuppressedError)` becomes an assertion on that shape — the `instanceof` form would silently +> assert nothing on the floor runtime. + **Prerequisite:** This plan assumes Phases 0, 1, 2, 3a, 3b, 4a, 4b, and 4c are implemented exactly as their plans specify, **plus Phase 7a's `Clock` seam** (added by the 2026-07-28 Phase 7a brainstorm's retrofit — see the -"`Clock` retrofit" note in `docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md`, Scope section). +"`Clock` retrofit" note in `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md`, Scope section). This inverts this plan's original numeric ordering relative to Phase 7; 7a's `config/clock.ts` must exist before Task 8 of this plan can be executed. Concretely: @@ -1846,7 +1856,7 @@ Expected: FAIL — `Cannot find module './engine.js'`. import {HttpStatusError, toHttpError} from '../body/http-status-error.js'; import type {Clock} from '../config/clock.js'; // Phase 7b retrofit: getGlobalLogger() call sites below, narrow blast radius (only this file's own emission -// points; no other phase depends on them). See docs/superpowers/specs/2026-07-28-phase7b-observability-design.md's +// points; no other phase depends on them). See docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md's // "Amendments to 5a and 5b" section. import {getGlobalLogger} from '../observability/logger.js'; import type {Request} from '../http/request.js'; @@ -2693,7 +2703,7 @@ git commit -m "feat(core): idempotency-key request recovery step" **Files:** - Verify unchanged: `packages/core/etc/core.api.md` - Verify unchanged: `packages/core/src/index.ts` -- Create: `docs/superpowers/plans/2026-07-26-phase5a-retry-checklist.md` +- Create: `docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md` **Interfaces:** - Consumes: every symbol from Tasks 1–11. @@ -2730,7 +2740,7 @@ classifier's `TimeoutError`-name check is the one place a runtime difference wou - [ ] **Step 4: Write the requirement checklist** -Create `docs/superpowers/plans/2026-07-26-phase5a-retry-checklist.md` in the same format as +Create `docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md` in the same format as `2026-07-24-phase3a-io-contracts-checklist.md` — `| ID | Level | Requirement gist | Status | Where |` tables, legend ✅ shipped / 🚫 never built / ⏳ deferred / N/A. @@ -2762,15 +2772,15 @@ State explicitly at the top whether the plan has been executed, matching the Pha The design doc's Deferred Items table is headed "add to the roadmap's Deferred Items Log" — writing the checklist does not discharge that. Append both rows to the log in -`docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`, in the log's existing column shape: +`docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`, in the log's existing column shape: `RETRY-29` (opt-in server-driven retry-classification override, not scheduled) and `RECOV-33` (client-identity header step, Phase 7a). Do not restate the justifications — link to this phase's design doc. - [ ] **Step 6: Commit** ```bash -git add docs/superpowers/plans/2026-07-26-phase5a-retry-checklist.md \ - docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md +git add docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry-checklist.md \ + docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md git commit -m "docs: Phase 5a requirement checklist" ``` diff --git a/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md new file mode 100644 index 0000000..b9d5174 --- /dev/null +++ b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md @@ -0,0 +1,139 @@ +# Phase 5b — Redirect Implementation Plan — Checklist + +Verification of [2026-07-26-phase5b-redirect.md](./2026-07-26-phase5b-redirect.md) against every requirement +ID in `docs/product-spec/10-redirect-handling.md` (`REDIR-1`–`REDIR-28`) plus `PIPE-40`, as dispositioned by +`docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md`. + +**Status: EXECUTED (2026-08-27).** Every task below is implemented, tested, and green across the full gate +sequence (`typecheck`, `lint`, `build`, `bun test` with coverage, `api:ci`, `lint:publish`, +`verify:dual-consumption`, `verify:consumer-types`, `verify:seam-1`, `verify:runtime-floor`, `test:node`, +`audit`). `packages/core/etc/core.api.md` and `packages/core/src/index.ts` are byte-identical to this phase's +starting point (`862bb46`, Phase 5a) — nothing in this phase reaches the public barrel. (Stated against +the branch point, not `main`: `main` currently sits three commits back at `8e55792`, so a diff against +it would show Phases 3, 4, and 5a's barrel changes and prove nothing about this one.) + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +## Files shipped + +| File | Requirements | Task | +|---|---|---| +| `packages/core/src/redirect/errors.ts` | `REDIR-6`, `REDIR-15` | 1 | +| `packages/core/src/redirect/codes.ts` | `REDIR-1`–`REDIR-5` | 2 | +| `packages/core/src/redirect/cross-origin.ts` | `REDIR-8`, `REDIR-11` | 3 | +| `packages/core/src/redirect/settings.ts` | `REDIR-17`, `REDIR-20`, `REDIR-26`, `REDIR-27` | 4 | +| `packages/core/src/redirect/decide.ts` | `REDIR-1`–`REDIR-21` | 5 | +| `packages/core/src/redirect/redirect-step.ts` | `REDIR-22`, `REDIR-23`, `PIPE-15`, `PIPE-36`, `PIPE-40` | 6 | +| `packages/core/src/redirect/strip-marker-step.ts` | `REDIR-11`(c) | 7 | +| `packages/core/src/recovery/release.ts` | `RECOV-12`, `RETRY-22`, `REDIR-22`(b) | review pass 1 | +| `test/node-conformance/redirect.test.mjs` | `REDIR-12`–`REDIR-14`, `REDIR-18`, `PIPE-40` on Node | 6 | + +Every production file has a colocated `*.test.ts`; 124 tests across the eight pairs. + +`recovery/release.ts` was not in the plan. It is `releaseQuietly`/`withReleaseFailure`, **extracted +unchanged** from `retry/engine.ts` during review pass 1 so this phase's two error paths consume them +rather than shipping a second copy of a helper whose identity guard is load-bearing. The move is +behavior-neutral for 5a — its suite passes untouched — and `engine.ts` now imports what it used to +define. + +## 10.1 Recognized codes and eligibility + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-1 | MUST | Redirect attempted only for 301/302/303/307/308; any other status returned verbatim without consulting redirect logic | ✅ | Task 2 (`REDIRECT_STATUSES`, `isRecognizedRedirect`), Task 5 (`decide()`'s first statement, before any allocation) | +| REDIR-2 | MUST | 300/304/305 never auto-followed even with a Location; 305 never redirects to a server-chosen proxy | ✅ | Task 2 — excluded from the set by construction; asserted for all three codes *with* a Location present | +| REDIR-3 | MUST | 301/302 followed only when the ORIGINAL method is in the allowed set (default `{GET, HEAD}`); method AND body preserved, no automatic POST→GET rewrite | ✅ | Task 2 (`isEligibleByCode`), Task 5 (`buildFollowRequest` carries `current.method` and the builder-prefilled body through) | +| REDIR-4 | MUST | 307/308 preserve method and body, followed only when the method is in the allowed set | ✅ | Task 2 — same predicate; 303 is the only status branched on, so the four method-preserving codes cannot drift apart | +| REDIR-5 | MUST | 303 not followed by default; when opted in, re-issued as GET with the body dropped and every `Content-*` header removed case-insensitively; the original method is irrelevant to whether it is followed | ✅ | Task 2 (the `allow303`-only gate, asserted against an *empty* allowed-method set), Task 5 (`stripContentHeaders`, `method: 'GET'`, `body(undefined)`) | + +## 10.2 Body and credential hygiene + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-6 | MUST | A followed method-preserving redirect re-sends the body, so it MUST be replayable; a non-replayable body fails with a clear error naming replayability, and the redirect is not attempted. 303 exempt | ✅ | Task 1 (`NonReplayableBodyError`), Task 5 — the gate is evaluated **before** any write is attempted, which is what separates it from 3b's `ConsumedBodyError` (a second-write failure). 303's exemption asserted with a single-use body | +| REDIR-7 | MUST | `Authorization` stripped before EVERY re-issue — same-origin and the 303 GET rebuild included | ✅ | Task 5 (`nextHopHeaders`, unconditional), asserted same-origin, cross-origin, on the 303 rebuild, and on a permitted downgrade; re-asserted end-to-end against the wire in Task 6 | +| REDIR-8 | MUST | Cross-origin iff the resolved target differs from the SEED origin in scheme, host (case-insensitive), or effective port (default when omitted) — never the immediately preceding hop | ✅ | Task 3 (`originOf`/`isCrossOrigin`) — the seed origin is computed once in the step and never advances with the chain (Task 6). A `fast-check` property asserts path/query/fragment never participate | +| REDIR-9 | MUST | On a cross-origin redirect (303 rebuild included), `Cookie` and `Proxy-Authorization` also stripped | ✅ | Task 5 | +| REDIR-10 | SHOULD | On a same-origin redirect the `Cookie` header is retained; only `Authorization` is stripped | ✅ | Task 5, asserted directly (both headers survive a same-origin hop) | +| REDIR-11 | MUST | A cross-origin re-issue carries an out-of-band signal telling the auth layer to skip stamping: (a) unforgeable — cleared on every re-issue before being conditionally set, (b) suppress-only, never causing a credential to be sent, (c) removed by the credential-attaching layer before dispatch | ✅ (a, b, c) | Task 3 (`CROSS_ORIGIN_MARKER_HEADER`, `withCrossOriginMarker` clears-then-sets in one `set` call), Task 5 (cleared unconditionally, set only when cross-origin), Task 7 (the `POST_AUTH` guard). (b) holds structurally — nothing in 5b reads the marker to *cause* a stamp; 5c's auth step is its first consumer. The porter caveat the requirement itself names ("a pipeline with no auth step forwards the internal marker to the transport") is closed here rather than left to 5c — see the cross-phase table **Review pass 1:** the guard step now early-returns when the marker is absent instead of rebuilding `Headers` and `Request` on every request through the pipeline, and `withRedirect()` removes any existing guard before re-installing, so a second call cannot seat a duplicate (`append` dedupes by `type` only for pillar stages) | +| REDIR-12 | MUST | Userinfo in the Location target dropped before re-issue; server-supplied embedded credentials never used | ✅ | Task 5 (`resolveLocation`), asserted in `bun test` and again on Node's own URL parser | + +## 10.3 Location resolution + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-13 | MUST | Resolution preserves the wire-exact, already-percent-encoded path/query/fragment, bracketed IPv6 literal hosts, and explicit ports; `%2F`→`/` or `%26`→`&` re-encoding forbidden | ✅ | Task 5 — nothing decodes or re-encodes; the only mutation is clearing userinfo. Asserted for `%2F`/`%26` and for `[2001:db8::1]:8443`, and repeated in `test/node-conformance/redirect.test.mjs` because the parser is the runtime's, not this package's | +| REDIR-14 | MUST | A relative Location resolved against the CURRENT hop's request URL per RFC 3986; absolute values used as-is after userinfo stripping | ✅ | Task 5 — `new URL(raw, currentUrl)`. The two-hop test in Task 6 uses a *relative* second Location precisely so "current hop, not seed" is load-bearing | +| REDIR-18 | MUST | A malformed or unresolvable Location — invalid URI, illegal characters, or an unsupported/unknown scheme — MUST NOT throw; the step returns the current response unfollowed | ✅ (total) / ⏳ (the log) | Task 5 — totality asserted by a `fast-check` property over arbitrary strings sanitized only to what the *lenient* inbound header validator admits. The unsupported-scheme half needed an explicit `http:`/`https:` gate: WHATWG `URL` parses `javascript:`, `data:`, `file:`, and `mailto:` without complaint and the downgrade guard passes all of them. The requirement's "logs the condition" clause is deferred — see the deferral table | +| REDIR-19 | MUST | A missing or empty Location returns the response unfollowed | ✅ | Task 5, asserted for both the absent header and an empty value | +| REDIR-27 | MAY | The header the target is read from is configurable, default `Location` | ✅ | Task 4 (`locationHeader`), Task 5 (read through the setting), asserted with a custom header name. **Review pass 1:** the value was validated non-blank but stored untrimmed and never checked against the header-name grammar, while `Headers.get()` neither trims nor validates — so `' Location '` was accepted and then silently matched nothing, leaving every redirect unfollowed with no error at any layer. Now trimmed before storage and validated with `hasForbiddenNameByte` (HTTP-17), the same guard 5a applies to `attemptHeaderName` | + +## 10.4 Loop, cap, and downgrade + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-15 | MUST | An HTTPS→HTTP downgrade across a single hop rejected by default with a clear error; opt-in permits it but MUST surface it observably; credential stripping applies regardless; evaluated per hop transition | ✅ (rejection, opt-in, stripping) / ⏳ (the observable surfacing) | Task 1 (`SchemeDowngradeError`), Task 5 — keyed to the CURRENT hop's scheme, not the seed's, so an HTTPS→HTTP→HTTPS chain flags only the hop that actually downgraded; asserted directly. Credential stripping on a *permitted* downgrade asserted separately. The "surface it observably" clause is a distinct obligation on the permitted path and is deferred with the rest of redirect's logging — see the deferral table | +| REDIR-16 | MUST | Loops detected by recording every visited absolute URI (seeded with the original request URI); revisiting one stops and returns the CURRENT response WITHOUT throwing, body left open | ✅ | Task 5 (the `visited` check), Task 6 (the set is seeded with the seed request's `href` and grown per followed hop). Asserted end-to-end: the loop response comes back identical and with `cancelCount() === 0` | +| REDIR-17 | MUST | Followed redirects capped by `maxHops` (default 3); on reaching the cap the last response is returned as-is even if itself a 3xx, without throwing; `maxHops: 0` disables following entirely | ✅ | Task 4 (default 3, `0` accepted as an ordinary value; **review pass 1** tightened the guard from finite-and-non-negative to `Number.isInteger`, so a fractional budget is rejected rather than silently truncated), Task 5 (`redirectsFollowed + 1 > maxHops`). Asserted end-to-end with a 4th 301 past a 3-hop cap — returned open, still a 301 — and with `maxHops: 0` on the first response. No special-case branch exists for `0`; the same gate produces it | +| REDIR-23 | SHOULD | Iterative loop, not unbounded recursion, so it is stack-safe regardless of `maxHops` | ✅ | Task 6 — a `for(;;)` with `await`; each iteration's frame is released before the next begins | + +## 10.5 The predicate and the fast path + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-20 | MUST | A configured predicate fully overrides the built-in follow decision and receives a read-only, DEFENSIVELY COPIED condition snapshot (current response, redirects already followed, insertion-ordered visited set including the current request's URI) so it cannot mutate live cycle-detection state | ✅, scoped | Task 4 (`RedirectCondition`/`RedirectPredicate`), Task 5. The snapshot is a real `new Set(visited)` copy, not the live set typed `ReadonlySet` — the type is erased at runtime, and the assertion that a predicate casting it away cannot poison loop detection is a direct test. **The override is scoped to code/method eligibility only**, not to the safety mechanics that follow it (userinfo stripping, credential hygiene, downgrade rejection, replayability, loop/cap) — a judgment call on ambiguous wording, recorded in the design doc's Deviation Ledger and asserted as ledgered behavior | +| REDIR-21 | SHOULD | The non-redirect fast path short-circuits before allocating a snapshot and MUST NOT consult a predicate; a recognized 3xx ALWAYS allocates the snapshot and consults the predicate, even with no usable Location | ✅ | Task 5 — the recognized-status check is `decide()`'s first statement, asserted by a predicate that records whether it was called; the "even with no usable Location" half asserted with a 301 carrying no Location at all | + +## 10.6 Lifecycle, ordering, and immutability + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-22 | MUST | Deterministic response-body lifecycle: (a) the prior redirect response closed before issuing a follow-up; (b) if building the follow-up throws, the current response closed before the error propagates; (c) on any "return current" outcome the response is left OPEN for the caller | ✅ | Task 6 — (a) the two-hop conformance test counts exactly one release per superseded hop; (b) `decideOrClose` wraps the decision because `decide()` invokes caller predicate code, asserted with a throwing predicate *and* with the downgrade rejection, both leaving `cancelCount() === 1`; (c) asserted on not-a-redirect, loop-detected, hop-cap, and cancelled paths. The error from (b) is rethrown **unchanged** — redirect's spec states no conversion, unlike `RETRY-40`. **Review pass 1:** both (b) paths originally did a bare `await response.close()`, which `Response.close()` is documented to reject from — so a failing release replaced the very error that was supposed to propagate. They now go through `releaseQuietly`/`withReleaseFailure`, keeping the decision error primary with the release failure as `suppressed` (`RECOV-12`); asserted for `SchemeDowngradeError` and for a caller predicate's own error. Path (a) is deliberately NOT quieted: there is no primary error to preserve, and `PIPE-40` makes the release part of the contract | +| REDIR-24 | MUST | The redirect follower wraps the credential-attaching layer — redirect OUTER, auth INSIDE, per hop | ✅ | Structural — 4c's `STAGE_ORDER` places `REDIRECT` before `AUTH`, and `redirectStep` is pinned to the `REDIRECT` pillar (`PIPE-36`), so a caller cannot invert the two. The clause's *consequence* — `REDIR-7`'s unconditional strip plus `REDIR-11`'s suppression signal — ships here; the auth step that runs inside the loop is 5c | +| REDIR-25 | MUST | The asynchronous pipeline MUST NOT follow redirects: no async redirect step ships, no async preset installs one, so a 3xx surfaces to the async caller verbatim | ✅ (preserved) | Structural — this phase ships **one** adapter, not 5a's two. There is no async redirect step and nothing to install one. The asymmetry with the sync pipeline is preserved rather than changed, so no documentation of a deviation is owed | +| REDIR-26 | MUST | The allowed-method set stored as an immutable defensive copy, decoupled from the caller's collection | ✅ | Task 4 — `new Set(merged.allowedMethods)`, asserted by mutating the caller's set after construction. Deliberately a copy and not a frozen `Set`: `Object.freeze` is shallow and does not disarm `Set.prototype.add`, so a "frozen set" would be a guarantee the runtime cannot keep. The settings object itself IS frozen | + +## 10.7 Observability + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| REDIR-28 | SHOULD | Each followed hop, loop detection, and scheme-downgrade event emitted as structured records, URLs through a redactor, redaction failures degraded to a placeholder; the malformed-Location event logs the raw string as the stated exception | ⏳ | **Phase 7b, Task 9.** 5b executes before 7b, so an `observability/logger.js` import here would not resolve — and 7b's own retrofit conformance test needs 5b's redirect step, so the dependency cannot run the other way. `redirect-step.ts` carries a TSDoc note at `redirectStep()` naming 7b's Task 9 as the owner. Same disposition, and the same cycle-breaking reason, as 5a's two `engine.ts` events | + +## Cross-cutting invariants (`§19`) — what appendix B actually checks for redirect + +Appendix B carries **no `REDIR-`-prefixed checkbox at all**. Redirect reaches its conformance checklist only +through the cross-cutting line at `appendix-b-conformance-test-checklist.md:81`, so these are the rows Phase 9 +will look for. + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| XCUT-16 | MUST | No credential stamped over a non-HTTPS transport; the guard applies only on the credential-attaching path, and a deliberately credential-free re-issue — explicitly "a marker-suppressed cross-origin redirect" — MAY proceed over any scheme | ✅ (5b's half) | Structural. 5b never attaches a credential; it only strips (`REDIR-7`) and signals suppression (`REDIR-11`). The carve-out this requirement names is exactly what `cross-origin.ts`'s marker produces. The enforcing half is 5c's auth step | +| XCUT-17 | MUST | Redirect credential hygiene: (a) strip `Authorization` before every re-issue, even same-origin; (b) cross-origin — judged against the seed, not the previous hop — additionally strip `Cookie`/`Proxy-Authorization` and ensure the caller's credential is not re-applied to the foreign host; (c) drop userinfo in the Location; (d) reject an HTTPS→HTTP downgrade by default, opt-in only, logging the deviation | ✅ (a, c, d-rejection) / ✅ (b, stripping half) / ⏳ (b's re-application half, and d's logging) | (a) Task 5, asserted same-origin, cross-origin, on the 303 rebuild, and on a permitted downgrade; every value stripped regardless of header casing. (b) Task 5 strips both, seed-judged; "not re-applied to the foreign host" needs an auth layer to refrain, so it is 5c's to close via the marker 5b produces. (c) Task 5, asserted in `bun test` and on Node's own parser. (d) Task 5 rejects by default with `SchemeDowngradeError` and permits only via `allowSchemeDowngrade`; the logging half travels with `REDIR-28` to Phase 7b | +| XCUT-19 | MUST | Default-deny log redaction of userinfo/query/fragment/headers/credentials/bodies | N/A in 5b | Vacuous while 5b emits nothing. Becomes live with Phase 7b's Task 9, which routes every URL field through `redactUrl()` | +| XCUT-20 | MUST | Observability never throws into the request path | N/A in 5b | Same — no emission sites exist here yet. 7b's `emitQuietly()` owns it | + +## Cross-phase obligations + +| Obligation | Status | Where | +|---|---|---| +| `PIPE-40` — a wrapping step releases every superseded intermediate response and never closes the one it hands back; on an abandoned re-drive the in-flight response is returned unclosed | ✅ **Resolved here** | Task 6's two-hop `FakeTransport` conformance test: three wire sends, exactly one release per intermediate response observed through `countingResponse()`'s stream hook, and `cancelCount() === 0` on the final response. Deferred out of 4c and targeted at "the first redirect step" by the roadmap — that is this phase. The abandon clause is asserted on the paths that genuinely return — loop detected, hop cap, and cancellation — each with `cancelCount() === 0`. **Its fourth named path, non-replayable body, is NOT one of them, and that is deliberate:** `PIPE-40` lists it among the responses "returned unclosed" while `REDIR-22`(b) lists the same trigger among those "closed before the error propagates", and `REDIR-6` settles the control flow by requiring that path to *fail with an error* rather than return. 5b closes and throws; the contradiction is recorded in the design's Deviation Ledger and deferred to Phase 10, and `redirect-step.test.ts` asserts the close-then-throw behavior with the reasoning inline | +| `PIPE-15` — a step that re-drives the chain takes a FRESH continuation per drive | ✅ | Task 6 — every dispatch, including the first, goes through `ctx.fork()`; `ctx.next()` is never called, since its single-invocation guard would trip on hop two | +| `PIPE-36` — a shipped pillar family locks its stage assignment | ✅ | Task 6 — satisfied structurally, as 5a's `retryStep` was: a factory returning a descriptor with `stage: 'REDIRECT'` baked in. Nothing to subclass, nothing to relocate | +| `PIPE-3` — the inert extension slots around each pillar | ✅ (consumed) | Task 7 — `stripCrossOriginMarkerStep()` is the first real occupant of `POST_AUTH`, which 4c shipped inert. No change to 4c was needed | +| `StepContext.signal` (5a's Task 1 amendment) | ✅ (consumed) | Task 6 — checked once per iteration, in the `follow` branch, before closing the hop and re-driving. No cancellable *wait* is needed here (unlike retry, nothing sleeps between hops), so this is one cheap read rather than a timer race | +| `FakeTransport` (5a's `@internal` double) | ✅ (reused unchanged) | Tasks 6 and 7 — consumed exactly as the roadmap said 5b and 5c would, with no edits to `testing/fake-transport.ts` | +| Node-runtime conformance (`CLAUDE.md`'s membership rule) | ✅ | `test/node-conformance/redirect.test.mjs` — Location resolution is delegated wholesale to the platform's WHATWG `URL`, an independent implementation on each runtime, and `PIPE-40`'s close counting rides on Web Streams. Thirteen cases: relative resolution, dot segments, protocol-relative, `%2F`/`%26` preservation, bracketed IPv6 with an explicit port, userinfo clearing, case/default-port normalization (what makes `REDIR-16`'s loop detection hold, since `visited` keys on `href`), non-URL-as-relative-reference, the malformed-absolute throw, the unsupported-scheme gate, and the three lifecycle paths | +| Public barrel unchanged | ✅ | Task 8 — `git diff --exit-code` on `core.api.md` and `index.ts` is empty, and `src/redirect/` gets no `index.ts`. Same "not yet" disposition 5a's `retry/` shipped with: 5c's promotion task is the first point any pillar-authoring surface goes public | + +## Deferred out of Phase 5b + +| Item | Target | Reason | +|---|---|---| +| `REDIR-28` — the hop, loop-detected, downgrade, and malformed-Location log events | Phase 7b (Task 9) | Cycle-breaking: 5b cannot import `observability/` (it does not exist at this plan's execution time), and 7b needs 5b's redirect step for its own retrofit conformance test. `redirect-step.ts` names 7b's Task 9 as the owner in its TSDoc | +| `REDIR-15`'s "surface it observably" clause on a *permitted* downgrade | Phase 7b (Task 9) | Travels with `REDIR-28`. The opt-in flag and the credential-stripping half both ship here; only the warning-level emission is outstanding. Note this is one obligation, not two: setting a boolean in a config file a year ago is not surfacing anything about the request that actually took the downgrade | +| A reason discriminant on `Decision`'s `'return-current'` variant | Not scheduled | 7b's amendment already flags this: without it, logging cannot distinguish loop-detected from hop-cap-exceeded from normal termination, so those two of `REDIR-28`'s four events stay open even after Task 9. Reshaping `Decision` touches every assertion in `decide.test.ts`; it is a `SHOULD`, so it did not earn that churn inside this phase | +| `AUTH-29` / the marker's *consumption* side — skip-stamping on a cross-origin re-issue, and the auth step's first-stripper role | Phase 5c | 5b only **produces** the marker and **defends** it with an independent guard. Nothing yet reads it for its intended purpose. When 5c ships, `stripCrossOriginMarkerStep()` stays installed as a redundant, idempotent backstop | +| `PIPE-2`'s auth-re-runs-per-hop clause | Phase 5c | Needs an auth step to re-run | +| The standard-resilience preset and public-barrel promotion of `redirectStep`/`withRedirect` | Phase 5c | The preset needs all three pillars installed; publishing a pillar-authoring surface early would freeze `StepDescriptor`/`Stage`/`PipelineBuilder` shapes 5c may still reshape | +| The predicate-override scope judgment (`REDIR-20`) | Phase 9 conformance sweep, or sooner | A judgment call made without the user present. Narrow and mechanical to reverse if wrong: gate `decide()`'s step 3 onward behind the predicate's answer. Recorded in the design doc's Deviation Ledger | diff --git a/docs/superpowers/specs/2026-07-26-phase5b-redirect-design.md b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md similarity index 91% rename from docs/superpowers/specs/2026-07-26-phase5b-redirect-design.md rename to docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md index 480ee7a..0ea50af 100644 --- a/docs/superpowers/specs/2026-07-26-phase5b-redirect-design.md +++ b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md @@ -339,6 +339,14 @@ Wiring redirect's log call sites is a one-file addition once Phase 7's `Logger` against a facade that doesn't exist yet would mean guessing its shape twice. Not re-litigated; consistent with 5a's precedent. +> **Amended again 2026-08-27 (execution).** The retrofit below is written as though it had already landed in +> this file's own code. It has not: Phase 5b executes *before* Phase 7b, so `redirect-step.ts` cannot import +> `observability/logger.js` or `observability/redaction.js` — and 7b needs 5b's redirect step for its own +> retrofit conformance test, so the dependency cannot run the other way. Phase 5b as executed emits **no** +> log events; `redirectStep()`'s TSDoc names Phase 7b's plan Task 9 as their owner, and the plan's own +> 2026-07-29 correction says the same. Read the paragraph below as 7b's specification of what it will add +> here, not as a description of shipped code. + *Current disposition:* `redirect-step.ts` emits three events via `getGlobalLogger()` — a per-hop event, a rejection event on the `'fail'` path, and the permitted-downgrade event described under "Scheme-downgrade guard". Two constraints bind every one of them, and neither is optional: @@ -399,8 +407,9 @@ hints — that parser is 5a's `pacing.ts`, untouched here). | Location resolution ends with an explicit `http:`/`https:` followable-scheme gate | Spec states "an unsupported scheme" is returned unfollowed, without saying how it is detected | WHATWG `URL` happily parses `javascript:`, `data:`, `file:`, and `mailto:`, and the scheme-downgrade guard passes them (none is `http:`) — without the gate the step would dispatch a server-supplied `javascript:` target | | The predicate's `RedirectCondition.visited` is a defensive copy, not the live set typed `ReadonlySet` | Spec: "a read-only, defensively-copied condition snapshot… so it cannot mutate the live cycle-detection state" | A `ReadonlySet` type annotation is erased at runtime; a predicate that casts it away could pre-seed or clear loop detection for the rest of the call. The spec's wording is about the object, not the type | | `maxHops: 0` is an ordinary cap value, not a special-cased early return | Spec states it as "disables redirect following entirely" | Falls out of the same cap gate every other `maxHops` value uses — a 0-hop budget always fails the "would this exceed the cap" check on the first follow attempt, producing identical observable behavior with no branch to get wrong | +| The non-replayable-body path CLOSES the in-flight response and throws, rather than returning it unclosed | **`PIPE-40` and `REDIR-22` contradict each other here, both at `MUST` level.** `PIPE-40`: "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `REDIR-22`: "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." | Resolved for `REDIR-22`, on three grounds. (1) `REDIR-6` independently settles the control flow — "the operation MUST fail with a clear error naming replayability" — so this path throws, and a response that is never *returned* cannot be "returned unclosed"; `PIPE-40`'s clause is written about a returned value. (2) Specific governs general: `§10` owns the redirect step's lifecycle, `PIPE-40` states the cross-cutting default. (3) Closing is the safer reading — the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths (cycle, budget) DO return, and both return unclosed as it requires. Flagged for Phase 10; if reversed, the change is one branch in `redirect-step.ts` | | No stage-pipeline recovery-chain adapter | 5a shipped two adapters (pillar + recovery) over one retry engine | `pipeline.md`/`PIPE-*` states plainly there is no async redirect pillar — the async standard pipeline does not follow redirects at the pipeline layer at all, so there is no second consumer to adapt for | -| ~~Redirect logging not implemented~~ — **superseded 2026-07-28 by the Phase 7b retrofit**; hop, rejection, and permitted-downgrade events now ship, redacted and contained | Spec: `SHOULD` emit structured records per hop/loop/downgrade event | Was: `Logger`/`LogEvent` seam is Phase 7 per the roadmap's Deferred Items Log. Now: only the loop-detected and malformed-Location events remain deferred, both blocked on a reason discriminant `decide()`'s `Decision` does not carry | +| Redirect logging not implemented in this phase — the Phase 7b retrofit (hop, rejection, permitted-downgrade events, redacted and contained) is specified above but **applied by 7b's Task 9**, not by 5b | Spec: `SHOULD` emit structured records per hop/loop/downgrade event | Was: `Logger`/`LogEvent` seam is Phase 7 per the roadmap's Deferred Items Log. Now: only the loop-detected and malformed-Location events remain deferred, both blocked on a reason discriminant `decide()`'s `Decision` does not carry | ## Deferred Items (add to the roadmap's Deferred Items Log) @@ -408,5 +417,6 @@ hints — that parser is 5a's `pacing.ts`, untouched here). |---|---|---|---| | `PIPE-40` — 2-hop-redirect conformance clause | Phase 4c, targeted here by the roadmap | **Resolved in Phase 5b** | Satisfied by the two-hop `FakeTransport` test above (wire-send count, per-hop close, final-response-open) | | `AUTH-29` / marker *consumption* (skip-stamping on a cross-origin re-issue, first-stripper role) | This brainstorm | **Phase 5c** | 5b only produces the marker and defends it with an independent guard step; nothing yet reads it for its intended purpose (suppressing credential stamping) — that is 5c's auth step | -| Redirect structured logging (`SHOULD`-level hop/loop/downgrade events) | This brainstorm | **Partially resolved 2026-07-28 (Phase 7b)** | Hop, rejection, and permitted-downgrade events ship in `redirect-step.ts`, URLs through `redactUrl()`, emissions through `emitQuietly()`. The loop-detected and malformed-Location events remain open — both need a reason discriminant on `decide()`'s `'return-current'` variant | +| Redirect structured logging (`SHOULD`-level hop/loop/downgrade events) | This brainstorm | **Phase 7b, Task 9** | Specified 2026-07-28 and unbuilt as of 5b's execution: hop, rejection, and permitted-downgrade events, URLs through `redactUrl()`, emissions through `emitQuietly()`. The loop-detected and malformed-Location events stay open even after that — both need a reason discriminant on `decide()`'s `'return-current'` variant | | Redirect predicate's scope over safety mechanics (see Deviation Ledger) | This brainstorm | Re-confirm at Phase 9 conformance sweep, or sooner if the user disagrees | A judgment call made without the user present; narrow and mechanical to reverse if wrong | +| `PIPE-40` vs `REDIR-22` on the non-replayable-body path | Review pass 3 | **Phase 10 reconciliation** | Two `MUST`s naming the same trigger and prescribing opposite dispositions. 5b implements `REDIR-22` (close, then throw) for the reasons in the Deviation Ledger; whichever way Phase 10 lands, one of the two spec sentences needs an erratum rather than a silent port-side choice | diff --git a/docs/superpowers/plans/2026-07-26-phase5b-redirect.md b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect.md similarity index 99% rename from docs/superpowers/plans/2026-07-26-phase5b-redirect.md rename to docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect.md index ccaab51..f9072b2 100644 --- a/docs/superpowers/plans/2026-07-26-phase5b-redirect.md +++ b/docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect.md @@ -6,7 +6,7 @@ comparison and the credential-suppression marker, the pure per-hop decision function, scheme-downgrade and loop/hop-cap guarding, and the pillar adapter plus its bundled marker-stripping safety net — satisfying `docs/product-spec/10-redirect-handling.md` (`REDIR-1`–`REDIR-*`), per -`docs/superpowers/specs/2026-07-26-phase5b-redirect-design.md`, and closing the roadmap's `PIPE-40` deferred item. +`docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-design.md`, and closing the roadmap's `PIPE-40` deferred item. > **Amended 2026-07-28 (Phase 7b retrofit):** Task 6's `redirect-step.ts` gains three `SHOULD`-level structured > log events via `getGlobalLogger()` — a hop event, a rejection event distinguishing `SchemeDowngradeError` @@ -25,7 +25,7 @@ loop/hop-cap guarding, and the pillar adapter plus its bundled marker-stripping > earlier "5b now depends on 7b first" wording was a cycle. **An agent executing this plan must skip the Phase > 7b retrofit blocks in Task 6** and build `redirect-step.ts` without any `observability/` import; Phase 7b's > plan Task 9 adds the three emission sites afterwards. See -> `docs/superpowers/plans/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. +> `docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. **Architecture:** A new `packages/core/src/redirect/` folder of seven files. `decide.ts` is a pure function — no I/O, no clock, no side effects beyond the `Request` value it returns — that resolves one hop's outcome from a @@ -1504,7 +1504,7 @@ Expected: FAIL — `Cannot find module './redirect-step.js'`. // Amended 2026-07-28 (Phase 7b retrofit): three getGlobalLogger() call sites below, every URL field through // redactUrl() and every emission through emitQuietly(). Narrow blast radius -- only this file's own emission // points; no other phase depends on them. See -// docs/superpowers/specs/2026-07-28-phase7b-observability-design.md's "Amendments to 5a and 5b" section. +// docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md's "Amendments to 5a and 5b" section. import {getGlobalLogger, type Logger} from '../observability/logger.js'; import {redactUrl} from '../observability/redaction.js'; import {invariant} from '../invariant.js'; @@ -1806,7 +1806,7 @@ git commit -m "feat(core): independent POST_AUTH marker guard + withRedirect() b **Files:** - Verify unchanged: `packages/core/etc/core.api.md` - Verify unchanged: `packages/core/src/index.ts` -- Create: `docs/superpowers/plans/2026-07-26-phase5b-redirect-checklist.md` +- Create: `docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md` **Interfaces:** - Consumes: every symbol from Tasks 1–7. @@ -1834,7 +1834,7 @@ Expected: every gate PASS. - [ ] **Step 4: Write the requirement checklist** -Create `docs/superpowers/plans/2026-07-26-phase5b-redirect-checklist.md`, same format as +Create `docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md`, same format as `2026-07-26-phase5a-retry-checklist.md` — `| ID | Level | Requirement gist | Status | Where |` tables, legend ✅ shipped / 🚫 never built / ⏳ deferred / N/A. @@ -1864,7 +1864,7 @@ State explicitly at the top whether the plan has been executed, matching the Pha - [ ] **Step 5: Commit** ```bash -git add docs/superpowers/plans/2026-07-26-phase5b-redirect-checklist.md +git add docs/work/mvp/phase5/phase5b/2026-07-26-phase5b-redirect-checklist.md git commit -m "docs: Phase 5b requirement checklist" ``` diff --git a/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md new file mode 100644 index 0000000..d87dea5 --- /dev/null +++ b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md @@ -0,0 +1,230 @@ +# Phase 5c — Auth Implementation Plan — Checklist + +Verification of [2026-07-26-phase5c-auth.md](./2026-07-26-phase5c-auth.md) against every requirement ID in +`docs/product-spec/11-authentication.md` (`AUTH-1`–`AUTH-38`) plus `PIPE-2`, `PIPE-24`, `PIPE-35`, and +`PIPE-39`, as dispositioned by +`docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md`. + +**Status: EXECUTED (2026-08-27).** Every task below is implemented, tested, and green across the full gate +sequence (`typecheck`, `lint`, `build`, `bun test` with coverage, `api:ci`, `lint:publish`, +`verify:dual-consumption`, `verify:consumer-types`, `verify:seam-1`, `verify:runtime-floor`, `test:node`, +`audit`). + +**This is the one phase whose barrel and API report are EXPECTED to change.** Every prior phase asserted +`packages/core/src/index.ts` and `packages/core/etc/core.api.md` byte-identical to its starting point; 5c is +the first point a caller can assemble a working pipeline, so the pillar-authoring surface is promoted here. +See "Public-barrel promotion" below. + +**Phase 7b retrofit deliberately skipped.** The plan carries an amendment installing `loggingStep()` into the +preset's `LOGGING` slot, and its own 2026-07-29 correction says an agent executing this plan must skip those +blocks: 5c runs before 7b, so `observability/logging-step.js` does not resolve at this plan's execution time. +`standardResilience()` installs the three pillars that exist; Phase 7b's plan Task 9 installs the fourth. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +## Files shipped + +| File | Requirements | Task | +|---|---|---| +| `packages/core/src/auth/errors.ts` | `AUTH-6`, `AUTH-28`, `AUTH-35` | 1 | +| `packages/core/src/auth/scheme.ts` | `AUTH-1` | 2 | +| `packages/core/src/auth/requirement.ts` | `AUTH-2` | 3 | +| `packages/core/src/auth/descriptor.ts` | `AUTH-3` | 4 | +| `packages/core/src/auth/resolve.ts` | `AUTH-4`–`AUTH-7` | 5 | +| `packages/core/src/auth/credential.ts` | `AUTH-8`–`AUTH-11` | 6 | +| `packages/core/src/auth/challenge.ts` | `AUTH-12`, `AUTH-13` | 7 | +| `packages/core/src/auth/md5.ts` | `AUTH-15`, `AUTH-17` | 8 | +| `packages/core/src/auth/basic.ts` | `AUTH-14` | 9 | +| `packages/core/src/auth/digest.ts` | `AUTH-15`–`AUTH-22` | 10 | +| `packages/core/src/auth/static-key.ts` | `AUTH-26` | 11 | +| `packages/core/src/auth/composing-handler.ts` | `AUTH-23`–`AUTH-25` | 12 | +| `packages/core/src/auth/bearer-cache.ts` | `AUTH-34`–`AUTH-37` | 13 | +| `packages/core/src/auth/auth-step.ts` | `AUTH-27`–`AUTH-33`, `AUTH-36`, `AUTH-38` | 14 | +| `packages/core/src/auth/preset.ts` | `PIPE-24`, `PIPE-39` | 16 | +| `packages/core/src/http/request-options.ts` (amended) | `AUTH-4`'s `perCall` tier | 14 | +| `packages/core/src/pipeline/builder.ts` (amended) | `PIPE-35` | 15 | +| `packages/core/src/pipeline/runtime.ts` (amended) | `PIPE-35` (`get transport()`) | 15 | +| `packages/core/src/index.ts` (amended) | public-barrel promotion | 16 | +| `test/node-conformance/auth.test.mjs` | `AUTH-14`, `AUTH-15`, `AUTH-17`, `AUTH-20`, `AUTH-21`, `AUTH-30`–`AUTH-33` on Node | 17 | + +Every production file has a colocated `*.test.ts`. + +`test/node-conformance/auth.test.mjs` was not in the plan's file list. It is required by +`test/node-conformance/README.md`'s membership rule: 5c reaches three runtime-provided globals Bun implements +independently of Node — `crypto.subtle.digest`, `crypto.getRandomValues`, and `btoa` — and every one of them +fails silently rather than loudly if the runtimes disagree. A wrong SHA-256 digest is still a well-formed hex +string; a Latin-1/UTF-8 mismatch in Basic stamping is still valid-looking base64. + +## 11.1 The descriptor/resolver model + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-1 | MUST | The scheme set is exactly `{OAUTH2, API_KEY, BASIC, DIGEST, NO_AUTH}`, `NO_AUTH` a sentinel rather than a wire scheme | ✅ | Task 2 — a string-literal union, not a TS `enum` (`erasableSyntaxOnly`). The five members are enforced by the TYPE, exhaustively at every branch (`preemptiveStamp` and `defaultChallengeHook` both close on `assertNever`), which is what makes adding a sixth a compile error rather than a silent fall-through. A companion `AUTH_SCHEMES` array shipped briefly and was cut: nothing enumerated it, and its only test asserted the array's five members against the union's five — the constant restated, not a behaviour | +| AUTH-2 | MUST | A requirement binds one scheme to its own scopes and params; immutable against post-construction mutation of the inputs; value equality over all three | ✅ | Task 3 — `createAuthRequirement` spreads `scopes` and copies `params` into a new `Map`, then freezes; `authRequirementsEqual` compares scheme, ordered scopes, and params. Scope ORDER is part of the value, asserted directly | +| AUTH-3 | MUST | A descriptor is a non-empty ordered preference list, rejects an empty list at construction, is immutable, and reports `allowsAnonymous` iff some requirement is `NO_AUTH` | ✅ | Task 4 — the empty-list rejection is `invariant()`, **not** a typed leaf: a caller assembling zero requirements has a bug, not an operational failure (`docs/knowledge/error-handling.md`'s programmer/operational split). This corrects the design doc's "`ArgumentError` reused from earlier phases" — no such class exists in any prior phase | +| AUTH-4 | MUST | Tier selection is per-call, then operation, then client; the first PRESENT tier is resolved against and a present-but-unsatisfiable tier never falls through | ✅ | Task 5 (`perCall ?? operation ?? client`), asserted with a satisfiable lower tier present under an unsatisfiable higher one. Task 14 gives `perCall` a genuinely per-call source via `RequestOptions.auth` and `StepContext.options` | +| AUTH-5 | MUST | Within the selected descriptor, the first requirement whose scheme is `NO_AUTH` or in the supplied available set wins, without inspecting any concrete credential | ✅ | Task 5 — `availableSchemes` is a `ReadonlySet<AuthScheme>`, and Task 14's `availableSchemesOf()` derives it from which credentials are configured, so no credential value can reach the resolver | +| AUTH-6 | MUST | All tiers absent is an argument error; an unsatisfiable selected descriptor fails with a distinct error carrying the required schemes in preference order and the available schemes | ✅ | Task 1 (`AuthResolutionError.unsatisfiable`, both lists as `readonly` FIELDS, copied), Task 5. All-tiers-absent is `invariant()`, per AUTH-3's note above; the test asserts it is NOT an `AuthResolutionError` | +| AUTH-7 | MUST | The resolver is stateless, concurrency-safe, and a deterministic pure function | ✅ | Task 5 — a module-level function with no captured state; asserted by identity (the same inputs return the very object the descriptor already holds, so nothing is allocated per call) | + +## 11.2 Credentials + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-8 | MUST | Every credential redacts its secret in any string/diagnostic form without corrupting the real fields; bearer tokens have VALUE equality, key credentials REFERENCE identity | ✅ | Task 6 — **all three** credential types are classes holding their secret in a `#` field, each with `toString` AND `Symbol.for('nodejs.util.inspect.custom')`, because `console.log` does not route object arguments through `toString`. `#`, not TS `private`: redaction is a RUNTIME-privacy requirement, and `private` is erased, leaving the secret reachable through `Object.keys`/`JSON.stringify`/default inspect — all three asserted for all three types. `BearerToken` was a bare `{token, expiresAt}` object at first, which redacted NOTHING and failed this requirement outright; it keeps AUTH-8's VALUE equality through `bearerTokensEqual`, a pure function, exactly as the data object did. `ApiKeyCredential`/`NameKeyCredential` deliberately have NO `equals` override, so `===` already gives reference identity. None of the three exposes a public secret accessor: the static keys are read only through the internal `credentialKey()` friend hook, so no secret appears on the published `.d.ts` | +| AUTH-9 | MUST | Secret and identity fields validated non-blank at construction | ✅ | Task 6 — `invariant()` on all four (bearer token, API key, name-key name and key), asserted for `''` and whitespace-only. All three types are NOMINAL with private constructors, so the validation cannot be routed around: a `TokenProvider` returning an object literal no longer type-checks, which was reachable while `BearerToken` was a structural interface | +| AUTH-10 | MUST | Bearer expiry optional (absent = never locally expires), evaluated additively with a grace margin: expired iff `expiresAt` is set and `now + margin > expiresAt` | ✅ | Task 6 (`isBearerTokenExpired`), with the boundary asserted at `now === expiresAt` (NOT expired) and `now === expiresAt + 1` | +| AUTH-11 | MUST | Provider fetch errors propagate, are never cached, and reach an async caller through the async channel, never a synchronous throw | ✅ | Tasks 6 + 13 — the cache does not catch around the provider call, so this falls out of the structure rather than needing a branch; asserted by a rejecting provider followed by a clean refetch. `TokenProvider` takes no parameters at all, because a coalesced fetch belongs to no single call and nothing could correctly populate one — see the Deviation Ledger. A provider that fails SYNCHRONOUSLY (throwing before returning a promise, or returning a non-thenable) is normalized onto the async channel by `invokeProvider`, so `AUTH-38`'s uniform error model holds for it too; left bare it escaped past the background refresh's own `.catch` before that catch was attached | + +## 11.3 Challenge parsing + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-12 | MUST | Parse RFC 7235 challenge headers: multiple comma-separated challenges, quoted values containing commas and `=`, backslash escapes, lower-cased scheme and param names, verbatim unquoted values, a bare scheme with an empty map, a token68 under a synthetic key | ✅ | Task 7 — hand-written with a quote-depth scanner, never `.split(',')`. The synthetic key is spelled `'token68'`, the requirement's own wording. A token68's trailing `=` padding (`Negotiate YWJj==`) is disambiguated from an auth-param by requiring a real token or quoted-string value after the `=` — only at the scheme tail, the one position RFC 7235 permits a positional token68 | +| AUTH-13 | MUST | Total: never throws; blank input yields `[]`; a malformed challenge recovers at the next top-level comma; an unterminated quoted string ends at EOF; params before a malformed tail are preserved | ✅ | Task 7 — a `fast-check` property asserts totality over arbitrary strings; two more assert quoted-comma non-splitting and single-challenge round-tripping. A comma inside a MALFORMED segment's quoted value is also not a recovery point, asserted separately | + +## 11.4 Stamping handlers + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-14 | MUST | `Basic ` + base64(UTF-8(`username:password`)), computed once; `basic` accepted case-insensitively; `Authorization`/`Proxy-Authorization` chosen by the caller from which challenge header the status carried; credentials non-empty but whitespace PERMITTED per RFC 7617 | ✅ | Task 9 — the value is computed at construction and closed over; the laxer non-empty rule is deliberately NOT the credential types' `.trim()` check, asserted both ways. Case-insensitivity is implemented in `parseChallenges`, which lower-cases the scheme before any handler sees it | +| AUTH-15 | MUST | Digest supports exactly `{MD5, MD5-sess, SHA-256, SHA-256-sess}`, `qop=auth` or absent; declines `auth-int`-only, unsupported algorithms, and mutual-auth verification | ✅ | Task 10 — `SUPPORTED_ALGORITHMS` is the closed set; `auth-int`-only and `MD4` both asserted declined. Mutual auth (`Authentication-Info`) is never emitted or verified, which is the requirement's own disposition | +| AUTH-16 | MUST | Satisfiable iff scheme is `digest`, `realm`+`nonce` present, `qop` absent or containing `auth`, algorithm supported or absent (defaulting MD5), preferring the algorithm earliest in the CONFIGURED list regardless of wire order | ✅ | Task 10 (`parseDigestChallenge` + `rank`), Task 12 (`composingHandler` sorts by handler order then `rank`). `rank` is a plan-time addition to `ChallengeHandler`: `canHandle` alone answers yes/no per challenge and cannot express a preference among several a handler could equally satisfy — which is exactly what RFC 7616's repeated-challenge algorithm discovery produces | +| AUTH-17 | MUST | HA1/HA2/response per RFC 7616/2069, lower-case hex of the selected algorithm | ✅ | Tasks 8 + 10 — `computeDigestResponse` is exported and unit-tested against five independently-computed vectors (MD5 qop, MD5 no-qop, MD5-sess, SHA-256, SHA-256-sess) because `stamp()` draws a fresh random cnonce and can never be pinned end-to-end. MD5 is hand-rolled (Web Crypto excludes it) and checked against RFC 1321's own vectors plus the 55/56/64-byte padding boundaries | +| AUTH-18 | MUST | `nc` tracked per server nonce, starting at `00000001`, incrementing only on reuse, rendered as exactly 8 lower-case hex digits, low 32 bits on overflow | ✅ | Task 10 (`NonceCountStore`) — a `fast-check` property asserts strict monotonicity for a fixed nonce; a distinct nonce asserted to start fresh; a no-`qop` stamp asserted NOT to consume a count | +| AUTH-19 | SHOULD | The per-nonce store is bounded (default 1024) and drained under the cap; evicting a live nonce is harmless | ✅ | Task 10 — an insert-then-DRAIN-IN-A-LOOP, not a pre-insert single evict, per `docs/knowledge/concurrency-and-async.md`'s XCUT-14 rule for a server-keyed map. The distinguishing test bursts 4096 fresh nonces and asserts the map is at exactly the cap after every admit — a single-victim-per-insert store passes the "an evicted nonce restarts at 1" probe but fails this one | +| AUTH-20 | MUST | The client nonce comes from a cryptographically strong source with ≥128 bits of entropy | ✅ | Task 10 — `globalThis.crypto.getRandomValues()` over 16 bytes, never `Math.random()`; asserted 32 hex characters and distinct across calls, on Bun and again on Node | +| AUTH-21 | MUST | UTF-8 hash input when the challenge advertises `charset=UTF-8`, ISO-8859-1 otherwise | ✅ | Task 10 — asserted by a non-ASCII password hashing differently under the two, and identically for an all-ASCII input, on both runtimes | +| AUTH-22 | MUST | Quote/escape the appropriate fields, leave `qop`/`nc`/`algorithm` unquoted with the full algorithm spelling, use the request-target as the digest-uri, emit `cnonce`/`nc`/`qop` only when `qop` is negotiated | ✅ | Task 10 — `opaque` is echoed back quoted when the challenge carried one and omitted entirely otherwise (RFC 7616 requires the client return it unchanged; a server binding state to it rejects a request without it). A quote inside a realm asserted escaped. The digest-uri is `pathname + search`, asserted through the pillar step against `/a?q=1` | +| AUTH-23 | MUST | Composed handlers delegate to the first handler in DECLARATION order whose can-handle passes; the handler list is defensively copied | ✅ | Task 12 — handler order is the primary sort key and beats wire-order challenge position, asserted with `basic` first on the wire and `digest` first in configuration. A handler pushed onto the caller's array after construction is asserted invisible | +| AUTH-24 | MUST | Handlers are safe for concurrent invocation; a per-handler mutable counter such as Digest's `nc` yields correct, non-duplicated counts under concurrent reuse of one nonce | ✅ | Task 10 — `next()` is one synchronous read-increment-write with no `await` between the read and the write. Node and Bun have no preemptive interleaving mid-statement, so "thread-safe primitives" collapses to that, the same collapse 5a documented for `BODY-3` | +| AUTH-25 | MUST | `Authorization` for `WWW-Authenticate`, `Proxy-Authorization` for `Proxy-Authenticate`, selected by an explicit proxy flag; no header at all when nothing is satisfiable | ✅ | Task 12 (handlers return the VALUE half only), Task 14 (`pickChallengeHeader` reads only the header matching the STATUS, so a 401 carrying a stray `Proxy-Authenticate` is not answered — asserted). A 407 answered into `Proxy-Authorization` with `Authorization` absent is asserted end-to-end. The "explicit proxy flag" is `ChallengeSelection.isProxy` inside `auth-step.ts`, consumed by `answerHeaderName`; it is NOT threaded into `ChallengeHandler.stamp`, which was tried and removed — see the Deviation Ledger. `AUTH-28`'s replay guard covers `Proxy-Authorization` as well as `Authorization`, asserted separately | +| AUTH-26 | MUST | A static key is written into the configured header (default `Authorization`), prefixed by the configured prefix and exactly one space; stateless after construction | ✅ | Task 11 — uniform over both credential shapes; `NameKeyCredential.name` is deliberately NOT consulted as a header name (it is the non-secret half of the redacted `toString`; a caller wanting it passes `headerName` explicitly). An explicitly-empty prefix still contributes its space, so the option's absent state stays reachable | + +## 11.5 The AUTH pillar step + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-27 | MUST | Exactly one auth step at the single AUTH pillar stage, nested inside both the redirect and retry loops, so auth executes per redirect hop and per retry attempt | ✅ | Task 14 — `stage: 'AUTH'` is baked into the descriptor, so PIPE-36's "not relocatable out of its pillar" holds structurally; `PILLAR_STAGES` already caps the slot at one. Task 16's preset installs redirect-then-retry-then-auth, asserted by flattened stage order | +| AUTH-28 | MUST | On ANY path where a credential will be attached, reject a non-HTTPS URL (case-insensitive) BEFORE any token fetch or header stamping, with an error naming the concrete step and the offending scheme | ✅ | Task 14, **both paths**. Outbound: skipped for `NO_AUTH` (matching the requirement's own qualifier), asserted to fire before the provider is called. Replay: the outbound guard is skipped entirely for `NO_AUTH` and nothing constrains a caller hook to preserve the URL, so a replacement carrying a credential header is guarded again — and the challenge response is closed before the throw, so the body is not leaked | +| AUTH-29 | MUST | A cross-origin re-issue marked by the redirect step is not stamped, has the internal marker stripped so it never reaches the wire, and skips the HTTPS guard; a same-origin re-issue is re-stamped normally. The mechanism can only SUPPRESS, never force | ✅ | Task 14 — the marker is read first and cleared unconditionally before either branch, so it cannot survive into a request built by the stamping logic. **Both halves**: the outbound suppression AND the challenge-reaction suppression — a marked hop returns its 401 untouched and unclosed, because answering it would stamp exactly the credential the outbound pass declined to send, onto a server-chosen foreign host, over a URL whose HTTPS guard was skipped. Suppress-only holds structurally: nothing reads the marker to cause a stamp. Joint conformance in Task 16 | +| AUTH-30 | MUST | A 401 with `WWW-Authenticate` consults the challenge hook; a non-null replacement closes the original and drives once through a fresh chain copy, with no further challenge handling; the default hook yields no replacement | ✅ | Task 14 — reconciled as ONE step with one pluggable hook and a scheme-dependent default body, not three mechanisms. Every dispatch goes through a fresh `ctx.fork()`. A second 401 on the replay is returned as-is (asserted: exactly two wire sends, not a loop). `API_KEY`/`NO_AUTH` never react, which is the requirement's literal "the default hook yields no replacement" | +| AUTH-31 | MUST | The replay is gated on body replayability: a non-replayable replacement skips the replay, surfaces the original unchanged, and MUST NOT close it | ✅ | Task 14, applied **uniformly** — the reference gates only its sync step and recommends (SHOULD) a port extend it; one unified step leaves exactly one place to apply it, closing that SHOULD. The gate covers the DISPATCH only, for both hook shapes: the hook always runs, and its replacement is then gated on `body.replayable`. There is deliberately no "skip the hook for a one-shot body" fast path — one shipped briefly and skipped `AUTH-36`'s eviction with it, which is recorded against `AUTH-36` below. Asserted for the default hook and for a caller hook returning a non-replayable replacement, including the response left uncancelled | +| AUTH-32 | MUST | A hook that throws, rejects, or throws synchronously closes the open 401 before propagating | ✅ | Task 14 (`runHook`) — asserted for a rejecting hook AND a synchronously-throwing one, both on Bun and on Node. The close goes through 4b's `releaseQuietly`/`withReleaseFailure`, not a bare `await response.close()`: `Response.close()` rethrows whatever cancelling the body raised, so the bare form discarded the hook's own error and surfaced the teardown failure in its place — the inversion `RECOV-12` forbids, and the one 5b's `decideOrClose` already guards against. `guardReplayScheme` was fixed the same way, where the masked error was `PlaintextCredentialError`. Asserted against a body whose `cancel()` rejects | +| AUTH-33 | MUST | A 401 without `WWW-Authenticate` is returned unchanged without consulting the hook | ✅ | Task 14 — asserted for a bare 401, for a 401 carrying only `Proxy-Authenticate`, and for a hook returning `undefined`; all three leave the response uncancelled | +| AUTH-38 | SHOULD | The HTTPS-guard failure and any hook error are delivered through the async channel, not a synchronous throw | ✅ | Task 14, satisfied structurally: the step's `fn` is `async`, so both become a rejected promise with no separate code path. Asserted by a hook that throws synchronously still surfacing as a rejection | + +## 11.6 The bearer token cache + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| AUTH-34 | MUST | Stamp `Authorization: Bearer <token>` from a token cached until a configurable refresh margin (default 30 s) before expiry; concurrent requests racing on a missing/expiring token yield at most one provider fetch, with a non-blocking hot-path read of a valid cached token | ✅ | Task 13 — one unified policy, not two stacks; the hot-path read is the fresh-zone branch of `AUTH-37`'s state machine. See the Deviation Ledger. The 30 s default is `AuthStepSettings.bearerMarginMs`; single-flight coalescing asserted at exactly one provider invocation for two concurrent callers — and, separately, for a burst of concurrent POST-EVICTION refreshes, which an earlier `refreshNow()` shape turned into one provider call per 401 (a mass revocation would have stampeded the identity provider); the method is now named `refreshPostEviction()`, because it may JOIN a sibling 401's fetch and what it actually guarantees is that no pre-eviction fetch is ever joined. The 30 s default is pinned from BOTH sides — a token expiring just inside it refreshes, one just outside it does not — since a single one-sided assertion cannot tell 30 s from 60 s, and `BearerCredential.marginMs`'s override and an explicit `0` are each asserted for EFFECT, not only for validation. The margin is validated as a finite, non-negative duration at BOTH doors (`bearerMarginMs` and `BearerCredential.marginMs`) and `createBearerToken` rejects a non-finite `expiresAt`: `nowMs + marginMs > expiresAt` is false for `NaN`, so an unvalidated margin made the cache read a long-dead token as fresh and serve it from the hot path forever, never calling the provider again | +| AUTH-35 | MUST | Reject a null token and a token already expired at fetch time (no margin); never cache a thrown provider error | ✅ | Task 13 — the null guard is a RUNTIME check at a deliberately widened boundary, because a plain-JS caller can return null regardless of `TokenProvider`'s non-nullable type. A rejection propagates through `finally` untouched, so nothing is cached; asserted by a clean refetch after each failure mode | +| AUTH-36 | MUST | On a 401 advertising a Bearer challenge, evict ONLY the exact cached token that produced it (matched on the stamped header value), re-stamp a single retry with a freshly fetched token, preserve a token another request already refreshed, surface the 401 unchanged when the rejected request carried no `Authorization` or the response advertises no Bearer challenge, and fire regardless of HTTP method | ✅ | Tasks 13 + 14 — `evict()` compares `` `Bearer ${cached.token}` `` to the rejected header value and RETURNS the survivor on a mismatch, which the hook then stamps. That return is what makes the preservation clause observable: the first shape preserved the token and then unconditionally fetched a replacement, overwriting it on the next tick and reducing the clause to a no-op. Asserted end-to-end with a gated two-drive interleaving — the second 401 stamps the preserved token and the provider is called twice, not three times. No method check exists anywhere on this path; AUTH-31's replayability gate is what protects a non-replayable body — and that gate now covers the DISPATCH only. An earlier shape short-circuited the whole hook for a one-shot body, which skipped the eviction too and left the token the server had just rejected in the cache; with `AUTH-10`'s never-expiring token that never aged out either, so a stream-only client re-sent the dead credential indefinitely. Asserted by two successive one-shot POSTs, where the second carries a freshly fetched token | +| AUTH-37 | MUST | A three-zone expiry policy without blocking: fresh stamps with no refresh; expiring-but-valid stamps immediately and kicks off a background refresh; expired/missing awaits a single-flight fetch; concurrent expiring/missing callers coalesce; a failed fetch is not cached; a failed BACKGROUND refresh is non-fatal | ✅ | Task 13 — all three zones asserted separately. The background rejection is swallowed EXPLICITLY and UNCONDITIONALLY: a bare `void` would leave an unhandled rejection that terminates the process under Node's default policy, and the narrowed catch that briefly rethrew `InvariantViolation` did exactly that for a fault in caller-supplied provider code — see the Deviation Ledger. A synchronously-failing provider is normalized onto the async channel first, so the non-fatal guarantee is not conditional on HOW the provider failed. The post-eviction path is `refreshPostEviction()`. It does NOT bypass coalescing — an earlier shape did, and turned a mass revocation into one provider call per 401 — it supersedes only fetches predating this eviction burst, joining a sibling 401's fetch at the same generation (see the `AUTH-34` row). A generation counter also stops a superseded fetch re-caching its token if it settles LAST | + +## Cross-phase requirements closed here + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PIPE-2 | MUST | The pillar precedence chain, and specifically that auth executes per redirect hop and per retry attempt | ✅ | Task 16's joint conformance test — a `standardResilience()` runtime over a scripted `302 (cross-origin), 302 (back to seed origin), 200`: the credential is present on hop 1, ABSENT on the cross-origin hop, and RE-STAMPED on the return to the seed origin. 5a's own suite already covers the per-attempt dimension | +| PIPE-24 | MUST | The standard-resilience preset installs into empty pillar slots only, rejecting the whole call if any is occupied | ✅ *(satisfied VACUOUSLY — see the Deviation Ledger)* | Task 16 — true BY CONSTRUCTION: `standardResilience()` takes a `Transport` and always starts from a fresh `PipelineBuilder`, so no slot can be occupied. **The requirement's validate-and-reject half has no implementation, because no input can reach it** — that is recorded as a deviation rather than left implied, so Phase 9 does not look for a check that was never written. A caller layering the preset onto a customized builder reaches for `seedFrom` instead | +| PIPE-35 | SHOULD | Two unambiguous ways to seed from an existing pipeline — FLATTEN (copy steps and transport, same loops) vs NEST (opaque transport, separate loops) — with the choice explicit, never accidental (MUST) | ✅ | Task 15 — `mode` has no default value, so a caller cannot seed by accident. `flatten` re-buckets by each descriptor's OWN stage (asserted against seeded array position) and pillar collisions apply exactly as any append sequence. `nest` sets the runtime as the transport, asserted by the outer step running before the inner one and by both layers occupying the same pillar independently. `Runtime.transport` was added to make flatten implementable at all | +| PIPE-39 | SHOULD | Convenience constructors including a standard pipeline installing the default resilience pillars | ✅ | Task 16 — `standardResilience()` installs redirect (through 5b's `withRedirect`, which seats the `POST_AUTH` marker guard alongside), retry, and auth. Four descriptors total, asserted | + +## Public-barrel promotion + +`packages/core/src/index.ts` gains two groups, and `packages/core/etc/core.api.md` is regenerated to match. + +**Both pass conditions failed on the first attempt and are now enforced mechanically rather than asserted.** +This section previously claimed zero `ae-forgotten-export` warnings and a compile-only consumer smoke check +covering the promoted surface. Neither was true: the committed report carried a live +`ae-forgotten-export` for `ExecutionContext`, and `scripts/verify-consumer-types.mjs` still exercised only +Phase 3's body/response surface. The cause was a single word — an `@internal` token inside the prose comment +above the barrel's context-family export, which `stripInternal` (inherited from `gts/tsconfig-google.json`) +takes as an instruction to delete the whole export from the emitted `.d.ts`. `typecheck`, `build`, and +`api:ci` all passed over it, because api-extractor recorded the warning as report TEXT rather than failing. + +Both are now gates, not claims: + +- `packages/core/api-extractor.json` sets `ae-forgotten-export` to `logLevel: "error"` with + `addToApiReportFile: false`, so a forgotten export FAILS `api:ci` instead of being written into the report. +- `scripts/verify-consumer-types.mjs` compiles a consumer that names every promoted symbol — the context + family included — builds an `AuthStepSettings`, assembles both a hand-built `PipelineBuilder` pipeline and + a `standardResilience()` client, and annotates a custom `Step`, all importing ONLY from the package entry + point on the declared `lib` with `types: []`. + +- **Group 1, the authoring surface:** `Stage`, `STAGE_ORDER`, `PILLAR_STAGES`, `Step`, `StepContext`, `Next`, + `StepDescriptor`, `PipelineBuilder`, `Runtime`, `retryStep`, `redirectStep`, `authStep`, + `standardResilience`. +- **Group 2, everything those signatures name:** `RetryStepOptions`, `RetrySettings`, `BackoffSettings`, + `Clock`, `RedirectSettings`, `RedirectPredicate`, `RedirectCondition`, `StandardResilienceOptions`, + `AuthStepSettings`, `AuthCredentialSet`, `BasicCredential`, `DigestCredential`, `BearerCredential`, + `ApiKeyCredentialConfig`, `ChallengeHook`, `AuthTiers`, `AuthScheme`, `DigestAlgorithm`, `AuthDescriptor`, + `AuthRequirement`, `TokenProvider`, the factories + `createAuthDescriptor`/`createAuthRequirement`/`createBearerToken` and the equality helpers beside them, the + `ApiKeyCredential`/`NameKeyCredential`/`BearerToken` classes (all three NOMINAL — they carry a `#` field, so + no object literal substitutes, `API_KEY` would otherwise be unreachable, and a `TokenProvider` cannot return + a hand-built token that skips `AUTH-9`'s validation), and the two error leaves + `AuthResolutionError`/`PlaintextCredentialError`. +- **Deliberately NOT promoted, after review:** `Challenge`, `ChallengeHandler`, and `DigestUriContext`. They + were only reachable because `AuthStepSettings` carried a public `handlers` field — a field that promised + composability the package does not offer, since `basicHandler`/`digestHandler` stay internal, so supplying + one handler silently LOST the credential-derived ones. `handlers` was removed and the three types went back + to internal. `challengeHook` covers the custom-scheme case with a shape a caller can actually satisfy. + `DigestChallengeUnsupportedError` was cut for the same reason (`docs/work/mvp/2026-09-04-open-items-dissolution.md` G11). +- **Not in the plan's list, promoted as a forced consequence:** `Step`, and the whole context family + (`ExecutionContext`, `DispatchContext`, `RequestContext`, `ExchangeContext`, `InstrumentationBundle`). + `StepDescriptor.fn` names `Step`, and `StepContext.context` names `ExecutionContext`, which is a union + alias — api-extractor refuses to analyze it while its members are unexported. Promoting `StepContext` + without them would leave a caller unable to type a custom step's `ctx`. Narrowing `StepContext.context` + instead was considered and rejected — `CTX-1` exists so a step can read the exchange's request and response. + Recorded as an ACCEPTED RISK in `docs/work/mvp/2026-09-04-open-items-dissolution.md` G10, with `InstrumentationBundle`'s two provisional + `unknown` members documented as provisional in the emitted `.d.ts` itself. +- **Still internal:** everything else under `src/auth/` — `parseChallenges`, `md5.ts`, `basicHandler`, + `digestHandler`, `NonceCountStore`, `computeDigestResponse`, `composingHandler`, `BearerTokenCache`, + `stampStaticKey`, `availableSchemesOf`, `AUTH_STEP_TYPE` — plus 5b's `withRedirect`, + `stripCrossOriginMarkerStep`, and the cross-origin marker functions. A caller BUILDS an `AuthStepSettings` + from the Group 2 factories and hands it to `authStep()`/`standardResilience()`; it never constructs handler + internals. + +`RequestOptions` gains exactly one member, `auth?: AuthDescriptor`, and a matching builder method. + +## Deviation Ledger (for Phase 10) + +| Deviation | Reference behavior | Justification | +|---|---|---| +| One bearer strategy (async three-zone), not two | The reference ships a sync single-flight strategy and a separate async three-zone strategy | This port has one `Promise`-only execution model (4c), so `AUTH-34`'s hot-path read is a branch of `AUTH-37`'s state machine, not a second stack. Same reasoning and shape as 5a's `RETRY-28` collapse | +| `AUTH-31`'s replayability gate applied uniformly | The reference applies it on the sync auth step only, and SHOULDs a port extend it | One unified step leaves exactly one place to apply it; closes the spec's own SHOULD | +| Basic and Digest never stamp preemptively | Not stated either way in §11; inferred from `AUTH-14`/`AUTH-23`–`AUTH-25`'s exclusively challenge-driven phrasing | Digest structurally cannot stamp before seeing `realm`/`nonce`, and no separate "preemptive Basic" ID exists to contradict treating both uniformly. Flagged as an interpretation, not a certainty — Phase 9's conformance sweep should re-check it against any reference fixtures it turns up | +| `AUTH-3`/`AUTH-6` construction failures use `invariant()`, not a typed leaf | The design doc assumed an `ArgumentError` "reused from earlier phases" | No such class exists in any prior phase. Both cases are PROGRAMMER errors under `docs/knowledge/error-handling.md`'s split, which requires `invariant`/`assertNever`, not a handled error. A plan-time fix, not a deviation from working code | +| `TokenProvider` takes NO parameters; cancellation is caller-side, never provider-side | The reference's provider also takes no cancellation, so this ends up matching it exactly | Cancellation of a token fetch is caller-side by construction. `AUTH-34` makes the fetch SHARED by every caller coalesced onto it, so it is owned by no single call: handing it one caller's signal (a plan-time addition, shipped briefly as an optional `{signal}` bag) let a stranger's abort reject callers who never aborted — including one who supplied no signal at all — and let a request that merely finished tear down a refresh other requests were joined to. `bearer-cache.ts` races each caller's own WAIT against that caller's own signal instead, cancelling the wait without cancelling the work. Since nothing can ever populate a signal parameter, the parameter was cut rather than left documented-as-never-filled: a slot a caller writes code against and then finds inert is worse than no slot. `docs/knowledge/concurrency-and-async.md`'s "pass the caller's signal down to the I/O primitive" rule is deliberately not applied, because its premise — that the call owns the I/O — is false for a coalesced fetch; its "every external I/O call must carry a deadline" rule is discharged by `TokenProvider`'s TSDoc making an `AbortSignal.timeout` the provider's own obligation. **The type is back to the design doc's original shape**, `() => Promise<BearerToken>` | +| `ChallengeHook` takes an options bag carrying the call signal | The reference's hook takes only the response and request | The hook is the sanctioned place for a custom OAuth2 refresh-token grant — network I/O on the request path — and unlike the token fetch it is NOT shared between callers, so the same rule that forbids handing a coalesced fetch one caller's signal positively requires handing the hook exactly that. Without it a hung hook pinned the auth step, every retry attempt nested under it, and the whole request. The parameter is optional and third, so an existing two-argument hook still type-checks. `authStep` checks the signal at two further points: BEFORE building or running the hook, so a call already abandoned when the challenge arrives never spends the default hook's IdP round trip (matching `redirectStep`'s pre-hop check), and again before the replay dispatch, for an abort that arrived while the hook was in flight. Both reads go through an `isAborted()` helper rather than an inline test — `AbortSignal.aborted` is a live getter, but TypeScript narrows it like an ordinary property and carries that narrowing across the `await`, so the second check does not compile when written inline. That is the compiler being confidently wrong about mutable external state, and `concurrency-and-async.md`'s re-validate-after-await rule is the one that governs | +| A Digest challenge whose echoed fields are not header-safe is DECLINED, not answered | `AUTH-22` says to quote and echo `realm`/`nonce`/`opaque`; it does not say what to do when they cannot be written | `HTTP-19` lets a received field-value carry obs-text, so `Digest realm="café"` — a real RFC 7616 shape, and the reason the spec has a `charset` parameter at all — arrives intact; `HTTP-18`'s outbound grammar will not let it back out, and relaxing that is off the table because it is the request-splitting defence. Building the header anyway threw `HeaderValidationError` out of the whole auth step, converting a challenge the caller could have inspected into an exception. `parseDigestChallenge` now declines, so `canHandle` is false and `AUTH-33` surfaces the 401 unchanged. **The consequence: `AUTH-21`'s UTF-8 branch is reachable for the HASH INPUT (a non-ASCII password works) but not for the realm ECHO.** A non-ASCII configured *username* is caller misconfiguration rather than wire data, so `digestHandler()` rejects it at construction instead | +| A failed background refresh is swallowed unconditionally, `InvariantViolation` included | `AUTH-37` says a failed background refresh MUST NOT fail the in-flight request (log-and-continue) | An earlier shape re-threw `InvariantViolation` from the fire-and-forget `.catch`, reasoning that a programmer error must crash loudly. That was wrong twice: the throw landed in a promise nobody awaits, so it did not surface at the fault — it killed the host process asynchronously, unattributable to any request, while the request that triggered it had already been served a valid token; and the fault it re-raised belongs to caller-supplied `TokenProvider` code, where a blank token is an operational fault (an empty environment variable, a malformed IdP payload) at least as often as a coding one. `docs/knowledge/error-handling.md`'s crash-loudly rule governs OUR invariants at the point WE detect them; it does not license re-raising someone else's failure into a detached promise | +| `docs/knowledge/error-handling.md` forbids "log and continue"; `AUTH-37` mandates it | `error-handling.md:22` — "a `catch` block must end in exactly one of three ways … 'log and continue' is none of these" | Standing, unresolved conflict between the styleguide and the normative spec, resolved in the spec's favour: `AUTH-37`'s clause is explicit and unconditional. The catch is blanket rather than narrowed to one expected type, which `error-handling.md:24` would also prefer otherwise, because the set of failures a caller-supplied provider can raise is not enumerable by this module. The LOG half is still missing and is tracked as a deferred item against Phase 7b | +| Duplicate auth-params within one challenge are last-wins | `AUTH-12` is silent on duplicates | RFC 7235's grammar does not admit them, so any input reaching this case is already malformed and `AUTH-13`'s leniency governs. `Map.set` gives last-wins for free; recorded because it is an unforced choice, not a derived one, and because parameter names are lower-cased first, so `realm` and `REALM` collide | +| `ChallengeHandler.stamp()` is async and takes an optional request context; `rank()` added | The design doc's prose gives `stamp()` a synchronous `string` return and no `rank` | SHA-256 Digest goes through `crypto.subtle.digest()`, which is asynchronous with no synchronous fallback; HA2 needs the method and request-target, which the challenge does not carry. `rank` is what expresses `AUTH-16`'s configured-preference-over-wire-order among several challenges one handler could equally satisfy | +| A generation counter guards the bearer cache against a superseded fetch | Not described either way | `refreshPostEviction()` drops the in-flight slot on the supersede branch, but the older fetch's own `then` would still publish its token into `cached` — re-caching exactly the token the server rejected whenever it settles after the fresh one. The counter is what makes `AUTH-37`'s "so the retry never re-sends the rejected token" hold in both resolution orders | +| `standardResilience()` installs only REDIRECT/RETRY/AUTH, not LOGGING | `docs/knowledge/pipeline.md`'s preset description includes instrumentation | Phase 7b has not shipped at this plan's execution point, and the plan's own 2026-07-29 correction routes the fourth `append` to 7b's Task 9. A scope boundary, not an omission | +| No async-variant preset | The reference's async standard pipeline (retry + instrumentation + caller-supplied scheduler) | 4c already dispositioned this port as one `Promise`-only execution model; there is no second pipeline to give a second preset to | +| The context family and `Step` promoted to public | Not addressed by the plan's Group 2 list | Forced by api-extractor: `StepDescriptor.fn` names `Step` and `StepContext.context` names the `ExecutionContext` union. See "Public-barrel promotion" above | +| `PIPE-24`'s validate-and-reject clause is structurally inexpressible here | `PIPE-24` requires the preset to validate up front that no target pillar is occupied and to reject the whole call, installing nothing, if any is | `standardResilience()` takes a `Transport`, not a builder or an existing pipeline, so it always starts from a fresh `PipelineBuilder` and no slot CAN be occupied. The requirement is satisfied vacuously — there is no code path implementing the validation, because there is no input that could fail it. Recorded so **Phase 9 does not hunt for a check that was never written**. A caller layering the preset onto a customized builder uses `PipelineBuilder.seedFrom(runtime, 'nest' \| 'flatten')`; if a future signature ever accepts a pre-populated builder, the validation becomes both expressible and mandatory | +| Helper functions sit ABOVE their callers in `auth-step.ts`, `bearer-cache.ts`, `digest.ts`, `challenge.ts` | `docs/knowledge/function-design.md` requires the step-down rule — each function above the functions it calls | Bottom-up (primitives first, the exported factory last) is the established shape of every step module since 5a's `retry-step.ts` and 5b's `redirect-step.ts`, and these four read as one family with them. Inverting four files to satisfy the rule costs more than the rule buys, and would leave 5c's modules the only ones ordered differently from their siblings. What WAS fixed is the inconsistency: `handleChallenge` was the single helper sitting below its caller, and now sits above it like the other sixteen | +| Several three-parameter functions take positional parameters, not an options object | `docs/knowledge/function-design.md` requires an options object at three or more parameters — stricter than this repo's `max-params: 3`, which they all pass | `createAuthRequirement(scheme, scopes, params)`, `digestHandler(username, password, options)`, `isBearerTokenExpired(token, nowMs, marginMs)` and `AuthResolutionError`'s `(message, requiredSchemes, availableSchemes)` all read unambiguously at their call sites, and every parameter is a distinct type, so no call can silently transpose two. The rule's real target is the boolean flag and the same-typed neighbour; both were fixed where they occurred — `hashHex` now takes a `HashInput` and `BearerTokenCache.refresh` takes an eviction generation rather than a `postEviction` boolean | +| `stamp` is the module's verb for a computation that RETURNS credential material rather than writing it | `docs/knowledge/naming-conventions.md` bars inventing a verb outside the client-verb taxonomy; `get`/`acquire` would be the sanctioned spellings | Seven symbols already share the vocabulary (`ChallengeHandler.stamp`, `ComposingHandler.stamp`, `stampStaticKey`, `BearerTokenCache.stamp`, `preemptiveStamp`, `stampContext`, and this checklist's own prose), and renaming half of them would leave the module reading in two dialects — worse than either end state. The vocabulary is now DEFINED once, in `ChallengeHandler`'s TSDoc: "to STAMP means to PRODUCE the value the caller writes, never to write it" | +| `ChallengeHandler.stamp` / `ComposingHandler.stamp` take no `isProxy` flag | `AUTH-25` phrases the origin-vs-proxy choice as "an explicit proxy flag" | The flag existed and was threaded from `auth-step.ts` through the composer into both handlers, and NEITHER read it: Basic's value is computed once at construction, and Digest's depends only on the challenge and the request-target. The only tests either could carry for it were tests asserting it changed nothing, which is a parameter kept alive by its own coverage. `AUTH-25`'s flag is still explicit — it is `ChallengeSelection.isProxy`, consumed by `answerHeaderName` in `auth-step.ts`, which is the one place that knows which challenge header the status carried. A future scheme whose VALUE varies by proxy-ness adds it back, and would then have something to assert | +| `AUTH-6`'s all-tiers-absent failure escapes the `DexpaceError` tree | `AUTH-6` calls for "an argument error" | It is an `InvariantViolation`, which extends `Error` directly, is internal-only, and is absent from the barrel — so a caller **cannot narrow on it**, and `catch (e) {if (e instanceof DexpaceError)}` misses it entirely. Sanctioned by the plan's Global Constraints (a caller with no tier configured has a bug, not an operational failure) and precedented by 5a's `retrySettings()` and 5b's `redirectSettings()`; recorded here because the residue is caller-visible, not merely internal. `authStep`'s `@throws` block DOES name it, precisely because a caller cannot narrow on it: the tag is the only place the failure mode is discoverable, so leaving it out would have hidden the one thing this row exists to record. (An earlier revision of this row claimed the opposite; the code was always the honest half.) | + +## Deferred Items (add to the roadmap's Deferred Items Log) + +| Item | Deferred from | Target | Reason | +|---|---|---|---| +| `standardResilience()` gains a `LOGGING` pillar step | Phase 5c | Phase 7b (Task 9) | No real logging step exists at 5c's execution point; the preset grows by one `append` | +| `AUTH-37`'s "log-and-continue" half for a failed background refresh | Phase 5c | Phase 7b | The failure is swallowed today — unconditionally, see the Deviation Ledger — because no `Logger` exists to record it. Until then a provider outage during a background refresh is invisible | +| Re-verification of the "Basic/Digest never preemptively stamp" reading against reference fixtures | Phase 5c | Phase 9 (conformance sweep) | Flagged as an interpretation in the Deviation Ledger, not a certainty | +| RFC 7616 §4 `username*` (RFC 5987) extended notation for a non-ASCII Digest username | Phase 5c | Unscoped | `digestHandler()` rejects a non-header-safe username at construction today. Implementing `username*` would let it be sent correctly rather than refused, and is the standard's own answer | +| A per-**operation** `AuthTiers` source | Phase 5c | Unscoped | `perCall` (via `RequestOptions.auth`) and `client` both have real sources as of Task 14; nothing in this roadmap ships a per-operation layer | +| ~~`DigestChallengeUnsupportedError` consumer confirmation~~ | Phase 5c | **CLOSED in 5c** | Cut before shipping rather than deferred: nothing constructed or caught it, and its stated purpose — a caller driving `digestHandler()` directly — was unreachable, since `digestHandler` is internal. Removing an exported error class is breaking, so cutting it now cost nothing. `docs/work/mvp/2026-09-04-open-items-dissolution.md` G11 | +| A caller-supplied `ChallengeHandler` list on `AuthStepSettings` | Phase 5c | Unscoped | `handlers` was removed at review: it forced three types onto the public barrel and could not compose with the built-in handlers, which stay internal. If a caller ever needs to ADD a handler rather than replace the whole reaction, the shape to ship is an append-semantics field plus public `basicHandler`/`digestHandler` factories — not the replace-semantics field that was cut | diff --git a/docs/superpowers/specs/2026-07-26-phase5c-auth-design.md b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-26-phase5c-auth-design.md rename to docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md index 15a63d6..e517941 100644 --- a/docs/superpowers/specs/2026-07-26-phase5c-auth-design.md +++ b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md @@ -6,14 +6,14 @@ types, the RFC 7235 challenge parser, the Basic/Digest/static-key stamping handlers, and the single AUTH pillar step that ties them together — satisfying `docs/product-spec/11-authentication.md` (`AUTH-1`–`AUTH-38`). This is the third and final sub-phase of the roadmap's Phase 5 split: 5a (retry, done), 5b (redirect — see [Phase 5b -design](./2026-07-26-phase5b-redirect-design.md)), 5c (this document, auth). 5c also closes items the roadmap's +design](../phase5b/2026-07-26-phase5b-redirect-design.md)), 5c (this document, auth). 5c also closes items the roadmap's Deferred Items Log parked here: `PIPE-35`'s `seedFrom`, `AUTH-29`/marker-consumption (5b produced the marker and left consumption to 5c), the standard-resilience preset (`PIPE-24`/`PIPE-39`), and public-barrel promotion of the pillar-step authoring surface. **Governing documents:** `docs/product-spec/11-authentication.md` (normative, cited by ID throughout), `docs/product-spec/10-redirect-handling.md` (`REDIR-7`–`REDIR-11`, `REDIR-24` — the cross-origin marker contract -5c consumes) plus **the [Phase 5b design](./2026-07-26-phase5b-redirect-design.md) itself**, which is the actual +5c consumes) plus **the [Phase 5b design](../phase5b/2026-07-26-phase5b-redirect-design.md) itself**, which is the actual source of truth for that marker's concrete shape (see "Alignment with 5b's shipped design" below — an earlier draft of this section guessed a different, incompatible shape before 5b's doc was found on disk), `docs/product-spec/08-execution-pipelines.md` §8.1 (`PIPE-2`, `PIPE-24`, `PIPE-35`, `PIPE-39`, `PIPE-40`), diff --git a/docs/superpowers/plans/2026-07-26-phase5c-auth.md b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth.md similarity index 99% rename from docs/superpowers/plans/2026-07-26-phase5c-auth.md rename to docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth.md index da0fa0a..a41f9c9 100644 --- a/docs/superpowers/plans/2026-07-26-phase5c-auth.md +++ b/docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth.md @@ -6,7 +6,7 @@ credential types, the RFC 7235 challenge parser, the Basic/Digest/static-key stamping handlers, the bearer token cache, the single AUTH pillar step, `PipelineBuilder.seedFrom()`, and the standard-resilience preset — satisfying `docs/product-spec/11-authentication.md` (`AUTH-1`–`AUTH-38`), per -`docs/superpowers/specs/2026-07-26-phase5c-auth-design.md`. Also closes `PIPE-35`'s `seedFrom`, `AUTH-29`'s +`docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-design.md`. Also closes `PIPE-35`'s `seedFrom`, `AUTH-29`'s marker-consumption side (5b produced the marker), `PIPE-24`/`PIPE-39`'s preset, and public-barrel promotion of the pillar-authoring surface. @@ -21,7 +21,7 @@ the pillar-authoring surface. > **An agent executing this plan must skip the Phase 7b retrofit blocks in Task 16**: build > `standardResilience()` installing the three pillars that exist by then (redirect, retry, auth), leaving > `LOGGING` empty. Phase 7b's plan Task 9 installs the fourth. See -> `docs/superpowers/plans/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. +> `docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md`'s Prerequisite and Task 9. **Architecture:** A new `packages/core/src/auth/` folder of fifteen files, plus one amendment to `packages/core/src/pipeline/builder.ts`. The descriptor/resolver half (`scheme.ts`/`requirement.ts`/ @@ -4128,7 +4128,7 @@ git commit -m "feat(core): standard-resilience preset + public pillar-authoring **Files:** - Verify: full gate sequence (already run at the end of Task 16; re-run here to confirm nothing regressed from the barrel edit's review) -- Create: `docs/superpowers/plans/2026-07-26-phase5c-auth-checklist.md` +- Create: `docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md` **Interfaces:** - Consumes: every symbol from Tasks 1–16. @@ -4153,7 +4153,7 @@ Expected: every gate PASS. `test:node` matters specifically here: `globalThis.cr - [ ] **Step 3: Write the requirement checklist** -Create `docs/superpowers/plans/2026-07-26-phase5c-auth-checklist.md`, same format as +Create `docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md`, same format as `2026-07-26-phase5a-retry-checklist.md`/`2026-07-26-phase5b-redirect-checklist.md` — `| ID | Level | Requirement gist | Status | Where |` tables, legend ✅ shipped / 🚫 never built / ⏳ deferred / N/A. @@ -4199,7 +4199,7 @@ State explicitly at the top whether the plan has been executed, matching the Pha - [ ] **Step 4: Commit** ```bash -git add docs/superpowers/plans/2026-07-26-phase5c-auth-checklist.md +git add docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth-checklist.md git commit -m "docs: Phase 5c requirement checklist" ``` diff --git a/docs/superpowers/specs/2026-07-28-phase6-segmentation-design.md b/docs/work/mvp/phase6/2026-07-28-phase6-segmentation-design.md similarity index 100% rename from docs/superpowers/specs/2026-07-28-phase6-segmentation-design.md rename to docs/work/mvp/phase6/2026-07-28-phase6-segmentation-design.md diff --git a/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-checklist.md b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-checklist.md new file mode 100644 index 0000000..3b93628 --- /dev/null +++ b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-checklist.md @@ -0,0 +1,136 @@ +# Phase 6a — Serde Implementation Plan — Checklist + +Verification of [2026-07-28-phase6a-serde.md](./2026-07-28-phase6a-serde.md) against every requirement ID in +`docs/product-spec/14-serialization-serde.md` (`SERDE-1`–`SERDE-30`), appendix C's `SEAM-19`–`SEAM-23`, and the +two Phase-0 deferrals this phase closes (`NFR-2`, `NFR-14`), as dispositioned by +`docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md`. + +**Status: EXECUTED (2026-08-27).** Every task below is implemented and tested. Deviations, deferrals, and the +requirement clauses satisfied by delegation rather than by code are recorded in `docs/work/mvp/2026-09-04-open-items-dissolution.md` §H — +**read that section alongside this table**; a row here marked ✅ against a delegated clause points at the §H +entry that says what "satisfied" means for it. + +**Legend:** ✅ Implemented and tested — ✅(t) Satisfied by construction, with a test as the only possible +evidence — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred (named target phase) — +N/A Not applicable in this port. + +**A note on the collapsed rows.** The design's "Collapsed Requirements" table +([design §Collapsed Requirements](./2026-07-28-phase6a-serde-design.md)) dispositions six MUSTs as N/A +or satisfied-by-construction. Phase 9's sweep must read that table rather than re-deriving them, or those six +read as uncovered. Every collapsed row below points back at it. + +## §3 — The wire-codec seam (`SEAM-19`–`SEAM-23`) + +Listed first because §3's seam requirements and §14's serde chapter overlap, and Phase 9 audits both indexes. + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SEAM-19 | MUST | Bundle of serializer + deserializer + undeclared-nowhere media type; concrete codecs outside core | ✅ | Task 2 — `packages/core/src/seams/serde.ts` (`interface Serde`). `mediaType` is a required non-optional field, asserted by a `@ts-expect-error` in `seams/serde.test.ts` that omitting it does not compile. Concrete codec lives in `packages/codec-json/`, a separate package | +| SEAM-20 | MUST | Four allocation profiles; encode failures a stable SDK type; stream-write I/O unwrapped; fixed-buffer overflow a bounds error | ✅ | Task 2 (`interface Serializer`), Task 9 (`packages/codec-json/src/json-serde.ts`). All four profiles present and asserted in `seams/serde.test.ts` ("all four SEAM-20 allocation profiles are present, including the fresh-string one") and exercised in `json-serde.test.ts`. Overflow → plain `RangeError`; encode failure → `SerializationError` | +| SEAM-21 | MUST | Explicit runtime type token, not an erased/inferred generic | ✅ | Task 2 + Task 7 — every decode entry point takes `Schema<T>`; `Serde` dropped its type parameter. Phase 2's `@internal` marking removed and the seam promoted to `packages/core/src/index.ts`, proven by `packages/core/src/index.public.test.ts` | +| SEAM-22 | MUST | Generic type capture rejects an unresolved type variable at construction | N/A | Collapsed — see the design's table. Nothing is inferred from an erased generic at runtime, so the state SEAM-22 guards against is unreachable; the compiler refusing a call site with no concrete schema is earlier and stronger | +| SEAM-23 | MUST | Stable SDK-owned failure hierarchy, adapters throw it instead of leaking the backing type, cause always chained | ✅ | Task 1 — `packages/core/src/serde/errors.ts`, tested in `serde/errors.test.ts` (header cites SEAM-23). **Structural deviation recorded at §H3:** two flat leaves under `DexpaceError` plus `isSerdeError`, not a `SerdeError` base class — the two-level cap 3b retrofitted for | + +## §14.1 — The bundle and its media type + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-1 | MUST | One bundle exposing exactly one encoder and one decoder for one wire format | ✅ | Task 2 (`interface Serde`), Task 9 (`jsonSerde()`). Round-trip through one bundle asserted in `codec-json/src/json-serde.property.test.ts` (fast-check) and `json-serde.test.ts` | +| SERDE-2 | MUST | Declared media type is the default `Content-Type` when a body is built from value + serde; never defaulted at the SPI | ✅ | Task 4 — `packages/core/src/body/serde-body.ts`. `body/serde-body.test.ts` asserts the default, an explicit override, and that a non-JSON serde stamps its own type. No format-agnostic fallback exists on the path. Cross-package proof in `codec-json/src/cross-package.test.ts` and `test/node-conformance/serde.test.mjs` | + +## §14.2 — Allocation profiles and stream ownership + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-3 | MUST | Read to EOF / write fully, never close or take ownership of the caller's stream | ✅ | Task 9 + Task 10 — `json-serde.ts` releases the writer/reader lock in a `finally` and never calls `close()`/`cancel()`. `json-serde.test.ts` uses close-counting stream wrappers on both directions; re-asserted against Node's own Web Streams in `test/node-conformance/serde.test.mjs` | +| SERDE-4 | MUST | Encode-into-buffer returns bytes written, honors offset, throws a non-serde `RangeError` on overflow or bad offset, leaves `[0, offset)` untouched | ✅ | Task 9 — `serializeInto`. Range-checked before encoding, so the buffer is never partially written. `json-serde.test.ts` asserts the byte count, the untouched prefix, the exactly-fitting boundary case, an out-of-range offset, and that the `RangeError` carries **no** `cause` | + +## §14.3 — The type witness + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-5 | MUST | Every decode takes an explicit runtime type witness | ✅ | Task 2 (`Schema<T>` mandatory on `deserialize`/`deserializeFrom`), Task 10. `seams/serde.test.ts` asserts the return type is driven by the schema argument, not by the bundle | +| SERDE-6 | MUST | Parametric targets expressible; a decoder that cannot resolve type arguments fails loudly | ✅ | Task 2 + Task 10 — schema combinators (`z.array(Dto)`) are the carrier; there is no raw path to fall back to, so the "silently decodes into the wrong type" failure is unreachable. Asserted in `seams/serde.test.ts` and `json-serde.test.ts` ("a parametric target is just a combinator schema — no carrier type exists") | +| SERDE-7 | MUST | Ergonomic reified/inline decode helper routes through the generic carrier | N/A | Collapsed — see the design's table. TypeScript has no reified generics; the schema *is* the reification and is already mandatory on every entry point, so there is no less-typed path for a helper to route away from | +| SERDE-8 | MUST | The carrier rejects construction with no type argument or an unresolved type variable | N/A | Collapsed — see the design's table. Vacuous: nothing is inferred from an erased generic at runtime | + +## §14.4 — Failures + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-9 | MUST | Failures surface as the SDK's stable serde type, chaining the original, with no backing-library type escaping | ✅ | Task 1 + Tasks 9/10 — `json-serde.ts` wraps every `JSON.parse`/`JSON.stringify` throw. `json-serde.test.ts` asserts the SDK type and the chained `cause` on malformed JSON, on a schema rejection, and across **every** allocation profile | +| SERDE-10 | MUST | Write path a serialization subtype, read path a deserialization subtype, both off a common root | ✅ | Task 1 — `SerializationError` / `DeserializationError`, both directly under `DexpaceError`, grouped by `isSerdeError`. Structure deviates (§H3); direction and catch-one-category both hold | +| SERDE-11 | SHOULD | Serde failures are unchecked, not a declared/checked exception | ✅ | By construction — JavaScript has no checked exceptions. Recorded in `serde/errors.test.ts`'s header comment as having nothing to assert | +| SERDE-12 | MUST | A genuine stream I/O error propagates unwrapped; only malformed-input / shape-mismatch / unencodable-value failures are wrapped | ✅ | Task 5 + Task 10 — `response-handlers.ts` rethrows anything already in this SDK's typed tree untouched (`e instanceof DexpaceError`, which covers all five FLAT `io/errors.ts` leaves plus `HttpStatusError`); the codec catches nothing off `read()`. `response-handlers.test.ts` asserts unwrapped propagation one case per leaf; `json-serde.test.ts` asserts the codec re-wraps nothing coming off the stream. **See §H8:** the narrower `instanceof IoError` guard this replaced was a real defect, and one residual limit remains — a *foreign* transport's stream error is indistinguishable from a non-conforming codec leaking one, so it is still surfaced as `DeserializationError` | +| SERDE-13 | MUST | A wire `null` into a non-null target fails naming the target, across every decode overload | ✅ | Task 10 — the single `decodeText` funnel both entry points route through, so "across every overload" is mechanically true for this codec. `json-serde.test.ts` asserts it on both entry points, asserts it fires *before* the schema runs, and asserts the documented `'the target type'` fallback label. **See §H9:** every target is treated as non-null, which is deliberate and has two named consequences | + +## §14.5 — The PATCH tri-state type + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-14 | MUST | Exactly three states, Present-of-null unrepresentable, covariant in the value type | ✅ | Task 3 — `packages/core/src/serde/tristate.ts`. `present<T>(value: NonNullable<T>)` makes the fourth state unrepresentable at the **type** level (earlier than the reference's runtime rejection), asserted by a `@ts-expect-error` in `serde/tristate.test.ts`; covariance asserted separately ("Absent and Null are assignable to any parameterization") | +| SERDE-15 | MUST | Absent omits the key, Null emits a wire null, Present emits the value | ✅ | Task 11 — `packages/codec-json/src/tristate-replacer.ts`. All three asserted in `tristate-replacer.test.ts`, plus the nested-Tristate and decoy-object cases; re-asserted on Node in `test/node-conformance/serde.test.mjs` | +| SERDE-16 | MUST | Missing key → Absent, explicit null → Null, value → Present with the element type preserved | ✅ | Task 12 — `packages/codec-json/src/tristate-schema.ts` (`tristate()`). `tristate-schema.test.ts` covers all three, and an `expectTypeOf` assertion covers the element-type preservation a runtime test cannot see | +| SERDE-17 | MUST | A tri-state field with no key on the wire resolves to Absent, via the field default rather than a null hook | ✅ | Task 12 — `tristateObject()` looks the key up and feeds a module-private missing sentinel to the field's schema, because a `JSON.parse` reviver never fires for an absent key. Asserted in `tristate-schema.test.ts` and on Node | +| SERDE-18 | SHOULD | Construction/consumption helpers; `ofNullable` can never yield Absent; fold, value-or-null, three predicates | ✅ | Task 3 — `absent`, `nullValue`, `present`, `ofNullable`, `foldTristate`, `valueOrNull`, `isAbsent`/`isNull`/`isPresent`. `ofNullable` never yielding Absent is asserted directly. `fold` is named `foldTristate` to avoid colliding with 4b's `Outcome.fold` in one barrel | +| SERDE-19 | MUST | The default codec configuration wires the tri-state semantics; opting out is explicit | ✅ | Task 9 — `jsonSerde()` installs the replacer by default; `{tristate: false}` is the only way out. `tristate-replacer.test.ts` asserts both branches, including that opting out makes Absent and Null indistinguishable | +| SERDE-20 | SHOULD | Top-level / array-element Tristate degrades to a wire null rather than throwing | ✅ | Task 11, in **two** places — the top-level position is resolved by `json-serde.ts`'s `encodeToText` *before* `JSON.stringify` runs (via `degradeTopLevelTristate`), because a replacer cannot tell the root from a key literally named `""`; the array-element position needs no code at all, since `JSON.stringify` itself emits `null` for an element whose replacer returned `undefined`. Do not collapse the two back together. `tristate-replacer.test.ts` covers top-level, array-element, nested-array (indices never shift at depth), and the `""`-key case that forced the split | + +## §14.6 — Codec configuration policy + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-21 | MUST | Reject cross-shape scalar coercions | ✅(t) | Collapsed — `JSON.parse` performs no coercion, so no code implements this and **the tests are the coverage**. All twelve enumerated pairs asserted in `packages/codec-json/src/conformance.test.ts`, including `empty-string → floating-point` | +| SERDE-22 | MUST | Permit representation-preserving conversions | ✅(t) | Collapsed — JavaScript has one numeric type, so integer→float widening is not a conversion. Three rows in `conformance.test.ts` | +| SERDE-23 | SHOULD | Ignore unknown/unexpected fields rather than failing | ✅ | **Delegated, not enforced — see §H2.** The policy is the caller's schema's; documented as a recommendation in `jsonSerde`'s TSDoc. Two rows in `conformance.test.ts` prove the delegation is real: the same codec and the same bytes give opposite outcomes for a permissive and a strict schema | +| SERDE-24 | SHOULD | Date/time emitted as ISO-8601, round-tripping to the same instant | ✅(t) | Collapsed — `Date.prototype.toJSON` emits ISO-8601. `conformance.test.ts` asserts the exact wire form and instant equality | +| SERDE-25 | SHOULD | The default-configuration factory returns a fresh instance per call | ✅ | Task 9 — `jsonSerde()` allocates and freezes per call, asserted in `json-serde.test.ts`. Trivial here: the requirement's rationale (mutable codec caches) has no analog — there is no engine and no cache | +| SERDE-26 | MUST | Never mutate a caller-supplied codec instance; operate on a private copy | N/A | Collapsed — see the design's table. There is no codec object to supply, copy, or mutate; `jsonSerde()` takes options, not an engine. The failure mode (reconfiguring an `ObjectMapper` the caller also uses) is unreachable | + +## §14.7 — Response handlers + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-27 | MUST | Stream the body through the deserializer without materializing; close on every path; missing body a serde exception naming the target; codec failure chained; mid-stream I/O unwrapped | ✅ | Task 5 — `packages/core/src/serde/response-handlers.ts` (`decodeResponse`). `response-handlers.test.ts` covers the success path, the missing body with and without a `typeName`, the wrapped codec failure, the already-typed failure that is not double-wrapped, the unwrapped stream failure, and both close-failure orderings. Close-on-every-path is built on 4b's `releaseQuietly`/`withReleaseFailure` (§H4 item 2), so a close failure never displaces the real one. **The no-materialize clause is honored at the seam and unavoidably broken by this codec — §H1** | +| SERDE-28 | MUST | Decode only 2xx; 4xx/5xx throws the mapped HTTP error with a bounded buffered body; any other non-2xx closes and raises a status-leading serde exception preserving ETag/Location | ✅ | Task 6 — `decodeSuccessResponse`. 4xx/5xx delegates to 3b's `toHttpError()` at the shared 1 MiB `BODY-30`/`HTTP-52` cap rather than building a second one. `response-handlers.test.ts` covers 2xx, 500, a non-canonical 599, 304 (ETag/Location preserved as readable fields), 1xx, the close-also-fails ordering, and the fallback label | + +## §14.8 — Concurrency and diagnostics + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SERDE-29 | SHOULD | A configured serde is safe to share across concurrent tasks | ✅(t) | Collapsed — single-threaded event loop, and the bundle is `Object.freeze`d and stateless. `conformance.test.ts` drives 200 genuinely interleaved round-trips through one bundle — each awaits a multi-chunk `deserializeFrom`, so the calls really do overlap — asserting no cross-talk | +| SERDE-30 | MAY | Absent/Null sentinels provide a stable, identity-free textual representation | ✅ | Task 3 — shipped as the exported `tristateToString()` free function rather than a `toString()` on the sentinels, which is a mismatch against the design's own coverage row; recorded at §H4 item 6. Asserted in `serde/tristate.test.ts` | + +## Phase-0 deferrals closed here + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| NFR-2 | SHOULD | Each optional capability a separately installable unit depending on core plus at most one third-party library | ✅ | Task 8 — `packages/codec-json` ships with `dependencies: {}` hard-committed and **zero** external libraries, its only edge to core being a peer. `scripts/verify-seam-1.mjs` was generalized from core-only to every package under `packages/`, and `scripts/verify-seam-1.test.mjs` drives that script against fixture trees to prove it still fails when it should. Codec half closed; the transport half stays Phase 8a | +| NFR-14 | SHOULD | Dependency and tool versions live in a single source of truth | ✅ | Task 8 — the workspace root's `workspaces.catalog` block single-sources `typescript`, `@microsoft/api-extractor`, `expect-type`, and `fast-check`; the root's own `devDependencies` and both member packages reference them as `"catalog:"`. Confirmed against the pinned Bun version (`.bun-version` 1.3.14; catalogs landed in 1.2.0), so the fallback Task 8 allowed for was not needed | +| Peer-dependency dedup | — | Every adapter declares `@dexpace/core` as a peer, guarding the dual-package hazard | ✅ | Task 8 + Task 13 — `peerDependencies` + `peerDependenciesMeta`, asserted for every non-core package by `verify-seam-1.mjs`. `codec-json/src/cross-package.test.ts` proves the **consequence** rather than the declaration: a `Tristate` built in core is recognized by the codec's replacer, because `TRISTATE_BRAND` is a registry-global `Symbol.for` | +| NFR-8 | SHOULD | Shrinker keep/retain configuration covering the runtime-wired SPI seams and the Tristate type | ⏳ | **Deferred to Phase 9 — §H5.** Both surfaces are created here and neither is keep-configured here: the keep-config and its guard are one workspace-wide deliverable. `plans/2026-07-28-phase9-cross-cutting-conformance.md` ships `@dexpace/shrink-test` with `@dexpace/codec-json` and `jsonSerde` already in `participatingPackages`. 6a's only obligation is that both stay reachable through the public barrels, which `index.public.test.ts` and `cross-package.test.ts` prove | +| NFR-9 | SHOULD | Automated shrink-survival regression guard wired into the default build | ⏳ | **Deferred to Phase 9 — §H5.** Same deliverable as `NFR-8` | + +## Open items this phase raised + +Recorded in full at `docs/work/mvp/2026-09-04-open-items-dissolution.md` §H. Summary, so a Phase 9 sweep does not have to reconstruct it: + +| § | Kind | Gist | +|---|---|---| +| H1 | Accepted deviation | The JSON codec buffers the whole body before parsing (`SERDE-27`); the seam itself does not | +| H2 | Accepted deviation | `SERDE-23` satisfied by delegation to the caller's schema, not by enforcement | +| H3 | Accepted deviation | No serde-specific error base class; two flat leaves plus `isSerdeError` (`SEAM-23`) | +| H4 | Recorded | Six places the shipped code departs from this plan as written | +| H5 | Deferred (Phase 9) | `NFR-8`/`NFR-9` shrinker keep-configuration | +| H6 | Deferred (Phase 10) | Assertion density, project-wide | +| H7 | Recorded | Coverage excludes `**/dist/**`; `bun test` is now build-dependent | +| H8 | Partly resolved / open | One bug fixed (the typed-tree pass-through); a *foreign* transport's stream error is still wrapped as `DeserializationError` | +| H9 | Open | Every decode target is treated as non-null; two named consequences | +| H10 | Open | One concept, two spellings: positional SPI vs. `DecodeTarget` at the handler layer | +| H11 | Open | `tristate()`/`tristateObject()` are format-agnostic but ship in a format-specific package | +| H12 | Open | `seams/index.ts` is an unimported internal barrel the corpus bans | +| H13 | Open | `test:scripts` runs in no CI job | +| H14 | Open | `decodeSuccessResponse`'s 4xx/5xx branch delegates to `toHttpError`, whose bare `finally { close() }` can displace the primary failure | +| H15 | Open | No `AbortSignal` on any of the four long-running async APIs this phase ships | +| H16 | Recorded | Deep-nesting encode diverges between Bun and Node; both outcomes are correct, so no gate | +| H17 | Recorded | `SERDE-20`'s array-element degradation comes from `JSON.stringify` itself; the replacer branch for it was dead | diff --git a/docs/superpowers/specs/2026-07-28-phase6a-serde-design.md b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md similarity index 97% rename from docs/superpowers/specs/2026-07-28-phase6a-serde-design.md rename to docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md index 264c7c7..6f92bb9 100644 --- a/docs/superpowers/specs/2026-07-28-phase6a-serde-design.md +++ b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md @@ -6,7 +6,7 @@ `SEAM-21`), `Tristate<T>`, the serde error leaves, `SERDE-2`'s media-type-as-default-`Content-Type` wiring, and `SERDE-27`/`SERDE-28`'s response handlers — plus the workspace's first second package, `@dexpace/codec-json`. Satisfies `docs/product-spec/14-serialization-serde.md` (`SERDE-1`–`SERDE-30`). First of the three sub-phases the -[Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: **6a** (this +[Phase 6 segmentation design](../2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: **6a** (this document, serde), 6b (SSE, `§13`), 6c (pagination, `§12`). **Governing documents:** `docs/product-spec/14-serialization-serde.md` (normative, cited by ID throughout), @@ -261,10 +261,12 @@ serde semantics) rather than methods. **Runtime-floor note on the close-failure path.** Preserving a decode failure as primary while carrying the close failure alongside it is what `SuppressedError` is for, and `SuppressedError` is **not available on the - declared `engines.node` floor** — it is a V8 global from the full Explicit Resource Management proposal, - absent on every 18.x runtime, and `esnext.disposable` in `lib` supplies only the type. This is the open - cross-phase decision recorded at `plans/2026-07-25-phase4b-recovery-chain.md:22-47`; 6a is a fourth site - alongside 5a, 6b and 6c, and whichever option lands there lands here unchanged. + declared `engines.node` floor** — it belongs to the full Explicit Resource Management proposal, which reached + Node only in 24.0.0, against a floor of `>=20.3`, and this package's `lib` does not supply its type either. + The cross-phase decision is **closed**: Phase 4b resolved it to branch (b) and shipped + `suppress(error, suppressed, message)` in `packages/core/src/suppress.ts` — native class where the runtime has + one, shape-compatible stand-in where it does not. 6a calls that helper and asserts its shape, never + `instanceof SuppressedError`. - `SERDE-28`: 2xx decodes. **4xx/5xx delegates to 3b's `toHttpError()`** — that function already buffers a bounded error body inside the response's own close-guaranteeing scope, at the shared 1 MiB cap `BODY-30`/`HTTP-52` define and `§14` itself points at. Building a second cap here would be a defect. Other non-2xx (1xx, an diff --git a/docs/superpowers/plans/2026-07-28-phase6a-serde.md b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde.md similarity index 99% rename from docs/superpowers/plans/2026-07-28-phase6a-serde.md rename to docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde.md index b4ef2ca..ae7a0ee 100644 --- a/docs/superpowers/plans/2026-07-28-phase6a-serde.md +++ b/docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde.md @@ -6,7 +6,7 @@ (closing `SEAM-21`), `Tristate<T>`, two serde error leaves, `serdeBody()`, and the two response handlers — plus the workspace's first second package, `@dexpace/codec-json`, satisfying `product-spec/14-serialization-serde.md` (`SERDE-1`–`SERDE-30`) per -`docs/superpowers/specs/2026-07-28-phase6a-serde-design.md`. +`docs/work/mvp/phase6/phase6a/2026-07-28-phase6a-serde-design.md`. **Architecture:** A new `packages/core/src/serde/` folder of five independent files with no folder-level barrel, plus one modified file in `src/seams/` (Phase 2's provisional `Serde<T>` is reshaped in place) and one new file in @@ -2584,7 +2584,7 @@ git commit -m "feat(codec-json): add the tristate() decode combinator resolving - Create: `packages/codec-json/src/conformance.test.ts` - Create: `packages/codec-json/etc/codec-json.api.md` (generated) - Create: `.changeset/phase6a-codec-json.md` -- Modify: `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (mark the three retargeted rows +- Modify: `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` (mark the three retargeted rows resolved) **Interfaces:** @@ -2759,7 +2759,7 @@ Initial release: `jsonSerde()`, the `Tristate` PATCH replacer (on by default), a - [ ] **Step 7: Close out the roadmap rows** -In `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`, mark these Deferred-Items-Log rows' +In `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`, mark these Deferred-Items-Log rows' target column **Resolved in Phase 6a**, adding one sentence of evidence each: - `NFR-2` — codec half closed; `packages/codec-json` ships with `dependencies: {}` and zero external libraries. @@ -2786,7 +2786,7 @@ command's output instead if one does not. - [ ] **Step 9: Commit** ```bash -git add packages/codec-json .changeset docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md +git add packages/codec-json .changeset docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md git commit -m "test(codec-json): guard the dual-package hazard and the collapsed SERDE requirements (SERDE-21/22/24/29)" ``` diff --git a/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-checklist.md b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-checklist.md new file mode 100644 index 0000000..ef716b5 --- /dev/null +++ b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-checklist.md @@ -0,0 +1,85 @@ +# Phase 6b — Server-Sent Events Implementation Plan — Checklist + +Verification of [2026-07-28-phase6b-sse.md](./2026-07-28-phase6b-sse.md) against every requirement ID in +`docs/product-spec/13-server-sent-events-and-streaming.md` (`SSE-1`–`SSE-41`), as dispositioned by +`docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md`. + +**Status: EXECUTED (2026-08-27).** Every task below is implemented and tested across 1,497 repository tests, 40 script tests, and 79 Node conformance tests. Deviations, deferrals, and design rationales are recorded in `docs/work/mvp/2026-09-04-open-items-dissolution.md` §I and the roadmap design. + +**Legend:** ✅ Implemented and tested — ✅(t) Satisfied by construction, with a test as the only possible evidence — ⏳ Deferred (named target phase) — N/A Not applicable in this port. + +--- + +## §13.1 — Event Model (`SSE-20`–`SSE-22`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-20 | MUST | Immutable `SseEvent` with defensively copied `data` lines | ✅ | Task 1 — `packages/core/src/sse/event.ts` (`makeSseEvent()`). `event.test.ts` asserts `Object.isFrozen(event)` and `Object.isFrozen(event.data)`. | +| SSE-21 | MUST | Value equality and string representation | ✅ | Task 1 — `packages/core/src/sse/event.ts` (`sseEventsEqual()`, `sseEventToString()`). Tested in `event.test.ts`. (Hash equality is N/A in JS, recorded in §I). | +| SSE-22 | MUST | `isSseEventEmpty` predicate (comment counts as content) | ✅ | Task 1 — `packages/core/src/sse/event.ts` (`isSseEventEmpty()`). Tested in `event.test.ts`. | + +--- + +## §13.2 — Line Framing, Parsing & Grammar (`SSE-1`–`SSE-19`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-1 | MUST | Dispatch on blank line & reset per-block accumulators | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-2 | MUST | `\n`, `\r`, and `\r\n` line framing | ✅ | Task 2 — `packages/core/src/sse/line-reader.ts`. Tested across split chunk boundaries in `line-reader.test.ts` and `line-reader.property.test.ts`. | +| SSE-3 | MUST | First-colon field splitting | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-4 | MUST | Present-but-empty recorded as `""`, distinct from absent (`undefined`) | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-5 | MUST | Single leading `U+0020` space stripped | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-6 | MUST | Leading `:` captures comment (latest wins) & dispatches | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-7 | MUST | Unknown fields silently ignored | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-8 | MUST | `data` lines accumulated in wire order as `string[]` | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-9 | MUST | `id` with `\u0000` dropped completely | ✅ | Task 3 — `packages/core/src/sse/parser.ts` & `event.ts`. Tested in `parser.test.ts` and `event.test.ts`. | +| SSE-10 | MUST | `event` not defaulted to `"message"` | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-11 | MUST | `retry` digits-only ASCII capped at `Number.MAX_SAFE_INTEGER` | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-12 | MUST | Leading UTF-8 BOM stripped via lookahead (`peek()`) once at stream start; later BOMs preserved | ✅ | Task 2 — `packages/core/src/sse/line-reader.ts`. Tested in `line-reader.test.ts`, `parser.test.ts`, and `test/node-conformance/sse.test.mjs`. | +| SSE-13 | MUST | Permissive dispatch (any of the 5 fields set emits event) | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-14 | MUST | EOF dispatch of pending fields | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-15 | MUST | Stable end sentinel (`SSE_END`) | ✅ | Task 2 + Task 3 — `packages/core/src/sse/line-reader.ts` & `parser.ts`. Tested in `line-reader.test.ts` and `parser.test.ts`. | +| SSE-16 | MUST | Single-pass; no `last-event-id` state retention across events | ✅ | Task 3 — `packages/core/src/sse/parser.ts`. Tested in `parser.test.ts`. | +| SSE-17 | MUST | Parser does not close or own `BufferedSource` | ✅ | Task 2 + Task 3 — `packages/core/src/sse/line-reader.ts` & `parser.ts`. Tested in `parser.test.ts`. | +| SSE-18 | MUST | Single-consumer model re-expressed as single-pass AsyncGenerator | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-19 | MUST | Configurable line cap (`maxLineBytes` / `SseLineTooLongError`) | ✅ | Task 2 — `packages/core/src/sse/line-reader.ts`. Tested in `line-reader.test.ts`. | + +--- + +## §13.3 — Stream Facade & Resource Management (`SSE-23`–`SSE-32`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-23 | MUST | Exactly-once resource release across all termination paths | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `lifecycle.test.ts` (6-path matrix). | +| SSE-24 | MUST | Clean stream termination automatically releases resource | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts` and `lifecycle.test.ts`. | +| SSE-25 | MUST | Partial consume release via iterator `.return()` / early `break` | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts` and `lifecycle.test.ts`. | +| SSE-26 | MUST | Re-iteration throws `SseStreamError` | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-27 | MUST | Post-close iteration throws `SseStreamError`; mid-pull close ends cleanly | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-28 | MUST | Idempotent `close()` | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-29 | MUST | Mid-stream failure releases resource first; close error suppressed | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-30 | MUST | Clean terminal release failure swallowed/reported out-of-band; explicit `close()` propagates | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-31 | MUST | Close during in-flight read mapped to `IoError` | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts` and `test/node-conformance/sse.test.mjs`. | +| SSE-32 | MUST | `sseStreamFrom` binds response body lifecycle; rejects bodyless response | ✅ | Task 7 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | + +--- + +## §13.4 — Typed Adapter (`SSE-33`–`SSE-36`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-33 | MUST | Typed adapter passes raw event name + newline-joined data | ✅ | Task 6 — `packages/core/src/sse/typed.ts`. Tested in `typed.test.ts`. | +| SSE-34 | MUST | `MapperOutcome<T>` union (`mapperValue`, `MAPPER_SKIP`, `MAPPER_DONE`) | ✅ | Task 6 — `packages/core/src/sse/typed.ts`. Tested in `typed.test.ts`. | +| SSE-35 | MUST | Lazy per-element mapping | ✅ | Task 6 — `packages/core/src/sse/typed.ts`. Tested in `typed.test.ts`. | +| SSE-36 | MUST | Throwing mapper releases resource before propagating error | ✅ | Task 6 — `packages/core/src/sse/typed.ts`. Tested in `typed.test.ts`. | + +--- + +## §13.5 — Boundaries, Flow Control & Isolation (`SSE-37`–`SSE-41`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-37 | MUST | Zero serde dependencies in core SSE | ✅ | Task 8 — `scripts/verify-sse-37.mjs`, asserted in CI and `test:scripts`. | +| SSE-38 | MUST | No reconnect or `Last-Event-ID` path in core SSE | ✅ | Task 8 — `scripts/verify-sse-37.mjs`, asserted in CI and `test:scripts`. | +| SSE-39 | MUST | Pull-based flow control (1:1 with consumer demand) | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-40 | MUST | Single-pass lazy view reusing reader | ✅ | Task 5 — `packages/core/src/sse/stream.ts`. Tested in `stream.test.ts`. | +| SSE-41 | MAY | Reactive adapter view (`Observable`) | ⏳ | Deferred to Phase 8b (`@dexpace/rx`). Recorded in §I and roadmap. | diff --git a/docs/superpowers/specs/2026-07-28-phase6b-sse-design.md b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md similarity index 97% rename from docs/superpowers/specs/2026-07-28-phase6b-sse-design.md rename to docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md index 460183d..09a3bcd 100644 --- a/docs/superpowers/specs/2026-07-28-phase6b-sse-design.md +++ b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md @@ -5,7 +5,7 @@ **Purpose:** Implement the SSE subsystem — the WHATWG line/field grammar as a state machine, the immutable `SseEvent` value, the resource-owning single-pass stream facade, and the typed adapter — satisfying `docs/product-spec/13-server-sent-events-and-streaming.md` (`SSE-1`–`SSE-41`). Second of the three sub-phases the -[Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: 6a (serde, `§14`), +[Phase 6 segmentation design](../2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: 6a (serde, `§14`), **6b** (this document, SSE), 6c (pagination, `§12`). **Governing documents:** `docs/product-spec/13-server-sent-events-and-streaming.md` (normative, cited by ID @@ -193,9 +193,9 @@ internal release routine, not inferred from context: consumer already received, which is the specific harm `SSE-30` names. - Release on an **explicit `close()`** — a failing close propagates. The caller asked; the caller hears. - Release with **an error already in flight** (`SSE-29`, `SSE-36`) — the primary error propagates with the close - failure attached via native `SuppressedError`, the same mechanism 5a uses. + failure attached via Phase 4b's guarded `suppress()` helper, the same mechanism 5a uses. -> **`SuppressedError` is blocked on a cross-phase decision, and 6b does not get to make it.** +> **`SuppressedError` was blocked on a cross-phase decision; it is now closed and 6b just calls the helper.** > `plans/2026-07-25-phase4b-recovery-chain.md:24-48` establishes that `SuppressedError` is a V8 global from the > full Explicit Resource Management proposal, absent on every 18.x runtime — Node backported > `Symbol.dispose`/`Symbol.asyncDispose` alone — while `engines.node` is `">=18.17"` and `verify:node-floor` @@ -325,7 +325,7 @@ Phase 9's sweep reads this table rather than re-deriving it. |---|---|---| | `BufferedSource` (`overStream`, `exhausted`, `readByte`, `readExactly`, `peek`, `close`) | 3a | `peek()` is `SSE-12`'s lookahead; `close()` is already idempotent and already rejects later reads, covering most of `SSE-27`/`SSE-28`. `exhausted()` — not a sentinel from `readByte()` — is how end of stream is detected, because `readByte()` rejects rather than returning one | | `IoError` | 3a | Every read-path failure surfaces as one shape, including the torn-down-mid-read case | -| `SuppressedError` usage pattern | 5a | `SSE-29`/`SSE-36`'s "close failure attached to the primary" is the identical mechanism — **and inherits 5a's unresolved blocker**, see below | +| `suppress()` usage pattern | 4b, 5a | `SSE-29`/`SSE-36`'s "close failure attached to the primary" is the identical mechanism, over 4b's guarded helper. The former cross-phase blocker is closed | | `Response.body` / `Response.close()` | 3b | `sseStreamFrom` binds to them; it does not reach for a transport — which is also half of why `SSE-38` holds by construction | | The `kind`-discriminated union idiom | 4b | `MapperOutcome<T>` mirrors `Outcome<T>`'s shape without extending its type | @@ -408,4 +408,4 @@ would publish a way to violate `SSE-17`'s non-ownership contract by accident. | `SSE-37` and `SSE-38` enforced by a build script, not by module-graph structure | `SSE-37`, `SSE-38` | The reference gets it free from package boundaries; this port puts serde in the same package, so the invariant needs a mechanical guard or it is only a convention. The script strips comments before the `SSE-38` marker scan and skips `*.test.ts` for markers only — a gate that failed on a TSDoc *documenting* the absence of reconnection would be deleted rather than obeyed | | `[Symbol.asyncDispose]` on `SseStream` is optional and runtime-guarded, not an `implements AsyncDisposable` | `styleguide/typescript/13` §13.1–13.2 | The symbol postdates the declared `>=18.17` floor that `verify:node-floor` pins, and TypeScript does not polyfill it for a declaring library. `close()` stays the supported path everywhere; dispose delegates to it. Cost: `await using` does not type-check against an optional member. Unconditional once the floor moves past 18.18 | | Byte-at-a-time line framing via `readByte()` | `docs/knowledge/performance.md:16-17` | `readByte()` is `readExactly(1)` underneath, so framing allocates per byte on the parse path. Deferred deliberately on the guide's own terms (`performance.md:4,24` — no micro-fix before a profile names the bottleneck) and recorded so Phase 10 revisits it with a `*.bench.ts` instead of rediscovering it. A bulk-`read()` framing is the fix if a profile calls for one; no observable contract changes | -| `SSE-29`/`SSE-36` construct a native `SuppressedError` | `NFR-10` / the declared `>=18.17` floor | Not 6b's deviation to take or reverse — inherited from the cross-phase blocker at `plans/2026-07-25-phase4b-recovery-chain.md:24-48`, which must resolve across 4b/5a/6a/6b/6c together. Listed here so Phase 10 sees 6b in that set | +| `SSE-29`/`SSE-36` pair errors through `suppress()` rather than native `SuppressedError` | none — a runtime-floor constraint, not a spec deviation | Inherited from 4b's F1 resolution (branch (b), 2026-08-26): the native class reached Node in 24.0.0, against a `>=20.3` floor. Listed here so Phase 10 sees 6b in that set | diff --git a/docs/superpowers/plans/2026-07-28-phase6b-sse.md b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse.md similarity index 98% rename from docs/superpowers/plans/2026-07-28-phase6b-sse.md rename to docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse.md index 306c580..ea2591a 100644 --- a/docs/superpowers/plans/2026-07-28-phase6b-sse.md +++ b/docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse.md @@ -5,7 +5,7 @@ **Goal:** Ship the SSE subsystem in `@dexpace/core` — the CR/LF/CRLF line reader, the WHATWG field-grammar state machine, the immutable `SseEvent` value, the resource-owning single-pass stream facade, and the typed adapter — satisfying `product-spec/13-server-sent-events-and-streaming.md` (`SSE-1`–`SSE-41`) per -`docs/superpowers/specs/2026-07-28-phase6b-sse-design.md`. +`docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md`. **Architecture:** A new `packages/core/src/sse/` folder of six independent files with no folder-level barrel. Byte access and lifecycle come from Phase 3a's `BufferedSource`; **line framing does not** — `IO-14` keeps a lone @@ -14,26 +14,18 @@ The parser is a stateful class (`SSE-15`/`SSE-16` need observable state, `SSE-17 generator lives one layer up in the facade, where ownership belongs. **Tech Stack:** TypeScript 5.8+, `bun test`, `fast-check` for the two chunk-independence/round-trip properties, -native `SuppressedError`. No new runtime dependencies. No `node:` imports. **No serde imports at all** — enforced +Phase 4b's guarded `suppress()` helper. No new runtime dependencies. No `node:` imports. **No serde imports at all** — enforced by a new build script, not by review. -> ### ⛔ BLOCKED on the same cross-phase item as Phase 4b — do not execute Tasks 5–7 yet +> ### ✅ F1 CLOSED — use `suppress()`, not `new SuppressedError(...)` > -> **`SuppressedError` does not exist on the declared runtime floor.** `SSE-29` and `SSE-36` are implemented here -> with `new SuppressedError(...)`, and `engines.node` is `">=18.17"`. `SuppressedError` is a V8 global from the -> full Explicit Resource Management proposal and is absent on every 18.x runtime — Node backported -> `Symbol.dispose`/`Symbol.asyncDispose` on their own, not the error type. Adding `esnext.disposable` to `lib` -> supplies the *type* only, so `new SuppressedError(...)` type-checks, passes `bun test` locally, and then throws -> `ReferenceError: SuppressedError is not defined` under Task 9's `bun run verify:node-floor` / `bun run -> test:node` on the pinned 18.17.0 runner. That is exactly the `NFR-10` trap -> `docs/knowledge/tooling-and-quality-gates.md:60-61` describes. -> -> This is **not 6b's decision to make**: `plans/2026-07-25-phase4b-recovery-chain.md:24-48` already raised it as -> a blocker naming Phases 5a, 6a, 6b and 6c, with two options on the table — raise `engines.node`, or add a -> runtime-guarded `suppress(primary, secondary)` helper in `packages/core/src/`. Whichever lands, lands in all -> five. If the guarded-helper option is chosen, every `new SuppressedError(...)` below becomes -> `suppress(primary, secondary, message)` and the `toBeInstanceOf(SuppressedError)` assertions become assertions -> on that helper's shape. Tasks 1–4 and 8 are unaffected and can proceed. +> Resolved 2026-08-26 in Phase 4b as branch (b): `packages/core/src/suppress.ts` ships +> `suppress(error, suppressed, message)`, which constructs the native `SuppressedError` when +> `globalThis.SuppressedError` exists and a shape-compatible stand-in (`name`, `error`, `suppressed`) when it +> does not. `SuppressedError` reached Node only in **24.0.0** and `engines.node` is `>=20.3`, so the direct form +> neither type-checks (it is not in this package's `lib`) nor runs on the floor. Every `new SuppressedError(...)` +> below becomes `suppress(...)`, and every `toBeInstanceOf(SuppressedError)` becomes an assertion on that shape — +> the `instanceof` form would silently assert nothing on the floor runtime. No decision left to make here. **Prerequisite:** Phases 0 through **5c** implemented as their plans specify. **6a is deliberately *not* a prerequisite** — `SSE-37` (MUST) forbids any serde dependency in core SSE, so this phase imports nothing 6a @@ -2387,7 +2379,7 @@ git commit -m "build: enforce the SSE no-serde and no-reconnect invariants mecha - Create: `packages/core/src/sse/lifecycle.test.ts` - Modify: `packages/core/etc/core.api.md` (regenerated) - Create: `.changeset/phase6b-sse.md` -- Modify: `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` +- Modify: `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` **Interfaces:** - Consumes: everything above. @@ -2555,7 +2547,7 @@ continuity, both of which remain the caller's responsibility. - [ ] **Step 6: Close out the roadmap rows** -In `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`: +In `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`: - Mark the collapsed-disposition row's 6b half satisfied, pointing at the Phase 6b design's "Collapsed Requirements" table (`SSE-18` re-expressed, `SSE-31` re-expressed but **not** collapsed). @@ -2574,7 +2566,7 @@ command's output instead if one does not. - [ ] **Step 8: Commit** ```bash -git add packages/core/src/index.ts packages/core/src/index.public.test.ts packages/core/src/sse/lifecycle.test.ts packages/core/etc/core.api.md .changeset/phase6b-sse.md docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md +git add packages/core/src/index.ts packages/core/src/index.public.test.ts packages/core/src/sse/lifecycle.test.ts packages/core/etc/core.api.md .changeset/phase6b-sse.md docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md git commit -m "feat(core): promote the SSE surface to the public barrel (SSE-23/26/37)" ``` diff --git a/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-checklist.md b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-checklist.md new file mode 100644 index 0000000..b3ae42c --- /dev/null +++ b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-checklist.md @@ -0,0 +1,85 @@ +# Phase 6c — Pagination Implementation Plan — Checklist + +Verification of pagination requirements (`PAGE-1`–`PAGE-36`) from `docs/product-spec/16-pagination.md` and `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md`, as dispositioned by `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`. + +**Status: EXECUTED (2026-08-27).** All tasks implemented, tested, and reviewed. Deviations and design ledger rows are recorded in `docs/work/mvp/2026-09-04-open-items-dissolution.md` §I. + +**Legend:** ✅ Implemented and tested — ✅(t) Satisfied by construction or type test — 🚫 Not built — ⏳ Deferred — N/A Not applicable. + +--- + +## §16.1 — Core Data Model & Strategy Contract (`PAGE-1`–`PAGE-5`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-1 | MUST | Items and pages views over one walk, server order preserved across page boundaries | ✅ | `Page`, `Paginator` in `packages/core/src/pagination/page.ts`, `paginator.ts`, tested in `paginator.test.ts` | +| PAGE-2 | MUST | Materialized items frozen, survive close; items never null | ✅ | `packages/core/src/pagination/page.ts`, tested in `page.test.ts` | +| PAGE-3 | MUST | Exactly one owned response per page; idempotent close delegates to `Response.close()` | ✅ | `packages/core/src/pagination/page.ts`, `Page.close()`, `Page[Symbol.asyncDispose]()`, tested in `page.test.ts` | +| PAGE-4 | MUST | `PageInfo` carries items + nextRequest; `undefined` signals end of stream | ✅ | `packages/core/src/pagination/page.ts` (`pageInfo`), tested in `page.test.ts` | +| PAGE-5 | MUST | `PaginationStrategy.parse` contract returning `Promise<PageInfo<T>>` | ✅ | `packages/core/src/pagination/strategy.ts`, tested in `strategy.test.ts` (ledger row I2) | + +--- + +## §16.2 — Paginator Lifecycle & Consumption Views (`PAGE-6`–`PAGE-15`, `PAGE-27`, `PAGE-31`–`PAGE-33`, `PAGE-36`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-6 | MUST | Page-lazy: zero wire exchanges before first probe | ✅ | `Paginator.#walk`, tested in `paginator.test.ts` | +| PAGE-7 | MUST | Forward-only walk, idempotent end probes | ✅ | `Paginator.#walk`, tested in `paginator.test.ts` | +| PAGE-8 | MUST | Independent iterations over item-level view (`items()`) | ✅ | `Paginator.items()`, tested in `paginator.test.ts` | +| PAGE-9 | MUST | `maxPages` cap: positive integer at construction, stops walk | ✅ | `Paginator`, `paginateWithFetchers`, tested in `paginator.test.ts`, `fetchers.test.ts` | +| PAGE-10 | MUST | Capped walk delivers exactly capped count even when strategy supplies nextRequest | ✅ | `Paginator.#walk`, tested in `paginator.test.ts` | +| PAGE-11 | MUST | Close response BEFORE yielding items on `items()` view | ✅ | `Paginator.items()`, tested in `lifecycle.test.ts` (ledger row I1) | +| PAGE-12 | MUST | Auto-close on abandon / break / exhaustion, scoped construct (`await using`) support | ✅ | `Page[Symbol.asyncDispose]`, `Paginator.#walk`, tested in `page.test.ts`, `lifecycle.test.ts` | +| PAGE-13 | MUST | Parse failure closes inline, close error suppressed | ✅ | `parseOrClose` in `paginator.ts`, tested in `lifecycle.test.ts` | +| PAGE-14 | MUST | Page-level view (`pages()`) is single-use; subsequent iterator throws | ✅ | `Paginator.pages()`, `paginateWithFetchers`, tested in `lifecycle.test.ts`, `fetchers.test.ts` | +| PAGE-15 | MUST | Close errors surface when walk or release fails | ✅ | `releaseHeldOnFailure`, `suppress()`, tested in `lifecycle.test.ts`, `fetchers.test.ts` | +| PAGE-27 | MUST | Every response closed exactly once (no double-close, no leak) | ✅ | `lifecycle.test.ts` (`test.each`), `test/node-conformance/pagination.test.mjs` | +| PAGE-28 | MUST | Underlying causes propagated unwrapped | ✅ | `lifecycle.test.ts`, `errors.test.ts` | +| PAGE-29 | MUST | Async parse boundary (`Promise<PageInfo<T>>`) | ✅(t) | `strategy.ts`, `strategy.test.ts` | +| PAGE-30 | MUST | Synchronous item array within page | ✅(t) | `page.ts`, `strategy.test.ts` | +| PAGE-31 | MUST | Stack safety across thousands of pages (iterative generator drive) | ✅ | `cancellation.test.ts` (5000 pages test) | +| PAGE-32 | MUST | Consumer throw discards return-phase close error, keeping consumer error primary | ✅ | `Paginator.#walk` finally, tested in `lifecycle.test.ts` | +| PAGE-33 | MUST | Race between abort and arrival drops and closes response | ✅ | `Paginator.#walk`, tested in `cancellation.test.ts` | +| PAGE-36 | MUST | Per-operation RequestOptions passed to every page exchange | ✅ | `Paginator.#walk`, tested in `paginator.test.ts` | + +--- + +## §16.3 — Built-in Strategies & Parsing (`PAGE-16`–`PAGE-20`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-16 | MUST | Built-in cursor strategy (single body read, null/empty/undefined ends) | ✅ | `packages/core/src/pagination/strategies.ts` (`cursorStrategy`), tested in `strategies.test.ts` | +| PAGE-17 | MUST | Built-in page number strategy (empty items ends, start page fallback) | ✅ | `packages/core/src/pagination/strategies.ts` (`pageNumberStrategy`), tested in `strategies.test.ts` | +| PAGE-18 | MUST | Built-in link header strategy (RFC 8288, case-insensitive `rel="next"`) | ✅ | `packages/core/src/pagination/strategies.ts`, `link-header.ts`, tested in `strategies.test.ts`, `link-header.test.ts` | +| PAGE-19 | MUST | Unresolvable Link header URL throws | ✅ | `packages/core/src/pagination/strategies.ts`, tested in `strategies.test.ts` | +| PAGE-20 | MUST | Multiple Link headers parsed and combined | ✅ | `packages/core/src/pagination/link-header.ts`, tested in `link-header.test.ts`, `strategies.test.ts` | + +--- + +## §16.4 — Verbatim Query Splicing (`PAGE-21`–`PAGE-24`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-21 | MUST | Verbatim query splice without URLSearchParams; untargeted params preserved byte-for-byte | ✅ | `packages/core/src/pagination/query-splice.ts`, tested in `query-splice.test.ts`, `query-splice.property.test.ts` | +| PAGE-22 | MUST | RFC 3986 percent-encoding in query components (`+` is data, not space) | ✅ | `packages/core/src/http/query-params.ts`, `query-splice.ts`, tested in `query-splice.test.ts` | +| PAGE-23 | MUST | Replace-first, append, remove query parameter maintaining order | ✅ | `packages/core/src/pagination/query-splice.ts`, tested in `query-splice.test.ts`, `query-splice.property.test.ts` | +| PAGE-24 | MUST | Non-query URL components preserved verbatim | ✅ | `packages/core/src/pagination/query-splice.ts`, tested in `query-splice.test.ts` | + +--- + +## §16.5 — Cancellation Integration (`PAGE-25`–`PAGE-26`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-25 | MUST | AbortSignal threaded into every exchange | ✅ | `Paginator.#walk`, tested in `cancellation.test.ts`, `test/node-conformance/pagination.test.mjs` | +| PAGE-26 | MUST | Page-granular cancellation (abort stops walk, in-flight response closed and dropped) | ✅ | `Paginator.#walk`, tested in `cancellation.test.ts`, `test/node-conformance/pagination.test.mjs` | + +--- + +## §16.6 — Fetcher-Based Front-End (`PAGE-34`–`PAGE-35`) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| PAGE-34 | MUST | Fetcher pagination (`first` once, `next` keys off link/token, returns `Page`) | ✅ | `packages/core/src/pagination/fetchers.ts` (`paginateWithFetchers`), tested in `fetchers.test.ts` | +| PAGE-35 | MUST | Mutable shared options bag threaded across fetcher calls | ✅ | `packages/core/src/pagination/fetchers.ts` (`PagingOptions`), tested in `fetchers.test.ts` | diff --git a/docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md similarity index 98% rename from docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md rename to docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md index f57f798..e1aa746 100644 --- a/docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md +++ b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md @@ -5,7 +5,7 @@ **Purpose:** Implement the pagination engine — the `Page` resource, the `PageInfo` strategy contract, the item-level and page-level views, the page cap, the three built-in strategies, the verbatim query splice, and the fetcher-based front-end — satisfying `docs/product-spec/12-pagination.md` (`PAGE-1`–`PAGE-36`). Last of the three -sub-phases the [Phase 6 segmentation design](./2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: 6a +sub-phases the [Phase 6 segmentation design](../2026-07-28-phase6-segmentation-design.md) splits Phase 6 into: 6a (serde, `§14`), 6b (SSE, `§13`), **6c** (this document, pagination). **Governing documents:** `docs/product-spec/12-pagination.md` (normative, cited by ID throughout), @@ -197,7 +197,7 @@ one of them would get an error instead of a walk. So `pages()` mints a fresh sin `items()` is deliberately not single-use at either level: `PAGE-14` scopes the restriction to the page-level view, and `PAGE-8` requires two independent iterations to work. -`PAGE-15`'s two-close-failures case uses native `SuppressedError`, the same mechanism 5a and 6b use — and it +`PAGE-15`'s two-close-failures case uses Phase 4b's guarded `suppress()` helper (never native `SuppressedError`, which reached Node only in 24.0.0 against a `>=20.3` floor), the same mechanism 5a and 6b use — and it applies to the walk's *own* failure paths too, not only to back-to-back close failures. A generator whose body throws and whose `finally` then fails to close the held page would surface the close error and lose the transport or parse failure the caller actually needs. So the drive routine catches, releases with the walk's failure kept @@ -377,7 +377,7 @@ async generator *is* the engine.** | `Transport` | 2 | `PAGE-25`'s transport-agnosticism; `Runtime` (4c) satisfies it, so a resilience pipeline drops in unchanged | | `RequestOptions` | 1 | `PAGE-36` threads the caller's instance; the engine never constructs one | | `FakeTransport`, `countingResponse()` | 5a | Scripted multi-response sequences, wire-send counting, and per-response close observation are exactly what `PAGE-6`/`PAGE-9`/`PAGE-27` need. 5a's design names `countingResponse()`'s `cancel()` hook as the **only** sanctioned way to observe a close — responses are frozen, so a spy assignment throws | -| `SuppressedError` usage pattern | 5a, 6b | `PAGE-13` and `PAGE-15` both need primary-plus-suppressed | +| `suppress()` usage pattern (never native `SuppressedError`) | 4b, 5a, 6b | `PAGE-13` and `PAGE-15` both need primary-plus-suppressed | ## File Layout @@ -442,7 +442,7 @@ implementation details; publishing them would publish a second URL-manipulation - The same iterator-level single-use assertion on `paginateWithFetchers()`' returned view, paired with a `firstCalls === 1` check — an unguarded view re-runs the first-page fetcher and breaks `PAGE-34` outright. - A transport failure surfacing unwrapped (`PAGE-28`), and a transport failure whose held-page release *also* - fails surfacing as `SuppressedError` with the transport failure primary (`PAGE-15`). + fails surfacing as a `suppress()` pairing with the transport failure primary (`PAGE-15`). - The capped fetcher walk asserting the next-page fetcher ran `N-1` times, not `N`, and that every page fetched was also closed — a cap checked at the wrong end of the loop over-fetches by one and leaks the page it refuses to deliver. diff --git a/docs/superpowers/plans/2026-07-28-phase6c-pagination.md b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination.md similarity index 99% rename from docs/superpowers/plans/2026-07-28-phase6c-pagination.md rename to docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination.md index b6af624..720aa5b 100644 --- a/docs/superpowers/plans/2026-07-28-phase6c-pagination.md +++ b/docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination.md @@ -5,7 +5,7 @@ **Goal:** Ship the pagination engine in `@dexpace/core` — `Page`/`PageInfo`, the strategy contract, the item- and page-level views, the page cap, the verbatim query splice, the three built-in strategies, and the fetcher-based front-end — satisfying `product-spec/12-pagination.md` (`PAGE-1`–`PAGE-36`) per -`docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md`. +`docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md`. **Architecture:** A new `packages/core/src/pagination/` folder of eight independent files with no folder-level barrel. Both consumption views are `async function*` generators over one drive routine, which is why `PAGE-6`'s @@ -2845,7 +2845,7 @@ git commit -m "feat(core): add the fetcher-based pagination front-end (PAGE-34/3 - Modify: `packages/core/src/index.public.test.ts` (append) - Modify: `packages/core/etc/core.api.md` (regenerated) - Create: `.changeset/phase6c-pagination.md` -- Modify: `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` +- Modify: `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` - Modify: `docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md` (the `PAGE-11` erratum note) **Interfaces:** @@ -2984,7 +2984,7 @@ rather than re-adding it.) - [ ] **Step 6: Close out the roadmap rows** -In `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md`: +In `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md`: - Mark the `PAGE-11` erratum row **Resolved in Phase 6c**, pointing at the erratum text above and at `lifecycle.test.ts`'s ordering assertion as the mechanical proof. @@ -3003,7 +3003,7 @@ command's output instead if one does not. - [ ] **Step 8: Commit** ```bash -git add packages/core/src/index.ts packages/core/src/index.public.test.ts packages/core/etc/core.api.md .changeset/phase6c-pagination.md docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md +git add packages/core/src/index.ts packages/core/src/index.public.test.ts packages/core/etc/core.api.md .changeset/phase6c-pagination.md docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md docs/sdk-design-nodejs/07-pagination-sse-and-serialization.md git commit -m "feat(core): promote the pagination surface and close Phase 6 (PAGE-1-36)" ``` diff --git a/docs/superpowers/specs/2026-07-28-phase7-segmentation-design.md b/docs/work/mvp/phase7/2026-07-28-phase7-segmentation-design.md similarity index 97% rename from docs/superpowers/specs/2026-07-28-phase7-segmentation-design.md rename to docs/work/mvp/phase7/2026-07-28-phase7-segmentation-design.md index cd9f922..ca879cc 100644 --- a/docs/superpowers/specs/2026-07-28-phase7-segmentation-design.md +++ b/docs/work/mvp/phase7/2026-07-28-phase7-segmentation-design.md @@ -5,8 +5,8 @@ **Purpose:** Record why and how Phase 7 ("Instrumentation & Configuration") splits before either sub-phase gets its own detailed design, mirroring the sizing review that split Phases 3, 4, 5, and 6. This document is the segmentation rationale only; the two sub-phases' full designs are -[7a (Configuration & Platform Primitives)](./2026-07-28-phase7a-configuration-design.md) and -[7b (Instrumentation & Observability)](./2026-07-28-phase7b-observability-design.md). +[7a (Configuration & Platform Primitives)](./phase7a/2026-07-28-phase7a-configuration-design.md) and +[7b (Instrumentation & Observability)](./phase7b/2026-07-28-phase7b-observability-design.md). ## 1. Sizing diff --git a/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md new file mode 100644 index 0000000..a6d07fa --- /dev/null +++ b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md @@ -0,0 +1,109 @@ +# Phase 7a — Configuration & Platform Primitives — Checklist + +**Status: EXECUTED.** Every ✅ below names code and tests that exist on this branch, not a plan step. Verified +against [2026-07-28-phase7a-configuration.md](./2026-07-28-phase7a-configuration.md) and +`docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md`, over every requirement ID in +`docs/product-spec/16-configuration.md` plus appendix C's `RECOV-33` and `NFR-15`. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +Deviations from the plan's own sketches, and everything left open, are recorded in +[`docs/work/mvp/2026-09-04-open-items-dissolution.md`](../../2026-09-04-open-items-dissolution.md) (entries K1–K12). + +## 16.1 Layered lookup + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-1 | MUST | Strict precedence: override → environment (exact key) → property (normalized key) → default | ✅ | `config/configuration.ts` `LayeredConfiguration.getString`; four tests, one per rung, in `configuration.test.ts` "layered precedence". The reference's fourth tier collapses to three — Node has no ambient key/value store distinct from `process.env`, so the *production* property source is empty while the seam stays substitutable (ledgered) | +| CFG-2 | MUST | A present-but-empty environment value is absent; the lookup falls through | ✅ | `getString`'s `fromEnv !== ''` guard; asserted falling through to the property layer *and* to the default | +| CFG-3 | MUST | Property layer queried under the lower-cased, underscore-to-dot key | ✅ | `normalizePropertyKey`; asserted end-to-end (`MAX_RETRY_ATTEMPTS` → `max.retry.attempts`) and at the seam (the injected property source records the key it was handed) | +| CFG-4 | MUST | A raw property accessor by exact name, no normalization | ✅ | `getRawProperty`; both directions asserted — the raw accessor resolves `https.proxyHost`, the normalizing one does not | +| CFG-38 | MUST | Typed accessors resolve through the full layered lookup before parsing | ✅ | `getInt`/`getBoolean`/`getDuration` all call `getString`; each has an env-only test proving it does not read the override map alone | + +## 16.2 Never-throw typed accessors + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-5 | MUST | Integer accessor never throws; default on absent/unparseable; negatives valid | ✅ | `getInt` gates on `STRICT_INTEGER` before `Number`. **Tightened past the plan's sketch:** the sketch used `Number.parseInt`, which resolves `"12abc"` to `12`; a trailing-tail case and a fractional case are both asserted to return the default. **Totality hardened after adversarial review (2026-08-27):** every layer read goes through `readLayer`, so a seam that *throws* (a file- or secrets-store-backed `SourceFn`) and a seam that answers with a **non-string** (any `Record`-backed lookup for a key named `__proto__`/`constructor`/`toString`, `process.env` included) both resolve to "this layer supplies nothing" instead of escaping as a foreign error or a `TypeError`. `-0` is normalized onto `0`. Open item G14 owns surfacing the swallowed seam failure once a `Logger` exists | +| CFG-6 | MUST | Boolean accessor strict: only case-insensitive `true`/`false` | ✅ | `getBoolean`; `1`/`0`/`yes`/`no`/`on`/`off` each asserted to fall through. Shares `readLayer`'s totality guarantee — see `CFG-5` | +| CFG-7 | MUST | Duration grammar: ISO-8601, `<number><unit>` over ms/s/m/h/d, bare number as ms; negative and unknown unit → default | ✅ | `config/duration.ts` — `parseDurationMs` + `parseIsoDuration`/`parseShorthandDuration`, its own module because the grammar is a concept separate from the layered lookup that consumes it (`docs/knowledge/module-organization.md:42`). Grammar cases and a `fast-check` totality property live in `duration.test.ts`; the accessor half — fallback on rejection, layered lookup first — stays in `configuration.test.ts`. `P`/`PT` with no component, and the ambiguous month designator `P5M`, are both rejected rather than silently read as zero or as minutes. The accessor shares `readLayer`'s totality guarantee — see `CFG-5`. A grammatically valid duration larger than any timer can honor is deliberately **not** rejected here (open item G13) | + +## 16.3 Immutability and derivation + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-8 | MUST | Built config immutable and shareable; override map copied at build | ✅ | `ConfigurationBuilder.build` copies the `Map`; `LayeredConfiguration` calls `Object.freeze(this)`. Both asserted — mutate-after-build, and `Object.isFrozen` | +| CFG-9 | MUST | Copy-on-write derive; map copied before the mutator runs; sources inherited by reference | ✅ | `LayeredConfiguration.derive`; three tests — receiver unchanged, sources inherited, a source-replacing mutator detaching only the copy | +| CFG-10 | MUST | Remove drops only the override layer; removing an unset key is a no-op | ✅ | `ConfigurationBuilder.remove`; both cases asserted through `derive` | +| CFG-11 | MUST | Env and property sources are substitutable seams; production delegates to the platform environment | ✅ | `SourceFn`, `withEnvSource`/`withPropertySource`, and `defaultConfiguration()` reading `globalThis.process?.env` with no `node:` import. No unit test touches the real environment except the one that exists to prove the production wiring does, which sets and removes its own uniquely-named key. The production seam reads through `Object.hasOwn`, so `process.env`'s inherited `constructor`/`toString`/`__proto__` cannot answer a lookup with a function; a caller-supplied seam that throws or answers with a non-string is absorbed by `readLayer` rather than propagated (see `CFG-5`, open item G14) | +| CFG-12 | SHOULD | Builders single-threaded only; the guarantee is about the built config | ✅ | Documented on `ConfigurationBuilder`. No enforcement is possible or wanted — a JS builder has no cross-thread reachability to guard | +| CFG-13 | SHOULD | Process-wide slot, last-write-wins, defaults to empty | ✅ | `getGlobalConfiguration`/`setGlobalConfiguration`; the default is asserted against a value captured at module load, so the claim survives any test order | +| CFG-14 | SHOULD | Well-known key constants for retry cap, log level, and the proxy variables | ✅ | `CFG_KEY_MAX_RETRY_ATTEMPTS`, `CFG_KEY_LOG_LEVEL`, `CFG_KEY_HTTP_PROXY`, `CFG_KEY_HTTPS_PROXY`, `CFG_KEY_NO_PROXY`; 7b consumes `CFG_KEY_LOG_LEVEL` for `OBS-35` | +| CFG-37 | MUST | Fail fast on a null/absent required argument; documented-nullable slots exempt | ✅ | `invariant()` on `put` (key and value), `remove`, `withEnvSource`, `withPropertySource`, `derive`, and `setGlobalConfiguration`; a lookup's own `fallback` is asserted to accept `undefined`. No new error class — `InvariantViolation` is the programmer-error signal. `setGlobalConfiguration` checks the *shape* it needs rather than only non-null, matching its siblings: a present-but-wrong `42` no longer reaches the process-wide slot to fail in an unrelated consumer later | + +## 16.4 Clock and async primitives + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-15 | MUST | Injectable three-operation time seam with a shared platform-backed default | ✅ | `config/clock.ts` — `Clock` and the frozen `defaultClock` | +| CFG-16 | MUST | Monotonic non-decreasing, relative-only; wall clock may move backwards and is not for elapsed time | ✅ | `monotonic()` is `performance.now()`, `now()` is `Date.now()`; the split is documented per member and the non-decreasing property is asserted | +| CFG-17 | MUST | `sleep` rejects negative, allows zero, and re-asserts cancellation status before propagating | ✅ | `sleep` rejects a negative *and* a non-finite `durationMs` with `InvariantViolation` — a programmer error takes the project's assertion signal, and it is a *rejection*, never a synchronous throw — resolves at `0` **through a real timer**, and rejects with **the signal's own `reason`** rather than a fresh error — the JS re-expression of "re-assert the interrupt status". The abort path clears the timer and the resolve path detaches the abort listener, so neither outlives the wait; `test:node` counts pending timers via `process.getActiveResourcesInfo()` to assert it rather than relying on the runner hanging. Both the already-aborted and the abort-mid-wait paths are asserted, and `test/node-conformance/config-primitives.test.mjs` re-asserts them on Node's `AbortSignal` (including the default `AbortError` reason Bun cannot verify). **Two corrections from adversarial review (2026-08-27):** zero no longer short-circuits to `Promise.resolve()`, which settled on the microtask queue and let a zero-backoff loop starve timers and I/O entirely (4.1M iterations in 300ms with a pending `setTimeout(fn, 0)` never running); and `durationMs` above `2 ** 31 - 1` is now rejected, because `setTimeout` silently clamps a larger delay to `1` and `sleep(2 ** 31)` returned in 7ms instead of 24.8 days. The ceiling case is a `test:node` case, since the clamp is Node timer behavior. The aborted check precedes the duration path, so cancellation still wins at zero | +| CFG-18 | SHOULD | A separate scheduled non-blocking delay yielding a cancellable future | 🚫 | Collapsed into `Clock.sleep`. Node has no carrier threads to distinguish "block this one" from "schedule that one" against; every timer is already non-blocking, and `CFG-18`'s "cancelling the future cancels the scheduled task" *is* the `clearTimeout` on `sleep`'s abort path. Ledgered | +| CFG-19 | SHOULD | Unwrap async-completion wrapper exceptions to the original throwable | N/A | Vacuous in this runtime: a `Promise` rejection carries the original error directly and there is no wrapper type to unwrap. Ledgered | +| CFG-20 | SHOULD | Interruptible-task future over an executor; cancel-with-interrupt vs. cancel-without | 🚫 | No executor or worker-pool vocabulary in this port; cancellation is `AbortSignal`-based throughout. Ledgered | +| CFG-21 | MUST | A cancelled future's closeable result closed on the discard path, null-safe | N/A | Vacuous for the same reason as `CFG-19`, and for the same reason as `CFG-20` — there is no interruptible-task future for the discard path to belong to. The equivalent obligation for real responses lives in `PIPE-40`, already shipped in Phase 4c. Ledgered | + +## 16.5 Proxy model + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-22 | MUST | Immutable model: type, address, glob list, optional credentials, challenge-handler slot, bypass flag; string form masks credentials | ✅ | `config/proxy.ts` — `ProxyOptions` and `createProxyOptions`, frozen instance and frozen glob list. **The plan's sketch omitted `toString` from the interface and its test faked one with a literal, testing nothing.** As shipped the masking contract is the free `formatProxyOptions(options)`, not a `toString(): string` interface member: every object satisfies such a member through `Object.prototype`, so declaring it would state a contract the type cannot enforce while forcing `Omit`/`Pick` gymnastics through the public API report. `createProxyOptions` additionally attaches an own `toString` that delegates to the same function — one masking implementation — so accidental interpolation into a log cannot leak the password; asserted through `String(options)`, through `formatProxyOptions` directly, and by a test pinning that the two agree. A hand-built literal renders `[object Object]`, which leaks nothing either. `challengeHandler` ships as a typed slot only — ⏳ Phase 8 | +| CFG-23 | MUST | Bypass-all short-circuits; otherwise any-glob match; `*`/`?`, escaped metacharacters, full-string, case-insensitive, compiled once | ✅ | `shouldBypassProxy`, `globMatches`, and a `WeakMap` cache keyed by the pattern array so compilation happens once per list — for a hand-built options object too, without a cache field on the public shape. Each dialect clause has its own test, including the apex-domain negative and a literal `+`. **Reimplemented after adversarial review (2026-08-27):** the `*` → `.*` regex backtracked catastrophically — the operator-supplied `NO_PROXY` entry `*a*a*a*a*a*a*a*a*a*b` against a 60-character host blocked the event loop for 38 seconds — and it escaped `\\` and then re-entered the `*` inside the escape, so `a\\*b` matched neither `a*b` nor `aXXXb`. A two-pointer wildcard walk over lower-cased code points replaces it: one backtrack anchor per `*` rather than a stack, `O(pattern × text)` at worst, 0.02ms on that case, and every regex metacharacter is literal by construction rather than by an escape list. The dialect has **no escape character** — `CFG-23` defines none — so `\\` is an ordinary literal; stated on the function and pinned by a test, alongside a timing guard | +| CFG-24 | MUST | Property tier first (https over http, port from the host's own layer, https-only credentials), then env URL (HTTPS_PROXY over HTTP_PROXY); never throws, invalid config → null + warning | ✅ (null) / ⏳ (warning, Phase 7b) | `resolveFromProperties` then `resolveFromEnvironment`. **Broader than the plan's sketch, which dropped the property tier entirely**: the tier is implemented against the substitutable property seam, so the conformance clauses (same-layer port, https-only credentials) are testable, while the production wiring's empty property seam leaves real behavior environment-only exactly as the design's ledger says. Two `fast-check` totality properties assert it never throws — the second generates **URL-shaped** values, because a bare `fc.string()` essentially never parses and so could not reach the credential decode at all: it missed a `URIError` escaping from `decodeURIComponent` on the lone `%` an un-encoded proxy password legitimately contains (fixed 2026-08-27 by moving the decode inside the `try`). An empty user name means no credentials on **both** tiers, so a blank `https.proxyUser` no longer fabricates a masked `***:***@`, and an IPv6 literal resolves to the same bare address from either tier rather than bracketed from one and bare from the other. **The requirement's "+ warning" half is not emitted** — there is no `Logger` in the package until 7b ships the `OBS-*` seam, so all three rejection paths return `null` silently; open item G10 names 7b as the owner | +| CFG-25 | MUST | Port explicit and within 0..65535; missing/non-numeric/out-of-range → null; no default-port guessing | ✅ | `parsePort` and `explicitPort`; the no-port URL, the out-of-range port, and the missing property port are each asserted to resolve to `null`. **Two corrections from adversarial review (2026-08-27):** the WHATWG parser normalizes a special scheme's default port to `''`, so `HTTP_PROXY=http://proxy:80` and `HTTPS_PROXY=https://proxy:443` — the two most common proxy configurations there are — both resolved to `null`; the requirement bans *guessing* an absent port, not honoring one the operator wrote, so the port is re-read under a non-special probe scheme that the parser leaves verbatim, and only the port comes from that probe. And `parsePort` gated on bare `Number()`, silently accepting `0x10` as port 16, `1e2` as 100, `0b11` as 3, and `80.0`/`+80`; it now requires a bare run of decimal digits, the same strictness `getInt`'s `STRICT_INTEGER` applies | +| CFG-26 | MUST | Property list (pipe) wins over env (comma); backslash escape honored; order is split → drop empty → unescape → trim | ✅ | `splitEscaped` + `parseNonProxyHosts`. **Also absent from the plan's sketch.** Both conformance strings are asserted verbatim (`a\|b\|c` → `[a\|b, c]`, `a\,b,c` → `[a,b, c]`), as is the consequence of trimming last — a whitespace-only fragment survives the drop and lands as an empty token | +| CFG-27 | MUST | A resolved list of exactly one bare `*` is bypass-all (resolution returns null), not a literal entry | ✅ | `resolveNonProxyHosts`; the multi-entry `*` case is asserted to stay an ordinary glob | +| CFG-28 | MAY | A convenience resolver MAY read from the global config; nothing may read proxy config implicitly at startup | ✅ (prohibition) / 🚫 (convenience) | The prohibition holds and is asserted: `resolveProxyOptions` touches the env seam only when invoked, and no module-level code reads it. The optional global-config convenience overload is not built — see open item G4 | + +## 16.6 Dates, identifiers, and value equality + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| CFG-29 | MUST | Canonical RFC 1123 formatting: zero-padded day, literal `GMT`, UTC | ✅ | `config/http-date.ts` `formatHttpDate`; the spec's own example string is asserted byte-for-byte. **Bounded after adversarial review (2026-08-27):** `padStart(4, '0')` cannot render a year outside 0000..9999, so year −1 emitted the malformed `00-1` and year 275760 emitted `275760`, neither of which survived a round-trip — `formatHttpDate` now raises `InvariantViolation` outside that span, a second and narrower bound than `Date`'s own. The `parse(format(x))` property generator was widened from its year-2100 ceiling to the full accepted span, which is why it never caught this | +| CFG-30 | MUST | Tolerant parsing: case-insensitive month, `GMT`/`UTC`/`+0000`/`+00:00`, informational weekday stripped not validated | ✅ | `parseHttpDate`; one test per tolerance clause, plus a contradictory-weekday case and a single-digit day | +| CFG-31 | MUST | Strict from the day-of-month on: blank input fails, a missing post-weekday comma fails | ✅ | Blank, whitespace-only, and comma-less forms all return `null`. Never `Date.parse`. Two additional strictness holes the plan's sketch left open are closed and tested: `Date.UTC` mapping a four-digit year like `0026` onto 1926, and a rolled-over calendar date like `31 Feb` | +| CFG-32 | MUST | Type-4 UUID, RFC 4122 layout, concurrency-safe, non-blocking PRNG, non-cryptographic output | ✅ | `config/identifiers.ts` `randomUuid` over `globalThis.crypto.getRandomValues` — no `node:crypto`. Layout, offsets, a 10 000-item collision batch, and concurrent generation are asserted; the non-cryptographic caveat is on the doc comment. Re-asserted on Node in the conformance suite. **Guarded after adversarial review (2026-08-27):** a runtime without global WebCrypto — a Node release below `engines.node`'s 20.3 floor, or a locked-down embedder — produced the bare `TypeError: Cannot read properties of undefined (reading 'getRandomValues')`, naming neither the SDK nor the floor. An `invariant` now names both, and `randomUuidFrom(webCrypto)` takes the source explicitly so the branch is reachable from a test without deleting a global (`docs/knowledge/testing.md:50`), the same shape `detectRuntimeIdentity(host)` uses | +| CFG-33 | MUST | Deep content equality, arrays element-by-element with recursion, null-safe, hash/equality consistent | ✅ | `config/equality.ts` `deepEqual`/`deepHash`; `fast-check` properties for reflexivity, symmetry, and hash consistency over generated nested trees. **No consumer in-tree yet** — deliberately withheld from the barrel (no requirement gives a caller direct access), so nothing imports it until 5a/5b's settings-validation and collection defensive-copy checks do; in-package consumers import `config/equality.js` directly. **The plan's own sketch test for this was wrong** — it asserted two arrays holding *distinct object literals* were equal, which CFG-33's "non-arrays fall back to ordinary equality" forbids; corrected, and the identity fallback is now asserted in both directions | +| CFG-34 | MUST | `NaN` equals `NaN`, `+0` ≠ `-0`, object array never equal to a primitive array of the same values | ✅ | `Object.is` per element; same module and same no-consumer-yet status as `CFG-33`; the plain-vs-typed and typed-vs-differently-typed splits are both asserted, as is the matching hash behavior for `NaN` and `-0` | +| CFG-35 | SHOULD | One shared classifier: 408, 429, and 5xx except 501/505; this exact set is a hard contract | ✅ (status axis) / ⏳ (throwable axis) | `config/retryable.ts` `RETRYABLE_STATUSES`/`isRetryableStatus`, with a `fast-check` property tying the set and the predicate together and a size assertion pinning the exact cardinality. **Made genuinely immutable after adversarial review (2026-08-27):** the `ReadonlySet` *type* is compile-time only and `Object.freeze` does not seal a `Set`'s internal slots, so `(RETRYABLE_STATUSES as Set<number>).add(418)` succeeded and permanently rewrote the process-wide classifier — a "hard contract" any consumer could edit. The cited precedent did not hold: `http/method.ts`'s `IDEMPOTENT_METHODS` is module-private, while this binding leaves through the package barrel. `add`/`delete`/`clear` are now own properties that throw, with the freeze stopping them being defined back. An audit of every non-primitive the barrel exports found only this one and `defaultClock`, which was already frozen. The requirement's second half — "a throwable is retryable iff it or any cause in its chain is an IO/timeout error, cycle-safe" — is Phase 5a's `classify.ts`, which re-exports this module rather than defining a second copy (open item G5) | +| CFG-36 | SHOULD | Static build/runtime descriptor resolved once; each falls back to a non-blank `unknown`; ordered `[sdk, runtime]` tokens, none blank | ✅ | `config/build-info.ts` `getBuildInfo`, frozen and cached — resolved on first access rather than at import, so the module keeps `sideEffects: false` and takes no import-time clock/host read (`docs/knowledge/module-organization.md:24`); the observable contract, one frozen instance for the process, is unchanged; `detectRuntimeIdentity` takes the host explicitly so the Deno, browser, blank-value, and undetectable branches are all reachable from a test without deleting a global. **Sanitized after adversarial review (2026-08-27):** the detected value is ambient (`process.version`, `Deno.version.deno`, `navigator.userAgent`) and was returned **untrimmed** despite the blank test trimming — `' v20.0.0 '` became `node/ v20.0.0 ` and the `^v` strip silently missed — and never validated, so one non-ASCII byte in a `navigator.userAgent` made the default `clientIdentityStep` reject **every** outbound request with a `HeaderValidationError`. `toUsableToken` now trims, requires non-blank, and requires header-safe, falling back to `unknown` otherwise; a `process.version` of exactly `'v'` no longer yields the versionless `node/`. The printable-ASCII predicate is local to `build-info.ts` rather than imported from `http/ascii-validation.ts`, to avoid adding a second outbound `config/ →` edge to the one open item G11 already tracks; the duplication itself is owned by open item G18 | + +## Cross-cutting + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| NFR-15 | SHOULD | Published artifacts embed self-identifying version metadata resolvable at runtime, never an `unknown` placeholder | ✅ | `packages/core/scripts/gen-version.mjs` writes `src/generated/version.ts` from `package.json`, wired as the `prebuild` step of both the package and root `build` scripts and committed so an unbuilt `bun test` still sees a real version. No runtime `package.json` read, so the browser/Workers builds are covered too. Asserted on Node in the conformance suite | +| RECOV-33 | MUST | Client-identity step: Append after the FIRST existing value preserving all others, Replace overwrites all, blank token line is a no-op emitting no header | ✅ (behavior) / ⏳ (public reachability) | `config/client-identity-step.ts` `clientIdentityStep`, installed at `PRE_REDIRECT`. Append rebuilds the value list with `set`-then-`add` so the header keeps its position among the others (a remove-then-re-add would move it to the end); `getAll` — not `get` — is what the multi-value test asserts, since `get` returns only the first value and would pass under a dropped-values bug. Not exported from the package barrel this phase (open item G1) | + +## Deferred out of Phase 7a + +| Item | Target | Reason | +|---|---|---| +| `ProxyOptions.challengeHandler` has no protocol behind it | Phase 8 (first concrete `Transport`) | The type carries the slot per `CFG-22`'s field list; nothing dispatches through it until a transport owns a proxy connection to be challenged over | +| No concrete `Transport` consumes `ProxyOptions` | Phase 8 | Contract lands with the model, wiring lands with the consumer — the `Serde<T>`-before-`codec-json` precedent | +| Whether `clientIdentityStep` joins `standardResilience()`'s default install list | Phase 9, or a future preset revision | Not installed by default here; no requirement mandates it, and the preset manages pillar slots, not the `PRE_REDIRECT` slot this step occupies | +| `clientIdentityStep` is not on the public barrel | Phase 5c (or whichever phase promotes the pipeline authoring surface) | Its `StepDescriptor` return type is `@internal`; see open item G1 | +| `CFG-35`'s throwable/cause-chain axis | Phase 5a | Belongs with the retry engine's error classification; 5a re-exports this module's status set rather than restating it | + +## Cross-phase retrofits + +The three single-sourcing retrofits into Phase 5a's (written, unexecuted) design and plan were applied as +document edits on 2026-07-28, ahead of this phase's execution — see that plan's amendment banner at +`docs/work/mvp/phase5/phase5a/2026-07-26-phase5a-retry.md:11`. Nothing in 5a's code exists yet, so this phase +changed no 5a source. What 5a must consume when it runs: + +- `RetryConfig.clock: Clock` from `config/clock.js`, replacing the ad hoc `now: () => number`. +- `parseHttpDate` from `config/http-date.js`, replacing `pacing.ts`'s private RFC 1123 parser. +- `RETRYABLE_STATUSES`/`isRetryableStatus` re-exported from `config/retryable.js`, replacing `classify.ts`'s + private copy. diff --git a/docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md similarity index 83% rename from docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md rename to docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md index e21cf6b..9ee86a5 100644 --- a/docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md +++ b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md @@ -7,7 +7,7 @@ dates, UUID generation, deep equality, the retryability classifier, the build/ru client-identity header step — satisfying `docs/product-spec/16-configuration.md` (`CFG-1`–`CFG-38`), `NFR-15` (self-identifying version metadata), and `RECOV-33` (client-identity step, appendix C). This is the first of two sub-phases the roadmap's Phase 7 ("Instrumentation & Configuration") splits into — see the -[segmentation design](./2026-07-28-phase7-segmentation-design.md). 7a leads; 7b (Observability, `§15`) trails and +[segmentation design](../2026-07-28-phase7-segmentation-design.md). 7a leads; 7b (Observability, `§15`) trails and consumes this phase's `Configuration`/`CFG-14` key constant for its log-level resolution (`OBS-35`). **Governing documents:** `docs/product-spec/16-configuration.md` (normative, cited by ID throughout), @@ -141,10 +141,18 @@ interface ProxyOptions { readonly credentials?: { readonly username: string; readonly password: string }; readonly challengeHandler?: unknown; // slot only; no challenge protocol shipped readonly bypassAll: boolean; - toString(): string; // credentials masked } -function shouldBypassProxy(options: ProxyOptions, host: string): boolean; // CFG-23 +// CFG-22's masking contract, as built. `toString(): string` was NOT kept on the interface: every object +// satisfies it via Object.prototype, so declaring it guarantees nothing while forcing Omit/Pick gymnastics +// through the public API. `createProxyOptions` attaches an own `toString` delegating here, so `String(options)` +// still masks for a factory-built instance. +function formatProxyOptions(options: ProxyOptions): string; // CFG-22, credentials masked +function createProxyOptions(init: ProxyOptionsInit): ProxyOptions; +function shouldBypassProxy( + options: Pick<ProxyOptions, 'bypassAll' | 'nonProxyHosts'>, // CFG-23 + host: string, +): boolean; function resolveProxyOptions(config: Configuration): ProxyOptions | null; // CFG-24–CFG-28, never throws ``` @@ -153,11 +161,25 @@ concrete `Transport` exists until Phase 8 (`@dexpace/transport-fetch`/`-undici`) `Serde<T>`-before-`codec-json` precedent exactly: the contract lands here, wiring lands with the first concrete consumer. Explicit scope boundary, recorded so it isn't later mistaken for an omission. -Node has no system-properties layer, so `CFG-24`'s "`https.proxyHost` preferred over `http.proxyHost`" collapses -to environment-only: `HTTPS_PROXY` preferred over `HTTP_PROXY`, parsed as `scheme://user:pass@host:port` — a -second, smaller instance of the same three-tier-to-two-tier collapse `Configuration` itself makes. Glob -compilation (`*` → "any run", `?` → "one char", metacharacter-escaped, full-string, case-insensitive) happens -once at construction and is cached on the `ProxyOptions` instance. +**As-built correction (2026-08-27).** This section originally said `CFG-24`'s "`https.proxyHost` preferred over +`http.proxyHost`" collapses to environment-only. That is true of the *production sources* and false of the +resolution logic, and the shipped code implements the wider reading. `resolveProxyOptions` consults the property +tier first — `https.proxyHost` over `http.proxyHost`, the port taken from the chosen host's own layer, +credentials read only from `https.proxyUser`/`https.proxyPassword` — and falls through to the environment URL +form (`HTTPS_PROXY` preferred over `HTTP_PROXY`, parsed as `scheme://user:pass@host:port`) only when the +property tier yields nothing. All of `CFG-26`'s non-proxy-host resolution (property pipe-list over environment +comma-list, backslash escape, split → drop empty → unescape → trim) is likewise built. It reads the same +substitutable property seam `Configuration` already carries for `CFG-3`/`CFG-4`, and in the default Node wiring +that seam is empty, so real behavior *is* environment-only exactly as the collapse describes. Without the tier, +`CFG-24`'s same-layer-port and https-only-credentials clauses and every clause of `CFG-26` would have been +silent gaps, and `CFG-4`'s `getRawProperty` would have had no consumer in the repository. + +Glob compilation (`*` → "any run", `?` → "one char", metacharacter-escaped, full-string, case-insensitive) +happens once at construction. As built, the compiled patterns are held in a module-level `WeakMap` keyed by the +pattern array itself rather than in a cache field on the `ProxyOptions` instance, which keeps the public shape +free of a field no requirement asks for and makes the caching work for a hand-built options object too. The +trade-off: a hand-built object whose `nonProxyHosts` array is not frozen caches its first compile permanently, +so a pattern pushed on afterwards is silently ignored. ## Dates, identifiers, and equality (`CFG-29`–`CFG-36`) @@ -233,6 +255,7 @@ doesn't manage). ``` packages/core/src/config/ configuration.ts # Configuration, ConfigurationBuilder, global slot, CFG-14 key constants + duration.ts # parseDurationMs -- CFG-7's grammar, split out of configuration.ts at review clock.ts # Clock seam, default implementation proxy.ts # ProxyOptions, shouldBypassProxy, resolveProxyOptions http-date.ts # formatHttpDate, parseHttpDate @@ -263,7 +286,16 @@ every phase since Phase 1), each pointing at its concrete file (e.g. `export {Cl `Configuration`, `ConfigurationBuilder`, `getGlobalConfiguration`/`setGlobalConfiguration`, the `CFG-14` key constants, `Clock`, `ProxyOptions`, `resolveProxyOptions`/`shouldBypassProxy`, `formatHttpDate`/`parseHttpDate`, `randomUuid`, `isRetryableStatus`, `getBuildInfo`, and `clientIdentityStep`/`ClientIdentitySettings` are -promoted this way. `deepEqual`/`deepHash` stay `@internal` (no root re-export) — no requirement calls for +promoted this way. + +**As built, two corrections to that list.** `createProxyOptions`/`ProxyOptionsInit` and `formatProxyOptions` +are promoted alongside `ProxyOptions` — a caller cannot otherwise build one with its `CFG-22` masking, and the +free formatter replaced the interface's `toString()` member (see §"Proxy model"). `clientIdentityStep`/ +`ClientIdentitySettings` are **not** promoted: `StepDescriptor` is `@internal` and api-extractor rejects a +`@public` export returning a forgotten one, so the step ships in-package until the phase that publishes the +pipeline authoring surface lands. Recorded as open item G1. + +`deepEqual`/`deepHash` stay `@internal` (no root re-export) — no requirement calls for callers to compare arbitrary values through the SDK's own API, and 5a's/5b's own equality needs (settings validation, collection defensive-copy checks) can import `config/equality.js` directly within the same package. @@ -300,7 +332,7 @@ reintroduce a second parser without a test noticing the import changed. | `Clock.sleep` is the only wait primitive; no separate blocking-sleep/async-delay pair | `CFG-15`/`CFG-17` (blocking sleep) vs. `CFG-18` (scheduled async delay) as two primitives | Node has no carrier threads to distinguish "block this one" from "schedule that one" against; both are already non-blocking `setTimeout`-backed `Promise`s. Same collapse class as `NFR-11` | | No interruptible-task-future / executor vocabulary (`CFG-20`/`CFG-21`) | JVM `ExecutorService`/`Future` cancel-with-interrupt semantics | No executor/worker-pool concept in this port; `Promise` cancellation is `AbortSignal`-based throughout, already covered by `Clock.sleep`'s signal parameter and 4a's cancellation model | | Async-wrapper unwrapping (`CFG-19`) is vacuous | JVM wraps async-completion exceptions requiring unwrap | `Promise` rejection carries the original error directly; no wrapper type exists in this runtime to unwrap | -| System-property layer collapses into environment-only (proxy and general config alike) | `CFG-1`/`CFG-24`'s four/system-properties-first precedence | Node has no ambient key/value store distinct from `process.env`; already the settled reasoning from `08-instrumentation-and-configuration.md`, restated here for `CFG-*`'s own conformance sweep | +| The *production sources* collapse to environment-only (proxy and general config alike); the property seam and the resolution tier that reads it are both built | `CFG-1`/`CFG-24`'s four/system-properties-first precedence | Node has no ambient key/value store distinct from `process.env`, so `defaultConfiguration()` wires the property seam to a function that always returns `undefined` and real behavior is environment-only — already the settled reasoning from `08-instrumentation-and-configuration.md`. The seam itself, `getRawProperty` (`CFG-4`), `resolveProxyOptions`'s property tier (`CFG-24`), and all of `CFG-26` ARE implemented against it, so every conformance clause is testable; only the production wiring collapses. Narrowed from an earlier, wider wording on 2026-08-27 — see `docs/work/mvp/2026-09-04-open-items-dissolution.md` K2 | | SDK version resolved via build-time codegen, not a runtime `package.json` read | N/A — JVM reads manifest attributes at class-load time | Core's runtime floor includes browsers/Workers with no filesystem; `import.meta.url` tricks are Node/Deno/Bun-only and would leave the browser build with the placeholder `NFR-15` forbids | ## Deferred Items (add to the roadmap's Deferred Items Log) diff --git a/docs/superpowers/plans/2026-07-28-phase7a-configuration.md b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration.md similarity index 97% rename from docs/superpowers/plans/2026-07-28-phase7a-configuration.md rename to docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration.md index 479ce04..4638af8 100644 --- a/docs/superpowers/plans/2026-07-28-phase7a-configuration.md +++ b/docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration.md @@ -1,14 +1,28 @@ # Phase 7a — Configuration & Platform Primitives Implementation Plan +> **EXECUTED — do not follow these sketches verbatim (banner added 2026-08-27).** This phase has shipped. +> The task sketches below are the pre-execution draft, and six of them are known to be wrong: Task 2's +> RFC 1123 parser (`Date.UTC` maps a four-digit year under 100 onto 1900-1999, and it does not reject a +> rolled-over calendar date such as `31 Feb`), Task 5's hash-consistency test (asserts two arrays of +> *distinct object literals* are equal, which `CFG-33`'s "non-arrays fall back to ordinary equality" +> forbids), Task 6's `getInt` (`Number.parseInt` resolves `"12abc"` to `12`, which `CFG-5` calls +> unparseable), Task 7's `CFG-22` test (asserts a hard-coded literal returned by a fake `toString`, testing +> nothing) and its omission of `CFG-26` entirely, and Task 8 Step 6 (rewrites `build` to `tsc -b`, replacing +> the working `tsc -p tsconfig.build.json`). All six were corrected in the shipped code and are itemized in +> [`docs/work/mvp/2026-09-04-open-items-dissolution.md`](../../2026-09-04-open-items-dissolution.md) K6. **The as-built record is +> [the checklist](./2026-07-28-phase7a-configuration-checklist.md) and the code, not this file.** The +> sketches are left in place deliberately, as the historical artifact of a completed phase. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Ship the layered `Configuration` model, the `Clock` seam, the proxy model, RFC 1123 dates, UUID generation, deep equality, the retryability classifier, the build/runtime version descriptor, and the client-identity header step in `@dexpace/core` — satisfying `docs/product-spec/16-configuration.md` (`CFG-1`–`CFG-38`), `NFR-15`, and appendix C's `RECOV-33`, per -`docs/superpowers/specs/2026-07-28-phase7a-configuration-design.md`. +`docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-design.md`. -**Architecture:** A new `packages/core/src/config/` folder of nine independent files, no folder-level barrel +**Architecture:** A new `packages/core/src/config/` folder of nine independent files (**ten as built** — see +the file-structure note below), no folder-level barrel (every public symbol re-exports from the existing package-root `packages/core/src/index.ts` instead). Every file is pure — no I/O beyond the injected env/property source functions `Configuration` accepts, and no `node:` imports anywhere. A build-time codegen script (`scripts/gen-version.mjs`) writes @@ -96,7 +110,9 @@ packages/core/scripts/ Every file has a colocated `*.test.ts` except `generated/version.ts` (generated, not hand-written) and `scripts/gen-version.mjs` (exercised indirectly by Task 8's own test asserting the generated output shape). -Nine production files, each one responsibility, none over ~100 lines. +Nine production files, each one responsibility, none over ~100 lines. **As built: ten** -- Pass 1 of the +review split `CFG-7`'s duration grammar out of `configuration.ts` into `config/duration.ts` on the +one-concept-per-file rule (`docs/knowledge/module-organization.md:42`). See the banner at the top of this file. --- @@ -1697,7 +1713,7 @@ git commit -m "feat(core): client-identity header step (RECOV-33), closes NFR-15 **Files:** - Modify: `packages/core/src/index.ts` - Verify: `packages/core/etc/core.api.md` -- Create: `docs/superpowers/plans/2026-07-28-phase7a-configuration-checklist.md` +- Create: `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md` **Interfaces:** - Consumes: every public symbol from Tasks 1–9. @@ -1762,7 +1778,7 @@ Expected: every gate PASS. - [ ] **Step 5: Write the requirement checklist** -Create `docs/superpowers/plans/2026-07-28-phase7a-configuration-checklist.md`, same `| ID | Level | +Create `docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md`, same `| ID | Level | Requirement gist | Status | Where |` table format as prior phase checklists (e.g. `2026-07-24-phase3a-io-contracts-checklist.md`), legend ✅ shipped / 🚫 never built / ⏳ deferred / N/A. @@ -1791,7 +1807,7 @@ State explicitly at the top whether the plan has been executed. ```bash git add packages/core/src/index.ts packages/core/etc/core.api.md \ - docs/superpowers/plans/2026-07-28-phase7a-configuration-checklist.md + docs/work/mvp/phase7/phase7a/2026-07-28-phase7a-configuration-checklist.md git commit -m "feat(core): promote Phase 7a's public surface; checklist" ``` diff --git a/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md new file mode 100644 index 0000000..1949635 --- /dev/null +++ b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md @@ -0,0 +1,98 @@ +# Phase 7b — Instrumentation & Observability — Checklist + +**Status: EXECUTED.** Every ✅ below names code and tests that exist on this branch, not a plan step. Verified +against `docs/product-spec/15-instrumentation-and-observability.md` over every requirement ID (`OBS-1` through `OBS-40`), +plus Task 1 through Task 10 deliverables, package builds, and API reports. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +## 15.1 Structured Logging Facade + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-1 | MUST | Zero-overhead no-op default; disabled levels allocate nothing and produce no output | ✅ | `observability/logger.ts`: `NOOP_LOGGER`, `NOOP_EVENT`, `createLogger` short-circuits disabled levels before building event; asserted in `logger.test.ts` | +| OBS-2 | MUST | Four severity levels (`error`, `warning`, `info`, `verbose`) | ✅ | `observability/logger.ts`: `LogLevel` union type and facade level mappings; asserted in `logger.test.ts` | +| OBS-3 | MUST | Fluent `LogEvent` builder; rejected empty keys; null rendered as string "null" | ✅ | `observability/logger.ts`: `RealLogEvent.field`, non-empty key invariant, `renderField` converts null/undefined to "null"; asserted in `logger.test.ts` | +| OBS-4 | MUST | Reserved "event" tag key exclusively managed via `.event()`; empty string clears tag | ✅ | `observability/logger.ts`: `RealLogEvent.event` sets `eventTag` or clears; `.field("event", ...)` guarded and throttled; asserted in `logger.test.ts` | +| OBS-5 | MUST | Precedence: per-event fields > global context > diagnostic context | ✅ | `observability/logger.ts`: folded in order diagnosticContext → globalFields → per-event fields; asserted in `logger.test.ts` | +| OBS-6 | MUST | Total field rendering; never throws; numbers/booleans/bigints type-preserving; hostile `toString`/`toPrimitive` guarded | ✅ | `observability/logger.ts`: `renderField`, `renderNonPrimitive`, `renderScalar`, `fast-check` property tests in `logger.test.ts` | +| OBS-7 | SHOULD | String field values capped at 8 KiB with `…[truncated]` marker; UTF-16 surrogate pair boundary safe | ✅ | `observability/logger.ts`: `truncate()` at 8192 chars avoiding surrogate splitting; primitives exempt; asserted in `logger.test.ts` | +| OBS-8 | MUST | Single-emission guarantee per `LogEvent`; subsequent `.emit()` calls are no-ops | ✅ | `observability/logger.ts`: `RealLogEvent.emitted` boolean flag; asserted in `logger.test.ts` | +| OBS-9 | MUST | `Logger.withContext` adds immutable context to all derived loggers | ✅ | `observability/logger.ts`: `withContext` returns new logger with merged `globalFields`; asserted in `logger.test.ts` | +| OBS-40 | SHOULD | Collision warning for `.field("event", ...)` throttled to at most once per logger at verbose | ✅ | `observability/logger.ts`: `CollisionWarningGate` emits at most once at `verbose`; asserted in `logger.test.ts` | + +## 15.2 Diagnostic Context Allow-List + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-10 | MUST | AsyncLocalStorage diagnostic context with default allow-list (`trace.id`, `span.id`); null allow-list folds all; null values skipped | ✅ | `observability/diagnostic-context.ts`: `withDiagnosticFields`, `getDiagnosticContext`, `DEFAULT_DIAGNOSTIC_ALLOW_LIST`, prototype-safe mapping; asserted in `diagnostic-context.test.ts` | + +## 15.3 Redaction Policy + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-11 | MUST | Userinfo in URLs is always scrubbed to `***:***@` | ✅ | `observability/redaction.ts`: `redactUrl` replaces user/password; asserted in `redaction.test.ts` | +| OBS-12 | MUST | Query parameters scrubbed unless allow-listed; default allow-list is `{'api-version'}`; names and encoding preserved | ✅ | `observability/redaction.ts`: `redactQueryString` with case-insensitive name matching; asserted in `redaction.test.ts` | +| OBS-13 | MUST | Fragment tokens in `key=value` format scrubbed; plain text fragments preserved | ✅ | `observability/redaction.ts`: `redactFragment` matches `k=v` pairs; asserted in `redaction.test.ts` | +| OBS-14 | MUST | Scheme, host, port, path untouched; no spurious `?` added; `?` in fragment not confused with query delimiter | ✅ | `observability/redaction.ts`: structured URL parsing and delimiter preservation; asserted in `redaction.test.ts` | +| OBS-15 | MUST | Total URL redaction: malformed URLs redact to `[malformed url]`, never throws | ✅ | `observability/redaction.ts`: try-catch fallback, fast-check property-tested across all strings in `redaction.test.ts` | +| OBS-16 | MUST | Header values containing URLs are redacted (absolute as URL, relative keeps path + `?***`) | ✅ | `observability/redaction.ts`: `redactAbsoluteOrRelativeUrl`; asserted in `redaction.test.ts` | +| OBS-17 | MUST | Sensitive headers scrubbed; Location and Content-Location redacted as URLs | ✅ | `observability/redaction.ts`: `redactHeaderValue` with default-deny allow-list and URL detection; asserted in `redaction.test.ts` | +| OBS-18 | MUST | Configurable dropped header policy (`mark` with `REDACTED` vs `omit` dropping header) | ✅ | `observability/redaction.ts`: `DroppedHeaderPolicy` support in `redactHeaderValue`; asserted in `redaction.test.ts` | +| OBS-19 | SHOULD | Dropped-header verbosity policy (WARN-every / WARN-first-per-name / verbose-only) for transport encoding drops | ⏳ | **Phase 8a**: Requires concrete `fetch` transport capable of detecting unencodable headers | + +## 15.4 Failure Containment + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-20 | MUST | Failure containment: logger exceptions and body-drain errors caught and surfaced as `http.instrumentation.logFailure` at verbose; tracer and meter exceptions propagate | ✅ | `observability/logging-step.ts` (`safeEmit`, `captureResponseBody`), `retry/engine.ts`, `redirect/redirect-step.ts`; asserted in `logging-step.test.ts` | + +## 15.5 Tracing & Context (W3C) + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-21 | MUST | Tracing SPI: `Span`, `Tracer`, `Scope`, `NOOP_SPAN`, `NOOP_TRACER`; non-recording span inert with idempotent `end()` | ✅ | `observability/tracing.ts`: `Span`, `Tracer`, `NOOP_SPAN`, `NOOP_TRACER`; asserted in `tracing.test.ts` | +| OBS-22 | MUST | `activateSpan` sets ambient span and restores previous on scope close, including on exception | ✅ | `observability/tracing.ts`: `activateSpan` using `createAsyncScopedStore`; asserted in `tracing.test.ts` | +| OBS-23 | MUST | `activateSpanForCorrelation` binds `trace.id` / `span.id` into diagnostic context for recording spans via `spanContext()` | ✅ | `observability/tracing.ts`: updates MDC and active span scope; asserted in `tracing.test.ts` | +| OBS-24 | MUST | Thread-local diagnostic context bridged across async boundaries via immutable snapshot (`withDiagnosticFields`) | ✅ | `observability/diagnostic-context.ts`: `withDiagnosticFields`, `pushDiagnosticFields`; asserted in `diagnostic-context.test.ts` | +| OBS-25 | MUST | Allocation-free no-op defaults used when tracing is disabled | ✅ | `observability/tracing.ts`: `NOOP_TRACER`, `NOOP_SPAN`; asserted in `tracing.test.ts` | +| OBS-26 | MUST | W3C-compliant identifiers: 32-hex trace id, 16-hex span id, flags, state, invalid all-zero sentinels | ✅ | `observability/tracing.ts`: `generateTraceId`, `generateSpanId`, `SpanContext`; asserted in `tracing.test.ts` | +| OBS-27 | MUST | Trace-id generation: W3C, Datadog (64-bit decimal), no-op sentinels; all-zero draws coerced non-zero | ✅ | `observability/tracing.ts`: `generateTraceId`, `createInstrumentationBundle`; asserted in `tracing.test.ts` | +| OBS-28 | SHOULD | Richer HTTP-tracer vocabulary (operation, per-attempt, and transport milestones) | ⏳ | **Phase 8a**: Interface + transport milestones wired with `fetch` transport | +| OBS-29 | MUST | HTTP-tracer lifecycle ordering contract (operationStarted, per-attempt, retries-exhausted, operationFailed/Succeeded) | ⏳ | **Phase 8a / Phase 9**: Lifecycle ordering verification with transport adapter | +| OBS-30 | MUST | Tracer and metrics callbacks must not throw; throwing tracer/meter propagates | ✅ | `observability/tracing.ts`, `observability/metrics.ts`, `observability/logging-step.ts`; asserted in `logging-step.test.ts` | + +## 15.6 Metrics SPI + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-31 | MUST | Metrics SPI: `Counter`, `Histogram`, `Meter`, `NOOP_METER`; zero external dependencies | ✅ | `observability/metrics.ts`: `NOOP_METER`, `NOOP_COUNTER`, `NOOP_HISTOGRAM`; asserted in `metrics.test.ts` | +| OBS-32 | SHOULD | Semconv naming (`http.client.request.count`, `http.client.request.duration`) with method, status, errorType attributes | ✅ | `observability/logging-step.ts`: counter and histogram tagged with method and status/errorType; asserted in `logging-step.test.ts` | +| OBS-33 | MUST | Counter documents non-negative increments; Histogram tolerates all inputs without throwing | ✅ | `observability/metrics.ts`: `Counter`, `Histogram`, `NOOP_HISTOGRAM`; asserted in `metrics.test.ts` | + +## 15.7 LOGGING Pillar Step & Event Vocabulary + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| OBS-34 | MUST | HTTP logging granularity (`none`, `headers`, `body`); span and metrics run independently of log level | ✅ | `observability/logging-step.ts`: `LoggingGranularity`, spans and metrics recorded under all granularities; asserted in `logging-step.test.ts` | +| OBS-35 | SHOULD | Ambient granularity resolution from Configuration `CFG_KEY_LOG_LEVEL` | ✅ | `observability/logging-step.ts`: `resolveGranularity` reads configuration key; asserted in `logging-step.test.ts` | +| OBS-36 | MUST | Request/response body preview capture bounded to preview size (default 8 KiB), non-buffering, size field emitted | ✅ | `observability/logging-step.ts`: `prepareRequestBody`, `captureResponseBody`, `http.request.body.size`, `http.response.body.size`; asserted in `logging-step.test.ts` | +| OBS-37 | SHOULD | Unknown-length response body skips capture; live stream delivered untouched | ✅ | `observability/logging-step.ts`: `captureResponseBody` skips when `content-length` missing; asserted in `logging-step.test.ts` | +| OBS-38 | SHOULD | Text bodies decoded safely with charset fallback; binary bodies rendered as `[binary N bytes captured]` | ✅ | `observability/logging-step.ts`: `isTextMediaType`, `decodeBodyText`, replacement on truncated multi-byte; asserted in `logging-step.test.ts` | +| OBS-39 | MUST | Structured event names `http.request` and `http.response` with standard field hierarchy and redacted `url.full` | ✅ | `observability/logging-step.ts`: `emitRequestEvent`, `emitResponseEvent`, `emitFailureEvent`; asserted in `logging-step.test.ts` | + +## 15.8 Adapter Packages + +| Package | Purpose | Status | Where | +|---|---|---|---| +| `@dexpace/logging-pino` | Pino logging adapter wrapping `pino` instance | ✅ | `packages/logging-pino/src/pino-logger.ts` | +| `@dexpace/logging-debug` | Debug logging adapter wrapping `debug` factory or debugger | ✅ | `packages/logging-debug/src/debug-logger.ts` | + +## 15.9 Subsystem Retrofits + +| Subsystem | Events Emitted | Status | Where | +|---|---|---|---| +| Retry Engine | `http.retry.delayOverrideFailed`, `http.retry.attemptFailed`, `http.retry.exhausted` (contained per OBS-20) | ✅ | `packages/core/src/retry/engine.ts` | +| Redirect Step | `http.redirect.hop`, `http.redirect.downgradePermitted`, `http.redirect.rejected` (contained per OBS-20) | ✅ | `packages/core/src/redirect/redirect-step.ts` | +| Standard Resilience Preset | `loggingStep` installed into `standardResilience` pipeline with options pass-through | ✅ | `packages/core/src/auth/preset.ts` | diff --git a/docs/superpowers/specs/2026-07-28-phase7b-observability-design.md b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-28-phase7b-observability-design.md rename to docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md index d29b3c0..2c33449 100644 --- a/docs/superpowers/specs/2026-07-28-phase7b-observability-design.md +++ b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md @@ -7,7 +7,7 @@ the redaction policy, tracing (`Tracer`/`Span`, real W3C trace-context generatio `LOGGING` pillar step — satisfying `docs/product-spec/15-instrumentation-and-observability.md`'s `OBS-1`–`OBS-27` and `OBS-30`–`OBS-40` (`OBS-19`, `OBS-28`, and `OBS-29` are deferred to Phase 8a by name — see Scope). This is the second of two sub-phases the roadmap's Phase 7 splits into — see the -[segmentation design](./2026-07-28-phase7-segmentation-design.md). 7b trails 7a and consumes its `Configuration` +[segmentation design](../2026-07-28-phase7-segmentation-design.md). 7b trails 7a and consumes its `Configuration` (`OBS-35`'s log-level resolution) and `CFG-14`'s log-level key constant. **Governing documents:** `docs/product-spec/15-instrumentation-and-observability.md` (normative, cited by ID diff --git a/docs/superpowers/plans/2026-07-28-phase7b-observability.md b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md similarity index 99% rename from docs/superpowers/plans/2026-07-28-phase7b-observability.md rename to docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md index 6dbed3d..f09d64f 100644 --- a/docs/superpowers/plans/2026-07-28-phase7b-observability.md +++ b/docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability.md @@ -7,7 +7,7 @@ redaction policy, tracing (`Tracer`/`Span`, real W3C trace-context generation), `LOGGING` pillar step in `@dexpace/core`, plus the `@dexpace/logging-pino` and `@dexpace/logging-debug` bridge packages — satisfying `docs/product-spec/15-instrumentation-and-observability.md`'s `OBS-1`–`OBS-18` and `OBS-20`–`OBS-27`, `OBS-30`–`OBS-40` (`OBS-19`/`OBS-28`/`OBS-29` → Phase 8a by name), per -`docs/superpowers/specs/2026-07-28-phase7b-observability-design.md`. +`docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md`. **Architecture:** A new `packages/core/src/observability/` folder of six files, no folder-level barrel. `diagnostic-context.ts` (no dependencies within this package) is built first; `logger.ts` — the facade, a @@ -2660,7 +2660,7 @@ git commit -m "feat(core): retry/redirect structured logging and the preset's LO **Files:** - Modify: `packages/core/src/index.ts` - Verify: `packages/core/etc/core.api.md` -- Create: `docs/superpowers/plans/2026-07-28-phase7b-observability-checklist.md` +- Create: `docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md` **Interfaces:** - Consumes: every public symbol from Tasks 1–6. @@ -2738,7 +2738,7 @@ git diff --exit-code packages/*/etc/*.api.md # an unreviewed API drift fails h - [ ] **Step 5: Write the requirement checklist** -Create `docs/superpowers/plans/2026-07-28-phase7b-observability-checklist.md`, same table format as prior +Create `docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md`, same table format as prior phase checklists, legend ✅ shipped / 🚫 never built / ⏳ deferred / N/A. Sections and their sources: @@ -2785,7 +2785,7 @@ State explicitly at the top whether the plan has been executed. ```bash git add packages/core/src/index.ts packages/core/etc/core.api.md \ - docs/superpowers/plans/2026-07-28-phase7b-observability-checklist.md + docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-checklist.md git commit -m "feat(core): promote Phase 7b's public surface; checklist" ``` diff --git a/docs/superpowers/specs/2026-07-28-phase8-segmentation-design.md b/docs/work/mvp/phase8/2026-07-28-phase8-segmentation-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-28-phase8-segmentation-design.md rename to docs/work/mvp/phase8/2026-07-28-phase8-segmentation-design.md index 26c7d64..9a49e6f 100644 --- a/docs/superpowers/specs/2026-07-28-phase8-segmentation-design.md +++ b/docs/work/mvp/phase8/2026-07-28-phase8-segmentation-design.md @@ -142,7 +142,7 @@ does not collapse (§5.2) plus `SSE-41` (§6), the reactive SSE adapter with bac (`ASYNC-21`), fatal/non-fatal error-family split, and documented source ownership. Reuses, does not rebuild: 7b's `AsyncLocalStorage`-backed diagnostic-context bridge -(`docs/superpowers/specs/2026-07-28-phase7b-observability-design.md` lines 151–169) for `ASYNC-8`–`ASYNC-12`'s +(`docs/work/mvp/phase7/phase7b/2026-07-28-phase7b-observability-design.md` lines 151–169) for `ASYNC-8`–`ASYNC-12`'s logging-context propagation — 7b's design already states `AsyncLocalStorage` auto-propagates across `await`, promise chains, and timers, covering "most of what `OBS-24`'s bridge... manually requires," with an explicit `captureDiagnosticSnapshot()`/`runWithSnapshot()` escape hatch already built for exactly the residual case diff --git a/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md new file mode 100644 index 0000000..27c68fd --- /dev/null +++ b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md @@ -0,0 +1,80 @@ +# Phase 8a — Transport Adapters — Checklist + +**Status: EXECUTED.** Every ✅ below names code and tests that exist on this branch, not a plan step. Verified +against `docs/product-spec/17-transport-adapter-conformance-contract.md` over every requirement ID +(`TRANSPORT-1` through `TRANSPORT-30`), plus the `SEAM`/`NFR` rows the roadmap parks on this phase. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +Paths are relative to the repo root. `run-suite.ts` means +`packages/transport-conformance/src/run-suite.ts`, the single suite both transports run through their own +`*.conformance.test.ts`, so no row below is proven for one transport and assumed for the other. + +## 17.1 Pipeline authority + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| TRANSPORT-1 | MUST | Native redirect following disabled; default off | ✅ | `fetch-transport.ts` pins `redirect: 'manual'`, `undici-transport.ts` pins `maxRedirections: 0` even behind a BYO dispatcher; asserted in `run-suite.ts` ("a 302 is returned raw") and per package in `fetch-transport.test.ts` / `undici-transport.test.ts` | +| TRANSPORT-2 | MUST | Native automatic retry disabled | ✅ | Satisfied by construction, not by a flag: `fetch` has no automatic-retry feature to disable, and `undici-transport.ts` composes a plain `Agent`/`ProxyAgent` — never a `RetryAgent` and never a retry interceptor. The only path to a retrying dispatcher is a caller supplying one as `dispatcher`, which is their own decision about their own client (SEAM-14). No standalone assertion: there is no observable knob to read back, and a test asserting "we did not import `RetryAgent`" would be a tautology over the import list | + +## 17.2 Cancellation and timeout classification + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| TRANSPORT-3 | MUST | Caller cancellation is terminal, never the retryable type; discriminated out-of-band | ✅ | `transport-shared/src/abort-mapping.ts` `abortToSdkError` branches on `isTimeoutSignal` (a structured `reason.name` check, not a message match); asserted in `abort-mapping.test.ts` and `run-suite.ts` | +| TRANSPORT-4 | MUST | Read/response timeout is a RETRYABLE transport failure, cancellation flag clear | ✅ | Same mapping returns `TransportFailureError` (an `IoError` subtype) for a timeout signal; asserted in `run-suite.ts` and `test/node-conformance/transport.test.mjs` | +| TRANSPORT-5 | MUST | Per-call timeout applies to that call only | ✅ | `composeSignal(signal, options?.timeoutMs ?? defaultTimeoutMs)` per send, never on the instance; asserted in `run-suite.ts` ("two concurrent calls are each bounded by their own timeout") | +| TRANSPORT-6 | SHOULD | A sub-resolution positive timeout is not truncated to zero | ✅ | N/A in mechanism — `AbortSignal.timeout(ms)` is already millisecond-resolution with no zero-means-no-timeout coercion — but asserted anyway in `run-suite.ts` ("a sub-resolution 1ms timeout still times out rather than hanging") so a future coarser implementation cannot regress it silently | +| TRANSPORT-7 | MUST | Cancelling in flight propagates into the native exchange and releases it | ✅ | The composed signal is forwarded to `fetch`/undici through `forkSignal`; asserted in `run-suite.ts` ("a cancelled exchange leaves no handle that stalls close()") | +| TRANSPORT-8 | MUST | A native-internal cancel completes terminal while a timeout on the same path stays retryable | ✅ (undici) / N/A (fetch) | `undici-transport.ts` maps `UND_ERR_DESTROYED`/`UND_ERR_ABORTED`/`UND_ERR_CLOSED` to `CancellationError`; asserted in `undici-transport.test.ts` (destroying the dispatcher mid-flight) with the timeout twin alongside it, and gated in `run-suite.ts` on `supportsInternalCancel`. `fetch` has no internal-cancel path distinct from an abort — the requirement's own text scopes it out | +| TRANSPORT-9 | MUST | An adaptation-race response is still closed | ✅ | Both transports re-check the composed signal after dispatch and cancel/`dump()` the native body before rejecting; asserted in `run-suite.ts` ("a timeout while headers are still pending releases the connection") | + +## 17.3 Header and body mapping + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| TRANSPORT-10 | MUST | Caller Content-Type authoritative; body-derived emitted only when none set | ✅ | `transport-shared/src/header-mapping.ts` `mapOutboundHeaders` checks `headers.get('content-type')` before stamping; asserted in `header-mapping.test.ts` (both directions) and end to end in `run-suite.ts` | +| TRANSPORT-11 | MUST | Framing headers dropped before dispatch, each drop logged at verbose | ✅ | `FETCH_FORBIDDEN_HEADERS` (adds `connection`) / `UNDICI_FORBIDDEN_HEADERS` (does not — §17's own note); drops routed through `createDropLogger`. Asserted in `header-mapping.test.ts`, per package in each transport's unit test, and through the drop log itself in `run-suite.ts` | +| TRANSPORT-12 | MUST | A wire-invalid header degrades to a drop, never a failed send | ✅ | `mapOutboundHeaders` catches per header; `fetch-transport.ts` additionally catches `Headers.append` rejections. Asserted in `header-mapping.test.ts` | +| TRANSPORT-13 | SHOULD | Configurable drop-log policy; case-insensitive, bounded dedup | ✅ | `transport-shared/src/drop-log.ts`: `'all' \| 'first-per-name' \| 'quiet'`, lower-cased keys, drain-to-cap at `MAX_LOGGED_DROP_NAMES`; asserted in `drop-log.test.ts` including the synthesised-name burst | +| TRANSPORT-14 | MUST | Lenient inbound copy; control-byte header dropped, obs-text value preserved | ✅ | `degradeInboundHeaders` writes through `Headers`'s lenient `addInbound` path; asserted in `header-mapping.test.ts`. **Not** asserted end to end: both native HTTP parsers reject a control byte in a header value at the wire (`Malformed_HTTP_Response`) before a transport ever sees it, so the fixture that would drive it is unbuildable — the degrade path is a real defence for hostile/synthetic responses and is tested at its source | + +## 17.4 Lifecycle and ownership + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| TRANSPORT-15 | MUST | Ownership-aware close; a BYO client is never shut down | ✅ | `undici-transport.ts` `selectDispatchers` returns an `owned` list that is empty for a BYO dispatcher; `close()` iterates only that list, in reverse acquisition order, and includes a transport-constructed `ProxyAgent`. Asserted in `undici-transport.test.ts`. `fetch`'s `close()` is a sanctioned no-op over a runtime global it does not own | +| TRANSPORT-16 | MUST | `close()` idempotent, non-blocking, interrupt-safe | ✅ | `#closing` memoizes one teardown so concurrent calls share it. undici's dispatchers are `destroy()`ed, not gracefully `close()`d, precisely because of the "no unbounded await" clause — a graceful close waits out every enqueued request; asserted in `undici-transport.test.ts` ("close does not wait out an in-flight request"). Idempotency asserted in `run-suite.ts` and both unit tests. Both transports are also `AsyncDisposable`, so `await using` is a single teardown path | +| TRANSPORT-17 | MUST | A single-use body is written to the wire exactly once | ✅ | `transport-shared/src/body-pump.ts` runs `writeTo` once per send and neither transport re-invokes it; asserted in `body-pump.test.ts`, `run-suite.ts` (a counting body whose bytes are read back off the wire), and `test/node-conformance/transport.test.mjs` | +| TRANSPORT-18 | MUST | Re-subscribable producer replays identical bytes | 🚫 | Deviation Ledger: neither `fetch` nor undici drives writes through a re-subscribable producer, so there is no native internal resend to make idempotent. The SDK's own retry layer (5a) re-invokes `send()` against the original `Body`, already gated by `RETRY-5`/`RETRY-7`'s replayability check | +| TRANSPORT-19 | SHOULD | An abandoned streaming subscription unblocks its producer; teardown idempotent | ✅ | `BodyPump.abandon` aborts the writer and awaits the producer's unwind; both transports call it on every non-delivering exit path, the adaptation-throw path included. Separately, both hold a handler on the producer's settlement through `producerFailure` for the whole send — without one, a producer that fails *after* the response was delivered (an early `413`, say) reaches the runtime's default `unhandledRejection` policy and takes the process down. Asserted in `body-pump.test.ts` (a producer parked on backpressure forever is released, twice over), `fetch-transport.test.ts` (a producer failure races the pending fetch and fails the send), and — for both transports — `run-suite.ts` ("a producer that fails after delivery does not escape as an unhandled rejection", driven by the `/early-response` fixture) | + +## 17.5 Failure and response mapping + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| TRANSPORT-20 | MUST | A no-response failure is the canonical retryable I/O subtype | ✅ | `packages/core/src/io/errors.ts` `TransportFailureError extends IoError`, so 5a's `classify.ts` cause-walk already treats it as always-retryable; asserted in `errors.test.ts`, `run-suite.ts`, and the Node layer. The subtyping costs a third hierarchy level against the styleguide's two-level cap — Deviation Ledger row 17. The converse is enforced too: `undici-transport.ts` `toDispatchError` keeps undici's argument-validation codes (`UND_ERR_INVALID_ARG`, `UND_ERR_NOT_SUPPORTED`) *outside* the `IoError` tree, because `classify.ts` is an allow-list and a permanent misconfiguration classified as retryable would spend a caller's whole budget re-proving itself. Asserted in `undici-transport.test.ts` with its retryable twin alongside | +| TRANSPORT-21 | MUST | A pre-dispatch failure arrives through the promise, never a synchronous throw | ✅ | `send` is `async` throughout; asserted in `run-suite.ts` | +| TRANSPORT-22 | MUST | An adaptation throw closes the native response first | ✅ | Both transports wrap `adaptResponse` and cancel (`body.cancel()`) / destroy (`body.destroy()`) before rethrowing; asserted by injection in `fetch-transport.test.ts` and `undici-transport.test.ts`, which is the only way to reach it — a conforming wire response has no field whose adaptation can fail | +| TRANSPORT-23 | MUST | Success never resolves to null | ✅ | The return type is `Promise<Response>` and `Response.newBuilder().build()` enforces its required fields; asserted in `run-suite.ts` | +| TRANSPORT-24 | MUST | Vendor status codes surfaced faithfully, body readable and closeable | ✅ | `Status.of` is total by construction (HTTP-10); asserted with a 520 in `run-suite.ts` and the Node layer | +| TRANSPORT-25 | MUST | Response body is a lazily-read stream; close cascades and releases the connection | ✅ | `fetch` hands over `Response.body` unbuffered; undici goes through `toDemandDrivenStream`, a pull-based adapter (deliberately not `Readable.toWeb`, which throws `ERR_INVALID_STATE` on Bun when closed without being drained, and deliberately not a `'data'`-listener adapter, which would buffer eagerly). Asserted in `run-suite.ts` against a dripping fixture — a first chunk in hand while the stream is still open — plus close-without-reading and idempotent close | +| TRANSPORT-26 | MUST | A body-less request is valid for any method; zero-length body substituted where required | ✅ | Neither native client rejects a null body, so no substitution is needed; asserted in `run-suite.ts` that a body-less POST dispatches with `Content-Length: 0` on the wire | +| TRANSPORT-27 | SHOULD | Malformed inbound Content-Type downgrades; absent Content-Length maps to -1 | ✅ / N/A | The Content-Type half is asserted in `run-suite.ts` (an unparseable type still delivers a 200 and a readable body — nothing parses it at the transport layer, so nothing can fail on it). The Content-Length half is **N/A in this port**: `Response.body` is a raw `ReadableStream`, and this port has no response-side declared-length field for a -1 sentinel to live in | +| TRANSPORT-28 | SHOULD | File body streams directly, honoring start/count; treated as replayable | ✅ (direct stream) / 🚫 (zero-copy) | `undici-transport.ts` `isFileBody` narrows structurally on `kind === 'file'` and dispatches `createReadStream(path, {start, end})`; asserted byte-exactly over the wire in `undici-transport.test.ts`. `fileBody()` is always `replayable: true` with a fresh handle per write (`body-file/src/file-body.ts`, `file-body.test.ts`). A literal kernel zero-copy path is a Deviation Ledger row — Node's HTTP client stack exposes no `sendfile`-shaped API for outbound bodies | +| TRANSPORT-29 | MUST | Concurrent-safe, effectively immutable after construction | ✅ | All per-request state lives in locals and the returned promise graph; every instance field is `readonly` except the memoized `#closing`. Asserted in `run-suite.ts` (20 concurrent sends, each response matched to its own request by a per-call header) and the Node layer | +| TRANSPORT-30 | SHOULD | Unsupported proxy features discoverable; credentials never logged, never answered to a 401 | ✅ | `undici-transport.ts` + `challenge-handler.ts`: a custom `challengeHandler` warns at construction and again on the first real 407, proxy auth falls back to Basic (`ProxyOptions.credentials`, passed to the `ProxyAgent` constructor), a 401 is never treated as a proxy challenge, and no credential reaches the logger on any path. A per-request `Proxy-Authorization` is dropped when a proxy is configured, because `ProxyAgent.dispatch` rejects one outright. Asserted in `challenge-handler.test.ts` and `undici-transport.test.ts`. **This is a deviation from the Phase 8a plan**, which specified a retry-with-stamped-credential flow; that flow is not implementable on undici — see `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` row 13. `transport-fetch` has no `proxy` option at all (design §6) | + +## Roadmap rows this phase closes + +| ID | Requirement gist | Status | Where | +|---|---|---|---| +| SEAM-12 | Concurrent-call conformance test | ✅ | Collapses onto `TRANSPORT-29`; `run-suite.ts` and `test/node-conformance/transport.test.mjs` | +| SEAM-14 | Close behavior: idempotent, ownership-aware, releases only self-created resources | ✅ | Collapses onto `TRANSPORT-15`/`TRANSPORT-16`; `undici-transport.test.ts` | +| SEAM-15 | Post-close `send()` behavior documented per adapter | ✅ | `fetch` keeps working (no-op close, nothing was released); undici rejects with the terminal `CancellationError`, since the dispatcher the send would route over no longer exists and no retry over it can succeed. Stated in each `close()` TSDoc and each package README | +| SEAM-16 | An abort after the promise resolved must not close the delivered body | ✅ | `transport-shared/src/signal-fork.ts`: both clients tie the body's lifetime to the signal they were handed, so each transport dispatches over a fork it detaches at delivery. Asserted in `signal-fork.test.ts`, `run-suite.ts`, and the Node layer | +| SEAM-30 | Cancel an orphaned response on the completion race | ✅ | Collapses onto `TRANSPORT-9`; `run-suite.ts` | +| NFR-2 | Each optional capability separately installable (core + ≤1 external lib) | ✅ | `transport-fetch`/`body-file`/`transport-shared` take zero external libs; `transport-undici` takes exactly one (`undici`). Gate-enforced by `scripts/verify-seam-1.mjs`'s per-package allow-list, which is now the *only* way to declare a runtime dependency: every package absent from it is still held to a hard-committed empty `dependencies` object, an omitted field included. `scripts/verify-seam-1.test.mjs` drives both halves | +| NFR-15 | The stamped identity actually reaches the wire | ✅ | `run-suite.ts` sends `getBuildInfo().identityTokens` as `User-Agent` and reads it back off the fixture server unmangled, for both transports | +| BODY-11/12/13 | File-backed body: fail-fast validation, recognizable by type, short-write detection | ✅ | `body-file/src/file-body.ts` + `file-body.test.ts`; recognition through `@dexpace/core`'s type-only `FileBodyDescriptor`. Neither transport depends on `@dexpace/body-file`, so a real `fileBody()` crossing a real transport has no home in either package's own suite — it is asserted in `test/node-conformance/transport.test.mjs` instead, whole and ranged, for both adapters. That is the only place the two halves meet: `transport-undici` bypasses `writeTo` entirely for its own `createReadStream` | diff --git a/docs/superpowers/specs/2026-07-28-phase8a-transport-design.md b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-28-phase8a-transport-design.md rename to docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-design.md index 6dafc43..1a4a4b0 100644 --- a/docs/superpowers/specs/2026-07-28-phase8a-transport-design.md +++ b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-design.md @@ -5,15 +5,15 @@ **Purpose:** Implement the concrete `Transport` implementations — `@dexpace/transport-fetch` and `@dexpace/transport-undici` — satisfying `docs/product-spec/17-transport-adapter-conformance-contract.md` (`TRANSPORT-1`–`TRANSPORT-30`), plus the nine Deferred Items Log rows the -[Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) routed here: `SEAM-30`/`SEAM-14`/ +[Phase 8 segmentation design](../2026-07-28-phase8-segmentation-design.md) routed here: `SEAM-30`/`SEAM-14`/ `SEAM-12` (Phase 2), the transport half of `NFR-2` and `NFR-15` (Phase 0), `FileBody` (Phase 3b brainstorm), and the `challengeHandler` protocol (Phase 7a brainstorm). This is the first of two Phase 8 sub-phases; 8b (`@dexpace/rx`, `§18`) has no dependency on this one and may execute in either order. **Governing documents:** `docs/product-spec/17-transport-adapter-conformance-contract.md` (normative, cited by ID -throughout), `docs/superpowers/specs/2026-07-28-phase8-segmentation-design.md` (the cut, the collapse tables, the -open items this document resolves), `docs/superpowers/specs/2026-07-23-phase2-seam-foundations-design.md` (the -`Transport` interface, `composeSignal`/`isTimeoutSignal`/`CancellationError`), `docs/superpowers/plans/ +throughout), `docs/work/mvp/phase8/2026-07-28-phase8-segmentation-design.md` (the cut, the collapse tables, the +open items this document resolves), `docs/work/mvp/phase2/2026-07-23-phase2-seam-foundations-design.md` (the +`Transport` interface, `composeSignal`/`isTimeoutSignal`/`CancellationError`), `docs/work/mvp/phase3/phase3b/ 2026-07-25-phase3b-body-lifecycle.md` (`Body`, `Request.body`, `Response.body`/`.close()`), `docs/sdk-design-nodejs/02-package-and-workspace-layout.md`, `docs/sdk-design-nodejs/03-seam-by-seam-idiomatic-mapping.md` §3.2, `docs/knowledge/{transport-adapter,concurrency-and-async,message-bodies,resource-management, diff --git a/docs/superpowers/plans/2026-07-28-phase8a-transport.md b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport.md similarity index 99% rename from docs/superpowers/plans/2026-07-28-phase8a-transport.md rename to docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport.md index 4f2201b..d9ca69c 100644 --- a/docs/superpowers/plans/2026-07-28-phase8a-transport.md +++ b/docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport.md @@ -6,7 +6,7 @@ `@dexpace/body-file`, and `@dexpace/transport-shared`, plus the unpublished `@dexpace/transport-conformance` devDependency and two small retrofits to `@dexpace/core` (`TransportFailureError`, `FileBodyDescriptor`) — satisfying `docs/product-spec/17-transport-adapter-conformance-contract.md` -(`TRANSPORT-1`–`TRANSPORT-30`) per `docs/superpowers/specs/2026-07-28-phase8a-transport-design.md`. +(`TRANSPORT-1`–`TRANSPORT-30`) per `docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-design.md`. **Architecture:** Five new workspace packages plus two amendments to already-shipped `@dexpace/core` files. Both transports implement the identical `Transport` interface (Phase 2, unchanged) and are proven against one shared diff --git a/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md new file mode 100644 index 0000000..a98b652 --- /dev/null +++ b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md @@ -0,0 +1,74 @@ +# Phase 8b — Async-Runtime Bridge — Checklist + +**Status: EXECUTED.** Every ✅ below names code and tests that exist on this branch, not a plan step. Verified +against `docs/product-spec/18-asynchronous-runtime-adapter-contract.md` (`ASYNC-1` through `ASYNC-22`), +`docs/product-spec/13-server-sent-events-and-streaming.md` (`SSE-41`), plus Task 1 through Task 4 deliverables, +package builds, and API reports. + +**One implementation deviation**, recorded in full in the plan's Self-Review: the `AsyncIterable`→`Observable` +bridge is `packages/rx/src/from-async-iterable.ts`, not RxJS's own `from()`, because `rxjs@7.8.2` does not reach +the source when a subscription is torn down while a pull is suspended — the `ASYNC-6` clause an idle SSE stream +depends on. Every row citing that module below is citing the reason it exists. The `🚫` rows that name Phase 8a +are **not satisfied yet**: they collapse onto `TRANSPORT-*` requirements no shipped package implements — see +`docs/work/mvp/2026-09-04-open-items-dissolution.md` §M2. + +**Legend:** ✅ Implemented and tested — 🚫 Not built (permanent simplification / collapse, named reason) — ⏳ Deferred +(named target phase) — N/A Not applicable in this port. + +## 18.1 Completion and failure delivery + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-1 | MUST | Single-value completion future delivers non-null Response on success, failure channel on no response | 🚫 | Collapses onto `TRANSPORT-23` (Phase 8a) — for `Transport`, `send()` returns `Promise<Response>` directly | +| ASYNC-2 | MUST | Construction-time failure via failure channel, not sync throw | 🚫 | Collapses onto `TRANSPORT-21` (Phase 8a) | + +## 18.2 Cancellation modes + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-3 | MUST | Cancel-with-interrupt vs without on worker thread | N/A | Node event loop has no worker-thread-pool transport model to interrupt (`SEAM-18` disposition) | +| ASYNC-4 | MUST | Ordered interrupt delivery preventing pooled-thread poisoning | N/A | Node event loop has no pooled worker threads to poison (`SEAM-18` disposition) | +| ASYNC-5 | MUST | Orphaned closeable result closed exactly once on race | 🚫 | Collapses onto `TRANSPORT-9` (Phase 8a) | +| ASYNC-6 | MUST | Bidirectional cancellation across adapter | ✅ | `packages/rx/src/from-async-iterable.ts`'s teardown (release the source, then `iterator.return()`). Asserted across all four paths — synchronous unsubscribe from inside `next()`, unsubscribe while a pull is suspended, unsubscribe before the first emission, and a rejected release — in `from-async-iterable.conformance.test.ts`, `sse.test.ts`, `pagination.test.ts`, and on real Node in `test/node-conformance/rx-bridge.test.mjs`. The same suite pins RxJS's native `from()` **failing** this clause | +| ASYNC-7 | SHOULD | Document interrupt-mode choice per adapter | N/A | Vacuous: no blocking worker thread calls to interrupt | + +## 18.3 Logging-context propagation + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-8 | SHOULD | Propagate logging context across thread/scheduler hops | ✅ | Node `AsyncLocalStorage` auto-propagation through promise chains/async iteration; the package installs no RxJS scheduler, which is what keeps the continuation chain intact. Stated in `sseEvents$`/`typedSse$` TSDoc, including the caller-introduced `observeOn`/`subscribeOn` boundary | +| ASYNC-9 | MUST | Save, install, restore logging context | ✅ | Node `AsyncLocalStorage` auto-propagation invariant | +| ASYNC-10 | MUST | Capture logging context at logical caller point | ✅ | Node `AsyncLocalStorage` captures per subscription at iteration pull time | +| ASYNC-11 | MUST | Safe when no logging context backend installed | ✅ | `AsyncLocalStorage` handles undefined store gracefully | +| ASYNC-12 | MUST | Explicit transfer at thread boundary where auto-inheritance absent | N/A | Single-threaded event loop; continuation-local storage auto-propagates | + +## 18.4 Error unwrapping and blocking bridge + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-13 | MUST | Unwrap async framework wrapper exceptions to original cause | ✅ | `packages/rx/src/from-async-iterable.ts` passes a thrown value straight to `subscriber.error`; asserted in `from-async-iterable.conformance.test.ts` (`RangeError` in, same `RangeError` out) and `sse.test.ts` (a throwing `SseMapper`) | +| ASYNC-14 | MUST | Async->sync blocking bridge honoring thread interruption | N/A | Inapplicable in Node — no blocking HTTP client bridge (`SEAM-18` disposition) | + +## 18.5 Lifecycle + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-15 | MUST | Close/dispose operation idempotent, ownership-aware, interrupt-safe | 🚫 | Owned by Phase 8a `TRANSPORT-15`/`16`; `@dexpace/rx` owns no background thread pools | +| ASYNC-16 | SHOULD | Graceful executor shutdown on close | 🚫 | Owned by Phase 8a | +| ASYNC-17 | SHOULD | No-op default close for lightweight/functional transports | 🚫 | Owned by Phase 8a (`transport-fetch`) | + +## 18.6 Delay, options, and streaming + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| ASYNC-18 | MUST | Non-blocking scheduled-delay primitive | N/A | Resolved N/A to this port: `@dexpace/rx` does no reconnection, retry, or backoff; SSE reconnection is caller-owned; pagination retry lives in pipeline layer | +| ASYNC-19 | MUST | Per-call request options threaded through overloads | N/A | Resolved N/A: `@dexpace/rx` wraps already-constructed `SseStream` / `Paginator` instances; does not initiate new HTTP calls | +| ASYNC-20 | MUST | Delivered Response body not closed on late future cancel | 🚫 | Restates `SEAM-16` / transport invariant owned by Phase 8a | +| ASYNC-21 | MUST | Reactive streaming adapter (SSE) honors backpressure, completes on end-of-source, propagates errors without swallowing, single-subscriber | ✅ | `packages/rx/src/sse.ts`: `sseEvents$`, `typedSse$`, over `from-async-iterable.ts`'s one-pull-per-emission loop. `from-async-iterable.conformance.test.ts` (poll-once-per-demand, complete-on-end, error passthrough), `sse.test.ts` (single-subscriber via `SSE-26`), `test/node-conformance/rx-bridge.test.mjs` | +| ASYNC-22 | MUST | Safe for concurrent calls | 🚫 | Collapses onto `TRANSPORT-29` (Phase 8a) | + +## 13.7 Server-Sent Events + +| ID | Level | Requirement gist | Status | Where | +|---|---|---|---|---| +| SSE-41 | MAY | Reactive SSE adapter with fatal/non-fatal split and documented source ownership | ✅ | `packages/rx/src/sse.ts`: `sseEvents$`, `typedSse$`. Source ownership is documented on both functions and in `packages/rx/README.md` (the adapter closes the stream on unsubscribe; the caller owns reconnection). The fatal/non-fatal split collapses — JavaScript has no catchable-fatal tier, per the design doc's Deviation Ledger. Asserted in `sse.test.ts` and `from-async-iterable.conformance.test.ts` | diff --git a/docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md similarity index 97% rename from docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md rename to docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md index a40f5ca..195296f 100644 --- a/docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md +++ b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md @@ -4,15 +4,15 @@ **Purpose:** Implement `@dexpace/rx`, exposing Phase 6's `Paginator`/`Page` and `SseStream` as RxJS `Observable`s — satisfying the non-collapsed subset of `docs/product-spec/18-asynchronous-runtime-adapter-contract.md` -(`ASYNC-*`) identified in the [Phase 8 segmentation design](./2026-07-28-phase8-segmentation-design.md) §5.2, plus +(`ASYNC-*`) identified in the [Phase 8 segmentation design](../2026-07-28-phase8-segmentation-design.md) §5.2, plus `SSE-41` (the reactive SSE adapter, Phase 6 brainstorm). Second of two Phase 8 sub-phases; has no dependency on 8a and may execute in either order. **Governing documents:** `docs/product-spec/18-asynchronous-runtime-adapter-contract.md`, `docs/product-spec/ -13-server-sent-events-and-streaming.md` (`SSE-41`, `SSE-26`'s single-pass rule), `docs/superpowers/specs/ +13-server-sent-events-and-streaming.md` (`SSE-41`, `SSE-26`'s single-pass rule), `docs/work/mvp/phase8/ 2026-07-28-phase8-segmentation-design.md` §4/§5.2/§7 (this document resolves every 8b open item that document -flagged), `docs/superpowers/specs/2026-07-28-phase6b-sse-design.md` (`SseStream`, `typedSseStream`), -`docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md` (`Paginator`, `Page`), `docs/superpowers/specs/ +flagged), `docs/work/mvp/phase6/phase6b/2026-07-28-phase6b-sse-design.md` (`SseStream`, `typedSseStream`), +`docs/work/mvp/phase6/phase6c/2026-07-28-phase6c-pagination-design.md` (`Paginator`, `Page`), `docs/work/mvp/phase7/phase7b/ 2026-07-28-phase7b-observability-design.md` (the `AsyncLocalStorage` diagnostic-context bridge this phase reuses), `docs/sdk-design-nodejs/02-package-and-workspace-layout.md`, `docs/knowledge/{concurrency-and-async, sse-streaming,pagination,observability,resource-management}.md`. diff --git a/docs/superpowers/plans/2026-07-28-phase8b-async-runtime.md b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime.md similarity index 85% rename from docs/superpowers/plans/2026-07-28-phase8b-async-runtime.md rename to docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime.md index 9e65970..7a0b8fc 100644 --- a/docs/superpowers/plans/2026-07-28-phase8b-async-runtime.md +++ b/docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime.md @@ -3,7 +3,7 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Ship `@dexpace/rx`, exposing Phase 6's `SseStream`/`typedSseStream` and `Paginator` as RxJS `Observable`s, -per `docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md`. Satisfies the non-collapsed subset of +per `docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-design.md`. Satisfies the non-collapsed subset of `docs/product-spec/18-asynchronous-runtime-adapter-contract.md` and `SSE-41`. **Architecture:** One thin package. The bridge logic itself is RxJS's own native `from(asyncIterable)` — this @@ -391,6 +391,52 @@ git commit -m "feat(rx): promote public barrel for @dexpace/rx" ## Self-Review +**Executed 2026-08-28.** All four tasks shipped; the full gate sequence is green. One deviation, recorded below +rather than left silent. + +### Deviation Ledger addition — the `AsyncIterable`→`Observable` bridge is hand-written + +**What the plan said.** Global Constraints: "Do not hand-write an `AsyncIterable`-to-`Observable` pull loop. Use +RxJS's own `from()`." Task 1 Step 3 named the escape hatch: if a conformance clause fails against the installed +RxJS, write the minimal wrapping `Observable` that closes *exactly* that clause and record it here. + +**What was found.** `rxjs@7.8.2`'s async-iterable path (`internal/observable/innerFrom.js`) is a bare `for await` +loop that tests `subscriber.closed` only *after* a pull resolves: + +```js +async function process(asyncIterable, subscriber) { + for await (const value of asyncIterable) { + subscriber.next(value); + if (subscriber.closed) return; + } + subscriber.complete(); +} +``` + +Unsubscribing while a pull is suspended therefore reaches the source only if and when the source produces +again. For pagination that is invisible (a page fetch always settles). For SSE it is the failure mode that +matters most: an idle event stream is *permanently* suspended on `next()`, so `subscription.unsubscribe()` +leaves the response body unreleased and the connection open until the server happens to send something. That is +`ASYNC-6`'s bidirectional-cancellation clause, unsatisfied — and `SSE-30`'s release obligation with it. + +**What was built.** `packages/rx/src/from-async-iterable.ts` (`@internal`, ~60 lines): the same pull loop, plus a +teardown that releases the caller-supplied source and drives `iterator.return()` on unsubscription. Release runs +*before* the iterator return, because closing the source is what settles the suspended pull that an async +generator's queued `return()` would otherwise sit behind. Scope is exactly the failing clause — no scheduler, no +error re-wrapping, no retry, no buffering. + +**How it stays honest.** `from-async-iterable.conformance.test.ts`'s last case asserts the *defect* in RxJS's own +`from()` (`returns` stays `0` after an idle unsubscribe) alongside this module's `1`. When a future RxJS closes +the gap that case fails, and the reviewer's instruction is in the file header: delete the module and go back to +`from()`. `test/node-conformance/rx-bridge.test.mjs` proves the same cancellation path on real Node, since +whether the release lands depends on Node's `ReadableStream.cancel()` and async-generator `return()` queueing. + +**Not a deviation from the product spec.** `ASYNC-6`/`ASYNC-21`/`SSE-41` are satisfied as written; the deviation +is from this plan's own implementation instruction, which named this outcome as an allowed one. Nothing is added +to `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md`, whose Phase 8b rows +(`ASYNC-21`'s fatal/non-fatal collapse, `ASYNC-18`'s full-port collapse) are unaffected. + + - [ ] Task 1's conformance suite passed against the installed RxJS version with no fallback needed — or, if a fallback was needed, it is scoped to exactly the failing clause and recorded in this section as a Deviation Ledger addition to the design doc. diff --git a/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-checklist.md b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-checklist.md new file mode 100644 index 0000000..77b5798 --- /dev/null +++ b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-checklist.md @@ -0,0 +1,120 @@ +# Phase 9 — Cross-Cutting Invariants & Conformance — Requirement Checklist + +Every `XCUT-1`–`XCUT-24` and `NFR-1`–`NFR-17` ID, mapped to the task that satisfies it and the evidence that +proves it. This is the first systematic tabulation of the `XCUT` family in this project — before Phase 9 the +grep across every spec and plan turned up incidental citations only, and **zero** `XCUT-N` citations in any +source file. + +**Legend.** ✅ satisfied, evidence cited · 🔁 satisfied by an earlier phase, retrofit citation added here · +📋 documented disposition, no test · ⚠️ satisfied with a finding filed against another phase. + +**Gate status at close:** `typecheck`, `lint`, `build`, `test` (2171 pass / 0 fail, 99.72% lines vs. the 80% +floor), `api`, `lint:publish`, `verify:dual-consumption`, `verify:consumer-types`, `verify:seam-1`, +`verify:sse-37`, `verify:runtime-floor`, `audit`, `shrink-test`, `test:node`, `test:scripts` — all green. + +--- + +## `XCUT` — cross-cutting invariants + +| ID | Status | Task | Evidence | +|---|---|---|---| +| `XCUT-1` | ✅ | 5 | `cancellation-and-timeout.conformance.test.ts` — 3 rows: an aborted in-flight request surfaces `CancellationError`; the retry pillar does not spend its remaining attempts (`dispatches() === 1`); the ambient signal stays aborted | +| `XCUT-2` | 🔁 | 5 | `seams/transport.test.ts` — citation already present from Phase 2; `isTimeoutSignal` discriminates on `signal.reason.name`, never a message string. No change needed | +| `XCUT-3` | ⚠️ | 5 | Same file — 3 rows: a 60 s backoff aborts in well under 5 s; a cancellation (not a timeout) is carried in the surfaced chain; no further attempt dispatched. **Finding N1** — the retry path surfaces a bare `AbortError`, not `CancellationError` | +| `XCUT-4` | ✅ | 6 | `error-taxonomy.conformance.test.ts` — 3 rows: a 5xx returns its fully-received response and converts via `toHttpError` to a status+body-carrying error; a refused connection rejects as `IoError` | +| `XCUT-5` | 🔁 | 6 | `retry/classify.test.ts` — already asserts 408, 429, 500/503/599 retryable and 501/505 not. This port has no separately-cached flag: the classifier is a pure function of the immutable `.status` | +| `XCUT-6` | ✅ | 6 | Same file — 2 rows: a `CustomTransientError extends IoError` declared *in the test file* is retried with no edit to `classify.ts`; a plain `Error` is not. Subtyping is this port's retryability capability (deviation ledger item 17) | +| `XCUT-7` | ✅ | 6 | Same file — 2 rows against a live `/status?code=N`: widening to `{501}` retries a 501; narrowing to `{503}` stops a 500 whose built-in classification is retryable | +| `XCUT-8` | ⚠️🔁 | 6 | `body/http-status-error.test.ts` — `toHttpError` returns `null` for 200/304, the absent/null convenience form `XCUT-8` permits. **Finding N2** — the public constructor still builds `HttpStatusError(200, …)` | +| `XCUT-9` | ✅ | 6 | `error-taxonomy.conformance.test.ts` — a self-referential `cause` is classified and surfaced unchanged; the test completing at all is the assertion | +| `XCUT-10` | ✅ | 7 | `retry-safety.conformance.test.ts` — all five rows of the requirement's own conformance clause, including the load-bearing one: a body-less POST failing with a *transport* error is still not retried | +| `XCUT-11` | ✅ | 8 | `concurrency-and-lifecycle.conformance.test.ts` — 24 interleaved requests through one shared pipeline pair every response to its own request; exactly one dispatch each, no double-sends | +| `XCUT-12` | 🔁 | 8 | `auth/bearer-cache.test.ts` — N concurrent callers coalesce to exactly one provider invocation, in both the expired and post-eviction zones | +| `XCUT-13` | ✅🔁 | 8 | `concurrency-and-lifecycle.conformance.test.ts` — 4 rows incl. a real transport closed twice, and an aborted signal not cleared by close. Retrofits on `fetch-transport.test.ts` / `undici-transport.test.ts` | +| `XCUT-14` | 🔁 | 8 | `context/store.test.ts` ("a burst past the cap converges to at or under the cap") and `auth/digest.test.ts` (1024-entry nonce counter, drain-to-cap). Retrofit rather than a new test — **neither map is reachable from a consumer-shaped test**, so a burst driven from `tests/` could assert liveness but never a bound | +| `XCUT-15` | 🔁 | 9 | `http/request.test.ts` (URL cloned per access, setters yield new instances) and `http/headers.test.ts` (builder defensively copies an ingested collection) | +| `XCUT-16` | ✅🔁 | 9 | `security-by-default.conformance.test.ts` — a bearer credential over `http://` throws `PlaintextCredentialError` with **`providerInvocations === 0`** and `dispatches() === 0`: the refusal lands before any token fetch. Retrofit on `auth/auth-step.test.ts` | +| `XCUT-17` | ✅🔁 | 9 | Same file — 4 rows over two genuinely distinct origins: Authorization dropped even same-origin; Cookie kept same-origin but dropped cross-origin. Clauses (c) userinfo and (d) downgrade retrofitted onto `redirect/decide.test.ts`, being unreachable over a plaintext fixture | +| `XCUT-18` | 🔁 | 9 | `http/headers.test.ts` — names reject C0 incl. HTAB, DEL and non-ASCII; outbound values reject the same except HTAB; inbound lenient on obs-text but not control bytes | +| `XCUT-19` | 🔁 | 9 | `observability/redaction.test.ts` (userinfo never allow-listable, query/fragment default-deny) and `auth/credential.test.ts` (all three credentials redact their secret in every string form) | +| `XCUT-20` | 🔁 | 9 | `observability/logging-step.test.ts` — a throwing `Logger` is caught and re-surfaced as `http.instrumentation.*`; the request still completes | +| `XCUT-21` | 🔁 | 8 | `auth/digest.test.ts` — the cnonce is drawn from `crypto.getRandomValues` at ≥128 bits, fresh per call (AUTH-20) | +| `XCUT-22` | 🔁 | 8 | `undici-transport.test.ts` ("a bring-your-own dispatcher is never closed by the transport") and `fetch-transport.test.ts`. Also asserted end-to-end at pipeline level: `Runtime.close()` leaves the caller's transport usable | +| `XCUT-23` | 📋 | — | **N/A by construction.** Every seam this port ships (`Transport`, `Serde`, the logger facade) is explicit-call-only; the classpath auto-discovery `SEAM-5`–`SEAM-10` describes is a permanent simplification never built. The ordering holds vacuously — there is nothing for an explicit install to beat. Deviation ledger, Phase 9 row 1 | +| `XCUT-24` | ✅🔁 | 10 | `diagnostic-previews.conformance.test.ts` — the requirement's own clause verbatim: a **10 MB** body with a 1 KiB cap. Text previews cap at 1024 chars, binary at `[binary 1024 bytes captured]`, `body.size` is 1024 not 10485760, no event field exceeds the cap, and the caller still reads all 10485760 bytes | + +**All 24 dispositioned. No silent gaps.** + +--- + +## `NFR` — non-functional requirements + +| ID | Status | Evidence | +|---|---|---| +| `NFR-1` | ✅ | Audited across all 11 packages: `@dexpace/core` declares zero `dependencies`. Gate-enforced by `verify:seam-1` | +| `NFR-2` | ✅ | Every adapter is core-as-peer plus at most one external library — `logging-debug`→`debug`, `logging-pino`→`pino`, `rx`→`rxjs`, `transport-undici`→`undici`, `transport-fetch`/`body-file`/`codec-json`→none. `@dexpace/transport-shared` is an internal sibling, not a third-party lib | +| `NFR-3` | ✅ | One committed `etc/*.api.md` per published package (9 of them); internals stay unexported | +| `NFR-4` | ✅ | `bun run api` verifies all 9 reports; blocking in CI | +| `NFR-5` | ✅ | `bunfig.toml` `coverageThreshold = 0.8`, blocking. Actual: 99.72% lines / 98.76% funcs. **Verified live twice** — raising the threshold to 0.999 makes the run exit 1, and a single new file at 66.67% function coverage failed the run on its own while the aggregate stayed at 98.5%. So Bun enforces the floor **per file**, not only in aggregate: stricter than `NFR-5`'s "minimum aggregate" wording requires, and the gate is demonstrably not dormant | +| `NFR-6` | ✅ | `tsc --noEmit` per package under `strict`; `typecheck` now covers `shrink-test` and `tests/` too | +| `NFR-7` | ✅ | `gts` + `strictTypeChecked`/`stylisticTypeChecked`, fatal. Every `eslint-disable` carries a `-- reason`, including the one added this phase in `error-taxonomy.conformance.test.ts` | +| `NFR-8` | 📋 | **Not applicable by design** — no reflection-driven discovery surface to keep-configure. Deviation ledger, Phase 9 row 2; `docs/knowledge/deliberate-deviations.md:55` (stale as of 2026-08-30 — read `docs/deviations.md` §10 instead) | +| `NFR-9` | ✅ | `@dexpace/shrink-test` (Tasks 1–3): esbuild bundle+minify+tree-shake, 24 KiB budget against a measured 16,671 bytes, then a **child-process** round trip. Guard proven non-vacuous: a separately-bundled `IoError` has a different class identity and `instanceof` is false across the boundary | +| `NFR-10` | ✅ | All 10 published packages declare `engines.node >= 20.3`; `verify:runtime-floor` gates target-vs-floor; `test:node` runs the floor and current LTS in CI | +| `NFR-11` | ✅ | No `Observable`/`rxjs`/`Subscriber`/`EventEmitter` anywhere in `core.api.md`; `rxjs` appears in no core source file | +| `NFR-12` | 📋 | Deferred to Phase 10 / first release, unchanged | +| `NFR-13` | ✅ | Swept every tracked `.ts`/`.mjs`/`.js`: **3 offenders fixed** (`eslint.config.js`, `scripts/knowledge.mjs`, `scripts/knowledge.test.mjs`). Now 0. `packages/core/scripts/gen-version.mjs` correctly carries it on line 2 under a shebang | +| `NFR-14` | ⚠️ | Root catalog holds `api-extractor`, `expect-type`, `fast-check`, `typescript`. **Finding N4** — `rxjs@^7.8.0` is restated in three places. Peer ranges (`debug`, `pino`) are correctly per-package, being part of each package's published contract | +| `NFR-15` | ✅ | `SDK_VERSION` generated at build time from `package.json`; resolves to the real version, never an "unknown" placeholder | +| `NFR-16` | 📋 | Deferred to first actual publish, unchanged | +| `NFR-17` | ✅ | Every gate above is a blocking CI step. `shrink-test` needed no fourteenth step: the suite lives under `packages/`, so `bun run test` already runs it | + +**All 17 dispositioned.** + +--- + +## Deviations recorded for Phase 10 + +| Deviation | Reference behavior | Justification | +|---|---|---| +| `XCUT-23`'s explicit-install / auto-discovery / loud-fail ordering is satisfied vacuously, not tested as a race | The JVM reference arbitrates a real classpath auto-discovery race for its SPI seams | Every seam this port ships is explicit-call-only; the auto-discovery mechanism was never built, so no competing resolution path exists for an explicit install to beat | +| `NFR-8`'s shrinker keep-configuration ships nothing | The reference ships ProGuard/R8 keep rules for its reflective/SPI surface | No reflection-driven discovery surface exists here. The risk that *does* carry over is the dual-package hazard, and `@dexpace/shrink-test` targets it — now with measured proof the hazard is real | +| `XCUT-6`'s "retryability capability" is subtyping, not a duck-typed flag | The reference queries a capability interface | `classify.ts`'s allow-list returns true for any `IoError`, so extending it opts a new failure in with no classifier edit. Already ledgered as item 17 | + +## Findings filed — `docs/work/mvp/2026-09-04-open-items-dissolution.md` Section N + +| # | Summary | Owner | +|---|---|---| +| N1 | Cancellation surfaces `CancellationError` from the transport but a bare `AbortError` from a retry backoff wait | 5a | +| N2 | `HttpStatusError`'s public constructor accepts a 200, fabricating the "successful exception" `XCUT-8` names | 3b | +| N3 | The plan's `grep -rn "unresolved 2026-07-25" docs/knowledge/` step cannot pass as written | Phase 10 | +| N4 | `rxjs` version restated in three places against `NFR-14`'s single-source-of-truth | Phase 10 | + +## Plan amendments + +The plan was written before any package existed, and several of its code blocks assume APIs that shipped +differently. Recorded so the next reader does not treat the plan as as-built: + +1. **Suite location.** `bunfig.toml` pins `[test] root = "packages"`, so a top-level `tests/` tree is invisible + to `bun test`. The root `test` script now passes both trees (`bun test ./packages ./tests`); CI runs + `bun run test --coverage`. The design's "exactly one new root script" no longer holds — `test` and + `typecheck` each grew, and CI's Test step changed. +2. **`StandardResilienceOptions.retry` is `RetryStepOptions`**, so settings nest under `.settings` — not the + plan's `Partial<RetrySettings>`. +3. **`isRetryableFailure` is `@internal`** and absent from core's barrel. Every `XCUT-6`/`7`/`9` row drives the + composed pipeline instead, which is what this suite is for anyway. +4. **`XCUT-6`'s test error extends `IoError`**, not a duck-typed `{isRetryable: true}` — no such capability + exists, by design. +5. **The dispatch counter wraps the transport, not `Runtime.send`.** The plan's placement counts caller + invocations and would read 1 whether retry re-issued four times or none, inverting every `XCUT-10` row. +6. **Two real listeners for the cross-origin hop.** The plan reused one port under `localhost` vs `127.0.0.1`; + the server binds `127.0.0.1` explicitly, so that name is not reliably resolvable. +7. **`XCUT-17`'s auth-re-stamp row is unreachable over a plaintext fixture** — `XCUT-16` forbids stamping a + credential over `http://`, which the suite asserts directly instead. +8. **`XCUT-13` on `Runtime` proves nothing about close.** `Runtime.close()` is a documented no-op (PIPE-27); + the real idempotence lives in the transports, so the test asserts both, plus the trap that a consumer who + only calls `runtime.close()` never closes the transport. +9. **Task 11's three documentation fixes were already applied at planning time** (commit `c6603aa`), so this + phase confirmed them rather than making them. +10. **`scripts/verify-nfr-audit.mjs` was never created.** The plan had it written, run once, then deleted; the + same checks were run directly, and their results are the `NFR-1`/`NFR-2` rows above. diff --git a/docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md similarity index 99% rename from docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md rename to docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md index 86d9473..3459872 100644 --- a/docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md +++ b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md @@ -13,8 +13,8 @@ a gap a prior phase left open. **Governing documents:** `docs/product-spec/19-cross-cutting-invariants-and-policies.md`, `docs/product-spec/20-non-functional-requirements-and-quality-bar.md`, -`docs/product-spec/appendix-b-conformance-test-checklist.md` (§B.8, §B.9), `docs/superpowers/specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md` -(Deferred Items Log), `docs/superpowers/plans/2026-07-25-checkpoint-scaffold-through-phase3a.md` (the one prior +`docs/product-spec/appendix-b-conformance-test-checklist.md` (§B.8, §B.9), `docs/work/mvp/2026-07-23-nodejs-sdk-v1-roadmap-design.md` +(Deferred Items Log), `docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md` (the one prior cross-phase audit this roadmap has produced — its structure and its unclosed action items are both inputs here), every prior phase's own design/plan (cited per-ID below). `docs/knowledge/{cross-cutting-invariants,testing, tooling-and-quality-gates,deliberate-deviations,seams-and-extensibility,resource-management,cancellation-and-timeouts, diff --git a/docs/superpowers/plans/2026-07-28-phase9-cross-cutting-conformance.md b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance.md similarity index 99% rename from docs/superpowers/plans/2026-07-28-phase9-cross-cutting-conformance.md rename to docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance.md index f5168df..e522335 100644 --- a/docs/superpowers/plans/2026-07-28-phase9-cross-cutting-conformance.md +++ b/docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance.md @@ -5,7 +5,7 @@ **Goal:** Prove `docs/product-spec/19-cross-cutting-invariants-and-policies.md` (`XCUT-1`–`XCUT-24`) and `docs/product-spec/20-non-functional-requirements-and-quality-bar.md` (`NFR-1`–`NFR-17`) hold across the composed workspace, ship `@dexpace/shrink-test` (`NFR-9`), and close three stale `unresolved 2026-07-25` markers in -`docs/knowledge/tooling-and-quality-gates.md` — per `docs/superpowers/specs/2026-07-28-phase9-cross-cutting-conformance-design.md`. +`docs/knowledge/tooling-and-quality-gates.md` — per `docs/work/mvp/phase9/2026-07-28-phase9-cross-cutting-conformance-design.md`. **Architecture:** One new private/unpublished devDependency package (`@dexpace/shrink-test`) plus one new top-level integration-test directory (`tests/conformance/xcut/`, the first use of `docs/knowledge/testing.md:8`'s @@ -901,7 +901,7 @@ Expected: PASS, 3 tests. - [ ] **Step 3: Retrofit `XCUT-12` and `XCUT-22`** -Add `XCUT-12` to 5c's existing credential-cache single-flight test's header comment (`docs/superpowers/plans/2026-07-26-phase5c-auth.md`'s own test already races N callers on an expiring token). Add `XCUT-22` to 8a's +Add `XCUT-12` to 5c's existing credential-cache single-flight test's header comment (`docs/work/mvp/phase5/phase5c/2026-07-26-phase5c-auth.md`'s own test already races N callers on an expiring token). Add `XCUT-22` to 8a's existing `undici-transport.test.ts` BYO-dispatcher test ("closing a transport built from a BYO Agent does not close that agent") — both comment-only. @@ -1105,7 +1105,7 @@ git commit -m "test(conformance): add XCUT-24 large-body diagnostic-preview test confirmed 2026-07-28 Phase 9 audit). The scaffold implements Bun (`bun.lock`, `.bun-version`, `bun install --frozen-lockfile` as the CI gate) throughout; the design's pnpm/`catalog:` framing describes a toolchain this repository does not use. Decision recorded at - `docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md:54`; the enforcement properties pnpm's + `docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-checklist.md:54`; the enforcement properties pnpm's layout gave for free (isolated linker, workspace catalogs) were restored separately — see the Bun workspace catalogs adopted in Phase 6a and the isolated linker set at the 2026-07-25 checkpoint. <sub>design `docs/sdk-design-nodejs/02-package-and-workspace-layout.md:50-51` · styleguide diff --git a/docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-checklist.md similarity index 98% rename from docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md rename to docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-checklist.md index d102d10..639a53c 100644 --- a/docs/superpowers/plans/2026-07-23-scaffold-milestone-checklist.md +++ b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-checklist.md @@ -9,7 +9,7 @@ requirement-ID prefixes. Seventeen of them (`HTTP`, `IO`, `BODY`, `CTX`, `PIPE`, `PAGE`, `SSE`, `SERDE`, `OBS`, `CFG`, `TRANSPORT`, `ASYNC`, `XCUT`) are behavioral contracts on domain code — this phase ships zero domain code, so none of them are evaluable yet. They aren't listed item-by-item below; they're tracked at their respective phases in -[2026-07-23-nodejs-sdk-v1-roadmap-design.md](../specs/2026-07-23-nodejs-sdk-v1-roadmap-design.md). Only the two +[2026-07-23-nodejs-sdk-v1-roadmap-design.md](../2026-07-23-nodejs-sdk-v1-roadmap-design.md). Only the two prefixes with toolchain/architectural-level applicability — `NFR` (17 requirements) and `SEAM` (2 of its 30 requirements: `SEAM-1`/`SEAM-2`, the architectural ones; `SEAM-3` onward are seam *behavior* contracts, equally out of scope until Phase 2) — are checked here. diff --git a/docs/superpowers/specs/2026-07-23-scaffold-milestone-design.md b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-design.md similarity index 98% rename from docs/superpowers/specs/2026-07-23-scaffold-milestone-design.md rename to docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-design.md index c9af5a7..27f86e2 100644 --- a/docs/superpowers/specs/2026-07-23-scaffold-milestone-design.md +++ b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-design.md @@ -4,7 +4,7 @@ **Purpose:** Bootstrap the `nodejs-sdk` repository from its current state (docs only, no `package.json`) to a buildable, lintable, testable, dual-consumable state with **zero domain code**. This is Phase 0 of the -[v1 roadmap](./2026-07-23-nodejs-sdk-v1-roadmap-design.md) and the only phase this document covers in detail. +[v1 roadmap](../2026-07-23-nodejs-sdk-v1-roadmap-design.md) and the only phase this document covers in detail. **Why this comes first:** every later phase is written under the styleguide and toolchain gates from line one. Building domain code before the gates exist means retrofitting lint rules, coverage floors, and API-compatibility diff --git a/docs/superpowers/plans/2026-07-23-scaffold-milestone.md b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone.md similarity index 99% rename from docs/superpowers/plans/2026-07-23-scaffold-milestone.md rename to docs/work/mvp/scaffold/2026-07-23-scaffold-milestone.md index 6f14afe..e6c886f 100644 --- a/docs/superpowers/plans/2026-07-23-scaffold-milestone.md +++ b/docs/work/mvp/scaffold/2026-07-23-scaffold-milestone.md @@ -773,7 +773,7 @@ exist there. ## Self-Review -**Spec coverage** (against `docs/superpowers/specs/2026-07-23-scaffold-milestone-design.md`): +**Spec coverage** (against `docs/work/mvp/scaffold/2026-07-23-scaffold-milestone-design.md`): - Workspace init (Bun, `.bun-version`, `bun.lock`, `packages/*`) → Task 1, 3. - Stub `@dexpace/core` with placeholder export → Task 3, 4. - Full toolchain gate table (package manager, lint, type strictness, explicit API surface, API-compatibility diff --git a/eslint.config.js b/eslint.config.js index b45f534..a7b4de7 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: MIT import {createRequire} from 'node:module'; import tseslint from 'typescript-eslint'; import gts from 'gts'; @@ -22,12 +23,27 @@ export default tseslint.config( rules: {'prettier/prettier': ['error', gtsPrettierOptions]}, }, { - // The root config and the `.mjs` verification scripts belong to no + // The root config, the `.mjs` verification scripts, the Node-runtime + // conformance suite, and the `.claude/skills` runners belong to no // TypeScript project; they get the gts/format baseline only, never the // type-aware tiers below. gts scopes its own Node globals to a fixed list - // of filenames that does not include `scripts/*.mjs`, so declare them here - // or `console`/`URL` trip `no-undef`. - files: ['eslint.config.js', 'scripts/*.mjs'], + // of filenames that includes none of these, so declare them here or + // `console`/`URL` trip `no-undef` — and, in the conformance suite, so do + // the Web Streams and `AbortSignal` globals that are the whole point of + // running it on Node. + // + // The Node-conformance entry below is one of the five files holding the + // `tests/` partition (CLAUDE.md's hard rule), and + // `scripts/verify-test-partition.mjs` blocks CI if it stops matching a real + // file. A stale glob here does not error: it silently drops the Node + // globals and buries the suite in `no-undef`. + files: [ + 'eslint.config.js', + 'scripts/*.mjs', + 'packages/*/scripts/*.mjs', + 'tests/node-conformance/*.mjs', + '.claude/skills/*/*.mjs', + ], languageOptions: {sourceType: 'module', globals: globals.node}, }, { diff --git a/examples/petstore/FINDINGS.md b/examples/petstore/FINDINGS.md new file mode 100644 index 0000000..dea4b3d --- /dev/null +++ b/examples/petstore/FINDINGS.md @@ -0,0 +1,278 @@ +# Petstore codegen spike — findings + +The deliverable of [issue #64](https://github.com/dexpace/nodejs-sdk/issues/64). This is the answer +the spike was built to produce: for each gap the issue hypothesised, what the example actually had +to hand-write, and whether it belongs in `packages/core/src/codegen/`. + +**Status: spike complete, all five hypotheses confirmed, five more gaps found.** The example is +throwaway. Nothing here is shipped, nothing is a package, and no CI step runs any of it. + +## What was built + +``` +examples/petstore/ +├── spec/petstore.openapi.json byte-identical to python-sdk/examples/petstore/spec/ +├── generate.mjs deterministic renderer, Prettier-formatted output 426 lines +├── generate.d.mts hand-written types for the script +├── tsconfig.json standalone; exists for eslint's projectService (finding 7) +├── src/ +│ ├── models.ts Pet / PetEvent / PetPatch + Schema<T> witnesses 158 +│ ├── operation.ts the Operation / OperationInput split (finding 3) 84 +│ ├── errors.ts PetStoreError / PetNotFoundError + StatusErrorMap 129 +│ ├── support.ts jsonBody, petPatchToWire, PET_PAGE_STRATEGY, mapper 154 +│ ├── service-core.ts the executor — the payload of the spike 290 +│ ├── fake-transport.ts local in-memory Transport (finding 5) 129 +│ └── _generated/ +│ ├── operations.ts the operation table 45 +│ └── client.ts the facade 82 +├── canary.test.ts 15 assertions end to end over the fake transport 365 +└── regen.test.ts re-render, byte-compare 34 +``` + +Everything under `src/` except `_generated/` is what a real service SDK would hand-write. That is +**944 lines, of which roughly 340 are the gap**: the executor's mechanical half, the status map, the +operation split, and the fake transport. The rest — models, schemas, binders — is per-service work +that no core change removes. + +## Verification + +All run by hand; none is a CI step. + +```bash +bun run build # @dexpace/core -> dist, and the rest +node examples/petstore/generate.mjs # rewrite src/_generated/ +bunx tsc -p examples/petstore/tsconfig.json --noEmit +bunx eslint examples/ # clean; and `bun run lint` covers it too +bun test ./examples/petstore # 15 pass, 0 fail +bun run test # 164 files — UNCHANGED from the pre-scaffold run +``` + +The isolation premise held on the number that mattered: `bun run test` collected **164 files before +the scaffold and 164 after**. It did not hold completely — see finding 7. + +--- + +## Confirmed hypotheses + +### 1. No executor tier — confirmed, and it is thin + +Nothing in `@dexpace/core` exports an object with `execute` / `executeRequest` / `paginate` / +`events` plus ownership-aware close. `src/service-core.ts` is that object. Stripped of its comments +it is about 90 lines, and it is thin for one specific reason worth recording: + +**`Runtime implements Transport` (PIPE-26) is what collapses the layer.** The same pipeline drops +into `new Paginator({transport: runtime, ...})`, into a bare `runtime.send()`, and into the response +`sseStreamFrom()` opens, with no adapter anywhere. Python needs a `ServiceCore` and an +`AsyncServiceCore`; Node needs one, and it delegates rather than bridges. + +**Ownership is free, not implemented.** `Runtime.close()` is a documented no-op that never touches +its terminal transport (PIPE-27), so "borrowed" costs no bookkeeping — the executor closes the +transport it built the preset around, and nothing else. Both close semantics are asserted in the +canary. + +**Verdict — belongs in core**, as `packages/core/src/codegen/service-core.ts`. It is the smallest of +the four gaps to lift and the one every service SDK would otherwise copy verbatim. + +### 2. No declarative status-to-error map — confirmed + +`decodeSuccessResponse` routes every 4xx/5xx through `toHttpError`, which produces `HttpStatusError` +and nothing else. `src/errors.ts` is the local `StatusErrorMap`: a `ReadonlyMap<number, ctor>` plus a +fallback, validated at construction, applied by `remapStatusError` in the executor's `catch`. + +Two things the spike learned that the issue did not anticipate: + +- **The Node disjointness rule is nearly structural.** Python must enforce that a mapped class + extends `HttpResponseError` and does not extend `OSError`, because both are reachable by + multiple inheritance. Node's tree is `DexpaceError` -> {`HttpStatusError`, `IoError`, + `TransportFailureError`, ...} and single inheritance makes overlap impossible. The check is still + worth having — nothing stops a caller mapping a 404 to an `IoError` subclass — but it is one + `instanceof` on the prototype, not a lattice walk. +- **The re-map is post-hoc, and that is lossy.** By the time `remapStatusError` runs, `toHttpError` + has already drained the body into its bounded buffer and closed the response. A mapped error class + can therefore only ever see what `HttpStatusError` kept: status, media type, and up to 1 MiB of + bytes. If the map lived in core it could construct the typed error **at the drain site**, and a + service error class could be handed a decoded error payload rather than raw bytes. + +**Verdict — belongs in core**, and specifically at the `toHttpError` call site rather than wrapped +around it, so the second point above stops being a limitation. + +### 3. `OperationDescriptor` merges the static and per-call halves — confirmed, and the fix is additive + +Four of `OperationDescriptor`'s six fields (`pathParams`, `query`, `headers`, `body`) change per +call, so it cannot be the module-level constant an operation table needs, and it has no slot for an +operation's declared auth. `src/operation.ts` splits it: + +```ts +Operation = {name, method, pathTemplate, auth?} // frozen once, at module load +OperationInput = {pathParams?, query?, headers?, body?} // per call +assemble(op, input) -> OperationDescriptor // two lines +``` + +**The compatibility question in the issue resolves in favour of "no break".** `Operation & +OperationInput` is exactly `OperationDescriptor` plus `name` and `auth`. Core can introduce both +halves and re-express `OperationDescriptor` as their union without touching a single published +signature; `buildRequest(baseUrl, operation)` keeps its exact shape and every existing caller keeps +compiling. + +**Verdict — belongs in core**, as a purely additive reshape. No deprecation, no major version. + +### 4. The `operation` auth tier has no source — confirmed, and now measured + +The deferral register recorded the `operation` tier as **BLOCKED — no source layer exists on this +roadmap**. This spike is that layer, and here is exactly what its absence costs. (That row was closed +on 2026-09-04 — the Phase 5c decision behind it is settled — and the register itself dissolved the +same day. What this section found was carried over to `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1, which is where the +live gap is now tracked.) + +`AuthTiers` is `perCall ?? operation ?? client`, resolved inside `authStep`. `RequestOptions.auth` +fills `perCall`; the step's own settings fill `client`; nothing fills `operation`. So the executor +folds the operation's descriptor into the `perCall` slot: + +```ts +const auth = call.auth ?? operation?.auth; // service-core.ts, requestOptions() +``` + +Three consequences, all real: + +1. **AUTH-4's precedence chain is reimplemented outside core.** The top two-thirds of it live in a + consumer's executor. Every generated SDK would carry the same two-line `??`. +2. **Core cannot tell the two tiers apart.** Once folded, a caller's genuine per-call override and + an operation's declared requirement occupy the same slot. The executor resolves the collision + before core sees it; core has no way to audit, log, or diagnose which tier actually won. +3. **`AuthTiers.operation` stays dead.** It is a documented public field with no writer anywhere in + the workspace. + +**What works correctly and needed no help:** presence-selects-the-tier. The canary asserts all three +outcomes — the operation tier beating a client `API_KEY` default, an operation with no descriptor +falling back to that default, and a present-but-unsatisfiable `OAUTH2` requirement raising +`AuthResolutionError` **with `transport.calls` still empty**. AUTH-4/AUTH-5/AUTH-6 are mechanically +right; only the plumbing is missing. + +**Verdict — the smallest useful fix is a second per-call slot.** Either `RequestOptions` gains +`operationAuth?: AuthDescriptor` (filling `AuthTiers.operation` in `effectiveTiers`), or +`StepContext.options` carries the operation descriptor separately. Either makes the fold above +disappear and `AuthTiers.operation` live. This is the fix `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1 carries. + +### 5. `FakeTransport` and `countingResponse` are unreachable — confirmed + +They live at `packages/core/src/testing/fake-transport.ts` and are absent from the package barrel. +Reaching them means deep-importing `packages/core/src/` while everything else resolves +`@dexpace/core` to `packages/core/dist/` — two copies of core, two `HttpStatusError` classes, every +cross-boundary `instanceof` silently false. The example wrote its own instead: `src/fake-transport.ts`, +129 lines including its own body-draining helper, because `Body` exposes `writeTo(sink)` and no byte +accessor. + +**Verdict — worth deciding, and the answer is probably a separate package.** Exporting the testing +helpers from `@dexpace/core`'s barrel puts test doubles in every production bundle and makes them +API-report surface with a compatibility promise. A `@dexpace/testing` package with core as a peer +dependency gets the sharing without either cost. Doing nothing is also defensible: the fake is 30 +mechanical lines, and every consumer writing their own is not a crisis. + +**A companion positive finding.** Python needs a `_PetPageStrategy` wrapper class that re-decodes +each raw page item, because its `CursorStrategy` is configured by wire field names and yields raw +documents. Node's `cursorStrategy` takes an `extract` callback instead, so the decode happens inside +it and the wrapper has **no twin here**. `support.ts`'s `PET_PAGE_STRATEGY` is one call. + +--- + +## New gaps, found while building + +### 6. There is no encode witness — `Schema<T>` is decode-only + +`Schema<T>` is `{parse(input: unknown): T}`. Nothing in the seam goes the other way. Python's +`Codec` is bidirectional: `_CODEC.encode(model)` produces the wire document, so its `json_body(model)` +is generic over every model. + +Node's `serdeBody(value, serde, mediaType)` encodes **whatever object it is handed**, with no field +mapping. So a model whose field names differ from its wire names — `petId` vs `pet_id`, `weightKg` +vs `weight_kg`, which is every real API — needs a hand-written projection per model: + +```ts +export function petPatchToWire(patch: PetPatch): Readonly<Record<string, unknown>> { + return {name: patch.name, tag: patch.tag, weight_kg: patch.weightKg}; +} +``` + +A generator can emit these — it knows both names from `components/schemas`. But there is no seam in +core to hang them on, so today the generated facade has to name a hand-written symbol from the +service's own shim, which is exactly what `client.ts` does. + +**Verdict — not core's job to solve, but core should state the shape.** An `Encoder<T>` mirror of +`Schema<T>`, or a `Codec<T> = {parse; toWire}` pair, would give a generator one thing to emit +instead of a naming convention between two files. + +### 7. `gts lint .` DOES reach `examples/` — the isolation claim was four-fifths right + +The plan's isolation list named `bunfig.toml`, `verify-test-partition.mjs`, the tsconfig projects and +api-extractor. All four hold. It missed **Lint**, which is a blocking CI step and runs `gts lint .` +from the repository root over every file in the tree. + +Two consequences, both handled here rather than by editing shared config: + +- **The example needs its own `tsconfig.json`.** `eslint.config.js` runs the type-aware tier with + `projectService: true`, which resolves each `.ts` file against the nearest enclosing tsconfig. With + none, lint fails with *"was not found by the project service"*. +- **Generated output has to be Prettier-clean**, because formatting is an error, not a warning. + Predicting Prettier's line breaking from a string-concatenating renderer is not viable, so + `generate.mjs` formats its own output through the same `gts/.prettierrc.json` that + `eslint.config.js` feeds the `prettier/prettier` rule, resolved the same way. The cost is that a + Prettier upgrade can change the checked-in bytes — and `regen.test.ts` is what says so. + +This is worth writing down for the next spike that assumes `examples/` is invisible. It is invisible +to four gates and fully visible to the fifth. + +### 8. `Paginator` has no status-mapping hook + +The engine hands every response to the strategy regardless of status. A mid-walk 500 therefore +reaches `extract`, fails schema validation, and surfaces as a `DeserializationError` — never as the +`StatusErrorMap`'s typed error, because the executor's mapping wraps `execute`, not the walk. + +A generated SDK cannot fix this without putting status handling into every strategy, which is the +duplication `StatusErrorMap` exists to prevent. Whatever shape finding 2 takes in core, `Paginator` +needs the same treatment — most cheaply as an optional `onErrorStatus` hook on `PaginatorInit`, or +by having the engine reject on a non-2xx before the strategy is consulted. + +Not exercised by the canary: the spike records the gap rather than asserting the current behaviour, +because asserting it would pin a shape that should change. + +### 9. `max-params: 3` bites a generated facade + +`updatePet(petId, patch, call)` is already at the repository's cap. Two path parameters plus a body +plus a call bag is four, and a real API has plenty of those. A generator targeting this repository's +lint rules must emit a single options object per method — or generated code needs an exemption. +Worth deciding before a generator exists, because it changes every rendered signature. + +### 10. The document's scheme vocabulary needs a mapping table + +`AuthScheme` is a closed union (`'OAUTH2' | 'API_KEY' | 'BASIC' | 'DIGEST' | 'NO_AUTH'`). The frozen +document says `bearer` and `apikey`. `generate.mjs` carries `SCHEME_BY_SPEC_NAME` and fails at +generation time on an unmapped name, which is the right time to fail. A real generator would derive +it from `components/securitySchemes` instead — but the closed union means the mapping is +**mandatory**, not a convenience, and it belongs in whatever codegen contract core publishes. + +### 11. No sync/async parity gate analogue — confirmed, nothing to port + +Node is async-only. One facade, no mode switch in the generator, no AST-normalising parity check. +Python's `tools/parity_check.py` and `test_petstore_parity.py` have no twin and need none. + +--- + +## Recommendation + +If `packages/core/src/codegen/` is built, it should contain, in descending order of value: + +| What | Why | Size | +|---|---|---| +| `Operation` / `OperationInput` / `assemble` | Finding 3. Purely additive; unblocks a real operation table. | ~40 lines | +| The `operation` auth-tier slot | Finding 4. Closes a register row that has been blocked since Phase 5c, and stops AUTH-4 being reimplemented per SDK. | ~15 lines, plus a `RequestOptions` field | +| `StatusErrorMap` + its application at the `toHttpError` site | Finding 2. Removes a hand-written `if (status === ...)` chain from every service SDK, and fixes the post-hoc losses. | ~70 lines | +| `ServiceCore` | Finding 1. The most code, the least judgement — a delegation layer over surfaces that already compose. | ~90 lines | +| A `Paginator` status hook | Finding 8. Without it, finding 2's fix has a hole exactly the width of a paginated endpoint. | ~10 lines | + +Findings 5, 6, 9 and 10 are decisions rather than code: whether testing helpers get a package, +whether the serde seam gains an encode half, whether generated code is exempt from `max-params`, and +where the scheme mapping is published. + +**What this spike deliberately did not do:** design any of those APIs. The issue's sequencing was +right — the executor was built first, against the core as it stands, so the shape of each gap is now +measured rather than guessed. diff --git a/examples/petstore/README.md b/examples/petstore/README.md new file mode 100644 index 0000000..7bebbcf --- /dev/null +++ b/examples/petstore/README.md @@ -0,0 +1,63 @@ +# Petstore codegen canary + +A **throwaway spike**, not a shipped example. It answers one question for +[issue #64](https://github.com/dexpace/nodejs-sdk/issues/64): does the Python SDK's codegen contract +port onto `@dexpace/core` as it stands, and what exactly is missing? + +The answer is [FINDINGS.md](./FINDINGS.md). Read that first — this file is only how to run it. + +Nothing here is a workspace package, nothing is published, and no CI step runs any of it. It lives +outside `packages/` and `tests/` on purpose. One gate does see it — `bun run lint` — which is +[finding 7](./FINDINGS.md#7-gts-lint--does-reach-examples--the-isolation-claim-was-four-fifths-right). + +## What it is + +A frozen OpenAPI document, a deterministic generator, a projection-only facade, a hand-written +executor, and an end-to-end canary over an in-memory transport. The document in `spec/` is +byte-identical to the Python witness's, so the same fixture drives both ports. + +The generator emits **data and delegation, never logic**: an operation table plus a facade whose +every method binds arguments into an `OperationInput` and calls the shared `ServiceCore`. Everything +behavioural — request assembly, pipeline, retry, auth resolution, error mapping, pagination, SSE — +stays in `@dexpace/core` or in the executor the spike was written to measure. + +## Layout + +| Path | What | +|---|---| +| `spec/petstore.openapi.json` | The frozen document. Never edited by anything here. | +| `generate.mjs` | Renders `src/_generated/`. Deterministic, Prettier-formatted. | +| `src/models.ts` | Hand-written models plus a `Schema<T>` per model. | +| `src/operation.ts` | The `Operation` / `OperationInput` split core does not have. | +| `src/errors.ts` | Typed errors plus the local `StatusErrorMap`. | +| `src/support.ts` | The binders the generated facade names. | +| `src/service-core.ts` | The executor — the payload of the spike. | +| `src/fake-transport.ts` | A local in-memory `Transport`. | +| `src/_generated/` | **Generated. Never hand-edit** — `regen.test.ts` fails if you do. | + +## Running it + +From the repository root, after `bun install --frozen-lockfile`: + +```bash +bun run build # required: the example resolves core via dist/ +node examples/petstore/generate.mjs # rewrite src/_generated/ +bun test ./examples/petstore # canary + regen guard +bunx tsc -p examples/petstore/tsconfig.json --noEmit +bunx eslint examples/ +``` + +`bun test ./examples/petstore` needs the `./` prefix — a bare `examples/petstore` is treated as a +test-name filter and matches nothing. + +## Regenerating + +`src/_generated/` is checked in and byte-compared against a fresh render on every test run. To +change what is generated, edit `generate.mjs` (or the frozen document), then: + +```bash +node examples/petstore/generate.mjs && bun test ./examples/petstore +``` + +A Prettier upgrade can also move the bytes, since the generator formats its output through +`gts/.prettierrc.json`. The regen test is what tells you to re-run the script. diff --git a/examples/petstore/canary.test.ts b/examples/petstore/canary.test.ts new file mode 100644 index 0000000..c46aac0 --- /dev/null +++ b/examples/petstore/canary.test.ts @@ -0,0 +1,365 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/canary.test.ts +// End-to-end canary for the generated petstore SDK against the head core, over an in-memory +// transport. Each scenario proves one certified capability reaches a caller THROUGH the generated +// facade rather than through a hand-written call: +// +// PAGE-1/PAGE-10/PAGE-16 listPets walks two cursor pages, splices `?cursor=`, honours maxPages +// SSE-33/SSE-34 watchPets maps frames and stops on the `[DONE]` sentinel +// BODY-30/HTTP-52 a 404 and a 500 arrive as the status map's typed errors +// AUTH-4/AUTH-5/AUTH-6 the operation tier beats the client default, falls back when absent, +// and fails loudly — with no request sent — when present-but-unsatisfiable +// SERDE-15/SERDE-19 a merge-patch body carries Absent / Null / Present intact +// SEAM-14/PIPE-27 an owned transport is closed; a borrowed runtime is not +// +// Run with `bun test ./examples/petstore` after `bun run build`. NOT part of `bun run test`. +import {expect, test} from 'bun:test'; +import { + ApiKeyCredential, + AuthResolutionError, + createAuthDescriptor, + createAuthRequirement, + createBearerToken, + isAbsent, + isNull, + isPresent, + nullValue, + present, + standardResilience, + type ApiKeyCredentialConfig, + type AuthStepSettings, + type BearerCredential, +} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; +import {PetStoreClient} from './src/_generated/client.js'; +import { + PETSTORE_ERRORS, + PetNotFoundError, + PetStoreError, + type StatusErrorMap, +} from './src/errors.js'; +import { + LocalFakeTransport, + readBodyBytes, + type ScriptedReply, +} from './src/fake-transport.js'; +import { + PET_PATCH_SCHEMA, + emptyPetPatch, + type Pet, + type PetEvent, + type PetPatch, +} from './src/models.js'; +import {ServiceCore} from './src/service-core.js'; + +const BASE = 'https://api.example.com'; +const SERDE = jsonSerde(); +const DECODER = new TextDecoder(); + +const API_KEY: ApiKeyCredentialConfig = { + credential: new ApiKeyCredential('k-123'), + headerName: 'X-Api-Key', +}; + +const BEARER: BearerCredential = { + provider: () => Promise.resolve(createBearerToken('t-abc')), +}; + +/** + * A client-tier default of `API_KEY`, with the bearer credential present or absent. + * + * Absent is the unsatisfiable case: `getPet` declares an `OAUTH2` requirement, AUTH-4 selects the + * tier by PRESENCE, and AUTH-5 judges satisfiability on configured credentials — so the call must + * fail rather than quietly fall through to the satisfiable `API_KEY` default below it. + */ +function authSettings(withBearer: boolean): AuthStepSettings { + return { + credentials: withBearer + ? {apiKey: API_KEY, bearer: BEARER} + : {apiKey: API_KEY}, + tiers: {client: createAuthDescriptor([createAuthRequirement('API_KEY')])}, + }; +} + +interface Harness { + readonly client: PetStoreClient; + readonly transport: LocalFakeTransport; +} + +function harness( + script: readonly ScriptedReply[], + options: { + readonly auth?: AuthStepSettings | undefined; + readonly errors?: StatusErrorMap | undefined; + } = {}, +): Harness { + const transport = new LocalFakeTransport(script); + const core = new ServiceCore({ + baseUrl: BASE, + transport, + serde: SERDE, + resilience: options.auth === undefined ? undefined : {auth: options.auth}, + errors: options.errors, + }); + return {client: new PetStoreClient(core), transport}; +} + +function petJson(id: string, name: string): string { + return JSON.stringify({id, name, tag: null}); +} + +const PAGE_ONE = JSON.stringify({ + data: [ + {id: '1', name: 'a', tag: null}, + {id: '2', name: 'b', tag: null}, + ], + next_cursor: 'c2', +}); + +const PAGE_TWO = JSON.stringify({ + data: [{id: '3', name: 'c', tag: null}], + next_cursor: null, +}); + +function sseBody(): string { + const frames = [ + JSON.stringify({kind: 'created', pet_id: '1'}), + JSON.stringify({kind: 'updated', pet_id: '1'}), + '[DONE]', + ]; + return frames.map(frame => `data: ${frame}\n\n`).join(''); +} + +function namedPatch(): PetPatch { + return {...emptyPetPatch(), name: present('Rex')}; +} + +// -------------------------------------------------------------------------------------------- +// paginate +// -------------------------------------------------------------------------------------------- + +test('listPets walks two cursor pages and yields typed pets', async () => { + const {client, transport} = harness([ + {status: 200, body: PAGE_ONE}, + {status: 200, body: PAGE_TWO}, + ]); + const pets: Pet[] = []; + for await (const pet of client.listPets().items()) pets.push(pet); + + expect(pets.map(pet => pet.name)).toEqual(['a', 'b', 'c']); + expect(transport.calls).toHaveLength(2); + // PAGE-16: the cursor is spliced onto the request that produced the page, not re-derived. + expect(transport.calls[1]?.request.url.search).toBe('?cursor=c2'); + await client.close(); +}); + +test('maxPages caps the walk at one exchange', async () => { + const {client, transport} = harness([ + {status: 200, body: PAGE_ONE}, + {status: 200, body: PAGE_TWO}, + ]); + const pets: Pet[] = []; + for await (const pet of client.listPets({maxPages: 1}).items()) + pets.push(pet); + + expect(pets.map(pet => pet.name)).toEqual(['a', 'b']); + expect(transport.calls).toHaveLength(1); + await client.close(); +}); + +// -------------------------------------------------------------------------------------------- +// events +// -------------------------------------------------------------------------------------------- + +test('watchPets yields mapped events and stops on the [DONE] sentinel', async () => { + const {client, transport} = harness([ + { + status: 200, + body: sseBody(), + headers: {'content-type': 'text/event-stream'}, + }, + ]); + const events: PetEvent[] = []; + for await (const event of client.watchPets()) events.push(event); + + expect(events).toEqual([ + {kind: 'created', petId: '1'}, + {kind: 'updated', petId: '1'}, + ]); + expect(transport.calls).toHaveLength(1); + await client.close(); +}); + +test('a failure status never reaches the SSE parser', async () => { + const {client} = harness([{status: 404, body: '{"message":"nope"}'}], { + errors: PETSTORE_ERRORS, + }); + const iterate = async (): Promise<void> => { + for await (const event of client.watchPets()) { + throw new Error(`expected no event, got ${event.kind}`); + } + }; + + const error = await iterate().catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(PetNotFoundError); + await client.close(); +}); + +// -------------------------------------------------------------------------------------------- +// typed errors through the status map +// -------------------------------------------------------------------------------------------- + +test('a 404 arrives as the mapped PetNotFoundError', async () => { + const {client} = harness([{status: 404, body: '{"message":"nope"}'}], { + errors: PETSTORE_ERRORS, + }); + + const error = await client + .updatePet('7', namedPatch()) + .catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(PetNotFoundError); + expect((error as PetNotFoundError).status).toBe(404); + expect((error as PetNotFoundError).preview).toBe('{"message":"nope"}'); + await client.close(); +}); + +test('an unmapped error status falls back to the table default', async () => { + const {client} = harness([{status: 500, body: 'boom'}], { + errors: PETSTORE_ERRORS, + }); + + // `maxRetries: 0` because a 500 is retryable (RETRY-1/CFG-35) and this test is about the + // mapping, not the budget — without it the walk burns the default attempts and their backoff. + const error = await client + .updatePet('7', namedPatch(), {maxRetries: 0}) + .catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(PetStoreError); + expect(error).not.toBeInstanceOf(PetNotFoundError); + await client.close(); +}); + +// -------------------------------------------------------------------------------------------- +// tiered auth +// -------------------------------------------------------------------------------------------- + +test('the operation tier wins over the client default', async () => { + const {client, transport} = harness( + [{status: 200, body: petJson('7', 'Rex')}], + {auth: authSettings(true)}, + ); + + // getPet declares OAUTH2 in the frozen document; the client default is API_KEY. + await client.getPet('7'); + + const headers = transport.calls[0]?.request.headers; + expect(headers?.get('Authorization')).toBe('Bearer t-abc'); + expect(headers?.get('X-Api-Key')).toBeUndefined(); + await client.close(); +}); + +test('an operation with no declared auth falls back to the client default', async () => { + const {client, transport} = harness( + [{status: 200, body: petJson('7', 'Rex')}], + {auth: authSettings(true)}, + ); + + await client.updatePet('7', namedPatch()); + + const headers = transport.calls[0]?.request.headers; + expect(headers?.get('X-Api-Key')).toBe('k-123'); + expect(headers?.get('Authorization')).toBeUndefined(); + await client.close(); +}); + +test('a present but unsatisfiable tier fails loudly, with no request sent', async () => { + const {client, transport} = harness( + [{status: 200, body: petJson('7', 'Rex')}], + {auth: authSettings(false)}, + ); + + const error = await client.getPet('7').catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(AuthResolutionError); + expect(transport.calls).toHaveLength(0); + await client.close(); +}); + +// -------------------------------------------------------------------------------------------- +// merge-patch three-state round trip +// -------------------------------------------------------------------------------------------- + +test('a merge-patch body carries Absent, Null and Present intact', async () => { + const {client, transport} = harness([ + {status: 200, body: petJson('7', 'Rex')}, + ]); + + // name -> Present, tag -> explicit Null, weightKg -> left Absent. + await client.updatePet('7', { + ...emptyPetPatch(), + name: present('Rex'), + tag: nullValue(), + }); + + const sent = transport.calls[0]?.request.body; + if (sent === undefined) throw new Error('expected the facade to send a body'); + expect(sent.mediaType).toBe('application/merge-patch+json'); + + const document: unknown = JSON.parse( + DECODER.decode(await readBodyBytes(sent)), + ); + // The Absent key is gone; the Null one survives as a wire null (SERDE-15). + expect(document).toEqual({name: 'Rex', tag: null}); + + const decoded = PET_PATCH_SCHEMA.parse(document); + expect(isPresent(decoded.name) && decoded.name.value).toBe('Rex'); + expect(isNull(decoded.tag)).toBe(true); + expect(isAbsent(decoded.weightKg)).toBe(true); + await client.close(); +}); + +// -------------------------------------------------------------------------------------------- +// lifecycle +// -------------------------------------------------------------------------------------------- + +test('closing a core that owns its transport closes it exactly once', async () => { + const transport = new LocalFakeTransport([{status: 200, body: '{}'}]); + const client = new PetStoreClient( + new ServiceCore({baseUrl: BASE, transport, serde: SERDE}), + ); + + await client.close(); + + expect(transport.closeCount).toBe(1); +}); + +test('closing a core that borrows a runtime leaves the transport alone', async () => { + const transport = new LocalFakeTransport([{status: 200, body: '{}'}]); + const runtime = standardResilience(transport); + const client = new PetStoreClient( + new ServiceCore({baseUrl: BASE, runtime, serde: SERDE}), + ); + + await client.close(); + + expect(transport.closeCount).toBe(0); +}); + +test('a core needs exactly one of transport or runtime', () => { + const transport = new LocalFakeTransport([{status: 200}]); + + expect(() => new ServiceCore({baseUrl: BASE, serde: SERDE})).toThrow( + TypeError, + ); + expect( + () => + new ServiceCore({ + baseUrl: BASE, + serde: SERDE, + transport, + runtime: standardResilience(transport), + }), + ).toThrow(TypeError); +}); diff --git a/examples/petstore/generate.d.mts b/examples/petstore/generate.d.mts new file mode 100644 index 0000000..e9b6b2a --- /dev/null +++ b/examples/petstore/generate.d.mts @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/generate.d.mts +/** + * Types for `generate.mjs`, hand-written because the generator is a plain ESM script. + * + * The alternative — `allowJs` in `tsconfig.json` — types `renderAll()` as + * `Promise<Map<string, any>>`, and `any` then flows into the regen test where + * `strictTypeChecked`'s `no-unsafe-*` rules reject it. Declaring the surface is both smaller and + * honest about what the script exports. + */ + +/** Parse the frozen OpenAPI document. */ +export declare function loadSpec(specPath?: string): unknown; + +/** Every operation in the document, sorted by `operationId`. */ +export declare function collectOperations(spec: unknown): unknown[]; + +/** Render the operation-table module's text. */ +export declare function renderOperations(ops: unknown[]): string; + +/** Render the facade module's text. */ +export declare function renderClient(ops: unknown[], className: string): string; + +/** Every generated file as `name -> Prettier-formatted content`; writes nothing. */ +export declare function renderAll( + specPath?: string, +): Promise<Map<string, string>>; + +/** Render and write every generated file; returns how many were written. */ +export declare function writeAll( + specPath?: string, + outDir?: string, +): Promise<number>; diff --git a/examples/petstore/generate.mjs b/examples/petstore/generate.mjs new file mode 100644 index 0000000..25b9a52 --- /dev/null +++ b/examples/petstore/generate.mjs @@ -0,0 +1,426 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/generate.mjs +/** + * Deterministic generator for the petstore codegen canary. + * + * Reads the frozen OpenAPI document (`spec/petstore.openapi.json`) — byte-identical to the one the + * Python witness uses — and renders the checked-in output under `src/_generated/`: + * + * - `operations.ts` — the operation table, pure `Operation` data; + * - `client.ts` — the projection-only facade. ONE facade, not two: Node is async-only, so the + * sync/async mode switch the Python generator carries has nothing to switch on and the parity + * gate that compares the two has no twin here. + * + * The output is deterministic — operations sorted by id, a fixed import order, no timestamps — so + * re-running reproduces the checked-in files byte for byte. `regen.test.ts` asserts exactly that. + * + * node examples/petstore/generate.mjs + * + * `renderAll()` is the pure entry point (name -> content) the regen test compares against the tree + * without touching the filesystem. + * + * **The rendered text is run through Prettier before it is returned.** Predicting Prettier's line + * breaking by hand is a losing game, and `gts lint .` at the repository root DOES lint + * `examples/` — formatting is an error there, not a warning. So the generator formats with the + * exact options `eslint.config.js` feeds the `prettier/prettier` rule, resolved from the same + * `gts/.prettierrc.json`. One consequence worth knowing: a Prettier upgrade can change the + * checked-in bytes, and the regen test is what tells you to re-run this script. + */ +import {readFileSync, writeFileSync} from 'node:fs'; +import {createRequire} from 'node:module'; +import {dirname, join} from 'node:path'; +import process from 'node:process'; +import {fileURLToPath, pathToFileURL} from 'node:url'; + +const require = createRequire(import.meta.url); + +/** The same file `eslint.config.js` sources, resolved the same way. */ +const PRETTIER_RC_PATH = require.resolve('gts/.prettierrc.json'); +const PRETTIER_OPTIONS = require(PRETTIER_RC_PATH); + +const HERE = dirname(fileURLToPath(import.meta.url)); +const SPEC_PATH = join(HERE, 'spec', 'petstore.openapi.json'); +const OUT_DIR = join(HERE, 'src', '_generated'); + +/** HTTP methods recognised in a path item, in OpenAPI order. */ +const HTTP_METHODS = ['get', 'put', 'post', 'delete', 'patch']; + +/** Matches a single `{name}` path placeholder. */ +const PLACEHOLDER = /\{([^{}]+)\}/g; + +/** + * The document's scheme vocabulary, mapped onto `AuthScheme` — core's closed union. + * + * The frozen document says `bearer`; core says `OAUTH2`. A real generator needs this table (or a + * `securitySchemes` walk that produces it), because `AuthScheme` is deliberately closed and an + * unmapped name is a generation-time failure rather than a runtime one. + */ +const SCHEME_BY_SPEC_NAME = { + apikey: 'API_KEY', + basic: 'BASIC', + bearer: 'OAUTH2', + digest: 'DIGEST', +}; + +/** Line 1 is NFR-13's SPDX marker, line 2 the repository's file-path comment convention. */ +function header(relativePath) { + return `// SPDX-License-Identifier: MIT\n// ${relativePath}`; +} + +/** Lazily-resolved Prettier, reached through gts so no root dependency is added. */ +let prettierPromise; + +function prettier() { + prettierPromise ??= import( + pathToFileURL(createRequire(PRETTIER_RC_PATH).resolve('prettier')).href + ).then(module => module.default ?? module); + return prettierPromise; +} + +/** `get_pet` -> `getPet`. */ +function camel(snake) { + return snake.replace(/_([a-z0-9])/g, (_match, ch) => ch.toUpperCase()); +} + +/** `get_pet` -> `GET_PET`. */ +function constantCase(snake) { + return snake.toUpperCase(); +} + +/** `PetPatch` -> `PET_PATCH`. */ +function constantCaseOfModel(pascal) { + return pascal.replace(/(?<!^)([A-Z])/g, '_$1').toUpperCase(); +} + +/** `PetPatch` -> `petPatch`. */ +function lowerCamelOfModel(pascal) { + return pascal.charAt(0).toLowerCase() + pascal.slice(1); +} + +/** Read the frozen document. */ +export function loadSpec(specPath = SPEC_PATH) { + return JSON.parse(readFileSync(specPath, 'utf8')); +} + +function optionalString(value) { + return value === undefined || value === null ? undefined : String(value); +} + +/** Normalise the `auth` extension into `{scheme, scopes}` records, schemes already mapped. */ +function authOf(value) { + if (!Array.isArray(value)) return []; + return value.map(alternative => { + const specName = String(alternative.scheme); + const scheme = SCHEME_BY_SPEC_NAME[specName]; + if (scheme === undefined) { + throw new Error(`unmapped auth scheme "${specName}" in the document`); + } + return {scheme, scopes: (alternative.scopes ?? []).map(String)}; + }); +} + +/** The single request-body media type, or undefined when the operation declares no body. */ +function bodyMediaTypeOf(entry) { + const content = entry.requestBody?.content; + if (content === undefined) return undefined; + const types = Object.keys(content).sort(); + if (types.length !== 1) { + throw new Error( + `expected exactly one request-body media type, got ${String(types.length)}`, + ); + } + return types[0]; +} + +/** Normalise one path-item operation into the record the renderers read. */ +function opFromEntry(path, method, entry) { + const ext = entry['x-dexpace'] ?? {}; + const body = ext.body ?? {}; + return { + operationId: String(entry.operationId), + summary: String(entry.summary ?? '') + .replace(/\s+/g, ' ') + .trim(), + method: method.toUpperCase(), + path, + kind: String(ext.kind), + pathParams: [...path.matchAll(PLACEHOLDER)].map(match => match[1]), + returns: optionalString(ext.returns), + bodyParam: optionalString(body.param), + bodyModel: optionalString(body.model), + bodyMediaType: bodyMediaTypeOf(entry), + itemModel: optionalString(ext.item_model), + strategy: optionalString(ext.strategy), + eventModel: optionalString(ext.event_model), + mapper: optionalString(ext.mapper), + auth: authOf(ext.auth), + }; +} + +/** Every operation in the document, sorted by id so rendering is stable. */ +export function collectOperations(spec) { + const ops = []; + for (const [path, item] of Object.entries(spec.paths ?? {})) { + for (const method of HTTP_METHODS) { + const entry = item[method]; + if (entry !== undefined) ops.push(opFromEntry(path, method, entry)); + } + } + return ops.sort((a, b) => (a.operationId < b.operationId ? -1 : 1)); +} + +const OPERATIONS_DOC = `/** + * Operation table for the petstore canary — GENERATED; do not edit. + * + * Rendered from \`examples/petstore/spec/petstore.openapi.json\` by + * \`examples/petstore/generate.mjs\`. Pure data: one frozen \`Operation\` per \`operationId\`, in + * id-sorted order. Re-render with \`node examples/petstore/generate.mjs\`. + */`; + +function authConstName(op) { + return `${constantCase(op.operationId)}_AUTH`; +} + +function renderRequirement(requirement) { + if (requirement.scopes.length === 0) { + return `createAuthRequirement('${requirement.scheme}')`; + } + const scopes = requirement.scopes.map(scope => `'${scope}'`).join(', '); + return `createAuthRequirement('${requirement.scheme}', [${scopes}])`; +} + +/** Render the operation-table module. */ +export function renderOperations(ops) { + const withAuth = ops.filter(op => op.auth.length > 0); + const lines = [ + header('examples/petstore/src/_generated/operations.ts'), + OPERATIONS_DOC, + '', + ]; + if (withAuth.length > 0) { + lines.push( + "import {createAuthDescriptor, createAuthRequirement} from '@dexpace/core';", + ); + } + lines.push("import type {Operation} from '../operation.js';", ''); + for (const op of withAuth) { + const requirements = op.auth.map(renderRequirement).join(', '); + lines.push( + `const ${authConstName(op)} = createAuthDescriptor([${requirements}]);`, + '', + ); + } + for (const op of ops) { + lines.push(`/** \`${op.method} ${op.path}\` — ${op.summary} */`); + lines.push( + `export const ${constantCase(op.operationId)}: Operation = Object.freeze<Operation>({`, + `name: '${op.operationId}',`, + `method: '${op.method}',`, + `pathTemplate: '${op.path}',`, + ); + if (op.auth.length > 0) lines.push(`auth: ${authConstName(op)},`); + lines.push('});', ''); + } + return lines.join('\n'); +} + +const CLIENT_DOC = `/** + * The petstore facade — GENERATED; do not edit. + * + * Rendered from \`examples/petstore/spec/petstore.openapi.json\` by + * \`examples/petstore/generate.mjs\`. Projection only: every method binds its arguments into an + * \`OperationInput\` and delegates to the shared \`ServiceCore\`, and carries no logic of its own. + * + * ONE facade, not two — Node is async-only, so the sync/async split the Python witness renders (and + * the AST-parity gate that keeps the two honest) has nothing to correspond to here. + */`; + +/** The runtime schema constant a model's decode witness is named by, in `models.ts`. */ +function schemaConst(model) { + return `${constantCaseOfModel(model)}_SCHEMA`; +} + +/** The hand-written encoder a body model is projected through, in `support.ts`. */ +function encoderName(model) { + return `${lowerCamelOfModel(model)}ToWire`; +} + +function sortedUnique(values) { + return [...new Set(values.filter(value => value !== undefined))].sort(); +} + +/** Render the facade's import block. Order is fixed, so the output is stable. */ +function renderClientImports(ops) { + const lines = []; + if (ops.some(op => op.kind === 'paginate')) { + lines.push("import type {Paginator} from '@dexpace/core';"); + } + const schemas = sortedUnique(ops.map(op => op.returns)).map(schemaConst); + if (schemas.length > 0) { + lines.push(`import {${schemas.join(', ')}} from '../models.js';`); + } + const models = sortedUnique([ + ...ops.map(op => op.returns), + ...ops.map(op => op.bodyModel), + ...ops.map(op => op.itemModel), + ...ops.map(op => op.eventModel), + ]); + if (models.length > 0) { + lines.push(`import type {${models.join(', ')}} from '../models.js';`); + } + if ( + ops.some(op => op.pathParams.length === 0 && op.bodyParam === undefined) + ) { + lines.push("import {NO_INPUT} from '../operation.js';"); + } + lines.push( + "import type {CallOptions, ServiceCore} from '../service-core.js';", + ); + const support = sortedUnique([ + ...ops.map(op => (op.bodyParam === undefined ? undefined : 'jsonBody')), + ...ops.map(op => + op.bodyModel === undefined ? undefined : encoderName(op.bodyModel), + ), + ...ops.map(op => op.strategy), + ...ops.map(op => op.mapper), + ]); + if (support.length > 0) { + lines.push(`import {${support.join(', ')}} from '../support.js';`); + } + lines.push("import * as operations from './operations.js';"); + return lines; +} + +/** The `OperationInput` literal for one operation, or `NO_INPUT` when it has nothing to bind. */ +function renderInput(op) { + const parts = []; + if (op.pathParams.length > 0) { + const pairs = op.pathParams + .map(name => `${name}: ${camel(name)}`) + .join(', '); + parts.push(`pathParams: {${pairs}}`); + } + if (op.bodyParam !== undefined && op.bodyModel !== undefined) { + parts.push( + `body: jsonBody(${encoderName(op.bodyModel)}(${camel(op.bodyParam)}), '${op.bodyMediaType}')`, + ); + } + return parts.length === 0 ? 'NO_INPUT' : `{${parts.join(', ')}}`; +} + +/** The declared parameters of a facade method, path params first, then any body, then the bag. */ +function renderParams(op, bagName, bagType) { + const params = op.pathParams.map(name => `${camel(name)}: string`); + if (op.bodyParam !== undefined && op.bodyModel !== undefined) { + params.push(`${camel(op.bodyParam)}: ${op.bodyModel}`); + } + params.push(`${bagName}: ${bagType} = {}`); + return params.join(', '); +} + +function renderUnaryMethod(op) { + const target = `{schema: ${schemaConst(op.returns)}, typeName: '${op.returns}'}`; + return [ + `/** \`${op.method} ${op.path}\` — ${op.summary} */`, + `${camel(op.operationId)}(${renderParams(op, 'call', 'CallOptions')}): Promise<${op.returns}> {`, + `return this.#core.execute(operations.${constantCase(op.operationId)}, ${renderInput(op)}, {...call, responseType: ${target}});`, + '}', + ].join('\n'); +} + +function renderPaginateMethod(op) { + const bagType = 'CallOptions & {maxPages?: number | undefined}'; + return [ + `/** \`${op.method} ${op.path}\` — ${op.summary} */`, + `${camel(op.operationId)}(${renderParams(op, 'paging', bagType)}): Paginator<${op.itemModel}> {`, + `return this.#core.paginate(operations.${constantCase(op.operationId)}, ${renderInput(op)}, {...paging, strategy: ${op.strategy}});`, + '}', + ].join('\n'); +} + +function renderEventsMethod(op) { + return [ + `/** \`${op.method} ${op.path}\` — ${op.summary} */`, + `${camel(op.operationId)}(${renderParams(op, 'streaming', 'CallOptions')}): AsyncIterable<${op.eventModel}> {`, + `return this.#core.events(operations.${constantCase(op.operationId)}, ${renderInput(op)}, {...streaming, mapper: ${op.mapper}});`, + '}', + ].join('\n'); +} + +function renderMethod(op) { + if (op.kind === 'paginate') return renderPaginateMethod(op); + if (op.kind === 'events') return renderEventsMethod(op); + return renderUnaryMethod(op); +} + +/** Render the facade module. */ +export function renderClient(ops, className) { + const blocks = [ + [ + '/** The petstore client — a projection over `ServiceCore`. */', + `export class ${className} {`, + 'readonly #core: ServiceCore;', + '', + 'constructor(core: ServiceCore) {', + 'this.#core = core;', + '}', + ].join('\n'), + ...ops.map(renderMethod), + [ + '/** Releases whatever the executor owns; a borrowed runtime is left alone. */', + 'close(): Promise<void> {', + 'return this.#core.close();', + '}', + ].join('\n'), + ]; + return [ + header('examples/petstore/src/_generated/client.ts'), + CLIENT_DOC, + '', + ...renderClientImports(ops), + '', + blocks.join('\n\n'), + '}', + '', + ].join('\n'); +} + +/** + * Render every generated file as `name -> content`, Prettier-formatted. + * + * Pure beyond reading the frozen document: it writes nothing. + */ +export async function renderAll(specPath = SPEC_PATH) { + const spec = loadSpec(specPath); + const ops = collectOperations(spec); + const className = String( + spec['x-dexpace-codegen']?.client_class ?? 'ApiClient', + ); + const raw = { + 'operations.ts': renderOperations(ops), + 'client.ts': renderClient(ops, className), + }; + const {format} = await prettier(); + const rendered = new Map(); + for (const [name, content] of Object.entries(raw)) { + rendered.set( + name, + await format(content, {...PRETTIER_OPTIONS, parser: 'typescript'}), + ); + } + return rendered; +} + +/** Render and write every generated file into `outDir`. */ +export async function writeAll(specPath = SPEC_PATH, outDir = OUT_DIR) { + const rendered = await renderAll(specPath); + for (const [name, content] of rendered) { + writeFileSync(join(outDir, name), content, 'utf8'); + } + return rendered.size; +} + +if (import.meta.url === pathToFileURL(process.argv[1] ?? '').href) { + const count = await writeAll(); + process.stdout.write(`generated ${String(count)} files into ${OUT_DIR}\n`); +} diff --git a/examples/petstore/regen.test.ts b/examples/petstore/regen.test.ts new file mode 100644 index 0000000..a64b43b --- /dev/null +++ b/examples/petstore/regen.test.ts @@ -0,0 +1,34 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/regen.test.ts +// The regen-diff guard: re-render the frozen document and byte-compare against the checked-in +// `src/_generated/` tree. A hand-edit to a generated file — or a generator change not reflected in +// the checked-in output — fails here, so the canary can only ever be regenerated, never patched. +// +// Run with `bun test ./examples/petstore`. Deliberately NOT part of `bun run test`: the example is +// outside `packages/` and `tests/`, and the root script names only those two trees. +import {readFileSync, readdirSync} from 'node:fs'; +import {dirname, join} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {expect, test} from 'bun:test'; +import {renderAll} from './generate.mjs'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const GENERATED = join(HERE, 'src', '_generated'); + +test('regenerating reproduces the checked-in output byte for byte', async () => { + const rendered = await renderAll(); + expect(rendered.size).toBeGreaterThan(0); + for (const [name, content] of rendered) { + const checkedIn = readFileSync(join(GENERATED, name), 'utf8'); + expect( + content, + `${name} is out of sync; re-run \`node examples/petstore/generate.mjs\` (never hand-edit a generated file)`, + ).toBe(checkedIn); + } +}); + +test('the generator accounts for every file in src/_generated', async () => { + const rendered = [...(await renderAll()).keys()].sort(); + const onDisk = readdirSync(GENERATED).sort(); + expect(rendered).toEqual(onDisk); +}); diff --git a/examples/petstore/spec/petstore.openapi.json b/examples/petstore/spec/petstore.openapi.json new file mode 100644 index 0000000..b13d8d9 --- /dev/null +++ b/examples/petstore/spec/petstore.openapi.json @@ -0,0 +1,115 @@ +{ + "openapi": "3.1.0", + "info": { + "title": "Dexpace Petstore Canary", + "version": "1.0.0", + "description": "A small, frozen OpenAPI document driving the petstore codegen canary. It is a fixture, not a real service: it exercises paginate, SSE events, a merge-patch body with three-valued fields, typed status errors, and a tiered auth requirement end to end through the generated facades." + }, + "x-dexpace-codegen": { + "package": "dexpace_petstore", + "client_class": "PetStoreClient", + "async_client_class": "AsyncPetStoreClient" + }, + "paths": { + "/pets": { + "get": { + "operationId": "list_pets", + "summary": "List pets, paginating by opaque cursor.", + "x-dexpace": { + "kind": "paginate", + "item_model": "Pet", + "strategy": "PET_PAGE_STRATEGY" + }, + "responses": { + "200": {"description": "A page of pets plus the next cursor."} + } + } + }, + "/pets/events": { + "get": { + "operationId": "watch_pets", + "summary": "Stream pet lifecycle events over Server-Sent Events.", + "x-dexpace": { + "kind": "events", + "event_model": "PetEvent", + "mapper": "PET_EVENT_MAPPER" + }, + "responses": { + "200": {"description": "An SSE stream of pet events."} + } + } + }, + "/pets/{pet_id}": { + "get": { + "operationId": "get_pet", + "summary": "Fetch one pet by id.", + "parameters": [ + {"name": "pet_id", "in": "path", "required": true, "schema": {"type": "string"}} + ], + "x-dexpace": { + "kind": "unary", + "returns": "Pet", + "auth": [{"scheme": "bearer", "scopes": ["pets:read"]}] + }, + "responses": { + "200": {"description": "The requested pet."}, + "404": {"description": "No pet with that id."} + } + }, + "patch": { + "operationId": "update_pet", + "summary": "Apply a merge-patch update to a pet.", + "parameters": [ + {"name": "pet_id", "in": "path", "required": true, "schema": {"type": "string"}} + ], + "requestBody": { + "required": true, + "content": { + "application/merge-patch+json": { + "schema": {"$ref": "#/components/schemas/PetPatch"} + } + } + }, + "x-dexpace": { + "kind": "unary", + "returns": "Pet", + "body": {"param": "patch", "model": "PetPatch"} + }, + "responses": { + "200": {"description": "The updated pet."}, + "404": {"description": "No pet with that id."} + } + } + } + }, + "components": { + "schemas": { + "Pet": { + "type": "object", + "required": ["id", "name"], + "properties": { + "id": {"type": "string"}, + "name": {"type": "string"}, + "tag": {"type": ["string", "null"]} + } + }, + "PetEvent": { + "type": "object", + "required": ["kind", "pet_id"], + "properties": { + "kind": {"type": "string"}, + "pet_id": {"type": "string"} + } + }, + "PetPatch": { + "type": "object", + "description": "A merge-patch body: an omitted field leaves the target unchanged, an explicit null clears it, and a value sets it.", + "properties": { + "name": {"type": ["string", "null"]}, + "tag": {"type": ["string", "null"]}, + "weight_kg": {"type": ["number", "null"]} + } + } + } + } +} diff --git a/examples/petstore/src/_generated/client.ts b/examples/petstore/src/_generated/client.ts new file mode 100644 index 0000000..0bc2002 --- /dev/null +++ b/examples/petstore/src/_generated/client.ts @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/_generated/client.ts +/** + * The petstore facade — GENERATED; do not edit. + * + * Rendered from `examples/petstore/spec/petstore.openapi.json` by + * `examples/petstore/generate.mjs`. Projection only: every method binds its arguments into an + * `OperationInput` and delegates to the shared `ServiceCore`, and carries no logic of its own. + * + * ONE facade, not two — Node is async-only, so the sync/async split the Python witness renders (and + * the AST-parity gate that keeps the two honest) has nothing to correspond to here. + */ + +import type {Paginator} from '@dexpace/core'; +import {PET_SCHEMA} from '../models.js'; +import type {Pet, PetEvent, PetPatch} from '../models.js'; +import {NO_INPUT} from '../operation.js'; +import type {CallOptions, ServiceCore} from '../service-core.js'; +import { + PET_EVENT_MAPPER, + PET_PAGE_STRATEGY, + jsonBody, + petPatchToWire, +} from '../support.js'; +import * as operations from './operations.js'; + +/** The petstore client — a projection over `ServiceCore`. */ +export class PetStoreClient { + readonly #core: ServiceCore; + + constructor(core: ServiceCore) { + this.#core = core; + } + + /** `GET /pets/{pet_id}` — Fetch one pet by id. */ + getPet(petId: string, call: CallOptions = {}): Promise<Pet> { + return this.#core.execute( + operations.GET_PET, + {pathParams: {pet_id: petId}}, + {...call, responseType: {schema: PET_SCHEMA, typeName: 'Pet'}}, + ); + } + + /** `GET /pets` — List pets, paginating by opaque cursor. */ + listPets( + paging: CallOptions & {maxPages?: number | undefined} = {}, + ): Paginator<Pet> { + return this.#core.paginate(operations.LIST_PETS, NO_INPUT, { + ...paging, + strategy: PET_PAGE_STRATEGY, + }); + } + + /** `PATCH /pets/{pet_id}` — Apply a merge-patch update to a pet. */ + updatePet( + petId: string, + patch: PetPatch, + call: CallOptions = {}, + ): Promise<Pet> { + return this.#core.execute( + operations.UPDATE_PET, + { + pathParams: {pet_id: petId}, + body: jsonBody(petPatchToWire(patch), 'application/merge-patch+json'), + }, + {...call, responseType: {schema: PET_SCHEMA, typeName: 'Pet'}}, + ); + } + + /** `GET /pets/events` — Stream pet lifecycle events over Server-Sent Events. */ + watchPets(streaming: CallOptions = {}): AsyncIterable<PetEvent> { + return this.#core.events(operations.WATCH_PETS, NO_INPUT, { + ...streaming, + mapper: PET_EVENT_MAPPER, + }); + } + + /** Releases whatever the executor owns; a borrowed runtime is left alone. */ + close(): Promise<void> { + return this.#core.close(); + } +} diff --git a/examples/petstore/src/_generated/operations.ts b/examples/petstore/src/_generated/operations.ts new file mode 100644 index 0000000..434b018 --- /dev/null +++ b/examples/petstore/src/_generated/operations.ts @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/_generated/operations.ts +/** + * Operation table for the petstore canary — GENERATED; do not edit. + * + * Rendered from `examples/petstore/spec/petstore.openapi.json` by + * `examples/petstore/generate.mjs`. Pure data: one frozen `Operation` per `operationId`, in + * id-sorted order. Re-render with `node examples/petstore/generate.mjs`. + */ + +import {createAuthDescriptor, createAuthRequirement} from '@dexpace/core'; +import type {Operation} from '../operation.js'; + +const GET_PET_AUTH = createAuthDescriptor([ + createAuthRequirement('OAUTH2', ['pets:read']), +]); + +/** `GET /pets/{pet_id}` — Fetch one pet by id. */ +export const GET_PET: Operation = Object.freeze<Operation>({ + name: 'get_pet', + method: 'GET', + pathTemplate: '/pets/{pet_id}', + auth: GET_PET_AUTH, +}); + +/** `GET /pets` — List pets, paginating by opaque cursor. */ +export const LIST_PETS: Operation = Object.freeze<Operation>({ + name: 'list_pets', + method: 'GET', + pathTemplate: '/pets', +}); + +/** `PATCH /pets/{pet_id}` — Apply a merge-patch update to a pet. */ +export const UPDATE_PET: Operation = Object.freeze<Operation>({ + name: 'update_pet', + method: 'PATCH', + pathTemplate: '/pets/{pet_id}', +}); + +/** `GET /pets/events` — Stream pet lifecycle events over Server-Sent Events. */ +export const WATCH_PETS: Operation = Object.freeze<Operation>({ + name: 'watch_pets', + method: 'GET', + pathTemplate: '/pets/events', +}); diff --git a/examples/petstore/src/errors.ts b/examples/petstore/src/errors.ts new file mode 100644 index 0000000..a3d7e5d --- /dev/null +++ b/examples/petstore/src/errors.ts @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/errors.ts +/** + * The typed error taxonomy for the petstore canary, and the declarative status-to-error map the + * executor consults. + * + * **This whole file is finding 2.** `@dexpace/core` produces exactly one class for a failure + * status: `decodeSuccessResponse` calls `toHttpError`, which returns `HttpStatusError` and nothing + * else. A service SDK that wants `PetNotFoundError` for a 404 therefore has to catch that one + * class and re-map it by status code — which is what {@link remapStatusError} does, and what every + * generated SDK would otherwise reimplement. + * + * The re-map is lossy in one respect worth recording: `toHttpError` has already drained and closed + * the response by the time the mapping runs, so the mapped error is built from the buffered + * `HttpStatusError`, never from the live response. That is fine here — the buffered copy carries + * the status, the media type and a bounded body — but it means a service error class can never see + * anything `HttpStatusError` did not keep. + */ +import { + DexpaceError, + HttpStatusError, + IoError, + TransportFailureError, +} from '@dexpace/core'; + +/** Base class for every typed petstore response error. */ +export class PetStoreError extends DexpaceError { + /** The response status that produced this error. */ + readonly status: number; + /** A bounded, non-consuming preview of the error body; `null` when there was none. */ + readonly preview: string | null; + + constructor(cause: HttpStatusError) { + super(`petstore: HTTP ${String(cause.status)}`, {cause}); + this.status = cause.status; + this.preview = cause.preview(); + } +} + +/** Raised for a 404 — no pet matched the requested id. */ +export class PetNotFoundError extends PetStoreError {} + +/** + * What a status maps to: a class constructible from the `HttpStatusError` core already produced. + * + * Typed against `DexpaceError` rather than `PetStoreError` so {@link createStatusErrorMap}'s + * validation is a real check on a caller-supplied class rather than a restatement of the parameter + * type. + */ +export type StatusErrorConstructor = new ( + cause: HttpStatusError, +) => DexpaceError; + +/** A declarative status-to-error table: the Node shape of Python's `StatusErrorMap`. */ +export interface StatusErrorMap { + /** Exact status matches, most specific. */ + readonly byStatus: ReadonlyMap<number, StatusErrorConstructor>; + /** Applied to any 4xx/5xx the table does not name. */ + readonly fallback: StatusErrorConstructor; +} + +/** + * Reject a mapped class that sits on the TRANSPORT branch of the error tree. + * + * Python enforces the equivalent rule (`HttpResponseError`, never `OSError`) so an + * `except OSError:` site cannot start catching service errors. The Node tree is + * `DexpaceError` -> {`HttpStatusError`, `IoError`, `TransportFailureError`, ...}, and single + * inheritance means a class cannot be on both branches — but nothing stops a caller mapping a 404 + * to a subclass of `IoError`, which is exactly what the rule forbids and what this rejects. + */ +function assertResponseBranch( + ctor: StatusErrorConstructor, + label: string, +): void { + if ( + ctor.prototype instanceof IoError || + ctor.prototype instanceof TransportFailureError + ) { + throw new TypeError( + `${label} is on the transport branch of the error tree; a mapped status error must not be`, + ); + } + if (!(ctor.prototype instanceof DexpaceError)) { + throw new TypeError(`${label} must extend DexpaceError`); + } +} + +/** + * Build a validated {@link StatusErrorMap}. + * + * Validation runs at construction, not per response: a misconfigured table is a programmer error + * and should surface where the table is written, not on the one production request that happens to + * receive the status it got wrong. + */ +export function createStatusErrorMap(init: { + readonly byStatus?: + Readonly<Record<number, StatusErrorConstructor>> | undefined; + readonly fallback: StatusErrorConstructor; +}): StatusErrorMap { + assertResponseBranch(init.fallback, 'the fallback error class'); + const byStatus = new Map<number, StatusErrorConstructor>(); + for (const [key, ctor] of Object.entries(init.byStatus ?? {})) { + assertResponseBranch(ctor, `the error class mapped to status ${key}`); + byStatus.set(Number(key), ctor); + } + return Object.freeze({byStatus, fallback: init.fallback}); +} + +/** + * Re-map an `HttpStatusError` through the table; anything else passes through untouched. + * + * `unknown` in, `unknown` out, so a call site can use it directly in a `catch` without narrowing + * first — and so a transport failure, a `DeserializationError`, or an `AuthResolutionError` reach + * the caller as themselves rather than being laundered into a service error. + */ +export function remapStatusError( + error: unknown, + map: StatusErrorMap | undefined, +): unknown { + if (map === undefined || !(error instanceof HttpStatusError)) return error; + const ctor = map.byStatus.get(error.status) ?? map.fallback; + return new ctor(error); +} + +/** The petstore's own table: a 404 is a `PetNotFoundError`, everything else a `PetStoreError`. */ +export const PETSTORE_ERRORS: StatusErrorMap = createStatusErrorMap({ + byStatus: {404: PetNotFoundError}, + fallback: PetStoreError, +}); diff --git a/examples/petstore/src/fake-transport.ts b/examples/petstore/src/fake-transport.ts new file mode 100644 index 0000000..08af2a0 --- /dev/null +++ b/examples/petstore/src/fake-transport.ts @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/fake-transport.ts +/** + * A local in-memory `Transport` for the canary. + * + * **This file is finding 5's evidence.** `@dexpace/core` already ships `FakeTransport` and + * `countingResponse` at `packages/core/src/testing/fake-transport.ts`, and neither is re-exported + * from the package entry point. Reaching them means deep-importing `packages/core/src/`, while the + * rest of the example resolves `@dexpace/core` to `packages/core/dist/` — two copies of core in one + * process, two `HttpStatusError` classes, and every `instanceof` across the boundary silently + * false. So the example writes its own, exactly as a real consumer would have to. + * + * It is not a hardship: the whole thing is a scripted list and one `send`. What it costs is that + * the fake is unshared, so nothing about it is certified by core's own suite. + */ +import {Headers, Protocol, Request, Response, Status} from '@dexpace/core'; +import type {Body, RequestOptions, Transport} from '@dexpace/core'; + +/** One scripted reply. */ +export interface ScriptedReply { + readonly status: number; + /** The response body as text; omitted means a body-less response. */ + readonly body?: string | undefined; + readonly headers?: Readonly<Record<string, string>> | undefined; +} + +/** One recorded send. */ +export interface RecordedCall { + readonly request: Request; + readonly options: RequestOptions | undefined; + readonly signal: AbortSignal | undefined; +} + +const TEXT_ENCODER = new TextEncoder(); + +function bodyStream(text: string): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(TEXT_ENCODER.encode(text)); + controller.close(); + }, + }); +} + +/** + * Drain a request body into bytes. + * + * `Body` exposes `writeTo(sink)` and no byte accessor, so reading what a facade actually sent means + * supplying a sink. Used by the merge-patch assertion, which has to see the encoded document. + */ +export async function readBodyBytes(body: Body): Promise<Uint8Array> { + const chunks: Uint8Array[] = []; + await body.writeTo( + new WritableStream<Uint8Array>({ + write(chunk: Uint8Array): void { + chunks.push(chunk); + }, + }), + ); + const total = chunks.reduce((sum, chunk) => sum + chunk.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; +} + +/** + * A scripted transport. Entries are served in order; once exhausted the last entry repeats, and a + * FRESH response — with a fresh body stream — is built per call, so a repeated entry is safe to + * consume more than once. + */ +export class LocalFakeTransport implements Transport { + readonly #script: readonly ScriptedReply[]; + readonly #calls: RecordedCall[] = []; + #closeCount = 0; + + constructor(script: readonly ScriptedReply[]) { + if (script.length === 0) { + throw new TypeError( + 'LocalFakeTransport needs at least one scripted reply', + ); + } + this.#script = [...script]; + } + + /** Every send this double served, in order. */ + get calls(): readonly RecordedCall[] { + return this.#calls; + } + + /** How many times `close()` was called — the owned/borrowed assertion reads this. */ + get closeCount(): number { + return this.#closeCount; + } + + send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + const reply = + this.#script[Math.min(this.#calls.length, this.#script.length - 1)]; + this.#calls.push({request, options, signal}); + if (reply === undefined) { + return Promise.reject(new Error('scripted reply index out of range')); + } + const headers = Headers.newBuilder(); + for (const [name, value] of Object.entries(reply.headers ?? {})) { + headers.setInbound(name, value); + } + return Promise.resolve( + Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(reply.status)) + .headers(headers.build()) + .body(reply.body === undefined ? null : bodyStream(reply.body)) + .build(), + ); + } + + close(): Promise<void> { + this.#closeCount += 1; + return Promise.resolve(); + } +} diff --git a/examples/petstore/src/models.ts b/examples/petstore/src/models.ts new file mode 100644 index 0000000..e66b3bc --- /dev/null +++ b/examples/petstore/src/models.ts @@ -0,0 +1,158 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/models.ts +/** + * Hand-written models for the petstore canary, plus a `Schema<T>` witness for each. + * + * Models are deliberately NOT generated — the generator reads `operationId`, the method, the path + * template and the `x-dexpace` extension, and nothing else (the issue's non-goals say so). What is + * written here is what a real generated SDK ships beside its facades. + * + * Two things are worth noticing while reading, because both are findings rather than style: + * + * 1. **Every model needs a decode witness AND a hand-written encode projection.** `Schema<T>` is a + * one-way seam: `parse(input: unknown): T`. There is no encode witness anywhere in core, so a + * model whose field names differ from its wire names — `petId` vs `pet_id`, `weightKg` vs + * `weight_kg` — has to carry a `toWire` function written by hand. See `support.ts`. + * + * 2. **`PetPatch`'s fields are `Tristate`, and that is the whole point of the merge-patch case.** + * Absent means "leave unchanged", Null means "clear", Present means "set". `@dexpace/codec-json` + * carries both halves: `tristateReplacer` on the encode side (installed by `jsonSerde()` by + * default) and `tristateObject` on the decode side. + */ +import {absent, type Schema, type Tristate} from '@dexpace/core'; +import {tristateObject} from '@dexpace/codec-json'; + +/** A pet as the service returns it. */ +export interface Pet { + readonly id: string; + readonly name: string; + /** `null` when the pet carries no tag; the wire field is nullable, not omissible. */ + readonly tag: string | null; +} + +/** One pet lifecycle event, delivered over SSE. Wire field `pet_id` becomes `petId` here. */ +export interface PetEvent { + readonly kind: string; + readonly petId: string; +} + +/** + * A merge-patch update body. + * + * Every field defaults to Absent through {@link emptyPetPatch}, so + * `{...emptyPetPatch(), name: present('Rex')}` sends only `name` and leaves the rest untouched. + */ +export interface PetPatch { + readonly name: Tristate<string>; + readonly tag: Tristate<string>; + readonly weightKg: Tristate<number>; +} + +/** A patch with every field Absent — the identity element a caller spreads over. */ +export function emptyPetPatch(): PetPatch { + return {name: absent(), tag: absent(), weightKg: absent()}; +} + +/** + * Narrow an already-parsed wire value to a JSON object. + * + * An array is `typeof 'object'` and non-null, so the array check is not decoration: without it a + * `[1, 2, 3]` arriving where a DTO was expected is silently reshaped into `{'0': 1, ...}` rather + * than rejected. + */ +function asObject( + input: unknown, + label: string, +): Readonly<Record<string, unknown>> { + if (typeof input !== 'object' || input === null || Array.isArray(input)) { + throw new TypeError(`${label}: expected a JSON object`); + } + return input as Readonly<Record<string, unknown>>; +} + +function requireString( + record: Readonly<Record<string, unknown>>, + key: string, + label: string, +): string { + const value = record[key]; + if (typeof value !== 'string') { + throw new TypeError(`${label}: "${key}" must be a string`); + } + return value; +} + +function nullableString( + record: Readonly<Record<string, unknown>>, + key: string, + label: string, +): string | null { + const value = record[key]; + if (value === undefined || value === null) return null; + if (typeof value !== 'string') { + throw new TypeError(`${label}: "${key}" must be a string or null`); + } + return value; +} + +/** Decode witness for {@link Pet}. */ +export const PET_SCHEMA: Schema<Pet> = Object.freeze({ + parse(input: unknown): Pet { + const record = asObject(input, 'Pet'); + return { + id: requireString(record, 'id', 'Pet'), + name: requireString(record, 'name', 'Pet'), + tag: nullableString(record, 'tag', 'Pet'), + }; + }, +}); + +/** Decode witness for {@link PetEvent}; renames the wire's `pet_id`. */ +export const PET_EVENT_SCHEMA: Schema<PetEvent> = Object.freeze({ + parse(input: unknown): PetEvent { + const record = asObject(input, 'PetEvent'); + return { + kind: requireString(record, 'kind', 'PetEvent'), + petId: requireString(record, 'pet_id', 'PetEvent'), + }; + }, +}); + +const STRING_SCHEMA: Schema<string> = Object.freeze({ + parse(input: unknown): string { + if (typeof input !== 'string') throw new TypeError('expected a string'); + return input; + }, +}); + +const NUMBER_SCHEMA: Schema<number> = Object.freeze({ + parse(input: unknown): number { + if (typeof input !== 'number') throw new TypeError('expected a number'); + return input; + }, +}); + +/** + * The wire-shaped half of {@link PetPatch}: `tristateObject` keys by WIRE name, so the rename to + * `weightKg` happens in {@link PET_PATCH_SCHEMA}'s own `parse` and not in the combinator. + */ +const PET_PATCH_WIRE_SCHEMA = tristateObject({ + name: STRING_SCHEMA, + tag: STRING_SCHEMA, + weight_kg: NUMBER_SCHEMA, +}); + +/** + * Decode witness for {@link PetPatch}. + * + * Only the canary uses it — a service does not normally decode its own request bodies. It is here + * so the merge-patch round trip is asserted on a `PetPatch`, not on a raw JSON document: an + * assertion over the document alone proves the encoder emitted the right bytes but nothing about + * the three states surviving a full round trip. + */ +export const PET_PATCH_SCHEMA: Schema<PetPatch> = Object.freeze({ + parse(input: unknown): PetPatch { + const wire = PET_PATCH_WIRE_SCHEMA.parse(asObject(input, 'PetPatch')); + return {name: wire.name, tag: wire.tag, weightKg: wire.weight_kg}; + }, +}); diff --git a/examples/petstore/src/operation.ts b/examples/petstore/src/operation.ts new file mode 100644 index 0000000..c32e8ee --- /dev/null +++ b/examples/petstore/src/operation.ts @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/operation.ts +/** + * The static/per-call split core does not have. + * + * **This file is finding 3.** `@dexpace/core`'s `OperationDescriptor` carries `method`, + * `pathTemplate`, `pathParams`, `query`, `headers` and `body` in one interface. Four of those six + * change on every call, so a descriptor cannot be the module-level constant a generated operation + * table needs — a generator using it directly would have to build a fresh object per call and + * would have nowhere to hang an operation's declared auth requirement. + * + * Splitting it costs nothing structurally: `Operation & OperationInput` is `OperationDescriptor` + * plus a `name` and an `auth` slot, and {@link assemble} is the two-line merge that proves it. The + * split is additive, so lifting it into core would not break the published `OperationDescriptor` — + * `OperationDescriptor` can stay exactly as it is and be re-expressed as the union of the two + * halves. + */ +import type { + AuthDescriptor, + Body, + Headers, + Method, + OperationDescriptor, + QueryParams, +} from '@dexpace/core'; + +/** + * The half that is fixed when the SDK is generated: everything the frozen OpenAPI document knows. + * + * A generated operation table is a module of these, one per `operationId`, each frozen once at + * module load and reused by every call. + */ +export interface Operation { + /** The `operationId` from the document; carried for diagnostics and tracing, never sent. */ + readonly name: string; + /** The HTTP method. */ + readonly method: Method; + /** The path template, `{name}` placeholders intact. */ + readonly pathTemplate: string; + /** + * The operation's declared auth requirement — AUTH-4's `operation` tier. + * + * Core has the slot (`AuthTiers.operation`) and no source for it; this field is that source. See + * FINDINGS.md, finding 4, for what the executor then has to do with it. + */ + readonly auth?: AuthDescriptor | undefined; +} + +/** + * The half that changes per call: exactly `OperationDescriptor` minus `method` and `pathTemplate`. + * + * `?: T | undefined` rather than `?: T` throughout, because `exactOptionalPropertyTypes` is on and + * a generated facade assigns every field including the ones it has nothing for. + */ +export interface OperationInput { + /** Values for the path template's `{name}` placeholders. */ + readonly pathParams?: Readonly<Record<string, string>> | undefined; + /** Query parameters appended after any the base URL already carries. */ + readonly query?: QueryParams | undefined; + /** Headers carried onto the assembled request as-is. */ + readonly headers?: Headers | undefined; + /** The already-encoded request body. */ + readonly body?: Body | undefined; +} + +/** An empty input — a frozen singleton, since a facade method with no arguments needs one per call. */ +export const NO_INPUT: OperationInput = Object.freeze({}); + +/** + * Merge the two halves back into the descriptor `buildRequest` takes. + * + * The whole of core's assembly seam is reachable through this one line, which is the point: the + * split is a re-shaping of the same data, not a parallel model. + */ +export function assemble( + operation: Operation, + input: OperationInput, +): OperationDescriptor { + return { + method: operation.method, + pathTemplate: operation.pathTemplate, + ...input, + }; +} diff --git a/examples/petstore/src/service-core.ts b/examples/petstore/src/service-core.ts new file mode 100644 index 0000000..1a87d64 --- /dev/null +++ b/examples/petstore/src/service-core.ts @@ -0,0 +1,294 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/service-core.ts +/** + * The executor tier a generated client delegates to. + * + * **This file is finding 1 — it is the payload of the spike.** Nothing in `@dexpace/core` exports + * an object with `execute` / `executeRequest` / `paginate` / `events` plus ownership-aware close, + * so every service SDK would write this. It turns out to be thin, and the reason it is thin is + * worth stating: `Runtime` implements `Transport`, so the pipeline drops straight into `Paginator`, + * into `sseStreamFrom`, and into a plain `send()` with no adapter between them. + * + * What is NOT thin, and is the actual finding, is everything the executor has to decide that core + * does not: + * + * - **Auth tier precedence** ({@link requestOptions}). Core's `AuthTiers` is + * `perCall ?? operation ?? client`, resolved inside `authStep`. Only `perCall` and `client` have + * a source; `RequestOptions.auth` fills `perCall` and the step's own settings fill `client`. So + * an operation's declared descriptor has to be folded into the `perCall` slot HERE, and the + * `call.auth ?? operation.auth` precedence — the top two-thirds of AUTH-4's chain — is + * reimplemented in this file. See FINDINGS.md, finding 4. + * - **Status-to-error mapping** ({@link ServiceCore.execute}). Core produces one class; the map is + * applied on the way out. See finding 2. + * - **Which failures reach a stream** ({@link ServiceCore.events}). `sseStreamFrom` does not look + * at the status, so a 404 would be parsed as an event stream unless the executor checks first. + * + * And one thing that is genuinely free: **ownership**. `Runtime.close()` is a documented no-op that + * never touches its terminal transport (PIPE-27), so "borrowed" needs no bookkeeping — the executor + * closes only what it built itself. + */ +import { + DexpaceError, + Paginator, + RequestOptions, + buildRequest, + decodeSuccessResponse, + sseStreamFrom, + standardResilience, + toHttpError, + typedSseStream, +} from '@dexpace/core'; +import type { + AuthDescriptor, + DecodeTarget, + PaginationStrategy, + Request, + Response, + Runtime, + Serde, + SseMapper, + SseStream, + StandardResilienceOptions, + Transport, +} from '@dexpace/core'; +import {assemble, type Operation, type OperationInput} from './operation.js'; +import {remapStatusError, type StatusErrorMap} from './errors.js'; + +/** Per-call overrides every entry point accepts. */ +export interface CallOptions { + /** AUTH-4's `perCall` tier. Beats the operation's own descriptor, which beats the client default. */ + readonly auth?: AuthDescriptor | undefined; + /** Per-call timeout, threaded through `RequestOptions`. */ + readonly timeoutMs?: number | undefined; + /** Per-call retry cap, threaded through `RequestOptions`. */ + readonly maxRetries?: number | undefined; + /** Cancellation for this call. */ + readonly signal?: AbortSignal | undefined; +} + +/** {@link ServiceCore.execute} and {@link ServiceCore.executeRequest}: what to decode into. */ +export interface ExecuteOptions<T> extends CallOptions { + /** The runtime type witness plus its diagnostic label. */ + readonly responseType: DecodeTarget<T>; +} + +/** {@link ServiceCore.paginate}: which strategy walks the collection, and how far. */ +export interface PaginateOptions<T> extends CallOptions { + readonly strategy: PaginationStrategy<T>; + /** Maximum page exchanges; unbounded when omitted. */ + readonly maxPages?: number | undefined; +} + +/** {@link ServiceCore.events}: how each SSE frame becomes a model. */ +export interface EventsOptions<T> extends CallOptions { + readonly mapper: SseMapper<T>; +} + +/** Everything a {@link ServiceCore} is built from. */ +export interface ServiceCoreInit { + /** The absolute base URL every operation is projected onto. */ + readonly baseUrl: string | URL; + /** + * A terminal transport the core OWNS: it is wrapped in `standardResilience()` and closed by + * {@link ServiceCore.close}. Mutually exclusive with `runtime`. + */ + readonly transport?: Transport | undefined; + /** + * An already-assembled pipeline the core BORROWS: used as-is and never closed. Mutually exclusive + * with `transport`. + */ + readonly runtime?: Runtime | undefined; + /** Pillar overrides, applied only on the owned-transport path where the preset is built here. */ + readonly resilience?: StandardResilienceOptions | undefined; + /** The wire codec. Required: core owns none (SEAM-1), so the executor has to be told. */ + readonly serde: Serde; + /** The declarative status-to-error table; omitted leaves `HttpStatusError` unmapped. */ + readonly errors?: StatusErrorMap | undefined; +} + +/** + * Carry the two auth tiers into `RequestOptions`, each in its own slot. + * + * This used to read `const auth = call.auth ?? operation?.auth`, which reimplemented the top + * two-thirds of AUTH-4's precedence chain in consumer code and left core unable to tell a genuine + * per-call override from an operation's declared requirement. `RequestOptions.operationAuth` landed + * 2026-09-04 (`docs/work/mvp/2026-09-04-open-items-dissolution.md` W1) and the fold is gone: core resolves + * `perCall ?? operation ?? client` itself, all three tiers distinguishable. + * + * Returning `undefined` when there is nothing to say still matters: an empty `RequestOptions` would + * still occupy the `perCall` slot as "no descriptor", and the point of the chain is that an ABSENT + * tier falls through while a PRESENT one does not. + */ +function requestOptions( + operation: Operation | undefined, + call: CallOptions, +): RequestOptions | undefined { + if ( + call.auth === undefined && + operation?.auth === undefined && + call.timeoutMs === undefined && + call.maxRetries === undefined + ) { + return undefined; + } + return RequestOptions.newBuilder() + .auth(call.auth) + .operationAuth(operation?.auth) + .timeoutMs(call.timeoutMs) + .maxRetries(call.maxRetries) + .build(); +} + +/** The shared executor every generated facade method delegates to. */ +export class ServiceCore { + readonly #baseUrl: string | URL; + readonly #runtime: Runtime; + readonly #ownedTransport: Transport | undefined; + readonly #serde: Serde; + readonly #errors: StatusErrorMap | undefined; + + constructor(init: ServiceCoreInit) { + const {transport, runtime} = init; + this.#baseUrl = init.baseUrl; + this.#serde = init.serde; + this.#errors = init.errors; + if (transport !== undefined && runtime === undefined) { + this.#runtime = standardResilience(transport, init.resilience); + this.#ownedTransport = transport; + } else if (runtime !== undefined && transport === undefined) { + this.#runtime = runtime; + this.#ownedTransport = undefined; + } else { + throw new TypeError( + 'ServiceCore takes exactly one of `transport` (owned) or `runtime` (borrowed)', + ); + } + } + + /** The pipeline every call goes through — owned or borrowed alike. */ + get runtime(): Runtime { + return this.#runtime; + } + + /** Assemble, send, and decode a 2xx into `T`; map a failure status through the error table. */ + async execute<T>( + operation: Operation, + input: OperationInput, + call: ExecuteOptions<T>, + ): Promise<T> { + const response = await this.dispatch(operation, input, call); + return this.#decode(response, call.responseType); + } + + /** The same, for a request a caller already built — an escape hatch out of the operation table. */ + async executeRequest<T>( + request: Request, + call: ExecuteOptions<T>, + ): Promise<T> { + const response = await this.#runtime.send( + request, + requestOptions(undefined, call), + call.signal, + ); + return this.#decode(response, call.responseType); + } + + /** Assemble and send, with no decode: the raw response, still open, still the caller's to close. */ + dispatch( + operation: Operation, + input: OperationInput, + call: CallOptions = {}, + ): Promise<Response> { + const request = buildRequest(this.#baseUrl, assemble(operation, input)); + return this.#runtime.send( + request, + requestOptions(operation, call), + call.signal, + ); + } + + /** + * A lazy walk over a paginated collection. + * + * Nothing is sent until the returned paginator is iterated (PAGE-6), so this method is + * synchronous and the generated facade needs no `await`. + */ + paginate<T>( + operation: Operation, + input: OperationInput, + paging: PaginateOptions<T>, + ): Paginator<T> { + return new Paginator<T>({ + transport: this.#runtime, + initialRequest: buildRequest(this.#baseUrl, assemble(operation, input)), + strategy: paging.strategy, + maxPages: paging.maxPages, + options: requestOptions(operation, paging), + signal: paging.signal, + }); + } + + /** + * A lazy stream of mapped SSE events. + * + * Lazy for the same reason `paginate` is: the request is sent on the first pull, so the facade + * method stays synchronous. The status check runs before the parser ever sees a byte. + */ + events<T>( + operation: Operation, + input: OperationInput, + streaming: EventsOptions<T>, + ): AsyncIterable<T> { + const open = async (): Promise<SseStream> => { + const response = await this.dispatch(operation, input, streaming); + await this.#failOnErrorStatus(response); + return sseStreamFrom(response, {signal: streaming.signal}); + }; + return { + async *[Symbol.asyncIterator](): AsyncGenerator<T> { + yield* typedSseStream(await open(), streaming.mapper); + }, + }; + } + + /** + * Release what this core created, and nothing else. + * + * An OWNED transport is closed. A BORROWED runtime is left alone — and so is the transport + * underneath it, which `Runtime.close()` would not have touched anyway (PIPE-27). + */ + async close(): Promise<void> { + if (this.#ownedTransport !== undefined) { + await this.#ownedTransport.close(); + } + } + + async #decode<T>(response: Response, target: DecodeTarget<T>): Promise<T> { + try { + return await decodeSuccessResponse( + response, + this.#serde.deserializer, + target, + ); + } catch (error: unknown) { + throw remapStatusError(error, this.#errors); + } + } + + /** + * Turn a non-2xx into the mapped typed error before any streaming reader is built. + * + * `toHttpError` covers 4xx/5xx and returns `null` for anything else, so an unfollowed 3xx or a + * 1xx lands in the second branch — closed, then reported as itself rather than being handed to + * an SSE parser that would read it as a malformed event stream. + */ + async #failOnErrorStatus(response: Response): Promise<void> { + if (response.status.isSuccess) return; + const failure = await toHttpError(response); + if (failure !== null) throw remapStatusError(failure, this.#errors); + await response.close(); + throw new DexpaceError( + `response status ${String(response.status.code)} is neither a success nor an error status`, + ); + } +} diff --git a/examples/petstore/src/support.ts b/examples/petstore/src/support.ts new file mode 100644 index 0000000..637439b --- /dev/null +++ b/examples/petstore/src/support.ts @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +// examples/petstore/src/support.ts +/** + * The hand-authored runtime binders the generated facade names. + * + * A generated SDK ships a small shim beside its facades; this is that shim. The facade holds no + * logic — it binds arguments and names a symbol from here. + * + * What this file measures: + * + * - **`jsonBody` is thin, and `petPatchToWire` is not.** Core's `serdeBody(value, serde, mediaType)` + * already does encode-plus-wrap, so the body binder is one line. The projection beside it is the + * cost: `Schema<T>` is decode-only, so a model whose field names differ from its wire names needs + * a hand-written encoder per model. See FINDINGS.md, finding 6. + * - **`PET_PAGE_STRATEGY` needed no decorator.** Python wraps the certified `CursorStrategy` in a + * `_PetPageStrategy` that re-decodes each raw item, because its strategy is configured by wire + * FIELD NAMES and hands back raw documents. Node's `cursorStrategy` takes an `extract` callback + * instead, so the decode happens inside it and the wrapper class disappears. See finding 5. + * - **`PET_EVENT_MAPPER` is a plain function.** `SseMapper<T>` is `(eventName, joinedData) => + * MapperOutcome<T>`, and `MAPPER_DONE` is the `[DONE]` sentinel's answer. + */ +import { + MAPPER_DONE, + cursorStrategy, + mapperValue, + serdeBody, + type Body, + type MapperOutcome, + type PaginationStrategy, + type Response, + type Schema, + type Serde, + type SseMapper, +} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; +import { + PET_EVENT_SCHEMA, + PET_SCHEMA, + type Pet, + type PetEvent, + type PetPatch, +} from './models.js'; + +/** + * The one shared codec instance every binder reuses. + * + * `jsonSerde()` freezes its bundle and holds no per-call state (SERDE-29), so a single instance + * serves every model and every concurrent call. The Tristate wiring is on by default, which is what + * makes the merge-patch body's three states survive the encode. + */ +const SERDE: Serde = jsonSerde(); + +const TEXT_ENCODER = new TextEncoder(); + +/** + * Encode an already-projected wire document into a request body. + * + * @param document - the wire-shaped value, not the model. The projection is the caller's job + * because core carries no encode witness — see {@link petPatchToWire}. + * @param mediaType - overrides the serde's own `application/json`; the petstore's PATCH operation + * declares `application/merge-patch+json` in the frozen document, and the generator passes it + * through. + */ +export function jsonBody(document: unknown, mediaType?: string): Body { + return serdeBody(document, SERDE, mediaType); +} + +/** + * Project a {@link PetPatch} onto its wire shape. + * + * Only the KEYS change here. The `Tristate` values are passed through untouched and resolved by + * `jsonSerde()`'s replacer at encode time — Absent drops the key, Null writes `null`, Present + * writes the value. Resolving them here instead would collapse Absent and Null before the replacer + * ever saw them, which is precisely the interop bug `Tristate` exists to prevent. + */ +export function petPatchToWire( + patch: PetPatch, +): Readonly<Record<string, unknown>> { + return {name: patch.name, tag: patch.tag, weight_kg: patch.weightKg}; +} + +/** One page of the `/pets` collection, as the frozen document describes it. */ +interface PetPage { + readonly items: readonly Pet[]; + readonly cursor: string | null; +} + +const PET_PAGE_SCHEMA: Schema<PetPage> = Object.freeze({ + parse(input: unknown): PetPage { + if (typeof input !== 'object' || input === null || Array.isArray(input)) { + throw new TypeError('pet page: expected a JSON object'); + } + const record = input as Readonly<Record<string, unknown>>; + const rawItems = record.data; + if (!Array.isArray(rawItems)) { + throw new TypeError('pet page: "data" must be an array'); + } + // `Array.isArray` narrows an `unknown` to `any[]`, and `any` would then flow into + // `PET_SCHEMA.parse` unchecked. Re-typing to `unknown[]` keeps the decode honest. + const data = rawItems as readonly unknown[]; + const next = record.next_cursor; + if (next !== null && next !== undefined && typeof next !== 'string') { + throw new TypeError('pet page: "next_cursor" must be a string or null'); + } + return { + items: data.map(item => PET_SCHEMA.parse(item)), + cursor: next ?? null, + }; + }, +}); + +/** + * Pagination for `listPets`: cursor continuation over a `data` array, splicing `?cursor=` onto the + * request that produced the page. + * + * `extract` reads the body exactly once and never closes the response — both are the strategy + * contract's obligations, and the paginator closes each page itself. + */ +export const PET_PAGE_STRATEGY: PaginationStrategy<Pet> = cursorStrategy<Pet>({ + parameterName: 'cursor', + extract: async (response: Response) => { + const page = SERDE.deserializer.deserialize(await response.bytes(), { + schema: PET_PAGE_SCHEMA, + typeName: 'PetPage', + }); + return {items: page.items, cursor: page.cursor}; + }, +}); + +/** The sentinel `watchPets` ends on, spelled exactly as the frozen document's fixture sends it. */ +const DONE_SENTINEL = '[DONE]'; + +/** + * SSE mapping for `watchPets`: `[DONE]` ends the stream, every other frame decodes into a + * {@link PetEvent}. + * + * Synchronous by contract — `SseMapper<T>` returns a `MapperOutcome<T>`, not a promise — which is + * why the decode goes through the deserializer's in-memory entry point rather than its streaming + * one. + */ +export const PET_EVENT_MAPPER: SseMapper<PetEvent> = ( + eventName: string | undefined, + joinedData: string, +): MapperOutcome<PetEvent> => { + if (joinedData === DONE_SENTINEL) return MAPPER_DONE; + return mapperValue( + SERDE.deserializer.deserialize(TEXT_ENCODER.encode(joinedData), { + schema: PET_EVENT_SCHEMA, + typeName: 'PetEvent', + }), + ); +}; diff --git a/examples/petstore/tsconfig.json b/examples/petstore/tsconfig.json new file mode 100644 index 0000000..bb43728 --- /dev/null +++ b/examples/petstore/tsconfig.json @@ -0,0 +1,31 @@ +{ + // Standalone and referenced by nothing. `bun run typecheck` names each package's project + // explicitly and this one is deliberately absent from that list, so the example is never a + // gate — check it by hand with `bunx tsc -p examples/petstore/tsconfig.json --noEmit`. + // + // It exists anyway, and is not optional: `eslint.config.js` runs the type-aware tier with + // `projectService: true`, which resolves each `.ts` file against the NEAREST enclosing + // tsconfig. Without this file — or with an `include` that misses a file — `gts lint .` fails + // at the repository root with "was not found by the project service", which is a blocking CI + // step. See FINDINGS.md, finding 7. + // + // `types: ["bun"]` mirrors `tests/tsconfig.json`: the canary and regen suites import + // `bun:test`, and nothing else in the tree supplies those globals. + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": ".", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ], + "types": [ + "bun" + ] + }, + "include": [ + "src/**/*.ts", + "*.test.ts" + ] +} diff --git a/package.json b/package.json index 6aeafe9..bec95f5 100644 --- a/package.json +++ b/package.json @@ -1,37 +1,80 @@ { "name": "nodejs-sdk", "private": true, + "license": "MIT", "type": "module", - "workspaces": [ - "packages/*" - ], + "workspaces": { + "packages": [ + "packages/*" + ], + "catalog": { + "@microsoft/api-extractor": "^7", + "expect-type": "^1.4.0", + "fast-check": "^3", + "rxjs": "^7.8.0", + "typescript": "^5.8" + } + }, "devDependencies": { "@arethetypeswrong/cli": "^0.18", "@changesets/cli": "^2", + "@dexpace/body-file": "workspace:*", + "@dexpace/codec-json": "workspace:*", "@dexpace/core": "workspace:*", + "@dexpace/logging-debug": "workspace:*", + "@dexpace/logging-pino": "workspace:*", + "@dexpace/rx": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@dexpace/transport-fetch": "workspace:*", + "@dexpace/transport-shared": "workspace:*", + "@dexpace/transport-undici": "workspace:*", "@eslint-community/eslint-plugin-eslint-comments": "^4", - "@microsoft/api-extractor": "^7", + "@microsoft/api-extractor": "catalog:", "@types/bun": "latest", "eslint": "^9", - "fast-check": "^3", + "fast-check": "catalog:", "globals": "^17.8.0", "gts": "^7", + "mitata": "^1", "publint": "^0.3", - "typescript": "^5.8", + "rxjs": "catalog:", + "typescript": "catalog:", "typescript-eslint": "^8" }, + "overrides": { + "fast-uri": "^3.1.5", + "js-yaml": "^4.3.1", + "tmp": "^0.2.6" + }, "scripts": { - "lint": "gts lint .", - "fix": "gts fix .", - "typecheck": "tsc -p packages/core/tsconfig.json --noEmit", - "build": "tsc -p packages/core/tsconfig.build.json", - "test": "bun test", - "api": "cd packages/core && bun run api:ci", - "lint:publish": "publint packages/core && attw --pack packages/core --ignore-rules cjs-resolves-to-esm", + "lint": "bun run build:deps && gts lint .", + "fix": "bun run build:deps && gts fix .", + "build:core": "tsc -b packages/core/tsconfig.build.json", + "build:deps": "bun run build:core && tsc -p packages/transport-shared/tsconfig.build.json && tsc -p packages/codec-json/tsconfig.build.json && tsc -p packages/transport-fetch/tsconfig.build.json", + "typecheck": "bun run build:deps && tsc -p packages/core/tsconfig.json --noEmit && tsc -p packages/codec-json/tsconfig.json --noEmit && tsc -p packages/logging-pino/tsconfig.json --noEmit && tsc -p packages/logging-debug/tsconfig.json --noEmit && tsc -p packages/body-file/tsconfig.json --noEmit && tsc -p packages/transport-shared/tsconfig.json --noEmit && tsc -p packages/transport-conformance/tsconfig.json --noEmit && tsc -p packages/transport-fetch/tsconfig.json --noEmit && tsc -p packages/transport-undici/tsconfig.json --noEmit && tsc -p packages/rx/tsconfig.json --noEmit && tsc -p packages/shrink-test/tsconfig.json --noEmit && tsc -p tests/tsconfig.json --noEmit", + "prebuild": "bun run --cwd packages/core prebuild", + "build": "bun run build:deps && tsc -p packages/codec-json/tsconfig.build.json && tsc -p packages/logging-pino/tsconfig.build.json && tsc -p packages/logging-debug/tsconfig.build.json && tsc -p packages/body-file/tsconfig.build.json && tsc -p packages/transport-fetch/tsconfig.build.json && tsc -p packages/transport-undici/tsconfig.build.json && tsc -p packages/rx/tsconfig.build.json", + "test": "bun test ./packages ./tests", + "knowledge": "node scripts/knowledge.mjs", + "test:examples": "bun test ./examples", + "test:scripts": "node --test 'scripts/*.test.mjs'", + "test:node": "node --test tests/node-conformance/*.test.mjs", + "shrink-test": "bun test ./packages/shrink-test", + "bench": "bun run packages/core/src/io/byte-queue.bench.ts", + "api": "cd packages/core && bun run api:ci && cd ../codec-json && bun run api:ci && cd ../logging-pino && bun run api:ci && cd ../logging-debug && bun run api:ci && cd ../body-file && bun run api:ci && cd ../transport-shared && bun run api:ci && cd ../transport-fetch && bun run api:ci && cd ../transport-undici && bun run api:ci && cd ../rx && bun run api:ci", + "lint:publish": "publint packages/core && attw --pack packages/core --ignore-rules cjs-resolves-to-esm && publint packages/codec-json && attw --pack packages/codec-json --ignore-rules cjs-resolves-to-esm && publint packages/logging-pino && attw --pack packages/logging-pino --ignore-rules cjs-resolves-to-esm && publint packages/logging-debug && attw --pack packages/logging-debug --ignore-rules cjs-resolves-to-esm && publint packages/body-file && attw --pack packages/body-file --ignore-rules cjs-resolves-to-esm && publint packages/transport-shared && attw --pack packages/transport-shared --ignore-rules cjs-resolves-to-esm && publint packages/transport-fetch && attw --pack packages/transport-fetch --ignore-rules cjs-resolves-to-esm && publint packages/transport-undici && attw --pack packages/transport-undici --ignore-rules cjs-resolves-to-esm && publint packages/rx && attw --pack packages/rx --ignore-rules cjs-resolves-to-esm", "audit": "bun audit --audit-level=high --prod", + "changeset": "node scripts/changeset.mjs", "verify:dual-consumption": "node scripts/verify-dual-consumption.mjs", + "verify:consumer-types": "node scripts/verify-consumer-types.mjs", "verify:seam-1": "node scripts/verify-seam-1.mjs", + "verify:sse-37": "node scripts/verify-sse-37.mjs", "verify:runtime-floor": "node scripts/verify-runtime-floor.mjs", - "verify:node-floor": "node scripts/verify-node-floor.mjs" + "verify:test-partition": "node scripts/verify-test-partition.mjs", + "verify:import-cycles": "node scripts/verify-import-cycles.mjs", + "verify:knowledge-structure": "node scripts/verify-knowledge-structure.mjs", + "knowledge:drift": "node scripts/knowledge-drift.mjs", + "verify:reproducible-build": "node scripts/verify-reproducible-build.mjs" } } + diff --git a/packages/body-file/README.md b/packages/body-file/README.md new file mode 100644 index 0000000..d29692f --- /dev/null +++ b/packages/body-file/README.md @@ -0,0 +1,44 @@ +# @dexpace/body-file + +A file-backed request `Body` for the dexpace SDK. Zero dependencies beyond a `@dexpace/core` peer — +`node:fs` is a runtime API, not an npm package, which is exactly why this lives here and not in +`@dexpace/core` (whose zero-`node:`-import invariant is hard). + +```sh +bun add @dexpace/body-file @dexpace/core +``` + +```typescript +import {Request} from '@dexpace/core'; +import {fileBody} from '@dexpace/body-file'; + +// Validated at construction, not at send time (HTTP-40, BODY-11). +const body = fileBody('./upload.bin', {start: 1024, count: 4096}); + +const request = Request.newBuilder() + .method('POST') + .url('https://example.com/v1/uploads') + .body(body) + .build(); +``` + +## Fail-fast construction + +`fileBody()` stats the path immediately and rejects all four ways it can be wrong, none of which +follows from another: the path must exist and be a **regular** file; `start >= 0`; `start <= size`; +`count >= 0`; and `start + count <= size`. The `start <= size` check earns its place — `count` +defaults to `size - start`, which goes *negative* for a start past end-of-file and then satisfies +the sum check, silently producing a zero-byte upload instead of an error. + +## Behavior worth knowing + +- `replayable` is always `true`, and `writeTo()` opens a **fresh** handle per call, so a retry + re-sends the same bytes (`HTTP-40`). +- `writeTo()` does not close the sink it was handed — closing belongs to whoever created it + (`BODY-8`) — and aborts it on failure so a consumer sees the error rather than a silently + truncated stream. The read handle is destroyed on every exit path, so a failed send strands no + file descriptor. +- A short read raises rather than reporting success (`BODY-13`). +- Transports recognize the result **structurally**, through `body.kind === 'file'`, never an + `instanceof` against this package: `@dexpace/transport-undici` dispatches straight off the file, + and neither transport depends on this package. diff --git a/packages/body-file/api-extractor.json b/packages/body-file/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/body-file/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/body-file/etc/body-file.api.md b/packages/body-file/etc/body-file.api.md new file mode 100644 index 0000000..24f20dc --- /dev/null +++ b/packages/body-file/etc/body-file.api.md @@ -0,0 +1,20 @@ +## API Report File for "@dexpace/body-file" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import type { FileBodyDescriptor } from '@dexpace/core'; + +// @public +export function fileBody(path: string, options?: FileBodyOptions): FileBodyDescriptor; + +// @public +export interface FileBodyOptions { + readonly count?: number; + readonly start?: number; +} + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/body-file/package.json b/packages/body-file/package.json new file mode 100644 index 0000000..91aed70 --- /dev/null +++ b/packages/body-file/package.json @@ -0,0 +1,51 @@ +{ + "name": "@dexpace/body-file", + "version": "0.0.0", + "description": "File body adapter with fail-fast node:fs validation for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/body-file" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/body-file/src/file-body.test.ts b/packages/body-file/src/file-body.test.ts new file mode 100644 index 0000000..1f48a9e --- /dev/null +++ b/packages/body-file/src/file-body.test.ts @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: MIT +// packages/body-file/src/file-body.test.ts +// Exercises: HTTP-40/BODY-11 (fail-fast construction validation, fresh handle per write, replayable), +// BODY-13 (short-write detection), BODY-12/TRANSPORT-28 (recognizable by type) +/* eslint-disable max-lines-per-function -- file body tests need full I/O lifecycle setup */ +import {afterEach, beforeEach, describe, expect, test} from 'bun:test'; +import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {fileBody} from './file-body.js'; + +let dir: string; +let filePath: string; + +beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'body-file-')); + filePath = join(dir, 'payload.bin'); + await writeFile(filePath, new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8])); +}); + +afterEach(async () => { + await rm(dir, {recursive: true, force: true}); +}); + +describe('fileBody (HTTP-40, BODY-11)', () => { + test('is recognizable by kind and replayable', () => { + const body = fileBody(filePath); + expect(body.kind).toBe('file'); + expect(body.replayable).toBe(true); + expect(body.contentLength).toBe(8); + expect(body.mediaType).toBeUndefined(); + }); + + test('rejects a nonexistent path at construction', () => { + expect(() => fileBody(join(dir, 'missing.bin'))).toThrow(); + }); + + test('rejects a directory path at construction', () => { + expect(() => fileBody(dir)).toThrow(); + }); + + test('rejects a negative start or out-of-range count at construction', () => { + expect(() => fileBody(filePath, {start: -1})).toThrow(); + expect(() => fileBody(filePath, {start: 4, count: 10})).toThrow(); + expect(() => fileBody(filePath, {count: -1})).toThrow(); + expect(() => fileBody(filePath, {start: 100})).toThrow(); + }); + + test('writeTo does not close the caller-owned sink', async () => { + const body = fileBody(filePath); + let closed = false; + const sink = new WritableStream<Uint8Array>({ + close() { + closed = true; + }, + write() { + // no-op: we only care about close tracking + }, + }); + await body.writeTo(sink); + expect(closed).toBe(false); + }); + + test('writeTo streams exactly the declared byte range', async () => { + const body = fileBody(filePath, {start: 2, count: 4}); + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(chunk); + }, + }); + await body.writeTo(sink); + const totalLength = chunks.reduce((acc, c) => acc + c.byteLength, 0); + const written = new Uint8Array(totalLength); + let offset = 0; + for (const chunk of chunks) { + written.set(chunk, offset); + offset += chunk.byteLength; + } + expect(written).toEqual(new Uint8Array([3, 4, 5, 6])); + }); + + test('writeTo handles 0 count', async () => { + const body = fileBody(filePath, {start: 0, count: 0}); + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(chunk); + }, + }); + await body.writeTo(sink); + expect(chunks.length).toBe(0); + }); + + test('writeTo opens a fresh handle on each call (replayable)', async () => { + const body = fileBody(filePath); + const first: number[] = []; + const second: number[] = []; + await body.writeTo( + new WritableStream({ + write(c) { + first.push(...c); + }, + }), + ); + await body.writeTo( + new WritableStream({ + write(c) { + second.push(...c); + }, + }), + ); + expect(second).toEqual(first); + }); + + test('writeTo propagates error from stream read or write', () => { + const body = fileBody(filePath); + const sink = new WritableStream<Uint8Array>({ + write: () => { + throw new Error('sink write error'); + }, + }); + expect(body.writeTo(sink)).rejects.toThrow('sink write error'); + }); +}); diff --git a/packages/body-file/src/file-body.ts b/packages/body-file/src/file-body.ts new file mode 100644 index 0000000..273e903 --- /dev/null +++ b/packages/body-file/src/file-body.ts @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT +// packages/body-file/src/file-body.ts +import {createReadStream, statSync} from 'node:fs'; +import type {FileBodyDescriptor} from '@dexpace/core'; +import {invariant} from './invariant.js'; + +/** + * Options for configuring a file-backed request body. + * + * @public + */ +export interface FileBodyOptions { + /** The starting byte offset within the file (default 0). */ + readonly start?: number; + /** The number of bytes to stream (default: remaining bytes from start to end of file). */ + readonly count?: number; +} + +/** + * Creates a file-backed request body descriptor with fail-fast construction validation (HTTP-40, BODY-11). + * + * @param path - the absolute or relative path to the regular file. + * @param options - optional byte range (start offset and count). + * @returns an immutable `FileBodyDescriptor`. + * @throws Error if the file does not exist, is not a regular file, or if the byte range is invalid. + * + * @public + */ +export function fileBody( + path: string, + options: FileBodyOptions = {}, +): FileBodyDescriptor { + const stats = statSync(path); + invariant(stats.isFile(), `not a regular file: ${path}`); + const start = options.start ?? 0; + invariant(start >= 0, `start must be non-negative, got ${String(start)}`); + invariant( + start <= stats.size, + `start (${String(start)}) exceeds file size (${String(stats.size)})`, + ); + const count = options.count ?? stats.size - start; + invariant(count >= 0, `count must be non-negative, got ${String(count)}`); + invariant( + start + count <= stats.size, + `start + count (${String(start + count)}) exceeds file size (${String(stats.size)})`, + ); + + return Object.freeze({ + kind: 'file' as const, + mediaType: undefined, + contentLength: count, + replayable: true, + path, + start, + count, + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + const writer = sink.getWriter(); + if (count === 0) { + writer.releaseLock(); + return; + } + let transferred = 0; + const stream = createReadStream(path, { + start, + end: start + count - 1, + }); + try { + for await (const chunk of stream) { + const bytes = chunk as Buffer; + await writer.write( + new Uint8Array(bytes.buffer, bytes.byteOffset, bytes.byteLength), + ); + transferred += bytes.byteLength; + } + invariant( + transferred === count, + `short write: transferred ${String(transferred)} of ${String(count)} bytes`, + ); + } catch (error) { + await writer.abort(error); + throw error; + } finally { + stream.destroy(); + writer.releaseLock(); + } + }, + }); +} diff --git a/packages/body-file/src/index.ts b/packages/body-file/src/index.ts new file mode 100644 index 0000000..aa1bb59 --- /dev/null +++ b/packages/body-file/src/index.ts @@ -0,0 +1,4 @@ +// SPDX-License-Identifier: MIT +// packages/body-file/src/index.ts +export {fileBody} from './file-body.js'; +export type {FileBodyOptions} from './file-body.js'; diff --git a/packages/body-file/src/invariant.ts b/packages/body-file/src/invariant.ts new file mode 100644 index 0000000..88a16de --- /dev/null +++ b/packages/body-file/src/invariant.ts @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT +// packages/body-file/src/invariant.ts + +export function invariant( + condition: boolean, + message: string, +): asserts condition { + if (!condition) throw new Error(message); +} diff --git a/packages/body-file/tsconfig.build.json b/packages/body-file/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/body-file/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/body-file/tsconfig.json b/packages/body-file/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/body-file/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/codec-json/README.md b/packages/codec-json/README.md new file mode 100644 index 0000000..8d440dc --- /dev/null +++ b/packages/codec-json/README.md @@ -0,0 +1,40 @@ +# @dexpace/codec-json + +The reference JSON wire codec for the dexpace SDK — `JSON.parse`/`JSON.stringify` behind the `Serde` +seam, with PATCH tri-state semantics wired in by default. Zero dependencies beyond a `@dexpace/core` +peer. + +```sh +bun add @dexpace/codec-json @dexpace/core +``` + +```typescript +import {decodeResponse, serdeBody, type Response} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; +import {z} from 'zod'; // any schema library works — this package depends on none + +const serde = jsonSerde(); +const User = z.object({id: z.number(), name: z.string()}); + +// Content-Type defaults to the serde's own media type: application/json +export const body = serdeBody({name: 'ada'}, serde); + +export async function readUser(response: Response) { + return decodeResponse(response, serde.deserializer, { + schema: User, + typeName: 'User', + }); +} +``` + +The schema you pass is both the runtime witness and the source of the static type — there is no +separate type argument to keep in sync. + +- **PATCH three-state fields** — `tristate()` and `tristateObject()`, documented on their own TSDoc in + [`src/tristate-schema.ts`](https://github.com/dexpace/nodejs-sdk/blob/main/packages/codec-json/src/tristate-schema.ts). Absent omits the key, Null emits a wire + `null`, Present emits the value; the wiring is on by default and `jsonSerde({tristate: false})` is + the only way out. +- **Unknown wire fields** — your schema's decision, not this codec's. The rationale and the + recommendation are on `jsonSerde`'s own TSDoc in [`src/json-serde.ts`](https://github.com/dexpace/nodejs-sdk/blob/main/packages/codec-json/src/json-serde.ts). +- **A top-level wire `null` never decodes**, and a top-level `undefined`, function, or symbol raises + `SerializationError` rather than encoding as `null`. Both are on `jsonSerde`'s TSDoc too. diff --git a/packages/codec-json/api-extractor.json b/packages/codec-json/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/codec-json/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/codec-json/etc/codec-json.api.md b/packages/codec-json/etc/codec-json.api.md new file mode 100644 index 0000000..3c37939 --- /dev/null +++ b/packages/codec-json/etc/codec-json.api.md @@ -0,0 +1,30 @@ +## API Report File for "@dexpace/codec-json" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import { Schema } from '@dexpace/core'; +import { Serde } from '@dexpace/core'; +import { Tristate } from '@dexpace/core'; + +// @public +export function jsonSerde(options?: JsonSerdeOptions): Serde; + +// @public +export interface JsonSerdeOptions { + readonly tristate?: boolean | undefined; +} + +// @public +export function tristate<T>(inner: Schema<T>): Schema<Tristate<T>>; + +// @public +export function tristateObject<S extends Record<string, Schema<unknown>>>(shape: S): Schema<{ + [K in keyof S]: Tristate<S[K] extends Schema<infer T> ? T : never>; +}>; + +// @public +export function tristateReplacer(key: string, value: unknown): unknown; + +``` diff --git a/packages/codec-json/package.json b/packages/codec-json/package.json new file mode 100644 index 0000000..df10c6e --- /dev/null +++ b/packages/codec-json/package.json @@ -0,0 +1,51 @@ +{ + "name": "@dexpace/codec-json", + "version": "0.0.0", + "description": "Reference JSON wire codec for the dexpace SDK: JSON.parse/JSON.stringify plus Tristate wiring and schema decode glue.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/codec-json" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/codec-json/src/abort-race.ts b/packages/codec-json/src/abort-race.ts new file mode 100644 index 0000000..357158b --- /dev/null +++ b/packages/codec-json/src/abort-race.ts @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/abort-race.ts + +/** + * One abort listener held for the length of a whole stream drive, plus the race that lets it settle + * an operation that is already *pending*. + * + * @internal + */ +export interface AbortRace { + /** + * Settle with `operation`, or reject with the signal's `reason` the moment it aborts — whichever + * happens first. + * + * Also rejects before `operation` is even consulted when the signal is already aborted, which is + * the between-chunks check the loop used to make for itself. + */ + race<T>(operation: Promise<T>): Promise<T>; + + /** Drop the abort listener. Call from the `finally` that releases the stream lock. */ + release(): void; +} + +/** The no-signal case: no listener to install, no race to run, no allocation per chunk. */ +const UNRACED: AbortRace = Object.freeze({ + race: <T>(operation: Promise<T>): Promise<T> => operation, + release: (): void => undefined, +}); + +/** + * Bind `signal` to a single listener that can interrupt any number of pending operations + * (SERDE-3, audit #67 / #79). + * + * `throwIfAborted()` between chunks is not enough on its own: a `reader.read()` that never resolves + * is never raced against anything, so the drain parks inside it, the call never settles, and + * `source.locked` stays `true` for the rest of the process — the opposite of the seam's promise that + * "an aborted call never leaves the caller's source locked". Racing the pending operation is what + * makes that promise true rather than aspirational. + * + * The signal's `reason` is surfaced verbatim, never re-typed: a caller aborting with its own error + * gets that error back, and a bare `abort()` gets the platform's `AbortError` `DOMException`, which + * is exactly what `throwIfAborted()` would have thrown. + * + * One listener per call, not one per chunk — a 10 000-chunk body would otherwise register and remove + * 10 000 listeners on a signal the caller may hold for the life of a request. + * + * @param signal - the caller's signal, or `undefined` when the call took none. + * @returns a race bound to `signal`, whose `release()` removes the listener. + * @internal + */ +export function abortRace(signal: AbortSignal | undefined): AbortRace { + if (signal === undefined) return UNRACED; + + let onAbort = (): void => undefined; + const aborted = new Promise<never>((_resolve, reject) => { + onAbort = (): void => { + // The seam documents that a caller sees its own abort `reason` verbatim, and a caller may + // abort with any value at all — `controller.abort('gone')` is legal. Re-typing it here would + // break that contract, and it is also exactly what `throwIfAborted()` throws. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- see above; re-enable if the seam ever narrows `reason` to an Error + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + // An abort can land while nothing is racing this promise — between two reads, or after the last + // one and before `release()`. That rejection would be an unhandled one, which takes the process + // down under Node's default policy (`docs/knowledge/harvested/cancellation-and-timeouts.md:26`). + // A no-op handler marks it handled without stopping `Promise.race` below from seeing it. + void aborted.catch(() => undefined); + + return Object.freeze({ + async race<T>(operation: Promise<T>): Promise<T> { + signal.throwIfAborted(); + // A pending `operation` that rejects after losing the race is still settled through + // `Promise.race`'s own handler, so it never becomes an unhandled rejection either — measured + // on Bun 1.3.14 and Node 20.3/26, where releasing a reader with a read outstanding rejects + // that read (`AbortError` on Bun, `TypeError` on Node). + return Promise.race([operation, aborted]); + }, + release(): void { + signal.removeEventListener('abort', onAbort); + }, + }); +} diff --git a/packages/codec-json/src/conformance.test.ts b/packages/codec-json/src/conformance.test.ts new file mode 100644 index 0000000..ff04f71 --- /dev/null +++ b/packages/codec-json/src/conformance.test.ts @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/conformance.test.ts +// Exercises the requirements the Phase 6a design dispositions as satisfied-by-construction: SERDE-21 (no +// cross-shape coercion), SERDE-22 (representation-preserving conversions still bind), SERDE-23 (unknown +// fields are the schema's decision, not the codec's), SERDE-24 (ISO-8601 dates round-trip), SERDE-29 (a +// bundle is safe to share once configured). +// +// No code in this repository implements SERDE-21 or SERDE-22 — `JSON.parse` performs no coercion, so there +// is nothing to switch off. These tests ARE the coverage, and Phase 9's sweep reads them as the evidence. +import {expect, test} from 'bun:test'; +import type {Schema} from '@dexpace/core'; +import {jsonSerde} from './json-serde.js'; + +const numberSchema: Schema<number> = { + parse: i => { + if (typeof i !== 'number') throw new Error('not a number'); + return i; + }, +}; +const intSchema: Schema<number> = { + parse: i => { + if (!Number.isInteger(i)) throw new Error('not an integer'); + // `as number`: `Number.isInteger` is not a type guard, but it cannot return true for a non-number. + return i as number; + }, +}; +const boolSchema: Schema<boolean> = { + parse: i => { + if (typeof i !== 'boolean') throw new Error('not a boolean'); + return i; + }, +}; +const stringSchema: Schema<string> = { + parse: i => { + if (typeof i !== 'string') throw new Error('not a string'); + return i; + }, +}; + +const decode = <T>(json: string, schema: Schema<T>): T => + jsonSerde().deserializer.deserialize(new TextEncoder().encode(json), { + schema, + typeName: 'Target', + }); + +// Typed as `Schema<unknown>` rather than inferred: `test.each` would otherwise widen the column to a +// union of the three schema types, which no single `decode` call site can accept. Schemas are +// covariant in their output, so each concrete schema is assignable here. +const COERCION_CASES: readonly (readonly [string, string, Schema<unknown>])[] = + [ + ['string → integer', '"5"', intSchema], + ['string → float', '"1.5"', numberSchema], + ['string → boolean', '"true"', boolSchema], + ['empty string → integer', '""', intSchema], + ['empty string → float', '""', numberSchema], + ['empty string → boolean', '""', boolSchema], + ['float → integer (lossy narrowing)', '1.5', intSchema], + ['boolean → integer', 'true', intSchema], + ['integer → boolean', '1', boolSchema], + ['boolean → float', 'true', numberSchema], + ['integer → string', '5', stringSchema], + ['boolean → string', 'true', stringSchema], + ]; + +test.each(COERCION_CASES)( + 'SERDE-21: %s is rejected, never silently reshaped', + (_name, json, schema) => { + expect(() => decode(json, schema)).toThrow(); + }, +); + +test('SERDE-22: an integer binds to a float target (JavaScript has one numeric type)', () => { + expect(decode('5', numberSchema)).toBe(5); +}); + +test('SERDE-22: an empty string binds to a textual target', () => { + expect(decode('""', stringSchema)).toBe(''); +}); + +test('SERDE-22: every well-typed value binds to its matching target', () => { + expect(decode('1.5', numberSchema)).toBe(1.5); + expect(decode('true', boolSchema)).toBe(true); + expect(decode('"text"', stringSchema)).toBe('text'); +}); + +test('SERDE-23: a permissive schema keeps an unknown wire field — the codec never rejects one', () => { + // The delegation, proven rather than asserted: nothing in this codec inspects the key set. A + // server adding a backward-compatible field does not break a client that has not regenerated. + const permissive: Schema<{id: number}> = { + parse: i => { + // `as {id: unknown}`: the wire shape this schema is written against, narrowed field by field + // on the next line rather than trusted. + const o = i as {id: unknown}; + if (typeof o.id !== 'number') throw new Error('not an id'); + // Returned as-is, extra keys included — this is what "ignore unknown fields" looks like when + // the schema, not the codec, owns the policy. `as {id: number}`: `id` was just checked; the + // extra keys are deliberately carried through and are outside the declared type. + return i as {id: number}; + }, + }; + + // `as Record<string, unknown>`: the decoded value's declared type is `{id: number}` and the point + // of the assertion is the key the type does NOT name, which is present at runtime. + const decoded = decode('{"id":1,"addedLater":true}', permissive) as Record< + string, + unknown + >; + + expect(decoded).toEqual({id: 1, addedLater: true}); +}); + +test("SERDE-23: a strict schema rejects the same payload — the policy is the schema's, either way", () => { + const strict: Schema<{id: number}> = { + parse: i => { + // `as Record<string, unknown>`: this schema's whole job is to inspect the key set, and + // `unknown` is not indexable. + const o = i as Record<string, unknown>; + const keys = Object.keys(o); + if (keys.length !== 1 || typeof o.id !== 'number') { + throw new Error(`unexpected keys: ${keys.join(',')}`); + } + return {id: o.id}; + }, + }; + + expect(decode('{"id":1}', strict)).toEqual({id: 1}); + // Same codec, same bytes, opposite outcome — because the schema changed, not the codec. + expect(() => decode('{"id":1,"addedLater":true}', strict)).toThrow(); +}); + +test('SERDE-24: a Date encodes as ISO-8601 and round-trips to the same instant', () => { + const instant = new Date('2026-07-28T12:34:56.789Z'); + + const encoded = new TextDecoder().decode( + jsonSerde().serializer.serialize({at: instant}), + ); + + expect(encoded).toBe('{"at":"2026-07-28T12:34:56.789Z"}'); + // `as {at: string}`: the wire shape, which the schema's whole job is to reconstitute. + const dateSchema: Schema<{at: Date}> = { + parse: i => ({at: new Date((i as {at: string}).at)}), + }; + expect(decode(encoded, dateSchema).at.getTime()).toBe(instant.getTime()); +}); + +/** + * A source that hands its payload back one byte at a time, so every read is a separate microtask. + * + * `deserializeFrom`'s read loop therefore yields between chunks, which is what makes 200 of these + * genuinely interleave. Wrapping synchronous `deserialize` calls in `Promise.resolve` would not: they + * run to completion during array construction, and the assertion would hold for a deeply stateful + * bundle too. + */ +const drip = (text: string): ReadableStream<Uint8Array> => { + const payload = new TextEncoder().encode(text); + let index = 0; + return new ReadableStream<Uint8Array>({ + pull(controller) { + if (index >= payload.length) { + controller.close(); + return; + } + controller.enqueue(payload.subarray(index, index + 1)); + index += 1; + }, + }); +}; + +test('SERDE-29: one bundle serves many concurrent operations without cross-talk', async () => { + const serde = jsonSerde(); + const identity: Schema<unknown> = {parse: i => i}; + let inFlight = 0; + let peakInFlight = 0; + + const results = await Promise.all( + Array.from({length: 200}, async (_, i) => { + inFlight += 1; + peakInFlight = Math.max(peakInFlight, inFlight); + try { + return await serde.deserializer.deserializeFrom( + drip(serde.serializer.serializeToString({i})), + {schema: identity}, + ); + } finally { + inFlight -= 1; + } + }), + ); + + expect(results).toEqual(Array.from({length: 200}, (_, i) => ({i}))); + // The test is worthless without this: it asserts the decodes really did overlap, so a bundle that + // carried per-operation state would be caught rather than run to completion one at a time. + expect(peakInFlight).toBe(200); +}); diff --git a/packages/codec-json/src/cross-package.test.ts b/packages/codec-json/src/cross-package.test.ts new file mode 100644 index 0000000..112cd01 --- /dev/null +++ b/packages/codec-json/src/cross-package.test.ts @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/cross-package.test.ts +// Exercises: SERDE-2 (core's serdeBody stamps the codec's own declared media type across the package +// boundary), SERDE-15 (a core-built Tristate keeps PATCH semantics through the codec), SEAM-1/NFR-2 (the +// codec's only edge to core is a peer, so brand identity has to survive that boundary). +// +// Guards the dual-package hazard sdk-design-nodejs/02 §2 describes. A `Tristate` constructed in @dexpace/core +// must be recognized by codec-json's replacer. Two non-identical copies of core in one tree would mean two +// distinct brand symbols and a silently wrong wire payload — a key emitted that the caller asked to omit. +import {expect, test} from 'bun:test'; +import { + absent, + isTristate, + nullValue, + present, + serdeBody, + TRISTATE_BRAND, +} from '@dexpace/core'; +import {jsonSerde} from './json-serde.js'; + +test('the brand symbol is registry-global, so two copies of core still agree', () => { + // Compared in this direction because `TRISTATE_BRAND` is a `unique symbol`: a plain `symbol` is + // not assignable to it, so it has to be the expectation rather than the subject. + expect(Symbol.for('@dexpace/core.Tristate')).toBe(TRISTATE_BRAND); +}); + +test('a Tristate constructed in core is recognized by codec-json', () => { + expect(isTristate(absent())).toBe(true); + expect(isTristate(nullValue())).toBe(true); + expect(isTristate(present(1))).toBe(true); +}); + +test('a core-constructed Tristate round-trips through the codec with PATCH semantics intact', () => { + const encoded = new TextDecoder().decode( + jsonSerde().serializer.serialize({ + keep: absent(), + clear: nullValue(), + set: present('v'), + }), + ); + + expect(encoded).toBe('{"clear":null,"set":"v"}'); +}); + +test('a caller object that merely has a kind field is not mistaken for a Tristate', () => { + const decoy = {kind: 'absent'}; + + expect(isTristate(decoy)).toBe(false); + expect( + new TextDecoder().decode(jsonSerde().serializer.serialize({x: decoy})), + ).toBe('{"x":{"kind":"absent"}}'); +}); + +test("core's serdeBody drives this codec end to end, stamping the codec's own media type (SERDE-2)", () => { + // The other direction of the same boundary: core consuming the codec through the `Serde` seam, + // rather than the codec consuming core's `Tristate`. + const body = serdeBody({name: 'ada', nickname: absent()}, jsonSerde()); + + expect(body.mediaType).toBe('application/json'); + expect(body.contentLength).toBe( + new TextEncoder().encode('{"name":"ada"}').length, + ); + expect(body.replayable).toBe(true); +}); diff --git a/packages/codec-json/src/index.ts b/packages/codec-json/src/index.ts new file mode 100644 index 0000000..24af8c3 --- /dev/null +++ b/packages/codec-json/src/index.ts @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/index.ts + +/** + * `@dexpace/codec-json` — the reference wire codec. + * + * Wraps `JSON.parse`/`JSON.stringify` behind `@dexpace/core`'s `Serde` seam. Depends on nothing + * beyond a `@dexpace/core` peer: schema validation is the caller's, supplied as a `Schema<T>` value + * at each decode call. + * + * @packageDocumentation + */ +export {jsonSerde} from './json-serde.js'; +export type {JsonSerdeOptions} from './json-serde.js'; +export {tristateReplacer} from './tristate-replacer.js'; +export {tristate, tristateObject} from './tristate-schema.js'; diff --git a/packages/codec-json/src/json-serde.property.test.ts b/packages/codec-json/src/json-serde.property.test.ts new file mode 100644 index 0000000..90ad55a --- /dev/null +++ b/packages/codec-json/src/json-serde.property.test.ts @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/json-serde.property.test.ts +// Exercises: SERDE-1 (the bundle's own serializer and deserializer round-trip each other). +import {test} from 'bun:test'; +import type {Schema} from '@dexpace/core'; +import fc from 'fast-check'; +import {jsonSerde} from './json-serde.js'; + +// `as T`: the round-trip is about bytes, not shape, so the witness deliberately validates nothing. +const identity = <T>(): Schema<T> => ({parse: input => input as T}); + +test('serialize → deserialize is the identity for any JSON value except null', () => { + fc.assert( + fc.property( + fc.jsonValue().filter(v => v !== null), + value => { + const serde = jsonSerde(); + const decoded = serde.deserializer.deserialize( + serde.serializer.serialize(value), + {schema: identity()}, + ); + return JSON.stringify(decoded) === JSON.stringify(value); + }, + ), + ); +}); + +test('serializeToString → deserialize agrees with the byte profile on the same values', () => { + fc.assert( + fc.property( + fc.jsonValue().filter(v => v !== null), + value => { + const {serializer} = jsonSerde(); + return ( + serializer.serializeToString(value) === + new TextDecoder().decode(serializer.serialize(value)) + ); + }, + ), + ); +}); + +test('serializeInto at any valid offset writes exactly what serialize produces', () => { + fc.assert( + fc.property( + fc.jsonValue().filter(v => v !== null), + fc.nat({max: 32}), + (value, offset) => { + const {serializer} = jsonSerde(); + const expected = serializer.serialize(value); + const target = new Uint8Array(offset + expected.length + 8).fill(0xaa); + + const written = serializer.serializeInto(value, target, offset); + + return ( + written === expected.length && + target + .slice(offset, offset + written) + .every((b, i) => b === expected[i]) && + target.slice(0, offset).every(b => b === 0xaa) + ); + }, + ), + ); +}); diff --git a/packages/codec-json/src/json-serde.test.ts b/packages/codec-json/src/json-serde.test.ts new file mode 100644 index 0000000..db85038 --- /dev/null +++ b/packages/codec-json/src/json-serde.test.ts @@ -0,0 +1,843 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/json-serde.test.ts +// Exercises: SERDE-1 (round-trip through one bundle), SERDE-2 (declared media type), SERDE-3 (never closes a +// caller stream), SERDE-4 (offset, byte count, RangeError with no cause), SERDE-5 (the schema witness drives +// the decode), SERDE-6 (parametric targets are combinator schemas), SERDE-9/SERDE-10 (library error never +// escapes; the directional leaves), SERDE-12 (the codec re-wraps nothing off the stream), SERDE-13 (a wire +// null into a non-null target, on every entry point, before the schema), SERDE-20 (a top-level unencodable +// value throws rather than sharing the Tristate degradation's old fallback), SERDE-25 (fresh instance per +// call), SEAM-20 (all four allocation profiles). +import {describe, expect, test} from 'bun:test'; +import { + DeserializationError, + SerializationError, + type Schema, +} from '@dexpace/core'; +import {jsonSerde} from './json-serde.js'; + +/** Distinguishes "the promise resolved" from a rejection value that happens to be falsy. */ +const RESOLVED = Symbol('resolved'); + +/** + * Settles `promise` and hands back whatever it rejected with. + * + * `expect(p).rejects.toX()` is typed `void` under `bun:test`, so awaiting it trips `await-thenable`. + */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return RESOLVED; + } catch (e: unknown) { + return e; + } +} + +/** How long a raced abort is given to settle a parked read or write before the case fails. */ +const SETTLE_MS = 250; + +/** + * Fails with a named error instead of letting the runner time out, so a regression reads as "the + * abort never settled the call" rather than as a five-second stall with no diagnosis. + * + * `Promise.race` keeps a handler on `promise`, so a later rejection of the losing side is never an + * unhandled one. + */ +async function settleWithin<T>(promise: Promise<T>, ms: number): Promise<T> { + let timer: ReturnType<typeof setTimeout> | undefined; + const deadline = new Promise<never>((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** One macrotask, which is long enough for a drain loop to reach its second, parked read. */ +function untilParked(): Promise<void> { + return new Promise<void>(resolve => { + setTimeout(resolve, 5); + }); +} + +test('declares application/json as its wire media type', () => { + expect(jsonSerde().mediaType).toBe('application/json'); +}); + +test('each call returns a fresh, frozen bundle (SERDE-25)', () => { + const a = jsonSerde(); + const b = jsonSerde(); + + expect(a).not.toBe(b); + expect(Object.isFrozen(a)).toBe(true); + expect(Object.isFrozen(a.serializer)).toBe(true); + expect(Object.isFrozen(a.deserializer)).toBe(true); +}); + +test('serialize encodes to UTF-8 JSON bytes', () => { + const bytes = jsonSerde().serializer.serialize({id: 1, name: 'ünïcode'}); + + expect(new TextDecoder().decode(bytes)).toBe('{"id":1,"name":"ünïcode"}'); +}); + +test('serializeToString is the fresh-string allocation profile SEAM-20 requires', () => { + const serde = jsonSerde(); + + expect(serde.serializer.serializeToString({id: 1, name: 'ünïcode'})).toBe( + '{"id":1,"name":"ünïcode"}', + ); + // The string and byte profiles are two views of one encoding, not two encoders that can drift. + expect(serde.serializer.serialize({a: 1})).toEqual( + new TextEncoder().encode(serde.serializer.serializeToString({a: 1})), + ); +}); + +test('a top-level value with no JSON representation throws, never encodes as null (SERDE-9)', () => { + // `JSON.stringify` returns the VALUE `undefined` for a top-level undefined, function, or symbol. + // Emitting the `null` literal instead — tempting, because a byte- or string-producing profile has + // to emit SOMETHING — substitutes a meaningful wire value ("clear this field", to a PATCH server) + // for a payload the caller cannot have meant to send. All three are unencodable values, which + // SERDE-9/SERDE-10 require surface as the stable serde type. SERDE-20's top-level Tristate + // degradation is the one case that legitimately encodes as `null`, and it is resolved before + // `JSON.stringify` runs rather than through this path — see tristate-replacer.test.ts. + const {serializer} = jsonSerde(); + + for (const unencodable of [undefined, () => 0, Symbol('x')]) { + expect(() => serializer.serializeToString(unencodable)).toThrow( + SerializationError, + ); + expect(() => serializer.serialize(unencodable)).toThrow(SerializationError); + expect(() => + serializer.serializeInto(unencodable, new Uint8Array(64)), + ).toThrow(SerializationError); + } +}); + +test('the unencodable-value message names the typeof, so the caller can see which it was', () => { + const {serializer} = jsonSerde(); + let caught: unknown; + try { + serializer.serializeToString(() => 0); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(SerializationError); + expect((caught as SerializationError).message).toContain('function'); + // A plain unencodable value has nothing to chain — there was no library error to preserve. + expect((caught as SerializationError).cause).toBeUndefined(); +}); + +test('a nested undefined or function still follows ordinary JSON.stringify rules', () => { + // Only the TOP level throws: nested, `JSON.stringify` drops the key, which is well-understood + // behaviour a caller relies on and is not this codec's to override. + const {serializer} = jsonSerde(); + + expect(serializer.serializeToString({a: 1, b: undefined})).toBe('{"a":1}'); + expect(serializer.serializeToString({a: 1, b: () => 0})).toBe('{"a":1}'); + expect(serializer.serializeToString([1, undefined, 2])).toBe('[1,null,2]'); +}); + +test('an unencodable value throws SerializationError, never the library type (SERDE-9)', () => { + const cyclic: Record<string, unknown> = {}; + cyclic.self = cyclic; + let caught: unknown; + + try { + jsonSerde().serializer.serialize(cyclic); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(SerializationError); + expect(caught).toHaveProperty('cause', expect.any(TypeError)); +}); + +test('every allocation profile routes an unencodable value through the same SDK type', () => { + const cyclic: Record<string, unknown> = {}; + cyclic.self = cyclic; + const {serializer} = jsonSerde(); + + expect(() => serializer.serializeToString(cyclic)).toThrow( + SerializationError, + ); + expect(() => serializer.serialize(cyclic)).toThrow(SerializationError); + expect(() => serializer.serializeInto(cyclic, new Uint8Array(64))).toThrow( + SerializationError, + ); +}); + +test('serializeInto honors an offset, returns the byte count, and leaves the prefix untouched (SERDE-4)', () => { + const target = new Uint8Array(64).fill(0xaa); + + const written = jsonSerde().serializer.serializeInto({a: 1}, target, 10); + + const expected = new TextEncoder().encode('{"a":1}'); + expect(written).toBe(expected.length); + expect(target.slice(10, 10 + written)).toEqual(expected); + expect(target.slice(0, 10)).toEqual(new Uint8Array(10).fill(0xaa)); + // Nothing past the written region is touched either — the buffer is the caller's. + expect(target.slice(10 + written)).toEqual( + new Uint8Array(64 - 10 - written).fill(0xaa), + ); +}); + +test('serializeInto with no offset writes at 0', () => { + const target = new Uint8Array(32); + + const written = jsonSerde().serializer.serializeInto({a: 1}, target); + + expect(target.slice(0, written)).toEqual(new TextEncoder().encode('{"a":1}')); +}); + +test('a payload that does not fit throws a plain RangeError with no cause (SERDE-4)', () => { + const target = new Uint8Array(3).fill(0xaa); + let caught: unknown; + + try { + jsonSerde().serializer.serializeInto({a: 1}, target); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(RangeError); + expect(caught).not.toBeInstanceOf(SerializationError); + expect((caught as RangeError).cause).toBeUndefined(); + // An overflow leaves the buffer exactly as it was — no partial write. + expect(target).toEqual(new Uint8Array(3).fill(0xaa)); +}); + +test('an out-of-range offset throws RangeError', () => { + const {serializer} = jsonSerde(); + + expect(() => + serializer.serializeInto({a: 1}, new Uint8Array(32), -1), + ).toThrow(RangeError); + expect(() => + serializer.serializeInto({a: 1}, new Uint8Array(32), 99), + ).toThrow(RangeError); + expect(() => + serializer.serializeInto({a: 1}, new Uint8Array(32), 1.5), + ).toThrow(RangeError); + expect(() => + serializer.serializeInto({a: 1}, new Uint8Array(32), Number.NaN), + ).toThrow(RangeError); +}); + +test('an exactly-fitting buffer is not an overflow', () => { + const expected = new TextEncoder().encode('{"a":1}'); + const target = new Uint8Array(expected.length); + + expect(jsonSerde().serializer.serializeInto({a: 1}, target)).toBe( + expected.length, + ); + expect(target).toEqual(expected); +}); + +test('serializeTo writes fully and never closes the caller-owned sink (SERDE-3)', async () => { + let closed = false; + let aborted = false; + const chunks: string[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(new TextDecoder().decode(chunk)); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + + await jsonSerde().serializer.serializeTo({a: 1}, sink); + + expect(chunks.join('')).toBe('{"a":1}'); + expect(closed).toBe(false); + expect(aborted).toBe(false); +}); + +test('serializeTo releases the writer lock, so the caller can keep using its own sink', async () => { + const chunks: string[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(new TextDecoder().decode(chunk)); + }, + }); + const serde = jsonSerde(); + + await serde.serializer.serializeTo({a: 1}, sink); + await serde.serializer.serializeTo({b: 2}, sink); + + expect(chunks.join('')).toBe('{"a":1}{"b":2}'); +}); + +test('serializeTo rejects an unencodable value without ever locking the caller-owned sink', async () => { + const cyclic: Record<string, unknown> = {}; + cyclic.self = cyclic; + const sink = new WritableStream<Uint8Array>(); + let caught: unknown; + + try { + await jsonSerde().serializer.serializeTo(cyclic, sink); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(SerializationError); + // Encoding happens before the lock is taken: a failed encode leaves the sink untouched and usable. + expect(sink.locked).toBe(false); +}); + +interface Dto { + readonly id: number; +} + +const dtoSchema: Schema<Dto> = { + parse(input: unknown): Dto { + // `as Dto`: probing one field on an `unknown` already proven a non-null object. + if ( + typeof input !== 'object' || + input === null || + typeof (input as Dto).id !== 'number' + ) { + throw new Error('not a Dto'); + } + return input as Dto; + }, +}; + +const bytes = (text: string): Uint8Array => new TextEncoder().encode(text); + +const streamOf = (text: string): ReadableStream<Uint8Array> => + new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(bytes(text)); + controller.close(); + }, + }); + +test('decode runs the schema over the parsed value (SERDE-5)', () => { + expect( + jsonSerde().deserializer.deserialize(bytes('{"id":3}'), { + schema: dtoSchema, + typeName: 'Dto', + }), + ).toEqual({id: 3}); +}); + +test('a parametric target is just a combinator schema — no carrier type exists (SERDE-6)', () => { + const arraySchema: Schema<readonly Dto[]> = { + // `as unknown[]`: JSON arrays arrive as `unknown`; the element schema validates each entry. + parse: input => (input as unknown[]).map(e => dtoSchema.parse(e)), + }; + + expect( + jsonSerde().deserializer.deserialize(bytes('[{"id":1},{"id":2}]'), { + schema: arraySchema, + }), + ).toEqual([{id: 1}, {id: 2}]); +}); + +test('malformed JSON throws DeserializationError with the library error chained (SERDE-9)', () => { + let caught: unknown; + + try { + jsonSerde().deserializer.deserialize(bytes('{not json'), { + schema: dtoSchema, + typeName: 'Dto', + }); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(DeserializationError); + expect(caught).toHaveProperty('cause', expect.any(SyntaxError)); +}); + +test('a schema rejection throws DeserializationError naming the target (SERDE-9)', () => { + let caught: unknown; + + try { + jsonSerde().deserializer.deserialize(bytes('{"id":"x"}'), { + schema: dtoSchema, + typeName: 'Dto', + }); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(DeserializationError); + expect(caught).toHaveProperty('message', expect.stringContaining('Dto')); + expect(caught).toHaveProperty('cause', expect.any(Error)); +}); + +/** + * Accepts anything, so a rejection can only have come from the codec. + * + * Driven through a permissive witness deliberately: with `dtoSchema` a deleted wire-null check still + * produces a `DeserializationError` naming `Dto` — the schema rejects `null` on its own and the wrapper + * message reads almost the same — so those assertions cannot tell the two rejectors apart. Verified by + * mutation: removing `decodeText`'s null branch left every `dtoSchema`-driven case green. + */ +const permissiveSchema = (): {schema: Schema<unknown>; ran: () => boolean} => { + let ran = false; + return { + schema: { + parse: i => { + ran = true; + return i; + }, + }, + ran: () => ran, + }; +}; + +/** The codec's own wire-null message, which a schema rejection cannot produce. */ +const WIRE_NULL_MESSAGE = + /wire null cannot be decoded into the non-null target/; + +test('a wire null into a non-null target fails naming the target, on every entry point (SERDE-13)', async () => { + const {deserializer} = jsonSerde(); + const first = permissiveSchema(); + const second = permissiveSchema(); + + expect(() => + deserializer.deserialize(bytes('null'), { + schema: first.schema, + typeName: 'Dto', + }), + ).toThrow(DeserializationError); + expect(() => + deserializer.deserialize(bytes('null'), { + schema: first.schema, + typeName: 'Dto', + }), + ).toThrow(WIRE_NULL_MESSAGE); + + const caught = await rejection( + deserializer.deserializeFrom(streamOf('null'), { + schema: second.schema, + typeName: 'Dto', + }), + ); + expect(caught).toBeInstanceOf(DeserializationError); + expect(caught).toHaveProperty( + 'message', + expect.stringMatching(WIRE_NULL_MESSAGE), + ); + expect(caught).toHaveProperty('message', expect.stringContaining('Dto')); + // Neither entry point reached the witness: the codec rejected, not the schema. + expect([first.ran(), second.ran()]).toEqual([false, false]); +}); + +test('the wire-null rejection is raised before the schema runs, so a permissive schema cannot swallow it', () => { + const permissive = permissiveSchema(); + + expect(() => + jsonSerde().deserializer.deserialize(bytes('null'), { + schema: permissive.schema, + typeName: 'Loose', + }), + ).toThrow(DeserializationError); + expect(permissive.ran()).toBe(false); +}); + +test('the null rejection falls back to a documented label when no typeName is given', () => { + const permissive = permissiveSchema(); + + expect(() => + jsonSerde().deserializer.deserialize(bytes('null'), { + schema: permissive.schema, + }), + ).toThrow( + /wire null cannot be decoded into the non-null target the target type/, + ); + expect(permissive.ran()).toBe(false); +}); + +test('deserializeFrom reads to EOF across multiple chunks and never cancels the source (SERDE-3)', async () => { + let cancelled = false; + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(bytes('{"id"')); + controller.enqueue(bytes(':42}')); + controller.close(); + }, + cancel() { + cancelled = true; + }, + }); + + expect( + await jsonSerde().deserializer.deserializeFrom(source, { + schema: dtoSchema, + typeName: 'Dto', + }), + ).toEqual({id: 42}); + expect(cancelled).toBe(false); +}); + +test('deserializeFrom releases the reader lock on the success path', async () => { + const source = streamOf('{"id":1}'); + + await jsonSerde().deserializer.deserializeFrom(source, { + schema: dtoSchema, + typeName: 'Dto', + }); + + expect(source.locked).toBe(false); +}); + +test('a genuine stream failure propagates unwrapped, and the lock is still released (SERDE-12)', async () => { + // Asserted with a plain sentinel rather than core's `IoError`: that class is deliberately + // package-private to `@dexpace/core` (Phase 3b froze `io/` as unexported), and the requirement is + // that the codec re-wraps NOTHING coming off the stream — which a sentinel proves more broadly. + const failure = new Error('socket reset'); + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.error(failure); + }, + }); + + const caught = await rejection( + jsonSerde().deserializer.deserializeFrom(source, { + schema: dtoSchema, + typeName: 'Dto', + }), + ); + + expect(caught).toBe(failure); + expect(caught).not.toBeInstanceOf(DeserializationError); +}); + +test('an empty body is a malformed payload, not a silent undefined (SERDE-9)', async () => { + const empty = new ReadableStream<Uint8Array>({ + start(controller) { + controller.close(); + }, + }); + + expect( + await rejection( + jsonSerde().deserializer.deserializeFrom(empty, { + schema: dtoSchema, + typeName: 'Dto', + }), + ), + ).toBeInstanceOf(DeserializationError); +}); + +test('a UTF-8 payload split mid-multi-byte-character across chunks decodes correctly', async () => { + const full = bytes('{"id":1,"n":"ü"}'); + const split = full.indexOf(0xc3); // the first byte of "ü" + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(full.slice(0, split + 1)); + controller.enqueue(full.slice(split + 1)); + controller.close(); + }, + }); + // `as {id: number; n: string}`: a deliberately loose schema; the assertion is about bytes, not shape. + const looseSchema: Schema<{id: number; n: string}> = { + parse: i => i as {id: number; n: string}, + }; + + expect( + await jsonSerde().deserializer.deserializeFrom(source, { + schema: looseSchema, + }), + ).toEqual({id: 1, n: 'ü'}); +}); + +test("serializeInto writes through a subarray VIEW at the view's own coordinates (SERDE-4)", () => { + // `target` is typed `Uint8Array`, and a caller carving one out of a pool hands over a view with a + // non-zero `byteOffset`. `offset` must be relative to the VIEW, the fit check must use the view's + // length rather than the backing buffer's, and nothing outside the view may be touched. + const backing = new Uint8Array(24).fill(0xaa); + const view = backing.subarray(4, 16); // 12 bytes, byteOffset 4 + const {serializer} = jsonSerde(); + + const written = serializer.serializeInto({a: 1}, view, 2); + const payload = serializer.serialize({a: 1}); + + expect(written).toBe(payload.length); + // Landed at byteOffset + offset, not at the backing buffer's absolute `offset`. Compared as plain + // arrays: `serialize` returns `Uint8Array<ArrayBufferLike>` and `subarray` here is + // `Uint8Array<ArrayBuffer>`, which `toEqual`'s overloads will not unify. + expect(Array.from(backing.subarray(6, 6 + written))).toEqual( + Array.from(payload), + ); + // Everything outside [6, 6+written) is untouched, on BOTH sides of the view. + expect(backing.subarray(0, 6).every(b => b === 0xaa)).toBe(true); + expect(backing.subarray(6 + written).every(b => b === 0xaa)).toBe(true); +}); + +test('the fit check measures the view, not the buffer behind it (SERDE-4)', () => { + // A 4-byte window onto a 64-byte buffer has room for 4 bytes, not 64. Measuring the backing + // buffer would let the write run past the window the caller actually lent out. + const backing = new Uint8Array(64).fill(0xaa); + const view = backing.subarray(0, 4); + + expect(() => jsonSerde().serializer.serializeInto({a: 1}, view, 0)).toThrow( + RangeError, + ); + expect(backing.every(b => b === 0xaa)).toBe(true); +}); + +describe('DecodeTarget object form, admitsNull, and {signal} (H9/H10/H15 batch, 2026-09-04)', () => { + const serde = jsonSerde(); + const passthrough: Schema<unknown> = {parse: (i: unknown) => i}; + + test('deserialize takes a DecodeTarget, not positional schema/typeName', () => { + const bytes = new TextEncoder().encode('{"a":1}'); + expect( + serde.deserializer.deserialize(bytes, {schema: passthrough}), + ).toEqual({a: 1}); + }); + + test('the typeName still reaches the error message through the target', () => { + const bytes = new TextEncoder().encode('null'); + expect(() => + serde.deserializer.deserialize(bytes, { + schema: passthrough, + typeName: 'Pet', + }), + ).toThrow(/non-null target Pet/); + }); + + test('admitsNull lets a top-level wire null through to the schema (SERDE-13 opt-in)', () => { + const bytes = new TextEncoder().encode('null'); + expect( + serde.deserializer.deserialize(bytes, { + schema: passthrough, + admitsNull: true, + }), + ).toBeNull(); + }); + + test('admitsNull is off by default, so the unconditional rejection is unchanged', () => { + const bytes = new TextEncoder().encode('null'); + expect(() => + serde.deserializer.deserialize(bytes, {schema: passthrough}), + ).toThrow(DeserializationError); + }); +}); + +describe('{signal} on the two stream-driving SPI methods (H15)', () => { + const serde = jsonSerde(); + const passthrough: Schema<unknown> = {parse: (i: unknown) => i}; + + test('deserializeFrom honors an already-aborted signal before reading', async () => { + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{"a":1}')); + controller.close(); + }, + }); + expect( + await rejection( + serde.deserializer.deserializeFrom( + source, + {schema: passthrough}, + {signal: AbortSignal.abort()}, + ), + ), + ).toBeInstanceOf(Error); + }); + + test('deserializeFrom leaves the source uncancelled when the signal aborts (SERDE-3)', async () => { + let cancelled = false; + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{"a":1}')); + controller.close(); + }, + cancel() { + cancelled = true; + }, + }); + await serde.deserializer + .deserializeFrom( + source, + {schema: passthrough}, + {signal: AbortSignal.abort()}, + ) + .catch(() => undefined); + expect(cancelled).toBe(false); + }); + + test('serializeTo honors an already-aborted signal and leaves the sink unclosed', async () => { + let closed = false; + const sink = new WritableStream<Uint8Array>({ + close() { + closed = true; + }, + }); + expect( + await rejection( + serde.serializer.serializeTo({a: 1}, sink, { + signal: AbortSignal.abort(), + }), + ), + ).toBeInstanceOf(Error); + expect(closed).toBe(false); + }); +}); + +describe('the options argument stays optional on both stream methods', () => { + const serde = jsonSerde(); + const passthrough: Schema<unknown> = {parse: (i: unknown) => i}; + + test('an absent options argument keeps both stream methods working', async () => { + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{"a":1}')); + controller.close(); + }, + }); + expect( + await serde.deserializer.deserializeFrom(source, {schema: passthrough}), + ).toEqual({a: 1}); + }); +}); + +// Module-scope, not describe-local: the pending-abort suite is split across sibling describes to +// stay inside `max-lines-per-function`, and both halves need these. +const PENDING_ABORT_SERDE = jsonSerde(); +const passthroughSchema: Schema<unknown> = {parse: (i: unknown) => i}; + +/** + * Hands over one chunk and then never produces another, so the drain parks *inside* + * `reader.read()` — the state a between-chunks signal check structurally cannot observe. + */ +function stallingSource( + first: string, + onCancel?: () => void, +): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(first)); + }, + pull() { + return new Promise<never>(() => undefined); + }, + cancel() { + onCancel?.(); + }, + }); +} + +describe('an abort that lands while a READ is pending (audit #67 / #79)', () => { + test('deserializeFrom settles with the caller reason and unlocks the source (SERDE-3)', async () => { + let cancelled = false; + const source = stallingSource('{"a":', () => { + cancelled = true; + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + // The whole point of the fix: the caller gets its stream back, still usable. + expect(source.locked).toBe(false); + expect(cancelled).toBe(false); + }); + + test('an abort with no reason surfaces the platform AbortError the seam documents', async () => { + const source = stallingSource('{"a":'); + const controller = new AbortController(); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(); + + expect(await settleWithin(settled, SETTLE_MS)).toMatchObject({ + name: 'AbortError', + }); + expect(source.locked).toBe(false); + }); +}); + +describe('an abort that lands while a WRITE is pending (audit #67 / #79)', () => { + test('serializeTo settles with the caller reason and unlocks the sink (SERDE-3)', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream<Uint8Array>({ + write() { + return new Promise<never>(() => undefined); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + + const settled = rejection( + PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + expect(sink.locked).toBe(false); + expect(closed).toBe(false); + expect(aborted).toBe(false); + }); + + test('a signal that never fires leaves both directions unchanged', async () => { + const controller = new AbortController(); + const source = new ReadableStream<Uint8Array>({ + start(streamController) { + streamController.enqueue(new TextEncoder().encode('{"a":')); + streamController.enqueue(new TextEncoder().encode('1}')); + streamController.close(); + }, + }); + const written: string[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + written.push(new TextDecoder().decode(chunk)); + }, + }); + + expect( + await PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ).toEqual({a: 1}); + await PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }); + + expect(written.join('')).toBe('{"a":1}'); + expect(source.locked).toBe(false); + expect(sink.locked).toBe(false); + }); +}); diff --git a/packages/codec-json/src/json-serde.ts b/packages/codec-json/src/json-serde.ts new file mode 100644 index 0000000..d33cf67 --- /dev/null +++ b/packages/codec-json/src/json-serde.ts @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/json-serde.ts +import { + DeserializationError, + SerializationError, + type DecodeTarget, + type Deserializer, + type Serde, + type Serializer, +} from '@dexpace/core'; +import {abortRace} from './abort-race.js'; +import { + degradeTopLevelTristate, + tristateReplacer, +} from './tristate-replacer.js'; + +/** + * Options for {@link jsonSerde}. + * + * @public + */ +export interface JsonSerdeOptions { + /** + * Install the `Tristate` PATCH wiring (SERDE-19). Without it, Absent and Null are + * indistinguishable on the wire, which silently turns "leave unchanged" into "clear". + * + * Set to `false` **only** when the caller has already installed equivalent wiring. Never silent — + * the caller has to name it. + * + * @defaultValue `true` + */ + readonly tristate?: boolean | undefined; +} + +/** + * Whether the Tristate wiring is installed, carried as one value rather than inferred from the + * replacer being `undefined`: the top-level degradation and the replacer are two halves of the same + * opt-in (SERDE-19), and reading one off the other couples them by convention alone. + */ +interface TristateWiring { + readonly installed: boolean; + readonly replacer: ((key: string, value: unknown) => unknown) | undefined; +} + +const ENCODER = new TextEncoder(); +const MEDIA_TYPE = 'application/json'; + +/** + * `JSON.stringify` is declared as returning `string`, and does not: it returns `undefined` for a + * top-level `undefined`, function, or symbol. Kept as a named seam even at one call site, so the + * correction is attached to the trap rather than buried in the caller's control flow. + */ +function stringifyOrUndefined( + value: unknown, + replacer: TristateWiring['replacer'], +): string | undefined { + return JSON.stringify(value, replacer); +} + +function encodeToText(value: unknown, wiring: TristateWiring): string { + // SERDE-20's top-level degradation runs HERE, before `JSON.stringify`, not inside the replacer. + // A replacer sees `key === ''` both for the top-level value and for an ordinary key named `''`, + // and cannot tell them apart — so handling it there emits a wire null for `{"": absent()}`, which + // SERDE-15 requires be omitted. This is the one place that knows which value is the root. + // Skipped entirely when the caller opted out: `{tristate: false}` means "I have installed + // equivalent wiring", and degrading behind their back would defeat that. + const root = wiring.installed ? degradeTopLevelTristate(value) : value; + + let text: string | undefined; + try { + text = stringifyOrUndefined(root, wiring.replacer); + } catch (e: unknown) { + throw new SerializationError('failed to encode value as JSON', {cause: e}); + } + + if (text === undefined) { + // Reachable only for a top-level `undefined`, function, or symbol — a top-level Tristate was + // resolved above. All three are unencodable values, which SERDE-9/SERDE-10 require surface as + // the stable serde type. Emitting `null` instead would silently substitute a meaningful wire + // value for a payload the caller could not have meant to send. + throw new SerializationError( + `a top-level ${typeof root} value has no JSON representation`, + ); + } + return text; +} + +function encodeToBytes(value: unknown, wiring: TristateWiring): Uint8Array { + return ENCODER.encode(encodeToText(value, wiring)); +} + +function makeSerializer(wiring: TristateWiring): Serializer { + return Object.freeze({ + serializeToString(value: unknown): string { + return encodeToText(value, wiring); + }, + + serialize(value: unknown): Uint8Array { + return encodeToBytes(value, wiring); + }, + + serializeInto(value: unknown, target: Uint8Array, offset = 0): number { + // Range-checked before encoding, so a bad offset costs nothing and the caller's buffer is + // never partially written. `Number.isInteger` also rejects NaN and Infinity. + if (!Number.isInteger(offset) || offset < 0 || offset > target.length) { + throw new RangeError( + `offset ${String(offset)} is out of range for a buffer of ${String(target.length)} bytes`, + ); + } + const bytes = encodeToBytes(value, wiring); + if (bytes.length > target.length - offset) { + // SERDE-4: an overflow is a RangeError, distinct from the serde type and with no cause + // chain. Thrown BEFORE `set`, so `[0, offset)` and everything past it are untouched. + throw new RangeError( + `encoded payload of ${String(bytes.length)} bytes does not fit in ${String( + target.length - offset, + )} available bytes`, + ); + } + target.set(bytes, offset); + return bytes.length; + }, + + async serializeTo( + value: unknown, + sink: WritableStream<Uint8Array>, + options?: {readonly signal?: AbortSignal | undefined}, + ): Promise<void> { + // Encoded before the lock is taken: a failed encode then leaves the caller's sink untouched + // and still usable, rather than locked-and-released around a write that never happened. + const bytes = encodeToBytes(value, wiring); + // Checked after the encode and before the lock, so an aborted call never takes the lock at + // all (SERDE-3). + const signal = options?.signal; + signal?.throwIfAborted(); + const writer = sink.getWriter(); + // After `getWriter()`, so a contended sink leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); + try { + // Raced, not merely checked before: one write is still one operation that can park + // indefinitely against a slow sink, and the abort has to reach it (audit #67 / #79). + await race.race(writer.write(bytes)); + } finally { + race.release(); + // Release the lock, never close: the sink is caller-owned (SERDE-3). A write still + // outstanding at this point stays outstanding — aborting it is the owner's call, not ours. + writer.releaseLock(); + } + }, + }); +} + +const UNNAMED_TARGET = 'the target type'; + +function decodeText<T>(text: string, decodeTarget: DecodeTarget<T>): T { + const {schema, typeName, admitsNull} = decodeTarget; + const target = typeName ?? UNNAMED_TARGET; + + let parsed: unknown; + try { + // `as unknown`: JSON.parse is typed `any`, which would silently infect everything downstream. + // The cast narrows *away* from `any`, the one direction the type-system chapter asks for at a + // boundary. + parsed = JSON.parse(text) as unknown; + } catch (e: unknown) { + throw new DeserializationError(`malformed JSON while decoding ${target}`, { + cause: e, + }); + } + + // SERDE-13, checked here rather than delegated: a schema library may or may not reject a bare + // null, and may or may not name the target when it does. Checking in the codec makes the + // behaviour uniform across every entry point and every schema library a caller might supply. + // This is also the single funnel that makes SERDE-13's "across every decode overload" true for + // this codec — `deserialize` and `deserializeFrom` both route through it. + // `admitsNull` is the caller stating what the schema value cannot: that `T` includes `null`. Off + // by default, so the rejection stays unconditional for every target that does not opt in. + if (parsed === null && admitsNull !== true) { + throw new DeserializationError( + `wire null cannot be decoded into the non-null target ${target}`, + ); + } + + try { + return schema.parse(parsed); + } catch (e: unknown) { + throw new DeserializationError( + `value does not match the schema for ${target}`, + {cause: e}, + ); + } +} + +function makeDeserializer(): Deserializer { + return Object.freeze({ + deserialize<T>(data: Uint8Array, target: DecodeTarget<T>): T { + return decodeText(new TextDecoder().decode(data), target); + }, + + async deserializeFrom<T>( + source: ReadableStream<Uint8Array>, + target: DecodeTarget<T>, + options?: {readonly signal?: AbortSignal | undefined}, + ): Promise<T> { + // `text` accumulates the WHOLE body before parsing, and is deliberately uncapped. + // + // SERDE-27 asks a decoder not to materialize the body. `JSON.parse` has no incremental form, + // so this codec cannot honor that — a limitation of the format, not of the seam: + // `decodeResponse` hands over the live stream and never buffers, and a codec with a streaming + // parser satisfies SERDE-27 fully behind this same interface. Recorded in the phase's + // Deviation Ledger. + // + // No byte cap: truncating a legitimate large payload is a worse failure than the memory it + // would save, and a caller who needs a bound imposes it on the transport, where the whole + // response is bounded at once. + // + // A streaming TextDecoder keeps multi-byte characters intact across chunk boundaries; decoding + // each chunk independently would corrupt any character split across two reads. + // Checked before the lock so an aborted call leaves the source neither locked nor cancelled + // (SERDE-3), and raced against every read so a stalled one stops too. + const signal = options?.signal; + signal?.throwIfAborted(); + const decoder = new TextDecoder('utf-8'); + const reader = source.getReader(); + // After `getReader()`, so a contended source leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); + let text = ''; + try { + for (;;) { + // Serial by necessity: each read depends on the previous one advancing the cursor. Raced + // rather than checked between reads: a source that stalls mid-body parks the loop inside + // `read()`, where a between-chunks check never runs again (audit #67 / #79). + const {done, value} = await race.race(reader.read()); + if (done) break; + text += decoder.decode(value, {stream: true}); + } + text += decoder.decode(); + } finally { + race.release(); + // Release the lock, never cancel: the source is caller-owned (SERDE-3). A stream failure + // surfaces from `read()` and propagates unwrapped (SERDE-12) — it is not caught here. + // Releasing with a read outstanding is legal on every supported runtime and unlocks the + // stream; the outstanding read rejects, and `Promise.race` above still owns that rejection. + reader.releaseLock(); + } + return decodeText(text, target); + }, + }); +} + +/** + * Build a fresh JSON `Serde` bundle (SERDE-1, SERDE-2, SERDE-25). + * + * The bundle is frozen and stateless, so one instance safely serves every DTO and every concurrent + * operation in an application (SERDE-29) — the payload type arrives as a `Schema` parameter of each + * decode call, not as a property of the bundle. + * + * **Unknown wire fields (SERDE-23).** This codec does not strip or reject them — that is your + * schema's decision. Prefer the permissive default (Zod's `.parse()` strips unknown keys; + * `.strict()` rejects them), so a server adding a backward-compatible field does not break clients + * that have not been regenerated yet. If you opt into a strict schema, you are opting out of that + * forward compatibility deliberately. + * + * **Coercion (SERDE-21/SERDE-22).** There is no coercion setting because there is no coercing codec: + * `JSON.parse` reshapes nothing, so `{"x":"5"}` yields the string `"5"` and a number-typed schema + * rejects it. Representation-preserving binding still works, because JavaScript has one numeric type. + * + * **Values with no JSON representation (SERDE-9/SERDE-10).** A top-level `undefined`, function, or + * symbol raises a `SerializationError` rather than encoding as the `null` literal. `JSON.stringify` + * returns the *value* `undefined` for all three, and substituting `null` would put a meaningful + * wire value in place of a payload the caller cannot have meant to send. Nested occurrences follow + * ordinary `JSON.stringify` rules (the key is dropped; an array element becomes `null`). + * + * **Top-level Tristate degradation (SERDE-20)** is resolved by this bundle's serializer *before* + * `JSON.stringify` runs, not by the replacer: a replacer cannot tell the top-level value from an + * ordinary key named `''`. A top-level Absent or Null therefore still encodes as `null` here, while + * `{"": absent()}` correctly omits the key. A caller composing their own + * `JSON.stringify(v, tristateReplacer)` gets the nested and array-element behaviour but not the + * top-level degradation — see {@link tristateReplacer}. + * + * **A top-level wire `null` decodes only into a target that admits one (SERDE-13).** A schema value + * carries no nullability this codec could read, so the rejection is unconditional *by default* and + * runs *before* the schema: a `200` whose entire body is the literal `null` raises a + * `DeserializationError`. Setting `admitsNull: true` on the `DecodeTarget` is the caller stating what + * the schema value cannot — that `T` includes `null` — and skips the check, which is the one case + * where `tristate(inner)` serves as a top-level target rather than a field combinator for use inside + * `tristateObject`. Checking after the schema instead would let a permissive schema such as + * `{parse: (i) => i}` return that `null` as a non-null `T`. + * + * @param options - opt out of the Tristate wiring; everything else is fixed by the format. + * @returns a frozen, stateless bundle safe to share across concurrent operations (SERDE-29). + * @public + */ +export function jsonSerde(options?: JsonSerdeOptions): Serde { + const installed = options?.tristate ?? true; + const wiring: TristateWiring = { + installed, + replacer: installed ? tristateReplacer : undefined, + }; + return Object.freeze({ + mediaType: MEDIA_TYPE, + serializer: makeSerializer(wiring), + deserializer: makeDeserializer(), + }); +} diff --git a/packages/codec-json/src/tristate-replacer.test.ts b/packages/codec-json/src/tristate-replacer.test.ts new file mode 100644 index 0000000..001f99c --- /dev/null +++ b/packages/codec-json/src/tristate-replacer.test.ts @@ -0,0 +1,165 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/tristate-replacer.test.ts +// Exercises: SERDE-15 (Absent omits the key, Null emits a wire null, Present emits the value — including +// under a key literally named `""`, which a top-level check cannot be distinguished from), SERDE-19 +// (installed by default; opt-out is explicit), SERDE-20 (degradation in the two positions that cannot omit +// a key: the top level, resolved in `jsonSerde()` before `JSON.stringify`, and an array element, which +// `JSON.stringify` itself renders as `null`). +import {expect, test} from 'bun:test'; +import {absent, nullValue, present} from '@dexpace/core'; +import {jsonSerde} from './json-serde.js'; +import {tristateReplacer} from './tristate-replacer.js'; + +const encode = (value: unknown, tristate = true): string => + new TextDecoder().decode(jsonSerde({tristate}).serializer.serialize(value)); + +test('Absent omits the key entirely (SERDE-15)', () => { + expect(encode({name: 'a', nickname: absent()})).toBe('{"name":"a"}'); +}); + +test('Null emits the key with a wire null (SERDE-15)', () => { + expect(encode({name: 'a', nickname: nullValue()})).toBe( + '{"name":"a","nickname":null}', + ); +}); + +test('Present emits the key with the encoded inner value (SERDE-15)', () => { + expect(encode({name: 'a', nickname: present('bee')})).toBe( + '{"name":"a","nickname":"bee"}', + ); +}); + +test('a Present carrying an object encodes the object, not the wrapper', () => { + expect(encode({at: present({deep: 1})})).toBe('{"at":{"deep":1}}'); +}); + +test('a nested Tristate inside a Present value is still rewritten', () => { + expect(encode({at: present({keep: absent(), clear: nullValue()})})).toBe( + '{"at":{"clear":null}}', + ); +}); + +test('the wiring is on by default (SERDE-19)', () => { + expect( + new TextDecoder().decode(jsonSerde().serializer.serialize({x: absent()})), + ).toBe('{}'); +}); + +test('opting out is explicit, and then Absent and Null become indistinguishable (SERDE-19)', () => { + const out = encode({x: absent()}, false); + + // Without the wiring the raw union shape leaks — which is exactly why the option must be named, + // never silent. The brand is a symbol, so JSON.stringify drops it and only `kind` survives. + expect(out).toBe('{"x":{"kind":"absent"}}'); + expect(encode({x: nullValue()}, false)).toBe('{"x":{"kind":"null"}}'); +}); + +test('a top-level Absent or Null degrades to a wire null rather than throwing (SERDE-20)', () => { + expect(encode(absent())).toBe('null'); + expect(encode(nullValue())).toBe('null'); +}); + +test('a top-level Present unwraps to its inner value', () => { + expect(encode(present(7))).toBe('7'); +}); + +test('an array-element Absent emits null rather than shifting or dropping the element (SERDE-20)', () => { + // Characterization, not a branch test: the replacer returns `undefined` here exactly as it does for + // an object key, and `JSON.stringify` renders a dropped ARRAY element as `null` on its own. That + // platform behaviour is where SERDE-20's array half comes from — see tristate-replacer.ts. + expect(encode([present(1), absent(), nullValue()])).toBe('[1,null,null]'); +}); + +test('a nested array keeps the same degradation, so indices never shift at depth', () => { + expect(encode({xs: [absent(), present('a')]})).toBe('{"xs":[null,"a"]}'); +}); + +test('a caller value that merely looks like a Tristate is left alone', () => { + expect(encode({x: {kind: 'absent'}})).toBe('{"x":{"kind":"absent"}}'); + expect(encode({x: {kind: 'present', value: 1}})).toBe( + '{"x":{"kind":"present","value":1}}', + ); +}); + +test('the replacer is exported so a caller can compose their own JSON.stringify call', () => { + expect( + JSON.stringify({keep: absent(), clear: nullValue()}, tristateReplacer), + ).toBe('{"clear":null}'); +}); + +// --- SERDE-15 under a key literally named "" --------------------------------------------------- +// +// `JSON.stringify` calls a replacer with `key === ''` for the top-level value AND for an ordinary +// key that is the empty string. Detecting "top level" as `key === ''` would therefore emit a wire +// `null` for `{"": absent()}` — silently turning "leave unchanged" into "clear", which is exactly +// the corruption SERDE-19 says the wiring exists to prevent. The top-level case resolves in +// `jsonSerde()` before `JSON.stringify` runs, so the replacer never has to guess; these cases pin +// that the replacer treats `""` as an ordinary key at every depth. + +test('an Absent under a key named "" is omitted like any other key (SERDE-15)', () => { + expect(encode({'': absent()})).toBe('{}'); + expect(encode({a: 1, '': absent()})).toBe('{"a":1}'); +}); + +test('a Null under a key named "" still emits the wire null', () => { + expect(encode({'': nullValue()})).toBe('{"":null}'); +}); + +test('a Present under a key named "" emits its inner value', () => { + expect(encode({'': present(7)})).toBe('{"":7}'); +}); + +test('the "" key behaves the same at depth and inside an array element', () => { + expect(encode({x: {'': absent()}})).toBe('{"x":{}}'); + expect(encode([{'': absent()}])).toBe('[{}]'); + expect(encode({xs: [{'': absent(), keep: 1}]})).toBe('{"xs":[{"keep":1}]}'); +}); + +test('a top-level Tristate degrades on every allocation profile, not only through serialize (SERDE-20)', () => { + // The degradation lives in `jsonSerde()`'s `encodeToText`, which every profile routes through — so + // the string profile and the byte profile cannot disagree about the root. + const {serializer} = jsonSerde(); + + expect(serializer.serializeToString(absent())).toBe('null'); + expect(new TextDecoder().decode(serializer.serialize(nullValue()))).toBe( + 'null', + ); + expect(serializer.serializeToString(present(7))).toBe('7'); +}); + +test('the exported replacer leaves the top-level position to the caller, which is documented', () => { + // The cost of resolving the `""`-key ambiguity: a caller composing their own `JSON.stringify` gets + // the nested and array-element behaviour but must route through `jsonSerde()` for the top level. + expect(JSON.stringify(absent(), tristateReplacer)).toBeUndefined(); + // Nested and array positions are unaffected. + expect(JSON.stringify({keep: absent()}, tristateReplacer)).toBe('{}'); + expect(JSON.stringify([absent()], tristateReplacer)).toBe('[null]'); +}); + +// --- a Tristate nested directly inside a Tristate ---------------------------------------------- +// +// `present()` takes `NonNullable<T>` and a Tristate is a non-null object, so `Tristate<Tristate<T>>` +// is well-typed; `tristate(tristate(inner))` builds one on the decode side. A replacer's return +// value is never fed back through the replacer, so a single unwrap would put this SDK's internal +// discriminant — `{"kind":"present","value":1}`, brand symbol and all — on the wire. The walk has to +// run to the bottom, which is what these cases pin. + +test('a Present wrapping a Present encodes the innermost value, not the wrapper', () => { + expect(encode({a: present(present(1))})).toBe('{"a":1}'); + expect(encode({a: present(present(present('x')))})).toBe('{"a":"x"}'); +}); + +test('a Present wrapping an Absent takes the Absent decision for its position', () => { + expect(encode({a: present(absent()), b: 1})).toBe('{"b":1}'); + expect(encode([present(absent())])).toBe('[null]'); +}); + +test('a Present wrapping a Null emits a wire null', () => { + expect(encode({a: present(nullValue())})).toBe('{"a":null}'); +}); + +test('the same resolution applies at the top level', () => { + expect(encode(present(present(1)))).toBe('1'); + expect(encode(present(absent()))).toBe('null'); + expect(encode(present(nullValue()))).toBe('null'); +}); diff --git a/packages/codec-json/src/tristate-replacer.ts b/packages/codec-json/src/tristate-replacer.ts new file mode 100644 index 0000000..b137e1c --- /dev/null +++ b/packages/codec-json/src/tristate-replacer.ts @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/tristate-replacer.ts +import {isTristate, type Tristate} from '@dexpace/core'; + +/** + * Resolve through a chain of nested Presents to the Tristate that actually decides the wire form. + * + * `present<T>(value: NonNullable<T>)` accepts a Tristate — one is a non-null object — so + * `Tristate<Tristate<T>>` is a well-typed value the public API constructs happily, and + * `tristate(tristate(inner))` produces one on the decode side. Without this walk the replacer + * returns the *inner sentinel object*, and `JSON.stringify` then serializes that object's own + * properties, putting `{"kind":"present","value":1}` — this SDK's internal discriminant — on the + * wire. A replacer's return value is never fed back through the replacer, so one unwrap is not + * enough; the walk has to run to the bottom. + * + * Module-private: both call sites (`degradeTopLevelTristate` and `tristateReplacer`) live in this + * file, so exporting it would widen the module's surface for no consumer. + * + * @param value - the outermost Tristate. + * @returns the innermost Tristate: never a Present whose value is itself a Tristate. + */ +function innermostTristate(value: Tristate<unknown>): Tristate<unknown> { + let current = value; + while (current.kind === 'present' && isTristate(current.value)) { + current = current.value; + } + return current; +} + +/** + * The wire form of a Tristate in a position that cannot omit a key (SERDE-20). + * + * Used for the **top-level** value, which `jsonSerde()`'s serializer resolves before calling + * `JSON.stringify` — see {@link tristateReplacer} for why that cannot happen inside the replacer. + * + * @param value - any top-level value; returned unchanged when it is not a Tristate. + * @returns the inner value for Present, `null` for Absent and Null alike. + * + * @internal + */ +export function degradeTopLevelTristate(value: unknown): unknown { + if (!isTristate(value)) return value; + const resolved = innermostTristate(value); + return resolved.kind === 'present' ? resolved.value : null; +} + +/** + * `JSON.stringify` replacer implementing PATCH three-state semantics (SERDE-15). + * + * Absent → the key is omitted entirely (a PATCH server reads that as "leave unchanged"). + * Null → the key is emitted with a wire `null` ("clear"). + * Present → the key is emitted with the inner value. + * + * Returning `undefined` from a replacer makes `JSON.stringify` drop the key — the exact mechanism + * SERDE-15 needs, built into the language. SERDE-20's array half comes from the same mechanism and + * needs no code here: an **array element** cannot be dropped without shifting every index after it, + * so `JSON.stringify` itself emits `null` for an element whose replacer returned `undefined`. This + * function therefore treats every non-top-level position identically. + * + * **This replacer does not handle the top-level position, and cannot.** `JSON.stringify` invokes a + * replacer for the top-level value with `key === ''`, but so does an ordinary object key that is + * literally the empty string — `{"": absent()}` is legal JSON at any depth, and the two cases are + * indistinguishable from `(key, value)` alone. Testing `key === ''` would emit a wire `null` for a + * `""` key that SERDE-15 requires be omitted, silently turning "leave unchanged" into "clear". + * `jsonSerde()` resolves a top-level Tristate *before* calling `JSON.stringify`, via this module's + * `degradeTopLevelTristate`, which is the one place that can tell the two apart. + * + * The consequence for a caller composing their own `JSON.stringify(value, tristateReplacer)` call: + * a **top-level** Tristate is not degraded here and `JSON.stringify` returns `undefined` for a + * top-level Absent. Nested and array-element positions behave exactly as documented above. Route + * through `jsonSerde()`'s serializer if the top-level case matters. + * + * Installed by `jsonSerde()` by default (SERDE-19). + * + * @param key - the key being serialized; unused, because every position this function sees takes the + * same decision — see the top-level note above. + * @param value - the value at that key, before encoding. + * @returns the value to encode, or `undefined` to omit the key. + * @public + */ +export function tristateReplacer(key: string, value: unknown): unknown { + if (!isTristate(value)) return value; + + const resolved = innermostTristate(value); + if (resolved.kind === 'present') return resolved.value; + + // Absent means "omit the key", which is SERDE-15's central interop invariant. In an array position + // `JSON.stringify` renders the dropped element as `null` on its own, which is where SERDE-20's + // array-element degradation actually comes from — no branch here produces it. + return resolved.kind === 'absent' ? undefined : null; +} diff --git a/packages/codec-json/src/tristate-schema.test.ts b/packages/codec-json/src/tristate-schema.test.ts new file mode 100644 index 0000000..2c68193 --- /dev/null +++ b/packages/codec-json/src/tristate-schema.test.ts @@ -0,0 +1,262 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/tristate-schema.test.ts +// Exercises: SERDE-14 (three states and only three — a Present can never carry null), SERDE-16 (missing → +// Absent, explicit null → Null, value → Present with element type preserved), SERDE-17 (a missing key resolves +// to Absent via the combinator's own default, not a JSON.parse reviver), SERDE-29 (both combinators return +// frozen schemas, so a shared schema cannot acquire state). +import {expect, test} from 'bun:test'; +import { + DeserializationError, + valueOrNull, + type Schema, + type Tristate, +} from '@dexpace/core'; +import {expectTypeOf} from 'expect-type'; +import {MISSING, tristate, tristateObject} from './tristate-schema.js'; + +/** A witness that validates nothing — these cases are about key resolution, not value shape. */ +const identity: Schema<unknown> = {parse: input => input}; + +const numberSchema: Schema<number> = { + parse(input: unknown): number { + if (typeof input !== 'number') throw new Error('not a number'); + return input; + }, +}; + +const stringSchema: Schema<string> = { + parse(input: unknown): string { + if (typeof input !== 'string') throw new Error('not a string'); + return input; + }, +}; + +test('an explicit null decodes to Null (SERDE-16)', () => { + expect(tristate(numberSchema).parse(null).kind).toBe('null'); +}); + +test('a present value decodes to Present with the inner schema applied (SERDE-16)', () => { + const decoded = tristate(numberSchema).parse(5); + + // Assert the fields directly. Spreading `decoded` into its own expectation would be a tautology + // that passes for any input. + expect(decoded.kind).toBe('present'); + expect(decoded.kind === 'present' ? decoded.value : undefined).toBe(5); +}); + +test('the inner schema still rejects a wrong-typed present value', () => { + expect(() => tristate(numberSchema).parse('5')).toThrow(); +}); + +test('the missing sentinel decodes to Absent (SERDE-17)', () => { + expect(tristate(numberSchema).parse(MISSING).kind).toBe('absent'); +}); + +test('undefined also decodes to Absent, so a hand-built object literal behaves the same way', () => { + expect(tristate(numberSchema).parse(undefined).kind).toBe('absent'); +}); + +test('tristateObject maps a missing key to Absent and a present key through the field schema (SERDE-17)', () => { + const schema = tristateObject({age: numberSchema}); + + expect(schema.parse({}).age.kind).toBe('absent'); + expect(schema.parse({age: null}).age.kind).toBe('null'); + const decoded = schema.parse({age: 30}).age; + expect(decoded.kind === 'present' ? decoded.value : undefined).toBe(30); +}); + +test('an explicitly-undefined key is Absent, not Null — the key exists but carried no wire value', () => { + expect( + tristateObject({age: numberSchema}).parse({age: undefined}).age.kind, + ).toBe('absent'); +}); + +test('tristateObject leaves non-tristate keys untouched at runtime', () => { + const schema = tristateObject({age: numberSchema}); + + const parsed = schema.parse({age: 1, other: 'kept'}); + + expect(parsed.age.kind).toBe('present'); + // `as Record<string, unknown>`: the pass-through keys are still THERE at runtime; the return type + // deliberately names only the `shape` keys, so reaching one is the caller's explicit widening. + // An index signature on the return type would have made every misspelled key compile silently. + expect((parsed as Record<string, unknown>).other).toBe('kept'); +}); + +test('tristateObject does not mutate the object it was handed', () => { + const schema = tristateObject({age: numberSchema}); + const source = {age: 1, other: 'kept'}; + + schema.parse(source); + + expect(source.age).toBe(1); +}); + +test('tristateObject rejects a non-object input rather than producing an empty shape', () => { + const schema = tristateObject({age: numberSchema}); + + expect(() => schema.parse(null)).toThrow(TypeError); + expect(() => schema.parse('not an object')).toThrow(TypeError); + expect(() => schema.parse(7)).toThrow(TypeError); +}); + +test('a field schema rejection propagates out of tristateObject', () => { + expect(() => + tristateObject({age: numberSchema}).parse({age: 'thirty'}), + ).toThrow(); +}); + +test("tristateObject preserves each field's element type through the mapped return (SERDE-16)", () => { + // `tristateObject`'s return is a mapped-plus-conditional type built behind an `as never`, so a + // runtime test cannot catch an inference regression here — only `expectTypeOf` can + // (docs/knowledge/harvested/testing.md:30). + const parsed = tristateObject({age: numberSchema, name: stringSchema}).parse( + {}, + ); + + expectTypeOf(parsed.age).toEqualTypeOf<Tristate<number>>(); + expectTypeOf(parsed.name).toEqualTypeOf<Tristate<string>>(); + // A key the shape never named is NOT reachable: the return type carries no index signature, so a + // misspelling is a compile error rather than a silent `unknown`. The pass-through keys still exist + // at runtime — a caller who wants them typed intersects at their own call site, where the DTO's + // real shape is known. + // @ts-expect-error — unnamed keys are absent from the mapped return type by design + const unreachable: unknown = parsed.somethingElse; + expect(unreachable).toBeUndefined(); +}); + +// --- SERDE-17: a missing key resolves to Absent, whatever it is NAMED -------------------------- +// +// `key in source` walks the prototype chain, and `JSON.parse` hands back objects rooted at +// `Object.prototype`. A field named after any of its eleven members therefore read as +// PRESENT-of-a-native-function when the wire had omitted it. `Object.hasOwn` is the fix; these +// cases pin it per name so a revert is loud. + +const PROTOTYPE_MEMBERS = Object.getOwnPropertyNames(Object.prototype).filter( + name => name !== '__proto__', +); + +test('a wire-omitted field named after an Object.prototype member is Absent, not Present (SERDE-17)', () => { + const shape = Object.fromEntries( + PROTOTYPE_MEMBERS.map(name => [name, identity]), + ); + // `JSON.parse`, not a literal: the prototype chain is the whole point of this case. + const parsed = tristateObject(shape).parse( + JSON.parse('{"unrelated":1}') as unknown, + ); + + for (const name of PROTOTYPE_MEMBERS) { + // `as Record<string, Tristate<unknown>>`: the shape is built dynamically, so the mapped return + // type cannot name these keys. + const field = (parsed as Record<string, Tristate<unknown>>)[name]; + expect(`${name}=${String(field?.kind)}`).toBe(`${name}=absent`); + } +}); + +test('a field named toString or constructor with no wire key resolves to Absent (SERDE-17)', () => { + const schema = tristateObject({toString: identity, constructor: identity}); + const parsed = schema.parse(JSON.parse('{"a":1}') as unknown); + + expect(parsed.toString.kind).toBe('absent'); + expect(parsed.constructor.kind).toBe('absent'); +}); + +test('a genuinely present key of the same name still decodes to Present', () => { + const schema = tristateObject({toString: identity, normal: identity}); + const parsed = schema.parse( + JSON.parse('{"toString":"mine","normal":1}') as unknown, + ); + + expect(parsed.toString.kind).toBe('present'); + expect(valueOrNull(parsed.toString)).toBe('mine'); + expect(valueOrNull(parsed.normal)).toBe(1); +}); + +// --- a shape field named __proto__ must not rewrite the RESULT's prototype --------------------- + +test('a __proto__ field in the shape yields Absent and leaves the result prototype intact', () => { + // Built with `defineProperty`: `{__proto__: x}` in a literal is the proto-setter syntax, so the + // key would never become an own property of the shape at all. + const shape: Record<string, Schema<unknown>> = {}; + Object.defineProperty(shape, '__proto__', { + value: identity, + enumerable: true, + writable: true, + configurable: true, + }); + + const parsed = tristateObject(shape).parse( + JSON.parse('{"a":1}') as unknown, + ) as Record<string, unknown>; + + expect(Object.getPrototypeOf(parsed)).toBe(Object.prototype); + const field = Object.getOwnPropertyDescriptor(parsed, '__proto__')?.value as + Tristate<unknown> | undefined; + expect(field?.kind).toBe('absent'); + // The sentinel's own members must not have leaked in through the chain. + expect((parsed as {kind?: unknown}).kind).toBeUndefined(); +}); + +test('a wire-level "__proto__" key is copied as data and pollutes nothing', () => { + const schema = tristateObject({name: identity}); + const parsed = schema.parse( + JSON.parse('{"__proto__":{"polluted":true},"name":"x"}') as unknown, + ) as Record<string, unknown>; + + expect(Object.getPrototypeOf(parsed)).toBe(Object.prototype); + expect(Object.getOwnPropertyNames(parsed)).toContain('__proto__'); + expect(({} as {polluted?: unknown}).polluted).toBeUndefined(); +}); + +// --- an array is not an object for this purpose (SERDE-16) ------------------------------------- + +test('tristateObject rejects an array rather than reshaping it into an index-keyed object', () => { + const schema = tristateObject({a: identity}); + + // An array is `typeof 'object'` and non-null, so a bare object check reshapes `[1,2,3]` into + // `{"0":1,"1":2,"2":3,"a":Absent}` — a shape mismatch laundered into a plausible-looking DTO. + expect(() => schema.parse([1, 2, 3])).toThrow(TypeError); + expect(() => schema.parse([])).toThrow(TypeError); +}); + +// --- SERDE-29: a shared schema must be unable to acquire state --------------------------------- + +test('both combinators return frozen schemas, like the bundle itself', () => { + expect(Object.isFrozen(tristate(identity))).toBe(true); + expect(Object.isFrozen(tristateObject({a: identity}))).toBe(true); +}); + +// SERDE-14 has three states. `present(null)` is a fourth, and the type system alone cannot keep it +// out: `present` takes `NonNullable<T>`, but `inner.parse`'s declared `T` is unconstrained, so the +// cast that satisfies the compiler is exactly where a normalizing schema slips through +// (audit #67 / #79). +const nullifying: Schema<unknown> = {parse: () => null}; +const erasing: Schema<unknown> = {parse: () => undefined}; + +test('an inner schema that normalizes a value to null is a decode failure (SERDE-14)', () => { + expect(() => tristate(nullifying).parse('a value')).toThrow( + DeserializationError, + ); + expect(() => tristate(nullifying).parse('a value')).toThrow( + /present Tristate cannot carry null/, + ); +}); + +test('an inner schema that normalizes a value to undefined is rejected the same way', () => { + expect(() => tristate(erasing).parse('a value')).toThrow( + DeserializationError, + ); +}); + +test('the wire null and missing-key paths still decode ahead of that check (SERDE-16)', () => { + // Neither reaches `inner.parse`, so a normalizing inner schema cannot turn a legitimate Null or + // Absent into a failure. + expect(tristate(nullifying).parse(null).kind).toBe('null'); + expect(tristate(nullifying).parse(MISSING).kind).toBe('absent'); +}); + +test('a nullifying field schema fails the whole tristateObject decode (SERDE-14)', () => { + expect(() => tristateObject({age: nullifying}).parse({age: 30})).toThrow( + DeserializationError, + ); +}); diff --git a/packages/codec-json/src/tristate-schema.ts b/packages/codec-json/src/tristate-schema.ts new file mode 100644 index 0000000..1e473b1 --- /dev/null +++ b/packages/codec-json/src/tristate-schema.ts @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/tristate-schema.ts +import { + absent, + DeserializationError, + nullValue, + present, + type Schema, + type Tristate, +} from '@dexpace/core'; + +/** + * The sentinel {@link tristateObject} feeds to a field's schema for "this key was not on the wire." + * + * `SERDE-17` is the awkward half of Tristate decoding: a `JSON.parse` reviver runs bottom-up per key + * and never fires for a key that is *absent*, so the raw JSON layer structurally cannot tell Absent + * from Null. The reference resolves this one layer up, in the codec's field-default machinery; this + * port resolves it one layer up too, in the schema combinator — {@link tristateObject} looks the key + * up on the parsed object and feeds this sentinel to the field's schema when it is missing. + * + * Not on the package's public barrel: no caller has to construct one, because {@link tristate} also + * accepts plain `undefined` for Absent. A plain `Symbol` rather than `Symbol.for` for the same + * reason — nothing crosses a package boundary on this identity, so the global registry buys nothing. + * + * @internal + */ +export const MISSING: unique symbol = Symbol('@dexpace/codec-json.missing'); + +/** + * Wrap a schema so it decodes the three PATCH states (SERDE-16). + * + * @param inner - the schema for the value a Present carries. + * @returns a schema producing `Tristate<T>`: a missing-key sentinel or `undefined` yields Absent, a wire + * `null` yields Null, anything else runs through `inner` and yields Present. + * @throws DeserializationError when `inner` resolves a present value to `null` or `undefined`. SERDE-14 + * has three states; a Present carrying nothing would be a fourth, and the wire said the key was there. + * @public + */ +export function tristate<T>(inner: Schema<T>): Schema<Tristate<T>> { + // Frozen for the same reason `jsonSerde()`'s bundle is (SERDE-29): a schema is shared across + // every concurrent decode that names it, so it must be stateless AND unable to acquire state. + return Object.freeze({ + parse(input: unknown): Tristate<T> { + if (input === MISSING || input === undefined) return absent(); + if (input === null) return nullValue(); + const value = inner.parse(input); + // The type system alone cannot hold SERDE-14's third state to non-null values: `present` takes + // `NonNullable<T>`, but `T` here is whatever the caller's schema declares, so a schema that + // NORMALIZES to null — a Zod `.transform()`, a "" → null cleanup — type-checks and produces + // `{kind: 'present', value: null}`, the fourth state the union exists to forbid. Checked at + // run time, and reported as a decode failure rather than an assertion: the input came off the + // wire, and the pairing of that input with that schema is what has no Tristate (audit #67 / + // #79). `undefined` is rejected with it — `NonNullable<T>` excludes both, and a Present of + // `undefined` is Absent wearing the wrong label. + if (value === null || value === undefined) { + throw new DeserializationError( + 'a present Tristate cannot carry null or undefined; the inner schema resolved a wire value to one (SERDE-14)', + ); + } + // `as NonNullable<T>`: the nullish cases have all returned or thrown above, a fact the + // compiler cannot derive through `inner.parse`'s unconstrained `T`. + return present<T>(value); + }, + }); +} + +/** + * Build an object schema whose named fields decode as Tristate, feeding an internal sentinel for keys the + * wire omitted (SERDE-17). + * + * Keys not named in `shape` pass through untouched **at runtime**, so this composes with a caller's + * own schema for the rest of the DTO rather than replacing it. The returned *type* names only the + * `shape` keys: an index signature would make every property access legal and typed `unknown`, + * silently accepting a misspelled field name. A caller who needs the pass-through keys typed + * intersects at their own call site, where the DTO's real shape is known. + * + * The input object is never mutated — the named fields are written onto a shallow copy. + * + * @param shape - a schema per Tristate-decoded field, keyed by wire name. + * @returns a schema producing an object whose named keys are `Tristate`-wrapped. + * @throws TypeError when the value being parsed is not a non-null object, or is an array. + * @throws DeserializationError when a field's own schema resolves a present wire value to `null` or + * `undefined`, which is {@link tristate}'s check applied per field (SERDE-14). + * @public + */ +export function tristateObject<S extends Record<string, Schema<unknown>>>( + shape: S, +): Schema<{ + [K in keyof S]: Tristate<S[K] extends Schema<infer T> ? T : never>; +}> { + const fields = Object.entries(shape).map( + ([key, inner]) => [key, tristate(inner)] as const, + ); + // Frozen alongside `tristate()`'s result, and for the same reason (SERDE-29). + return Object.freeze({ + parse(input: unknown) { + // An array is `typeof 'object'` and non-null, so a bare object check let one through and + // silently reshaped `[1,2,3]` into `{"0":1,"1":2,"2":3, ...}`. A JSON array arriving where a + // DTO was expected is a shape mismatch, which SERDE-16 wants rejected rather than reshaped. + if (typeof input !== 'object' || input === null || Array.isArray(input)) { + throw new TypeError('tristateObject expects a non-array object'); + } + // `as Record<string, unknown>`: the guard above established it is a non-null object; + // TypeScript narrows to `object`, which is not indexable. + const source = input as Record<string, unknown>; + const out: Record<string, unknown> = {...source}; + for (const [key, schema] of fields) { + // `Object.hasOwn`, never `key in source`: `in` walks the prototype chain, and `JSON.parse` + // hands back objects rooted at `Object.prototype`. A field named after any of its eleven + // members (`toString`, `constructor`, `valueOf`, `hasOwnProperty`, ...) then read as + // PRESENT-of-a-native-function when the wire had omitted it, which SERDE-17 requires + // resolve to Absent. + const raw = Object.hasOwn(source, key) ? source[key] : MISSING; + // `defineProperty`, never `out[key] = ...`: assignment to the key `__proto__` invokes + // `Object.prototype`'s setter, which would replace the RESULT object's prototype with a + // Tristate sentinel instead of writing a field. No global pollution either way — the + // spread above already copies a wire-level `__proto__` as a plain own property — but the + // returned DTO silently gained `kind`/`value` through the chain and lost the field. + Object.defineProperty(out, key, { + value: schema.parse(raw), + writable: true, + enumerable: true, + configurable: true, + }); + } + // `as never`: the declared return is a mapped-plus-conditional type the compiler cannot see + // this loop building key by key. The type-level test is what actually checks it — no runtime + // test can. + return out as never; + }, + }); +} diff --git a/packages/codec-json/tsconfig.build.json b/packages/codec-json/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/codec-json/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/codec-json/tsconfig.json b/packages/codec-json/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/codec-json/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/core/README.md b/packages/core/README.md new file mode 100644 index 0000000..bd60350 --- /dev/null +++ b/packages/core/README.md @@ -0,0 +1,173 @@ +# @dexpace/core + +The transport-agnostic HTTP core of the dexpace SDK: an immutable request/response domain model, a +staged policy pipeline, and the seams everything else plugs into. **Zero runtime dependencies**, ESM +only, Node ≥ 20.3. + +It is deliberately not an HTTP client — it never opens a socket. Pair it with a transport. + +```sh +bun add @dexpace/core @dexpace/transport-fetch +``` + +```typescript +import {Request, standardResilience} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const client = standardResilience(fetchTransport()); + +const response = await client.send( + Request.newBuilder().url('https://api.example.com/v1/things').build(), +); +try { + console.log(response.status.code, await response.text()); +} finally { + await response.close(); // the caller owns the body, always (BODY-15) +} +``` + +That is the whole zero-to-one path. `standardResilience()` returns a `Runtime` with redirect, retry, +auth and logging already installed in the order `AUTH-27` requires — redirect wraps retry wraps auth +— so a retry re-resolves credentials and a redirect hop re-stamps them. + +## Which package do I install? + +`@dexpace/core` alone gets you the models and the pipeline. Everything that touches a platform API +lives in a sibling package, because core's zero-dependency and zero-`node:`-import invariants are +hard (`SEAM-1`, gate-enforced by `bun run verify:seam-1`). + +| You need | Install | +|---|---| +| To send a request, no extra dependencies | `@dexpace/transport-fetch` | +| Connection pools, proxies, real `close()` semantics | `@dexpace/transport-undici` | +| A JSON wire codec behind the `Serde` seam | `@dexpace/codec-json` | +| A file-backed request body (`node:fs`) | `@dexpace/body-file` | +| Logs routed to `pino` or `debug` | `@dexpace/logging-pino`, `@dexpace/logging-debug` | +| RxJS `Observable` views of SSE and pagination | `@dexpace/rx` | + +Every one of them declares `@dexpace/core` as a **peer**, never a dependency: two copies of core in +one install would defeat the branded symbols and `instanceof` checks the seams rely on. + +## The five things worth knowing before reading source + +**1. Models are frozen and builder-built.** There is no public constructor on `Request`, `Response`, +`Headers`, `QueryParams`, `RequestOptions` or `RequestConditions` — `newBuilder()` is the only way +in, so validation cannot be routed around (`HTTP-2`). `newBuilder()` on an *instance* returns a +pre-filled builder that deep-copies every collection, so deriving never aliases the source +(`HTTP-3`). + +```typescript +import {Request} from '@dexpace/core'; + +const request = Request.newBuilder().url('https://api.example.com/v1/things').build(); + +const authorized = request + .newBuilder() + .headers(request.headers.newBuilder().set('Authorization', 'Bearer …').build()) + .build(); +``` + +**2. `Status` is total.** `Status.of(599)` succeeds, reports `isServerError`, and has `name === +undefined` and `isRecognized === false`; `Status.recognized(599)` returns `undefined` so a caller can +tell a vendor code from a registered one. An unrecognized code is never an error — a server is free +to invent one. + +**3. A body is a producer, not a buffer.** `byteArrayBody`, `stringBody`, `formUrlEncodedBody`, +`multipartBody`, `streamBody` and `serdeBody` are the factories; the classes are exported as types +only. `body.replayable` decides whether a retry can re-send it, and `materialize(body)` buys +replayability by buffering. `streamBody` is single-use by construction. + +**4. The caller owns the response body.** `response.close()` is yours to call, on every path, +including the ones where an error is propagating. Nothing in the pipeline closes a response it hands +you. + +**5. Errors are a two-level tree.** `DexpaceError` at the root, then leaves — the ten HTTP +domain-model errors (`RequiredFieldError`, `HeaderValidationError`, and the rest, grouped by the +`isDomainModelError` guard rather than by a class tier), `IoError`, `HttpStatusError`, +`AuthResolutionError`, `PaginationError`, `SerializationError`/`DeserializationError`, +`SseStreamError`, `CancellationError`. Exactly one sanctioned third level: `TransportFailureError +extends IoError`, so `catch (e) { if (e instanceof IoError) }` still catches a transport failure +(`docs/deviations.md` item 17). Wrap-and-rethrow always passes `{cause}`. + +## Building a pipeline yourself + +`standardResilience()` is a preset over `PipelineBuilder`. When it is the wrong shape, layer onto it: + +```typescript +import {PipelineBuilder, standardResilience, type Step} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const stamp: Step = async (request, ctx) => + ctx.next( + request + .newBuilder() + .headers(request.headers.newBuilder().set('X-Client-Phase', ctx.context.kind).build()) + .build(), + ); + +const runtime = PipelineBuilder.seedFrom( + standardResilience(fetchTransport(), {retry: {settings: {maxAttempts: 5}}}), + 'flatten', +) + .append({type: Symbol('x-client-phase'), stage: 'POST_SERDE', fn: stamp}) + .build(); +``` + +Steps run in `STAGE_ORDER`, sixteen stages from `PRE_REDIRECT` to `SEND`. The five **pillar** stages +— `REDIRECT`, `RETRY`, `AUTH`, `LOGGING`, `SERDE` (`PILLAR_STAGES`) — admit exactly one step each and +raise on a second; the surrounding `PRE_`/`POST_` stages stack. +`seedFrom(runtime, 'flatten' | 'nest')` is how the preset composes with a customized builder, rather +than the preset growing a "skip occupied slots" branch. + +**Install redirects with `withRedirect(builder)`, not with bare `redirectStep()`.** The redirect +pillar marks a cross-origin hop with an internal header, and a second `POST_AUTH` step strips it +before dispatch; `withRedirect` seats both, and `stripCrossOriginMarkerStep()` is that guard on its +own. Both became public on 2026-09-02 (`docs/work/mvp/2026-09-04-open-items-dissolution.md` U7) — before that only +`standardResilience()` and `seedFrom` could produce a safe redirect pipeline. A hand-built pipeline +that installs `redirectStep()` and neither guard forwards the marker to the wire. + +`Runtime` implements `Transport`, so a pipeline is substitutable for the transport it wraps. +`Runtime.close()` is a documented no-op: the pipeline never owns the transport it was given +(`PIPE-27`). + +## Beyond request/response + +- **Serde.** `Serde`/`Serializer`/`Deserializer`/`Schema` are the seam; `decodeResponse` and + `decodeSuccessResponse` are the response handlers; `Tristate` models PATCH's + absent/null/present distinction so `{}` and `{"x": null}` stop being the same wire message. Core + ships no codec — `@dexpace/codec-json` is the reference one. +- **Server-Sent Events.** `sseStreamFrom(response)` yields an `SseStream` of `SseEvent`; + `typedSseStream(stream, mapper)` decodes into your own models. Single-pass over a response body + this stream does not own, and no reconnect path in core (`SSE-37`/`SSE-38`, gate-enforced). +- **Pagination.** `Paginator` iterates `items()` or `pages()`; `cursorStrategy`, + `pageNumberStrategy` and `linkHeaderStrategy` cover the three shipped shapes, and + `PaginationStrategy` is the seam for the rest. A `Page` is closed before its items are yielded + (`PAGE-11`). +- **Configuration.** `Configuration` is a layered lookup — explicit override, then the environment + source under the exact key, then the property source under a normalized (lower-cased, dotted) key, + then your fallback — built through `ConfigurationBuilder`. Both sources are caller-supplied seams + (`CFG-11`), so a test substitutes them without touching the real environment. + `getGlobalConfiguration()`/`setGlobalConfiguration()` hold the process-wide slot. +- **Observability.** `Logger` is a facade with `NOOP_LOGGER` as the default; `createLogger(sink)` + adapts anything. `Tracer`/`Span`/`Meter` are duck-typed, so an OpenTelemetry object satisfies them + with no adapter and no registration. + +## Where the details are + +This README gets you running. It is deliberately not the API reference — that is generated and +gate-verified, and a hand-written third copy would drift: + +- **Every exported symbol, with its signature:** + [`etc/core.api.md`](https://github.com/dexpace/nodejs-sdk/blob/main/packages/core/etc/core.api.md), regenerated by `bun run api:local` and + verified in CI by `bun run api`. +- **What each symbol means, `@throws` included:** the TSDoc, which ships in the emitted `.d.ts` and + shows up on hover. +- **How the packages compose, with worked cross-package examples:** + [`docs/sdk-documentation/`](https://github.com/dexpace/nodejs-sdk/blob/main/docs/sdk-documentation). +- **What is normative:** [`docs/product-spec/`](https://github.com/dexpace/nodejs-sdk/blob/main/docs/product-spec). Every `HTTP-N`, `SEAM-N`, + `RETRY-N` identifier in this README and in the source is an entry there. + +Every link above is absolute on purpose. `package.json` ships `files: ["dist"]`, so none of these +paths exist in the published tarball, and no manifest carries a `repository` field for npm's renderer +to rewrite a relative link with — so on npmjs.com a relative one renders broken. That is `U8`'s +failure class, and the first place to check when adding a link here. diff --git a/packages/core/api-extractor.json b/packages/core/api-extractor.json index 75aa61d..455423c 100644 --- a/packages/core/api-extractor.json +++ b/packages/core/api-extractor.json @@ -10,5 +10,13 @@ }, "dtsRollup": { "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } } } diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index 330de32..cfcf87d 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -4,31 +4,387 @@ ```ts +// @public +export function absent(): Tristate<never>; + +// @public +export function activateSpan(span: Span): Scope; + +// @public +export function activateSpanForCorrelation(span: Span): Scope; + +// @public +export class AllocationLimitError extends DexpaceError { + constructor(requested: number, limit: number, options?: ErrorOptions); + readonly limit: number; + readonly requested: number; +} + +// @public +export class AnchorNotFoundError extends DexpaceError { + constructor(anchorType: symbol, operation: string, options?: ErrorOptions); + readonly anchorType: symbol; + readonly operation: string; +} + +// @public +export class ApiKeyCredential { + [INSPECT](): string; + constructor(key: string); + toString(): string; +} + +// @public +export interface ApiKeyCredentialConfig { + readonly credential: ApiKeyCredential | NameKeyCredential; + readonly headerName?: string | undefined; + readonly prefix?: string | undefined; +} + +// @public +export interface AuthCredentialSet { + readonly apiKey?: ApiKeyCredentialConfig | undefined; + readonly basic?: BasicCredential | undefined; + readonly bearer?: BearerCredential | undefined; + readonly digest?: DigestCredential | undefined; +} + +// @public +export interface AuthDescriptor { + readonly allowsAnonymous: boolean; + readonly requirements: readonly AuthRequirement[]; +} + +// @public +export interface AuthRequirement { + readonly params: ReadonlyMap<string, string>; + readonly scheme: AuthScheme; + readonly scopes: readonly string[]; +} + +// @public +export function authRequirementsEqual(a: AuthRequirement, b: AuthRequirement): boolean; + +// @public +export class AuthResolutionError extends DexpaceError { + constructor(message: string, requiredSchemes?: readonly string[], availableSchemes?: readonly string[]); + readonly availableSchemes: readonly string[] | undefined; + readonly requiredSchemes: readonly string[] | undefined; + static unsatisfiable(requiredSchemes: readonly string[], availableSchemes: readonly string[]): AuthResolutionError; +} + +// @public +export type AuthScheme = 'OAUTH2' | 'API_KEY' | 'BASIC' | 'DIGEST' | 'NO_AUTH'; + +// @public +export function authStep(settings: AuthStepSettings): StepDescriptor; + +// @public +export interface AuthStepSettings { + readonly bearerMarginMs?: number | undefined; + readonly challengeHook?: ChallengeHook | undefined; + readonly clock?: Pick<Clock, 'now'> | undefined; + readonly credentials: AuthCredentialSet; + readonly tiers: AuthTiers; +} + +// @public +export interface AuthTiers { + readonly client?: AuthDescriptor | undefined; + readonly operation?: AuthDescriptor | undefined; + readonly perCall?: AuthDescriptor | undefined; +} + +// @public +export interface BackoffSettings { + readonly fixedDelayMs?: number | undefined; + readonly initialDelayMs: number; + readonly jitter: number; + readonly maxDelayMs: number; + readonly multiplier: number; +} + +// @public +export class BasicCredential { + [INSPECT](): string; + constructor(username: string, password: string); + toString(): string; + readonly username: string; +} + +// @public +export interface BearerCredential { + readonly marginMs?: number | undefined; + readonly provider: TokenProvider; +} + +// @public +export class BearerToken { + [INSPECT](): string; + readonly expiresAt: number | undefined; + get token(): string; + toString(): string; +} + +// @public +export function bearerTokensEqual(a: BearerToken, b: BearerToken): boolean; + +// @public +interface Body_2 { + readonly contentLength: number; + readonly kind: 'byte-array' | 'string' | 'stream' | 'form-urlencoded' | 'multipart' | 'file'; + readonly mediaType: string | undefined; + readonly replayable: boolean; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} +export { Body_2 as Body } + // @public export interface Builder<T> { build(): T; } +// @public +export interface BuildInfo { + readonly identityTokens: readonly string[]; + readonly runtimeIdentity: string; + readonly sdkVersion: string; +} + // @public export function buildRequest(baseUrl: string | URL, operation: OperationDescriptor): Request_2; +// @public +export class ByteArrayBody implements Body_2 { + constructor(bytes: Uint8Array, mediaType?: string); + readonly contentLength: number; + readonly kind: "byte-array"; + readonly mediaType: string | undefined; + readonly replayable = true; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +// @public +export function byteArrayBody(bytes: Uint8Array, mediaType?: string): ByteArrayBody; + // @public export class CancellationError extends DexpaceError { constructor(message: string, options?: ErrorOptions); } +// @public +export const CFG_KEY_HTTP_PROXY = "HTTP_PROXY"; + +// @public +export const CFG_KEY_HTTPS_PROXY = "HTTPS_PROXY"; + +// @public +export const CFG_KEY_LOG_LEVEL = "DEXPACE_LOG_LEVEL"; + +// @public +export const CFG_KEY_MAX_RETRY_ATTEMPTS = "DEXPACE_MAX_RETRY_ATTEMPTS"; + +// @public +export const CFG_KEY_NO_PROXY = "NO_PROXY"; + +// @public +export type ChallengeHook = (response: Response_2, request: Request_2, options?: { + readonly signal?: AbortSignal | undefined; +}) => Promise<Request_2 | undefined>; + +// @public +export interface ClientIdentitySettings { + readonly headerName?: string | undefined; + readonly mode?: 'append' | 'replace' | undefined; + readonly tokens?: readonly string[] | undefined; +} + +// @public +export function clientIdentityStep(settings?: ClientIdentitySettings): StepDescriptor; + +// @public +export interface Clock { + monotonic(): number; + now(): number; + sleep(durationMs: number, signal?: AbortSignal): Promise<void>; +} + +// @public +export class ClosedResourceError extends DexpaceError { + constructor(resource: string, options?: ErrorOptions); + readonly resource: string; +} + // @public export function composeSignal(userSignal?: AbortSignal, timeoutMs?: number): AbortSignal | undefined; +// @public +export interface Configuration { + derive(mutate: (builder: ConfigurationBuilder) => void): Configuration; + getBoolean(key: string, fallback: boolean): boolean; + getDuration(key: string, fallbackMs: number): number; + getInt(key: string, fallback: number): number; + getRawProperty(key: string, fallback?: string): string | undefined; + getString(key: string, fallback?: string): string | undefined; +} + +// @public +export class ConfigurationBuilder { + build(): Configuration; + put(key: string, value: string): this; + remove(key: string): this; + withEnvSource(source: SourceFn): this; + withPropertySource(source: SourceFn): this; +} + +// @public +export class ConsumedBodyError extends DexpaceError { + constructor(bodyKind: string, options?: ErrorOptions); + readonly bodyKind: string; +} + +// @public +export interface Counter { + add(delta: number, attributes?: Readonly<Record<string, unknown>>): void; +} + +// @public +export function createAuthDescriptor(requirements: readonly AuthRequirement[]): AuthDescriptor; + +// @public +export function createAuthRequirement(scheme: AuthScheme, scopes?: readonly string[], params?: ReadonlyMap<string, string>): AuthRequirement; + +// @public +export function createBearerToken(token: string, expiresAt?: number): BearerToken; + +// @public +export function createInstrumentationBundle(tracerFactory?: (operationName: string) => Tracer): InstrumentationBundle; + +// @public +export function createLogger(sink: (level: LogLevel, fields: ReadonlyMap<string, unknown>) => void, options?: CreateLoggerOptions): Logger; + +// @public +export interface CreateLoggerOptions { + readonly diagnosticAllowList?: readonly string[] | null | undefined; + // (undocumented) + readonly globalFields?: Readonly<Record<string, unknown>> | undefined; + readonly isLevelEnabled?: ((level: LogLevel) => boolean) | undefined; +} + +// @public +export function createProxyOptions(init: ProxyOptionsInit): ProxyOptions; + +// @public +export class CrossStageEditError extends DexpaceError { + constructor(anchorStage: Stage, incomingStage: Stage, options?: ErrorOptions); + readonly anchorStage: Stage; + readonly incomingStage: Stage; +} + +// @public +export class CursorAlreadyAdvancedError extends DexpaceError { + constructor(stage: Stage, options?: ErrorOptions); + readonly stage: Stage; +} + +// @public +export function cursorStrategy<T>(init: { + extract: (response: Response_2) => Promise<{ + items: readonly T[]; + cursor?: string | null | undefined; + }>; + parameterName?: string | undefined; +}): PaginationStrategy<T>; + +// @public +export function decodeResponse<T>(response: Response_2, deserializer: Deserializer, target: DecodeTarget<T>): Promise<T>; + +// @public +export function decodeSuccessResponse<T>(response: Response_2, deserializer: Deserializer, target: DecodeTarget<T>): Promise<T>; + +// @public +export interface DecodeTarget<T> { + readonly admitsNull?: boolean | undefined; + readonly schema: Schema<T>; + readonly typeName?: string | undefined; +} + +// @public +export const defaultClock: Clock; + +// @public +export function defaultConfiguration(): Configuration; + +// @public +export class DeserializationError extends DexpaceError { + constructor(message: string, options?: DeserializationErrorOptions); + readonly etag: string | null; + readonly location: string | null; + readonly status: number | undefined; +} + +// @public +export interface DeserializationErrorOptions extends SerdeErrorOptions { + readonly etag?: string | null | undefined; + readonly location?: string | null | undefined; + readonly status?: number | undefined; +} + +// @public +export interface Deserializer { + deserialize<T>(data: Uint8Array, target: DecodeTarget<T>): T; + deserializeFrom<T>(source: ReadableStream<Uint8Array>, target: DecodeTarget<T>, options?: { + readonly signal?: AbortSignal | undefined; + }): Promise<T>; +} + // @public export class DexpaceError extends Error { constructor(message: string, options?: ErrorOptions); } // @public -export class DomainModelError extends DexpaceError { +export type DigestAlgorithm = 'MD5' | 'MD5-sess' | 'SHA-256' | 'SHA-256-sess'; + +// @public +export class DigestCredential { + [INSPECT](): string; + constructor(username: string, password: string, algorithmPreference?: readonly DigestAlgorithm[]); + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + toString(): string; + readonly username: string; } +// @public +export interface DispatchConfig { + readonly options?: RequestOptions | undefined; + readonly requestChain: RequestRecoveryChain; + readonly responseChain: ResponseRecoveryChain; + readonly signal?: AbortSignal | undefined; + readonly transport: Transport; +} + +// @public +export interface DispatchContext { + readonly instrumentation: InstrumentationBundle; + readonly key: symbol; + readonly kind: 'dispatch'; +} + +// @public +export function dispatchWithRecovery(request: Request_2, config: DispatchConfig): Promise<Response_2>; + +// @public +export type DroppedHeaderPolicy = 'mark' | 'omit'; + +// @public +class EndOfStreamError_2 extends DexpaceError { + constructor(delivered: number, requested: number, options?: ErrorOptions); + readonly delivered: number; + readonly requested: number; +} +export { EndOfStreamError_2 as EndOfStreamError } + // @public export class ETag { static readonly ANY: ETag; @@ -40,9 +396,101 @@ export class ETag { } // @public -export class EtagParseError extends DomainModelError { +export class EtagParseError extends DexpaceError { +} + +// @public +export interface ExchangeContext { + readonly instrumentation: InstrumentationBundle; + readonly key: symbol; + readonly kind: 'exchange'; + readonly operationName: string | undefined; + readonly request: Request_2; + readonly response: Response_2; +} + +// @public +export type ExecutionContext = DispatchContext | RequestContext | ExchangeContext; + +// @public +export function failure<T>(error: unknown): Outcome<T>; + +// @public +export interface FetcherPage<T> { + readonly continuationToken?: string | undefined; + readonly nextLink?: string | undefined; + readonly page: Page<T>; +} + +// @public +export interface FetcherPaginationInit<T> { + first: (options: PagingOptions) => Promise<FetcherPage<T> | undefined>; + maxPages?: number | undefined; + next: (key: string, options: PagingOptions) => Promise<FetcherPage<T> | undefined>; } +// @public +export interface FileBodyDescriptor extends Body_2 { + // (undocumented) + readonly count: number; + // (undocumented) + readonly kind: 'file'; + // (undocumented) + readonly path: string; + // (undocumented) + readonly start: number; +} + +// @public +export function fold<T, R>(outcome: Outcome<T>, onSuccess: (value: T) => R, onFailure: (error: unknown) => R): R; + +// @public +export function foldTristate<T, R>(tristate: Tristate<T>, branches: TristateBranches<T, R>): R; + +// @public +export function formatHttpDate(epochMs: number): string; + +// @public +export function formatProxyOptions(options: ProxyOptions): string; + +// @public +export class FormBodyValidationError extends DexpaceError { + constructor(field: string, value: unknown, options?: ErrorOptions); + readonly field: string; +} + +// @public +export class FormUrlEncodedBody implements Body_2 { + constructor(input: FormUrlEncodedInput); + readonly contentLength: number; + readonly kind: "form-urlencoded"; + readonly mediaType = "application/x-www-form-urlencoded"; + readonly params: QueryParams; + readonly replayable = true; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +// @public +export function formUrlEncodedBody(input: FormUrlEncodedInput): FormUrlEncodedBody; + +// @public +export type FormUrlEncodedInput = QueryParams | ReadonlyMap<string, FormUrlEncodedValue | readonly FormUrlEncodedValue[]> | Record<string, FormUrlEncodedValue | readonly FormUrlEncodedValue[]> | readonly (readonly [string, FormUrlEncodedValue])[]; + +// @public +export type FormUrlEncodedValue = string | number | boolean | bigint | null; + +// @public +export function getActiveSpan(): Span; + +// @public +export function getBuildInfo(): BuildInfo; + +// @public +export function getGlobalConfiguration(): Configuration; + +// @public +export function getGlobalLogger(): Logger; + // @public export class HeaderName { equals(other: HeaderName): boolean; @@ -62,80 +510,441 @@ class Headers_2 { static newBuilder(): HeadersBuilder; newBuilder(): HeadersBuilder; } -export { Headers_2 as Headers } +export { Headers_2 as Headers } + +// @public +export class HeadersBuilder implements Builder<Headers_2> { + add(name: string | HeaderName, value: string): this; + addInbound(name: string | HeaderName, value: string): this; + build(): Headers_2; + set(name: string | HeaderName, value: string | null): this; + setInbound(name: string | HeaderName, value: string | null): this; +} + +// @public +export class HeaderValidationError extends DexpaceError { + constructor(kind: 'name' | 'value', offendingName: string, _offendingValue: string | undefined); + readonly escapedName: string; + readonly kind: 'name' | 'value'; +} + +// @public +export interface Histogram { + record(value: number, attributes?: Readonly<Record<string, unknown>>): void; +} + +// @public +export class HttpRange { + static bounded(start: number, length: number): HttpRange; + get kind(): RangeKind; + get length(): number | undefined; + static open(start: number): HttpRange; + static parse(raw: string): HttpRange; + get raw(): string; + get start(): number | undefined; + static suffix(suffixLength: number): HttpRange; + get suffixLength(): number | undefined; +} + +// @public +export class HttpRangeValidationError extends DexpaceError { +} + +// @public +export class HttpStatusError extends DexpaceError { + constructor(status: number, bodyBytes: Uint8Array | undefined, mediaType: string | undefined, options?: ErrorOptions); + body(): Body_2 | undefined; + preview(charset?: string): string | null; + readonly status: number; +} + +// @public +export class HttpStatusValidationError extends DexpaceError { + constructor(status: number, options?: ErrorOptions); + readonly status: number; +} + +// @public +export interface IdempotencyKeyOptions { + readonly generate: () => string; + readonly headerName?: string | undefined; + readonly methods?: ReadonlySet<Method> | undefined; + readonly respectExisting?: boolean | undefined; +} + +// @public +export function idempotencyKeyStep(options: IdempotencyKeyOptions): RequestStep; + +// @public +export interface InstrumentationBundle { + readonly activeSpan: unknown; + readonly isRemote: boolean; + readonly isValid: boolean; + readonly spanId: string; + readonly traceFlags: number; + readonly traceId: string; + readonly traceIdEncoding: string; + readonly tracerFactory: (operationName: string) => unknown; + readonly traceState: string; +} + +// @public +export class IoError extends DexpaceError { + constructor(message: string, options?: ErrorOptions); +} + +// @public +export function isAbsent<T>(tristate: Tristate<T>): tristate is { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'absent'; +}; + +// @public +export function isBodyError(error: unknown): error is ConsumedBodyError | MultipartBoundaryError | FormBodyValidationError; + +// @public +export function isDomainModelError(error: unknown): error is RequiredFieldError | HeaderValidationError | MediaTypeParseError | ProtocolParseError | UrlConstructionError | RequestOptionsValidationError | EtagParseError | HttpRangeValidationError | RequestConditionsValidationError | RequestBodyNotAllowedError; + +// @public +export function isIoError(error: unknown): error is IoError | EndOfStreamError_2 | SourceContractViolationError | ClosedResourceError | AllocationLimitError; + +// @public +export function isNull<T>(tristate: Tristate<T>): tristate is { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'null'; +}; + +// @public +export function isPresent<T>(tristate: Tristate<T>): tristate is { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'present'; + readonly value: T; +}; + +// @public +export function isRetryableStatus(code: number): boolean; + +// @public +export function isSerdeError(e: unknown): e is SerializationError | DeserializationError; + +// @public +export function isSseEventEmpty(event: SseEvent): boolean; + +// @public +export function isTimeoutSignal(signal: AbortSignal): boolean; + +// @public +export function isTristate(value: unknown): value is Tristate<unknown>; + +// @public +export function linkHeaderStrategy<T>(init: { + extract: (response: Response_2) => Promise<readonly T[]>; + headerName?: string | undefined; +}): PaginationStrategy<T>; + +// @public +export interface LogEvent { + cause(error: unknown): this; + emit(): void; + event(name: string): this; + field(key: string, value: unknown): this; +} + +// @public +export interface Logger { + atLevel(level: LogLevel): LogEvent; + withContext(fields: Readonly<Record<string, unknown>>): Logger; +} + +// @public +export const LOGGING_STEP_TYPE: unique symbol; + +// @public +export type LoggingGranularity = 'none' | 'headers' | 'body'; + +// @public +export function loggingStep(settings?: LoggingStepSettings): StepDescriptor; + +// @public +export interface LoggingStepSettings { + readonly clock?: Clock | undefined; + readonly configKey?: string | undefined; + readonly droppedHeaderPolicy?: DroppedHeaderPolicy | undefined; + readonly granularity?: LoggingGranularity | undefined; + readonly logger?: Logger | undefined; + readonly meter?: Meter | undefined; + readonly previewSizeBytes?: number | undefined; + readonly severity?: LogLevel | undefined; + readonly tracerFactory?: (() => Tracer) | undefined; +} + +// @public +export type LogLevel = 'error' | 'warning' | 'info' | 'verbose'; + +// @public +export function makeSseEvent(fields: SseEventFields): SseEvent; + +// @public +export const MAPPER_DONE: MapperOutcome<never>; + +// @public +export const MAPPER_SKIP: MapperOutcome<never>; + +// @public +export type MapperOutcome<T> = { + readonly kind: 'value'; + readonly value: T; +} | { + readonly kind: 'skip'; +} | { + readonly kind: 'done'; +}; + +// @public +export function mapperValue<T>(value: T): MapperOutcome<T>; + +// @public +export function materialize(body: Body_2): Promise<Body_2>; + +// @public +export class MediaType { + get charset(): string | undefined; + equals(other: MediaType): boolean; + matches(pattern: MediaType): boolean; + static of(type: string, subtype: string, parameters?: ReadonlyMap<string, string>): MediaType; + parameter(key: string): string | undefined; + static parse(raw: string): MediaType; + render(): string; + get subtype(): string; + get type(): string; +} + +// @public +export class MediaTypeParseError extends DexpaceError { +} + +// @public +export interface Meter { + // (undocumented) + createCounter(name: string, options?: { + readonly unit?: string; + readonly description?: string; + }): Counter; + // (undocumented) + createHistogram(name: string, options?: { + readonly unit?: string; + readonly description?: string; + }): Histogram; +} + +// @public +export type Method = 'GET' | 'HEAD' | 'POST' | 'PUT' | 'DELETE' | 'CONNECT' | 'OPTIONS' | 'TRACE' | 'PATCH'; + +// @public +export class MultipartBody implements Body_2 { + constructor(parts: readonly MultipartPart[], boundary?: string); + readonly contentLength: number; + readonly kind: "multipart"; + readonly mediaType: string; + static newBuilder(): MultipartBodyBuilder; + newBuilder(): MultipartBodyBuilder; + readonly replayable: boolean; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +// @public +export function multipartBody(parts: readonly MultipartPart[], boundary?: string): MultipartBody; + +// @public +export class MultipartBodyBuilder implements Builder<MultipartBody> { + addPart(part: MultipartPart): this; + boundary(boundary: string | undefined): this; + build(): MultipartBody; + parts(parts: readonly MultipartPart[]): this; +} + +// @public +export class MultipartBoundaryError extends DexpaceError { + constructor(boundary: string, options?: ErrorOptions); + readonly boundary: string; +} + +// @public +export interface MultipartPart { + readonly body: Body_2; + readonly filename?: string | undefined; + readonly name: string; +} + +// @public +export class NameKeyCredential { + [INSPECT](): string; + constructor(name: string, key: string); + readonly name: string; + toString(): string; +} + +// @public +export type Next = (request?: Request_2) => Promise<Response_2>; + +// @public +export class NonReplayableBodyError extends DexpaceError { + constructor(targetUrl: string, options?: ErrorOptions); + readonly targetUrl: string; +} + +// @public +export const NOOP_LOGGER: Logger; + +// @public +export const NOOP_METER: Meter; + +// @public +export const NOOP_SPAN: Span; + +// @public +export const NOOP_TRACER: Tracer; + +// @public +export function nullValue(): Tristate<never>; + +// @public +export function ofNullable<T>(value: T | null | undefined): Tristate<T>; + +// @public +export class OperationAssemblyError extends DexpaceError { + constructor(message: string, parameterName: string); + readonly parameterName: string; +} + +// @public +export interface OperationDescriptor { + readonly body?: Body_2 | undefined; + readonly headers?: Headers_2 | undefined; + readonly method: Method; + readonly pathParams?: Readonly<Record<string, string>> | undefined; + readonly pathTemplate: string; + readonly query?: QueryParams | undefined; +} + +// @public +export type Outcome<T> = { + readonly kind: 'success'; + readonly value: T; +} | { + readonly kind: 'failure'; + readonly error: unknown; +}; + +// @public +export class Page<T> { + constructor(response: Response_2, items: readonly T[]); + close(): Promise<void>; + readonly headers: Headers_2; + readonly items: readonly T[]; + readonly request: Request_2; + readonly status: Status; +} + +// @public +export interface PageInfo<T> { + readonly items: readonly T[]; + readonly nextRequest: Request_2 | undefined; +} + +// @public +export function pageInfo<T>(items: readonly T[], nextRequest?: Request_2): PageInfo<T>; + +// @public +export function pageNumberStrategy<T>(init: { + extract: (response: Response_2) => Promise<readonly T[]>; + parameterName?: string | undefined; + startPage?: number | undefined; +}): PaginationStrategy<T>; + +// @public +export function paginateWithFetchers<T>(init: FetcherPaginationInit<T>): AsyncIterable<Page<T>>; + +// @public +export class PaginationError extends DexpaceError { + constructor(message: string, options?: ErrorOptions); +} // @public -export class HeadersBuilder implements Builder<Headers_2> { - add(name: string | HeaderName, value: string): this; - addInbound(name: string | HeaderName, value: string): this; - build(): Headers_2; - set(name: string | HeaderName, value: string | null): this; - setInbound(name: string | HeaderName, value: string | null): this; +export interface PaginationStrategy<T> { + parse(response: Response_2, template: Request_2): Promise<PageInfo<T>>; } // @public -export class HeaderValidationError extends DomainModelError { - constructor(kind: 'name' | 'value', offendingName: string, _offendingValue: string | undefined); - readonly escapedName: string; - readonly kind: 'name' | 'value'; +export class Paginator<T> { + constructor(init: PaginatorInit<T>); + items(): AsyncIterable<T>; + pages(): AsyncIterable<Page<T>>; } // @public -export class HttpRange { - static bounded(start: number, length: number): HttpRange; - get kind(): RangeKind; - get length(): number | undefined; - static open(start: number): HttpRange; - static parse(raw: string): HttpRange; - get raw(): string; - get start(): number | undefined; - static suffix(suffixLength: number): HttpRange; - get suffixLength(): number | undefined; +export interface PaginatorInit<T> { + readonly initialRequest: Request_2; + readonly maxPages?: number | undefined; + readonly options?: RequestOptions | undefined; + readonly signal?: AbortSignal | undefined; + readonly strategy: PaginationStrategy<T>; + readonly transport: Transport; } // @public -export class HttpRangeValidationError extends DomainModelError { +export interface PagingOptions { + [key: string]: unknown; + continuationToken?: string | undefined; + nextLink?: string | undefined; } // @public -export function isTimeoutSignal(signal: AbortSignal): boolean; +export function parseHttpDate(raw: string): number | null; // @public -export class MediaType { - get charset(): string | undefined; - equals(other: MediaType): boolean; - matches(pattern: MediaType): boolean; - static of(type: string, subtype: string, parameters?: ReadonlyMap<string, string>): MediaType; - parameter(key: string): string | undefined; - static parse(raw: string): MediaType; - render(): string; - get subtype(): string; - get type(): string; -} +export const PILLAR_STAGES: ReadonlySet<Stage>; // @public -export class MediaTypeParseError extends DomainModelError { +export class PillarCollisionError extends DexpaceError { + constructor(stage: Stage, existingType: symbol, incomingType: symbol, options?: ErrorOptions); + readonly existingType: symbol; + readonly incomingType: symbol; + readonly stage: Stage; } // @public -export type Method = 'GET' | 'HEAD' | 'POST' | 'PUT' | 'DELETE' | 'CONNECT' | 'OPTIONS' | 'TRACE' | 'PATCH'; +export class PipelineBuilder { + constructor(transport: Transport, options?: PipelineOptions); + append(descriptor: StepDescriptor): this; + appendAll(descriptors: readonly StepDescriptor[]): this; + build(): Runtime; + insertAfter(anchorType: symbol, descriptor: StepDescriptor): this; + insertBefore(anchorType: symbol, descriptor: StepDescriptor): this; + prepend(descriptor: StepDescriptor): this; + prependAll(descriptors: readonly StepDescriptor[]): this; + reload(descriptors: readonly StepDescriptor[]): this; + remove(type: symbol): this; + replace(anchorType: symbol, descriptor: StepDescriptor): this; + static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder; +} // @public -export class OperationAssemblyError extends DexpaceError { - constructor(message: string, parameterName: string); - readonly parameterName: string; +export interface PipelineOptions { + readonly instrumentation?: InstrumentationBundle | undefined; + readonly operationName?: string | undefined; } // @public -export interface OperationDescriptor { - readonly body?: unknown; - readonly headers?: Headers_2 | undefined; - readonly method: Method; - readonly pathParams?: Readonly<Record<string, string>> | undefined; - readonly pathTemplate: string; - readonly query?: QueryParams | undefined; +export class PlaintextCredentialError extends DexpaceError { + constructor(stepName: string, scheme: string); + readonly scheme: string; + readonly stepName: string; } +// @public +export function present<T>(value: NonNullable<T>): Tristate<T>; + // @public export class Protocol { equals(other: Protocol): boolean; @@ -146,9 +955,40 @@ export class Protocol { } // @public -export class ProtocolParseError extends DomainModelError { +export class ProtocolParseError extends DexpaceError { +} + +// @public +export interface ProxyCredentials { + readonly password: string; + readonly username: string; +} + +// @public +export interface ProxyOptions { + readonly bypassAll: boolean; + readonly challengeHandler?: unknown; + readonly credentials?: ProxyCredentials | undefined; + readonly host: string; + readonly nonProxyHosts: readonly string[]; + readonly port: number; + readonly type: ProxyType; +} + +// @public +export interface ProxyOptionsInit { + readonly bypassAll?: boolean | undefined; + readonly challengeHandler?: unknown; + readonly credentials?: ProxyCredentials | undefined; + readonly host: string; + readonly nonProxyHosts?: readonly string[] | undefined; + readonly port: number; + readonly type: ProxyType; } +// @public +export type ProxyType = 'http' | 'socks4' | 'socks5'; + // @public export class QueryParams { encode(): string; @@ -167,12 +1007,41 @@ export class QueryParamsBuilder implements Builder<QueryParams> { build(): QueryParams; } +// @public +export function randomUuid(): string; + // @public export type RangeKind = 'bounded' | 'suffix' | 'open'; +// @public +export type RecoveryStep = (outcome: Outcome<Response_2>) => Promise<Outcome<Response_2>>; + +// @public +export interface RedirectCondition { + readonly redirectsFollowed: number; + readonly response: Response_2; + readonly visited: ReadonlySet<string>; +} + +// @public +export type RedirectPredicate = (condition: Readonly<RedirectCondition>) => boolean; + +// @public +export interface RedirectSettings { + readonly allow303: boolean; + readonly allowedMethods: ReadonlySet<Method>; + readonly allowSchemeDowngrade: boolean; + readonly locationHeader: string; + readonly maxHops: number; + readonly predicate?: RedirectPredicate | undefined; +} + +// @public +export function redirectStep(overrides?: Partial<RedirectSettings>): StepDescriptor; + // @public class Request_2 { - get body(): unknown; + get body(): Body_2 | undefined; equals(other: Request_2): boolean; get headers(): Headers_2; get method(): Method; @@ -183,13 +1052,13 @@ class Request_2 { export { Request_2 as Request } // @public -export class RequestBodyNotAllowedError extends DomainModelError { +export class RequestBodyNotAllowedError extends DexpaceError { constructor(method: string); } // @public export class RequestBuilder implements Builder<Request_2> { - body(body: unknown): this; + body(body: Body_2 | undefined): this; build(): Request_2; headers(headers: Headers_2): this; method(method: Method): this; @@ -213,40 +1082,73 @@ export class RequestConditionsBuilder implements Builder<RequestConditions> { } // @public -export class RequestConditionsValidationError extends DomainModelError { +export class RequestConditionsValidationError extends DexpaceError { +} + +// @public +export interface RequestContext { + readonly instrumentation: InstrumentationBundle; + readonly key: symbol; + readonly kind: 'request'; + readonly operationName: string | undefined; + readonly request: Request_2; } // @public export class RequestOptions { + get auth(): AuthDescriptor | undefined; static readonly EMPTY: RequestOptions; get maxRetries(): number | undefined; static newBuilder(): RequestOptionsBuilder; newBuilder(): RequestOptionsBuilder; + get operationAuth(): AuthDescriptor | undefined; tag(key: string): string | undefined; get timeoutMs(): number | undefined; } // @public export class RequestOptionsBuilder implements Builder<RequestOptions> { + auth(descriptor: AuthDescriptor | undefined): this; build(): RequestOptions; maxRetries(value: number | undefined): this; + operationAuth(descriptor: AuthDescriptor | undefined): this; tags(entries: ReadonlyMap<string, string>): this; timeoutMs(value: number | undefined): this; } // @public -export class RequestOptionsValidationError extends DomainModelError { +export class RequestOptionsValidationError extends DexpaceError { } // @public -export class RequiredFieldError extends DomainModelError { +export class RequestRecoveryChain { + constructor(steps: readonly RequestStep[]); + apply(request: Request_2): Promise<Request_2>; +} + +// @public +export type RequestStep = (request: Request_2) => Promise<Request_2>; + +// @public +export class RequiredFieldError extends DexpaceError { constructor(fieldName: string); readonly fieldName: string; } +// @public +export class ReservedStageError extends DexpaceError { + constructor(operation: string, options?: ErrorOptions); + readonly operation: string; +} + +// @public +export function resolveProxyOptions(config: Configuration): ProxyOptions | null; + // @public class Response_2 { - get body(): unknown; + get body(): ReadableStream<Uint8Array> | null; + bytes(): Promise<Uint8Array>; + close(): Promise<void>; get headers(): Headers_2; static newBuilder(): ResponseBuilder; newBuilder(): ResponseBuilder; @@ -254,12 +1156,13 @@ class Response_2 { get reasonPhrase(): string | undefined; get request(): Request_2; get status(): Status; + text(): Promise<string>; } export { Response_2 as Response } // @public export class ResponseBuilder implements Builder<Response_2> { - body(body: unknown): this; + body(body: ReadableStream<Uint8Array> | null): this; build(): Response_2; headers(headers: Headers_2): this; protocol(protocol: Protocol): this; @@ -268,6 +1171,229 @@ export class ResponseBuilder implements Builder<Response_2> { status(status: Status): this; } +// @public +export class ResponseRecoveryChain { + constructor(responseSteps: readonly ResponseStep[], recoverySteps: readonly RecoveryStep[]); + apply(outcome: Outcome<Response_2>): Promise<Outcome<Response_2>>; +} + +// @public +export type ResponseStep = (response: Response_2) => Promise<Response_2>; + +// @public +export const RETRYABLE_STATUSES: ReadonlySet<number>; + +// @public +export function retryAttempts(error: unknown): readonly unknown[]; + +// @public +export class RetryDiscardedResponseError extends DexpaceError { + constructor(status: number, options?: ErrorOptions); + readonly status: number; +} + +// @public +export interface RetrySettings extends BackoffSettings { + readonly attemptHeaderName?: string | undefined; + readonly maxAttempts: number; + readonly retryableStatuses: ReadonlySet<number>; + readonly totalTimeoutMs?: number | undefined; +} + +// @public +export function retryStep(options?: RetryStepOptions): StepDescriptor; + +// @public +export interface RetryStepOptions { + readonly clock?: Clock | undefined; + readonly delayOverride?: ((attempt: number) => number | undefined) | undefined; + readonly random?: (() => number) | undefined; + readonly settings?: Partial<RetrySettings> | undefined; +} + +// @public +export class Runtime implements Transport { + close(): Promise<void>; + send(request: Request_2, options?: RequestOptions, signal?: AbortSignal): Promise<Response_2>; + get steps(): readonly StepDescriptor[]; + get transport(): Transport; +} + +// @public +export interface Schema<T> { + parse(input: unknown): T; +} + +// @public +export class SchemeDowngradeError extends DexpaceError { + constructor(fromUrl: string, toUrl: string, options?: ErrorOptions); + readonly fromUrl: string; + readonly toUrl: string; +} + +// @public +export interface Scope { + // (undocumented) + close(): void; +} + +// @public +export interface Serde { + readonly deserializer: Deserializer; + readonly mediaType: string; + readonly serializer: Serializer; +} + +// @public +export function serdeBody(value: unknown, serde: Serde, mediaType?: string): Body_2; + +// @public +export interface SerdeErrorOptions { + readonly cause?: unknown; +} + +// @public +export class SerializationError extends DexpaceError { + constructor(message: string, options?: SerdeErrorOptions); +} + +// @public +export interface Serializer { + serialize(value: unknown): Uint8Array; + serializeInto(value: unknown, target: Uint8Array, offset?: number): number; + serializeTo(value: unknown, sink: WritableStream<Uint8Array>, options?: { + readonly signal?: AbortSignal | undefined; + }): Promise<void>; + serializeToString(value: unknown): string; +} + +// @public +export function setGlobalConfiguration(config: Configuration): void; + +// @public +export function setGlobalLogger(logger: Logger): void; + +// @public +export function shouldBypassProxy(options: Pick<ProxyOptions, 'bypassAll' | 'nonProxyHosts'>, host: string): boolean; + +// @public +export class SourceContractViolationError extends DexpaceError { + constructor(message: string, options?: ErrorOptions); +} + +// @public +export type SourceFn = (key: string) => string | undefined; + +// @public +export interface Span { + // (undocumented) + end(): void; + // (undocumented) + readonly isRecording: boolean; + // (undocumented) + recordException(error: unknown): this; + // (undocumented) + setAttribute(key: string, value: unknown): this; + // (undocumented) + spanContext?(): SpanContext | undefined; +} + +// @public +export interface SpanContext { + // (undocumented) + readonly spanId: string; + // (undocumented) + readonly traceFlags?: number | undefined; + // (undocumented) + readonly traceId: string; + // (undocumented) + readonly traceState?: string | undefined; +} + +// @public +export interface SseEvent { + // (undocumented) + readonly comment: string | undefined; + readonly data: readonly string[]; + // (undocumented) + readonly event: string | undefined; + // (undocumented) + readonly id: string | undefined; + // (undocumented) + readonly retryMs: number | undefined; +} + +// @public +export interface SseEventFields { + // (undocumented) + readonly comment?: string | undefined; + // (undocumented) + readonly data?: readonly string[] | undefined; + // (undocumented) + readonly event?: string | undefined; + // (undocumented) + readonly id?: string | undefined; + // (undocumented) + readonly retryMs?: number | undefined; +} + +// @public +export function sseEventsEqual(a: SseEvent, b: SseEvent): boolean; + +// @public +export function sseEventToString(event: SseEvent): string; + +// @public +export class SseLineTooLongError extends DexpaceError { + constructor(limitBytes: number, options?: ErrorOptions); + readonly limitBytes: number; +} + +// @public +export type SseMapper<T> = (eventName: string | undefined, joinedData: string) => MapperOutcome<T>; + +// @public +export class SseStream implements AsyncIterable<SseEvent> { + [Symbol.asyncIterator](): AsyncIterator<SseEvent>; + close(): Promise<void>; +} + +// @public +export class SseStreamError extends DexpaceError { + constructor(message: string, options?: ErrorOptions); +} + +// @public +export function sseStreamFrom(response: Response_2, options?: SseStreamFromOptions): SseStream; + +// @public +export interface SseStreamFromOptions extends SseStreamOptions { + readonly maxLineBytes?: number | undefined; + readonly signal?: AbortSignal | undefined; +} + +// @public +export interface SseStreamOptions { + readonly onReleaseFailure?: ((error: unknown) => void) | undefined; +} + +// @public +export type Stage = 'PRE_REDIRECT' | 'REDIRECT' | 'POST_REDIRECT' | 'PRE_RETRY' | 'RETRY' | 'POST_RETRY' | 'PRE_AUTH' | 'AUTH' | 'POST_AUTH' | 'PRE_LOGGING' | 'LOGGING' | 'POST_LOGGING' | 'PRE_SERDE' | 'SERDE' | 'POST_SERDE' | 'SEND'; + +// @public +export const STAGE_ORDER: readonly Stage[]; + +// @public +export function standardResilience(transport: Transport, options?: StandardResilienceOptions): Runtime; + +// @public +export interface StandardResilienceOptions extends PipelineOptions { + readonly auth?: AuthStepSettings | undefined; + readonly logging?: LoggingStepSettings | undefined; + readonly redirect?: Partial<RedirectSettings> | undefined; + readonly retry?: RetryStepOptions | undefined; +} + // @public export class Status { get code(): number; @@ -284,6 +1410,79 @@ export class Status { static recognized(code: number): Status | undefined; } +// @public +export function statusMappingStep(response: Response_2): Promise<Response_2>; + +// @public +export type Step = (request: Request_2, ctx: StepContext) => Promise<Response_2>; + +// @public +export interface StepContext { + readonly context: ExecutionContext; + readonly fork?: (() => Next) | undefined; + readonly next: Next; + readonly options?: RequestOptions | undefined; + readonly signal?: AbortSignal | undefined; +} + +// @public +export interface StepDescriptor { + readonly fn: Step; + readonly stage: Stage; + readonly type: symbol; +} + +// @public +export class StreamBody implements Body_2 { + constructor(stream: ReadableStream<Uint8Array>, mediaType?: string, contentLength?: number); + readonly contentLength: number; + readonly kind: "stream"; + readonly mediaType: string | undefined; + readonly replayable = false; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +// @public +export function streamBody(stream: ReadableStream<Uint8Array>, mediaType?: string, contentLength?: number): StreamBody; + +// @public +export class StringBody implements Body_2 { + constructor(text: string, mediaType?: string); + readonly contentLength: number; + readonly kind: "string"; + readonly mediaType: string; + readonly replayable = true; + readonly text: string; + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +// @public +export function stringBody(text: string, mediaType?: string): StringBody; + +// @public +export function stripCrossOriginMarkerStep(): StepDescriptor; + +// @public +export function success<T>(value: T): Outcome<T>; + +// @public +export interface SuppressedErrorLike extends Error { + readonly error: unknown; + readonly suppressed: unknown; +} + +// @public +export function toHttpError(response: Response_2): Promise<HttpStatusError | null>; + +// @public +export type TokenProvider = () => Promise<BearerToken>; + +// @public +export interface Tracer { + // (undocumented) + startSpan(name: string): Span; +} + // @public export interface Transport { close(): Promise<void>; @@ -291,7 +1490,61 @@ export interface Transport { } // @public -export class UrlConstructionError extends DomainModelError { +export class TransportFailureError extends IoError { + constructor(message: string, options?: ErrorOptions); +} + +// @public +export type Tristate<T> = { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'absent'; +} | { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'null'; +} | { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'present'; + readonly value: T; +}; + +// @public +export const TRISTATE_BRAND: unique symbol; + +// @public +export interface TristateBranches<T, R> { + readonly onAbsent: () => R; + readonly onNull: () => R; + readonly onPresent: (value: T) => R; +} + +// @public +export function tristateToString<T>(tristate: Tristate<T>): string; + +// @public +export class TypedResponse<T> { + constructor(response: Response_2, parse: (response: Response_2) => Promise<T>); + get headers(): Response_2['headers']; + get protocol(): string; + get reason(): string | undefined; + get request(): Request_2; + get status(): Response_2['status']; + value(): Promise<T>; +} + +// @public +export function typedSseStream<T>(stream: SseStream, mapper: SseMapper<T>): AsyncIterable<T>; + +// @public +export class UrlConstructionError extends DexpaceError { } +// @public +export function valueOrNull<T>(tristate: Tristate<T>): T | null; + +// @public +export function withRedirect(builder: PipelineBuilder, overrides?: Partial<RedirectSettings>): PipelineBuilder; + +// @public +export function wrapCancellation(error: unknown): Outcome<never>; + ``` diff --git a/packages/core/package.json b/packages/core/package.json index 835e89a..535722e 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -6,7 +6,12 @@ "main": "./dist/index.js", "types": "./dist/index.d.ts", "engines": { - "node": ">=18.17" + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/core" }, "exports": { ".": { @@ -20,6 +25,7 @@ "sideEffects": false, "dependencies": {}, "scripts": { + "prebuild": "node scripts/gen-version.mjs", "build": "tsc -p tsconfig.build.json", "typecheck": "tsc --noEmit", "test": "bun test", @@ -28,6 +34,6 @@ "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" }, "devDependencies": { - "expect-type": "^1.4.0" + "expect-type": "catalog:" } } diff --git a/packages/core/scripts/gen-version.mjs b/packages/core/scripts/gen-version.mjs new file mode 100644 index 0000000..9f9d6d0 --- /dev/null +++ b/packages/core/scripts/gen-version.mjs @@ -0,0 +1,38 @@ +#!/usr/bin/env node +// SPDX-License-Identifier: MIT +// packages/core/scripts/gen-version.mjs +// Writes src/generated/version.ts as a plain string-literal constant taken from package.json's +// version field, so @dexpace/core never needs a runtime package.json read -- that would require +// node:fs / import.meta.url tricks unavailable on the browser/Workers half of core's runtime floor, +// and would leave those builds reporting the "unknown" placeholder NFR-15 forbids. +// +// Runs as the package's prebuild step. Its output is committed too, so a `bun test` run that has not +// built still sees a real (if stale) version rather than a placeholder. +import {mkdirSync, readFileSync, writeFileSync} from 'node:fs'; +import {dirname, join} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +const packageRoot = join(dirname(fileURLToPath(import.meta.url)), '..'); +const outputPath = join(packageRoot, 'src', 'generated', 'version.ts'); + +const {version} = JSON.parse( + readFileSync(join(packageRoot, 'package.json'), 'utf8'), +); + +if (typeof version !== 'string' || !/^[\w.+-]+$/.test(version)) { + throw new Error( + `gen-version: packages/core/package.json's "version" is missing or not a bare version string: ${String(version)}`, + ); +} + +const contents = `// SPDX-License-Identifier: MIT +// packages/core/src/generated/version.ts +// Generated by scripts/gen-version.mjs from package.json -- do not edit by hand. + +/** The published version of \`@dexpace/core\`, compiled in at build time (NFR-15). @internal */ +export const SDK_VERSION = '${version}'; +`; + +mkdirSync(dirname(outputPath), {recursive: true}); +writeFileSync(outputPath, contents); +console.log(`gen-version: wrote ${outputPath} (${version})`); diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts new file mode 100644 index 0000000..f8e266e --- /dev/null +++ b/packages/core/src/auth/auth-step.test.ts @@ -0,0 +1,1755 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/auth-step.test.ts +// Exercises: XCUT-16 (a credential is NEVER stamped over a non-HTTPS transport; the refusal is loud and +// lands before any token fetch or header write, and it applies only on the credential-attaching path -- +// a marker-suppressed cross-origin re-issue may proceed credential-free over any scheme), +// AUTH-27 (exactly one AUTH-stage descriptor, pinned to the pillar), AUTH-28 (HTTPS guard, +// NO_AUTH exempt, re-applied on the replay path -- unconditionally once the outbound pass guarded the +// hop, whatever header names the replacement carries, and additionally on an Authorization or +// Proxy-Authorization the hook adds to an unguarded NO_AUTH hop), AUTH-29 (the cross-origin marker skips the guard and +// stamping, is cleared from the outbound headers, and suppresses the challenge reaction too -- so the +// credential cannot re-enter via the 401), AUTH-25 (a 407 is answered from Proxy-Authenticate into +// Proxy-Authorization), AUTH-12/AUTH-13 (EVERY value of the matching challenge header is parsed, each +// value on its own, so a repeated WWW-Authenticate/Proxy-Authenticate offers its later challenges too +// and a malformed earlier value cannot swallow them), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives +// exactly once through a fresh fork()), AUTH-31 (a non-replayable replacement body surfaces the +// original challenge unchanged and unclosed), AUTH-32 (a throwing hook closes the challenge response +// before propagating), AUTH-33 (no matching challenge header, or a hook yielding nothing -> unchanged), +// AUTH-36 (OAUTH2's default hook evicts the exact rejected token and re-stamps -- including behind a +// non-replayable body, where only the REPLAY is skipped), AUTH-4 (a per-call RequestOptions.auth +// descriptor overrides the configured tiers, via ctx.options), AUTH-5/AUTH-6 (resolution against the +// derived available-scheme set), RECOV-12 (a failing release never masks the primary error), +// AUTH-34/AUTH-35 (a refresh margin is validated as a finite, non-negative duration). +import {describe, expect, test} from 'bun:test'; +import {streamBody} from '../body/stream-body.js'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {RequestOptions} from '../http/request-options.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {Cursor} from '../pipeline/cursor.js'; +import type {Transport} from '../seams/transport.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {CROSS_ORIGIN_MARKER_HEADER} from '../redirect/cross-origin.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {invariant} from '../invariant.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import { + AUTH_STEP_TYPE, + authStep, + availableSchemesOf, + type AuthCredentialSet, +} from './auth-step.js'; +import { + createBearerToken, + ApiKeyCredential, + BasicCredential, + DigestCredential, +} from './credential.js'; +import {createAuthDescriptor} from './descriptor.js'; +import {AuthResolutionError, PlaintextCredentialError} from './errors.js'; +import {createAuthRequirement} from './requirement.js'; +import type {AuthScheme} from './scheme.js'; + +// Constructed inline rather than imported: 4c keeps `aRequestContext()` file-local to `cursor.test.ts`, +// and importing across `*.test.ts` files is not acceptable -- the same call 5a's and 5b's step suites made. +function aRequestContext(request: Request): ExecutionContext { + return createRequestContext(request); +} + +function aRequest(url = 'https://example.com/a'): Request { + return Request.newBuilder().url(url).build(); +} + +function markedRequest(url: string): Request { + return Request.newBuilder() + .url(url) + .headers(Headers.newBuilder().add(CROSS_ORIGIN_MARKER_HEADER, '1').build()) + .build(); +} + +/** The optional per-drive inputs, bundled so `runThrough` stays within `max-params`. */ +interface DriveOverrides { + readonly request?: Request | undefined; + readonly options?: RequestOptions | undefined; + readonly signal?: AbortSignal | undefined; +} + +// `Transport`, not `FakeTransport`: the only thing this helper does with it is hand it to `Cursor`, +// and narrowing to what is actually used is what lets the gated double below be driven through it too +// (`docs/knowledge/harvested/api-design.md` -- accept the narrowest interface describing the members used). +function runThrough( + descriptor: StepDescriptor, + transport: Transport, + overrides: DriveOverrides = {}, +): Promise<Response> { + const request = overrides.request ?? aRequest(); + return new Cursor({ + steps: [descriptor], + transport, + request, + context: aRequestContext(request), + options: overrides.options, + signal: overrides.signal, + }).advance(); +} + +function tiersFor(scheme: AuthScheme): { + client: ReturnType<typeof createAuthDescriptor>; +} { + return {client: createAuthDescriptor([createAuthRequirement(scheme)])}; +} + +/** + * A challenge response: `countingResponse` plus the challenge header. `ResponseBuilder` carries the + * SAME body instance through `newBuilder()`, so the rebuilt response still reports through the + * original's release counter. `setInbound`, not `set`: these are inbound headers, and a real server may + * send obs-text in a realm (HTTP-19). + */ +function challengeResponse( + status: number, + headerName: string, + headerValue: string, +): {response: Response; cancelCount: () => number} { + const base = countingResponse(status); + const response = base.response + .newBuilder() + .headers( + base.response.headers + .newBuilder() + .setInbound(headerName, headerValue) + .build(), + ) + .build(); + return {response, cancelCount: base.cancelCount}; +} + +/** + * A challenge response carrying the same challenge header SEVERAL times — the wire shape RFC 7616 + * §3.3 recommends for algorithm discovery, and the one `@dexpace/transport-undici` hands over as + * separate entries rather than one comma-joined value. + * + * `addInbound` in a loop, never `setInbound`: `set` REPLACES, so building the fixture with it would + * quietly assert against a single-valued header and the row would pass against the very bug it exists + * to catch. + */ +function repeatedChallengeResponse( + status: number, + headerName: string, + headerValues: readonly string[], +): {response: Response; cancelCount: () => number} { + const base = countingResponse(status); + const builder = base.response.headers.newBuilder(); + for (const value of headerValues) builder.addInbound(headerName, value); + const response = base.response.newBuilder().headers(builder.build()).build(); + return {response, cancelCount: base.cancelCount}; +} + +/** A one-shot request body: `StreamBody.replayable` is `false` (AUTH-31's gate). */ +function oneShotPost(url = 'https://example.com/a'): Request { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.close(); + }, + }); + return Request.newBuilder() + .method('POST') + .url(url) + .body(streamBody(stream, 'text/plain', 0)) + .build(); +} + +const CANCEL_FAILURE = new Error('cancel exploded'); + +/** + * A 401 whose body `cancel()` REJECTS with a non-`TypeError` -- the one thing `Response.close()` is + * documented to rethrow. Models a transport releasing over an already-broken socket. Same shape 5b's + * `redirect-step.test.ts` uses for its own RECOV-12 coverage. + */ +function hostileChallenge(value = 'Basic realm="x"'): Response { + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw CANCEL_FAILURE; + }, + }); + return Response.newBuilder() + .request(aRequest()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(401)) + .headers(Headers.newBuilder().setInbound('WWW-Authenticate', value).build()) + .body(body) + .build(); +} + +/** + * A macrotask boundary, so a fire-and-forget background refresh's whole then/finally chain has + * drained regardless of how many microtask hops it takes. Same helper `bearer-cache.test.ts` uses. + */ +function drainMacrotask(): Promise<void> { + return new Promise(resolve => { + setTimeout(resolve, 0); + }); +} + +/** + * Drives three requests through `descriptor`, draining between them, and reports the `Authorization` + * value each one actually put on the wire. + * + * Three drives is the shortest sequence that can observe a refresh MARGIN at all: the first fills an + * empty cache (where no margin is consulted), the second is the one the margin either does or does + * not push into AUTH-37's expiring-but-valid zone, and the third reveals whether that zone's + * background refresh actually happened. + */ +async function stampsOverThreeDrives( + descriptor: StepDescriptor, +): Promise<readonly (string | undefined)[]> { + const transport = new FakeTransport([ + countingResponse(200).response, + countingResponse(200).response, + countingResponse(200).response, + ]); + for (let drive = 0; drive < 3; drive += 1) { + await runThrough(descriptor, transport); + await drainMacrotask(); + } + return transport.calls.map(call => call.request.headers.get('Authorization')); +} + +/** + * A provider issuing `t1` at `firstExpiresAt` and then `t2` far out of any margin's reach, counting + * its calls. Every test below pins the clock at 0, so `firstExpiresAt` IS t1's remaining lifetime. + */ +function agingTokenProvider(firstExpiresAt: number): { + readonly credentials: AuthCredentialSet; + readonly callCount: () => number; +} { + let issued = 0; + return { + credentials: { + bearer: { + provider: () => { + issued += 1; + return Promise.resolve( + createBearerToken( + `t${String(issued)}`, + issued === 1 ? firstExpiresAt : 10_000_000, + ), + ); + }, + }, + }, + callCount: () => issued, + }; +} + +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('availableSchemesOf (AUTH-5)', () => { + test('is empty for an empty credential set', () => { + expect([...availableSchemesOf({})]).toEqual([]); + }); + + test('maps each configured credential to its scheme', () => { + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p'), + bearer: {provider: () => Promise.resolve(createBearerToken('t'))}, + apiKey: {credential: new ApiKeyCredential('k')}, + }; + expect([...availableSchemesOf(credentials)].sort()).toEqual([ + 'API_KEY', + 'BASIC', + 'DIGEST', + 'OAUTH2', + ]); + }); +}); + +describe('authStep: resolution and the preemptive stamp (AUTH-26..AUTH-28, AUTH-34)', () => { + test('is pinned to the AUTH pillar stage (AUTH-27)', () => { + const descriptor = authStep({credentials: {}, tiers: tiersFor('NO_AUTH')}); + expect(descriptor.stage).toBe('AUTH'); + expect(descriptor.type).toBe(AUTH_STEP_TYPE); + }); + + test('NO_AUTH stamps nothing and never triggers the HTTPS guard, even over plain HTTP (AUTH-28)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const descriptor = authStep({credentials: {}, tiers: tiersFor('NO_AUTH')}); + + await runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }); + + expect( + transport.calls[0]?.request.headers.get('Authorization'), + ).toBeUndefined(); + }); + + test('API_KEY stamps preemptively via the configured header/prefix (AUTH-26)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }; + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + + await runThrough(descriptor, transport); + + expect(transport.calls[0]?.request.headers.get('X-Api-Key')).toBe('secret'); + }); + + test('OAUTH2 stamps a cached bearer token preemptively (AUTH-34)', async () => { + const transport = new FakeTransport([ + countingResponse(200).response, + countingResponse(200).response, + ]); + let calls = 0; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + calls += 1; + return Promise.resolve( + createBearerToken(`t${String(calls)}`, 100_000), + ); + }, + }, + }; + const descriptor = authStep({ + credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + await runThrough(descriptor, transport); + await runThrough(descriptor, transport); + + expect(transport.calls[0]?.request.headers.get('Authorization')).toBe( + 'Bearer t1', + ); + // The second call reads the still-fresh cached token rather than refetching. + expect(transport.calls[1]?.request.headers.get('Authorization')).toBe( + 'Bearer t1', + ); + expect(calls).toBe(1); + }); +}); + +describe('authStep: the HTTPS guard and tier resolution (AUTH-6/AUTH-28)', () => { + test('a credentialed scheme over plain HTTP throws PlaintextCredentialError before any send (AUTH-28)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: {credential: new ApiKeyCredential('secret')}, + }; + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + + const error = await rejectionOf( + runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }), + ); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect((error as PlaintextCredentialError).scheme).toBe('API_KEY'); + expect(transport.sendCount).toBe(0); + }); + + test('the guard fires before the token fetch, not after (AUTH-28)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + let fetched = false; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + fetched = true; + return Promise.resolve(createBearerToken('t', 100_000)); + }, + }, + }; + const descriptor = authStep({ + credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + await rejectionOf( + runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }), + ); + + expect(fetched).toBe(false); + }); + + test('an unsatisfiable tier surfaces AuthResolutionError (AUTH-6)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const descriptor = authStep({credentials: {}, tiers: tiersFor('BASIC')}); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(AuthResolutionError); + expect(transport.sendCount).toBe(0); + }); + + test('BASIC/DIGEST never stamp preemptively -- the outbound request carries no Authorization', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + await runThrough(descriptor, transport); + + expect( + transport.calls[0]?.request.headers.get('Authorization'), + ).toBeUndefined(); + }); +}); + +describe('authStep: the cross-origin marker (AUTH-29)', () => { + test('AUTH-29: a cross-origin-marked request skips the guard and stamping, marker cleared', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: {credential: new ApiKeyCredential('secret')}, + }; + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + // Plain HTTP -- would normally trip the guard, but the marker skips it (AUTH-29). + const marked = markedRequest('http://example.com/a'); + + await runThrough(descriptor, transport, {request: marked}); + + const sent = transport.calls[0]?.request; + expect(sent?.headers.get('Authorization')).toBeUndefined(); + expect(sent?.headers.has(CROSS_ORIGIN_MARKER_HEADER)).toBe(false); + }); + + test('AUTH-29: the marker is cleared even on the ordinary same-origin path', async () => { + // An unmarked request has nothing to clear, but a marked HTTPS request on a stamping path must + // still not forward the header -- clearing happens before the branch, not inside one of them. + const transport = new FakeTransport([countingResponse(200).response]); + const descriptor = authStep({credentials: {}, tiers: tiersFor('NO_AUTH')}); + + await runThrough(descriptor, transport, { + request: markedRequest('https://example.com/a'), + }); + + expect( + transport.calls[0]?.request.headers.has(CROSS_ORIGIN_MARKER_HEADER), + ).toBe(false); + }); + + test('AUTH-29: a cross-origin-marked request does NOT answer a challenge either', async () => { + // The suppression covers the whole hop. Answering the challenge here would stamp exactly the + // credential the outbound pass declined to send, onto the server-chosen foreign host, over a URL + // whose HTTPS guard was skipped -- the precise leak AUTH-29 exists to prevent. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport, { + request: markedRequest('http://evil.example/a'), + }); + + expect(transport.sendCount).toBe(1); // no re-drive was attempted + expect(response).toBe(challenged.response); // unchanged and unclosed -- the caller owns it + expect(challenged.cancelCount()).toBe(0); + }); +}); + +describe('authStep: challenge detection (AUTH-25/AUTH-33)', () => { + test('a 407 is answered from Proxy-Authenticate into Proxy-Authorization (AUTH-25)', async () => { + const challenged = challengeResponse( + 407, + 'Proxy-Authenticate', + 'Basic realm="p"', + ); + const success = countingResponse(200); + const transport = new FakeTransport([ + challenged.response, + success.response, + ]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Proxy-Authorization') + ?.startsWith('Basic '), + ).toBe(true); + expect( + transport.calls[1]?.request.headers.get('Authorization'), + ).toBeUndefined(); + }); + + test('a 401 carrying only Proxy-Authenticate is NOT answered (AUTH-25)', async () => { + const challenged = challengeResponse( + 401, + 'Proxy-Authenticate', + 'Basic realm="p"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(1); + expect(response).toBe(challenged.response); + }); +}); + +describe('authStep: challenge detection, negative cases (AUTH-33)', () => { + test('a 401 without WWW-Authenticate is returned unchanged (AUTH-33)', async () => { + const the401 = countingResponse(401); + const transport = new FakeTransport([the401.response]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport); + + expect(response).toBe(the401.response); + expect(transport.sendCount).toBe(1); + expect(the401.cancelCount()).toBe(0); + }); + + test('a non-challenge status is returned untouched', async () => { + const success = countingResponse(200); + const transport = new FakeTransport([success.response]); + const descriptor = authStep({credentials: {}, tiers: tiersFor('NO_AUTH')}); + + expect(await runThrough(descriptor, transport)).toBe(success.response); + }); +}); + +describe('authStep: the challenge replay (AUTH-30/AUTH-31)', () => { + test('a 401 with a Basic challenge re-drives exactly once with the stamped Authorization (AUTH-30)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const success = countingResponse(200); + const transport = new FakeTransport([ + challenged.response, + success.response, + ]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Authorization') + ?.startsWith('Basic '), + ).toBe(true); + expect(response).toBe(success.response); + expect(challenged.cancelCount()).toBe(1); // AUTH-30: the original is closed before the re-drive + }); + + test('no nested re-challenge: a second 401 on the replay is returned as-is (AUTH-30)', async () => { + const first = challengeResponse(401, 'WWW-Authenticate', 'Basic realm="x"'); + const second = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([first.response, second.response]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); // exactly one replay, not a loop + expect(response).toBe(second.response); + expect(second.cancelCount()).toBe(0); // the surfaced response is the caller's, left open + }); +}); + +describe('authStep: answering a Digest challenge (AUTH-15..AUTH-22)', () => { + test('a Digest challenge is answered with a Digest header value (AUTH-15..22)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Digest realm="r", nonce="n", qop="auth"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const credentials: AuthCredentialSet = { + digest: new DigestCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('DIGEST')}); + + await runThrough(descriptor, transport, { + request: aRequest('https://example.com/a?q=1'), + }); + + const value = transport.calls[1]?.request.headers.get('Authorization'); + expect(value?.startsWith('Digest ')).toBe(true); + // AUTH-22: the digest-uri is the request-target, path AND query. + expect(value).toContain('uri="/a?q=1"'); + }); +}); + +describe('authStep: repeated challenge headers (AUTH-12/AUTH-16/AUTH-25)', () => { + test('a later WWW-Authenticate entry is answered when the first is unsupported', async () => { + // RFC 7616 §3.3's algorithm-discovery shape: one header per algorithm, strongest first. Reading + // only `headers.get(...)` saw the SHA-512-256 line, found nothing satisfiable, and surfaced the + // 401 — while the identical pair comma-joined into ONE value authenticated (audit #67 / #74). + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce="n", algorithm=SHA-512-256', + 'Digest realm="r", nonce="n", algorithm=SHA-256, qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + const value = transport.calls[1]?.request.headers.get('Authorization'); + expect(value).toContain('algorithm=SHA-256'); + expect(value).toContain('qop=auth'); + }); + + test('a repeated Proxy-Authenticate is read the same way (AUTH-25)', async () => { + const challenged = repeatedChallengeResponse(407, 'Proxy-Authenticate', [ + 'Negotiate abc123', + 'Basic realm="p"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Proxy-Authorization') + ?.startsWith('Basic '), + ).toBe(true); + }); +}); + +describe('authStep: one offered challenge declined, the next answered (AUTH-13/AUTH-16)', () => { + test('a malformed FIRST entry cannot swallow a satisfiable later one (AUTH-13)', async () => { + // The row that fixes the parse strategy rather than only the read. Comma-joining the two values + // before parsing lets the unterminated quoted string in the first run on into the second — the + // scanner closes it at the `"` of `realm="r"`, and the whole satisfiable challenge disappears + // into a realm value. Parsing each value on its own bounds the damage at the value that carries + // it, which is what AUTH-13's "recovers to the next top-level comma" is reaching for. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="unterminated', + 'Digest realm="r", nonce="n", qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="r"', + ); + }); + + test('a challenge with an empty nonce is declined and the next one answered', async () => { + // Both halves of the same 401: a truncated first challenge that parses to `nonce: ''`, and a + // well-formed second. Declining is only useful if the step then keeps looking, which is what + // AUTH-25's "return no header when it cannot satisfy ANY offered challenge" requires -- the + // quantifier is over the whole offer, not over the first entry. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce=', + 'Digest realm="second", nonce="n", qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="second"', + ); + }); +}); + +describe('authStep: the replayability gate (AUTH-31)', () => { + test('an unsatisfiable challenge leaves the response unchanged (AUTH-25/AUTH-33)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Negotiate abc123', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(1); + expect(response).toBe(challenged.response); + expect(challenged.cancelCount()).toBe(0); + }); + + // AUTH-31 gates the DISPATCH only. The hook still runs for a one-shot body -- there is deliberately + // no "skip the hook when the body is one-shot" fast path (see `handleChallenge`), because OAUTH2's + // default hook evicts the rejected token on the way past and that work is not wasted. What this + // test pins is the replay gate's own three obligations; the eviction half is pinned separately by + // 'a revoked token is evicted even though the replay is skipped' below. + test('a non-replayable body surfaces the original 401 unchanged and unclosed, with no replay (AUTH-31)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const credentials: AuthCredentialSet = { + basic: new BasicCredential('u', 'p'), + }; + const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); + + const response = await runThrough(descriptor, transport, { + request: oneShotPost(), + }); + + expect(response).toBe(challenged.response); + expect(transport.sendCount).toBe(1); // no replacement dispatch was attempted + expect(challenged.cancelCount()).toBe(0); // the caller owns it -- MUST NOT be closed + }); + + test('AUTH-31 also gates a caller hook that returns a non-replayable replacement', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: () => Promise.resolve(oneShotPost()), + }); + + const response = await runThrough(descriptor, transport); + + expect(response).toBe(challenged.response); + expect(transport.sendCount).toBe(1); + expect(challenged.cancelCount()).toBe(0); + }); +}); + +describe('authStep: challenge-hook failure and override (AUTH-30/AUTH-32/AUTH-33)', () => { + test('a throwing challengeHook closes the 401 before propagating (AUTH-32)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: () => Promise.reject(new Error('hook exploded')), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect((error as Error).message).toBe('hook exploded'); + expect(challenged.cancelCount()).toBe(1); + }); + + test('a hook throwing SYNCHRONOUSLY also closes the 401 (AUTH-32/AUTH-38)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: (): Promise<Request | undefined> => { + throw new Error('sync boom'); + }, + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect((error as Error).message).toBe('sync boom'); + expect(challenged.cancelCount()).toBe(1); + }); + + test('a hook yielding nothing leaves the 401 unchanged and unclosed (AUTH-33)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: () => Promise.resolve(undefined), + }); + + const response = await runThrough(descriptor, transport); + + expect(response).toBe(challenged.response); + expect(challenged.cancelCount()).toBe(0); + }); +}); + +describe('authStep: hook override and non-reactive schemes (AUTH-30)', () => { + test('a caller-supplied challengeHook takes precedence over the scheme default (AUTH-30)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + let hookInvoked = false; + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + challengeHook: (_response, request) => { + hookInvoked = true; + return Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('Authorization', 'Custom xyz') + .build(), + ) + .build(), + ); + }, + }); + + await runThrough(descriptor, transport); + + expect(hookInvoked).toBe(true); + expect(transport.calls[1]?.request.headers.get('Authorization')).toBe( + 'Custom xyz', + ); + }); +}); + +describe('authStep: schemes with no reactive behavior (AUTH-30)', () => { + test('API_KEY does not react to a 401 -- static credentials have no reactive behavior (AUTH-30)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const credentials: AuthCredentialSet = { + apiKey: {credential: new ApiKeyCredential('secret')}, + }; + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(1); + expect(response).toBe(challenged.response); + }); +}); + +describe('authStep: the OAUTH2 default hook (AUTH-36)', () => { + test('OAUTH2 default hook evicts the exact rejected token and re-stamps (AUTH-36)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Bearer realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + let calls = 0; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + calls += 1; + return Promise.resolve( + createBearerToken(`t${String(calls)}`, 100_000), + ); + }, + }, + }; + const descriptor = authStep({ + credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + await runThrough(descriptor, transport); + + expect(transport.calls[0]?.request.headers.get('Authorization')).toBe( + 'Bearer t1', + ); + // Evicted t1, fetched genuinely fresh. + expect(transport.calls[1]?.request.headers.get('Authorization')).toBe( + 'Bearer t2', + ); + expect(calls).toBe(2); + }); +}); + +/** + * Holds one nominated send until {@link GatedTransport.release} is called, so a two-drive + * interleaving can be pinned instead of left to the scheduler. Everything else delegates to the + * scripted double. + */ +class GatedTransport implements Transport { + readonly #inner: FakeTransport; + readonly #gatedEntry: number; + #entered = 0; + #release: (() => void) | undefined; + readonly #gate: Promise<void>; + + constructor(inner: FakeTransport, gatedEntry: number) { + this.#inner = inner; + this.#gatedEntry = gatedEntry; + this.#gate = new Promise<void>(resolve => { + this.#release = resolve; + }); + } + + release(): void { + this.#release?.(); + } + + async send(request: Request): Promise<Response> { + // Counted on ENTRY, not off the inner double's `sendCount`: a gated call has not reached the + // inner transport yet, so `sendCount` would still be pointing at the gated position and every + // later call would gate too -- a deadlock, which is exactly what the first shape of this did. + this.#entered += 1; + if (this.#entered === this.#gatedEntry) await this.#gate; + return this.#inner.send(request); + } + + async close(): Promise<void> { + // Nothing to release; the inner double owns no resources. + } +} + +describe('authStep: OAUTH2 preserves a token another request refreshed (AUTH-36)', () => { + test('a 401 on a token the cache has already replaced stamps the survivor, with no second fetch', async () => { + // AUTH-36's "preserving a token another request already refreshed", at the seam where it is + // actually observable. Two drives both stamp `t1` off one single-flight fetch. Drive A's 401 + // runs to completion first -- evicting `t1` and caching `t2` -- and only then is drive B's 401 + // released. B's rejected header (`t1`) no longer matches the cache (`t2`), so the eviction + // PRESERVES `t2` and the retry stamps it. Burning a third provider call to re-derive the same + // token, which the earlier unconditional-`refreshNow()` shape did, is what makes the clause a + // no-op rather than a behaviour. + // Scripted in the order the inner double actually SEES them, which the gate pins: A's 401, A's + // replay, then B's 401 and B's replay once released. + const inner = new FakeTransport([ + challengeResponse(401, 'WWW-Authenticate', 'Bearer realm="x"').response, + countingResponse(200).response, + challengeResponse(401, 'WWW-Authenticate', 'Bearer realm="x"').response, + countingResponse(200).response, + ]); + const transport = new GatedTransport(inner, 2); // hold drive B's first send + let calls = 0; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + calls += 1; + return Promise.resolve( + createBearerToken(`t${String(calls)}`, 100_000), + ); + }, + }, + }; + const descriptor = authStep({ + credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + const driveA = runThrough(descriptor, transport); + const driveB = runThrough(descriptor, transport); + await driveA; + transport.release(); + await driveB; + + expect(calls).toBe(2); // the initial fetch and A's post-eviction fetch. B fetched nothing. + expect(inner.calls[2]?.request.headers.get('Authorization')).toBe( + 'Bearer t1', // B's original stamp, the one the server rejected + ); + expect(inner.calls[3]?.request.headers.get('Authorization')).toBe( + 'Bearer t2', // the PRESERVED token, stamped without a third fetch + ); + }); +}); + +describe('authStep: OAUTH2 declines a non-Bearer challenge (AUTH-36)', () => { + test('OAUTH2 leaves a 401 unchanged when it advertises no Bearer challenge (AUTH-36)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + let calls = 0; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + calls += 1; + return Promise.resolve( + createBearerToken(`t${String(calls)}`, 100_000), + ); + }, + }, + }; + const descriptor = authStep({ + credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(1); + expect(response).toBe(challenged.response); + expect(calls).toBe(1); // no eviction-driven refetch + }); +}); + +describe('authStep: the replay HTTPS guard (AUTH-28)', () => { + test('AUTH-28 is re-applied to a challenge replacement that carries a credential', async () => { + // The outbound guard is SKIPPED for NO_AUTH, and nothing constrains a caller hook to preserve the + // URL -- so without a second guard a hook answering a challenge stamps a credential over plaintext. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), // outbound guard skipped entirely + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('Authorization', 'Basic c3B5') + .build(), + ) + .build(), + ), + }); + + const error = await rejectionOf( + runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }), + ); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); // the replacement never reached the wire + expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw, not leaked + }); +}); + +describe('authStep: once guarded, the replay stays guarded (AUTH-28)', () => { + test('a replacement carrying a NON-standard credential header over plaintext is refused (AUTH-28)', async () => { + // The reported hole (audit #67 / #71): the guard tested two header NAMES, and + // `ApiKeyCredentialConfig.headerName` lets this very step stamp any header it is told to. A hook + // that downgrades the URL and answers with `X-Api-Key` therefore went out in clear text, with no + // `PlaintextCredentialError`. The rule is now "the outbound pass guarded this hop, so the replay + // is guarded too", which does not depend on reading header names at all. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }, + tiers: tiersFor('API_KEY'), // the outbound pass DID guard: the seed is https + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .url('http://example.com/a') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); // the replacement never reached the wire + expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw + }); + + test('a header-free replacement over plaintext is refused too, once the hop was guarded (AUTH-28)', async () => { + // The other half of the same rule, and the reason it is stated as "once guarded, always guarded" + // rather than as a wider header list: a hook is free to invent a credential carrier this step has + // never heard of, so no enumeration of names can be complete. The scheme that made the outbound + // pass credentialed is what the replay inherits. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: {credential: new ApiKeyCredential('secret')}, + }, + tiers: tiersFor('API_KEY'), + challengeHook: (_response, request) => + Promise.resolve( + Request.newBuilder() + .url('http://example.com/b') + .method(request.method) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); + }); +}); + +describe('authStep: an unguarded NO_AUTH hop (AUTH-28/AUTH-29)', () => { + test('a credential-free replacement over plaintext is NOT blocked by the replay guard (AUTH-28/AUTH-29)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const success = countingResponse(200); + const transport = new FakeTransport([ + challenged.response, + success.response, + ]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: (_response, request) => + Promise.resolve( + request.newBuilder().url('http://example.com/b').build(), + ), + }); + + const response = await runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }); + + expect(transport.sendCount).toBe(2); + expect(response).toBe(success.response); + }); +}); + +describe('authStep: per-call configuration and injected seams (AUTH-4/AUTH-11)', () => { + test('a per-call RequestOptions.auth descriptor overrides the configured tiers (AUTH-4)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }; + // Configured tiers resolve to API_KEY; the per-call descriptor demands NO_AUTH and must win. + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + const options = RequestOptions.newBuilder() + .auth(createAuthDescriptor([createAuthRequirement('NO_AUTH')])) + .build(); + + await runThrough(descriptor, transport, {options}); + + expect( + transport.calls[0]?.request.headers.get('X-Api-Key'), + ).toBeUndefined(); + }); + + test('a per-call descriptor that is unsatisfiable does NOT fall through to the client tier (AUTH-4)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: {credential: new ApiKeyCredential('secret')}, + }; + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + const options = RequestOptions.newBuilder() + .auth(createAuthDescriptor([createAuthRequirement('BASIC')])) + .build(); + + const error = await rejectionOf( + runThrough(descriptor, transport, {options}), + ); + + expect(error).toBeInstanceOf(AuthResolutionError); + expect(transport.sendCount).toBe(0); + }); +}); + +describe('authStep: the operation tier (AUTH-4, docs/work/mvp/2026-09-04-open-items-dissolution.md W1)', () => { + test('RequestOptions.operationAuth fills the operation tier and beats the client tier (AUTH-4, docs/work/mvp/2026-09-04-open-items-dissolution.md W1)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }; + // The client tier resolves to API_KEY; the operation tier demands NO_AUTH and must win, so no + // API-key header is stamped. + const descriptor = authStep({credentials, tiers: tiersFor('API_KEY')}); + const options = RequestOptions.newBuilder() + .operationAuth(createAuthDescriptor([createAuthRequirement('NO_AUTH')])) + .build(); + + await runThrough(descriptor, transport, {options}); + + expect( + transport.calls[0]?.request.headers.get('X-Api-Key'), + ).toBeUndefined(); + }); + + test('a per-call descriptor still beats an operation descriptor (AUTH-4 precedence)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const credentials: AuthCredentialSet = { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }; + // perCall ?? operation ?? client, with the operation tier unsatisfiable: if it were consulted + // the call would raise AuthResolutionError instead of stamping. + const descriptor = authStep({credentials, tiers: {}}); + const options = RequestOptions.newBuilder() + .auth(createAuthDescriptor([createAuthRequirement('API_KEY')])) + .operationAuth(createAuthDescriptor([createAuthRequirement('BASIC')])) + .build(); + + await runThrough(descriptor, transport, {options}); + + expect(transport.calls[0]?.request.headers.get('X-Api-Key')).toBe('secret'); + }); +}); + +describe('authStep: answering an unrecognized scheme through challengeHook', () => { + // There is deliberately no `AuthStepSettings.handlers`: `challengeHook` is the ONE caller-facing + // extension point, and it covers the case a handler list was reaching for -- a scheme none of the + // built-in handlers recognizes -- without putting `ChallengeHandler` on the public barrel where + // neither `basicHandler` nor `digestHandler` is reachable to compose with. + test('a challengeHook answers a scheme no built-in handler recognizes', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Custom realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('Authorization', 'Custom abc') + .build(), + ) + .build(), + ), + }); + + await runThrough(descriptor, transport); + + expect(transport.calls[1]?.request.headers.get('Authorization')).toBe( + 'Custom abc', + ); + }); +}); + +describe('authStep: the call signal', () => { + test('the call signal reaches the challenge hook (AUTH-30)', async () => { + // A hook is the sanctioned place for a custom OAuth2 refresh grant, i.e. network I/O on the + // request path, so it must be able to observe the caller's cancellation. + const transport = new FakeTransport([ + challengeResponse(401, 'WWW-Authenticate', 'Basic realm="x"').response, + countingResponse(200).response, + ]); + const controller = new AbortController(); + let observed: AbortSignal | undefined; + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: (_response, request, options) => { + observed = options?.signal; + return Promise.resolve(request); + }, + }); + const request = aRequest(); + + await new Cursor({ + steps: [descriptor], + transport, + request, + context: aRequestContext(request), + signal: controller.signal, + }).advance(); + + expect(observed).toBe(controller.signal); + }); + + // There is deliberately no "the provider is not given the call signal" test any more: after M6, + // `TokenProvider` is `() => Promise<BearerToken>` and has no parameter to populate, so the property + // is structural. A test for it would only be exercising the type checker. +}); + +describe('authStep: a failing release never masks the primary error (RECOV-12)', () => { + test("a rejecting close() keeps the HOOK's own error primary (AUTH-32)", async () => { + // `Response.close()` rethrows whatever cancelling the body raised, so a bare + // `await response.close(); throw error;` discarded the hook's failure and surfaced the teardown + // failure in its place -- the inversion RECOV-12 forbids, and the one 5b's `decideOrClose` + // already guards against with the same two helpers. + const hookFailure = new Error('hook exploded'); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: () => Promise.reject(hookFailure), + }); + + const error = await rejectionOf( + runThrough(descriptor, new FakeTransport([hostileChallenge()])), + ); + + const suppressed = error as SuppressedErrorLike; + expect(suppressed.error).toBe(hookFailure); + expect(suppressed.suppressed).toBe(CANCEL_FAILURE); + }); + + test('a rejecting close() keeps PlaintextCredentialError primary on the replay guard (AUTH-28)', async () => { + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + // A replacement that downgrades to http AND carries a credential: AUTH-28 must refuse it, and + // that refusal is what the caller has to be able to see. + challengeHook: () => + Promise.resolve( + Request.newBuilder() + .url('http://example.com/a') + .headers( + Headers.newBuilder().set('Authorization', 'Bearer t').build(), + ) + .build(), + ), + }); + + const error = await rejectionOf( + runThrough(descriptor, new FakeTransport([hostileChallenge()])), + ); + + const suppressed = error as SuppressedErrorLike; + expect(suppressed.error).toBeInstanceOf(PlaintextCredentialError); + expect(suppressed.suppressed).toBe(CANCEL_FAILURE); + }); +}); + +describe('authStep: a non-replayable body still evicts (AUTH-31 vs AUTH-36)', () => { + test('a revoked token is evicted even though the replay is skipped', async () => { + // AUTH-31 gates the REPLAY on replayability; AUTH-36's eviction is a separate sentence. An + // earlier shape skipped the whole hook for a one-shot body, which left the token the server had + // just rejected sitting in the cache -- and a token with no `expiresAt` (AUTH-10's "never locally + // expires") never aged out either, so a stream-only client re-sent the dead credential forever. + let issued = 0; + const credentials: AuthCredentialSet = { + bearer: { + provider: () => { + issued += 1; + return Promise.resolve(createBearerToken(`t${String(issued)}`)); + }, + }, + }; + const descriptor = authStep({credentials, tiers: tiersFor('OAUTH2')}); + + const first = challengeResponse( + 401, + 'WWW-Authenticate', + 'Bearer realm="x"', + ); + const firstDrive = await runThrough( + descriptor, + new FakeTransport([first.response]), + { + request: oneShotPost(), + }, + ); + const second = challengeResponse( + 401, + 'WWW-Authenticate', + 'Bearer realm="x"', + ); + const secondTransport = new FakeTransport([second.response]); + await runThrough(descriptor, secondTransport, {request: oneShotPost()}); + + // AUTH-31 still holds: the original is surfaced unchanged and NOT closed. + expect(firstDrive.status.code).toBe(401); + expect(first.cancelCount()).toBe(0); + // AUTH-36 now also holds: the second request carries a freshly fetched token, not the dead one. + expect(secondTransport.calls[0]?.request.headers.get('Authorization')).toBe( + 'Bearer t2', + ); + }); +}); + +describe('authStep: refresh-margin validation (AUTH-34/AUTH-35)', () => { + // `nowMs + marginMs > expiresAt` is false for a NaN margin, so BOTH the margin check and AUTH-35's + // no-margin check say "not expired" and the cache serves a dead token from the hot path forever. + // Same rule and wording 5a's `retrySettings()` and 5b's `redirectSettings()` apply. + test('rejects a non-finite bearerMarginMs', () => { + expect(() => + authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + bearerMarginMs: Number.NaN, + }), + ).toThrow('finite, non-negative duration'); + }); + + test('rejects a negative bearerMarginMs', () => { + expect(() => + authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + bearerMarginMs: -1, + }), + ).toThrow('finite, non-negative duration'); + }); + + test('rejects a non-finite per-credential marginMs', () => { + expect(() => + authStep({ + credentials: { + bearer: { + provider: () => Promise.resolve(createBearerToken('t')), + marginMs: Number.NaN, + }, + }, + tiers: tiersFor('OAUTH2'), + }), + ).toThrow('finite, non-negative duration'); + }); +}); + +describe('authStep: the bearer refresh margin, in effect (AUTH-34/AUTH-37)', () => { + // The margin was validated at construction but its EFFECT was unasserted: both + // `AuthStepSettings.bearerMarginMs`'s 30 s default and `BearerCredential.marginMs`'s override could + // be deleted outright and every test still passed. AUTH-34 names the 30 s default itself, and + // `marginMs` is public surface, so both need a test that fails when the number changes. + // The two tests below pin the default from BOTH sides, deliberately. A single "a token 20 s out + // gets refreshed" assertion is satisfied by any margin >= 20 s, so it cannot tell 30 s from 60 s; + // the pair brackets the boundary at exactly 30 000 ms. + test("a token expiring just INSIDE AUTH-34's 30 s default is refreshed in the background", async () => { + const aging = agingTokenProvider(29_999); + const descriptor = authStep({ + credentials: aging.credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + const stamped = await stampsOverThreeDrives(descriptor); + + // Drive 2 stamps the stale-but-valid t1 and kicks off the refresh; drive 3 sees t2 (AUTH-37). + expect(stamped).toEqual(['Bearer t1', 'Bearer t1', 'Bearer t2']); + expect(aging.callCount()).toBe(2); + }); + + test('a token expiring just OUTSIDE the 30 s default stays in the fresh zone', async () => { + const aging = agingTokenProvider(30_001); + const descriptor = authStep({ + credentials: aging.credentials, + tiers: tiersFor('OAUTH2'), + clock: {now: () => 0}, + }); + + const stamped = await stampsOverThreeDrives(descriptor); + + expect(stamped).toEqual(['Bearer t1', 'Bearer t1', 'Bearer t1']); + expect(aging.callCount()).toBe(1); + }); + + test('a per-credential marginMs overrides the step-wide one', async () => { + const aging = agingTokenProvider(29_999); + const bearer = aging.credentials.bearer; + invariant(bearer !== undefined, 'agingTokenProvider configures a bearer'); + const descriptor = authStep({ + credentials: {bearer: {...bearer, marginMs: 30_000}}, + tiers: tiersFor('OAUTH2'), + bearerMarginMs: 0, // the step-wide margin alone would leave t1 in the fresh zone forever + clock: {now: () => 0}, + }); + + const stamped = await stampsOverThreeDrives(descriptor); + + expect(stamped).toEqual(['Bearer t1', 'Bearer t1', 'Bearer t2']); + expect(aging.callCount()).toBe(2); + }); + + test('an explicit zero margin beats the default and suppresses the background refresh', async () => { + const aging = agingTokenProvider(29_999); // inside the default margin, outside a zero one + const descriptor = authStep({ + credentials: aging.credentials, + tiers: tiersFor('OAUTH2'), + bearerMarginMs: 0, + clock: {now: () => 0}, + }); + + const stamped = await stampsOverThreeDrives(descriptor); + + expect(stamped).toEqual(['Bearer t1', 'Bearer t1', 'Bearer t1']); + expect(aging.callCount()).toBe(1); + }); +}); + +describe('authStep: the replay HTTPS guard covers Proxy-Authorization too (AUTH-25/AUTH-28)', () => { + test('a replacement carrying only Proxy-Authorization over plaintext is refused', async () => { + // AUTH-28 says ANY path where a credential will be attached, and AUTH-25 makes + // `Proxy-Authorization` exactly such a path for a 407. The guard's `Authorization` arm was + // asserted and this one was not, so dropping it left a proxy credential able to go out over + // plaintext with the whole suite green. + const challenged = challengeResponse( + 407, + 'Proxy-Authenticate', + 'Basic realm="p"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), // outbound guard skipped entirely + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('Proxy-Authorization', 'Basic c3B5') + .build(), + ) + .build(), + ), + }); + + const error = await rejectionOf( + runThrough(descriptor, transport, { + request: aRequest('http://example.com/a'), + }), + ); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); // the replacement never reached the wire + expect(challenged.cancelCount()).toBe(1); // and the 407 was closed before the throw + }); +}); + +describe('authStep: a challenge this client cannot echo (AUTH-21/AUTH-22)', () => { + test('a non-ASCII Digest realm surfaces the 401 unchanged rather than throwing', async () => { + // HTTP-19 lets a received field-value carry obs-text, so `realm="café"` -- a real RFC 7616 shape, + // which is why the spec has a `charset` parameter at all -- reaches us intact. HTTP-18 will not + // let it back out. `parseDigestChallenge` declines, so AUTH-33 surfaces the 401 open and + // unchanged; building the header anyway threw HeaderValidationError out of the whole step. + const challenge = challengeResponse( + 401, + 'WWW-Authenticate', + 'Digest realm="café", nonce="n", algorithm=MD5, charset=UTF-8', + ); + const transport = new FakeTransport([challenge.response]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + const response = await runThrough(descriptor, transport); + + expect(response.status.code).toBe(401); + expect(transport.sendCount).toBe(1); // no replay + expect(challenge.cancelCount()).toBe(0); // AUTH-33: returned open, the caller's to close + }); +}); + +describe('authStep: cancellation (AUTH-30)', () => { + test('a call already aborted at entry runs no step, no hook, and no wire send (V15)', async () => { + // The default OAUTH2 hook does an IdP round trip and the BASIC/DIGEST one does key derivation. + // Neither is worth doing for a caller who has already gone, so the hook is not even built. + // + // Strengthened 2026-09-02: the cursor now refuses the walk at the first step boundary, so a + // pre-aborted call spends NO wire send either and rejects with `CancellationError` rather than + // handing back the 401. The auth step's OWN abort guard is what the next test covers -- an + // abort arriving during the hook, which the cursor never sees. + let hookRan = false; + const controller = new AbortController(); + controller.abort(); + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: (_response, request) => { + hookRan = true; + return Promise.resolve(request); + }, + }); + + const {CancellationError} = await import('../seams/transport.js'); + const error = await rejectionOf( + runThrough(descriptor, transport, {signal: controller.signal}), + ); + + expect(error).toBeInstanceOf(CancellationError); + expect(hookRan).toBe(false); + expect(transport.sendCount).toBe(0); + expect(challenged.cancelCount()).toBe(0); // never dispatched, so nothing to close + }); + + test('an abort arriving DURING the hook still spends no second wire send', async () => { + // `redirectStep` checks `signal?.aborted` before each hop and the retry engine before each + // attempt; the auth step must not be the one pillar that dispatches for a caller who has gone. + const controller = new AbortController(); + const transport = new FakeTransport([ + challengeResponse(401, 'WWW-Authenticate', 'Basic realm="x"').response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {}, + tiers: tiersFor('NO_AUTH'), + challengeHook: (_response, request) => { + controller.abort(); // the caller gives up while the hook is running + return Promise.resolve(request); + }, + }); + + const response = await runThrough(descriptor, transport, { + signal: controller.signal, + }); + + expect(response.status.code).toBe(401); // surfaced open, like every other no-replay outcome + expect(transport.sendCount).toBe(1); + }); +}); diff --git a/packages/core/src/auth/auth-step.ts b/packages/core/src/auth/auth-step.ts new file mode 100644 index 0000000..042eadf --- /dev/null +++ b/packages/core/src/auth/auth-step.ts @@ -0,0 +1,844 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/auth-step.ts +import type {Clock} from '../config/clock.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {assertNever, invariant} from '../invariant.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; +import { + clearCrossOriginMarker, + hasCrossOriginMarker, +} from '../redirect/cross-origin.js'; +import {basicHandler} from './basic.js'; +import {BearerTokenCache} from './bearer-cache.js'; +import {parseChallenges} from './challenge.js'; +import type {Challenge, ChallengeHandler} from './challenge.js'; +import {composingHandler, type ComposingHandler} from './composing-handler.js'; +import { + credentialPassword, + type ApiKeyCredential, + type BasicCredential, + type DigestCredential, + type NameKeyCredential, + type TokenProvider, +} from './credential.js'; +import type {AuthDescriptor} from './descriptor.js'; +import {digestHandler} from './digest.js'; +import {PlaintextCredentialError} from './errors.js'; +import {resolveAuthRequirement, type AuthTiers} from './resolve.js'; +import type {AuthScheme} from './scheme.js'; +import {stampStaticKey} from './static-key.js'; + +/** + * The token source and refresh margin for the `OAUTH2` scheme. + * + * @public + */ +export interface BearerCredential { + /** The token source (AUTH-11). */ + readonly provider: TokenProvider; + /** Per-credential refresh margin; falls back to {@link AuthStepSettings.bearerMarginMs}. */ + readonly marginMs?: number | undefined; +} + +/** + * The static key, header, and prefix for the `API_KEY` scheme (AUTH-26). + * + * @public + */ +export interface ApiKeyCredentialConfig { + /** The key. Both credential classes are nominal, so no object literal substitutes for one. */ + readonly credential: ApiKeyCredential | NameKeyCredential; + /** The header to write. Defaults to `Authorization`. */ + readonly headerName?: string | undefined; + /** A scheme prefix, written followed by exactly one space. */ + readonly prefix?: string | undefined; +} + +/** + * Which schemes a caller has actually configured a credential for. + * + * This shape is designed by this phase — neither the product spec nor the design doc names one. It is + * both the credential material the step stamps with, and the source `availableSchemesOf()` derives + * AUTH-5's `availableSchemes` from, which is what keeps resolution from ever inspecting a concrete + * credential value. + * + * @public + */ +export interface AuthCredentialSet { + /** Enables the `BASIC` scheme. */ + readonly basic?: BasicCredential | undefined; + /** Enables the `DIGEST` scheme. */ + readonly digest?: DigestCredential | undefined; + /** Enables the `OAUTH2` scheme. */ + readonly bearer?: BearerCredential | undefined; + /** Enables the `API_KEY` scheme. */ + readonly apiKey?: ApiKeyCredentialConfig | undefined; +} + +/** + * AUTH-5: derives the satisfiable-scheme set from which credentials are configured, without exposing + * any credential value to resolution. + * + * `NO_AUTH` is deliberately absent: AUTH-5 makes it satisfiable unconditionally, so membership here + * would be redundant and would let a caller's empty credential set read as "nothing is available" + * while a `NO_AUTH` requirement still resolves. + * + * @param credentials - the configured credential set. + * @returns the schemes with a matching credential. + * + * @internal + */ +export function availableSchemesOf( + credentials: AuthCredentialSet, +): ReadonlySet<AuthScheme> { + const schemes = new Set<AuthScheme>(); + if (credentials.basic !== undefined) schemes.add('BASIC'); + if (credentials.digest !== undefined) schemes.add('DIGEST'); + if (credentials.bearer !== undefined) schemes.add('OAUTH2'); + if (credentials.apiKey !== undefined) schemes.add('API_KEY'); + return schemes; +} + +/** + * The handler list is derived from `credentials`, digest-first — "callers order stronger schemes + * first" (AUTH-23). Both handlers need a username and password to do anything, so a zero-argument + * `[digestHandler(), basicHandler()]` default is not constructible, which is why this is derived + * rather than defaulted. + * + * There is deliberately no caller override. An earlier shape took `AuthStepSettings.handlers`, which + * forced `Challenge`/`ChallengeHandler`/`DigestUriContext` onto the public barrel to make the field + * callable — and then delivered less than it promised: `basicHandler`/`digestHandler` stay internal, + * so a caller supplying one handler silently LOST the credential-derived ones rather than composing + * with them. {@link AuthStepSettings.challengeHook} already covers the custom-scheme case end to end, + * with a shape a caller can actually satisfy. + */ +function buildHandlers( + credentials: AuthCredentialSet, +): readonly ChallengeHandler[] { + const handlers: ChallengeHandler[] = []; + // `credentialPassword()`, not a public `.password` property: AUTH-8's secret stays off the + // published surface and this function is the one sanctioned reader, exactly as `stampStaticKey` + // is for `credentialKey()`. + if (credentials.digest !== undefined) { + handlers.push( + digestHandler( + credentials.digest.username, + credentialPassword(credentials.digest), + {algorithmPreference: credentials.digest.algorithmPreference}, + ), + ); + } + if (credentials.basic !== undefined) { + handlers.push( + basicHandler( + credentials.basic.username, + credentialPassword(credentials.basic), + ), + ); + } + return handlers; +} + +/** + * AUTH-30's pluggable 401/407 reaction. + * + * Returning `undefined` means "no replacement" — the challenge response is surfaced unchanged. A + * returned request is driven exactly once through a fresh copy of the downstream chain, with no + * further challenge handling on that drive. + * + * @public + */ +export type ChallengeHook = ( + response: Response, + request: Request, + options?: { + /** + * The calling request's cancellation, threaded straight through from `StepContext.signal`. + * + * A hook is the sanctioned place to run a custom OAuth2 refresh-token grant, which is external + * I/O on the request path -- and `docs/knowledge/harvested/concurrency-and-async.md` is explicit that a + * signal accepted at the top of a call chain must reach the actual I/O primitive, or it is + * decoration. Without this a hung hook pinned the auth step, every retry attempt nested under + * it, and the whole request, with no way for the caller to abort. + */ + readonly signal?: AbortSignal | undefined; + }, +) => Promise<Request | undefined>; + +/** + * Everything {@link authStep} accepts. + * + * @public + */ +export interface AuthStepSettings { + /** Which schemes are available, and the material to stamp them with. */ + readonly credentials: AuthCredentialSet; + /** + * The operation and client tiers, fixed at construction. The `perCall` slot may additionally be + * supplied per call via `RequestOptions.auth` (AUTH-4), which wins over any `perCall` value + * configured here. + */ + readonly tiers: AuthTiers; + /** + * Replaces the scheme-dependent default 401/407 reaction entirely — e.g. a custom OAuth2 + * refresh-token grant (AUTH-30). + */ + readonly challengeHook?: ChallengeHook | undefined; + /** + * Refresh margin ahead of a bearer token's expiry. + * + * @defaultValue 30000 — AUTH-34's "default 30 seconds". + */ + readonly bearerMarginMs?: number | undefined; + /** + * Wall-clock source for bearer expiry evaluation, injected so the three-zone policy is testable + * through the step and not only through the cache directly. Reading `Date.now()` inside the cache + * would be a second, uncontrollable clock — `bearer-cache.ts` takes an injected `nowMs` precisely so + * its one caller can supply a controllable one, and this is that caller. + * + * Typed as the `now()` half of {@link Clock}, not a bare `() => number` and not the whole `Clock`: + * `RetryStepOptions.clock` is a full `Clock`, and one instance has to satisfy both slots or a + * caller who fakes time for retry and forgets auth gets two clocks disagreeing inside one pipeline. + * Narrowing to the member actually used means no caller has to implement `monotonic`/`sleep` for a + * step that never sleeps. + * + * @defaultValue a `now()` reading `Date.now()` + */ + readonly clock?: Pick<Clock, 'now'> | undefined; +} + +/** + * A refresh margin must be a finite, non-negative duration -- the same rule and the same wording + * 5a's `retrySettings()` and 5b's `redirectSettings()` apply to every numeric setting they take, and + * an invalid value is a PROGRAMMER error there and here alike, so it trips `invariant()` rather than + * a typed error leaf. + * + * Not decorative. `isBearerTokenExpired` is `nowMs + marginMs > expiresAt`, so a `NaN` margin -- the + * shape `Number(process.env.MARGIN_MS)` produces for an unset variable -- makes BOTH the margin + * comparison and AUTH-35's no-margin comparison false. The cache then reads a long-dead token as + * fresh, returns it from the hot path, and never calls the provider again: a revoked credential + * stamped onto every request, indefinitely and silently. A large negative margin does the same. + */ +function validateMarginMs(label: string, value: number | undefined): void { + if (value === undefined) return; + invariant( + Number.isFinite(value) && value >= 0, + `${label} must be a finite, non-negative duration, got ${String(value)}`, + ); +} + +/** + * AUTH-4: the two per-call slots (`RequestOptions.auth` and `RequestOptions.operationAuth`, both via + * `StepContext.options`) fill the `perCall` and `operation` tiers. Each is applied only when present, + * so a configured tier is never overwritten with `undefined` — `{...configured, perCall: undefined}` + * would erase a `perCall` the step was constructed with (docs/work/mvp/2026-09-04-open-items-dissolution.md W1). + */ +function effectiveTiers( + configured: AuthTiers, + perCall: AuthDescriptor | undefined, + operation: AuthDescriptor | undefined, +): AuthTiers { + const withPerCall = + perCall === undefined ? configured : {...configured, perCall}; + return operation === undefined ? withPerCall : {...withPerCall, operation}; +} + +/** Stable identity for pillar-slot occupancy and anchor matching (PIPE-6/PIPE-18). @internal */ +export const AUTH_STEP_TYPE: unique symbol = Symbol('dexpace.auth'); + +/** AUTH-28: case-insensitive, evaluated before any token fetch or header write. */ +function requireHttps(url: URL, scheme: AuthScheme): void { + if (url.protocol.toLowerCase() !== 'https:') { + throw new PlaintextCredentialError('authStep', scheme); + } +} + +interface StampContext { + readonly scheme: AuthScheme; + readonly credentials: AuthCredentialSet; + readonly bearerCache: BearerTokenCache; + readonly marginMs: number; + readonly nowMs: number; + readonly signal: AbortSignal | undefined; +} + +function withHeader(request: Request, name: string, value: string): Request { + return request + .newBuilder() + .headers(request.headers.newBuilder().set(name, value).build()) + .build(); +} + +/** + * Whether the caller has given up. + * + * A function, not two inline `signal?.aborted === true` tests, and the indirection is load-bearing: + * `AbortSignal.aborted` is a LIVE getter that flips while an `await` is outstanding, but TypeScript + * narrows it like an ordinary property and carries that narrowing straight across the await. The + * second check in {@link handleChallenge} -- the one that exists precisely because the world moved + * during the hook -- therefore reads as `'false | undefined' and 'true' have no overlap` and fails to + * compile, which is the compiler being confidently wrong about mutable external state. Routing every + * read through a call re-reads the getter each time. + * + * `docs/knowledge/harvested/concurrency-and-async.md`: "state checked before an `await` must be re-validated + * after every `await` that could have let the world move." + */ +function isAborted(signal: AbortSignal | undefined): boolean { + return signal?.aborted === true; +} + +/** AUTH-25: which challenge header carried the offer decides which header the answer goes into. */ +function answerHeaderName(isProxy: boolean): string { + return isProxy ? 'Proxy-Authorization' : 'Authorization'; +} + +/** + * `OAUTH2` and `API_KEY` stamp preemptively — no server round-trip is needed to know what to send. + * `BASIC`, `DIGEST`, and `NO_AUTH` never do: Digest structurally cannot stamp before seeing the + * server's `realm`/`nonce`, AUTH-14/AUTH-23–AUTH-25 phrase Basic entirely in terms of answering a + * parsed challenge, and `NO_AUTH` has nothing to stamp. + * + * An exhaustive `switch` closing on `assertNever`, not an if-chain: `AuthScheme` is a closed + * discriminant, and `docs/knowledge/harvested/data-modeling.md` bars an if-chain over one because it gives no + * exhaustiveness guarantee and falls through silently when a variant is added — and the value that + * would fall through here is a credential-stamping decision. + */ +async function preemptiveStamp( + request: Request, + context: StampContext, +): Promise<Request> { + switch (context.scheme) { + case 'OAUTH2': { + const bearer = context.credentials.bearer; + invariant( + bearer !== undefined, + 'resolved OAUTH2 but no bearer credential configured', + ); + const token = await context.bearerCache.stamp({ + provider: bearer.provider, + marginMs: bearer.marginMs ?? context.marginMs, + nowMs: context.nowMs, + signal: context.signal, + }); + return withHeader(request, 'Authorization', `Bearer ${token.token}`); + } + case 'API_KEY': { + const apiKey = context.credentials.apiKey; + invariant( + apiKey !== undefined, + 'resolved API_KEY but no apiKey credential configured', + ); + const {headerName, headerValue} = stampStaticKey( + apiKey.credential, + apiKey, + ); + return withHeader(request, headerName, headerValue); + } + case 'BASIC': + case 'DIGEST': + case 'NO_AUTH': + return request; // challenge-driven, or no credential at all + default: + return assertNever(context.scheme); + } +} + +/** + * What the outbound pass decided, carried into the challenge pass. + * + * `crossOrigin` has to survive past the dispatch: AUTH-29's suppression covers the WHOLE hop, so the + * challenge reaction needs the same answer the outbound pass computed, and the marker itself is gone + * from the request by then. + */ +interface OutboundPlan { + readonly crossOrigin: boolean; + /** + * Whether the outbound pass ran {@link requireHttps} on this hop -- i.e. whether this is one of + * AUTH-28's "paths where a credential will be attached". Carried past the dispatch because the + * replay inherits it: see {@link guardReplayScheme}. + */ + readonly guarded: boolean; + readonly outbound: Request; +} + +/** + * AUTH-29 then AUTH-28, in that order. + * + * The cross-origin check comes FIRST, and the marker is cleared unconditionally before either branch + * — it must never reach the wire, and clearing up front means it cannot survive into a request built + * by the stamping logic. A marked hop then skips the HTTPS guard AND preemptive stamping entirely, + * forwarding the cleared request credential-free; AUTH-29 makes that skip deliberate, so a + * server-chosen downgrade hop is forwarded rather than hard-failing. + * + * On an unmarked hop the HTTPS guard runs only where a credential will actually be attached — + * `NO_AUTH` is exempt, matching AUTH-28's own qualifier — and before any token fetch or header write. + */ +async function planOutbound( + seedRequest: Request, + context: StampContext, +): Promise<OutboundPlan> { + const crossOrigin = hasCrossOriginMarker(seedRequest.headers); + const cleared = seedRequest + .newBuilder() + .headers(clearCrossOriginMarker(seedRequest.headers)) + .build(); + + if (crossOrigin) return {crossOrigin, guarded: false, outbound: cleared}; + const guarded = context.scheme !== 'NO_AUTH'; + if (guarded) requireHttps(cleared.url, context.scheme); + return { + crossOrigin, + guarded, + outbound: await preemptiveStamp(cleared, context), + }; +} + +interface ChallengeSelection { + /** Every value the matching challenge header carried, in wire order. */ + readonly values: readonly string[]; + readonly isProxy: boolean; +} + +/** + * AUTH-25: a 401 is answered from `WWW-Authenticate`, a 407 from `Proxy-Authenticate`. Reading only + * the header that matches the status keeps the pairing honest — a 401 carrying a stray + * `Proxy-Authenticate` must not produce a `Proxy-Authorization`, and vice versa. + * + * `getAll`, not `get`. RFC 9110 §5.3 lets a server send a list-valued field either comma-joined into + * one line or repeated across several, and RFC 7616 §3.3 recommends the repeated form for Digest + * algorithm discovery — one challenge per algorithm, strongest first. `get()` returns the FIRST value + * only, so a 401 offering `algorithm=SHA-512-256` (unsupported here) on line one and + * `algorithm=SHA-256, qop="auth"` on line two ended with no `Authorization` at all, while the same two + * challenges comma-joined authenticated (audit #67 / #74). Which shape reaches this step is the + * transport's accident, not the server's intent: `@dexpace/transport-fetch` joins repeated values, + * `@dexpace/transport-undici` keeps them apart, and both are legal. + */ +function pickChallengeHeader( + response: Response, +): ChallengeSelection | undefined { + const isProxy = response.status.code !== 401; + const values = response.headers.getAll( + isProxy ? 'Proxy-Authenticate' : 'WWW-Authenticate', + ); + return values.length === 0 ? undefined : {values, isProxy}; +} + +/** + * Every challenge the selected header offered, in wire order (AUTH-12). + * + * Each value is parsed on its OWN, and the lists are concatenated — never joined into one string + * first. `parseChallenges` is total (AUTH-13), but its recovery is bounded by the string it is handed: + * an unterminated quoted string terminates at end-of-input, so joining lets a malformed earlier value + * swallow a satisfiable later one whole. Parsing per value keeps the blast radius of a broken header + * line inside that line. + */ +function challengesOf(selection: ChallengeSelection): readonly Challenge[] { + return selection.values.flatMap(value => parseChallenges(value)); +} + +interface DefaultHookContext { + readonly scheme: AuthScheme; + readonly credentials: AuthCredentialSet; + readonly bearerCache: BearerTokenCache; + readonly composing: ComposingHandler; + readonly marginMs: number; + readonly nowMs: number; + readonly signal: AbortSignal | undefined; +} + +/** AUTH-36: evict the exact rejected token, fetch a genuinely fresh one, re-stamp once. */ +async function oauth2ChallengeHook( + request: Request, + selection: ChallengeSelection, + context: DefaultHookContext, +): Promise<Request | undefined> { + const bearer = context.credentials.bearer; + invariant( + bearer !== undefined, + 'resolved OAUTH2 but no bearer credential configured', + ); + const headerName = answerHeaderName(selection.isProxy); + const rejected = request.headers.get(headerName); + // AUTH-36: no Authorization on the rejected request -> surface the challenge unchanged. + if (rejected === undefined) return undefined; + const challenges: readonly Challenge[] = challengesOf(selection); + if (!challenges.some(challenge => challenge.scheme === 'bearer')) { + return undefined; // AUTH-36: the response advertises no Bearer challenge + } + + // AUTH-36's preservation clause, made observable: `evict()` clears the cache only when the cached + // token IS the rejected one, and hands back the survivor otherwise. A survivor means another + // request already refreshed past this 401, so the retry stamps THAT rather than burning a second + // provider fetch to arrive at the same place. + const preserved = context.bearerCache.evict(rejected); + if (preserved !== undefined) { + return withHeader(request, headerName, `Bearer ${preserved.token}`); + } + + // AUTH-37's post-eviction clause: a fetch that STARTED after this 401, so the retry cannot re-send + // the rejected token. Plain `stamp()` would coalesce onto a fetch that may have started before this + // 401 came back, and AUTH-11 permits a provider that caches internally, so that fetch can resolve + // to exactly the token the server just rejected. `refreshPostEviction` still coalesces concurrent + // 401s onto one fetch (AUTH-34) -- it supersedes pre-401 fetches only. + // + // The margin is INERT on this path and is passed anyway: `refresh()` validates the fetched token + // against a zero margin (AUTH-35) and never reads `BearerFetch.marginMs`, which only `stamp()` + // consults. It is resolved identically to the preemptive path rather than hard-coded, so the two + // call sites cannot drift apart if `refresh()` ever grows a margin-dependent branch -- and so a + // reader comparing them does not have to work out which of two spellings is the intended one. + const token = await context.bearerCache.refreshPostEviction({ + provider: bearer.provider, + marginMs: bearer.marginMs ?? context.marginMs, + nowMs: context.nowMs, + signal: context.signal, + }); + return withHeader(request, headerName, `Bearer ${token.token}`); +} + +/** + * AUTH-23–AUTH-25: delegate to the composing handler; no replacement when nothing is satisfiable. + * + * `selection.isProxy` reaches {@link answerHeaderName} and nothing else. The handlers produce the + * header VALUE only, and neither of them varies it by proxy-ness, so the flag stops here rather than + * being threaded into a contract that cannot use it. + */ +async function basicDigestChallengeHook( + request: Request, + selection: ChallengeSelection, + context: DefaultHookContext, +): Promise<Request | undefined> { + const challenges = challengesOf(selection); + const url = request.url; // HTTP-5: a fresh URL per access, so read it once. + const requestTarget = `${url.pathname}${url.search}`; + const value = await context.composing.stamp(challenges, { + method: request.method, + requestTarget, + }); + if (value === undefined) return undefined; + return withHeader(request, answerHeaderName(selection.isProxy), value); +} + +/** + * The scheme-dependent default hook body. AUTH-30's generic contract governs INVOCATION; this decides + * what each resolved scheme does with a parsed challenge. `API_KEY`/`NO_AUTH` never react — static or + * absent credentials have no reactive behavior, which is exactly AUTH-30's "the default hook yields no + * replacement". + * + * Exhaustive `switch` + `assertNever`, not an if-chain, for the same reason as `preemptiveStamp`: a + * sixth `AuthScheme` added later must not silently inherit the BASIC/DIGEST branch's stamping. + */ +async function defaultChallengeHook( + response: Response, + request: Request, + context: DefaultHookContext, +): Promise<Request | undefined> { + const selection = pickChallengeHeader(response); + if (selection === undefined) return undefined; + + switch (context.scheme) { + case 'OAUTH2': + return oauth2ChallengeHook(request, selection, context); + case 'BASIC': + case 'DIGEST': + return basicDigestChallengeHook(request, selection, context); + case 'API_KEY': + case 'NO_AUTH': + return undefined; + default: + return assertNever(context.scheme); + } +} + +/** What {@link guardReplayScheme} needs. Bundled to stay inside `max-params`. */ +interface ReplayGuardInput { + readonly replacement: Request; + readonly response: Response; + readonly scheme: AuthScheme; + /** {@link OutboundPlan.guarded} for this hop. */ + readonly outboundGuarded: boolean; +} + +/** + * AUTH-28 on the REPLAY path. The outbound guard is not sufficient here: it is skipped entirely for + * `NO_AUTH`, and nothing constrains a caller-supplied hook to preserve the request URL. A replay + * carrying a credential is by definition "a path where a credential will be attached", and AUTH-28 + * says ANY such path. + * + * **Once guarded, always guarded.** When the outbound pass ran the guard, so does the replay -- + * unconditionally, without inspecting a single header name. The rule used to be "the replacement + * carries `Authorization` or `Proxy-Authorization`", and that missed the case this step creates + * itself: `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, so a hook + * answering a 401 with `X-Api-Key: SECRET` over a downgraded `http://` URL went out in clear text + * with no `PlaintextCredentialError` (audit #67 / #71). Deriving the credential-carrying names from + * configuration instead was considered and rejected: a `challengeHook` may invent a carrier this step + * has never been told about, so no enumeration can be complete, whereas "this hop is credentialed" + * is a fact the outbound pass already decided. + * + * The header test survives as a SECOND arm rather than being replaced, because it still reaches + * somewhere the first cannot: a `NO_AUTH` hop is never guarded outbound, and a hook that answers its + * challenge with an `Authorization` header is attaching a credential all the same. + * + * The challenge response is closed before the throw, for the same reason AUTH-32 closes it on a hook + * throw: this is past the point where the caller still owns it, so propagating unclosed leaks the body. + */ +async function guardReplayScheme(input: ReplayGuardInput): Promise<void> { + const {replacement, response, scheme, outboundGuarded} = input; + const attachesCredential = + outboundGuarded || + replacement.headers.has('Authorization') || + replacement.headers.has('Proxy-Authorization'); + if (!attachesCredential) return; + try { + requireHttps(replacement.url, scheme); + } catch (error) { + // The GUARD's error stays primary. `Response.close()` rethrows whatever cancelling the body + // raised, so a bare `await response.close()` here replaced `PlaintextCredentialError` -- typed, + // caller-catchable, security-relevant -- with the teardown failure, the inversion RECOV-12 + // forbids. Same helpers 4b built and 5b's `decideOrClose` uses. + throw withReleaseFailure(error, await releaseQuietly(response)); + } +} + +/** What {@link runHook} needs besides the hook itself. Bundled to stay inside `max-params`. */ +interface HookInvocation { + readonly response: Response; + readonly request: Request; + readonly signal: AbortSignal | undefined; +} + +/** + * AUTH-32: a hook that throws, or whose promise rejects, closes the open challenge response before the + * error propagates. + * + * The HOOK's error stays primary. `Response.close()` rethrows whatever cancelling the body raised, so + * a bare `await response.close()` here discarded the hook's own failure and surfaced the teardown + * failure in its place -- RECOV-12's "attaching any close error as suppressed so it never masks the + * primary", inverted. `releaseQuietly`/`withReleaseFailure` are 4b's helpers, shared with the retry + * engine and 5b's `decideOrClose`. + */ +async function runHook( + hook: ChallengeHook, + invocation: HookInvocation, +): Promise<Request | undefined> { + const {response, request, signal} = invocation; + try { + return await hook(response, request, {signal}); + } catch (error) { + throw withReleaseFailure(error, await releaseQuietly(response)); + } +} + +interface ChallengeDrive { + readonly response: Response; + readonly outbound: Request; + /** {@link OutboundPlan.guarded}, carried in for {@link guardReplayScheme}. */ + readonly outboundGuarded: boolean; + readonly fork: () => (request?: Request) => Promise<Response>; + readonly settings: AuthStepSettings; + readonly hookContext: DefaultHookContext; +} + +/** + * AUTH-30–AUTH-33: the 401/407 reaction, split out of the pillar closure to keep both under the + * 70-line cap and to give the response-lifecycle rules one place to live. + * + * The challenge response is returned OPEN — the caller's to close — on every no-replay outcome (no + * matching challenge header, a one-shot body, a hook yielding nothing, a non-replayable replacement). + * It is CLOSED before the replay dispatch, and before propagating a hook throw or a replay-path guard + * failure. + */ +async function handleChallenge(drive: ChallengeDrive): Promise<Response> { + const {response, outbound, outboundGuarded, fork, settings, hookContext} = + drive; + + const selection = pickChallengeHeader(response); + // AUTH-33: no matching challenge header -> unchanged, and the hook is never consulted. + if (selection === undefined) return response; + + // There is deliberately NO "skip the hook when the body is one-shot" fast path here. An earlier + // shape had one, on the reasoning that the default hook would only fetch a replacement that is + // then thrown away -- which is wrong on inspection: OAUTH2's hook EVICTS the rejected token and + // populates the cache for every subsequent request, so the work is not wasted. Skipping it left a + // server-revoked token cached behind every non-replayable request, and a token with no `expiresAt` + // (AUTH-10's "never locally expires") never aged out either, so a stream-only client re-sent the + // dead credential forever. AUTH-36's eviction clause and AUTH-31's replay gate are separate + // sentences; only the DISPATCH below is gated. + + // The caller had already abandoned this call before the challenge even arrived. The default OAUTH2 + // hook does an IdP round trip and the BASIC/DIGEST one does key derivation, so running either here + // is pure waste -- and `redirectStep` makes the same call, returning the current response open + // rather than doing more work. The signal threaded into `runHook` below covers the other case: an + // abort arriving while a hook is already in flight. + if (isAborted(hookContext.signal)) return response; + + const hook: ChallengeHook = + settings.challengeHook ?? + ((res, req) => defaultChallengeHook(res, req, hookContext)); + + const replacement = await runHook(hook, { + response, + request: outbound, + signal: hookContext.signal, + }); + // AUTH-33: the hook yielded nothing -> the challenge response is surfaced unchanged and unclosed. + if (replacement === undefined) return response; + + // AUTH-31, applied uniformly: a non-replayable replacement body skips the replay, surfaces the + // original unchanged, and MUST NOT close it -- the caller owns it. The reference applies this gate on + // its sync step only and recommends a port extend it; with one unified step there is exactly one + // place to apply it, so it covers OAUTH2's evict-and-retry too. + if (replacement.body !== undefined && !replacement.body.replayable) { + return response; + } + + // The caller gave up WHILE the hook ran -- the pre-hook check above cannot see this one. Surfacing + // the challenge open and unclosed is the same answer every other no-replay outcome gives; spending + // a second wire send on a request nobody is waiting for is the one thing that must not happen. + if (isAborted(hookContext.signal)) return response; + + // AUTH-28 + await guardReplayScheme({ + replacement, + response, + scheme: hookContext.scheme, + outboundGuarded, + }); + + await response.close(); // AUTH-30: the original is closed before the replacement is driven. + // AUTH-30: exactly once, through a FRESH chain copy, with no further challenge handling on it. + return fork()(replacement); +} + +/** + * The single AUTH pillar step (AUTH-27–AUTH-33). + * + * One pluggable challenge-reaction extension point ({@link AuthStepSettings.challengeHook}) with a + * scheme-dependent default body — not three competing mechanisms. AUTH-30's contract (consult the + * hook, close the original on a non-null replacement, re-drive once through a fresh chain copy, no + * nested re-challenge) governs every scheme uniformly; AUTH-23–AUTH-26 and AUTH-34–AUTH-37 describe + * what the DEFAULT hook does for each resolved scheme. + * + * `stage: 'AUTH'` is baked into the descriptor this factory returns, which is how PIPE-36 is satisfied + * structurally. `ctx.fork` is asserted rather than checked — AUTH is in `PILLAR_STAGES`, so its + * absence means the descriptor was installed somewhere it cannot be, a programmer error. Every + * dispatch, INCLUDING the first, goes through a fresh `ctx.fork()` rather than `ctx.next()`, since a + * challenge may drive the chain a second time and `next()`'s single-invocation guard would trip + * (PIPE-15). + * + * Nested inside both redirect (5b) and retry (5a) per AUTH-27's "redirect wraps retry wraps auth", so + * it re-resolves and re-stamps per redirect hop and per retry attempt (PIPE-2). + * + * Both challenge statuses are handled: a 401 is answered from `WWW-Authenticate` into `Authorization`, + * a 407 from `Proxy-Authenticate` into `Proxy-Authorization` (AUTH-25). A cross-origin-marked hop + * answers neither (AUTH-29). + * + * AUTH-38 is satisfied structurally: `fn` is `async`, so the HTTPS-guard failure and any hook error + * reach the caller as a rejected promise rather than a synchronous throw. + * + * @param settings - credentials, tiers, and the optional challenge hook and clock overrides. + * @returns the descriptor to install in a pipeline's AUTH slot. + * @throws PlaintextCredentialError — as a rejected promise — when the resolved scheme would attach a + * credential over a non-HTTPS URL (AUTH-28), on the outbound pass and again on a challenge replay. + * A replay whose hop was guarded outbound is guarded again whatever URL and headers the hook chose, + * so a hook that downgrades the scheme fails here rather than on the wire. + * Recover by fixing the endpoint's scheme; retrying will not help. + * @throws AuthResolutionError — as a rejected promise — when the selected tier lists no scheme with a + * matching configured credential (AUTH-6; AUTH-4 governs only WHICH tier is selected), or when the + * token provider returns a null or already-expired token (AUTH-35). The first is a configuration + * fault; the second is transient and the next request retries the fetch. + * @throws HeaderValidationError — as a rejected promise — when the credential material will not fit in + * a header value: a `TokenProvider` yielding a token with a control character passes AUTH-9's + * non-blank check but fails HTTP-18's outbound grammar at the write. + * @throws an assertion failure (a caller bug, not a catchable condition) — synchronously from this factory when `bearerMarginMs` or + * `BearerCredential.marginMs` is not a finite, non-negative duration, or a configured Digest/Basic + * credential is blank or not header-safe; and as a rejected promise from `send()` when no auth tier + * is configured at all (AUTH-6). All are caller misconfigurations, not operational failures. + * @throws Anything a caller-supplied `TokenProvider` or `challengeHook` raises, unwrapped and + * unconverted — the same pass-through stance the redirect step takes for its `predicate`. + * + * @example + * ```ts + * const runtime = new PipelineBuilder(transport) + * .append(authStep({ + * credentials: {bearer: {provider: () => fetchToken({signal: AbortSignal.timeout(5_000)})}}, + * tiers: {client: createAuthDescriptor([createAuthRequirement('OAUTH2')])}, + * })) + * .build(); + * ``` + * + * @public + */ +export function authStep(settings: AuthStepSettings): StepDescriptor { + // Built ONCE per installed step, not per request. The bearer cache is the one piece of shared + // mutable state, and sharing it across calls is the point -- AUTH-34's single-flight coalescing only + // works if concurrent calls meet at the same instance. + const bearerCache = new BearerTokenCache(); + const availableSchemes = availableSchemesOf(settings.credentials); + const composing = composingHandler(buildHandlers(settings.credentials)); + validateMarginMs('authStep bearerMarginMs', settings.bearerMarginMs); + validateMarginMs( + 'BearerCredential marginMs', + settings.credentials.bearer?.marginMs, + ); + const bearerMarginMs = settings.bearerMarginMs ?? 30_000; + const readNow = settings.clock?.now.bind(settings.clock) ?? Date.now; + + return { + type: AUTH_STEP_TYPE, + stage: 'AUTH', + fn: async (seedRequest, ctx) => { + const {fork, signal} = ctx; + invariant( + fork !== undefined, + 'authStep must occupy the AUTH pillar stage', + ); + + const {scheme} = resolveAuthRequirement( + effectiveTiers( + settings.tiers, + ctx.options?.auth, + ctx.options?.operationAuth, + ), + availableSchemes, + ); + // One clock read per hop, threaded into every expiry evaluation this hop performs, so the + // preemptive stamp and a challenge-driven refresh cannot disagree about "now" mid-call. + const nowMs = readNow(); + const stampContext: StampContext = { + scheme, + credentials: settings.credentials, + bearerCache, + marginMs: bearerMarginMs, + nowMs, + signal, + }; + + const {crossOrigin, guarded, outbound} = await planOutbound( + seedRequest, + stampContext, + ); + + const response = await fork()(outbound); + const status = response.status.code; + if (status !== 401 && status !== 407) return response; + + // AUTH-29, second half: the marker suppresses stamping for the WHOLE hop, not just the outbound + // pass. Answering a challenge here would stamp exactly the credential `planOutbound` declined + // to send -- onto the server-chosen foreign host, over a URL whose HTTPS guard was deliberately + // skipped. The challenge is the caller's to handle, so the response is returned untouched and + // unclosed. + if (crossOrigin) return response; + + return handleChallenge({ + response, + outbound, + outboundGuarded: guarded, + fork, + settings, + hookContext: {...stampContext, composing}, + }); + }, + }; +} diff --git a/packages/core/src/auth/basic.test.ts b/packages/core/src/auth/basic.test.ts new file mode 100644 index 0000000..d2c9512 --- /dev/null +++ b/packages/core/src/auth/basic.test.ts @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/basic.test.ts +// Exercises: AUTH-14 ('Basic ' + base64(UTF-8(username:password)), computed once; accepts a basic +// challenge case-insensitively; whitespace-only credentials are PERMITTED -- RFC 7617's laxer rule, +// deliberately different from the credential types' stricter non-blank check in credential.ts), +// AUTH-25 (the handler returns the header VALUE only, never picks the header name). +import {describe, expect, test} from 'bun:test'; +import {InvariantViolation} from '../invariant.js'; +import {basicHandler} from './basic.js'; + +const basicChallenge = {scheme: 'basic', params: new Map<string, string>()}; + +describe('basicHandler', () => { + test('produces "Basic " + base64(UTF-8(username:password))', async () => { + const handler = basicHandler('Aladdin', 'open sesame'); + const value = await handler.stamp(basicChallenge); + expect(value).toBe('Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='); + }); + + test('handles non-ASCII credentials via UTF-8 encoding', async () => { + const handler = basicHandler('üser', 'päss'); + const value = await handler.stamp(basicChallenge); + expect(value.startsWith('Basic ')).toBe(true); + // A naive Latin-1 btoa would produce a different, wrong encoding. + expect(value).toBe( + `Basic ${btoa( + String.fromCharCode(...new TextEncoder().encode('üser:päss')), + )}`, + ); + }); + + test('canHandle accepts "basic" (parseChallenges already lower-cases the scheme)', () => { + const handler = basicHandler('u', 'p'); + expect(handler.canHandle(basicChallenge)).toBe(true); + expect(handler.canHandle({scheme: 'digest', params: new Map()})).toBe( + false, + ); + }); + + test('whitespace-only credentials are permitted (RFC 7617, laxer than credential.ts)', () => { + expect(() => basicHandler(' ', ' ')).not.toThrow(); + }); + + test('a truly empty username or password is rejected', () => { + expect(() => basicHandler('', 'p')).toThrow(InvariantViolation); + expect(() => basicHandler('u', '')).toThrow(InvariantViolation); + }); + + test('the encoded value is computed once, at construction', async () => { + const handler = basicHandler('u', 'p'); + const first = await handler.stamp(basicChallenge); + const second = await handler.stamp(basicChallenge); + expect(first).toBe(second); + }); + + test('declares no rank -- it has no algorithm variants to prefer among (AUTH-16)', () => { + // `'rank' in handler`, not `handler.rank` -- reading an unbound method off an object literal + // trips `@typescript-eslint/unbound-method`, and presence is what the assertion is about anyway. + expect('rank' in basicHandler('u', 'p')).toBe(false); + }); +}); diff --git a/packages/core/src/auth/basic.ts b/packages/core/src/auth/basic.ts new file mode 100644 index 0000000..c16d949 --- /dev/null +++ b/packages/core/src/auth/basic.ts @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/basic.ts +import {invariant} from '../invariant.js'; +import type {Challenge, ChallengeHandler} from './challenge.js'; + +/** + * `btoa` is Latin-1: it throws on any code point above U+00FF and mis-encodes the rest. Encoding to + * UTF-8 bytes first and handing `btoa` one character per byte is what makes a non-ASCII password + * base64 to the bytes RFC 7617 specifies. `globalThis.btoa` is used rather than `node:buffer` to keep + * the package portable (SEAM-1, `sdk-design-nodejs/06`). + */ +function toBase64Utf8(text: string): string { + const bytes = new TextEncoder().encode(text); + let binary = ''; + for (const byte of bytes) binary += String.fromCharCode(byte); + return btoa(binary); +} + +/** + * The Basic challenge handler (AUTH-14). + * + * The header value is `Basic ` plus base64 of the UTF-8 encoding of `username:password`, computed + * ONCE at construction and closed over — "computed once" is AUTH-14's own wording, not a performance + * nicety. + * + * Credentials are validated as non-empty but whitespace IS permitted, per RFC 7617's laxer rule. + * This is deliberately NOT the stricter `.trim().length > 0` check `credential.ts`'s types apply: a + * caller intentionally using a whitespace-only password is unusual but RFC 7617-legal, and rejecting + * it here would be this port inventing a restriction the requirement declines to make. + * + * Challenge-reactive only, never preemptive: `authStep()` engages this handler on a 401/407, never on + * the outbound pass. See `auth-step.ts` for that reading of AUTH-14/AUTH-23–AUTH-25. + * + * @param username - the user id. Must be non-empty; whitespace permitted. + * @param password - the password. Must be non-empty; whitespace permitted. + * @returns a stateless handler that answers `basic` challenges. + * @throws InvariantViolation when either credential is empty — a caller misconfiguration. + * + * @internal + */ +export function basicHandler( + username: string, + password: string, +): ChallengeHandler { + invariant(username.length > 0, 'Basic username must not be empty'); + invariant(password.length > 0, 'Basic password must not be empty'); + const value = `Basic ${toBase64Utf8(`${username}:${password}`)}`; + + return { + // `challenge.scheme` arrives lower-cased from `parseChallenges`, which is where AUTH-14's + // case-insensitivity is actually implemented. + canHandle: (challenge: Challenge): boolean => challenge.scheme === 'basic', + // Zero parameters, and that is the whole contract: the value was computed at construction, so + // neither the challenge nor the request-target can change it. AUTH-25's + // Authorization/Proxy-Authorization choice is the caller's, made from which challenge header the + // status carried. + stamp: (): Promise<string> => Promise.resolve(value), + }; +} diff --git a/packages/core/src/auth/bearer-cache.test.ts b/packages/core/src/auth/bearer-cache.test.ts new file mode 100644 index 0000000..f6f8b92 --- /dev/null +++ b/packages/core/src/auth/bearer-cache.test.ts @@ -0,0 +1,686 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/bearer-cache.test.ts +// Exercises: XCUT-12 (hot-path credential-cache reads take no lock while valid, and refresh is +// single-flight under a lock scoped to THIS cache -- the concurrent-coalescing tests below are the +// SHOULD's conformance clause: "race N threads on an expiring token; assert exactly one fetch"), +// AUTH-34 (fresh-zone hot-path read, no refresh), AUTH-35 (null/expired provider result +// throws and is never cached; a rejecting provider propagates and is never cached), AUTH-37 +// (expiring-but-valid zone: stale value returned, background refresh fired, a FAILED background +// refresh non-fatal and not an unhandled rejection; expired/missing zone: single-flight await, +// concurrent callers coalesce to exactly one provider invocation; the post-eviction path +// (`refreshPostEviction`) fetches genuinely fresh, while concurrent post-eviction refreshes still +// coalesce onto ONE fetch), AUTH-36 +// (eviction matched on the stamped header value; the survivor is returned so the preservation clause +// is observable), AUTH-11 (a provider error propagates through the async channel and is never +// cached), AUTH-38 (a provider that fails SYNCHRONOUSLY still reaches the async channel, so a +// background refresh stays non-fatal and refreshPostEviction never throws synchronously), AUTH-34's +// cancellation shape (the shared fetch carries no caller signal; each caller races its own, and a +// long-lived signal reused across many fetches does not accumulate abort listeners). +// +// Every `nowMs` below is injected, and the cache validates fetched tokens against that SAME injected +// clock -- so `expiresAt` values are small synthetic epochs, not wall-clock instants. A cache that +// reached for `Date.now()` internally would reject every one of these tokens. +import {describe, expect, test} from 'bun:test'; +import {BearerTokenCache, type BearerFetch} from './bearer-cache.js'; +import { + createBearerToken, + type BearerToken, + type TokenProvider, +} from './credential.js'; +import {AuthResolutionError} from './errors.js'; + +function providerReturning(token: ReturnType<typeof createBearerToken>): { + provider: TokenProvider; + callCount: () => number; +} { + let invocations = 0; + const provider: TokenProvider = () => { + invocations += 1; + return Promise.resolve(token); + }; + return {provider, callCount: () => invocations}; +} + +/** Fails the test if invoked. Used to assert a path did NOT reach the provider. */ +function unexpectedProvider(why: string): TokenProvider { + return () => Promise.reject(new Error(`provider must not be called: ${why}`)); +} + +/** The four fetch parameters are bundled (`BearerFetch`); this keeps the call sites readable. */ +function fetchWith( + provider: TokenProvider, + marginMs: number, + nowMs: number, +): BearerFetch { + return {provider, marginMs, nowMs, signal: undefined}; +} + +/** + * A macrotask boundary -- not a fixed number of microtask hops -- so a fire-and-forget refresh's whole + * then/finally chain has drained regardless of how many ticks it takes. + */ +function drainMacrotask(): Promise<void> { + return new Promise(resolve => { + setTimeout(resolve, 0); + }); +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this runner's + * type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper keeps the + * assertion honest without a lint suppression. Same shape 5a's and 5b's step suites settled on. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('BearerTokenCache: the fresh and expiring zones (AUTH-34/AUTH-37)', () => { + test('a fresh cached token is returned without invoking the provider (AUTH-34)', async () => { + const cache = new BearerTokenCache(); + const fresh = providerReturning(createBearerToken('t1', 10_000)); + await cache.stamp(fetchWith(fresh.provider, 1000, 0)); // primes the cache + // nowMs=0, expiresAt=10000, margin=1000 -- not expiring. + const result = await cache.stamp( + fetchWith(unexpectedProvider('the cached token is still fresh'), 1000, 0), + ); + expect(result.token).toBe('t1'); + expect(fresh.callCount()).toBe(1); + }); + + test('a token with no expiry is always in the fresh zone (AUTH-10/AUTH-34)', async () => { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith(providerReturning(createBearerToken('forever')).provider, 0, 0), + ); + const result = await cache.stamp( + fetchWith( + unexpectedProvider('a token with no expiry never expires locally'), + 60_000, + Number.MAX_SAFE_INTEGER, + ), + ); + expect(result.token).toBe('forever'); + }); + + test('expiring-but-valid: returns the stale token AND fires a background refresh (AUTH-37)', async () => { + const cache = new BearerTokenCache(); + const initial = providerReturning(createBearerToken('t1', 1000)); + // primes: expiresAt=1000, nowMs=0, margin=500 -- not yet expiring + await cache.stamp(fetchWith(initial.provider, 500, 0)); + + const refreshed = providerReturning(createBearerToken('t2', 5000)); + // nowMs=900: expiring (900+500 > 1000) but not expired (900 > 1000 is false) + const result = await cache.stamp(fetchWith(refreshed.provider, 500, 900)); + expect(result.token).toBe('t1'); // stale value returned immediately + await drainMacrotask(); + const after = await cache.stamp( + fetchWith(unexpectedProvider('the refresh already cached t2'), 500, 900), + ); + expect(after.token).toBe('t2'); + }); +}); + +describe('BearerTokenCache: the expired/missing zone (AUTH-37)', () => { + test('expired/missing: awaits a fresh fetch', async () => { + const cache = new BearerTokenCache(); + // The FETCHED token must itself be valid at the injected `nowMs` -- a provider handing back an + // already-expired token is AUTH-35's rejection case, covered separately below. + const {provider, callCount} = providerReturning( + createBearerToken('t1', 10_000), + ); + const result = await cache.stamp(fetchWith(provider, 0, 5000)); // nothing cached + expect(result.token).toBe('t1'); + expect(callCount()).toBe(1); + }); + + test('an EXPIRED cached token awaits a fresh fetch rather than being stamped (AUTH-37)', async () => { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('t1', 1000)).provider, + 0, + 0, + ), + ); + const {provider, callCount} = providerReturning( + createBearerToken('t2', 9000), + ); + const result = await cache.stamp(fetchWith(provider, 0, 5000)); // t1 expired at 1000 + expect(result.token).toBe('t2'); + expect(callCount()).toBe(1); + }); +}); + +describe('BearerTokenCache: a failed background refresh is non-fatal (AUTH-37)', () => { + test('a FAILING background refresh is non-fatal and never becomes an unhandled rejection (AUTH-37)', async () => { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('t1', 1000)).provider, + 500, + 0, + ), + ); + + const failing: TokenProvider = () => + Promise.reject(new Error('refresh backend down')); + // Expiring-but-valid: stamps t1, refresh fails in the background. + const result = await cache.stamp(fetchWith(failing, 500, 900)); + // The still-valid token was already stamped -- the failure changes nothing. + expect(result.token).toBe('t1'); + + // Drain past the fire-and-forget chain; an unhandled rejection would surface here. + await drainMacrotask(); + + // t1 is still cached and still served -- a failed refresh must not evict what it failed to replace. + const after = await cache.stamp( + fetchWith( + unexpectedProvider('t1 is still cached and still valid at this nowMs'), + 0, + 900, + ), + ); + expect(after.token).toBe('t1'); + }); + + test("AUTH-37's LOG half: the swallowed refresh failure reaches the global logger", async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + + try { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('t1', 1000)).provider, + 500, + 0, + ), + ); + const failing: TokenProvider = () => + Promise.reject(new Error('refresh backend down')); + await cache.stamp(fetchWith(failing, 500, 900)); + await drainMacrotask(); + + const refreshFailures = events.filter( + e => e.get('event') === 'http.auth.bearerRefreshFailed', + ); + expect(refreshFailures).toHaveLength(1); + expect(String(refreshFailures[0]?.get('cause'))).toContain( + 'refresh backend down', + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); + +describe('BearerTokenCache: single-flight and cancellation (AUTH-11/AUTH-34)', () => { + test('concurrent expired/missing callers coalesce to exactly one provider invocation (single-flight)', async () => { + let resolveProvider: + ((token: ReturnType<typeof createBearerToken>) => void) | undefined; + let invocations = 0; + const provider: TokenProvider = () => { + invocations += 1; + return new Promise(resolve => { + resolveProvider = resolve; + }); + }; + const cache = new BearerTokenCache(); + + const first = cache.stamp(fetchWith(provider, 0, 0)); + const second = cache.stamp(fetchWith(provider, 0, 0)); + expect(invocations).toBe(1); // the second caller coalesced onto the first's in-flight fetch + + resolveProvider?.(createBearerToken('t1', 10_000)); + const [firstResult, secondResult] = await Promise.all([first, second]); + expect(firstResult.token).toBe('t1'); + expect(secondResult.token).toBe('t1'); + }); +}); + +describe('BearerTokenCache: cancellation is per-caller, not per-fetch (AUTH-34)', () => { + // A coalesced fetch is owned by no single call, so it carries no caller signal. That is structural + // rather than asserted: `TokenProvider` is `() => Promise<BearerToken>` and has no parameter to + // populate. What IS asserted below is the behaviour that replaces it -- each caller races its own + // wait against its own signal. + test("an aborting caller stops waiting without cancelling a coalesced caller's fetch", async () => { + let resolveProvider: + ((token: ReturnType<typeof createBearerToken>) => void) | undefined; + let invocations = 0; + const provider: TokenProvider = () => { + invocations += 1; + return new Promise(resolve => { + resolveProvider = resolve; + }); + }; + const cache = new BearerTokenCache(); + const controller = new AbortController(); + + const aborting = cache.stamp({ + provider, + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }); + const patient = cache.stamp(fetchWith(provider, 0, 0)); // no signal at all + expect(invocations).toBe(1); + + const givenUp = new Error('caller A gave up'); + controller.abort(givenUp); + // N1/XCUT-1: the SDK's own terminal type, with the caller's reason kept as the cause. + const {CancellationError} = await import('../seams/transport.js'); + const abortRejection = (await rejectionOf(aborting)) as Error; + expect(abortRejection).toBeInstanceOf(CancellationError); + expect(abortRejection.cause).toBe(givenUp); + + // The shared fetch was never cancelled, so B still gets its token. + resolveProvider?.(createBearerToken('t1', 10_000)); + expect((await patient).token).toBe('t1'); + expect(invocations).toBe(1); + }); + + test('a caller whose signal is already aborted rejects without starting a fetch', async () => { + const cache = new BearerTokenCache(); + const controller = new AbortController(); + controller.abort(new Error('already gone')); + + const rejected = await rejectionOf( + cache.stamp({ + provider: unexpectedProvider('the caller had already aborted'), + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }), + ); + + const {CancellationError} = await import('../seams/transport.js'); + expect(rejected as Error).toBeInstanceOf(CancellationError); + expect((rejected as Error).cause).toHaveProperty('message', 'already gone'); + }); +}); + +describe('BearerTokenCache: a signal outliving many fetches (AUTH-34)', () => { + test('a long-lived signal driving many sequential fetches still aborts exactly one waiter', async () => { + // `raceAbort` attaches an abort listener per WAIT and removes it in a `finally`. Without that + // removal a caller signal that outlives many token fetches -- one request driving a long + // paginated sweep, say -- accumulates one dead listener per fetch until Node's + // MaxListenersExceededWarning fires. The listener COUNT is asserted directly in + // `tests/node-conformance/auth.test.mjs`, where `node:events`' `getEventListeners` is available; + // this is the behavioural half, on Bun: after many settled fetches the signal must still drive + // exactly the one waiter outstanding when it fires, not a backlog of stale ones. + const cache = new BearerTokenCache(); + const controller = new AbortController(); + let rejections = 0; + for (let round = 0; round < 8; round += 1) { + const {provider} = providerReturning( + createBearerToken(`t${String(round)}`, 10_000), + ); + await cache.stamp({ + provider, + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }); + cache.evict(`Bearer t${String(round)}`); // force the next round back into the fetch path + } + + let release: ((token: BearerToken) => void) | undefined; + const parked = cache.stamp({ + provider: () => + new Promise<BearerToken>(resolve => { + release = resolve; + }), + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }); + controller.abort(new Error('the sweep was cancelled')); + if ((await rejectionOf(parked)) !== undefined) rejections += 1; + + expect(rejections).toBe(1); + release?.(createBearerToken('unused', 10_000)); + }); +}); + +describe('BearerTokenCache: a provider that fails SYNCHRONOUSLY (AUTH-37/AUTH-38)', () => { + // `TokenProvider` is caller-supplied and its declared return type is a promise, but a plain-JS + // provider can throw before returning one -- the same boundary AUTH-35's `null` guard distrusts. + const syncThrowing: TokenProvider = () => { + throw new Error('provider exploded synchronously'); + }; + + test('a failed BACKGROUND refresh stays non-fatal: the valid token is still stamped (AUTH-37)', async () => { + const cache = new BearerTokenCache(); + const {provider} = providerReturning(createBearerToken('t1', 1000)); + await cache.stamp(fetchWith(provider, 0, 0)); // primes the cache + + // nowMs=900, margin=200 -> expiring-but-valid. AUTH-37: stamp the stale token, refresh in the + // background, and the background failure MUST NOT fail this request. A bare `provider()` call + // threw straight out of `stamp` here, past the `void ... .catch(...)` that had not been attached + // yet, rejecting a request that had a perfectly good token to send. + const stamped = await cache.stamp(fetchWith(syncThrowing, 200, 900)); + + expect(stamped.token).toBe('t1'); + }); + + test('refreshPostEviction rejects rather than throwing synchronously (AUTH-38)', async () => { + const cache = new BearerTokenCache(); + + const rejected = await rejectionOf( + cache.refreshPostEviction(fetchWith(syncThrowing, 0, 0)), + ); + + expect(rejected as Error).toHaveProperty( + 'message', + 'provider exploded synchronously', + ); + }); + + test('stamp rejects rather than throwing synchronously on the expired/missing path', async () => { + const cache = new BearerTokenCache(); + + const rejected = await rejectionOf( + cache.stamp(fetchWith(syncThrowing, 0, 0)), + ); + + expect(rejected as Error).toHaveProperty( + 'message', + 'provider exploded synchronously', + ); + }); +}); + +describe('BearerTokenCache: provider failure handling (AUTH-11/AUTH-35)', () => { + test('a null provider result throws AuthResolutionError (AUTH-35)', async () => { + const cache = new BearerTokenCache(); + // A plain-JS caller can hand back null regardless of TokenProvider's non-nullable return type; + // AUTH-35 requires a RUNTIME guard, so the cast is the point of the test, not a workaround. + const nullish = (() => Promise.resolve(null)) as unknown as TokenProvider; + expect( + await rejectionOf(cache.stamp(fetchWith(nullish, 0, 0))), + ).toBeInstanceOf(AuthResolutionError); + }); + + test('an already-expired provider result throws and is never cached (AUTH-35)', async () => { + const cache = new BearerTokenCache(); + const alreadyExpired: TokenProvider = () => + Promise.resolve(createBearerToken('t1', -1)); // expiresAt in the past + expect( + await rejectionOf(cache.stamp(fetchWith(alreadyExpired, 0, 1000))), + ).toBeInstanceOf(AuthResolutionError); + + const {provider: recovers, callCount} = providerReturning( + createBearerToken('t2', 10_000), + ); + const result = await cache.stamp(fetchWith(recovers, 0, 1000)); + expect(result.token).toBe('t2'); + // The earlier rejection left nothing cached to short-circuit this call. + expect(callCount()).toBe(1); + }); + + test('a rejecting provider propagates and is never cached (AUTH-11)', async () => { + const cache = new BearerTokenCache(); + const boom = new Error('network down'); + const failing: TokenProvider = () => Promise.reject(boom); + expect(await rejectionOf(cache.stamp(fetchWith(failing, 0, 0)))).toBe(boom); + + const {provider: recovers} = providerReturning( + createBearerToken('t1', 10_000), + ); + const result = await cache.stamp(fetchWith(recovers, 0, 0)); + expect(result.token).toBe('t1'); // no stale rejection cached -- this call fetches cleanly + }); +}); + +describe('BearerTokenCache: refreshPostEviction supersedes a pre-401 fetch (AUTH-37)', () => { + test('does NOT coalesce onto a fetch that was already in flight', async () => { + // The exact hazard: a background refresh started BEFORE the 401 came back. AUTH-11 permits a + // provider that caches internally, so that older fetch can resolve to the very token the server + // rejected. A `stamp()` here would coalesce onto it and re-send the rejected token. + const cache = new BearerTokenCache(); + const resolvers: ((token: ReturnType<typeof createBearerToken>) => void)[] = + []; + let invocations = 0; + const provider: TokenProvider = () => { + invocations += 1; + return new Promise(resolve => { + resolvers.push(resolve); + }); + }; + + const stale = cache.stamp(fetchWith(provider, 0, 0)); // starts fetch #1 and parks it in flight + expect(invocations).toBe(1); + + const fresh = cache.refreshPostEviction(fetchWith(provider, 0, 0)); + expect(invocations).toBe(2); // a SECOND provider call, not a handle on the first + + resolvers[0]?.(createBearerToken('rejected-token', 10_000)); + resolvers[1]?.(createBearerToken('genuinely-fresh', 10_000)); + await stale; + expect((await fresh).token).toBe('genuinely-fresh'); + }); + + test('a superseded fetch resolving LAST still cannot re-cache the rejected token', async () => { + // Same hazard, opposite resolution order -- the one a generation-less cache gets wrong: the + // pre-401 fetch settles after the fresh one and would otherwise overwrite it. + const cache = new BearerTokenCache(); + const resolvers: ((token: ReturnType<typeof createBearerToken>) => void)[] = + []; + const provider: TokenProvider = () => + new Promise(resolve => { + resolvers.push(resolve); + }); + + const stale = cache.stamp(fetchWith(provider, 0, 0)); + const fresh = cache.refreshPostEviction(fetchWith(provider, 0, 0)); + + resolvers[1]?.(createBearerToken('genuinely-fresh', 10_000)); + await fresh; + resolvers[0]?.(createBearerToken('rejected-token', 10_000)); + await stale; + await drainMacrotask(); + + const served = await cache.stamp( + fetchWith( + unexpectedProvider('the fresh token is cached and valid'), + 0, + 0, + ), + ); + expect(served.token).toBe('genuinely-fresh'); + }); +}); + +describe('BearerTokenCache: refreshPostEviction caches and re-drives (AUTH-36/AUTH-37)', () => { + test('the EVICTION path supersedes a pre-401 fetch that resolves late', async () => { + // The sibling above drives `stamp` + `refreshPostEviction` directly. This one goes through AUTH-36's + // actual 401 sequence -- evict, then refreshPostEviction -- with the pre-401 background fetch resolving + // to exactly the token the server rejected, which AUTH-11 expressly permits a + // internally-caching provider to do. + const cache = new BearerTokenCache(); + let releasePreFetch: (() => void) | undefined; + const slow: TokenProvider = () => + new Promise(resolve => { + releasePreFetch = () => { + resolve(createBearerToken('rejected-token', 10_000)); + }; + }); + + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('rejected-token', 1000)).provider, + 0, + 0, + ), + ); + await cache.stamp(fetchWith(slow, 200, 900)); // parks a pre-401 background fetch in flight + + expect(cache.evict('Bearer rejected-token')).toBeUndefined(); + const fresh = await cache.refreshPostEviction( + fetchWith( + providerReturning(createBearerToken('genuinely-fresh', 100_000)) + .provider, + 0, + 900, + ), + ); + expect(fresh.token).toBe('genuinely-fresh'); + + releasePreFetch?.(); + await drainMacrotask(); + + const served = await cache.stamp( + fetchWith( + unexpectedProvider('the post-eviction token is cached and valid'), + 0, + 1000, + ), + ); + expect(served.token).toBe('genuinely-fresh'); + }); + + test('caches its result like any other fetch', async () => { + const cache = new BearerTokenCache(); + const {provider, callCount} = providerReturning( + createBearerToken('t1', 10_000), + ); + await cache.refreshPostEviction(fetchWith(provider, 0, 0)); + const again = await cache.stamp( + fetchWith( + unexpectedProvider('refreshPostEviction() populated the cache'), + 0, + 0, + ), + ); + expect(again.token).toBe('t1'); + expect(callCount()).toBe(1); + }); +}); + +describe('BearerTokenCache: a 401 burst coalesces (AUTH-34/AUTH-37)', () => { + test('N concurrent post-eviction refreshes share ONE provider fetch, not N', async () => { + // A server-side revocation 401s every in-flight request at once. Superseding the pre-401 fetch is + // required (AUTH-37), but starting one provider call per 401 is the thundering herd AUTH-34's + // "at most one provider fetch" clause forbids. + const cache = new BearerTokenCache(); + let calls = 0; + let release: ((token: BearerToken) => void) | undefined; + const provider = (): Promise<BearerToken> => { + calls += 1; + return new Promise<BearerToken>(resolve => { + release = resolve; + }); + }; + + const burst = [ + cache.refreshPostEviction(fetchWith(provider, 0, 0)), + cache.refreshPostEviction(fetchWith(provider, 0, 0)), + cache.refreshPostEviction(fetchWith(provider, 0, 0)), + cache.refreshPostEviction(fetchWith(provider, 0, 0)), + ]; + release?.(createBearerToken('fresh', 10_000)); + const tokens = await Promise.all(burst); + + expect(calls).toBe(1); + expect(tokens.map(token => token.token)).toEqual([ + 'fresh', + 'fresh', + 'fresh', + 'fresh', + ]); + }); + + test('a LATER 401 still supersedes: it does not join the settled burst fetch', async () => { + const cache = new BearerTokenCache(); + const first = providerReturning(createBearerToken('t1', 10_000)); + await cache.refreshPostEviction(fetchWith(first.provider, 0, 0)); + const second = providerReturning(createBearerToken('t2', 10_000)); + + const result = await cache.refreshPostEviction( + fetchWith(second.provider, 0, 0), + ); + + expect(result.token).toBe('t2'); + expect(second.callCount()).toBe(1); + }); + + test('a stamp()-driven fetch sitting at the current generation is NOT joined by refreshPostEviction', async () => { + // The guard is `inFlightEvictionGeneration`, not the generation counter alone: an ordinary + // single-flight fetch must still be superseded, or a pre-401 fetch could hand back the very token + // the server just rejected (AUTH-37). + const cache = new BearerTokenCache(); + let release: ((token: BearerToken) => void) | undefined; + const stale = (): Promise<BearerToken> => + new Promise<BearerToken>(resolve => { + release = resolve; + }); + const pending = cache.stamp(fetchWith(stale, 0, 0)); + + const fresh = providerReturning(createBearerToken('fresh', 10_000)); + const result = await cache.refreshPostEviction( + fetchWith(fresh.provider, 0, 0), + ); + + expect(result.token).toBe('fresh'); + expect(fresh.callCount()).toBe(1); + release?.(createBearerToken('rejected', 10_000)); + await pending; + }); +}); + +describe('BearerTokenCache: evict (AUTH-36)', () => { + test('evicts only when the header value matches the exact cached token', async () => { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('t1', 10_000)).provider, + 0, + 0, + ), + ); + // The survivor is RETURNED, which is what makes AUTH-36's "preserving a token another request + // already refreshed" observable rather than a no-op the next fetch overwrites. + expect(cache.evict('Bearer some-other-token')?.token).toBe('t1'); + const result = await cache.stamp( + fetchWith( + unexpectedProvider('a non-matching evict() must not clear the cache'), + 0, + 0, + ), + ); + expect(result.token).toBe('t1'); + }); + + test('a matching evict() forces the next call to refetch', async () => { + const cache = new BearerTokenCache(); + await cache.stamp( + fetchWith( + providerReturning(createBearerToken('t1', 10_000)).provider, + 0, + 0, + ), + ); + expect(cache.evict('Bearer t1')).toBeUndefined(); + const {provider, callCount} = providerReturning( + createBearerToken('t2', 10_000), + ); + const result = await cache.stamp(fetchWith(provider, 0, 0)); + expect(result.token).toBe('t2'); + expect(callCount()).toBe(1); + }); + + test('is a no-op returning undefined when nothing is cached', () => { + expect(new BearerTokenCache().evict('Bearer anything')).toBeUndefined(); + }); +}); diff --git a/packages/core/src/auth/bearer-cache.ts b/packages/core/src/auth/bearer-cache.ts new file mode 100644 index 0000000..2b2f783 --- /dev/null +++ b/packages/core/src/auth/bearer-cache.ts @@ -0,0 +1,366 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/bearer-cache.ts +import { + isBearerTokenExpired, + type BearerToken, + type TokenProvider, +} from './credential.js'; +import {AuthResolutionError} from './errors.js'; +import {abortToSdkError} from '../cancellation.js'; +import {getGlobalLogger} from '../observability/logger.js'; + +/** + * The value {@link BearerTokenCache.inFlightEvictionGeneration} carries when the in-flight fetch (if + * any) came from the ordinary {@link BearerTokenCache.stamp} path. Never a real generation: the + * counter only ever increments from 0, so no post-eviction fetch can collide with it. + */ +const NO_EVICTION_GENERATION = -1; + +/** + * AUTH-37's record half: a background refresh that failed is non-fatal, and is *logged*. + * + * Swallowed rather than re-raised for the reason stated at the call site -- a bare `void` leaves an + * unhandled rejection that terminates the process under Node's default policy, asynchronously and + * unattributable to any request, for a fault in caller-supplied `TokenProvider` code. The log is + * what makes "continue" honest rather than silent. + */ +function warnRefreshFailed(error: unknown): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('http.auth.bearerRefreshFailed') + .cause(error) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request -- and this one is not even on a request + // path, so a throw here would be the detached rejection the catch above exists to prevent. + } +} + +/** + * One token fetch's inputs. + * + * Bundled rather than passed positionally: `max-params` is 3, and + * `docs/knowledge/harvested/function-design.md` requires an options object at three or more parameters anyway. + * + * @internal + */ +export interface BearerFetch { + /** The token source (AUTH-11). */ + readonly provider: TokenProvider; + /** AUTH-34's refresh margin: how long before expiry a token counts as expiring. */ + readonly marginMs: number; + /** + * The injected clock reading. Never `Date.now()` inside this class — see {@link + * BearerTokenCache.refresh}. + */ + readonly nowMs: number; + /** + * The calling request's cancellation. + * + * It is NOT handed to the provider. A fetch reached through AUTH-34's single-flight coalescing is + * shared by every caller that joined it, so cancelling it on one caller's signal would reject + * callers who never aborted -- and a caller who supplied no signal at all. Instead each caller + * RACES the shared promise against its own signal ({@link raceAbort}): an aborting caller stops + * waiting, and the work the others are joined to keeps running. {@link TokenProvider} therefore + * takes no parameters at all, and carries the deadline obligation that follows from the fetch + * itself being uncancellable. + */ + readonly signal: AbortSignal | undefined; +} + +/** + * Calls `provider` so that a SYNCHRONOUS failure reaches the async channel like every other provider + * failure (AUTH-37, AUTH-38). + * + * A `TokenProvider` is caller-supplied code. Its declared return type is `Promise<BearerToken>`, but + * a plain-JS provider can throw before returning, or return something that is not a promise at all -- + * the same boundary the `null | undefined` widening in {@link BearerTokenCache.refresh} already + * distrusts, distrusted the same way. Left as a bare `provider(...)` call, such a failure escaped past + * `stamp`'s `void ... .catch(...)` before the catch was ever attached, turning AUTH-37's expressly + * non-fatal background refresh into a fatal one and rejecting a request that had a perfectly good + * cached token to stamp. + * + * An `async` wrapper, NOT `Promise.resolve().then(provider)`: an async function body runs + * synchronously up to its first `await`, so the provider is still invoked in the same tick as the + * `inFlight` assignment. Deferring it by a microtask would put an await between the single-flight + * check and the assignment -- the one thing the guard's lock-free correctness rests on. Returning + * `provider()` unawaited is likewise deliberate: `await`ing it here would trip `return-await` outside + * a try, and buys nothing, because the `async` keyword already converts a synchronous throw into a + * rejection. + * + * No `signal` is passed because {@link TokenProvider} takes no parameters at all: a coalesced fetch + * is owned by no single call, so there is nothing a caller signal could correctly mean here. See + * {@link BearerFetch.signal} for what happens instead; the provider owns its own deadline. + * + * Collapse this back into a bare `provider()` call only if `TokenProvider` stops being + * caller-supplied code. + */ +async function invokeProvider(provider: TokenProvider): Promise<BearerToken> { + return provider(); +} + +/** + * Starts (or joins) a fetch and awaits it, but stops waiting when `signal` aborts -- WITHOUT + * cancelling the fetch, which is shared by every caller coalesced onto it (AUTH-34). + * + * Takes a factory rather than a promise so the already-aborted check runs BEFORE any fetch is + * started, and so `start()` is still invoked in the caller's own synchronous span: an `async` + * function body runs to its first `await` synchronously, which is what keeps the single-flight + * assignment and the generation bump un-interleaved. + * + * `new Promise` with a synchronous executor adapting an event-emitter callback is the one shape + * `docs/knowledge/harvested/concurrency-and-async.md` sanctions for it, and the listener is removed on every + * exit so a long-lived caller signal does not accumulate one per token fetch. + * + * A `pending` that rejects after losing the race is still settled through `Promise.race`'s own + * handler, so it never becomes an unhandled rejection. + */ +async function raceAbort( + start: () => Promise<BearerToken>, + signal: AbortSignal | undefined, +): Promise<BearerToken> { + // Before `start()`, so an already-dead caller never opens a fetch it cannot use -- + // `concurrency-and-async.md`'s "check the signal before each expensive step". + // + // Mapped through `abortToSdkError` rather than rethrown verbatim (N1/XCUT-1): a cancelled token + // fetch and a cancelled transport dispatch are the same event to a caller, and used to arrive as + // two different types. The caller's own reason is kept as `.cause`. + if (signal?.aborted === true) throw abortToSdkError(signal, signal.reason); + const pending = start(); + if (signal === undefined) return pending; + let onAbort = (): void => undefined; + try { + return await Promise.race([ + pending, + new Promise<never>((_resolve, reject) => { + onAbort = (): void => { + reject(abortToSdkError(signal, signal.reason)); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }), + ]); + } finally { + signal.removeEventListener('abort', onAbort); + } +} + +/** + * The single-flight, three-zone bearer token cache (AUTH-34, AUTH-35, AUTH-37). + * + * The async three-zone policy is shipped unconditionally. This port has one `Promise`-only pipeline + * execution model (4c), so AUTH-34's "non-blocking hot-path read of a valid cached token" is the + * fresh-zone branch of this same state machine, not a second stack — directly parallel to 5a's + * one-retry-engine disposition of RETRY-28. + * + * Single-flight is a plain field, not a lock. On Node and Bun the only hazard would be two logical + * calls both observing "no in-flight fetch" before either assigns the slot, and that cannot happen + * because nothing awaits between the check and the assignment in {@link BearerTokenCache.refresh} — + * the same synchronous-guard collapse as Digest's nonce counter. + * + * One instance per configured {@link TokenProvider}; every test constructs its own. + * + * @internal + */ +export class BearerTokenCache { + private cached: BearerToken | undefined; + private inFlight: Promise<BearerToken> | undefined; + /** + * Bumped by {@link BearerTokenCache.refreshPostEviction} to supersede every fetch already in + * flight. A superseded fetch still resolves to its own caller, but must not publish its token into + * `cached` or clear the newer fetch's `inFlight` slot — otherwise a pre-401 fetch resolving late + * would re-cache exactly the token the server rejected, which is the outcome AUTH-37 forbids. + */ + private generation = 0; + /** + * The generation an EVICTION-DRIVEN fetch currently in flight was started at, or + * {@link NO_EVICTION_GENERATION} when the in-flight fetch (if any) came from the ordinary + * {@link BearerTokenCache.stamp} path. + * + * This is what lets {@link BearerTokenCache.refreshPostEviction} coalesce without re-opening the + * hazard it exists to close. A pre-401 fetch must never be joined -- it can resolve to the very + * token the server just rejected -- but two 401s on the SAME token arriving together should share + * one fetch, or a mass revocation turns every in-flight request into its own provider call, which + * is exactly the thundering herd AUTH-34's single-flight clause forbids. Comparing this against + * `generation` separates the two cases exactly: only a fetch started by `refreshPostEviction` AT + * the current generation is a genuine post-eviction fetch. + */ + private inFlightEvictionGeneration = NO_EVICTION_GENERATION; + + /** + * AUTH-34/AUTH-37's three zones: fresh (stamp, no refresh), expiring-but-valid (stamp the stale + * token, refresh in the background), expired or missing (await a fresh single-flight fetch). + * + * @param fetchOptions - the provider, margin, injected clock reading, and call signal. + * @returns the token to stamp. + * @throws AuthResolutionError when the provider yields null or an already-expired token (AUTH-35). + */ + async stamp(fetchOptions: BearerFetch): Promise<BearerToken> { + const {marginMs, nowMs} = fetchOptions; + if (this.cached !== undefined) { + const expiring = isBearerTokenExpired(this.cached, nowMs, marginMs); + if (!expiring) return this.cached; // fresh zone: stamp, no refresh + const expired = isBearerTokenExpired(this.cached, nowMs, 0); // AUTH-35: no margin at fetch time + if (!expired) { + const stillValid = this.cached; + // Expiring-but-valid zone: fire-and-forget, and the catch is BLANKET on purpose. AUTH-37 is + // unconditional -- "a failed/unusable BACKGROUND refresh MUST NOT fail the in-flight + // request (log-and-continue)" -- and a bare `void this.refresh(...)` would leave the + // rejection unhandled, which under Node's default policy terminates the process. + // + // An earlier shape re-threw an InvariantViolation here, reasoning that a programmer error + // must crash loudly. That was wrong twice over. The throw landed in a promise nobody awaits, + // so it did not surface at the fault -- it killed the host process asynchronously, with no + // request to attribute it to, while the request that triggered it had already been served a + // valid token. And the fault it re-raised is not ours: a blank token from a caller-supplied + // `TokenProvider` is an operational fault (an empty environment variable, a malformed IdP + // payload) as often as a coding one. `error-handling.md`'s crash-loudly rule governs OUR + // invariants at the point WE detect them; it does not license re-raising someone else's + // failure into a detached promise. + // + // AUTH-37's "log-and-continue" is now BOTH halves. The log arrived on 2026-09-02, once 7b's + // `getGlobalLogger()` existed to write to; until then the rejection was swallowed with no + // trace at all. Continue is unchanged: the still-valid token was already returned and a + // failed refresh evicts nothing. + // + // Not raced against `fetchOptions.signal`: this refresh belongs to the cache, not to the + // request that happened to trigger it, and it must outlive that request's cancellation. + void this.refresh(fetchOptions, NO_EVICTION_GENERATION).catch( + (error: unknown) => { + warnRefreshFailed(error); + }, + ); + return stillValid; + } + } + // Expired/missing zone: await a fresh single-flight fetch, but only until this caller's own + // signal fires (AUTH-34, and `concurrency-and-async.md`'s honour-the-signal rule). + return raceAbort( + () => this.refresh(fetchOptions, NO_EVICTION_GENERATION), + fetchOptions.signal, + ); + } + + /** + * AUTH-37's post-eviction path: a fetch guaranteed to have STARTED after a 401 in this eviction + * burst, never before one. + * + * {@link BearerTokenCache.stamp} is not a substitute. It routes through `refresh`, which hands back + * an already-in-flight promise — and that fetch may have started BEFORE the 401 arrived. AUTH-11 + * explicitly permits a provider that caches or refreshes internally, so such a fetch can resolve to + * the very token the server just rejected, which is precisely what AUTH-37's "re-stamp a single + * retry with a freshly fetched token" forbids. + * + * It does NOT bypass single-flight wholesale, which an earlier shape did: under a mass revocation + * every in-flight request gets its own 401, and starting one provider fetch per 401 is the + * thundering herd AUTH-34's "at most one provider fetch" clause exists to prevent. Coalescing is + * gated on {@link BearerTokenCache.inFlightEvictionGeneration} instead, so concurrent 401s share + * one post-eviction fetch while a pre-401 fetch is still always superseded. That is why the name + * is `refreshPostEviction` rather than `refreshNow`: this call may JOIN a sibling 401's fetch, and + * what it actually guarantees is that no fetch predating this eviction burst is ever joined. + * + * @param fetchOptions - the provider, margin, injected clock reading, and call signal. + * @returns the freshly fetched token. + * @throws AuthResolutionError when the provider yields null or an already-expired token (AUTH-35). + */ + // `async` for AUTH-38's uniform error model: this path runs caller-supplied provider code, and a + // provider that fails BEFORE returning a promise would otherwise throw synchronously out of a + // method whose declared return type is `Promise<BearerToken>`. + async refreshPostEviction(fetchOptions: BearerFetch): Promise<BearerToken> { + return raceAbort( + () => this.startPostEviction(fetchOptions), + fetchOptions.signal, + ); + } + + /** + * The join-or-supersede decision, split out so {@link raceAbort} can gate it on the caller's signal + * without the generation bump drifting out of the caller's synchronous span. Nothing awaits between + * the `inFlight` read and the write, which is what makes single-flight lock-free. + */ + private startPostEviction(fetchOptions: BearerFetch): Promise<BearerToken> { + if ( + this.inFlight !== undefined && + this.inFlightEvictionGeneration === this.generation + ) { + // Another 401 in this same burst already started a post-eviction fetch: join it (AUTH-34). + return this.inFlight; + } + this.generation += 1; // supersede every fetch already in flight + this.inFlight = undefined; // drop the pre-401 fetch's claim on the slot before starting a new one + return this.refresh(fetchOptions, this.generation); + } + + /** + * AUTH-36: clears the cache only when the currently-cached token is the exact one that produced the + * 401, matched on the stamped header value — so a token another in-flight request already refreshed + * survives. + * + * The survivor is RETURNED, not merely left in place, because that is the only way AUTH-36's + * "preserving a token another request already refreshed" clause becomes observable. Preserving it + * and then unconditionally fetching a replacement — which is what the caller did before — overwrote + * the preserved token on the next tick and made the whole clause a no-op. + * + * @param rejectedHeaderValue - the `Authorization` value the rejected request carried. + * @returns the cached token when it is NOT the rejected one (another request already refreshed it, + * so the retry should stamp this instead of fetching again), or `undefined` when the rejected + * token was evicted or nothing was cached. + */ + evict(rejectedHeaderValue: string): BearerToken | undefined { + if (this.cached === undefined) return undefined; + if (`Bearer ${this.cached.token}` === rejectedHeaderValue) { + this.cached = undefined; + return undefined; + } + return this.cached; + } + + /** + * `nowMs` is threaded in rather than read from `Date.now()`: `stamp()` already takes an injected + * clock, and a refresh validating against the ambient wall clock while its caller reasons about an + * injected one would be a second, invisible clock — it would reject every token under synthetic + * time and be uncontrollable in production. + */ + private refresh( + fetchOptions: BearerFetch, + evictionGeneration: number, + ): Promise<BearerToken> { + // Returns the RAW shared promise, never one raced against a caller signal. `raceAbort` is applied + // by the two public entry points instead, so the background refresh in `stamp()` -- which belongs + // to the cache rather than to any one request -- is deliberately left unraced. + if (this.inFlight !== undefined) return this.inFlight; // coalesce concurrent expiring/missing callers + const generation = this.generation; + // The generation is passed in rather than derived from a boolean flag: the ordinary path writes + // `NO_EVICTION_GENERATION` so a later `refreshPostEviction` cannot mistake a `stamp()`-driven + // fetch that happens to sit at the current generation for a post-eviction one and join it. + this.inFlightEvictionGeneration = evictionGeneration; + const pending = invokeProvider(fetchOptions.provider) + .then((token: BearerToken | null | undefined) => { + // `token` is widened at this ONE boundary on purpose. `TokenProvider`'s declared return type + // is non-nullable, so comparing the un-widened value against null trips + // `@typescript-eslint/no-unnecessary-condition` from the strict-type-checked tier -- but + // AUTH-35 requires a RUNTIME guard, because a plain-JS caller (or a mis-typed `any` + // boundary) can hand back null regardless of what the type says. Widening states that intent + // instead of suppressing the rule. + if ( + token === null || + token === undefined || + isBearerTokenExpired(token, fetchOptions.nowMs, 0) + ) { + throw new AuthResolutionError( + 'token provider returned a null or already-expired token', + ); // AUTH-35 + } + if (generation === this.generation) this.cached = token; + return token; + }) + .finally(() => { + // Never cache a rejection (AUTH-11/AUTH-35) -- it already propagates through `finally` + // untouched, so no `catch` is added. Guarded on the generation so a superseded fetch cannot + // clear a newer fetch's slot. + if (generation === this.generation) this.inFlight = undefined; + }); + this.inFlight = pending; + return pending; + } +} diff --git a/packages/core/src/auth/challenge.test.ts b/packages/core/src/auth/challenge.test.ts new file mode 100644 index 0000000..df5b787 --- /dev/null +++ b/packages/core/src/auth/challenge.test.ts @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/challenge.test.ts +// Exercises: AUTH-12 (scheme/param names lower-cased, values verbatim, token68 under its synthetic +// key), AUTH-13 (total: blank -> [], malformed recovers at the next top-level comma, unterminated +// quote ends at EOF, params before a malformed tail kept), and the multi-challenge/comma-ambiguity +// case that is the whole reason this parser cannot be a `.split(',')`. +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {parseChallenges} from './challenge.js'; + +describe('a single challenge', () => { + test('scheme and param names are lower-cased; values are verbatim', () => { + const [challenge] = parseChallenges('BASIC Realm="MixedCase"'); + expect(challenge?.scheme).toBe('basic'); + expect(challenge?.params.get('realm')).toBe('MixedCase'); + }); + + test('a bare scheme with no params gets an empty parameter map', () => { + const [challenge] = parseChallenges('NTLM'); + expect(challenge?.scheme).toBe('ntlm'); + expect(challenge?.params.size).toBe(0); + }); + + test('a token68 value is recorded under the synthetic key', () => { + const [challenge] = parseChallenges( + 'Negotiate a87421000492aa874209af8bc028', + ); + expect(challenge?.scheme).toBe('negotiate'); + // AUTH-12 names this key literally as 'token68'. + expect(challenge?.params.get('token68')).toBe( + 'a87421000492aa874209af8bc028', + ); + }); + + test("token68's own '=' padding is part of the value, not an assignment", () => { + const [challenge] = parseChallenges('Negotiate YWJj=='); + expect(challenge?.params.get('token68')).toBe('YWJj=='); + }); + + test('an unquoted token value is accepted', () => { + const [challenge] = parseChallenges('Digest realm=simple, qop=auth'); + expect(challenge?.params.get('realm')).toBe('simple'); + expect(challenge?.params.get('qop')).toBe('auth'); + }); + + // Only the WRAPPER is frozen, and the name says so. `Object.freeze` on the `params` `Map` would + // not stop `.set()`, so freezing it would be a comment that lies; the `ReadonlyMap` TYPE is the + // only guard on the parameters, for the same reason `createAuthRequirement` records for its own + // params map -- `Challenge` is `@internal`, never reaches a consumer, and nothing in this package + // re-casts `Challenge['params']` back to `Map`. + test('the returned challenge wrapper is frozen', () => { + const [challenge] = parseChallenges('Basic realm="a"'); + expect(Object.isFrozen(challenge)).toBe(true); + }); +}); + +describe('multiple comma-separated challenges', () => { + test('a top-level comma between two DIFFERENT auth-params of the SAME challenge does not start a new one', () => { + const challenges = parseChallenges( + 'Digest realm="a", nonce="n", qop="auth"', + ); + expect(challenges).toHaveLength(1); + expect(challenges[0]?.params.get('realm')).toBe('a'); + expect(challenges[0]?.params.get('nonce')).toBe('n'); + expect(challenges[0]?.params.get('qop')).toBe('auth'); + }); + + test('two distinct challenges are both recovered, each with its own params', () => { + const challenges = parseChallenges( + 'Basic realm="a", Digest realm="b", nonce="n"', + ); + expect(challenges).toHaveLength(2); + expect(challenges[0]).toEqual({ + scheme: 'basic', + params: new Map([['realm', 'a']]), + }); + expect(challenges[1]?.scheme).toBe('digest'); + expect(challenges[1]?.params.get('realm')).toBe('b'); + expect(challenges[1]?.params.get('nonce')).toBe('n'); + }); + + test('a comma INSIDE a quoted value never splits the challenge', () => { + const [challenge] = parseChallenges('Digest realm="a, b", nonce="n"'); + expect(challenge?.params.get('realm')).toBe('a, b'); + expect(challenge?.params.get('nonce')).toBe('n'); + }); + + test('wire order is preserved', () => { + const challenges = parseChallenges('Digest realm="d", Basic realm="b"'); + expect(challenges.map(c => c.scheme)).toEqual(['digest', 'basic']); + }); +}); + +describe('quoted-string handling', () => { + test('a backslash escape is unquoted', () => { + const [challenge] = parseChallenges(String.raw`Digest realm="a\"b"`); + expect(challenge?.params.get('realm')).toBe('a"b'); + }); + + test('an unterminated quoted string terminates at end-of-input (AUTH-13)', () => { + const [challenge] = parseChallenges('Digest realm="abc'); + expect(challenge?.params.get('realm')).toBe('abc'); + }); + + test('an equals sign inside a quoted value is not an assignment', () => { + const [challenge] = parseChallenges('Digest realm="a=b", nonce="n"'); + expect(challenge?.params.get('realm')).toBe('a=b'); + expect(challenge?.params.get('nonce')).toBe('n'); + }); +}); + +describe('totality and recovery (AUTH-13)', () => { + test('blank input yields an empty list', () => { + expect(parseChallenges('')).toEqual([]); + expect(parseChallenges(' ')).toEqual([]); + }); + + test('a malformed segment recovers at the next top-level comma, keeping prior params', () => { + const challenges = parseChallenges( + 'Digest realm="a", =bad, Basic realm="b"', + ); + expect(challenges).toHaveLength(2); + expect(challenges[0]).toEqual({ + scheme: 'digest', + params: new Map([['realm', 'a']]), + }); + expect(challenges[1]).toEqual({ + scheme: 'basic', + params: new Map([['realm', 'b']]), + }); + }); + + test('a leading auth-param with no scheme ahead of it is discarded, not crashed on', () => { + const challenges = parseChallenges('realm="orphan", Basic realm="b"'); + expect(challenges).toHaveLength(1); + expect(challenges[0]?.scheme).toBe('basic'); + }); + + test('a comma inside the malformed segment’s quoted value is not a recovery point', () => { + const challenges = parseChallenges( + 'Basic realm="a", ="x,y", Digest realm="b"', + ); + expect(challenges.map(c => c.scheme)).toEqual(['basic', 'digest']); + }); + + test('an escaped quote inside a malformed segment does not end its quoted run', () => { + // The `\"` keeps the run open, so the comma that follows is still inside quotes and is not a + // recovery point; recovery lands on the one after the closing quote. + const challenges = parseChallenges( + String.raw`Basic realm="a", ="x\",y", Digest realm="b"`, + ); + expect(challenges.map(c => c.scheme)).toEqual(['basic', 'digest']); + }); + + test('stray commas collapse rather than emitting empty challenges', () => { + expect(parseChallenges(',,,')).toEqual([]); + expect( + parseChallenges('Basic,,Digest realm="b"').map(c => c.scheme), + ).toEqual(['basic', 'digest']); + }); +}); + +describe('parser totality, as properties (AUTH-13)', () => { + test('property: never throws for arbitrary input', () => { + fc.assert( + fc.property(fc.string(), raw => { + expect(() => parseChallenges(raw)).not.toThrow(); + }), + ); + }); + + test('property: never throws and always terminates over a metacharacter corpus', () => { + // `fc.string()` above rarely produces dense clusters of the characters that actually drive this + // parser's recovery branches, and one of those branches (`readSchemeTail` resetting to its saved + // position when a token68 read comes back empty) advances nothing on its own -- termination rests + // on the outer loop consuming instead. This enumerates that alphabet exhaustively at length 5. + const alphabet = [' ', ',', '=', '"', '\\', '/', '!', '@', 'a', '\t']; + for (let seed = 0; seed < 100_000; seed += 1) { + let text = ''; + let n = seed; + for (let i = 0; i < 5; i += 1) { + text += alphabet[n % alphabet.length] ?? ''; + n = Math.floor(n / alphabet.length); + } + expect(() => parseChallenges(text)).not.toThrow(); + } + }); + + test('a 100 KB quoted value stays linear and yields one challenge', () => { + expect( + parseChallenges(`Basic realm="${'x'.repeat(100_000)}"`), + ).toHaveLength(1); + }); + + test('property: a well-formed single challenge round-trips scheme + one param exactly', () => { + fc.assert( + fc.property( + fc.stringMatching(/^[A-Za-z][A-Za-z0-9]{1,10}$/u), + fc.stringMatching(/^[a-zA-Z0-9 ]{0,20}$/u), + (scheme, value) => { + const [challenge] = parseChallenges(`${scheme} realm="${value}"`); + expect(challenge?.scheme).toBe(scheme.toLowerCase()); + expect(challenge?.params.get('realm')).toBe(value); + }, + ), + ); + }); + + test('property: a comma inside a quoted value never splits the challenge', () => { + fc.assert( + fc.property(fc.stringMatching(/^[a-zA-Z0-9]{0,8}$/u), fragment => { + const challenges = parseChallenges( + `Digest realm="${fragment},${fragment}", nonce="n"`, + ); + expect(challenges).toHaveLength(1); + expect(challenges[0]?.params.get('realm')).toBe( + `${fragment},${fragment}`, + ); + }), + ); + }); +}); diff --git a/packages/core/src/auth/challenge.ts b/packages/core/src/auth/challenge.ts new file mode 100644 index 0000000..5428702 --- /dev/null +++ b/packages/core/src/auth/challenge.ts @@ -0,0 +1,326 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/challenge.ts + +/** + * One parsed RFC 7235 challenge (AUTH-12): the scheme plus its auth-params. + * + * Scheme and parameter names are lower-cased; parameter values are stored verbatim after unquoting. + * + * @internal + */ +export interface Challenge { + /** The challenge scheme, lower-cased. */ + readonly scheme: string; + /** Auth-params, keys lower-cased, values verbatim after unquoting. */ + readonly params: ReadonlyMap<string, string>; +} + +/** + * What a handler needs from the request being stamped, beyond the challenge itself. + * + * Digest's HA2 is computed over the method and the request-target (RFC 7616 §3.4.3), neither of which + * the challenge carries. Passed as a small context object rather than the whole `Request` so a handler + * cannot reach the body or headers it has no business reading. + * + * @internal + */ +export interface DigestUriContext { + /** The request method, upper-case, as it goes on the wire. */ + readonly method: string; + /** The digest-uri: the request-target (path plus query) of the request being stamped. */ + readonly requestTarget: string; +} + +/** + * One challenge-reactive stamping strategy (AUTH-23–AUTH-25). Implemented by `basic.ts` and + * `digest.ts`, composed by `composing-handler.ts`. + * + * Throughout `auth/`, to STAMP means to PRODUCE the value the caller writes, never to write it: every + * `stamp` in this module (`ChallengeHandler.stamp`, `ComposingHandler.stamp`, `stampStaticKey`, + * `BearerTokenCache.stamp`) returns credential material and leaves the header write to `auth-step.ts`. + * + * Declared here rather than in `composing-handler.ts` so both implementations can be written before + * the composer exists, and because `challenge.ts` already owns the {@link Challenge} type both + * methods operate on. + * + * `stamp()` is asynchronous because Digest's SHA-256/SHA-256-sess algorithms compute HA1/HA2/response + * through `crypto.subtle.digest()`, and Web Crypto offers no synchronous digest to fall back to. + * Basic's implementation resolves immediately. + * + * @internal + */ +export interface ChallengeHandler { + /** + * AUTH-16/AUTH-25: whether this handler can answer `challenge`. + * + * @param challenge - the offered challenge. + * @returns `true` when this handler can produce a header value for it. + */ + canHandle(challenge: Challenge): boolean; + + /** + * Produces the header VALUE only — the caller picks `Authorization` vs `Proxy-Authorization` from + * which challenge header the status actually carried (AUTH-25). + * + * There is deliberately no `isProxy` parameter. It was one, threaded from `auth-step.ts` through + * `composing-handler.ts` into both implementations, and NEITHER read it: Basic's value is computed + * once at construction and Digest's depends only on the challenge and the request-target, so the + * only test either could carry for the parameter was one asserting it changed nothing. AUTH-25's + * origin-vs-proxy choice lives entirely in `auth-step.ts`'s `answerHeaderName`, which is the one + * place that knows which challenge header the status carried. A future scheme whose VALUE differs + * by proxy-ness would add it back — and would then have something to assert about it. + * + * @param challenge - the challenge being answered; `canHandle` has already passed. + * @param request - the request being stamped. Optional so a handler needing neither method nor + * target (Basic) is callable with one argument; Digest asserts its presence. + * @returns the header value. + */ + stamp(challenge: Challenge, request?: DigestUriContext): Promise<string>; + + /** + * AUTH-16's "earliest in the configured preference list, not wire order": when a server offers + * several challenges a single handler could equally satisfy — RFC 7616 uses repeated Digest + * challenges differing only by `algorithm` as an algorithm-discovery mechanism — `canHandle` alone + * can only answer yes/no per challenge, never express a preference among them. + * + * Lower is more preferred. Optional: a handler with no algorithm variants (Basic) omits it and is + * treated as rank 0. + * + * @param challenge - the offered challenge. + * @returns the preference rank; lower wins. + */ + rank?(challenge: Challenge): number; +} + +/** + * AUTH-12 names this key literally: a token68 value is "recorded under a synthetic key", spelled + * `token68` in the requirement's own text. A genuine `token68=...` auth-param does not exist in RFC + * 7235's grammar — token68 is positional, never `name=value` — so no collision with a real parameter + * is possible. + */ +const TOKEN68_KEY = 'token68'; +const TOKEN_CHAR = /[!#$%&'*+\-.^_`|~0-9A-Za-z]/u; +const TOKEN68_CHAR = /[A-Za-z0-9\-._~+/]/u; + +interface Scanner { + readonly text: string; + pos: number; +} + +/** + * The one place BWS (RFC 7230's optional SP/HTAB run) is skipped, shared by the Scanner-driven walk + * and the two lookahead predicates below -- which cannot take a `Scanner`, because they must not + * advance one. + */ +function skipBwsFrom(text: string, from: number): number { + let index = from; + while (index < text.length && (text[index] === ' ' || text[index] === '\t')) { + index += 1; + } + return index; +} + +function skipSpaces(scanner: Scanner): void { + scanner.pos = skipBwsFrom(scanner.text, scanner.pos); +} + +function readToken(scanner: Scanner): string { + const start = scanner.pos; + while ( + scanner.pos < scanner.text.length && + TOKEN_CHAR.test(scanner.text[scanner.pos] ?? '') + ) + scanner.pos += 1; + return scanner.text.slice(start, scanner.pos); +} + +function readToken68Tail(scanner: Scanner): string { + const start = scanner.pos; + while (scanner.pos < scanner.text.length) { + const char = scanner.text[scanner.pos] ?? ''; + // '=' is token68's own padding, not an assignment: the caller only reaches here after ruling out + // a `name=value` reading. + if (!TOKEN68_CHAR.test(char) && char !== '=') break; + scanner.pos += 1; + } + return scanner.text.slice(start, scanner.pos); +} + +/** + * Honors backslash escapes (AUTH-12). An unterminated string ends at end-of-input rather than + * throwing (AUTH-13). + */ +function readQuotedString(scanner: Scanner): string { + scanner.pos += 1; // opening quote, already confirmed present by the caller + let value = ''; + while (scanner.pos < scanner.text.length) { + // `?? ''` throughout, not a cast: `noUncheckedIndexedAccess` types every index read as + // `string | undefined`, and the loop bound already rules the undefined out. + const char = scanner.text[scanner.pos] ?? ''; + if (char === '\\' && scanner.pos + 1 < scanner.text.length) { + value += scanner.text[scanner.pos + 1] ?? ''; + scanner.pos += 2; + continue; + } + if (char === '"') { + scanner.pos += 1; + return value; + } + value += char; + scanner.pos += 1; + } + return value; +} + +/** + * Consumes, without capturing, up to and including the next top-level comma — the recovery path for a + * malformed segment (AUTH-13). Quote depth is tracked, so a comma inside a quoted value is not a + * recovery point. + */ +function skipToNextTopLevelComma(scanner: Scanner): void { + let inQuotes = false; + while (scanner.pos < scanner.text.length) { + const char = scanner.text[scanner.pos]; + if (char === '\\' && inQuotes && scanner.pos + 1 < scanner.text.length) { + scanner.pos += 2; + continue; + } + if (char === '"') inQuotes = !inQuotes; + else if (char === ',' && !inQuotes) { + scanner.pos += 1; + return; + } + scanner.pos += 1; + } +} + +/** Whether the next non-whitespace character is the `=` of a `name=value` auth-param. */ +function peekIsParamAssignment(text: string, fromPos: number): boolean { + return text[skipBwsFrom(text, fromPos)] === '='; +} + +/** + * Whether a `name=value` reading is viable: an `=` followed, after optional BWS, by a real token or + * quoted-string value. + * + * Only the position immediately after a scheme name needs this stricter test, because that is the one + * place RFC 7235 permits a positional `token68` — and a token68 may END in one or more `=` (base64 + * padding), so `Negotiate YWJj==` would otherwise be misread as an auth-param `ywjj` with an empty + * value. Everywhere else inside a challenge the loose `=` test stands, so a genuinely empty auth-param + * value stays an empty auth-param rather than being re-read as a token68 that cannot appear there. + */ +function peekIsValuedParam(text: string, fromPos: number): boolean { + const equalsAt = skipBwsFrom(text, fromPos); + if (text[equalsAt] !== '=') return false; + const next = text[skipBwsFrom(text, equalsAt + 1)] ?? ''; + return next === '"' || TOKEN_CHAR.test(next); +} + +function readValue(scanner: Scanner): string { + return scanner.text[scanner.pos] === '"' + ? readQuotedString(scanner) + : readToken(scanner); +} + +interface MutableChallenge { + readonly scheme: string; + readonly params: Map<string, string>; +} + +/** Reads one `name=value` pair into `current`. The caller has already confirmed the `=` follows. */ +function readParamInto( + scanner: Scanner, + name: string, + current: MutableChallenge, +): void { + skipSpaces(scanner); + scanner.pos += 1; // '=' + skipSpaces(scanner); + current.params.set(name.toLowerCase(), readValue(scanner)); + skipSpaces(scanner); + if (scanner.text[scanner.pos] === ',') scanner.pos += 1; +} + +/** Reads the optional token68-or-first-param tail immediately following a freshly-read scheme name. */ +function readSchemeTail(scanner: Scanner, current: MutableChallenge): void { + skipSpaces(scanner); + if (scanner.pos >= scanner.text.length || scanner.text[scanner.pos] === ',') + return; + const savedPos = scanner.pos; + const maybeName = readToken(scanner); + if (maybeName !== '' && peekIsValuedParam(scanner.text, scanner.pos)) { + readParamInto(scanner, maybeName, current); + return; + } + scanner.pos = savedPos; + const token68 = readToken68Tail(scanner); + if (token68 !== '') current.params.set(TOKEN68_KEY, token68); + skipSpaces(scanner); + if (scanner.text[scanner.pos] === ',') scanner.pos += 1; +} + +/** + * Parses an RFC 7235 `WWW-Authenticate`/`Proxy-Authenticate` value into its ordered challenge list + * (AUTH-12). + * + * Total by construction (AUTH-13): it never throws, for any input. Blank input yields `[]`; a + * malformed segment recovers at the next top-level comma through a quote-depth-tracked scan — never a + * naive `.split(',')`, which breaks on a quoted value containing a comma; params parsed before a + * malformed tail are kept; an unterminated quoted string terminates at end-of-input. + * + * Hand-written for the same reason 5a's RFC 1123 `Retry-After` date parser was: there is no built-in + * RFC 7235 parser to lean on, and a general-purpose header splitter would not honor quoted-string + * commas. + * + * @param headerValue - the raw header value. + * @returns the challenges, in wire order. Never throws. + * + * @internal + */ +export function parseChallenges(headerValue: string): readonly Challenge[] { + const challenges: MutableChallenge[] = []; + const scanner: Scanner = {text: headerValue, pos: 0}; + + for (;;) { + skipSpaces(scanner); + if (scanner.pos >= scanner.text.length) break; + if (scanner.text[scanner.pos] === ',') { + scanner.pos += 1; + continue; + } + + const token = readToken(scanner); + if (token === '') { + skipToNextTopLevelComma(scanner); + continue; + } + + if (peekIsParamAssignment(scanner.text, scanner.pos)) { + // A `name=value` with no scheme ahead of it: attach it to the challenge in progress, or discard + // the segment when the header opens with one. + const current = challenges.at(-1); + if (current === undefined) { + skipToNextTopLevelComma(scanner); + continue; + } + readParamInto(scanner, token, current); + continue; + } + + const current: MutableChallenge = { + scheme: token.toLowerCase(), + params: new Map(), + }; + challenges.push(current); + readSchemeTail(scanner, current); + } + + // `Object.freeze` is SHALLOW, and `params` is a `Map`, which `Object.freeze` cannot make read-only + // at all -- `.set()` still succeeds on a frozen Map. The `ReadonlyMap` TYPE is therefore the only + // guard on the parameters, exactly as `createAuthRequirement` documents for its own params map, and + // it holds for the same reason: `Challenge` is `@internal`, never reaches a consumer, and nothing + // in this package re-casts `Challenge['params']` back to `Map`. + return challenges.map(entry => + Object.freeze({scheme: entry.scheme, params: entry.params}), + ); +} diff --git a/packages/core/src/auth/composing-handler.test.ts b/packages/core/src/auth/composing-handler.test.ts new file mode 100644 index 0000000..ca11957 --- /dev/null +++ b/packages/core/src/auth/composing-handler.test.ts @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/composing-handler.test.ts +// Exercises: AUTH-23 (ordered handler list, defensively copied; first configured handler wins), +// AUTH-24 (handler order beats wire-order challenge position), AUTH-25 (returns the value half only +// -- no header-name decision here, and no header at all when nothing is satisfiable), and the +// rank-based tie-break that carries AUTH-16's algorithm preference. +import {describe, expect, test} from 'bun:test'; +import type {Challenge, ChallengeHandler} from './challenge.js'; +import {composingHandler} from './composing-handler.js'; + +function fakeHandler( + scheme: string, + value: string, + rank = 0, +): ChallengeHandler { + return { + canHandle: (challenge: Challenge): boolean => challenge.scheme === scheme, + stamp: (): Promise<string> => Promise.resolve(value), + rank: (): number => rank, + }; +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this runner's + * type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper keeps the + * assertion honest without a lint suppression. Same shape 5a's and 5b's step suites settled on. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('composingHandler', () => { + test('delegates to the first CONFIGURED handler that can satisfy any offered challenge', async () => { + const handler = composingHandler([ + fakeHandler('digest', 'digest-value'), + fakeHandler('basic', 'basic-value'), + ]); + const challenges: readonly Challenge[] = [ + {scheme: 'basic', params: new Map()}, + {scheme: 'digest', params: new Map()}, + ]; + // basic appears FIRST on the wire, but digest's HANDLER is configured first -- handler order wins. + expect(await handler.stamp(challenges)).toBe('digest-value'); + }); + + test('falls through to a later handler when the first cannot satisfy anything offered', async () => { + const handler = composingHandler([ + fakeHandler('digest', 'digest-value'), + fakeHandler('basic', 'basic-value'), + ]); + expect(await handler.stamp([{scheme: 'basic', params: new Map()}])).toBe( + 'basic-value', + ); + }); + + test('returns undefined when no handler can satisfy any offered challenge', async () => { + const handler = composingHandler([fakeHandler('digest', 'x')]); + expect( + await handler.stamp([{scheme: 'basic', params: new Map()}]), + ).toBeUndefined(); + }); + + test('returns undefined for an empty challenge list', async () => { + const handler = composingHandler([fakeHandler('digest', 'x')]); + expect(await handler.stamp([])).toBeUndefined(); + }); + + test('returns undefined when no handlers are configured at all', async () => { + const handler = composingHandler([]); + expect( + await handler.stamp([{scheme: 'digest', params: new Map()}]), + ).toBeUndefined(); + }); +}); + +describe('composingHandler ranking and delegation (AUTH-16/AUTH-23)', () => { + test('within one handler satisfying multiple challenges, rank breaks the tie', async () => { + const digestLike: ChallengeHandler = { + canHandle: challenge => challenge.scheme === 'digest', + stamp: challenge => + Promise.resolve( + `value-for-${challenge.params.get('algorithm') ?? 'default'}`, + ), + rank: challenge => + challenge.params.get('algorithm') === 'SHA-256' ? 0 : 1, + }; + const handler = composingHandler([digestLike]); + const challenges: readonly Challenge[] = [ + {scheme: 'digest', params: new Map([['algorithm', 'MD5']])}, + {scheme: 'digest', params: new Map([['algorithm', 'SHA-256']])}, + ]; + expect(await handler.stamp(challenges)).toBe('value-for-SHA-256'); + }); + + test('rank never outranks handler order -- a worse-ranked earlier handler still wins', async () => { + const handler = composingHandler([ + fakeHandler('digest', 'digest-value', 99), + fakeHandler('basic', 'basic-value', 0), + ]); + const challenges: readonly Challenge[] = [ + {scheme: 'basic', params: new Map()}, + {scheme: 'digest', params: new Map()}, + ]; + expect(await handler.stamp(challenges)).toBe('digest-value'); + }); + + test('a handler with no rank() defaults to 0 and does not crash the sort', async () => { + const noRank: ChallengeHandler = { + canHandle: (): boolean => true, + stamp: (): Promise<string> => Promise.resolve('no-rank-value'), + }; + const handler = composingHandler([noRank]); + expect(await handler.stamp([{scheme: 'anything', params: new Map()}])).toBe( + 'no-rank-value', + ); + }); +}); + +describe('composingHandler isolation and error propagation (AUTH-23)', () => { + test('defensively copies the handler list at construction (AUTH-23)', async () => { + const handlers = [fakeHandler('basic', 'v1')]; + const handler = composingHandler(handlers); + handlers.push(fakeHandler('digest', 'v2')); + // 'digest' was pushed after construction, so the composed handler must not see it. + expect( + await handler.stamp([{scheme: 'digest', params: new Map()}]), + ).toBeUndefined(); + }); + + // There is deliberately no companion test for an `isProxy` flag. One was threaded through this + // composer into both handlers and NEITHER read it, so the only assertion either could carry was + // that it changed nothing; AUTH-25's origin-vs-proxy choice lives in `auth-step.ts`'s + // `answerHeaderName`, which is where it is actually asserted. + test('passes the request context through to the winning handler', async () => { + let observedRequest: {method: string; requestTarget: string} | undefined; + const recorder: ChallengeHandler = { + canHandle: (): boolean => true, + stamp: (_challenge, request): Promise<string> => { + observedRequest = request; + return Promise.resolve('v'); + }, + }; + const handler = composingHandler([recorder]); + await handler.stamp([{scheme: 'x', params: new Map()}], { + method: 'GET', + requestTarget: '/y', + }); + expect(observedRequest).toEqual({method: 'GET', requestTarget: '/y'}); + }); + + test("a rejecting handler's failure propagates rather than being swallowed as 'no replacement'", async () => { + const boom: ChallengeHandler = { + canHandle: (): boolean => true, + stamp: (): Promise<string> => + Promise.reject(new Error('handler blew up')), + }; + const error = await rejectionOf( + composingHandler([boom]).stamp([{scheme: 'x', params: new Map()}]), + ); + expect((error as Error).message).toBe('handler blew up'); + }); +}); diff --git a/packages/core/src/auth/composing-handler.ts b/packages/core/src/auth/composing-handler.ts new file mode 100644 index 0000000..8fa9012 --- /dev/null +++ b/packages/core/src/auth/composing-handler.ts @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/composing-handler.ts +import type { + Challenge, + ChallengeHandler, + DigestUriContext, +} from './challenge.js'; + +/** + * Ordered delegation over a fixed handler list (AUTH-23–AUTH-25). + * + * @internal + */ +export interface ComposingHandler { + /** + * Answers the best challenge any configured handler can satisfy. + * + * @param challenges - every challenge the response offered, in wire order. + * @param request - the request being stamped, for handlers that need method and target. + * @returns the header VALUE, or `undefined` when no handler can satisfy any offered challenge — + * which the auth step reads as "no replacement request" (AUTH-25). + */ + stamp( + challenges: readonly Challenge[], + request?: DigestUriContext, + ): Promise<string | undefined>; +} + +interface Candidate { + readonly handlerIndex: number; + readonly rank: number; + readonly handler: ChallengeHandler; + readonly challenge: Challenge; +} + +function collectCandidates( + handlers: readonly ChallengeHandler[], + challenges: readonly Challenge[], +): Candidate[] { + const candidates: Candidate[] = []; + handlers.forEach((handler, handlerIndex) => { + for (const challenge of challenges) { + if (handler.canHandle(challenge)) { + candidates.push({ + handlerIndex, + rank: handler.rank?.(challenge) ?? 0, + handler, + challenge, + }); + } + } + }); + return candidates; +} + +/** + * AUTH-23: handler CONFIGURATION order is the primary key — "the first handler in declaration order + * whose can-handle check passes" wins regardless of where its satisfiable challenge sits on the wire. + * `rank` is the secondary key, carrying AUTH-16's algorithm-preference-over-wire-order rule within a + * single handler. + */ +function bestCandidate( + candidates: readonly Candidate[], +): Candidate | undefined { + return [...candidates].sort( + (a, b) => a.handlerIndex - b.handlerIndex || a.rank - b.rank, + )[0]; +} + +/** + * Composes an ordered handler list into one challenge answerer (AUTH-23–AUTH-25). + * + * The list is defensively copied at construction (AUTH-23), so a caller mutating its array afterwards + * cannot change which handlers this composer consults. Callers order stronger schemes first — the + * auth step builds `[digest, basic]`. + * + * Returns `undefined` — meaning "no replacement request" — when no handler can satisfy any offered + * challenge (AUTH-25). It never throws: an unsatisfiable challenge is an ordinary outcome the auth + * step turns into "leave the 401 unchanged" (AUTH-33), not an error condition. + * + * AUTH-25's `Authorization`-vs-`Proxy-Authorization` choice is NOT threaded through here: this + * composer and both handlers produce the VALUE half only, and `auth-step.ts` picks the header name + * from which challenge header the status actually carried. + * + * Handlers are stateless apart from Digest's per-nonce counter, which is safe for concurrent + * invocation (AUTH-24). + * + * @param handlers - the handlers, strongest first. + * @returns the composed handler. + * + * @internal + */ +export function composingHandler( + handlers: readonly ChallengeHandler[], +): ComposingHandler { + const configured = [...handlers]; + return { + stamp: async (challenges, request): Promise<string | undefined> => { + const candidate = bestCandidate( + collectCandidates(configured, challenges), + ); + if (candidate === undefined) return undefined; + return candidate.handler.stamp(candidate.challenge, request); + }, + }; +} diff --git a/packages/core/src/auth/credential.test.ts b/packages/core/src/auth/credential.test.ts new file mode 100644 index 0000000..f15189e --- /dev/null +++ b/packages/core/src/auth/credential.test.ts @@ -0,0 +1,346 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/credential.test.ts +// Exercises: AUTH-8 (BearerToken is value-equal; ApiKeyCredential/NameKeyCredential are +// reference-equal via bare `===`, no equals() override; ALL FIVE credential types -- bearer, API key, +// name key, Basic and Digest -- redact their secret in every string/diagnostic form and none is +// reachable through JSON.stringify/Object.keys, including when a whole AuthCredentialSet is handed to +// `util.inspect`), AUTH-9 (blank +// rejected as a programmer error, and every type is nominal so the validation cannot be routed +// around), AUTH-10 (expiry math: undefined never locally expires; expired iff nowMs + marginMs > +// expiresAt), AUTH-26 (`credentialKey()` is the sole read path for a static key's secret), +// AUTH-14/AUTH-16 (the Basic and Digest credentials carry the username and the algorithm preference +// as non-secret fields, which AUTH-8 permits to stay visible). +import {describe, expect, test} from 'bun:test'; +// The ONE `node:` import in this package's tests, and it is the point of the AuthCredentialSet rows +// below: the reported leak was `util.inspect(credentials)` printing `password: 'hunter2'`, so the row +// that proves it closed has to drive the real `util.inspect`, not the hook it happens to call. +import {inspect} from 'node:util'; +import {InvariantViolation} from '../invariant.js'; +import type {AuthCredentialSet} from './auth-step.js'; +import type {DigestAlgorithm} from './digest.js'; +import { + ApiKeyCredential, + BasicCredential, + BearerToken, + DigestCredential, + NameKeyCredential, + bearerTokensEqual, + createBearerToken, + credentialKey, + credentialPassword, + isBearerTokenExpired, +} from './credential.js'; + +// `util.inspect` is not imported: nothing else in `packages/core` reaches for a `node:` module, and +// the hook under test is reachable directly. `console.log`/`util.inspect` call exactly this method. +const INSPECT = Symbol.for('nodejs.util.inspect.custom'); + +function inspectOf( + value: + | ApiKeyCredential + | BasicCredential + | BearerToken + | DigestCredential + | NameKeyCredential, +): string { + const hooks = value as unknown as Record<symbol, (() => string) | undefined>; + const hook = hooks[INSPECT]; + expect(typeof hook).toBe('function'); + return hook === undefined ? '<no inspect hook>' : hook.call(value); +} + +describe('BearerToken', () => { + test('value equality over token + expiry', () => { + const a = createBearerToken('t', 1000); + const b = createBearerToken('t', 1000); + expect(bearerTokensEqual(a, b)).toBe(true); + }); + + test('differing token or expiry is not equal', () => { + expect( + bearerTokensEqual(createBearerToken('a'), createBearerToken('b')), + ).toBe(false); + expect( + bearerTokensEqual(createBearerToken('t', 1), createBearerToken('t', 2)), + ).toBe(false); + }); + + test('an absent expiry and a set expiry are not equal', () => { + expect( + bearerTokensEqual(createBearerToken('t'), createBearerToken('t', 1)), + ).toBe(false); + }); + + test('is frozen', () => { + expect(Object.isFrozen(createBearerToken('t'))).toBe(true); + }); + + test('rejects a blank or whitespace-only token (AUTH-9)', () => { + expect(() => createBearerToken('')).toThrow(InvariantViolation); + expect(() => createBearerToken(' ')).toThrow(InvariantViolation); + }); + + test('rejects a non-finite expiresAt', () => { + // `isBearerTokenExpired` is `nowMs + marginMs > expiresAt`, so a NaN expiry makes every + // comparison false: the token reads as permanently fresh, the cache serves it from the hot path + // forever, and no provider call ever happens to notice. Rejected at construction rather than + // discovered as a dead credential in production. + expect(() => createBearerToken('t', Number.NaN)).toThrow( + 'expiresAt must be a finite epoch', + ); + expect(() => createBearerToken('t', Number.POSITIVE_INFINITY)).toThrow( + 'expiresAt must be a finite epoch', + ); + }); + + test('undefined expiresAt never locally expires (AUTH-10)', () => { + const token = createBearerToken('t'); + expect(isBearerTokenExpired(token, Number.MAX_SAFE_INTEGER, 0)).toBe(false); + expect(isBearerTokenExpired(token, Number.MAX_SAFE_INTEGER, 60_000)).toBe( + false, + ); + }); + + test('expired iff nowMs + marginMs > expiresAt (AUTH-10)', () => { + const token = createBearerToken('t', 1000); + expect(isBearerTokenExpired(token, 999, 0)).toBe(false); + expect(isBearerTokenExpired(token, 1000, 0)).toBe(false); // exactly at expiry, not yet past it + expect(isBearerTokenExpired(token, 1001, 0)).toBe(true); + expect(isBearerTokenExpired(token, 900, 200)).toBe(true); // margin pushes it over + }); +}); + +describe('BearerToken redaction and nominality (AUTH-8/AUTH-9)', () => { + test('toString and inspect redact the token but keep the expiry', () => { + const token = createBearerToken('super-secret', 1000); + expect(token.toString()).not.toContain('super-secret'); + expect(String(token)).not.toContain('super-secret'); + expect(token.toString()).toContain('1000'); + expect(inspectOf(token)).not.toContain('super-secret'); + }); + + test('JSON.stringify and Object.keys cannot reach the token -- #private, not TS private', () => { + const token = createBearerToken('super-secret', 1000); + expect(JSON.stringify(token)).not.toContain('super-secret'); + expect(Object.keys(token)).toEqual(['expiresAt']); + }); + + test('the accessor still hands the real token to the stamping path', () => { + expect(createBearerToken('super-secret').token).toBe('super-secret'); + }); + + test('nominal: an object literal is not a BearerToken, so AUTH-9 cannot be bypassed', () => { + // The compile-time half is the point -- a `TokenProvider` returning `{token: '', expiresAt: + // undefined}` no longer type-checks -- and this asserts the runtime half: the constructor is + // private, so `createBearerToken` (which validates) is the only construction path. + const literal = {token: '', expiresAt: undefined}; + expect(literal instanceof BearerToken).toBe(false); + expect(createBearerToken('t') instanceof BearerToken).toBe(true); + }); +}); + +describe('ApiKeyCredential (AUTH-8)', () => { + test('two instances with identical fields are NOT equal -- reference identity only', () => { + expect( + new ApiKeyCredential('secret') === new ApiKeyCredential('secret'), + ).toBe(false); + }); + + test('toString and inspect redact the key', () => { + const credential = new ApiKeyCredential('super-secret'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify cannot reach the key either -- #private, not TS private', () => { + expect(JSON.stringify(new ApiKeyCredential('super-secret'))).not.toContain( + 'super-secret', + ); + expect(Object.keys(new ApiKeyCredential('super-secret'))).toEqual([]); + }); + + test('rejects a blank or whitespace-only key (AUTH-9)', () => { + expect(() => new ApiKeyCredential('')).toThrow(InvariantViolation); + expect(() => new ApiKeyCredential(' ')).toThrow(InvariantViolation); + }); + + test('the secret is reachable ONLY through the internal credentialKey() hook', () => { + const credential = new ApiKeyCredential('secret'); + expect(credentialKey(credential)).toBe('secret'); + // No public `key` accessor: the friend hook is the whole read path, so the secret never appears + // on the published surface (AUTH-8). + expect('key' in credential).toBe(false); + }); +}); + +describe('NameKeyCredential (AUTH-8)', () => { + test('two instances with identical fields are NOT equal', () => { + expect( + new NameKeyCredential('n', 'k') === new NameKeyCredential('n', 'k'), + ).toBe(false); + }); + + test('toString redacts the key but names the name', () => { + const credential = new NameKeyCredential('x-api-key', 'super-secret'); + expect(credential.toString()).toContain('x-api-key'); + expect(credential.toString()).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('rejects a blank or whitespace-only name or key (AUTH-9)', () => { + expect(() => new NameKeyCredential('', 'k')).toThrow(InvariantViolation); + expect(() => new NameKeyCredential('n', '')).toThrow(InvariantViolation); + expect(() => new NameKeyCredential(' ', 'k')).toThrow(InvariantViolation); + }); + + test('the secret is reachable only through credentialKey(); .name stays public', () => { + const credential = new NameKeyCredential('x-api-key', 'secret'); + expect(credentialKey(credential)).toBe('secret'); + expect(credential.name).toBe('x-api-key'); + expect('key' in credential).toBe(false); + }); + + test('JSON.stringify reaches the name but never the key', () => { + const serialized = JSON.stringify( + new NameKeyCredential('x-api-key', 'super-secret'), + ); + expect(serialized).toContain('x-api-key'); + expect(serialized).not.toContain('super-secret'); + }); +}); + +describe('BasicCredential (AUTH-8/AUTH-14)', () => { + test('two instances with identical fields are NOT equal -- reference identity only', () => { + expect( + new BasicCredential('u', 'p') === new BasicCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep the username', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credential.toString()).toContain('alice'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(JSON.stringify(credential)).toContain('alice'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual(['username']); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + // No public `password` property: that is exactly what the structural interface this class + // replaced put on the published surface (AUTH-8). + expect('password' in credential).toBe(false); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new BasicCredential('u', 'p'))).toBe(true); + }); +}); + +describe('DigestCredential (AUTH-8/AUTH-16)', () => { + test('two instances with identical fields are NOT equal', () => { + expect( + new DigestCredential('u', 'p') === new DigestCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep username and preference', () => { + const credential = new DigestCredential('bob', 'super-secret', ['SHA-256']); + expect(credential.toString()).toContain('bob'); + expect(credential.toString()).toContain('SHA-256'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(JSON.stringify(credential)).toContain('bob'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual([ + 'username', + 'algorithmPreference', + ]); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + expect('password' in credential).toBe(false); + }); + + test('an omitted algorithmPreference stays undefined -- the handler owns the default', () => { + // `digestHandler` applies AUTH-16's strongest-first default. Materializing it here would put a + // second copy of that list on the public surface, free to drift from the one that is used. + expect( + new DigestCredential('bob', 'p').algorithmPreference, + ).toBeUndefined(); + }); + + test('the algorithm preference is copied and frozen, not aliased (HTTP-3)', () => { + const supplied: DigestAlgorithm[] = ['SHA-256', 'MD5']; + const credential = new DigestCredential('bob', 'p', supplied); + supplied.push('MD5-sess'); + + expect(credential.algorithmPreference).toEqual(['SHA-256', 'MD5']); + expect(Object.isFrozen(credential.algorithmPreference)).toBe(true); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new DigestCredential('u', 'p'))).toBe(true); + }); +}); + +describe('a whole AuthCredentialSet is diagnostic-safe (AUTH-8)', () => { + /** Every scheme's material at once -- the shape a caller hands `authStep()`. */ + function everyCredential(): AuthCredentialSet { + return { + basic: new BasicCredential('alice', 'basic-hunter2'), + digest: new DigestCredential('bob', 'digest-hunter2', ['SHA-256']), + apiKey: { + credential: new NameKeyCredential('x-api-key', 'api-key-hunter2'), + headerName: 'X-Api-Key', + }, + bearer: { + provider: () => + Promise.resolve(createBearerToken('bearer-hunter2', 1000)), + }, + }; + } + + test('util.inspect prints no secret from any of the four schemes', () => { + const rendered = inspect(everyCredential(), {depth: null}); + + expect(rendered).not.toContain('basic-hunter2'); + expect(rendered).not.toContain('digest-hunter2'); + expect(rendered).not.toContain('api-key-hunter2'); + // The non-secret fields AUTH-8 permits to stay visible are still there, which is what makes the + // rendering worth printing at all. + expect(rendered).toContain('alice'); + expect(rendered).toContain('bob'); + expect(rendered).toContain('x-api-key'); + }); + + test('util.inspect honours the inspect hook on a BearerToken too', () => { + // `inspectOf` proves the hook exists; this proves `util.inspect` actually calls it. + expect(inspect(createBearerToken('bearer-hunter2', 1000))).not.toContain( + 'bearer-hunter2', + ); + }); + + test('JSON.stringify serializes no secret from any of the four schemes', () => { + const serialized = JSON.stringify(everyCredential()); + + expect(serialized).not.toContain('basic-hunter2'); + expect(serialized).not.toContain('digest-hunter2'); + expect(serialized).not.toContain('api-key-hunter2'); + }); +}); diff --git a/packages/core/src/auth/credential.ts b/packages/core/src/auth/credential.ts new file mode 100644 index 0000000..93d95df --- /dev/null +++ b/packages/core/src/auth/credential.ts @@ -0,0 +1,484 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/credential.ts +import {invariant} from '../invariant.js'; +import type {DigestAlgorithm} from './digest.js'; + +// No `as unique symbol` cast: TypeScript rejects `unique symbol` in a type assertion (TS1335). A +// `const` initialized directly by a `Symbol.for()` call already gets the `unique symbol` type, which +// is what makes it usable as a computed member name below. +const INSPECT: unique symbol = Symbol.for('nodejs.util.inspect.custom'); + +/** + * TypeScript has no friend classes, so {@link createBearerToken} -- a plain function, not a member -- + * reaches the private constructor through this module-scoped `let`, assigned exactly once inside the + * class's `static {}` block. Init-once wiring, not mutable state; the same shape every builder-based + * model in `src/http/` uses. + */ +let createToken: (token: string, expiresAt: number | undefined) => BearerToken; + +/** + * An OAuth2 bearer token (AUTH-8, AUTH-9, AUTH-10). + * + * A class with `#token`, not a frozen data object, for two reasons AUTH-8 and AUTH-9 make between + * them: + * + * - **Redaction.** AUTH-8 requires EVERY credential type to redact its secret in any + * string/diagnostic representation. A plain `{token, expiresAt}` object redacts nothing: + * `console.log` prints the token, `JSON.stringify` serializes it, and any structured logger walking + * the object graph carries it into a log sink. `#token` is unreachable to all three, and the + * `toString`/inspect pair below gives the redacted form those paths fall back to. `expiresAt` stays + * an ordinary public field -- AUTH-8 explicitly permits non-secret fields to remain visible. + * - **Nominality.** `TokenProvider` returns a `BearerToken`, so with a structural interface a provider + * could hand back an object literal and bypass AUTH-9's non-blank validation entirely. `#token` + * makes the type nominal and the `private` constructor makes {@link createBearerToken} the only way + * to build one, so the validation cannot be routed around. + * + * AUTH-8's VALUE equality is unaffected: it lives in {@link bearerTokensEqual}, a pure function over + * the two fields, exactly as it did when this was a data object. There is deliberately no `equals` + * member -- the key credentials below need reference identity, and keeping equality out of both + * classes keeps that distinction in one place. + * + * @public + */ +export class BearerToken { + readonly #token: string; + /** Epoch ms; `undefined` means "never locally expires" (AUTH-10). Non-secret, so visible. */ + readonly expiresAt: number | undefined; + + private constructor(token: string, expiresAt: number | undefined) { + this.#token = token; + this.expiresAt = expiresAt; + Object.freeze(this); + } + + static { + createToken = (token, expiresAt) => new BearerToken(token, expiresAt); + } + + /** The opaque token, never blank (AUTH-9). Read by the stamping path that writes the header. */ + get token(): string { + return this.#token; + } + + /** + * AUTH-8's redacted string form. The expiry survives; the token does not. + * + * @returns the representation with the token masked. + */ + toString(): string { + return `BearerToken{token=***, expiresAt=${String(this.expiresAt)}}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`, which do not route object arguments through + * `toString`. Node-specific but harmless elsewhere -- an unrecognized well-known symbol is simply + * never read. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * Builds a frozen {@link BearerToken}. The only way to construct one: the constructor is `private`, so + * AUTH-9's validation below cannot be bypassed by a provider returning a hand-built value. + * + * @param token - the opaque token. Must not be blank. + * @param expiresAt - epoch ms at which the token expires, or `undefined` for "never locally expires". + * When present it must be a finite number. + * @returns the frozen token. + * @throws an assertion failure (a caller bug, not a catchable condition) when `token` is blank (AUTH-9), or when `expiresAt` is present but not + * finite -- both caller misconfigurations. + * + * @public + */ +export function createBearerToken( + token: string, + expiresAt?: number, +): BearerToken { + invariant(token.trim().length > 0, 'bearer token must not be blank'); // AUTH-9 + // A `NaN` expiry makes every comparison in `isBearerTokenExpired` false, so the token reads as + // permanently fresh and the cache stamps a dead credential forever -- silently, and with no + // provider call to notice. Rejected at construction for the same reason `authStep` rejects a + // non-finite margin: it is the identical bug arriving through the other door. + invariant( + expiresAt === undefined || Number.isFinite(expiresAt), + `bearer token expiresAt must be a finite epoch, got ${String(expiresAt)}`, + ); + return createToken(token, expiresAt); +} + +/** + * AUTH-8's VALUE equality for {@link BearerToken}, over the real token and expiry. + * + * A pure function rather than an `equals` member, deliberately: the two key credentials next door + * need REFERENCE identity, and keeping equality out of every credential class is what stops one of + * them acquiring value semantics by accident. The redacted string form has no bearing on it. + * + * @param a - the left token. + * @param b - the right token. + * @returns `true` when token and expiry both match. + * + * @public + */ +export function bearerTokensEqual(a: BearerToken, b: BearerToken): boolean { + return a.token === b.token && a.expiresAt === b.expiresAt; +} + +/** + * AUTH-10: expired at `nowMs` with grace margin `marginMs` if and only if an expiry is set and + * `nowMs + marginMs` strictly exceeds it. An exact hit on the expiry instant is NOT yet expired. + * + * @param token - the token to test. + * @param nowMs - the reference time, epoch ms. + * @param marginMs - the refresh grace margin in ms; `0` evaluates true expiry. + * @returns `true` when the token is expired under that margin. + * + * @internal + */ +export function isBearerTokenExpired( + token: BearerToken, + nowMs: number, + marginMs: number, +): boolean { + return token.expiresAt !== undefined && nowMs + marginMs > token.expiresAt; +} + +/** + * The friend-class hooks for the two key credentials' secrets. `static-key.ts` -- a different module, + * and the ONLY sanctioned reader -- reaches them through {@link credentialKey}. + * + * A public `get key()` would have been simpler and was the first shape here, but it re-opens exactly + * the leak the `#key` note below argues against: it puts the secret back on the published `.d.ts`, + * reachable as `credential.key` by any consumer, any diagnostic helper walking accessors, and any + * future logging step. Init-once wiring assigned in each class's `static {}` block, not mutable state. + */ +let readApiKey: (credential: ApiKeyCredential) => string; +let readNameKey: (credential: NameKeyCredential) => string; + +/** + * The in-package read hook for a static key credential's secret (AUTH-26's stamping path). + * + * Exported (still internal-only, absent from the package barrel) because the one caller -- + * `stampStaticKey` -- lives in another module and TypeScript has no friend-class visibility to + * express that with. + * + * @param credential - the credential whose secret is being stamped. + * @returns the raw key. + * + * @internal + */ +export function credentialKey( + credential: ApiKeyCredential | NameKeyCredential, +): string { + return credential instanceof ApiKeyCredential + ? readApiKey(credential) + : readNameKey(credential); +} + +/** + * A static API key (AUTH-8, AUTH-9, AUTH-26). + * + * AUTH-8 requires REFERENCE equality here — "two instances with identical fields are NOT equal" — so + * this is a class with a private field and deliberately NO `equals` override: `===`, the language + * default, already gives exactly those semantics. + * + * `#key`, not `private key`, is the deliberate exception to `docs/knowledge/harvested/data-modeling.md`'s + * `private`-by-default rule, and the same note requires the justification be written down: AUTH-8's + * redaction is a RUNTIME-privacy requirement, not a compile-time one. `private` is erased, leaving the + * secret reachable through `credential['key']`, `Object.keys`, `JSON.stringify`, and a default + * `util.inspect` — exactly the accidental-leak paths the redacted `toString`/inspect exist to close. + * `#key` is genuinely unreachable, and the nominality it induces is load-bearing besides: it is what + * stops a caller substituting an object literal for a validated credential. + * + * @public + */ +export class ApiKeyCredential { + readonly #key: string; + + /** + * @param key - the secret key. Must not be blank. + * @throws an assertion failure (a caller bug, not a catchable condition) when `key` is blank (AUTH-9). + */ + constructor(key: string) { + invariant(key.trim().length > 0, 'ApiKeyCredential key must not be blank'); // AUTH-9 + this.#key = key; + } + + static { + readApiKey = credential => credential.#key; + } + + /** + * AUTH-8's redacted string form. + * + * @returns a fixed representation with the key masked. + */ + toString(): string { + return 'ApiKeyCredential{key=***}'; + } + + /** + * The same redaction for `console.log`/`util.inspect`, which do not route object arguments through + * `toString`. Node-specific but harmless elsewhere — an unrecognized well-known symbol is simply + * never read. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * A named static key — the header-name/secret pair AUTH-26 stamps (AUTH-8, AUTH-9). + * + * `#key` for the same runtime-privacy reason as {@link ApiKeyCredential}; `name` is non-secret, which + * AUTH-8 explicitly permits to stay visible, so it is an ordinary public field. + * + * @public + */ +export class NameKeyCredential { + /** The non-secret identifier — a header name, a key id. AUTH-8 permits this to stay visible. */ + readonly name: string; + readonly #key: string; + + /** + * @param name - the non-secret identifier. Must not be blank. + * @param key - the secret key. Must not be blank. + * @throws an assertion failure (a caller bug, not a catchable condition) when either is blank (AUTH-9). + */ + constructor(name: string, key: string) { + invariant( + name.trim().length > 0, + 'NameKeyCredential name must not be blank', + ); // AUTH-9 + invariant(key.trim().length > 0, 'NameKeyCredential key must not be blank'); // AUTH-9 + this.name = name; + this.#key = key; + } + + static { + readNameKey = credential => credential.#key; + } + + /** + * AUTH-8's redacted string form: the name survives, the key does not. + * + * @returns the representation with the key masked. + */ + toString(): string { + return `NameKeyCredential{name=${this.name}, key=***}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`. See {@link ApiKeyCredential} for why both + * hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * The friend-class hooks for the two password credentials' secrets, and the reason those two are + * classes at all. `auth-step.ts`'s `buildHandlers` -- the ONLY sanctioned reader -- reaches them + * through {@link credentialPassword}. + * + * They shipped as structural interfaces with a public `readonly password: string` until 2026-09-04, + * which put a live password on the published `.d.ts` and, worse, on the object graph: `util.inspect` + * of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and + * `JSON.stringify` serialized it. AUTH-8's redaction clause names bearer, API-key and name-key + * explicitly; reading it as covering EVERY credential type is a deliberate widening, recorded in + * `docs/deviations.md` (found by audit #67 / #71). + */ +let readBasicPassword: (credential: BasicCredential) => string; +let readDigestPassword: (credential: DigestCredential) => string; + +/** + * The in-package read hook for a Basic or Digest credential's password. + * + * Exported (still internal-only, absent from the package barrel) because the one caller -- + * `buildHandlers` in `auth-step.ts` -- lives in another module and TypeScript has no friend-class + * visibility to express that with. The same shape {@link credentialKey} uses for the static keys. + * + * @param credential - the credential whose password is about to reach a handler. + * @returns the raw password. + * + * @internal + */ +export function credentialPassword( + credential: BasicCredential | DigestCredential, +): string { + return credential instanceof BasicCredential + ? readBasicPassword(credential) + : readDigestPassword(credential); +} + +/** + * Username and password for the `BASIC` scheme (AUTH-8, AUTH-14). + * + * `#password`, not `readonly password`, for the runtime-privacy reason {@link ApiKeyCredential} + * states at length: `private` is erased and a plain property is reachable through + * `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect`. The + * redacted `toString`/inspect pair below is what those paths get instead. `username` is non-secret, + * which AUTH-8 explicitly permits to stay visible. + * + * Reference equality, like the two key credentials and for the same reason: there is deliberately no + * `equals` member anywhere in this module. + * + * **Validation is not repeated here.** AUTH-14's rule -- non-empty, whitespace permitted, which is + * deliberately laxer than `.trim().length > 0` -- lives in `basicHandler()`, and `authStep()` builds + * a handler for every configured credential at construction, so a blank password still fails + * synchronously from that factory. Restating the rule here would put a second copy of it one edit + * away from disagreeing with the copy that is actually applied to the wire. + * + * @public + */ +export class BasicCredential { + /** The user id (AUTH-14: non-empty; whitespace permitted). Non-secret, so visible. */ + readonly username: string; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + */ + constructor(username: string, password: string) { + this.username = username; + this.#password = password; + Object.freeze(this); + } + + static { + readBasicPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username survives, the password does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + return `BasicCredential{username=${this.username}, password=***}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`, which do not route object arguments through + * `toString`. See {@link ApiKeyCredential} for why both hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * Username, password, and algorithm preference for the `DIGEST` scheme (AUTH-8, AUTH-16). + * + * `#password` for the same runtime-privacy reason as {@link BasicCredential}. `username` and + * `algorithmPreference` are non-secret and stay visible. + * + * **Validation is not repeated here**, for the reason {@link BasicCredential} states: AUTH-16's + * acceptable-set rule and the blank/header-safety checks live in `digestHandler()`, which + * `authStep()` builds at construction. + * + * @public + */ +export class DigestCredential { + /** The user id. Non-secret, so visible. */ + readonly username: string; + /** + * Preferred-first order, and also the acceptable set (AUTH-16). `undefined` means strongest-first + * over all four supported algorithms -- the default is applied by `digestHandler()`, not + * materialized here, so there is only ever one copy of that list. + */ + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + * @param algorithmPreference - preferred-first acceptable algorithms; copied and frozen, so a + * caller mutating the array afterwards cannot change what this credential accepts (HTTP-3's + * no-aliasing rule, applied to the one collection this type holds). + */ + constructor( + username: string, + password: string, + algorithmPreference?: readonly DigestAlgorithm[], + ) { + this.username = username; + this.#password = password; + this.algorithmPreference = + algorithmPreference === undefined + ? undefined + : Object.freeze([...algorithmPreference]); + Object.freeze(this); + } + + static { + readDigestPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username and the algorithm preference survive, the password + * does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + const preference = + this.algorithmPreference === undefined + ? 'default' + : this.algorithmPreference.join('|'); + return `DigestCredential{username=${this.username}, password=***, algorithmPreference=${preference}}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`. See {@link ApiKeyCredential} for why both + * hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * AUTH-11: an async token source. A plain function type, no class. + * + * A throwing or rejecting provider propagates and is never cached — `bearer-cache.ts` simply does not + * catch around the call, so that falls out of the structure rather than needing an explicit branch. + * + * **A provider MUST carry its own deadline.** It takes NO parameters, deliberately -- not even an + * optional `{signal}` bag. AUTH-34 coalesces every concurrent caller racing on a missing or expiring + * token onto ONE fetch, so that fetch belongs to no single request: handing it one caller's signal + * would let a stranger's cancellation reject callers who never aborted, including a caller who + * supplied no signal at all, and would let a request that merely finished tear down a refresh other + * requests are joined to. `bearer-cache.ts` races each caller's own WAIT against that caller's own + * signal instead, which cancels the wait without cancelling the work. + * + * Since nothing can ever populate a signal parameter, there is no signal parameter -- a slot + * documented as never filled is worse than no slot, because a caller writes code against it and then + * wonders why cancelling does nothing. The consequence is that nothing outside the provider can bound + * the fetch, so the provider must bound itself: + * + * ```ts + * const provider: TokenProvider = () => fetchToken({signal: AbortSignal.timeout(5_000)}); + * ``` + * + * `docs/knowledge/harvested/concurrency-and-async.md`'s "every external I/O call must carry a deadline" is the + * rule this discharges; its "pass the caller's signal down to the I/O primitive" rule is the one + * deliberately not applied here, because the premise it rests on -- that the call owns the I/O -- is + * false for a coalesced fetch. Recorded in the phase checklist's Deviation Ledger. + * + * @public + */ +export type TokenProvider = () => Promise<BearerToken>; diff --git a/packages/core/src/auth/descriptor.test.ts b/packages/core/src/auth/descriptor.test.ts new file mode 100644 index 0000000..67340e1 --- /dev/null +++ b/packages/core/src/auth/descriptor.test.ts @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/descriptor.test.ts +// Exercises: AUTH-3 (non-empty, immutable, ordered; empty list rejected as a programmer error via +// invariant(), not a typed operational leaf -- see the plan's Global Constraints). +import {describe, expect, test} from 'bun:test'; +import {InvariantViolation} from '../invariant.js'; +import {createAuthDescriptor} from './descriptor.js'; +import {createAuthRequirement} from './requirement.js'; + +describe('createAuthDescriptor', () => { + test('preserves requirement order', () => { + const descriptor = createAuthDescriptor([ + createAuthRequirement('DIGEST'), + createAuthRequirement('BASIC'), + ]); + expect(descriptor.requirements.map(r => r.scheme)).toEqual([ + 'DIGEST', + 'BASIC', + ]); + }); + + test('allowsAnonymous is true iff any requirement is NO_AUTH', () => { + expect( + createAuthDescriptor([createAuthRequirement('NO_AUTH')]).allowsAnonymous, + ).toBe(true); + expect( + createAuthDescriptor([createAuthRequirement('BASIC')]).allowsAnonymous, + ).toBe(false); + expect( + createAuthDescriptor([ + createAuthRequirement('BASIC'), + createAuthRequirement('NO_AUTH'), + ]).allowsAnonymous, + ).toBe(true); + }); + + test('rejects an empty requirement list (AUTH-3) -- a programmer error, not AuthResolutionError', () => { + expect(() => createAuthDescriptor([])).toThrow(InvariantViolation); + }); + + test('is frozen, including the requirements array', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('BASIC')]); + expect(Object.isFrozen(descriptor)).toBe(true); + expect(Object.isFrozen(descriptor.requirements)).toBe(true); + }); + + test('defensively copies the requirement list', () => { + const requirements = [createAuthRequirement('BASIC')]; + const descriptor = createAuthDescriptor(requirements); + requirements.push(createAuthRequirement('NO_AUTH')); + expect(descriptor.requirements.length).toBe(1); + }); +}); diff --git a/packages/core/src/auth/descriptor.ts b/packages/core/src/auth/descriptor.ts new file mode 100644 index 0000000..8cd9c9e --- /dev/null +++ b/packages/core/src/auth/descriptor.ts @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/descriptor.ts +import {invariant} from '../invariant.js'; +import type {AuthRequirement} from './requirement.js'; + +/** + * AUTH-3: a non-empty, immutable, ordered list of requirements in preference order. + * + * @public + */ +export interface AuthDescriptor { + /** The requirements, in preference order. Never empty. */ + readonly requirements: readonly AuthRequirement[]; + /** `true` if and only if some requirement's scheme is `NO_AUTH` (AUTH-3). */ + readonly allowsAnonymous: boolean; +} + +/** + * Builds a frozen {@link AuthDescriptor}, copying the requirement list so later caller-side mutation + * cannot reach the stored value (AUTH-3). + * + * An empty list is a PROGRAMMER error — a caller assembling zero requirements has a bug, not an + * operational failure — so it goes through `invariant()`, the same call 5a's `retrySettings()` and + * 5b's `redirectSettings()` made, rather than a typed error leaf. + * + * @param requirements - the requirements, in preference order. Must be non-empty. + * @returns the frozen descriptor. + * @throws an assertion failure (a caller bug, not a catchable condition) when `requirements` is empty (AUTH-3). + * + * @public + */ +export function createAuthDescriptor( + requirements: readonly AuthRequirement[], +): AuthDescriptor { + invariant( + requirements.length > 0, + 'AuthDescriptor requires at least one AuthRequirement', + ); + return Object.freeze({ + requirements: Object.freeze([...requirements]), + allowsAnonymous: requirements.some( + requirement => requirement.scheme === 'NO_AUTH', + ), + }); +} diff --git a/packages/core/src/auth/digest.test.ts b/packages/core/src/auth/digest.test.ts new file mode 100644 index 0000000..93b474e --- /dev/null +++ b/packages/core/src/auth/digest.test.ts @@ -0,0 +1,567 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/digest.test.ts +// Exercises: XCUT-14 (the per-nonce counter is a server-keyed, process-lived map, so it carries a hard +// 1024-entry cap and drains back under it with a loop after each insert -- asserted below), XCUT-21 +// (the client nonce is drawn from a CSPRNG with >= 128 bits of entropy, never a non-cryptographic RNG), +// AUTH-15 (exactly {MD5, MD5-sess, SHA-256, SHA-256-sess}, qop=auth or absent, declines +// auth-int and unsupported algorithms), AUTH-16 (satisfiability: scheme/realm/nonce/qop/algorithm, +// and configured-preference order over wire order -- realm and nonce must carry a VALUE, so a +// truncated `nonce=` is declined rather than echoed back as `nonce=""`), AUTH-17 (HA1/HA2/response per RFC 7616/2069, +// verified against independently-computed vectors), AUTH-18/AUTH-19 (nonce count: starts at 1, +// increments only on nonce reuse, 8 lower-case hex digits, bounded and drained to the cap), +// AUTH-20 (client nonce from crypto.getRandomValues, >=128 bits), AUTH-21 (UTF-8 vs ISO-8859-1 by +// charset), AUTH-22 (quoting, and nc/qop emitted only when qop negotiated -- with cnonce emitted for +// every -sess algorithm whatever qop says, a recorded departure from AUTH-22's letter: RFC 7616 3.4.2 +// folds cnonce into a -sess HA1, so a server cannot verify a -sess response that omits it), AUTH-25 +// (Authorization vs Proxy-Authorization is the CALLER's job -- stamp() returns only the value). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import type {Challenge, DigestUriContext} from './challenge.js'; +import { + NonceCountStore, + computeDigestResponse, + digestHandler, +} from './digest.js'; + +const REALM = 'testrealm@host.com'; +const NONCE = 'dcd98b7102dd2f0e8b11d0f600bfb0c093'; +const CNONCE = '0a4f113b'; +const NC = '00000001'; +const BASE = { + realm: REALM, + nonce: NONCE, + isUtf8: true, + method: 'GET', + uri: '/dir/index.html', + username: 'Mufasa', + password: 'Circle Of Life', + cnonce: CNONCE, + nc: NC, +} as const; + +const REQUEST_CONTEXT: DigestUriContext = { + method: 'GET', + requestTarget: '/dir/index.html', +}; + +function digestChallenge(params: Record<string, string>): Challenge { + return {scheme: 'digest', params: new Map(Object.entries(params))}; +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this runner's + * type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper keeps the + * assertion honest without a lint suppression. Same shape 5a's and 5b's step suites settled on. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('computeDigestResponse (verified against RFC 2617/7616 vectors)', () => { + test('MD5, qop=auth', async () => { + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5', + hasQopAuth: true, + }), + ).toBe('6629fae49393a05397450978507c4ef1'); + }); + + test('MD5, no qop (RFC 2069 form)', async () => { + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5', + hasQopAuth: false, + }), + ).toBe('670fd8c2df070c60b045671b8b24ff02'); + }); + + test('MD5-sess, qop=auth', async () => { + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5-sess', + hasQopAuth: true, + }), + ).toBe('8e3825c57e897f5a0dec6c2d4e5059d0'); + }); + + test('MD5-sess, no qop -- cnonce still folds into HA1 (RFC 7616 3.4.2)', async () => { + // The vector the `-sess`-without-`qop` fix is pinned on. HA1 is + // H(H(user:realm:pass):nonce:cnonce) for every `-sess` algorithm, whatever `qop` the server + // offered, and the response is then RFC 2069's H(HA1:nonce:HA2) because no qop was negotiated. + // Recomputed independently from the same RFC 2617 3.5 inputs the vectors above use. + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5-sess', + hasQopAuth: false, + }), + ).toBe('4726bc10c33fa6cb357eb27807b1cce8'); + }); + + test('SHA-256, qop=auth', async () => { + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'SHA-256', + hasQopAuth: true, + }), + ).toBe('5abdd07184ba512a22c53f41470e5eea7dcaa3a93a59b630c13dfe0a5dc6e38b'); + }); + + test('SHA-256-sess, qop=auth', async () => { + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'SHA-256-sess', + hasQopAuth: true, + }), + ).toBe('b8822e12417cb7750f4e2b8515f0dcf25b7dd26993e80bee1426201446a7f59b'); + }); +}); + +describe('computeDigestResponse charset and determinism (AUTH-17/AUTH-21)', () => { + test('AUTH-21: the charset changes the hash for a non-ASCII password', async () => { + const utf8 = await computeDigestResponse({ + ...BASE, + password: 'pässwörd', + algorithm: 'MD5', + hasQopAuth: true, + isUtf8: true, + }); + const latin1 = await computeDigestResponse({ + ...BASE, + password: 'pässwörd', + algorithm: 'MD5', + hasQopAuth: true, + isUtf8: false, + }); + expect(utf8).not.toBe(latin1); + }); + + test('AUTH-21: an all-ASCII input hashes identically under either charset', async () => { + const utf8 = await computeDigestResponse({ + ...BASE, + algorithm: 'MD5', + hasQopAuth: true, + isUtf8: true, + }); + const latin1 = await computeDigestResponse({ + ...BASE, + algorithm: 'MD5', + hasQopAuth: true, + isUtf8: false, + }); + expect(utf8).toBe(latin1); + }); + + test('is deterministic -- the same inputs recompute to the same response (AUTH-17)', async () => { + const input = {...BASE, algorithm: 'SHA-256', hasQopAuth: true} as const; + expect(await computeDigestResponse(input)).toBe( + await computeDigestResponse(input), + ); + }); +}); + +describe('NonceCountStore (AUTH-18/19)', () => { + test('starts at 1 for a first-seen nonce', () => { + expect(new NonceCountStore().next('n1')).toBe(1); + }); + + test('increments only on reuse of the SAME nonce', () => { + const store = new NonceCountStore(); + expect(store.next('n1')).toBe(1); + // A different nonce starts fresh; it does not inherit n1's count. + expect(store.next('n2')).toBe(1); + expect(store.next('n1')).toBe(2); + expect(store.next('n1')).toBe(3); + }); + + test('property: a fixed nonce produces a strictly increasing sequence', () => { + fc.assert( + fc.property(fc.integer({min: 1, max: 200}), calls => { + const fresh = new NonceCountStore(); + let previous = 0; + for (let i = 0; i < calls; i += 1) { + const count = fresh.next('fixed'); + expect(count).toBeGreaterThan(previous); + previous = count; + } + }), + ); + }); + + test('bounded at 1024 entries, oldest evicted first (AUTH-19)', () => { + const store = new NonceCountStore(); + for (let i = 0; i < 1024; i += 1) store.next(`nonce-${String(i)}`); + store.next('nonce-1024'); // 1025th distinct nonce -- evicts 'nonce-0' + expect(store.next('nonce-0')).toBe(1); // evicted -- starts over, not 2 + }); + + test('drains back UNDER the cap after every admit, not one victim per insert (AUTH-19/XCUT-14)', () => { + // The distinguishing case for drain-to-cap vs pre-insert check-then-evict: a long run of fresh + // server-chosen nonces. A single-victim-per-insert store stays pinned at (or above) the bound + // forever without converging; the loop must leave the map at exactly the cap after each admit. + const store = new NonceCountStore(); + for (let i = 0; i < 4096; i += 1) { + store.next(`burst-${String(i)}`); + expect(store.size).toBeLessThanOrEqual(1024); + } + expect(store.size).toBe(1024); + }); +}); + +describe('digestHandler: credential validation (AUTH-9/AUTH-22)', () => { + test('rejects blank credentials', () => { + expect(() => digestHandler('', 'p')).toThrow(InvariantViolation); + expect(() => digestHandler('u', ' ')).toThrow(InvariantViolation); + }); + + test('rejects a username that is not header-safe (AUTH-22)', () => { + // AUTH-22 writes the username verbatim into the Authorization value, and HTTP-18 admits only + // HTAB plus printable ASCII there. Caller configuration, so it fails fast and loudly at + // construction rather than being declined silently per request the way a server realm is. + // RFC 7616 §4's `username*` encoding would lift this and is deferred. + expect(() => digestHandler('björn', 'p')).toThrow('header-safe'); + }); + + test('canHandle declines a challenge whose realm cannot be echoed (AUTH-22/HTTP-18)', () => { + // A received field-value may legally carry obs-text (HTTP-19), so `realm="café"` reaches us + // intact -- but it cannot go back out. Declining makes AUTH-33 surface the 401 unchanged, which + // beats building the header anyway and throwing HeaderValidationError out of the whole step. + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: 'café', nonce: NONCE, algorithm: 'MD5'}), + ), + ).toBe(false); + }); + + test('canHandle declines an opaque or nonce that cannot be echoed (AUTH-22/HTTP-18)', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, opaque: 'ö'}), + ), + ).toBe(false); + expect( + handler.canHandle(digestChallenge({realm: REALM, nonce: 'nö'})), + ).toBe(false); + }); +}); + +describe('digestHandler: a realm or nonce with no value (AUTH-16)', () => { + test('canHandle declines an EMPTY realm or nonce, not only an absent one (AUTH-16)', () => { + // A truncated `WWW-Authenticate: Digest realm="r", nonce=` parses to `nonce: ''` -- AUTH-12 stores + // values verbatim after unquoting and AUTH-13 keeps what it parsed before the malformed tail, both + // correctly. The `=== undefined` test downstream then read that as present, and the client sent + // `nonce=""` back: a response computed over an empty nonce, which no server can have issued + // (audit #67 / #74). AUTH-16's "carries realm and nonce" is read as carrying a VALUE. + const handler = digestHandler('u', 'p'); + expect(handler.canHandle(digestChallenge({realm: REALM, nonce: ''}))).toBe( + false, + ); + expect(handler.canHandle(digestChallenge({realm: '', nonce: NONCE}))).toBe( + false, + ); + }); + + test('rank is worst-possible for an empty realm or nonce, so a sibling wins', () => { + // The declining half is only useful if the composer can still get past it: `rank` and `canHandle` + // must agree, or a challenge that cannot be stamped would still sort first (AUTH-25's "return no + // header when it cannot satisfy any" applies per challenge, not per response). + const handler = digestHandler('u', 'p'); + expect(handler.rank?.(digestChallenge({realm: REALM, nonce: ''}))).toBe( + Number.MAX_SAFE_INTEGER, + ); + }); +}); + +describe('digestHandler: challenge selection (AUTH-15/AUTH-16)', () => { + test('canHandle accepts a well-formed Digest challenge', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, qop: 'auth'}), + ), + ).toBe(true); + }); + + test('canHandle accepts a qop list that merely CONTAINS auth', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, qop: 'auth-int, auth'}), + ), + ).toBe(true); + }); + + test('canHandle rejects a non-Digest scheme', () => { + expect( + digestHandler('u', 'p').canHandle({scheme: 'basic', params: new Map()}), + ).toBe(false); + }); + + test('canHandle rejects a missing realm or nonce', () => { + const handler = digestHandler('u', 'p'); + expect(handler.canHandle(digestChallenge({nonce: NONCE}))).toBe(false); + expect(handler.canHandle(digestChallenge({realm: REALM}))).toBe(false); + }); + + test('canHandle declines an auth-int-only qop (AUTH-15)', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, qop: 'auth-int'}), + ), + ).toBe(false); + }); + + test('canHandle declines an unsupported algorithm (AUTH-15)', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD4'}), + ), + ).toBe(false); + }); + + test('canHandle matches the algorithm name case-insensitively', () => { + const handler = digestHandler('u', 'p'); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'sha-256'}), + ), + ).toBe(true); + }); +}); + +describe('digestHandler stamping (AUTH-17..AUTH-22, AUTH-25)', () => { + test('canHandle defaults to MD5 when algorithm is absent', () => { + const handler = digestHandler('u', 'p', {algorithmPreference: ['MD5']}); + expect( + handler.canHandle(digestChallenge({realm: REALM, nonce: NONCE})), + ).toBe(true); + }); + + test('canHandle honors a caller-restricted algorithm preference', () => { + const handler = digestHandler('u', 'p', {algorithmPreference: ['SHA-256']}); + expect( + handler.canHandle( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5'}), + ), + ).toBe(false); + }); + + test('rank reflects preference-list order, for composing-handler.ts to sort by', () => { + const handler = digestHandler('u', 'p', { + algorithmPreference: ['SHA-256', 'MD5'], + }); + const sha = handler.rank?.( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'SHA-256'}), + ); + const md5Rank = handler.rank?.( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5'}), + ); + expect(sha).toBeLessThan(md5Rank ?? Number.POSITIVE_INFINITY); + }); + + test('rank is worst-possible for a challenge it cannot handle', () => { + const handler = digestHandler('u', 'p'); + expect(handler.rank?.({scheme: 'basic', params: new Map()})).toBe( + Number.MAX_SAFE_INTEGER, + ); + }); + + test('stamp() produces a well-formed Digest header value, qop negotiated', async () => { + const handler = digestHandler('Mufasa', 'Circle Of Life'); + const challenge = digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + }); + const value = await handler.stamp(challenge, REQUEST_CONTEXT); + expect(value.startsWith('Digest ')).toBe(true); + expect(value).toContain('username="Mufasa"'); + expect(value).toContain(`realm="${REALM}"`); + expect(value).toContain('uri="/dir/index.html"'); + expect(value).toContain('qop=auth'); + expect(value).toMatch(/nc=[0-9a-f]{8}/u); + expect(value).toMatch(/response="[0-9a-f]+"/u); + }); +}); + +describe('digestHandler nonce counting and preconditions (AUTH-18/AUTH-25)', () => { + test('stamp() draws a fresh >=128-bit client nonce per call (AUTH-20)', async () => { + const handler = digestHandler('u', 'p'); + const challenge = digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + }); + const first = /cnonce="([0-9a-f]+)"/u.exec( + await handler.stamp(challenge, REQUEST_CONTEXT), + ); + const second = /cnonce="([0-9a-f]+)"/u.exec( + await handler.stamp(challenge, REQUEST_CONTEXT), + ); + expect(first?.[1]).toHaveLength(32); // 16 bytes rendered as hex + expect(first?.[1]).not.toBe(second?.[1]); + }); + + test('stamp() emits the FULL algorithm spelling, unquoted (AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + algorithm: 'SHA-256-sess', + }), + REQUEST_CONTEXT, + ); + expect(value).toContain('algorithm=SHA-256-sess'); + }); + + test('stamp() escapes a quote inside a realm rather than emitting it raw (AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({realm: 'a"b', nonce: NONCE}), + REQUEST_CONTEXT, + ); + expect(value).toContain(String.raw`realm="a\"b"`); + }); +}); + +describe('digestHandler opaque and qop emission (AUTH-22)', () => { + test('stamp() echoes the challenge opaque back, quoted (AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + const challenge = digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + opaque: '5ccc069c403ebaf9f0171e9517f40e41', + }); + const value = await handler.stamp(challenge, REQUEST_CONTEXT); + expect(value).toContain('opaque="5ccc069c403ebaf9f0171e9517f40e41"'); + }); + + test('stamp() omits opaque entirely when the challenge carried none (AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE}), + REQUEST_CONTEXT, + ); + expect(value).not.toContain('opaque'); + }); +}); + +describe('digestHandler -sess without qop (AUTH-17/AUTH-22)', () => { + test('stamp() emits cnonce -- and only cnonce -- for a -sess algorithm with no qop', async () => { + // A `-sess` HA1 is H(H(u:r:p):nonce:cnonce). Hashing a fresh random cnonce and then leaving it + // off the wire made every such response unverifiable by construction, and AUTH-30 bounds the + // replay to one 401, so the request simply failed (audit #67 / #74). RFC 7616 3.4: "cnonce: This + // parameter MUST be used by all implementations". `nc` and `qop` stay conditional on a negotiated + // qop -- a `deviations.md` row records the departure from AUTH-22's letter. + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + expect(value).toContain('cnonce="'); + expect(value).not.toContain('qop='); + expect(value).not.toContain('nc='); + }); + + test('the emitted cnonce is the one the response was computed with (-sess, no qop)', async () => { + // The row that makes the header VERIFIABLE rather than merely populated: a cnonce emitted from a + // second draw would satisfy the assertion above and still be worthless to the server. Recomputing + // HA1 from the header's own cnonce has to reproduce the header's own response. + const handler = digestHandler('Mufasa', 'Circle Of Life'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + const cnonce = /cnonce="(?<value>[0-9a-f]+)"/u.exec(value)?.groups?.value; + const response = /response="(?<value>[0-9a-f]+)"/u.exec(value)?.groups + ?.value; + expect(cnonce).toBeDefined(); + expect(response).toBe( + await computeDigestResponse({ + ...BASE, + isUtf8: false, // no charset on the challenge -- AUTH-21's ISO-8859-1 branch + algorithm: 'MD5-sess', + hasQopAuth: false, + cnonce: cnonce ?? '', + nc: '', + }), + ); + }); +}); + +describe('digestHandler nonce-count sequencing (AUTH-18)', () => { + test('stamp() omits cnonce/nc/qop for a NON-sess algorithm with no qop (AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + // No `algorithm` parameter, so AUTH-16's default applies: plain MD5, whose HA1 does not involve + // the client nonce at all. AUTH-22's "emit cnonce/nc/qop only when qop is negotiated" is exactly + // right here, and this is the branch it was written for. + const challenge = digestChallenge({realm: REALM, nonce: NONCE}); + const value = await handler.stamp(challenge, REQUEST_CONTEXT); + expect(value).not.toContain('qop='); + expect(value).not.toContain('cnonce='); + expect(value).not.toContain('nc='); + }); + + test('two successive stamp() calls against the SAME nonce increment nc (AUTH-18)', async () => { + const handler = digestHandler('u', 'p'); + const challenge = digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + }); + const first = await handler.stamp(challenge, REQUEST_CONTEXT); + const second = await handler.stamp(challenge, REQUEST_CONTEXT); + expect(first).toContain('nc=00000001'); + expect(second).toContain('nc=00000002'); + }); + + test('a no-qop stamp does not consume a nonce count (AUTH-18/AUTH-22)', async () => { + const handler = digestHandler('u', 'p'); + await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE}), + REQUEST_CONTEXT, + ); + const withQop = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, qop: 'auth'}), + REQUEST_CONTEXT, + ); + expect(withQop).toContain('nc=00000001'); + }); + + test('stamp() rejects a challenge canHandle() would decline', async () => { + const handler = digestHandler('u', 'p'); + expect( + await rejectionOf( + handler.stamp({scheme: 'basic', params: new Map()}, REQUEST_CONTEXT), + ), + ).toBeInstanceOf(InvariantViolation); + }); + + test('stamp() rejects a missing DigestUriContext -- it cannot compute HA2 without one', async () => { + const handler = digestHandler('u', 'p'); + expect( + await rejectionOf( + handler.stamp(digestChallenge({realm: REALM, nonce: NONCE})), + ), + ).toBeInstanceOf(InvariantViolation); + }); +}); diff --git a/packages/core/src/auth/digest.ts b/packages/core/src/auth/digest.ts new file mode 100644 index 0000000..fdfaa2a --- /dev/null +++ b/packages/core/src/auth/digest.ts @@ -0,0 +1,508 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/digest.ts +import {hasForbiddenOutboundValueByte} from '../http/ascii-validation.js'; +import {invariant} from '../invariant.js'; +import type { + Challenge, + ChallengeHandler, + DigestUriContext, +} from './challenge.js'; +import {md5, toHex} from './md5.js'; + +/** + * AUTH-15: exactly these four algorithms are supported. `auth-int` and every other algorithm is + * declined rather than approximated. + * + * @public + */ +export type DigestAlgorithm = 'MD5' | 'MD5-sess' | 'SHA-256' | 'SHA-256-sess'; + +// `as const`, not a bare `readonly` annotation, so the CONSTANT_CASE is honest: `naming-conventions.md` +// reserves that casing for deeply immutable values, and a bare `readonly DigestAlgorithm[]` annotation +// is a compile-time claim only -- the array stays mutable at runtime through a cast. +const SUPPORTED_ALGORITHMS = [ + 'MD5', + 'MD5-sess', + 'SHA-256', + 'SHA-256-sess', +] as const satisfies readonly DigestAlgorithm[]; + +// Strongest first. AUTH-16 makes this list the PREFERENCE order, applied regardless of the order the +// server offered its challenges in. +const DEFAULT_ALGORITHM_PREFERENCE = [ + 'SHA-256-sess', + 'SHA-256', + 'MD5-sess', + 'MD5', +] as const satisfies readonly DigestAlgorithm[]; + +const NONCE_COUNT_LIMIT = 1024; + +/** + * Digest handler tuning. + * + * @internal + */ +export interface DigestOptions { + /** + * Preferred-first order, and also the ACCEPTABLE set: an algorithm absent from this list is + * declined outright (AUTH-16). Defaults to `['SHA-256-sess', 'SHA-256', 'MD5-sess', 'MD5']`. + */ + readonly algorithmPreference?: readonly DigestAlgorithm[] | undefined; +} + +function baseAlgorithm(algorithm: DigestAlgorithm): 'MD5' | 'SHA-256' { + return algorithm.startsWith('MD5') ? 'MD5' : 'SHA-256'; +} + +/** + * AUTH-21's non-UTF-8 branch. ISO-8859-1 is a byte-for-byte code-unit copy for every character it can + * represent; a character outside the codebook has no ISO-8859-1 encoding at all, and truncating is the + * same lossy answer every other Latin-1 encoder gives. A server that expects such a character is + * required to advertise `charset=UTF-8`, which routes to the other branch. + */ +function encodeLatin1(input: string): Uint8Array<ArrayBuffer> { + const bytes = new Uint8Array(input.length); + for (let i = 0; i < input.length; i += 1) bytes[i] = input.charCodeAt(i); + return bytes; +} + +/** + * AUTH-21's UTF-8 branch, copied into a freshly-allocated buffer. + * + * The copy is not incidental: `crypto.subtle.digest` takes a `BufferSource`, which excludes a view + * that might sit on a `SharedArrayBuffer`, and `TextEncoder.encode` is typed as the wider + * `Uint8Array<ArrayBufferLike>`. Re-narrowing with a cast would assert something the type system + * cannot check; allocating an exact-typed buffer costs one copy of a string that is never more than a + * few hundred bytes. + */ +function encodeUtf8(input: string): Uint8Array<ArrayBuffer> { + const encoded = new TextEncoder().encode(input); + const bytes = new Uint8Array(encoded.length); + bytes.set(encoded); + return bytes; +} + +/** AUTH-17/AUTH-21: one hash, over one encoding of one string. */ +interface HashInput { + readonly base: 'MD5' | 'SHA-256'; + readonly input: string; + /** AUTH-21: UTF-8 when the challenge advertised `charset=UTF-8`, ISO-8859-1 otherwise. */ + readonly isUtf8: boolean; +} + +// An options object rather than three positional parameters: `function-design.md` requires one at +// three or more parameters, and unconditionally for any boolean parameter -- `hashHex(base, s, true)` +// says nothing at the call site about what the `true` selects. +async function hashHex({base, input, isUtf8}: HashInput): Promise<string> { + const bytes = isUtf8 ? encodeUtf8(input) : encodeLatin1(input); + // MD5 is hand-rolled because Web Crypto excludes it; SHA-256 goes through Web Crypto rather than + // `node:crypto` to keep the package portable (SEAM-1, `sdk-design-nodejs/06`). + if (base === 'MD5') return toHex(md5(bytes)); + return toHex( + new Uint8Array(await globalThis.crypto.subtle.digest('SHA-256', bytes)), + ); +} + +/** + * The per-server-nonce request counter (AUTH-18, AUTH-19). + * + * `nc` starts at 1 for a first-seen nonce, increments only on reuse of that same nonce, and wraps to + * the low 32 bits on overflow. Bounded at 1024 entries with insertion-order eviction — `Map` + * iteration order IS insertion order, so the oldest key is `keys().next().value` and no separate LRU + * structure is needed. Evicting a live nonce is harmless: its count restarts at 1, which is + * spec-legal for a nonce the server has just re-issued. + * + * The eviction is an insert-THEN-drain, never a pre-insert check-then-evict: `next()` admits the + * nonce first and only then brings the map back under the cap, which is how + * `docs/knowledge/harvested/concurrency-and-async.md` (XCUT-14) and AUTH-19 both word it — "drained back under + * the cap after admitting a nonce". The key space is the SERVER's, since it picks the nonces, so a + * pre-insert evict would leave a burst sitting above the cap rather than converging to it. + * + * The drain is a `while` rather than an `if` purely as defence, and the distinction is NOT currently + * observable: `next()` grows the map by at most one entry per call, so the body can run at most once + * and the two spellings are equivalent today. The loop is what keeps the bound true if `NONCE_COUNT_LIMIT` + * is ever lowered at runtime or a second writer is ever added. + * + * AUTH-24's concurrency clause: `next()` is one synchronous read-increment-write with no `await` + * between the read and the write, so two concurrent callers cannot observe the same count. Node and + * Bun have no preemptive interleaving mid-statement — the same collapse 5a documented for BODY-3's + * materialize-once guard. + * + * @internal + */ +export class NonceCountStore { + private readonly counts = new Map<string, number>(); + + /** + * The number of nonces currently tracked. + * + * Exposed so the bound itself is assertable — otherwise "drained back under the cap" is testable + * only through the indirect "an evicted nonce restarts at 1" probe, which passes for a + * single-victim-per-insert store that never converges. + */ + get size(): number { + return this.counts.size; + } + + /** + * Returns the nonce count to send with this request (AUTH-18). + * + * @param nonce - the server-chosen nonce being answered. + * @returns `1` the first time this nonce is seen, one more than the previous value on each reuse, + * wrapping to the low 32 bits on overflow. + */ + next(nonce: string): number { + const current = this.counts.get(nonce); + const count = current === undefined ? 1 : (current + 1) >>> 0; + this.counts.set(nonce, count); + + // The just-admitted nonce sits at the TAIL of insertion order (a `set` on an existing key leaves + // the map's size unchanged, so the loop is not entered at all on a reuse), which is why draining + // from the head can never evict the live nonce this call is answering with. + while (this.counts.size > NONCE_COUNT_LIMIT) { + const oldest = this.counts.keys().next().value; + invariant( + oldest !== undefined, + 'nonce-count store is over its bound but reports no oldest entry', + ); + this.counts.delete(oldest); + } + + return count; + } +} + +/** AUTH-18: exactly 8 lower-case hex digits. */ +function formatNonceCount(count: number): string { + return count.toString(16).padStart(8, '0'); +} + +/** AUTH-20: at least 128 bits from a CSPRNG. Never `Math.random()`. */ +function generateClientNonce(): string { + const bytes = new Uint8Array(16); + globalThis.crypto.getRandomValues(bytes); + return toHex(bytes); +} + +interface ParsedDigestChallenge { + readonly algorithm: DigestAlgorithm; + readonly realm: string; + readonly nonce: string; + /** + * Whether `qop=auth` was negotiated. Named for what it HOLDS, not for the wire parameter: `qop` on + * the wire is a string (`auth`, `auth-int`), so a boolean called `qop` reads as that value. + */ + readonly hasQopAuth: boolean; + readonly isUtf8: boolean; + /** + * AUTH-22 names `opaque` in the must-quote list, which only makes sense if it is emitted: RFC 7616 + * requires the client return the server's opaque value unchanged, and a server that binds session + * state to it rejects a request that omits it. Absent when the challenge carried none. + */ + readonly opaque: string | undefined; +} + +/** + * AUTH-22 echoes `realm`, `nonce`, and `opaque` VERBATIM into the `Authorization` value, and HTTP-18's + * outbound grammar admits only HTAB plus printable ASCII. A received challenge is held to the laxer + * inbound rule (HTTP-19 permits obs-text, exactly so a Latin-1 field is not silently dropped), so a + * server may legitimately hand us a realm this client cannot echo -- `Digest realm="café"` is a real + * RFC 7616 shape, which is why the spec has a `charset` parameter at all. + * + * Declining such a challenge makes `canHandle` false, so the composer finds no candidate and AUTH-33 + * surfaces the 401 unchanged. That is strictly better than the alternative it replaces, which was to + * build the header anyway and throw `HeaderValidationError` out of the whole auth step -- turning a + * challenge the caller could have inspected into an exception. Relaxing the outbound rule instead was + * never an option: HTTP-17/18/19's strictness is the request-splitting defence. + * + * The consequence is that AUTH-21's UTF-8 branch is reachable for the HASH INPUT (where a non-ASCII + * password lives and works) but not for the realm ECHO. RFC 7616 §4's `username*` (RFC 5987) extended + * notation is the standard answer and is deferred; both are recorded in the Deviation Ledger. + */ +function isHeaderSafeEcho(info: { + readonly realm: string; + readonly nonce: string; + readonly opaque: string | undefined; +}): boolean { + return ( + !hasForbiddenOutboundValueByte(info.realm) && + !hasForbiddenOutboundValueByte(info.nonce) && + (info.opaque === undefined || !hasForbiddenOutboundValueByte(info.opaque)) + ); +} + +/** + * AUTH-16: satisfiable if and only if the scheme is `digest`, `realm` and `nonce` both carry a + * non-empty value, `qop` is absent or contains `auth`, the algorithm (defaulting to `MD5`) is in the + * caller's configured preference list, and every field AUTH-22 echoes back is header-safe + * ({@link isHeaderSafeEcho}). + * + * "Carries realm and nonce" is read as carrying a VALUE, not merely a key. A truncated header — + * `Digest realm="r", nonce=` — parses to `nonce: ''`, which is AUTH-12 and AUTH-13 both behaving + * exactly as specified: values are stored verbatim after unquoting, and what was parsed before a + * malformed tail is kept. Testing `=== undefined` here accepted that and sent `nonce=""` back, a + * response computed over a nonce no server can have issued. The challenge is declined instead, so the + * next one is tried and a 401 offering nothing else surfaces unchanged (AUTH-25, AUTH-33). + */ +function parseDigestChallenge( + challenge: Challenge, + preference: readonly DigestAlgorithm[], +): ParsedDigestChallenge | undefined { + if (challenge.scheme !== 'digest') return undefined; + const realm = challenge.params.get('realm'); + const nonce = challenge.params.get('nonce'); + if (realm === undefined || realm === '') return undefined; + if (nonce === undefined || nonce === '') return undefined; + + const qopRaw = challenge.params.get('qop'); + const hasQop = qopRaw !== undefined; + // AUTH-15: an `auth-int`-only challenge is DECLINED, not silently downgraded. + if ( + hasQop && + !qopRaw.split(',').some(entry => entry.trim().toLowerCase() === 'auth') + ) { + return undefined; + } + + const algorithmRaw = challenge.params.get('algorithm'); + const algorithm = + algorithmRaw === undefined + ? 'MD5' + : SUPPORTED_ALGORITHMS.find( + candidate => candidate.toLowerCase() === algorithmRaw.toLowerCase(), + ); + if (algorithm === undefined || !preference.includes(algorithm)) { + return undefined; + } + + const isUtf8 = + (challenge.params.get('charset') ?? '').toLowerCase() === 'utf-8'; + const info: ParsedDigestChallenge = { + algorithm, + realm, + nonce, + hasQopAuth: hasQop, + isUtf8, + opaque: challenge.params.get('opaque'), + }; + return isHeaderSafeEcho(info) ? info : undefined; +} + +/** + * Everything {@link computeDigestResponse} needs. Bundled into one object because the computation + * genuinely takes eleven inputs and `max-params` is 3. + * + * @internal + */ +export interface DigestComputationInput { + /** The negotiated algorithm; `-sess` variants fold the nonce and cnonce into HA1. */ + readonly algorithm: DigestAlgorithm; + /** The challenge's realm. */ + readonly realm: string; + /** The server-chosen nonce. */ + readonly nonce: string; + /** Whether `qop=auth` was negotiated. `false` selects RFC 2069's shorter response input. */ + readonly hasQopAuth: boolean; + /** AUTH-21: UTF-8 hash input when the challenge advertised `charset=UTF-8`, ISO-8859-1 otherwise. */ + readonly isUtf8: boolean; + /** The request method. */ + readonly method: string; + /** The digest-uri: the request-target. */ + readonly uri: string; + /** The user id. */ + readonly username: string; + /** The password. */ + readonly password: string; + /** The client nonce; ignored when `qop` is `false`. */ + readonly cnonce: string; + /** The 8-hex-digit nonce count; ignored when `qop` is `false`. */ + readonly nc: string; +} + +/** + * Computes HA1, HA2, and the Digest response per RFC 7616/2069 (AUTH-17), in lower-case hex. + * + * Exported, and taking a single bundled parameter, so it can be unit-tested directly against fixed, + * independently-verified vectors: `digestHandler()`'s own `stamp()` always generates a fresh random + * cnonce (AUTH-20), so its output can never be asserted against a fixed expected hash end-to-end. + * + * @param input - the full computation input. + * @returns the response value, lower-case hex. + * + * @internal + */ +export async function computeDigestResponse( + input: DigestComputationInput, +): Promise<string> { + const base = baseAlgorithm(input.algorithm); + const isUtf8 = input.isUtf8; + const ha1Plain = await hashHex({ + base, + input: `${input.username}:${input.realm}:${input.password}`, + isUtf8, + }); + const ha1 = input.algorithm.endsWith('-sess') + ? await hashHex({ + base, + input: `${ha1Plain}:${input.nonce}:${input.cnonce}`, + isUtf8, + }) + : ha1Plain; + const ha2 = await hashHex({ + base, + input: `${input.method}:${input.uri}`, + isUtf8, + }); + const responseInput = input.hasQopAuth + ? `${ha1}:${input.nonce}:${input.nc}:${input.cnonce}:auth:${ha2}` + : `${ha1}:${input.nonce}:${ha2}`; + return hashHex({base, input: responseInput, isUtf8}); +} + +/** AUTH-22's quoting: a quoted-string with `\` and `"` escaped. */ +function quote(value: string): string { + return `"${value.replaceAll('\\', '\\\\').replaceAll('"', '\\"')}"`; +} + +interface HeaderValueParams { + readonly username: string; + readonly info: ParsedDigestChallenge; + readonly uri: string; + readonly response: string; + readonly cnonce: string; + readonly nc: string; +} + +/** + * AUTH-22: quotes `username`/`realm`/`nonce`/`uri`/`response`/`cnonce`/`opaque`; leaves + * `qop`/`nc`/`algorithm` unquoted, with the full algorithm spelling; emits `nc`/`qop` only when `qop` + * was actually negotiated. + * + * **`cnonce` is emitted for every `-sess` algorithm, negotiated `qop` or not** — a deliberate + * departure from AUTH-22's letter, recorded in `docs/deviations.md`. A `-sess` HA1 is + * `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so a server handed a `-sess` response with + * no `cnonce` cannot recompute HA1 and cannot verify anything: the request is unanswerable by + * construction, and AUTH-30 bounds the replay to one 401, so it simply fails. RFC 7616 §3.4 states it + * outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's wording is RFC + * 2617's RFC 2069-compatibility form, which predates `-sess` entirely. + * + * `nc` stays conditional and stays out: RFC 2069's response input is `H(HA1:nonce:HA2)`, with no nonce + * count in it, so emitting one would advertise a count the response was not computed over. + */ +function buildHeaderValue(params: HeaderValueParams): string { + const {username, info, uri, response, cnonce, nc} = params; + const parts = [ + `username=${quote(username)}`, + `realm=${quote(info.realm)}`, + `nonce=${quote(info.nonce)}`, + `uri=${quote(uri)}`, + `algorithm=${info.algorithm}`, + `response=${quote(response)}`, + ]; + // AUTH-22: `opaque` is quoted and echoed back verbatim when the challenge carried one. + if (info.opaque !== undefined) parts.push(`opaque=${quote(info.opaque)}`); + if (info.hasQopAuth) + parts.push('qop=auth', `nc=${nc}`, `cnonce=${quote(cnonce)}`); + else if (info.algorithm.endsWith('-sess')) + parts.push(`cnonce=${quote(cnonce)}`); + return `Digest ${parts.join(', ')}`; +} + +/** + * The Digest challenge handler (AUTH-15–AUTH-22). + * + * Cryptographic primitives are split across two sources for portability: `md5.ts` for MD5/MD5-sess, + * which Web Crypto deliberately excludes, and `crypto.subtle.digest('SHA-256', …)` for the SHA-256 + * pair. The client nonce comes from `crypto.getRandomValues()` (AUTH-20). + * + * The per-nonce counter is the one piece of mutable state, and it needs no lock: nothing awaits + * between its read and its write (AUTH-24). + * + * Challenge-reactive only — Digest structurally cannot stamp before seeing the server's + * `realm`/`nonce`. + * + * @param username - the user id. Must not be blank, and must be header-safe (printable ASCII): + * AUTH-22 writes it into the `Authorization` value verbatim. + * @param password - the password. Must not be blank. May hold any character — it only ever reaches + * the hash input, which is where AUTH-21's UTF-8/Latin-1 choice applies. + * @param options - algorithm preference; omitted means strongest-first over all four. + * @returns a handler that answers satisfiable `digest` challenges. + * @throws InvariantViolation when either credential is blank, or the username carries a byte HTTP-18 + * forbids in an outbound header value — both caller misconfigurations. + * + * @internal + */ +export function digestHandler( + username: string, + password: string, + options?: DigestOptions, +): ChallengeHandler { + invariant(username.trim().length > 0, 'Digest username must not be blank'); + invariant(password.trim().length > 0, 'Digest password must not be blank'); + // Configuration, not wire data: a non-ASCII username is the caller's own mistake, so it fails fast + // and loudly at construction rather than being declined silently per-request the way an + // unechoable server realm is. RFC 7616 §4's `username*` encoding would lift this and is deferred. + invariant( + !hasForbiddenOutboundValueByte(username), + 'Digest username must be header-safe (printable ASCII); RFC 7616 username* encoding is not yet supported', + ); + const preference = + options?.algorithmPreference ?? DEFAULT_ALGORITHM_PREFERENCE; + const nonceCounts = new NonceCountStore(); + + return { + canHandle: (challenge: Challenge): boolean => + parseDigestChallenge(challenge, preference) !== undefined, + + // AUTH-16: among several Digest challenges differing only by algorithm, prefer the one earliest + // in the CONFIGURED list, not the one earliest on the wire. + rank: (challenge: Challenge): number => { + const parsed = parseDigestChallenge(challenge, preference); + return parsed === undefined + ? Number.MAX_SAFE_INTEGER + : preference.indexOf(parsed.algorithm); + }, + + stamp: async ( + challenge: Challenge, + request?: DigestUriContext, + ): Promise<string> => { + const info = parseDigestChallenge(challenge, preference); + invariant( + info !== undefined, + 'digestHandler.stamp called with a challenge canHandle() would reject', + ); + invariant( + request !== undefined, + 'digestHandler.stamp requires a DigestUriContext (method + requestTarget)', + ); + + const cnonce = generateClientNonce(); + const nc = info.hasQopAuth + ? formatNonceCount(nonceCounts.next(info.nonce)) + : ''; + const response = await computeDigestResponse({ + algorithm: info.algorithm, + realm: info.realm, + nonce: info.nonce, + hasQopAuth: info.hasQopAuth, + isUtf8: info.isUtf8, + method: request.method, + uri: request.requestTarget, + username, + password, + cnonce, + nc, + }); + return buildHeaderValue({ + username, + info, + uri: request.requestTarget, + response, + cnonce, + nc, + }); + }, + }; +} diff --git a/packages/core/src/auth/errors.test.ts b/packages/core/src/auth/errors.test.ts new file mode 100644 index 0000000..3de4668 --- /dev/null +++ b/packages/core/src/auth/errors.test.ts @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/errors.test.ts +// Exercises: AUTH-6 (the resolution error names required and available schemes, and copies both +// lists onto its own frozen fields), AUTH-28 (the plaintext guard names the step and scheme), +// AUTH-35 (the resolution error's message-only construction path). +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import {AuthResolutionError, PlaintextCredentialError} from './errors.js'; + +describe('AuthResolutionError', () => { + test('a plain message constructs directly', () => { + const error = new AuthResolutionError( + 'token provider returned an expired token', + ); + expect(error.name).toBe('AuthResolutionError'); + expect(error.message).toContain('expired'); + }); + + test('the message-only path carries no scheme lists (AUTH-35)', () => { + const error = new AuthResolutionError('token provider returned null'); + expect(error.requiredSchemes).toBeUndefined(); + expect(error.availableSchemes).toBeUndefined(); + }); + + test('unsatisfiable() names both the required and available schemes', () => { + const error = AuthResolutionError.unsatisfiable( + ['BASIC', 'DIGEST'], + ['API_KEY'], + ); + expect(error.message).toContain('BASIC'); + expect(error.message).toContain('DIGEST'); + expect(error.message).toContain('API_KEY'); + }); + + test('unsatisfiable() also carries them as indexable fields, not only as prose (AUTH-6)', () => { + const error = AuthResolutionError.unsatisfiable( + ['BASIC', 'DIGEST'], + ['API_KEY'], + ); + expect(error.requiredSchemes).toEqual(['BASIC', 'DIGEST']); // preference order preserved + expect(error.availableSchemes).toEqual(['API_KEY']); + }); + + test('unsatisfiable() copies the caller arrays rather than aliasing them', () => { + const required = ['BASIC']; + const error = AuthResolutionError.unsatisfiable(required, []); + required.push('DIGEST'); + expect(error.requiredSchemes).toEqual(['BASIC']); + }); + + test('descends from DexpaceError, so a caller can catch the whole taxonomy', () => { + expect(new AuthResolutionError('x')).toBeInstanceOf(DexpaceError); + }); +}); + +describe('AuthResolutionError copies its scheme lists (AUTH-6)', () => { + test('the constructor copies, so a caller mutating its array cannot reach the error', () => { + const required = ['BASIC']; + const available = ['DIGEST']; + const error = new AuthResolutionError('nope', required, available); + required.push('OAUTH2'); + available.push('API_KEY'); + expect(error.requiredSchemes).toEqual(['BASIC']); + expect(error.availableSchemes).toEqual(['DIGEST']); + }); + + test('unsatisfiable() delegates to that one copy site', () => { + const required = ['BASIC']; + const error = AuthResolutionError.unsatisfiable(required, []); + required.push('DIGEST'); + expect(error.requiredSchemes).toEqual(['BASIC']); + }); +}); + +describe('PlaintextCredentialError', () => { + test('names the step and the resolved scheme', () => { + const error = new PlaintextCredentialError('authStep', 'BASIC'); + expect(error.message).toContain('authStep'); + expect(error.message).toContain('BASIC'); + }); + + test('carries them as fields too (error-handling.md)', () => { + const error = new PlaintextCredentialError('authStep', 'BASIC'); + expect(error.stepName).toBe('authStep'); + expect(error.scheme).toBe('BASIC'); + expect(error.name).toBe('PlaintextCredentialError'); + }); +}); diff --git a/packages/core/src/auth/errors.ts b/packages/core/src/auth/errors.ts new file mode 100644 index 0000000..57d7145 --- /dev/null +++ b/packages/core/src/auth/errors.ts @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * AUTH-6 (the selected tier lists no satisfiable scheme) and AUTH-35 (a `TokenProvider` returned + * null or an already-expired token). + * + * AUTH-6, not AUTH-4: AUTH-4 governs only WHICH tier is selected — most-specific-present, with no + * fall-through — and it is AUTH-6 that requires "a distinct auth-resolution error (carrying both the + * required schemes in preference order and the available schemes)" when that tier turns out to be + * unsatisfiable. + * + * The scheme lists are `readonly` FIELDS, not only interpolated prose. AUTH-6 requires the error to + * carry both the required schemes in preference order and the available schemes, and + * `docs/knowledge/harvested/error-handling.md` requires identifying inputs to be `readonly` fields "so they + * survive serialization and appear in structured logs". Both are `undefined` on the AUTH-35 + * construction path, which has no scheme lists to carry. + * + * Typed `readonly string[]` rather than `readonly AuthScheme[]`: this module is the taxonomy leaf + * every other auth module depends on, and `scheme.ts` has no reason to depend back on it. A union of + * string literals is assignable to `string`, so callers pass `AuthScheme[]` values unchanged. + * + * @public + */ +export class AuthResolutionError extends DexpaceError { + /** The selected tier's schemes, in declared preference order. Absent on the AUTH-35 path. */ + readonly requiredSchemes: readonly string[] | undefined; + /** The schemes a credential was actually configured for. Absent on the AUTH-35 path. */ + readonly availableSchemes: readonly string[] | undefined; + + /** + * Both lists are COPIED, not aliased. They are typed `readonly` and this class is public surface, + * so a caller-owned array stored by reference would leave a `readonly` field whose contents change + * after the error was constructed. `unsatisfiable()` below delegates here rather than copying a + * second time, so there is exactly one copy site. + * + * @param message - the human-readable failure description. + * @param requiredSchemes - the selected tier's schemes, in preference order. + * @param availableSchemes - the schemes a credential was configured for. + */ + constructor( + message: string, + requiredSchemes?: readonly string[], + availableSchemes?: readonly string[], + ) { + super(message); + this.requiredSchemes = + requiredSchemes === undefined + ? undefined + : Object.freeze([...requiredSchemes]); + this.availableSchemes = + availableSchemes === undefined + ? undefined + : Object.freeze([...availableSchemes]); + } + + /** + * AUTH-6's unsatisfiable-descriptor case: the caller configured a tier, but none of its listed + * schemes has a matching credential. + * + * @param requiredSchemes - the selected tier's schemes, in preference order. + * @param availableSchemes - the schemes a credential was configured for. + * @returns the error, with both lists copied onto its own fields by the constructor. + */ + static unsatisfiable( + requiredSchemes: readonly string[], + availableSchemes: readonly string[], + ): AuthResolutionError { + return new AuthResolutionError( + `no requirement is satisfiable; required one of [${requiredSchemes.join(', ')}], available: [${availableSchemes.join(', ')}]`, + requiredSchemes, + availableSchemes, + ); + } +} + +/** + * AUTH-28: a credential would have been attached to a non-HTTPS URL. + * + * The offending URL is deliberately NOT carried — a URL can hold userinfo and query-string secrets, + * and `docs/knowledge/harvested/error-handling.md` bars interpolating secrets into a message that travels into + * logs. The step name and scheme identify the fault without that risk. + * + * @public + */ +export class PlaintextCredentialError extends DexpaceError { + /** The concrete step that refused, as AUTH-28 requires the error to name. */ + readonly stepName: string; + /** The resolved auth scheme whose credential would have been stamped. */ + readonly scheme: string; + + /** + * @param stepName - the concrete step that refused. + * @param scheme - the resolved auth scheme. + */ + constructor(stepName: string, scheme: string) { + super( + `${stepName} refuses to send a ${scheme} credential over a non-HTTPS URL`, + ); + this.stepName = stepName; + this.scheme = scheme; + } +} diff --git a/packages/core/src/auth/md5.test.ts b/packages/core/src/auth/md5.test.ts new file mode 100644 index 0000000..8348a2e --- /dev/null +++ b/packages/core/src/auth/md5.test.ts @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/md5.test.ts +// Exercises: AUTH-15, AUTH-17 (MD5 correctness against RFC 1321's own test vectors, and the +// lower-case hex rendering the Digest response is built from). +import {describe, expect, test} from 'bun:test'; +import {md5, toHex} from './md5.js'; + +function md5Hex(input: string): string { + return toHex(md5(new TextEncoder().encode(input))); +} + +describe('md5 (RFC 1321 test vectors)', () => { + test('the empty string', () => { + expect(md5Hex('')).toBe('d41d8cd98f00b204e9800998ecf8427e'); + }); + + test('"a"', () => { + expect(md5Hex('a')).toBe('0cc175b9c0f1b6a831c399e269772661'); + }); + + test('"abc"', () => { + expect(md5Hex('abc')).toBe('900150983cd24fb0d6963f7d28e17f72'); + }); + + test('"message digest"', () => { + expect(md5Hex('message digest')).toBe('f96b697d7cb7938d525a2f31aaf161d0'); + }); + + test('the lowercase alphabet, exercising a multi-block input', () => { + expect(md5Hex('abcdefghijklmnopqrstuvwxyz')).toBe( + 'c3fcd3d76192e4007dfb496cca67e13b', + ); + }); + + test('the 62-character alphanumeric vector', () => { + expect( + md5Hex('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'), + ).toBe('d174ab98d277d9f5a5611c2c9f419d9f'); + }); + + test('the 80-digit vector, exercising a two-block input', () => { + expect(md5Hex('1234567890'.repeat(8))).toBe( + '57edf4a22be3c955ac49da2e2107b67a', + ); + }); + + test('a 55-byte input, the last length that pads into a single block', () => { + expect(md5Hex('a'.repeat(55))).toBe('ef1772b6dff9a122358552954ad0df65'); + }); + + test('a 56-byte input, the first length that forces a second block', () => { + expect(md5Hex('a'.repeat(56))).toBe('3b0c8ac703f828b04c6c197006d17218'); + }); + + test('a 64-byte input, exactly one block before padding', () => { + expect(md5Hex('a'.repeat(64))).toBe('014842d480b571495a4a0363793f7367'); + }); + + test('non-ASCII bytes hash by their UTF-8 encoding', () => { + expect(md5Hex('é')).toBe('66ddcd97cfdeabb2f6fb8a999b4bc76f'); + }); + + test('the digest is 16 bytes', () => { + expect(md5(new Uint8Array()).length).toBe(16); + }); + + test('is pure -- the same input hashes identically twice', () => { + const input = new TextEncoder().encode('repeat me'); + expect(toHex(md5(input))).toBe(toHex(md5(input))); + }); +}); + +describe('toHex', () => { + test('pads each byte to two lower-case hex digits', () => { + expect(toHex(new Uint8Array([0, 15, 255]))).toBe('000fff'); + }); + + test('renders the empty array as the empty string', () => { + expect(toHex(new Uint8Array())).toBe(''); + }); +}); diff --git a/packages/core/src/auth/md5.ts b/packages/core/src/auth/md5.ts new file mode 100644 index 0000000..6653893 --- /dev/null +++ b/packages/core/src/auth/md5.ts @@ -0,0 +1,154 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/md5.ts + +/** + * RFC 1321 MD5, hand-rolled and dependency-free. + * + * Web Crypto's `subtle.digest()` deliberately excludes MD5 — the algorithm is out of the standard on + * security grounds — yet RFC 7616 Digest still requires MD5/MD5-sess for interop with servers that + * have not adopted SHA-256 (AUTH-15). Adding an npm dependency for it would violate SEAM-1's + * zero-runtime-dependency rule, and reaching for `node:crypto` would cost the portability to + * browsers/Deno/Workers that `sdk-design-nodejs/06` picks Web Crypto to keep. + * + * @packageDocumentation + */ + +const SHIFTS = [ + 7, 12, 17, 22, 7, 12, 17, 22, 7, 12, 17, 22, 7, 12, 17, 22, 5, 9, 14, 20, 5, + 9, 14, 20, 5, 9, 14, 20, 5, 9, 14, 20, 4, 11, 16, 23, 4, 11, 16, 23, 4, 11, + 16, 23, 4, 11, 16, 23, 6, 10, 15, 21, 6, 10, 15, 21, 6, 10, 15, 21, 6, 10, 15, + 21, +] as const; + +// `/*#__PURE__*/`, because this is a top-level CALL, and `docs/knowledge/harvested/performance.md` is explicit +// that modules must do no work at import time — a top-level call is a side effect the bundler must +// preserve, and that pins the module in the bundle. `@dexpace/core` declares `"sideEffects": false`; +// without the annotation a bundler cannot prove these 64 `Math.sin` calls are pure, so `md5.ts` and +// its table are retained by every consumer that transitively imports anything reaching them, +// including one that never touches Digest. +// Deeply immutable via the freeze, which is what earns the CONSTANT_CASE (naming-conventions.md). +const CONSTANTS: readonly number[] = /*#__PURE__*/ Object.freeze( + Array.from( + {length: 64}, + (_, i) => Math.floor(Math.abs(Math.sin(i + 1)) * 2 ** 32) >>> 0, + ), +); + +function leftRotate(value: number, bits: number): number { + return ((value << bits) | (value >>> (32 - bits))) >>> 0; +} + +/** RFC 1321 §3.1: pad to a multiple of 64 bytes with a single 0x80, zeros, then the original bit length. */ +function pad(message: Uint8Array): Uint8Array { + const bitLength = BigInt(message.length) * 8n; + const paddingLength = (((56 - ((message.length + 1) % 64)) % 64) + 64) % 64; + const result = new Uint8Array(message.length + 1 + paddingLength + 8); + result.set(message); + result[message.length] = 0x80; + // `result` is freshly allocated, so its byteOffset is 0 and the buffer view needs no offset. + new DataView(result.buffer).setBigUint64(result.length - 8, bitLength, true); + return result; +} + +/** The three state words RFC 1321's per-round auxiliary function reads. Bundled so `roundFunction` + * stays within `max-params`. */ +interface RoundWords { + readonly b: number; + readonly c: number; + readonly d: number; +} + +function roundFunction(i: number, words: RoundWords): number { + const {b, c, d} = words; + if (i < 16) return (b & c) | (~b & d); + if (i < 32) return (d & b) | (~d & c); + if (i < 48) return b ^ c ^ d; + return c ^ (b | ~d); +} + +function messageIndex(i: number): number { + if (i < 16) return i; + if (i < 32) return (5 * i + 1) % 16; + if (i < 48) return (3 * i + 5) % 16; + return (7 * i) % 16; +} + +interface State { + a: number; + b: number; + c: number; + d: number; +} + +function processBlock(words: readonly number[], state: State): State { + let {a, b, c, d} = state; + for (let i = 0; i < 64; i += 1) { + const f = + (roundFunction(i, {b, c, d}) + + a + + (CONSTANTS[i] ?? 0) + + (words[messageIndex(i)] ?? 0)) >>> + 0; + a = d; + d = c; + c = b; + b = (b + leftRotate(f, SHIFTS[i] ?? 0)) >>> 0; + } + return { + a: (state.a + a) >>> 0, + b: (state.b + b) >>> 0, + c: (state.c + c) >>> 0, + d: (state.d + d) >>> 0, + }; +} + +/** + * Computes the RFC 1321 MD5 digest of `message` (AUTH-15–AUTH-17). + * + * Pure: no shared state, no allocation the caller can observe, safe for concurrent invocation + * (AUTH-24). + * + * @param message - the bytes to hash. + * @returns the 16-byte digest. + * + * @internal + */ +export function md5(message: Uint8Array): Uint8Array { + const data = pad(message); + // `data` comes straight from `pad`, so it is a fresh, offset-0 view over its own buffer. + const view = new DataView(data.buffer); + let state: State = { + a: 0x67452301, + b: 0xefcdab89, + c: 0x98badcfe, + d: 0x10325476, + }; + + for (let chunkStart = 0; chunkStart < data.length; chunkStart += 64) { + const words = Array.from({length: 16}, (_, i) => + view.getUint32(chunkStart + i * 4, true), + ); + state = processBlock(words, state); + } + + const digest = new Uint8Array(16); + const outView = new DataView(digest.buffer); + outView.setUint32(0, state.a, true); + outView.setUint32(4, state.b, true); + outView.setUint32(8, state.c, true); + outView.setUint32(12, state.d, true); + return digest; +} + +/** + * Renders bytes as lower-case hex, two digits per byte — the form AUTH-17 requires for HA1/HA2 and + * the Digest response. + * + * @param bytes - the bytes to render. + * @returns the lower-case hex string, twice as long as `bytes`. + * + * @internal + */ +export function toHex(bytes: Uint8Array): string { + return Array.from(bytes, byte => byte.toString(16).padStart(2, '0')).join(''); +} diff --git a/packages/core/src/auth/preset.test.ts b/packages/core/src/auth/preset.test.ts new file mode 100644 index 0000000..733207b --- /dev/null +++ b/packages/core/src/auth/preset.test.ts @@ -0,0 +1,288 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/preset.test.ts +// Exercises: PIPE-24 ("installs into empty pillar slots only" -- true by construction, since the preset +// always starts from a fresh PipelineBuilder), PIPE-39 (installs exactly the pillars that exist), and +// jointly with 5b: PIPE-2's "auth executes per redirect hop, not once for the whole call" plus +// AUTH-29's marker-CONSUMPTION side (5b produced the marker and routed consumption here), OBS-29 + +// CTX-16 (the preset forwards an instrumentation bundle and an operation name to the built pipeline). +import {describe, expect, test} from 'bun:test'; +import {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; +import {PipelineBuilder} from '../pipeline/builder.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {CROSS_ORIGIN_MARKER_HEADER} from '../redirect/cross-origin.js'; +import {REDIRECT_STEP_TYPE} from '../redirect/redirect-step.js'; +import {withRedirect} from '../redirect/strip-marker-step.js'; +import {RETRY_STEP_TYPE} from '../retry/retry-step.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {AUTH_STEP_TYPE} from './auth-step.js'; +import {createBearerToken} from './credential.js'; +import {createAuthDescriptor} from './descriptor.js'; +import {standardResilience, type StandardResilienceOptions} from './preset.js'; +import {createAuthRequirement} from './requirement.js'; +import {LOGGING_STEP_TYPE} from '../observability/logging-step.js'; + +function aRequest(url = 'https://example.com/start'): Request { + return Request.newBuilder().url(url).build(); +} + +// `FakeTransport` does not itself set a Location -- 5b's `decide()` reads it off `Response.headers`, so a +// scripted 3xx entry must carry one explicitly. `setInbound`, not `set`: Location is an inbound header. +function withLocation(response: Response, location: string): Response { + return response + .newBuilder() + .headers( + response.headers.newBuilder().setInbound('Location', location).build(), + ) + .build(); +} + +function bearerOptions(): StandardResilienceOptions { + return { + auth: { + credentials: { + bearer: { + provider: () => Promise.resolve(createBearerToken('tok', 60_000)), + }, + }, + tiers: {client: createAuthDescriptor([createAuthRequirement('OAUTH2')])}, + clock: {now: () => 0}, + }, + }; +} + +describe('standardResilience', () => { + test('installs the resilience pillars plus 5b’s marker guard and 7b logging (PIPE-24/PIPE-39)', () => { + const runtime = standardResilience( + new FakeTransport([countingResponse(200).response]), + bearerOptions(), + ); + const types = runtime.steps.map(step => step.type); + + expect(types).toContain(REDIRECT_STEP_TYPE); + expect(types).toContain(RETRY_STEP_TYPE); + expect(types).toContain(AUTH_STEP_TYPE); + expect(types).toContain(LOGGING_STEP_TYPE); + // redirectStep + its POST_AUTH marker guard + retryStep + authStep + loggingStep. + expect(types).toHaveLength(5); + }); + + test('the pillars flatten in redirect-then-retry-then-auth-then-logging order (AUTH-27/PIPE-2)', () => { + const runtime = standardResilience( + new FakeTransport([countingResponse(200).response]), + bearerOptions(), + ); + const order = runtime.steps.map(step => step.stage); + + expect(order.indexOf('REDIRECT')).toBeLessThan(order.indexOf('RETRY')); + expect(order.indexOf('RETRY')).toBeLessThan(order.indexOf('AUTH')); + expect(order.indexOf('AUTH')).toBeLessThan(order.indexOf('LOGGING')); + }); + + test('NO_AUTH is the default when no auth option is supplied', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + const runtime = standardResilience(transport); + + await runtime.send(aRequest('https://example.com')); + + expect( + transport.calls[0]?.request.headers.get('Authorization'), + ).toBeUndefined(); + }); + + test('the default NO_AUTH step does not trip the HTTPS guard on a plain-HTTP call (AUTH-28)', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + + await standardResilience(transport).send(aRequest('http://example.com')); + + expect(transport.sendCount).toBe(1); + }); +}); + +describe('standardResilience with redirects (PIPE-2 + AUTH-29, jointly with 5b)', () => { + test('joint conformance (PIPE-2 + AUTH-29): credential absent on the cross-origin hop, restamped on return to same-origin', async () => { + const toCrossOrigin = withLocation( + countingResponse(302).response, + 'https://evil.example/mid', + ); + const backToSeedOrigin = withLocation( + countingResponse(302).response, + 'https://example.com/final', + ); + const finalHop = countingResponse(200); + const transport = new FakeTransport([ + toCrossOrigin, + backToSeedOrigin, + finalHop.response, + ]); + + const runtime = standardResilience(transport, bearerOptions()); + const response = await runtime.send(aRequest()); + + expect(transport.calls).toHaveLength(3); + // Seed hop: same-origin, stamped. + expect(transport.calls[0]?.request.headers.get('Authorization')).toBe( + 'Bearer tok', + ); + // Cross-origin hop: suppressed (AUTH-29). + expect( + transport.calls[1]?.request.headers.get('Authorization'), + ).toBeUndefined(); + // Back to the seed origin: re-stamped, proving auth re-runs PER HOP (PIPE-2). + expect(transport.calls[2]?.request.headers.get('Authorization')).toBe( + 'Bearer tok', + ); + expect(response).toBe(finalHop.response); + }); +}); + +describe('the cross-origin marker is load-bearing on both sides (REDIR-11/AUTH-29)', () => { + test('the internal cross-origin marker never reaches the wire (REDIR-11/AUTH-29)', async () => { + const toCrossOrigin = withLocation( + countingResponse(302).response, + 'https://evil.example/mid', + ); + const transport = new FakeTransport([ + toCrossOrigin, + countingResponse(200).response, + ]); + + await standardResilience(transport, bearerOptions()).send(aRequest()); + + for (const call of transport.calls) { + expect(call.request.headers.has(CROSS_ORIGIN_MARKER_HEADER)).toBe(false); + } + }); + + test('neither the redirect guard nor the marker check alone is sufficient -- both are independently necessary', async () => { + // A minimal AUTH-stage step that IGNORES the cross-origin marker and always stamps -- standing in + // for "what would happen if 5c's marker check were removed". With THIS step installed instead of the + // real authStep(), the credential leaks onto the cross-origin hop, proving the marker suppresses + // something observable rather than headers merely happening to come out empty. + const leakyAuthStep: StepDescriptor = { + type: Symbol('leaky-auth'), + stage: 'AUTH', + fn: (request, ctx) => { + const stamped = request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('Authorization', 'Bearer leaked') + .build(), + ) + .build(); + return ctx.next(stamped); + }, + }; + const toCrossOrigin = withLocation( + countingResponse(302).response, + 'https://evil.example/mid', + ); + const transport = new FakeTransport([ + toCrossOrigin, + countingResponse(200).response, + ]); + + const runtime = withRedirect(new PipelineBuilder(transport)) + .append(leakyAuthStep) + .build(); + await runtime.send(aRequest()); + + // 5b's redirect step already strips Authorization unconditionally on every re-issue (REDIR-7), so + // this variant demonstrates the OTHER half: a leaky auth step re-attaches a credential redirect just + // stripped, proving 5b's stripping alone is not sufficient either. Both layers are load-bearing. + expect(transport.calls[1]?.request.headers.get('Authorization')).toBe( + 'Bearer leaked', + ); + }); +}); + +describe('standardResilience logging options (Phase 7b)', () => { + test('logging options pass through to the installed logging step', async () => { + const {createLogger} = await import('../observability/logger.js'); + const events: string[] = []; + const testLogger = createLogger((_level, fields) => { + const name = fields.get('event'); + if (typeof name === 'string') events.push(name); + }); + + const transport = new FakeTransport([countingResponse(200).response]); + const runtime = standardResilience(transport, { + logging: {logger: testLogger, granularity: 'headers'}, + }); + + await runtime.send(aRequest()); + + expect(events).toEqual(['http.request', 'http.response']); + }); +}); + +describe('standardResilience instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle opens one operation span per send, and the name reaches the step', async () => { + const spanNames: string[] = []; + const factoryNames: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + const tracer: Tracer = { + startSpan(name: string): Span { + spanNames.push(name); + return span; + }, + }; + let seen: string | undefined; + const probe: StepDescriptor = { + type: Symbol('operation-name-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen = + 'operationName' in ctx.context + ? ctx.context.operationName + : undefined; + return ctx.next(request); + }, + }; + + const runtime = PipelineBuilder.seedFrom( + standardResilience( + new FakeTransport([ + countingResponse(200).response, + countingResponse(200).response, + ]), + { + instrumentation: createInstrumentationBundle(operationName => { + factoryNames.push(operationName); + return tracer; + }), + operationName: 'GetUser', + }, + ), + 'flatten', + ) + .append(probe) + .build(); + + await runtime.send(aRequest()); + await runtime.send(aRequest()); + + // `Runtime.send()` asks the factory for the operation's own tracer once per call (OBS-29's 1:1 + // binding), and the LOGGING pillar asks for one labelled with CTX-16's operation name per attempt. + expect( + factoryNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(factoryNames.filter(name => name === 'GetUser')).toHaveLength(2); + expect( + spanNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(seen).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/auth/preset.ts b/packages/core/src/auth/preset.ts new file mode 100644 index 0000000..6317375 --- /dev/null +++ b/packages/core/src/auth/preset.ts @@ -0,0 +1,124 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/preset.ts +import {PipelineBuilder, type PipelineOptions} from '../pipeline/builder.js'; +import type {Runtime} from '../pipeline/runtime.js'; +import type {RedirectSettings} from '../redirect/settings.js'; +import {withRedirect} from '../redirect/strip-marker-step.js'; +import {retryStep, type RetryStepOptions} from '../retry/retry-step.js'; +import type {Transport} from '../seams/transport.js'; +import {authStep, type AuthStepSettings} from './auth-step.js'; +import {createAuthDescriptor} from './descriptor.js'; +import {createAuthRequirement} from './requirement.js'; +import { + loggingStep, + type LoggingStepSettings, +} from '../observability/logging-step.js'; + +/** + * Per-pillar overrides for {@link standardResilience}. Every slot is optional; an omitted one takes + * that pillar's own defaults. + * + * @public + */ +export interface StandardResilienceOptions extends PipelineOptions { + /** Retry settings and injected seams; omitted yields 5a's spec defaults. */ + readonly retry?: RetryStepOptions | undefined; + /** Redirect policy overrides; omitted yields 5b's spec defaults. */ + readonly redirect?: Partial<RedirectSettings> | undefined; + /** + * Auth configuration. Required if any credential tier is meant to apply; omitted installs a + * `NO_AUTH`-only step, which stamps nothing and never trips the HTTPS guard. + */ + readonly auth?: AuthStepSettings | undefined; + /** Logging and observability settings; omitted installs the default inert step. */ + readonly logging?: LoggingStepSettings | undefined; +} + +// Built lazily rather than as a top-level `const NO_AUTH_SETTINGS = ...`: a module-scope factory call +// is import-time work a bundler must preserve (`docs/knowledge/harvested/performance.md`), and it would pin +// descriptor.ts/requirement.ts into every bundle that imports the preset. The allocation is per call, +// but the preset is constructed once per client, not per request. +function noAuthSettings(): AuthStepSettings { + return { + credentials: {}, + tiers: {client: createAuthDescriptor([createAuthRequirement('NO_AUTH')])}, + }; +} + +/** + * Assembles the standard resilience pipeline: redirect, then retry, then auth, then logging (PIPE-24, PIPE-39). + * + * The order is AUTH-27's "redirect wraps retry wraps auth", so the auth step re-resolves and + * re-stamps per redirect hop and per retry attempt (PIPE-2). Redirect is installed through 5b's + * `withRedirect()`, which seats the pillar step AND its `POST_AUTH` cross-origin-marker guard + * together, so the internal marker can never reach the wire. + * + * PIPE-24's "installs into empty pillar slots only" is true BY CONSTRUCTION: this function always + * starts from a fresh `PipelineBuilder`, so no slot can be occupied and no runtime check is needed. A + * caller wanting to layer this preset onto an already-customized builder reaches for + * {@link PipelineBuilder.seedFrom} (`'nest'` or `'flatten'`) rather than this function growing a + * "skip occupied slots" branch — the two features compose. + * + * `LOGGING` installs {@link loggingStep} to emit telemetry and metrics around dispatches. `SERDE` remains + * reserved with no shipped behavior anywhere in this roadmap's current scope. + * + * The two inherited {@link PipelineOptions} fields are pipeline-wide rather than per-pillar: + * `instrumentation` is the bundle every context of every call carries, and the source of the one + * `http.client.operation` span each `send()` opens (`OBS-29`); `operationName` is `CTX-16`'s advisory + * label. Omitting `instrumentation` leaves the no-op bundle in place, which opens no span at all — it + * is the only way to switch tracing on for a preset-built pipeline. + * + * This function only assembles the pipeline. The failures below surface from the returned runtime's + * `send()`, and are documented here because this factory is where a caller chooses the auth + * configuration that determines whether they can occur at all. + * + * @param transport - the terminal transport. Never closed by the pipeline (PIPE-27). + * **Exceeding the redirect hop cap does NOT throw** (REDIR-17): the current 3xx response is returned + * to the caller unfollowed. See {@link redirectStep}. Stated before the tags below because TSDoc + * folds trailing prose into the preceding block tag. + * + * @param options - per-pillar overrides. + * @returns the built, immutable runtime. + * @throws PlaintextCredentialError — from the returned runtime's `send()` — when a credentialed scheme + * meets a non-HTTPS URL (AUTH-28). + * @throws AuthResolutionError — from the returned runtime's `send()` — when no configured credential + * satisfies the resolved auth tier (AUTH-6; AUTH-4 governs only WHICH tier is selected), or a token + * provider returns a null or already-expired token (AUTH-35). + * @throws HeaderValidationError — from the returned runtime's `send()` — when credential material + * will not fit in a header value (HTTP-18). + * @throws SchemeDowngradeError — from the returned runtime's `send()` — when a redirect attempts an HTTPS to HTTP downgrade not permitted by settings (REDIR-14/15). + * @throws an assertion failure (a caller bug, not a catchable condition) — synchronously from this function — when any pillar's settings are + * invalid, including a non-finite bearer refresh margin or a non-header-safe Digest username. A + * caller-supplied `TokenProvider` or `challengeHook` error passes through `send()` unwrapped. + * + * @example + * ```ts + * const client = standardResilience(transport, { + * auth: { + * credentials: {apiKey: {credential: new ApiKeyCredential(process.env.API_KEY ?? '')}}, + * tiers: {client: createAuthDescriptor([createAuthRequirement('API_KEY')])}, + * }, + * }); + * const response = await client.send( + * Request.newBuilder().url('https://api.example.com/v1/things').build(), + * ); + * ``` + * + * @public + */ +export function standardResilience( + transport: Transport, + options: StandardResilienceOptions = {}, +): Runtime { + // The two `PipelineOptions` fields are pipeline-wide rather than per-pillar, so they go to the + // builder rather than into a step's settings; everything below installs one pillar each. + const builder = new PipelineBuilder(transport, { + instrumentation: options.instrumentation, + operationName: options.operationName, + }); + return withRedirect(builder, options.redirect) + .append(retryStep(options.retry)) + .append(authStep(options.auth ?? noAuthSettings())) + .append(loggingStep(options.logging)) + .build(); +} diff --git a/packages/core/src/auth/requirement.test.ts b/packages/core/src/auth/requirement.test.ts new file mode 100644 index 0000000..ddb2c26 --- /dev/null +++ b/packages/core/src/auth/requirement.test.ts @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/requirement.test.ts +// Exercises: AUTH-2 (frozen data shape, defensive copies of scopes/params, value equality). +import {describe, expect, test} from 'bun:test'; +import {authRequirementsEqual, createAuthRequirement} from './requirement.js'; + +describe('createAuthRequirement', () => { + test('defaults scopes to empty and params to an empty map', () => { + const requirement = createAuthRequirement('BASIC'); + expect(requirement.scopes).toEqual([]); + expect(requirement.params.size).toBe(0); + }); + + test('is frozen', () => { + expect(Object.isFrozen(createAuthRequirement('BASIC'))).toBe(true); + }); + + test('freezes the scopes array too', () => { + expect( + Object.isFrozen(createAuthRequirement('OAUTH2', ['read']).scopes), + ).toBe(true); + }); + + test('defensively copies the scopes array', () => { + const scopes = ['read']; + const requirement = createAuthRequirement('OAUTH2', scopes); + scopes.push('write'); + expect(requirement.scopes).toEqual(['read']); + }); + + test('defensively copies the params map', () => { + const params = new Map([['tenant', 'a']]); + const requirement = createAuthRequirement('OAUTH2', [], params); + params.set('tenant', 'b'); + expect(requirement.params.get('tenant')).toBe('a'); + }); +}); + +describe('authRequirementsEqual', () => { + test('true for identical scheme/scopes/params, regardless of construction order', () => { + const a = createAuthRequirement( + 'OAUTH2', + ['read', 'write'], + new Map([['tenant', 'x']]), + ); + const b = createAuthRequirement( + 'OAUTH2', + ['read', 'write'], + new Map([['tenant', 'x']]), + ); + expect(authRequirementsEqual(a, b)).toBe(true); + }); + + test('false for a differing scheme', () => { + expect( + authRequirementsEqual( + createAuthRequirement('BASIC'), + createAuthRequirement('DIGEST'), + ), + ).toBe(false); + }); + + test('false for differing scopes', () => { + const a = createAuthRequirement('OAUTH2', ['read']); + const b = createAuthRequirement('OAUTH2', ['write']); + expect(authRequirementsEqual(a, b)).toBe(false); + }); + + test('scope ORDER is part of the value, not a set comparison', () => { + const a = createAuthRequirement('OAUTH2', ['read', 'write']); + const b = createAuthRequirement('OAUTH2', ['write', 'read']); + expect(authRequirementsEqual(a, b)).toBe(false); + }); + + test('false for a differing scope count', () => { + const a = createAuthRequirement('OAUTH2', ['read']); + const b = createAuthRequirement('OAUTH2', ['read', 'write']); + expect(authRequirementsEqual(a, b)).toBe(false); + }); + + test('false for differing params', () => { + const a = createAuthRequirement('OAUTH2', [], new Map([['tenant', 'x']])); + const b = createAuthRequirement('OAUTH2', [], new Map([['tenant', 'y']])); + expect(authRequirementsEqual(a, b)).toBe(false); + }); + + test('false for a differing param count', () => { + const a = createAuthRequirement('OAUTH2', [], new Map([['tenant', 'x']])); + const b = createAuthRequirement( + 'OAUTH2', + [], + new Map([ + ['tenant', 'x'], + ['region', 'eu'], + ]), + ); + expect(authRequirementsEqual(a, b)).toBe(false); + }); +}); diff --git a/packages/core/src/auth/requirement.ts b/packages/core/src/auth/requirement.ts new file mode 100644 index 0000000..89ea94c --- /dev/null +++ b/packages/core/src/auth/requirement.ts @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/requirement.ts +import type {AuthScheme} from './scheme.js'; + +/** + * AUTH-2: one scheme bound to its own OAuth scopes and params. + * + * A frozen data shape plus a pure equality function — the same "data and functions, not objects" + * call 4a made for context types and 4c made for `Stage`, rather than a class with an `equals()` + * method. + * + * @public + */ +export interface AuthRequirement { + /** The bound scheme. */ + readonly scheme: AuthScheme; + /** Meaningful only for `OAUTH2`; preserved verbatim, never inspected by resolution (AUTH-2). */ + readonly scopes: readonly string[]; + /** Scheme-specific parameters, preserved verbatim and never inspected by resolution (AUTH-2). */ + readonly params: ReadonlyMap<string, string>; +} + +/** + * Builds a frozen {@link AuthRequirement}, defensively copying both collections so a caller mutating + * its inputs afterwards cannot reach the stored value (AUTH-2). + * + * @param scheme - the scheme this requirement binds. + * @param scopes - OAuth scopes; meaningful only for `OAUTH2`. + * @param params - scheme-specific parameters. + * @returns the frozen requirement. + * + * @public + */ +export function createAuthRequirement( + scheme: AuthScheme, + scopes: readonly string[] = [], + params: ReadonlyMap<string, string> = new Map(), +): AuthRequirement { + // `Object.freeze` is SHALLOW. `docs/knowledge/harvested/data-modeling.md` requires a frozen value object to + // hold only primitives or already-frozen/read-only values, never a mutable object that stays + // writable behind the freeze. `new Map(params)` satisfies AUTH-2's literal clause -- caller-side + // mutation cannot reach the stored value -- but leaves the copy itself writable behind the + // `ReadonlyMap` type. Rebuilding a frozen Map on every read is not worth it for a value this small + // and this rarely read; instead the copy is made once here, and nothing in this package ever + // re-casts `AuthRequirement['params']` back to `Map` (AUTH-2 also bars resolution from inspecting + // params at all), which is what keeps the `ReadonlyMap` type honest in practice. + return Object.freeze({ + scheme, + scopes: Object.freeze([...scopes]), + params: new Map(params), + }); +} + +function scopesEqual(a: readonly string[], b: readonly string[]): boolean { + return a.length === b.length && a.every((scope, index) => scope === b[index]); +} + +function paramsEqual( + a: ReadonlyMap<string, string>, + b: ReadonlyMap<string, string>, +): boolean { + return ( + a.size === b.size && [...a].every(([key, value]) => b.get(key) === value) + ); +} + +/** + * AUTH-2's value-based equality: over scheme, scopes (ordered), and params. + * + * @param a - the left requirement. + * @param b - the right requirement. + * @returns `true` when all three components match. + * + * @public + */ +export function authRequirementsEqual( + a: AuthRequirement, + b: AuthRequirement, +): boolean { + return ( + a.scheme === b.scheme && + scopesEqual(a.scopes, b.scopes) && + paramsEqual(a.params, b.params) + ); +} diff --git a/packages/core/src/auth/resolve.test.ts b/packages/core/src/auth/resolve.test.ts new file mode 100644 index 0000000..8b87d37 --- /dev/null +++ b/packages/core/src/auth/resolve.test.ts @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/resolve.test.ts +// Exercises: AUTH-4 (perCall ?? operation ?? client, first PRESENT wins, no fallthrough on failure), +// AUTH-5 (first requirement whose scheme is NO_AUTH or in availableSchemes wins), AUTH-6 (all tiers +// absent is a programmer error), AUTH-7 (pure function, no hidden state). +import {describe, expect, test} from 'bun:test'; +import {InvariantViolation} from '../invariant.js'; +import {createAuthDescriptor} from './descriptor.js'; +import {AuthResolutionError} from './errors.js'; +import {createAuthRequirement} from './requirement.js'; +import {resolveAuthRequirement} from './resolve.js'; + +describe('tier selection (AUTH-4)', () => { + test('perCall wins when present, even if operation/client are also present', () => { + const requirement = resolveAuthRequirement( + { + perCall: createAuthDescriptor([createAuthRequirement('BASIC')]), + operation: createAuthDescriptor([createAuthRequirement('DIGEST')]), + client: createAuthDescriptor([createAuthRequirement('API_KEY')]), + }, + new Set(['BASIC', 'DIGEST', 'API_KEY']), + ); + expect(requirement.scheme).toBe('BASIC'); + }); + + test('operation wins over client when perCall is absent', () => { + const requirement = resolveAuthRequirement( + { + operation: createAuthDescriptor([createAuthRequirement('DIGEST')]), + client: createAuthDescriptor([createAuthRequirement('API_KEY')]), + }, + new Set(['DIGEST', 'API_KEY']), + ); + expect(requirement.scheme).toBe('DIGEST'); + }); + + test('client is used when it is the only tier present', () => { + const requirement = resolveAuthRequirement( + {client: createAuthDescriptor([createAuthRequirement('API_KEY')])}, + new Set(['API_KEY']), + ); + expect(requirement.scheme).toBe('API_KEY'); + }); + + test('a lower tier is NEVER consulted once a higher one is present, even if unsatisfiable', () => { + expect(() => + resolveAuthRequirement( + { + perCall: createAuthDescriptor([createAuthRequirement('DIGEST')]), + client: createAuthDescriptor([createAuthRequirement('BASIC')]), + }, + // would satisfy client's tier, but perCall is present and DIGEST is not available + new Set(['BASIC']), + ), + ).toThrow(AuthResolutionError); + }); + + test('an explicitly-undefined higher tier is treated as absent', () => { + const requirement = resolveAuthRequirement( + { + perCall: undefined, + client: createAuthDescriptor([createAuthRequirement('BASIC')]), + }, + new Set(['BASIC']), + ); + expect(requirement.scheme).toBe('BASIC'); + }); +}); + +describe('within-descriptor selection (AUTH-5)', () => { + test('the first requirement whose scheme is available wins, in preference order', () => { + const descriptor = createAuthDescriptor([ + createAuthRequirement('OAUTH2'), + createAuthRequirement('BASIC'), + ]); + const requirement = resolveAuthRequirement( + {client: descriptor}, + new Set(['BASIC']), + ); + expect(requirement.scheme).toBe('BASIC'); + }); + + test('NO_AUTH always wins regardless of availableSchemes', () => { + const descriptor = createAuthDescriptor([ + createAuthRequirement('NO_AUTH'), + createAuthRequirement('BASIC'), + ]); + const requirement = resolveAuthRequirement({client: descriptor}, new Set()); + expect(requirement.scheme).toBe('NO_AUTH'); + }); + + test('scopes and params are never inspected -- only the scheme decides', () => { + const descriptor = createAuthDescriptor([ + createAuthRequirement('OAUTH2', ['read'], new Map([['tenant', 'x']])), + ]); + const requirement = resolveAuthRequirement( + {client: descriptor}, + new Set(['OAUTH2']), + ); + expect(requirement.scopes).toEqual(['read']); + expect(requirement.params.get('tenant')).toBe('x'); + }); + + test('an unsatisfiable descriptor throws AuthResolutionError naming both schemes', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('DIGEST')]); + try { + resolveAuthRequirement({client: descriptor}, new Set(['BASIC'])); + throw new Error('expected a throw'); + } catch (error) { + expect(error).toBeInstanceOf(AuthResolutionError); + expect((error as Error).message).toContain('DIGEST'); + expect((error as Error).message).toContain('BASIC'); + } + }); + + test('the thrown error carries required schemes in PREFERENCE order (AUTH-6)', () => { + const descriptor = createAuthDescriptor([ + createAuthRequirement('DIGEST'), + createAuthRequirement('OAUTH2'), + ]); + try { + resolveAuthRequirement({client: descriptor}, new Set(['BASIC'])); + throw new Error('expected a throw'); + } catch (error) { + expect((error as AuthResolutionError).requiredSchemes).toEqual([ + 'DIGEST', + 'OAUTH2', + ]); + expect((error as AuthResolutionError).availableSchemes).toEqual([ + 'BASIC', + ]); + } + }); +}); + +describe('AUTH-6: all tiers absent', () => { + test('is a programmer error, not AuthResolutionError', () => { + expect(() => resolveAuthRequirement({}, new Set())).toThrow( + InvariantViolation, + ); + expect(() => resolveAuthRequirement({}, new Set())).not.toThrow( + AuthResolutionError, + ); + }); +}); + +describe('AUTH-7: purity', () => { + test('the same inputs always resolve to an equal requirement', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('BASIC')]); + const first = resolveAuthRequirement( + {client: descriptor}, + new Set(['BASIC']), + ); + const second = resolveAuthRequirement( + {client: descriptor}, + new Set(['BASIC']), + ); + // Same object identity: resolve() picks from the existing descriptor, building nothing new. + expect(first).toBe(second); + }); +}); diff --git a/packages/core/src/auth/resolve.ts b/packages/core/src/auth/resolve.ts new file mode 100644 index 0000000..80b7ec6 --- /dev/null +++ b/packages/core/src/auth/resolve.ts @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/resolve.ts +import {invariant} from '../invariant.js'; +import type {AuthDescriptor} from './descriptor.js'; +import {AuthResolutionError} from './errors.js'; +import type {AuthRequirement} from './requirement.js'; +import type {AuthScheme} from './scheme.js'; + +/** + * AUTH-4's three configuration tiers, most specific first. Every slot is optional; at least one must + * be present at resolution time. + * + * @public + */ +export interface AuthTiers { + /** The per-call override, sourced from `RequestOptions.auth` by the AUTH pillar step. */ + readonly perCall?: AuthDescriptor | undefined; + /** + * The per-operation tier, sourced from `RequestOptions.operationAuth` by the AUTH pillar step. + * Selection is `perCall ?? operation ?? client`. + * + * It had no source until 2026-09-04, and the cost of that was measured rather than assumed: a + * consumer with per-operation descriptors had to fold them into `perCall` itself, which + * reimplemented this very precedence rule outside core and left core unable to tell a genuine + * per-call override from an operation's declared requirement. `examples/petstore/FINDINGS.md` §4 + * is that measurement; `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1 records the fix. + */ + readonly operation?: AuthDescriptor | undefined; + /** The client-wide tier, fixed at step construction. */ + readonly client?: AuthDescriptor | undefined; +} + +/** + * Resolves the single {@link AuthRequirement} a call should satisfy (AUTH-4, AUTH-5, AUTH-7). + * + * Tier selection is `perCall ?? operation ?? client` — the first tier PRESENT, not the first that + * succeeds. If the selected tier lists no satisfiable scheme, {@link AuthResolutionError} is thrown + * naming that tier's schemes; a lower tier is never consulted, because the caller asked for the + * override explicitly (AUTH-4). + * + * Satisfiability is judged on scheme identity alone (AUTH-5): `NO_AUTH` always, otherwise membership + * in `availableSchemes`. No concrete credential value is ever inspected, which is why the caller + * derives `availableSchemes` from the credential types it configured rather than passing credentials + * in. + * + * Pure and stateless (AUTH-7): the returned requirement is the very object the descriptor already + * holds, not a copy. + * + * @param tiers - the three configuration tiers; at least one must be present. + * @param availableSchemes - the schemes a credential is actually configured for. + * @returns the first satisfiable requirement from the selected tier, in declared order. + * @throws AuthResolutionError when the selected tier lists no satisfiable scheme (AUTH-6). + * @throws an assertion failure (a caller bug, not a catchable condition) when every tier is absent — a caller misconfiguration, not an + * operational failure (AUTH-6, per the plan's Global Constraints). + * + * @public + */ +export function resolveAuthRequirement( + tiers: AuthTiers, + availableSchemes: ReadonlySet<AuthScheme>, +): AuthRequirement { + const descriptor = tiers.perCall ?? tiers.operation ?? tiers.client; + invariant( + descriptor !== undefined, + 'resolveAuthRequirement: at least one auth tier must be configured', + ); + + const match = descriptor.requirements.find( + requirement => + requirement.scheme === 'NO_AUTH' || + availableSchemes.has(requirement.scheme), + ); + if (match === undefined) { + const requiredSchemes = descriptor.requirements.map( + requirement => requirement.scheme, + ); + throw AuthResolutionError.unsatisfiable(requiredSchemes, [ + ...availableSchemes, + ]); + } + return match; +} diff --git a/packages/core/src/auth/scheme.ts b/packages/core/src/auth/scheme.ts new file mode 100644 index 0000000..618cb10 --- /dev/null +++ b/packages/core/src/auth/scheme.ts @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/scheme.ts + +/** + * AUTH-1: the recognized auth scheme set. `NO_AUTH` is a distinct sentinel meaning "may run + * anonymously / skip credential stamping", not a wire scheme. + * + * A string-literal union, not a TypeScript `enum` — `erasableSyntaxOnly` bars enums, and the scheme + * set has no behavior beyond identity and ordering. Same call 4c made for `Stage`. + * + * @public + */ +export type AuthScheme = 'OAUTH2' | 'API_KEY' | 'BASIC' | 'DIGEST' | 'NO_AUTH'; + +// There is deliberately no `AUTH_SCHEMES` array beside the union. One shipped briefly, documented +// "for enumeration", and nothing ever enumerated it: `availableSchemesOf` derives AUTH-5's set from +// which credentials are configured, and every other reader branches on the union exhaustively. Its +// only test asserted the array's five members against the union's five members, which is the +// constant restated rather than a behaviour, and would have passed against any five-element array. diff --git a/packages/core/src/auth/static-key.test.ts b/packages/core/src/auth/static-key.test.ts new file mode 100644 index 0000000..bb470d7 --- /dev/null +++ b/packages/core/src/auth/static-key.test.ts @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/static-key.test.ts +// Exercises: AUTH-26 (uniform over ApiKeyCredential/NameKeyCredential; default header Authorization; +// prefix + exactly one space when set; stateless -- no challenge involved). +import {describe, expect, test} from 'bun:test'; +import {ApiKeyCredential, NameKeyCredential} from './credential.js'; +import {stampStaticKey} from './static-key.js'; + +describe('stampStaticKey', () => { + test('defaults to the Authorization header, no prefix', () => { + const stamp = stampStaticKey(new ApiKeyCredential('secret')); + expect(stamp.headerName).toBe('Authorization'); + expect(stamp.headerValue).toBe('secret'); + }); + + test('applies a configured prefix with exactly one separating space', () => { + const stamp = stampStaticKey(new ApiKeyCredential('secret'), { + prefix: 'Bearer', + }); + expect(stamp.headerValue).toBe('Bearer secret'); + }); + + test('an empty prefix still contributes its separating space, rather than being ignored', () => { + // `undefined` means "no prefix"; `''` is a caller who explicitly configured one. Collapsing the + // two would make the option's absent state unreachable. + expect( + stampStaticKey(new ApiKeyCredential('secret'), {prefix: ''}).headerValue, + ).toBe(' secret'); + }); + + test('honors a configured header name', () => { + const stamp = stampStaticKey(new NameKeyCredential('x-api-key', 'secret'), { + headerName: 'X-Api-Key', + }); + expect(stamp.headerName).toBe('X-Api-Key'); + expect(stamp.headerValue).toBe('secret'); + }); + + test('treats NameKeyCredential uniformly with ApiKeyCredential -- only the secret is read, not .name', () => { + const stamp = stampStaticKey( + new NameKeyCredential('ignored-here', 'secret'), + ); + expect(stamp.headerName).toBe('Authorization'); + expect(stamp.headerValue).toBe('secret'); + }); + + test('is stateless -- the same credential stamps identically every call', () => { + const credential = new ApiKeyCredential('secret'); + expect(stampStaticKey(credential)).toEqual(stampStaticKey(credential)); + }); +}); diff --git a/packages/core/src/auth/static-key.ts b/packages/core/src/auth/static-key.ts new file mode 100644 index 0000000..7e8d480 --- /dev/null +++ b/packages/core/src/auth/static-key.ts @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/auth/static-key.ts +import { + credentialKey, + type ApiKeyCredential, + type NameKeyCredential, +} from './credential.js'; + +/** + * Where and how a static key is written (AUTH-26). + * + * @internal + */ +export interface StaticKeyOptions { + /** The header to write. Defaults to `Authorization` (AUTH-26). */ + readonly headerName?: string | undefined; + /** A scheme prefix; when set it is written followed by exactly one space (AUTH-26). */ + readonly prefix?: string | undefined; +} + +/** + * The header name/value pair a static-key stamp produces. + * + * @internal + */ +export interface StaticKeyStamp { + /** The header to write. */ + readonly headerName: string; + /** The value to write, prefix already applied. */ + readonly headerValue: string; +} + +/** + * AUTH-26: writes the secret into the configured header, prefixed by the configured prefix and one + * space when set. + * + * Uniform over both credential shapes. `NameKeyCredential.name` is deliberately NOT consulted: it is + * non-secret metadata for the redacted `toString` in `credential.ts`, not a header name — a caller + * that wants the name to select the header passes it as `options.headerName`, explicitly. + * + * Stateless after construction, and no challenge is involved: a static key is stamped preemptively, + * never in reaction to a 401. + * + * @param credential - the API key or name-key credential to stamp. + * @param options - header name and prefix overrides. + * @returns the header name and value to write. + * + * @internal + */ +export function stampStaticKey( + credential: ApiKeyCredential | NameKeyCredential, + options?: StaticKeyOptions, +): StaticKeyStamp { + const headerName = options?.headerName ?? 'Authorization'; + // `credentialKey()`, not a public `credential.key` getter: AUTH-8's secret stays off the published + // surface and this module is the one sanctioned reader. + const key = credentialKey(credential); + const headerValue = + options?.prefix === undefined ? key : `${options.prefix} ${key}`; + return {headerName, headerValue}; +} diff --git a/packages/core/src/body/body.test.ts b/packages/core/src/body/body.test.ts new file mode 100644 index 0000000..9f65ef4 --- /dev/null +++ b/packages/core/src/body/body.test.ts @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/body.test.ts +// Exercises: BODY-11/TRANSPORT-28 (FileBodyDescriptor recognition contract) +import {describe, expect, test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import type {Body, FileBodyDescriptor} from './body.js'; + +describe('FileBodyDescriptor (BODY-11/TRANSPORT-28 recognition contract)', () => { + test('is a Body with a discriminated file kind and structural fields', () => { + expectTypeOf<FileBodyDescriptor>().toExtend<Body>(); + expectTypeOf<FileBodyDescriptor['kind']>().toEqualTypeOf<'file'>(); + expectTypeOf<FileBodyDescriptor['path']>().toEqualTypeOf<string>(); + expectTypeOf<FileBodyDescriptor['start']>().toEqualTypeOf<number>(); + expectTypeOf<FileBodyDescriptor['count']>().toEqualTypeOf<number>(); + }); + + test("Body['kind'] accepts 'file' without a cast", () => { + const kind: Body['kind'] = 'file'; + expect(kind).toBe('file'); + }); +}); diff --git a/packages/core/src/body/body.ts b/packages/core/src/body/body.ts new file mode 100644 index 0000000..e2d9778 --- /dev/null +++ b/packages/core/src/body/body.ts @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/body.ts + +/** + * The core domain interface for HTTP message bodies. + * + * @public + */ +export interface Body { + /** + * The discriminant that narrows this interface to a concrete variant, per the styleguide's + * discriminated-union-over-independent-classes pattern -- there is deliberately no base class. + */ + readonly kind: + | 'byte-array' + | 'string' + | 'stream' + | 'form-urlencoded' + | 'multipart' + | 'file'; + /** + * The media type to send as `Content-Type`, or `undefined` when the body declares none. + * + * Absence is `undefined`, never `null`, matching the domain model everywhere else. + */ + readonly mediaType: string | undefined; + /** + * The exact byte count `writeTo` will emit, or -1 when it is not known ahead of the write + * (BODY-35). A transport stamps this into `Content-Length`, so it must never disagree with the + * bytes actually written. + */ + readonly contentLength: number; + /** + * Whether writing more than once yields byte-for-byte identical output (BODY-4/BODY-5). + * + * Consulted by Phase 5's retry, redirect, and auth steps before re-sending a request; a + * single-use body must be run through `materialize` first. + */ + readonly replayable: boolean; + /** + * Writes the body once into `sink`, closing it on success and aborting it on failure so a partially + * written body is never signalled to the transport as a complete one. + * + * @param sink - the destination. The body owns closing it -- the caller only supplies it -- and + * aborts it rather than closing it when the write fails, so a truncated payload is never signalled + * downstream as a complete one. + * @throws ConsumedBodyError when a single-use body is written a second time (BODY-3). + * @throws EndOfStreamError when a stream body's byte count disagrees with its declared + * `contentLength` (HTTP-39/BODY-10). + */ + writeTo(sink: WritableStream<Uint8Array>): Promise<void>; +} + +/** + * The structural recognition contract a transport narrows on (`body.kind === 'file'`) to dispatch a + * file-specific send path (TRANSPORT-28). Type-only — `\@dexpace/core` never constructs one; the concrete + * factory lives in `\@dexpace/body-file`, which can depend on `node:fs` precisely because it is not core. + * + * @public + */ +export interface FileBodyDescriptor extends Body { + readonly kind: 'file'; + readonly path: string; + readonly start: number; + readonly count: number; +} diff --git a/packages/core/src/body/errors.test.ts b/packages/core/src/body/errors.test.ts new file mode 100644 index 0000000..d55a2b4 --- /dev/null +++ b/packages/core/src/body/errors.test.ts @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/errors.test.ts +// Exercises: BODY-3 (ConsumedBodyError), HTTP-51 (MultipartBoundaryError) +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import { + ConsumedBodyError, + isBodyError, + MultipartBoundaryError, +} from './errors.js'; + +describe('body errors', () => { + test('ConsumedBodyError descends from DexpaceError and names the body kind', () => { + const error = new ConsumedBodyError('stream'); + expect(error).toBeInstanceOf(DexpaceError); + expect(error.bodyKind).toBe('stream'); + expect(error.message).toContain('stream'); + }); + + test('MultipartBoundaryError descends from DexpaceError and names the offending boundary', () => { + const error = new MultipartBoundaryError('bad boundary'); + expect(error).toBeInstanceOf(DexpaceError); + expect(error.boundary).toBe('bad boundary'); + }); + + test('isBodyError groups both leaves without a class tier', () => { + expect(isBodyError(new ConsumedBodyError('stream'))).toBe(true); + expect(isBodyError(new MultipartBoundaryError('x'))).toBe(true); + expect(isBodyError(new DexpaceError('other'))).toBe(false); + }); +}); diff --git a/packages/core/src/body/errors.ts b/packages/core/src/body/errors.ts new file mode 100644 index 0000000..e1ffbee --- /dev/null +++ b/packages/core/src/body/errors.ts @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * A single-use body's second write (BODY-3). `bodyKind` names which Body variant refused the write. + * + * @example + * ```ts + * try { + * await body.writeTo(sink); + * } catch (error) { + * if (error instanceof ConsumedBodyError) { + * // materialize() first if you need to send this body more than once + * } + * } + * ``` + * @public + */ +export class ConsumedBodyError extends DexpaceError { + /** The `Body.kind` of the variant that refused the write. */ + readonly bodyKind: string; + + constructor(bodyKind: string, options?: ErrorOptions) { + super( + `${bodyKind} body already consumed -- single-use bodies cannot be written twice`, + options, + ); + this.bodyKind = bodyKind; + } +} + +/** + * A caller-supplied multipart boundary violates RFC 2046's grammar (HTTP-51). + * + * @public + */ +export class MultipartBoundaryError extends DexpaceError { + /** The rejected boundary, exactly as supplied. */ + readonly boundary: string; + + constructor(boundary: string, options?: ErrorOptions) { + super(`invalid multipart boundary: ${JSON.stringify(boundary)}`, options); + this.boundary = boundary; + } +} + +/** + * A form field that cannot be rendered into an `x-www-form-urlencoded` body (HTTP-38/BODY-35) -- a + * non-string field name, or a value that is neither a primitive nor `null`. Raised rather than dropping + * the field, which would put a silently incomplete body on the wire. + * + * @public + */ +export class FormBodyValidationError extends DexpaceError { + /** The form field name whose value could not be rendered. */ + readonly field: string; + + constructor(field: string, value: unknown, options?: ErrorOptions) { + super( + `form field ${JSON.stringify(field)} has an unsupported value of type ${typeof value} -- use a string, number, boolean, bigint, or null`, + options, + ); + this.field = field; + } +} + +/** + * A {@link HttpStatusError} construction whose status is not in HTTP-11's 400-599 error band, or is + * not an integer at all. + * + * `XCUT-8` requires the status-to-exception mapping to reject a non-error status "rather than + * fabricate a 'successful exception'". `toHttpError` always satisfied that — it returns `null` for + * anything outside the band — but the published constructor validated nothing, so + * `new HttpStatusError(200, …)` built exactly the object the requirement forbids and contradicted + * the class's own documented invariant. Enforced from 2026-09-02. + * + * A two-level leaf under {@link DexpaceError}, matching its siblings in this file. It never joined + * the `DomainModelError` tier `http/errors.ts` carried at the time; that tier has since been + * flattened onto {@link DexpaceError} as well, and `isDomainModelError` groups what hung off it. + * + * Deliberately NOT part of {@link isBodyError}. That guard groups the three failures a caller meets + * while *working with* a body; this one reports a programmer error at the moment an error object is + * constructed, and widening the guard's return type would change a published signature for a case + * no body-handling `catch` wants to see. + * + * @public + */ +export class HttpStatusValidationError extends DexpaceError { + /** The rejected status value, exactly as supplied. */ + readonly status: number; + + /** + * @param status - the rejected status value. + * @param options - standard error options; pass `{cause}` when wrapping a caught error. + */ + constructor(status: number, options?: ErrorOptions) { + super( + `HttpStatusError status must be an integer in HTTP-11's 400-599 error band, got ${String(status)}`, + options, + ); + this.status = status; + } +} + +/** + * Type guard for body errors. + * + * @public + */ +export function isBodyError( + error: unknown, +): error is + ConsumedBodyError | MultipartBoundaryError | FormBodyValidationError { + return ( + error instanceof ConsumedBodyError || + error instanceof MultipartBoundaryError || + error instanceof FormBodyValidationError + ); +} diff --git a/packages/core/src/body/freeze-body.ts b/packages/core/src/body/freeze-body.ts new file mode 100644 index 0000000..f40507b --- /dev/null +++ b/packages/core/src/body/freeze-body.ts @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/freeze-body.ts +import type {Body} from './body.js'; + +/** + * Freezes a fully-constructed {@link Body}, as the last statement of its constructor. + * + * `readonly` is erased at run time, so without this a caller can reassign a body's own metadata after + * construction and desynchronize it from the bytes `writeTo` emits: + * + * ```ts + * const body = byteArrayBody(Uint8Array.from([1, 2, 3])); + * (body as {contentLength: number}).contentLength = 999; // declared 999, writes 3 + * ``` + * + * That is the same declared-length-versus-written-bytes drift `HTTP-51` makes `MultipartBody` share one + * framing routine to prevent, and that `HTTP-1`/`XCUT-15` make it defensively copy its parts array for -- + * left open one level up, on the field the transport actually stamps into `Content-Length`. + * + * A named helper rather than five inlined `Object.freeze(this)` calls so the reason lives in one place; + * the freeze is shallow and is never relied on to cascade, matching the domain-model convention + * `packages/core/src/http/` already follows. `#private` fields are unaffected, which is why + * `StreamBody`'s consumed-once flag still works on a frozen instance. + */ +export function freezeBody(body: Body): void { + Object.freeze(body); +} diff --git a/packages/core/src/body/http-status-error.test.ts b/packages/core/src/body/http-status-error.test.ts new file mode 100644 index 0000000..46d2975 --- /dev/null +++ b/packages/core/src/body/http-status-error.test.ts @@ -0,0 +1,258 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/http-status-error.test.ts +// Exercises: HTTP-52/BODY-30 (1 MiB cap, replayable re-serve, buffered inside close-guaranteeing scope), +// BODY-31 (4xx/5xx only, no-body response returned unchanged), BODY-33 (non-consuming preview), +// HTTP-42 (preview decodes with the media type's charset, falling back to UTF-8, never throwing), +// XCUT-8 (the status-to-exception mapping factory refuses to fabricate a "successful exception": +// toHttpError returns null for 1xx/2xx/3xx rather than an error, which is the absent/null +// convenience form XCUT-8 explicitly permits in place of a throwing strict mapper. The port ships +// only that form, and since 2026-09-02 the CONSTRUCTOR enforces the 400-599 band too, so the +// guarantee holds at both levels rather than only at the factory). +import {describe, expect, test} from 'bun:test'; +import {Headers} from '../http/headers.js'; +import {HttpStatusError} from './http-status-error.js'; +import {HttpStatusValidationError} from './errors.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {toHttpError} from './http-status-error.js'; + +function readableOf(bytes: Uint8Array): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start: c => { + c.enqueue(bytes); + c.close(); + }, + }); +} + +function responseWith( + status: number, + body: ReadableStream<Uint8Array> | null, + headers: Headers = Headers.newBuilder().build(), +): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(headers) + .body(body) + .build(); +} + +describe('toHttpError (BODY-31)', () => { + test('returns null for a non-error response', async () => { + expect(await toHttpError(responseWith(200, null))).toBeNull(); + expect(await toHttpError(responseWith(304, null))).toBeNull(); + }); + + test('returns an HttpStatusError for 4xx and 5xx', async () => { + expect(await toHttpError(responseWith(404, null))).not.toBeNull(); + expect(await toHttpError(responseWith(500, null))).not.toBeNull(); + }); +}); + +describe("the constructor refuses a status outside HTTP-11's error band (N2, XCUT-8)", () => { + test('rejects a non-error status -- the "successful exception" XCUT-8 forbids', () => { + for (const status of [200, 204, 301, 399]) { + expect(() => new HttpStatusError(status, undefined, undefined)).toThrow( + HttpStatusValidationError, + ); + } + }); + + test('rejects a status outside 100-599 entirely', () => { + expect(() => new HttpStatusError(600, undefined, undefined)).toThrow( + HttpStatusValidationError, + ); + expect(() => new HttpStatusError(0, undefined, undefined)).toThrow( + HttpStatusValidationError, + ); + }); + + test('rejects a non-integer or non-finite status', () => { + expect(() => new HttpStatusError(404.5, undefined, undefined)).toThrow( + HttpStatusValidationError, + ); + expect(() => new HttpStatusError(Number.NaN, undefined, undefined)).toThrow( + HttpStatusValidationError, + ); + expect( + () => new HttpStatusError(Number.POSITIVE_INFINITY, undefined, undefined), + ).toThrow(HttpStatusValidationError); + }); + + test('accepts the whole band, inclusive at both edges', () => { + for (const status of [400, 404, 500, 599]) { + expect(new HttpStatusError(status, undefined, undefined).status).toBe( + status, + ); + } + }); +}); + +describe('toHttpError survives a failing close (H14/P1, RECOV-12)', () => { + /** + * A body whose `cancel()` hook fails INDEPENDENTLY of the read, which is the only shape that makes + * the masking observable: for a plain errored `ReadableStream` the read error and the cancel error + * are the same object, so nothing is masked. Here the stream reads cleanly to completion and only + * teardown fails. + */ + /** Fails the memoized close ONCE, so the drain's own release meets the stored rejection. */ + async function failFirstClose(response: Response): Promise<void> { + let closeError: unknown; + try { + await response.close(); + } catch (error: unknown) { + closeError = error; + } + expect(String(closeError)).toContain('CLOSE FAILED'); + } + + function bodyFailingToCancel(bytes: Uint8Array): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start: c => { + c.enqueue(bytes); + c.close(); + }, + cancel: () => { + throw new Error('CLOSE FAILED'); + }, + }); + } + + test('a 5xx whose close() fails still yields the HttpStatusError the docs promise', async () => { + const response = responseWith( + 500, + bodyFailingToCancel(new TextEncoder().encode('boom')), + ); + await failFirstClose(response); + + const error = await toHttpError(response); + expect(error).toBeInstanceOf(HttpStatusError); + expect(error?.status).toBe(500); + }); + + test("the release failure is not lost: it rides along as the error's cause", async () => { + const response = responseWith( + 500, + bodyFailingToCancel(new TextEncoder().encode('boom')), + ); + await failFirstClose(response); + + const error = await toHttpError(response); + expect(String((error as Error | null)?.cause)).toContain('CLOSE FAILED'); + }); + + test('a bodiless 4xx whose close() fails still yields the HttpStatusError', async () => { + const response = responseWith(404, bodyFailingToCancel(new Uint8Array())); + await failFirstClose(response); + // Drain the (already-cancelled) body path: the body is non-null, so this exercises the drain + // branch; the bodiless branch is covered by the null-body cases above. + const error = await toHttpError(response); + expect(error).toBeInstanceOf(HttpStatusError); + expect(error?.status).toBe(404); + }); +}); + +describe('HttpStatusError (HTTP-52/BODY-30)', () => { + test('carries the status', async () => { + expect((await toHttpError(responseWith(404, null)))?.status).toBe(404); + }); + + test('buffers the body and re-serves it as a replayable, independently readable Body', async () => { + const bytes = new TextEncoder().encode('not found'); + const error = await toHttpError(responseWith(404, readableOf(bytes))); + const body = error?.body(); + expect(body?.replayable).toBe(true); + + const chunks: Uint8Array[] = []; + await body?.writeTo(new WritableStream({write: c => void chunks.push(c)})); + expect(new TextDecoder().decode(chunks[0])).toBe('not found'); + + const chunksAgain: Uint8Array[] = []; + await error + ?.body() + ?.writeTo(new WritableStream({write: c => void chunksAgain.push(c)})); + expect(new TextDecoder().decode(chunksAgain[0])).toBe('not found'); + }); + + test('drops bytes beyond the 1 MiB cap but still drains and closes the connection', async () => { + const big = new Uint8Array(2 * 1024 * 1024).fill(65); + const error = await toHttpError(responseWith(500, readableOf(big))); + expect(error?.body()?.contentLength).toBe(1024 * 1024); + }); + + test('when the response has no body, the error carries an undefined body and null preview (BODY-31)', async () => { + const error = await toHttpError(responseWith(500, null)); + expect(error?.body()).toBeUndefined(); + expect(error?.preview()).toBeNull(); + }); + + test('preview is non-consuming and repeatable (BODY-33)', async () => { + const error = await toHttpError( + responseWith(500, readableOf(new TextEncoder().encode('boom'))), + ); + expect(error?.preview()).toBe('boom'); + expect(error?.preview()).toBe('boom'); + }); +}); + +function contentType(value: string): Headers { + return Headers.newBuilder().add('content-type', value).build(); +} + +describe('preview charset resolution (HTTP-42, BODY-33)', () => { + const cafeLatin1 = Uint8Array.from([0x63, 0x61, 0x66, 0xe9]); // "café" in ISO-8859-1 + + test('decodes with the charset declared by the response media type', async () => { + const error = await toHttpError( + responseWith( + 500, + readableOf(cafeLatin1), + contentType('text/plain; charset=iso-8859-1'), + ), + ); + expect(error?.preview()).toBe('café'); + }); + + test('an explicit charset argument still wins', async () => { + const error = await toHttpError( + responseWith(500, readableOf(cafeLatin1), contentType('text/plain')), + ); + expect(error?.preview('iso-8859-1')).toBe('café'); + }); + + test('an unknown charset falls back to UTF-8 instead of raising a RangeError', async () => { + const error = await toHttpError( + responseWith( + 500, + readableOf(new TextEncoder().encode('ok')), + contentType('text/plain; charset=bogus-charset'), + ), + ); + expect(error?.preview()).toBe('ok'); + expect(error?.preview('also-bogus')).toBe('ok'); + }); + + test('defaults to UTF-8 when no media type was sent', async () => { + const error = await toHttpError( + responseWith(500, readableOf(new TextEncoder().encode('héllo'))), + ); + expect(error?.preview()).toBe('héllo'); + }); + + test('body() drops an inbound media type that is not outbound-safe (HTTP-18/HTTP-19)', async () => { + // HTTP-19 admits obs-text (>= 0x80) inbound; HTTP-18 forbids it outbound. Re-serving a received + // content-type on an outbound Body must drop it, never raise from an accessor on an error object. + const headers = Headers.newBuilder() + .addInbound('content-type', 'text/plain; note="\u00e9"') + .build(); + const error = await toHttpError( + responseWith(500, readableOf(Uint8Array.from([1])), headers), + ); + expect(error?.body()?.mediaType).toBeUndefined(); + expect(error?.preview()).toBe('\u0001'); // still previews, charset resolution falls back + }); +}); diff --git a/packages/core/src/body/http-status-error.ts b/packages/core/src/body/http-status-error.ts new file mode 100644 index 0000000..09c4bab --- /dev/null +++ b/packages/core/src/body/http-status-error.ts @@ -0,0 +1,187 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/http-status-error.ts +import {decodeBodyText, resolveCharset} from '../http/charset.js'; +import {DexpaceError} from '../http/errors.js'; +import {HttpStatusValidationError} from './errors.js'; +import type {Response} from '../http/response.js'; +import {invariant} from '../invariant.js'; +import { + releaseQuietly, + releasedCleanly, + withReleaseFailure, +} from '../recovery/release.js'; +import type {Body} from './body.js'; +import {headerSafeMediaType} from './media-type-safety.js'; +import {byteArrayBody} from './simple-bodies.js'; + +// Fixed by HTTP-52. Deliberately NOT BODY-34's shared preview cap, which is configurable and covers the +// two logging tees only -- a spec-fixed value cannot be the configurable one. +const ERROR_BODY_CAP_BYTES = 1024 * 1024; // 1 MiB, HTTP-52/BODY-30 + +/** + * A 4xx/5xx response turned into an exception (HTTP-52/BODY-30, BODY-31). + * + * @public + */ +export class HttpStatusError extends DexpaceError { + /** + * The response status code, always in HTTP-11's 400-599 error band (BODY-31). + * + * "Always" is enforced by the constructor as of 2026-09-02, not merely asserted here. It was a + * documented-but-unchecked invariant before that, which is what let a consumer build the + * "successful exception" `XCUT-8` forbids. + */ + readonly status: number; + readonly #bodyBytes: Uint8Array | undefined; + readonly #mediaType: string | undefined; + + /** + * @param status - the response status; MUST be an integer in HTTP-11's 400-599 error band. + * @param bodyBytes - the buffered error body, capped at 1 MiB (HTTP-52/BODY-30), or `undefined`. + * @param mediaType - the response's `Content-Type`, used to decode {@link HttpStatusError.preview}. + * @param options - standard error options; pass `{cause}` when wrapping a caught error. + * @throws {@link HttpStatusValidationError} when `status` is not an integer in 400-599. `XCUT-8` + * requires the mapping to reject a non-error status rather than fabricate a "successful + * exception"; `toHttpError` is the total form that returns `null` instead of throwing. + */ + // eslint-disable-next-line max-params -- constructor parameters fixed by error model + constructor( + status: number, + bodyBytes: Uint8Array | undefined, + mediaType: string | undefined, + options?: ErrorOptions, + ) { + super(`HTTP ${String(status)}`, options); + if (!Number.isInteger(status) || status < 400 || status > 599) { + throw new HttpStatusValidationError(status); + } + this.status = status; + this.#bodyBytes = bodyBytes; + this.#mediaType = mediaType; + } + + /** + * The buffered error body, re-served as a replayable Body -- readable independently and repeatably + * after the transport connection was released (BODY-30). Undefined when there was no body. + */ + body(): Body | undefined { + return this.#bodyBytes === undefined + ? undefined + : // Dropped rather than raised when the received content-type is not outbound-safe: an inbound + // value may legally carry obs-text (HTTP-19) that an outbound body may not (HTTP-18). + byteArrayBody(this.#bodyBytes, headerSafeMediaType(this.#mediaType)); + } + + /** + * Non-consuming preview from the buffered copy (BODY-33). Null for no body. + * + * Decodes with `charset` when given, otherwise with the charset declared by the response's media type, + * falling back to UTF-8 when that is absent or unknown -- the same resolution `Response.text()` uses + * (HTTP-42). Never throws: an unrecognized label falls back rather than raising a RangeError out of a + * method on an error object, where a caller is least able to handle another exception. + */ + preview(charset?: string): string | null { + if (this.#bodyBytes === undefined) return null; + return decodeBodyText( + this.#bodyBytes, + charset ?? resolveCharset(this.#mediaType), + ); + } +} + +/** + * Turns a 4xx/5xx response into an HttpStatusError, buffering at most 1 MiB of the body inside the + * response's own close-guaranteeing scope (HTTP-52/BODY-30). Returns null for a non-error response + * (BODY-31) -- the caller keeps the response, body intact. + * + * **A failing release can no longer replace the result** (RECOV-12). The drain used to end its work + * in a bare `finally` block that awaited `response.close()`; `Response.close()` memoizes its + * release promise, so a close that had already failed handed the same rejection back and it + * replaced the `HttpStatusError` this function was about to build -- the error never existed, and + * every caller documenting `@throws HttpStatusError on 4xx/5xx` lied. Release now goes through + * `releaseQuietly`, so: + * + * - a **read** failure stays primary, with the release failure suppressed under it + * (`withReleaseFailure`), exactly as every other subsystem does it; + * - a **successful** read returns the `HttpStatusError` even when the release failed, carrying that + * failure as its `cause` so it is recorded rather than dropped. + * + * @param response - the response to convert; it is released either way (BODY-16). + * @returns the error for a 4xx/5xx, or `null` for any other status. + * @throws Whatever reading the response body raises; the response is released either way (BODY-16). + * If releasing ALSO fails, the read failure stays primary and the release failure rides along + * suppressed. + * @public + */ +export async function toHttpError( + response: Response, +): Promise<HttpStatusError | null> { + // BODY-31: error statuses only, i.e. HTTP-11's 400-599 band. A bare `code < 400` would sweep a + // non-standard 6xx -- which HTTP-10 requires Status.of to accept and return -- into the error path + // and consume a body BODY-31 says must be handed back intact. + if (!response.status.isError) return null; + const mediaType = response.headers.get('content-type'); + if (response.body === null) { + const releaseFailure = await releaseQuietly(response); + return new HttpStatusError( + response.status.code, + undefined, + mediaType, + releaseOptions(releaseFailure), + ); + } + const chunks: Uint8Array[] = []; + let total = 0; + // Acquired INSIDE the try, for the same reason Response.bytes does it: `getReader()` throws when an + // external consumer already holds the lock, and acquiring it above the try skipped the close on + // exactly that path -- holding the connection open (HTTP-52/BODY-30). + let reader: ReadableStreamDefaultReader<Uint8Array> | undefined; + let readFailure: {readonly error: unknown} | undefined; + try { + reader = response.body.getReader(); + for (;;) { + // Serial by necessity: each read depends on the previous one advancing the cursor. + const {done, value} = await reader.read(); + if (done) break; + if (total >= ERROR_BODY_CAP_BYTES) continue; // keep draining to release the connection; drop the bytes + const room = ERROR_BODY_CAP_BYTES - total; + const piece = value.length > room ? value.subarray(0, room) : value; + chunks.push(piece); + total += piece.length; + } + } catch (error: unknown) { + readFailure = {error}; + } + // Release before close(): cancel() rejects with TypeError on a locked stream (see Response.bytes). + reader?.releaseLock(); + const releaseFailure = await releaseQuietly(response); + if (readFailure !== undefined) { + throw withReleaseFailure(readFailure.error, releaseFailure); + } + invariant( + total <= ERROR_BODY_CAP_BYTES, + `buffered ${String(total)} bytes past the ${String(ERROR_BODY_CAP_BYTES)} cap`, + ); + + const bytes = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + bytes.set(chunk, offset); + offset += chunk.length; + } + return new HttpStatusError( + response.status.code, + bytes, + mediaType, + releaseOptions(releaseFailure), + ); +} + +/** + * Turns {@link releaseQuietly}'s opaque token into `ErrorOptions`. A clean release yields + * `undefined`, so the common path constructs exactly what it always did; a failed one rides along as + * `cause`, which is the only slot a RETURNED error has for a secondary failure. + */ +function releaseOptions(releaseToken: unknown): ErrorOptions | undefined { + return releasedCleanly(releaseToken) ? undefined : {cause: releaseToken}; +} diff --git a/packages/core/src/body/index.ts b/packages/core/src/body/index.ts new file mode 100644 index 0000000..8d4f328 --- /dev/null +++ b/packages/core/src/body/index.ts @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/index.ts +// Internal-facing barrel for product-spec §6. Everything except the two logging tees is also promoted to +// packages/core/src/index.ts (Step 2) -- this file is the superset a future in-tree consumer (e.g. Phase +// 7's pipeline) imports from directly. +export type {Body, FileBodyDescriptor} from './body.js'; +export { + ConsumedBodyError, + FormBodyValidationError, + isBodyError, + MultipartBoundaryError, +} from './errors.js'; +export {HttpStatusError, toHttpError} from './http-status-error.js'; +export {materialize} from './materialize.js'; +export { + multipartBody, + MultipartBody, + MultipartBodyBuilder, + type MultipartPart, +} from './multipart-body.js'; +export {withRequestLogging, type LoggedBody} from './request-body-logging.js'; +export { + withResponseLogging, + type LoggedResponseBody, +} from './response-body-logging.js'; +export { + byteArrayBody, + ByteArrayBody, + formUrlEncodedBody, + FormUrlEncodedBody, + type FormUrlEncodedInput, + type FormUrlEncodedValue, + stringBody, + StringBody, +} from './simple-bodies.js'; +export {streamBody, StreamBody} from './stream-body.js'; +export {TypedResponse} from './typed-response.js'; diff --git a/packages/core/src/body/materialize.test.ts b/packages/core/src/body/materialize.test.ts new file mode 100644 index 0000000..2582d8c --- /dev/null +++ b/packages/core/src/body/materialize.test.ts @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/materialize.test.ts +// Exercises: BODY-3/HTTP-37 (materialize-once) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {ConsumedBodyError} from './errors.js'; +import {byteArrayBody} from './simple-bodies.js'; +import {materialize} from './materialize.js'; +import {streamBody} from './stream-body.js'; + +function readableOf(...bytes: number[]): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from(bytes)); + controller.close(); + }, + }); +} + +async function drainBody(body: { + writeTo: (sink: WritableStream<Uint8Array>) => Promise<void>; +}): Promise<Uint8Array> { + const chunks: Uint8Array[] = []; + await body.writeTo(new WritableStream({write: c => void chunks.push(c)})); + const total = chunks.reduce((s, c) => s + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; +} + +describe('materialize', () => { + test('returns an already-replayable body unchanged', async () => { + const body = byteArrayBody(Uint8Array.from([1, 2])); + expect(await materialize(body)).toBe(body); + }); + + test('drains a single-use body into a fresh replayable ByteArrayBody', async () => { + const materialized = await materialize(streamBody(readableOf(1, 2, 3))); + expect(materialized.replayable).toBe(true); + expect(materialized.kind).toBe('byte-array'); + expect([...(await drainBody(materialized))]).toEqual([1, 2, 3]); + }); + + test('the materialized body is writable more than once, byte-for-byte identical', async () => { + const materialized = await materialize(streamBody(readableOf(9, 8))); + expect([...(await drainBody(materialized))]).toEqual([9, 8]); + expect([...(await drainBody(materialized))]).toEqual([9, 8]); + }); + + test('preserves the original mediaType', async () => { + const materialized = await materialize( + streamBody(readableOf(1), 'text/plain'), + ); + expect(materialized.mediaType).toBe('text/plain'); + }); + + test('under N concurrent callers exactly one drains; every other observes ConsumedBodyError (BODY-3)', async () => { + await fc.assert( + fc.asyncProperty(fc.integer({min: 2, max: 8}), async callers => { + const body = streamBody(readableOf(1, 2, 3)); + const results = await Promise.allSettled( + Array.from({length: callers}, () => materialize(body)), + ); + + const fulfilled = results.filter(r => r.status === 'fulfilled'); + expect(fulfilled.length).toBe(1); + for (const result of results.filter(r => r.status === 'rejected')) { + expect(result.reason).toBeInstanceOf(ConsumedBodyError); + } + }), + {seed: 0x3b}, + ); + }); +}); diff --git a/packages/core/src/body/materialize.ts b/packages/core/src/body/materialize.ts new file mode 100644 index 0000000..f001d0c --- /dev/null +++ b/packages/core/src/body/materialize.ts @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/materialize.ts +import {invariant} from '../invariant.js'; +import type {Body} from './body.js'; +import {byteArrayBody} from './simple-bodies.js'; + +/** + * Returns `body` unchanged if already replayable; otherwise drains its single write into a fresh + * replayable ByteArrayBody, after which the original is treated as consumed (BODY-3/HTTP-37). + * + * @throws ConsumedBodyError when `body` is single-use and has already been written (BODY-3). + * @throws Whatever the delegate's `writeTo` raises -- an EndOfStreamError from a stream body whose + * byte count disagrees with its declared length, for instance (HTTP-39/BODY-10). + * @public + */ +export async function materialize(body: Body): Promise<Body> { + if (body.replayable) return body; + const chunks: Uint8Array[] = []; + let total = 0; + const collector = new WritableStream<Uint8Array>({ + write: chunk => { + chunks.push(chunk); + total += chunk.length; + }, + }); + await body.writeTo(collector); + + const bytes = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + bytes.set(chunk, offset); + offset += chunk.length; + } + invariant( + offset === total, + `materialized ${String(offset)} bytes, expected ${String(total)}`, + ); + + const replayed = byteArrayBody(bytes, body.mediaType); + invariant(replayed.replayable, 'materialize must return a replayable body'); // BODY-3's postcondition + return replayed; +} diff --git a/packages/core/src/body/media-type-safety.ts b/packages/core/src/body/media-type-safety.ts new file mode 100644 index 0000000..b7015ce --- /dev/null +++ b/packages/core/src/body/media-type-safety.ts @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/media-type-safety.ts +import {hasForbiddenOutboundValueByte} from '../http/ascii-validation.js'; +import {MediaTypeParseError} from '../http/errors.js'; + +/** + * Rejects a media type that is not header-safe, using the same predicate as outbound header-value + * validation (HTTP-26). + * + * `Body.mediaType` is interpolated into a multipart part header verbatim (HTTP-51), so a CR/LF inside it + * is a header-injection primitive: it can append arbitrary headers, close the header block outright, and + * forge a closing boundary, all while the shared framing routine keeps the declared content length + * consistent with the corrupted bytes. Validating at construction closes it at the source -- a media type + * containing a control character is never legitimate. + */ +export function assertHeaderSafeMediaType(mediaType: string | undefined): void { + if (mediaType === undefined) return; + if (hasForbiddenOutboundValueByte(mediaType)) { + throw new MediaTypeParseError( + `media type must not contain a control character or non-ASCII byte: ${JSON.stringify(mediaType)}`, + ); + } +} + +/** + * Returns `mediaType` when it is header-safe, otherwise undefined. + * + * For media types that arrive from the wire rather than from a caller. HTTP-19 deliberately lets an + * inbound header value carry obs-text (>= 0x80) that HTTP-18 forbids outbound, so re-serving a received + * `content-type` on an outbound body can legitimately fail {@link assertHeaderSafeMediaType}. Dropping + * the media type is the right trade there -- raising from an accessor on an error object is not. + */ +export function headerSafeMediaType( + mediaType: string | undefined, +): string | undefined { + if (mediaType === undefined) return undefined; + return hasForbiddenOutboundValueByte(mediaType) ? undefined : mediaType; +} diff --git a/packages/core/src/body/multipart-body.test.ts b/packages/core/src/body/multipart-body.test.ts new file mode 100644 index 0000000..d616008 --- /dev/null +++ b/packages/core/src/body/multipart-body.test.ts @@ -0,0 +1,401 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/multipart-body.test.ts +// Exercises: BODY-2 (composite replayability, unknown-length collapse), HTTP-51 (shared framing routine, +// boundary generation/validation, header quoting, a boundary parameter rendered so an RFC 9110 parser +// can read it, and a part media type that cannot break the framing), HTTP-26 (a media type is +// header-safe), RECOV-12 (a close failure never masks the primary failure) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {MediaTypeParseError} from '../http/errors.js'; +import type {Body} from './body.js'; +import {MultipartBoundaryError} from './errors.js'; +import { + MultipartBody, + MultipartBodyBuilder, + multipartBody, +} from './multipart-body.js'; +import {byteArrayBody, stringBody} from './simple-bodies.js'; +import {streamBody} from './stream-body.js'; + +function emptyStream(): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }); +} + +function oneByteStream(): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(c) { + c.enqueue(Uint8Array.from([1])); + c.close(); + }, + }); +} + +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves. */ +async function rejection(promise: Promise<unknown>): Promise<Error> { + try { + await promise; + } catch (error: unknown) { + return error as Error; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +function collectingSink(): { + sink: WritableStream<Uint8Array>; + written: () => Uint8Array; +} { + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write: c => void chunks.push(c), + }); + return { + sink, + written: () => { + const total = chunks.reduce((sum, c) => sum + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; + }, + }; +} + +// `Uint8Array<ArrayBuffer>`, not the bare alias: `BodyInit` excludes a view over a `SharedArrayBuffer`, +// so the default `ArrayBufferLike` parameter is not assignable to the platform `Response` below. +async function drainBytes(body: { + writeTo: (sink: WritableStream<Uint8Array>) => Promise<void>; +}): Promise<Uint8Array<ArrayBuffer>> { + const chunks: Uint8Array[] = []; + await body.writeTo(new WritableStream({write: c => void chunks.push(c)})); + const total = chunks.reduce((s, c) => s + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; +} + +async function drain(body: { + writeTo: (sink: WritableStream<Uint8Array>) => Promise<void>; +}): Promise<string> { + return new TextDecoder().decode(await drainBytes(body)); +} + +describe('MultipartBody replayability and length (BODY-2)', () => { + test('replayable when every part is replayable', () => { + expect(multipartBody([{name: 'a', body: stringBody('x')}]).replayable).toBe( + true, + ); + }); + + test('not replayable when any part is not', () => { + const body = multipartBody([ + {name: 'a', body: stringBody('x')}, + {name: 'b', body: streamBody(oneByteStream())}, + ]); + expect(body.replayable).toBe(false); + }); + + test('declared length collapses to -1 if any part length is unknown (BODY-2)', () => { + expect( + multipartBody([{name: 'a', body: streamBody(emptyStream())}]) + .contentLength, + ).toBe(-1); + }); + + test('declared length equals the bytes actually written when every part length is known', async () => { + const body = multipartBody( + [{name: 'a', body: stringBody('hello')}], + 'FIXEDBOUNDARY', + ); + const rendered = await drain(body); + expect(new TextEncoder().encode(rendered).length).toBe(body.contentLength); + }); +}); + +describe('MultipartBody framing and headers (HTTP-51)', () => { + test('frames one part with boundary, headers, body, and a CRLF-terminated trailer', async () => { + const rendered = await drain( + multipartBody( + [ + { + name: 'field', + body: byteArrayBody(new TextEncoder().encode('value')), + }, + ], + 'B', + ), + ); + expect(rendered).toBe( + '--B\r\nContent-Disposition: form-data; name="field"\r\n\r\nvalue\r\n--B--\r\n', + ); + }); + + test('includes filename and Content-Type when the part has them', async () => { + const rendered = await drain( + multipartBody( + [ + { + name: 'file', + filename: 'a.txt', + body: byteArrayBody(Uint8Array.from([1]), 'text/plain'), + }, + ], + 'B', + ), + ); + expect(rendered).toContain('filename="a.txt"'); + expect(rendered).toContain('Content-Type: text/plain\r\n'); + }); + + test('quotes/escapes a quote or backslash in a part name, and strips embedded CR/LF (HTTP-51)', async () => { + const rendered = await drain( + multipartBody([{name: 'a"b\\c\r\nd', body: stringBody('x')}], 'B'), + ); + expect(rendered).toContain('name="a\\"b\\\\cd"'); + }); +}); + +describe('MultipartBody boundary generation and validation (HTTP-51)', () => { + test('a valid caller-supplied boundary is accepted', () => { + expect(() => + multipartBody([{name: 'a', body: stringBody('x')}], 'valid-boundary_1'), + ).not.toThrow(); + }); + + test('an invalid caller-supplied boundary throws MultipartBoundaryError', () => { + expect(() => + multipartBody([{name: 'a', body: stringBody('x')}], 'trailing space '), + ).toThrow(MultipartBoundaryError); + expect(() => + multipartBody([{name: 'a', body: stringBody('x')}], ''), + ).toThrow(MultipartBoundaryError); + }); + + test('an unsupplied boundary is generated and spec-valid', () => { + const body = multipartBody([{name: 'a', body: stringBody('x')}]); + expect(body.mediaType).toMatch( + /^multipart\/form-data; boundary=dexpace-[A-Za-z0-9]{32}$/, + ); + }); + + test('two generated boundaries differ', () => { + const a = multipartBody([{name: 'a', body: stringBody('x')}]); + const b = multipartBody([{name: 'a', body: stringBody('x')}]); + expect(a.mediaType).not.toBe(b.mediaType); + }); +}); + +describe('the rendered boundary parameter is a parseable one (HTTP-51)', () => { + // RFC 2046 `bchars` and RFC 9110 `tchar` are different sets: ' ', ',', ':', '=', '?', '/', '(' and + // ')' are legal in a boundary and illegal bare in a header parameter value. `validateBoundary` admits + // the first grammar and the renderer owes the second, so a boundary the constructor accepts must come + // back out quoted rather than bare. + test('a boundary that is not a bare token is quoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'a,b').mediaType, + ).toBe('multipart/form-data; boundary="a,b"'); + }); + + test('a boundary that IS a bare token is left unquoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1').mediaType, + ).toBe('multipart/form-data; boundary=plain-1'); + // The generated default stays byte-identical: it is drawn from ALPHA/DIGIT only. + expect( + multipartBody([{name: 'a', body: stringBody('x')}]).mediaType, + ).toMatch(/^multipart\/form-data; boundary=dexpace-[A-Za-z0-9]{32}$/); + }); + + test.each([['a,b'], ['bound ary'], ['a:b'], ['a=b'], ['a?b'], ['(a)/b']])( + 'the header a peer receives round-trips through a real parameter parser: %p', + async boundary => { + // The runtime's own multipart parser, standing in for the peer. Bun's happens to tolerate the + // unquoted form, so this is the regression guard and NOT the reproducer: Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`, which is why the + // same case is also in `tests/node-conformance/body-lifecycle.test.mjs`. Two independent + // parsers disagreeing about our own Content-Type is exactly what that tree is for. + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const response = new globalThis.Response(await drainBytes(body), { + headers: {'content-type': body.mediaType}, + }); + expect((await response.formData()).get('field')).toBe('value'); + }, + ); +}); + +describe('MultipartBodyBuilder (HTTP-2, HTTP-3)', () => { + test('static newBuilder and instance newBuilder pre-populates parts and boundary', async () => { + const original = MultipartBody.newBuilder() + .addPart({name: 'p1', body: stringBody('v1')}) + .boundary('CUSTOMB') + .build(); + + expect(original.contentLength).toBeGreaterThan(0); + + const derived = original + .newBuilder() + .addPart({name: 'p2', body: stringBody('v2')}) + .build(); + expect(derived.mediaType).toBe('multipart/form-data; boundary=CUSTOMB'); + const rendered = await drain(derived); + expect(rendered).toContain('name="p1"'); + expect(rendered).toContain('name="p2"'); + }); + + test('MultipartBodyBuilder.parts sets the parts list', async () => { + const builder = new MultipartBodyBuilder(); + builder.parts([{name: 'a', body: stringBody('1')}]); + const body = builder.build(); + expect(await drain(body)).toContain('name="a"'); + }); +}); + +describe('MultipartBody property tests (HTTP-51)', () => { + test('declared length always equals the bytes written, for any part set (HTTP-51)', async () => { + await fc.assert( + fc.asyncProperty( + fc.array(fc.record({name: fc.string(), content: fc.string()}), { + minLength: 1, + maxLength: 8, + }), + async specs => { + const body = multipartBody( + specs.map(s => ({name: s.name, body: stringBody(s.content)})), + ); + const written = new TextEncoder().encode(await drain(body)).length; + expect(written).toBe(body.contentLength); + }, + ), + {seed: 0x3b}, + ); + }); + + test('a part name containing CR/LF or a quote never breaks the framing (HTTP-51)', async () => { + await fc.assert( + fc.asyncProperty(fc.string(), async name => { + const rendered = await drain( + multipartBody( + [{name, body: byteArrayBody(new TextEncoder().encode('x'))}], + 'B', + ), + ); + const headerBlock = rendered.slice(0, rendered.indexOf('\r\n\r\n')); + // exactly two CRLFs of framing (boundary line, disposition line) -- no injected extras + expect(headerBlock.split('\r\n').length).toBe(2); + }), + {seed: 0x3b}, + ); + }); +}); + +// A hand-rolled Body bypassing the bundled factories' construction-time validation -- MultipartPart +// accepts any Body, so the framing routine cannot assume the media type was already checked. +function forgedBody(mediaType: string): Body { + return { + kind: 'byte-array', + mediaType, + contentLength: 1, + replayable: true, + writeTo: async sink => { + const writer = sink.getWriter(); + await writer.write(Uint8Array.from([120])); + await writer.close(); + }, + }; +} + +describe('a part media type cannot break the framing (HTTP-51)', () => { + test('a media type carrying CR/LF is refused, not interpolated', () => { + const part = { + name: 'f', + body: forgedBody('text/plain\r\nX-Injected: pwned'), + }; + expect(() => multipartBody([part], 'BOUNDARY')).toThrow( + MediaTypeParseError, + ); + }); + + test('a media type that would forge a closing boundary is refused', () => { + const part = { + name: 'f', + body: forgedBody('text/plain\r\n\r\nSMUGGLED\r\n--BOUNDARY--'), + }; + // Without this the declared contentLength still matches the written bytes -- the shared framing + // routine counts the forged bytes too, so the wire is consistently, silently wrong. + expect(() => multipartBody([part], 'BOUNDARY')).toThrow( + MediaTypeParseError, + ); + }); +}); + +describe('MultipartBody failure propagation (RECOV-12)', () => { + test('surfaces the sink failure, not a close TypeError', () => { + const body = multipartBody([{name: 'a', body: stringBody('x')}], 'B'); + const sink = new WritableStream<Uint8Array>({ + write: () => { + throw new Error('SOCKET GONE'); + }, + }); + expect(body.writeTo(sink)).rejects.toThrow('SOCKET GONE'); + }); +}); + +describe('the declared length is verified against what is written (HTTP-51)', () => { + // MultipartPart.body is the public `Body` interface, so a caller-supplied implementation can + // report one length and write another. The shared framing routine keeps the FRAMING consistent + // but takes each part's own contentLength on trust, which desynchronizes the value a transport + // stamps into Content-Length from what is actually on the socket. + function lyingBody(declared: number, actual: number): Body { + return { + kind: 'byte-array', + mediaType: undefined, + contentLength: declared, + replayable: true, + writeTo: async (sink: WritableStream<Uint8Array>): Promise<void> => { + const writer = sink.getWriter(); + await writer.write(new Uint8Array(actual).fill(65)); + await writer.close(); + }, + }; + } + + test('a part that overruns its declared length is stopped before the extra bytes reach the sink', async () => { + const {sink, written} = collectingSink(); + const body = multipartBody([{name: 'a', body: lyingBody(1, 5)}], 'B'); + const declared = body.contentLength; + + expect((await rejection(body.writeTo(sink))).name).toBe('EndOfStreamError'); + // Same reasoning as StreamBody's overrun check: once the length is stamped, a byte past it sits + // where the peer reads it as the start of the next message. + expect(written().length).toBeLessThanOrEqual(declared); + }); + + test('a part that writes fewer bytes than it declared fails rather than sending a short body', async () => { + const {sink} = collectingSink(); + const body = multipartBody([{name: 'a', body: lyingBody(5, 1)}], 'B'); + expect((await rejection(body.writeTo(sink))).name).toBe('EndOfStreamError'); + }); + + test('an unknown-length composite is not length-checked at all', async () => { + // contentLength collapses to -1, so there is no declared value to disagree with. + const body = multipartBody( + [{name: 'a', body: streamBody(oneByteStream())}], + 'B', + ); + expect(body.contentLength).toBe(-1); + await body.writeTo(collectingSink().sink); + }); +}); diff --git a/packages/core/src/body/multipart-body.ts b/packages/core/src/body/multipart-body.ts new file mode 100644 index 0000000..e1affd3 --- /dev/null +++ b/packages/core/src/body/multipart-body.ts @@ -0,0 +1,332 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/multipart-body.ts +import type {Builder} from '../http/builder.js'; +import {MediaType} from '../http/media-type.js'; +import {EndOfStreamError} from '../io/errors.js'; +import {invariant} from '../invariant.js'; +import type {Body} from './body.js'; +import {MultipartBoundaryError} from './errors.js'; +import {freezeBody} from './freeze-body.js'; +import {assertHeaderSafeMediaType} from './media-type-safety.js'; +import {withBodyWriter} from './write-body.js'; + +/** + * A part inside a {@link MultipartBody}. + * + * @public + */ +export interface MultipartPart { + /** The form field name, rendered into `Content-Disposition` and quoted/escaped (HTTP-51). */ + readonly name: string; + /** An optional upload filename, quoted/escaped the same way as {@link MultipartPart.name}. */ + readonly filename?: string | undefined; + /** The part's payload. Its `mediaType` becomes the part's `Content-Type` when present. */ + readonly body: Body; +} + +const BOUNDARY_CHARS = + 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'; +// RFC 2046 bchars grammar: 1-70 chars, last char not a space. +const BOUNDARY_PATTERN = + /^[A-Za-z0-9'()+_,\-./:=? ]{1,69}[A-Za-z0-9'()+_,\-./:=?]$/; +const SINGLE_CHAR_BOUNDARY_PATTERN = /^[A-Za-z0-9'()+_,\-./:=?]$/; +const CRLF = new TextEncoder().encode('\r\n'); + +function generateBoundary(): string { + const bytes = new Uint8Array(32); + crypto.getRandomValues(bytes); + let boundary = 'dexpace-'; + for (const byte of bytes) { + const char = BOUNDARY_CHARS[byte % BOUNDARY_CHARS.length]; + invariant(char !== undefined, 'boundary character must be defined'); + boundary += char; + } + return boundary; +} + +function validateBoundary(boundary: string): void { + const valid = + boundary.length === 1 + ? SINGLE_CHAR_BOUNDARY_PATTERN.test(boundary) + : BOUNDARY_PATTERN.test(boundary); + if (!valid) throw new MultipartBoundaryError(boundary); +} + +// Escapes a quote/backslash so it cannot break the quoted-string grammar, and strips CR/LF outright so +// they can never break the header framing (HTTP-51). +function quoteParam(value: string): string { + return value.replace(/[\\"]/g, ch => `\\${ch}`).replace(/[\r\n]/g, ''); +} + +// The shared framing routine HTTP-51 requires: both computeContentLength and writeTo call this for every +// part, so the declared length and the written bytes cannot drift. +function renderPartHeader(part: MultipartPart, boundary: string): Uint8Array { + let header = `--${boundary}\r\n`; + header += `Content-Disposition: form-data; name="${quoteParam(part.name)}"`; + if (part.filename !== undefined) + header += `; filename="${quoteParam(part.filename)}"`; + header += '\r\n'; + if (part.body.mediaType !== undefined) { + // Defence in depth: the bundled Body implementations validate at construction, but `MultipartPart` + // accepts any `Body`, and this value is interpolated raw. A CR/LF here would append arbitrary + // headers, close the header block, or forge a closing boundary -- and because this routine is shared + // with computeContentLength, the declared length would agree with the corrupted bytes (HTTP-51). + assertHeaderSafeMediaType(part.body.mediaType); + header += `Content-Type: ${part.body.mediaType}\r\n`; + } + header += '\r\n'; + return new TextEncoder().encode(header); +} + +/** + * HTTP-51: the `Content-Type` a peer actually parses. + * + * RFC 2046 `bchars` and RFC 9110 `tchar` are different sets. `BOUNDARY_PATTERN` above admits ' ', ',', + * ':', '=', '?', '/', '(' and ')', none of which is a `tchar`, so interpolating the boundary bare + * produces a parameter value that stops at the first offending byte -- `boundary=a,b` reads as + * `boundary=a` plus a junk parameter, and the peer then never finds a delimiter. Node's own FormData + * parser rejects such a body outright with `TypeError: Failed to parse body as FormData`. + * + * Rendered through {@link MediaType} rather than a second quoting routine here: it is the module that + * owns HTTP-25's token-or-quoted-string decision, and `parse(render(x)) === x` is its guarantee. A + * boundary that IS a bare token still renders bare, so the generated default is byte-identical to what + * this class emitted before. + * + * Narrowing `validateBoundary` to `tchar` instead was rejected: HTTP-51 asks that a boundary VIOLATING + * the RFC 2046 grammar be refused, not that a conforming one be. The defect is in the rendering. + */ +function renderMediaType(boundary: string): string { + return MediaType.of( + 'multipart', + 'form-data', + new Map([['boundary', boundary]]), + ).render(); +} + +function trailerBytes(boundary: string): Uint8Array { + return new TextEncoder().encode(`--${boundary}--\r\n`); +} + +function computeContentLength( + parts: readonly MultipartPart[], + boundary: string, +): number { + let total = 0; + for (const part of parts) { + if (part.body.contentLength === -1) return -1; // BODY-2: any unknown part collapses the whole + total += + renderPartHeader(part, boundary).length + + part.body.contentLength + + CRLF.length; + } + return total + trailerBytes(boundary).length; +} + +/** + * Counts what reaches the sink and refuses a chunk that would carry the message past `declared` + * (HTTP-51). + * + * The shared framing routine guarantees the declared length and the emitted bytes agree about the + * FRAMING, but it takes each part's own `contentLength` on trust -- and `MultipartPart.body` is the + * public `Body` interface, so a caller-supplied implementation can report one length and write + * another. That desynchronizes the value a transport stamps into `Content-Length` from what is + * actually on the socket, which is the precise drift HTTP-51 exists to prevent. + * + * Refused BEFORE the write, not tallied after the loop, for the same reason `StreamBody.#writeExactly` + * checks early: once the length is stamped, an overrun byte sits where the peer reads it as the start + * of the next message, and a thrown error cannot recall bytes already written. + */ +function boundedWriter( + writer: WritableStreamDefaultWriter<Uint8Array>, + declared: number, +): {write: (chunk: Uint8Array) => Promise<void>; written: () => number} { + let written = 0; + return { + write: async (chunk: Uint8Array): Promise<void> => { + if (declared !== -1 && written + chunk.length > declared) { + throw new EndOfStreamError(written + chunk.length, declared); + } + written += chunk.length; + await writer.write(chunk); + }, + written: () => written, + }; +} + +// Wraps the bounded write as a WritableStream whose close() does not close the real sink -- multiple +// parts share one underlying writer, and only the outer writeTo's own scope closes it. +function nonClosingSink( + write: (chunk: Uint8Array) => Promise<void>, +): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({write}); +} + +/** + * A composite body (BODY-2, HTTP-51). Replayable iff every part is; declared length collapses to unknown + * if any part's length is unknown. + * + * @public + */ +export class MultipartBody implements Body { + /** Discriminates this variant within the {@link Body} union. */ + readonly kind = 'multipart' as const; + /** + * `multipart/form-data` carrying the boundary this instance frames its parts with, with the + * `boundary` parameter quoted whenever it is not a bare RFC 9110 token (HTTP-51). + */ + readonly mediaType: string; + /** The total framed byte count, or -1 when any part's own length is unknown (BODY-2). */ + readonly contentLength: number; + /** `true` only when every part is replayable -- composite replayability (BODY-2). */ + readonly replayable: boolean; + readonly #parts: readonly MultipartPart[]; + readonly #boundary: string; + + constructor(parts: readonly MultipartPart[], boundary?: string) { + if (boundary !== undefined) validateBoundary(boundary); + this.#boundary = boundary ?? generateBoundary(); + this.#parts = [...parts]; + this.mediaType = renderMediaType(this.#boundary); + this.replayable = this.#parts.every(part => part.body.replayable); + this.contentLength = computeContentLength(this.#parts, this.#boundary); + invariant( + this.contentLength === -1 || + this.contentLength >= trailerBytes(this.#boundary).length, + `framing computed an impossible length ${String(this.contentLength)}`, + ); + freezeBody(this); // HTTP-1: see freeze-body.ts + } + + /** + * Starts an empty builder (HTTP-3). + * + * @returns a fresh {@link MultipartBodyBuilder}. + */ + static newBuilder(): MultipartBodyBuilder { + return new MultipartBodyBuilder(); + } + + /** + * Derives a builder pre-populated with this instance's parts and boundary, aliasing neither (HTTP-3). + * + * @returns a {@link MultipartBodyBuilder} holding a copy of this body's state. + */ + newBuilder(): MultipartBodyBuilder { + return new MultipartBodyBuilder() + .parts(this.#parts) + .boundary(this.#boundary); + } + + /** + * Writes every part framed by this body's boundary, then the closing trailer, then closes `sink` + * (BODY-2). + * + * Two mechanisms keep {@link MultipartBody.contentLength} and these bytes from drifting (HTTP-51), + * and both are needed. The shared framing routine that computed the length also produces the + * framing here, which covers the delimiters and part headers; and the write is bounded and totalled + * against the declared length, which covers what the routine cannot -- each part's own reported + * `contentLength`, taken on trust from an interface any caller can implement. + * + * @param sink - the destination; this body's to close, the caller's only to supply. Each part + * receives a non-closing adapter over the same writer, so no part can end the message early. + * @throws EndOfStreamError when the bytes actually written disagree with + * {@link MultipartBody.contentLength} — which happens when a caller-supplied part `Body` reports one + * length and writes another (HTTP-51). + * @throws {@link ConsumedBodyError} when a single-use part is written a second time (BODY-3) -- a + * non-replayable composite needs no guard of its own; the offending part's own guard fires. + */ + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + await withBodyWriter(sink, async writer => { + const bounded = boundedWriter(writer, this.contentLength); + for (const part of this.#parts) { + await bounded.write(renderPartHeader(part, this.#boundary)); + await part.body.writeTo(nonClosingSink(bounded.write)); + await bounded.write(CRLF); + } + await bounded.write(trailerBytes(this.#boundary)); + // HTTP-51: a part that writes FEWER bytes than it declared is the mirror of the overrun the + // bounded writer refuses, and just as wrong on the wire. Raised inside the writer scope so + // withBodyWriter aborts rather than signalling a clean close over a short body. + if ( + this.contentLength !== -1 && + bounded.written() !== this.contentLength + ) { + throw new EndOfStreamError(bounded.written(), this.contentLength); + } + }); + } +} + +/** + * Creates a MultipartBody (BODY-2, HTTP-51). + * + * @throws MultipartBoundaryError when `boundary` violates RFC 2046's bchars grammar (HTTP-51). + * @throws MediaTypeParseError when a part's media type contains a control character or non-ASCII byte, + * which would let it break out of the part header it is rendered into (HTTP-26/HTTP-51). + * @public + */ +export function multipartBody( + parts: readonly MultipartPart[], + boundary?: string, +): MultipartBody { + return new MultipartBody(parts, boundary); +} + +/** + * Builder for {@link MultipartBody}. + * + * @public + */ +export class MultipartBodyBuilder implements Builder<MultipartBody> { + #parts: MultipartPart[] = []; + #boundary: string | undefined; + + /** + * Replaces the whole parts list, copying it so the builder never aliases the caller's array. + * + * @param parts - the parts, in the order they will be framed. + * @returns this builder, for chaining. + */ + parts(parts: readonly MultipartPart[]): this { + this.#parts = [...parts]; + return this; + } + + /** + * Appends one part, keeping whatever was added before. + * + * @param part - the part to append. + * @returns this builder, for chaining. + */ + addPart(part: MultipartPart): this { + this.#parts.push(part); + return this; + } + + /** + * Sets the boundary, or clears it so `build()` generates a fresh random one. + * + * Prefer the generated default; see {@link multipartBody} for the RFC 2046 non-appearance + * obligation a caller-supplied delimiter carries and that this class cannot check. + * + * @param boundary - an RFC 2046 `bchars` delimiter that appears in no part, or `undefined`. + * @returns this builder, for chaining. + */ + boundary(boundary: string | undefined): this { + this.#boundary = boundary; + return this; + } + + /** + * Frames the accumulated parts into an immutable {@link MultipartBody}. + * + * @returns the frozen body. + * @throws {@link MultipartBoundaryError} when the configured boundary violates RFC 2046's bchars + * grammar (HTTP-51). + * @throws MediaTypeParseError when a part's media type contains a control character or non-ASCII byte + * (HTTP-26/HTTP-51). + */ + build(): MultipartBody { + return new MultipartBody(this.#parts, this.#boundary); + } +} diff --git a/packages/core/src/body/request-body-logging.test.ts b/packages/core/src/body/request-body-logging.test.ts new file mode 100644 index 0000000..6380af3 --- /dev/null +++ b/packages/core/src/body/request-body-logging.test.ts @@ -0,0 +1,281 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/request-body-logging.test.ts +// Exercises: BODY-17 (mirror + forward the full untruncated payload), BODY-18 (tap clears at the start +// of every write), BODY-19 (tap cap, full payload unaffected), BODY-20 (partial-failure snapshot), BODY-21 +// (replayable/materialize pass through, preserving the tap CAP without sharing its buffer), BODY-37 (no +// backing-buffer escape hatch), plus the decorator's own sink ownership: an abort must reach the +// primary sink rather than stopping at the adapter (RECOV-12) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import {withRequestLogging} from './request-body-logging.js'; +import {rejection} from '../io/test-support/rejection.js'; +import type {Body} from './body.js'; +import {ConsumedBodyError} from './errors.js'; +import {byteArrayBody} from './simple-bodies.js'; +import {streamBody} from './stream-body.js'; + +/** Sentinel distinguishing "abort() never ran" from "abort(undefined)". */ +const NOT_ABORTED = Symbol('not-aborted'); + +/** A healthy sink that records which teardown path the body took. */ +function observableSink(): { + sink: WritableStream<Uint8Array>; + aborted: () => unknown; + closed: () => boolean; +} { + let aborted: unknown = NOT_ABORTED; + let closed = false; + const sink = new WritableStream<Uint8Array>({ + write: () => undefined, + close: () => void (closed = true), + abort: reason => void (aborted = reason), + }); + return {sink, aborted: () => aborted, closed: () => closed}; +} + +function collectingSink(): { + sink: WritableStream<Uint8Array>; + written: () => Uint8Array; +} { + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write: c => void chunks.push(c), + }); + return { + sink, + written: () => { + const total = chunks.reduce((s, c) => s + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; + }, + }; +} + +describe('withRequestLogging mirroring and caps (BODY-17..20)', () => { + test('forwards the full payload untruncated regardless of the tap cap (BODY-17, BODY-19)', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2, 3, 4, 5])), + 2, + ); + const {sink, written} = collectingSink(); + await logged.writeTo(sink); + expect([...written()]).toEqual([1, 2, 3, 4, 5]); + expect([...logged.snapshot()]).toEqual([1, 2]); + }); + + test('the tap clears at the start of every write (BODY-18)', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([9, 9])), + 10, + ); + await logged.writeTo(collectingSink().sink); + await logged.writeTo(collectingSink().sink); + expect([...logged.snapshot()]).toEqual([9, 9]); // not [9, 9, 9, 9] + }); + + test('a tap cap of 0 mirrors nothing while still forwarding everything', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2])), + 0, + ); + const {sink, written} = collectingSink(); + await logged.writeTo(sink); + expect([...written()]).toEqual([1, 2]); + expect(logged.snapshot().length).toBe(0); + }); + + test('a partial write failure still leaves the bytes mirrored up to that point (BODY-20)', () => { + const failing = new WritableStream<Uint8Array>({ + write: (_chunk, controller) => { + controller.error(new Error('boom')); + }, + }); + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2, 3])), + 10, + ); + expect(logged.writeTo(failing)).rejects.toThrow(); + expect(logged.snapshot().length).toBeGreaterThan(0); + }); +}); + +describe('withRequestLogging replayability, materialize, and protection (BODY-21, 32, 37)', () => { + test('replayable passes through the delegate verbatim (BODY-21)', () => { + expect( + withRequestLogging(byteArrayBody(Uint8Array.from([1])), 10).replayable, + ).toBe(true); + const singleUse = withRequestLogging( + streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + ), + 10, + ); + expect(singleUse.replayable).toBe(false); + }); + + test('materialize() returns a still-logged, now-replayable wrapper preserving the tap (BODY-21)', async () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([7, 7])); + controller.close(); + }, + }); + const logged = withRequestLogging(streamBody(stream), 10); + expect(logged.replayable).toBe(false); + + const materialized = await logged.materialize(); + expect(materialized.replayable).toBe(true); + expect(typeof materialized.snapshot).toBe('function'); + + const {sink, written} = collectingSink(); + await materialized.writeTo(sink); + expect([...written()]).toEqual([7, 7]); + expect([...materialized.snapshot()]).toEqual([7, 7]); + }); + + test('exposes no direct handle onto the tap buffer -- snapshot is the only read path (BODY-37)', () => { + const logged = withRequestLogging(byteArrayBody(Uint8Array.from([1])), 10); + expect(Object.keys(logged)).not.toContain('tap'); + expect(Object.keys(logged)).not.toContain('buffer'); + }); + + test('the primary always receives the exact payload, independent of the tap cap (BODY-17)', async () => { + await fc.assert( + fc.asyncProperty( + fc.uint8Array({minLength: 0, maxLength: 512}), + fc.integer({min: 0, max: 600}), + async (payload, tapCap) => { + const logged = withRequestLogging(byteArrayBody(payload), tapCap); + const {sink, written} = collectingSink(); + await logged.writeTo(sink); + + expect([...written()]).toEqual([...payload]); // wire body never reduced or altered + expect(logged.snapshot().length).toBe( + Math.min(payload.length, tapCap), + ); // tap bounded + }, + ), + {seed: 0x3b}, + ); + }); + + test('a negative tap cap is rejected at construction (BODY-32)', () => { + expect(() => + withRequestLogging(byteArrayBody(Uint8Array.from([1])), -1), + ).toThrow(InvariantViolation); + }); +}); + +function bytesStream(...values: number[]): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from(values)); + controller.close(); + }, + }); +} + +describe('materialize does not alias the tap (BODY-21)', () => { + test('each wrapper keeps its own buffer, so one write cannot rewrite the other preview', async () => { + const logged = withRequestLogging( + streamBody(bytesStream(1, 2, 3), undefined, 3), + 100, + ); + const materialized = await logged.materialize(); + + const {sink} = collectingSink(); + await materialized.writeTo(sink); + + expect([...materialized.snapshot()]).toEqual([1, 2, 3]); + // BODY-18 clears the tap at the start of every write. With one shared ByteQueue, a Phase 7 retry + // loop's second attempt silently rewrites the preview the first-attempt wrapper is still holding. + expect([...logged.snapshot()]).toEqual([]); + }); + + test('the materialized wrapper still honours the configured cap', async () => { + const logged = withRequestLogging( + streamBody(bytesStream(1, 2, 3), undefined, 3), + 2, + ); + const materialized = await logged.materialize(); + const {sink} = collectingSink(); + await materialized.writeTo(sink); + expect([...materialized.snapshot()]).toEqual([1, 2]); + }); +}); + +describe('the decorator owns the sink it was handed (BODY-17, RECOV-12)', () => { + // The adapter stream the tee hands the delegate must forward BOTH teardown paths. Without an + // `abort` algorithm on it the delegate's abort stops at the decorator -- the adapter's default + // abort is a no-op -- so the real sink is never told the message is broken and a truncated body + // can be committed downstream as a complete one. + + test('a delegate failure aborts the primary sink rather than closing it', async () => { + // A declared length the stream cannot satisfy: withBodyWriter aborts, and that abort has to + // reach the caller's sink through the tee. + const {sink, aborted, closed} = observableSink(); + const short = streamBody(bytesStream(1, 2), undefined, 5); + const logged = withRequestLogging(short, 10); + + expect((await rejection(logged.writeTo(sink))).name).toBe( + 'EndOfStreamError', + ); + expect(aborted()).not.toBe(NOT_ABORTED); + expect(closed()).toBe(false); + }); + + test('a delegate that refuses before writing still tears the primary sink down', async () => { + // ConsumedBodyError is raised before the adapter is ever touched, so neither of its handlers + // runs and only writeTo's own catch can release the writer it took. + const {sink, aborted, closed} = observableSink(); + const body = streamBody(bytesStream()); + await body.writeTo(new WritableStream()); + const logged = withRequestLogging(body, 10); + + expect(await rejection(logged.writeTo(sink))).toBeInstanceOf( + ConsumedBodyError, + ); + expect(aborted()).toBeInstanceOf(ConsumedBodyError); + expect(closed()).toBe(false); + }); + + test('a delegate that resolves without closing the adapter still closes the primary', async () => { + // Body.writeTo's contract is that the body closes the sink it was given. A delegate that just + // resolves would otherwise strand the caller's sink open and locked, with nothing thrown. + const {sink, aborted, closed} = observableSink(); + const rogue: Body = { + kind: 'byte-array', + mediaType: undefined, + contentLength: 1, + replayable: true, + writeTo: async (target: WritableStream<Uint8Array>): Promise<void> => { + const writer = target.getWriter(); + await writer.write(Uint8Array.from([1])); + writer.releaseLock(); // resolves without close() or abort() + }, + }; + + await withRequestLogging(rogue, 10).writeTo(sink); + expect(closed()).toBe(true); + expect(aborted()).toBe(NOT_ABORTED); + }); + + test('a successful write closes the primary sink and never aborts it', async () => { + const {sink, aborted, closed} = observableSink(); + await withRequestLogging(byteArrayBody(Uint8Array.from([1])), 10).writeTo( + sink, + ); + expect(closed()).toBe(true); + expect(aborted()).toBe(NOT_ABORTED); + }); +}); diff --git a/packages/core/src/body/request-body-logging.ts b/packages/core/src/body/request-body-logging.ts new file mode 100644 index 0000000..c52165d --- /dev/null +++ b/packages/core/src/body/request-body-logging.ts @@ -0,0 +1,141 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/request-body-logging.ts +import {invariant} from '../invariant.js'; +import {ByteQueue} from '../io/byte-queue.js'; +import {MAX_BYTE_ARRAY_LENGTH} from '../io/limits.js'; +import type {Body} from './body.js'; +import {materialize} from './materialize.js'; + +/** + * A {@link Body} that also mirrors what it writes into a bounded, readable tap (BODY-17..21). + * + * @internal + */ +export interface LoggedBody extends Body { + /** A copy of the tap's current contents -- at most tapCapBytes of the most recent write (BODY-19). */ + snapshot(): Uint8Array; + /** Materializes the delegate while preserving the logging wrapper and the tap (BODY-21). */ + materialize(): Promise<LoggedBody>; +} + +/** + * One `writeTo` call's plumbing: the primary sink's writer, this wrapper's tap, and the cap. + * + * Extracted from the closure so the adapter-stream construction can live in its own function without + * tripping `max-params`, mirroring `response-body-logging.ts`'s `DrainState`. + */ +interface TapState { + readonly writer: WritableStreamDefaultWriter<Uint8Array>; + readonly tap: ByteQueue; + readonly cap: number; + /** Whether the delegate has already ended the adapter, by closing or aborting it. */ + settled: boolean; +} + +/** + * The adapter stream handed to the delegate: mirrors up to `cap` bytes of each chunk, then forwards the + * chunk whole (BODY-17, BODY-19). + * + * The `abort` handler is load-bearing, not symmetry for its own sake. A `Body.writeTo` aborts its sink + * on failure so the transport learns the message is broken (see `write-body.ts`); without an `abort` + * algorithm here the adapter's default is a no-op, so the abort STOPS AT THE DECORATOR -- the real sink + * is left open, still locked, and a truncated body can be committed downstream as a complete one. + */ +function tappedSink(state: TapState): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write: async chunk => { + if (state.tap.size < state.cap) { + const room = state.cap - state.tap.size; + // BODY-20/IO-27: mirror BEFORE forwarding, so a failing primary write still captures + // the chunk that failed. + state.tap.writeBytes( + room >= chunk.length ? chunk : chunk.subarray(0, room), + ); + } + await state.writer.write(chunk); // BODY-19: the full payload always reaches the primary + invariant( + state.tap.size <= state.cap, + `tap grew past its ${String(state.cap)}-byte cap`, + ); + }, + close: async () => { + state.settled = true; + await state.writer.close(); + }, + abort: async (reason: unknown) => { + state.settled = true; + await state.writer.abort(reason); + }, + }); +} + +/** + * Mirrors up to tapCapBytes of each writeTo call into an internal tap while forwarding the full, + * untruncated payload to the primary sink (BODY-17). The tap clears at the start of every write so a + * retry against a replayable delegate does not accumulate stale bytes (BODY-18). No handle onto the tap's + * buffer escapes: `snapshot()` returns a fresh, independent copy of the current contents (BODY-19, + * `../io/byte-queue.ts:96`), so a caller holding one cannot observe or disturb a later write. Each + * `materialize()` wraps its own `ByteQueue` for the same reason (BODY-21). + * + * @internal Consumed by the LOGGING pillar step (OBS-36). + */ +export function withRequestLogging( + delegate: Body, + tapCapBytes: number, +): LoggedBody { + // BODY-32: reject a negative cap, clamp to the platform's max single-array size. Without the guard a + // negative cap makes `tap.size < cap` permanently false and the tee silently mirrors nothing. + invariant( + tapCapBytes >= 0, + `tapCapBytes must be non-negative, got ${String(tapCapBytes)}`, + ); + const cap = Math.min(tapCapBytes, MAX_BYTE_ARRAY_LENGTH); + + function wrap(inner: Body): LoggedBody { + // Per-wrapper, never hoisted to the factory scope. BODY-21 asks materialize() to preserve the tap + // *cap*, not to share the buffer: two live wrappers over one ByteQueue means BODY-18's clear-on-write + // in the materialized wrapper silently rewrites the preview the pre-materialization wrapper is still + // holding -- which is precisely what a Phase 7 retry loop does between attempts. + const tap = new ByteQueue(); + return Object.freeze({ + kind: inner.kind, + mediaType: inner.mediaType, + contentLength: inner.contentLength, + get replayable() { + return inner.replayable; + }, + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + tap.clear(); // BODY-18 + const state: TapState = { + writer: sink.getWriter(), + tap, + cap, + settled: false, + }; + try { + await inner.writeTo(tappedSink(state)); + } catch (error: unknown) { + // A delegate that refuses before it ever touches the adapter -- ConsumedBodyError on a + // second write of a single-use body -- reaches neither handler in `tappedSink`, so the + // primary writer would stay locked and open forever. Best-effort, and never allowed to + // displace the primary failure (RECOV-12). + if (!state.settled) { + await state.writer.abort(error).catch(() => undefined); + } + throw error; + } + // `Body.writeTo`'s contract is that the body closes the sink it was given. This wrapper is + // the one place that takes a writer on behalf of someone else's `Body`, so a delegate that + // resolves without closing would strand the caller's sink open and locked with nothing + // thrown to notice it by. Honouring the contract on the delegate's behalf is the repair. + if (!state.settled) await state.writer.close(); + }, + snapshot(): Uint8Array { + return tap.snapshot(); + }, + materialize: async () => wrap(await materialize(inner)), + }); + } + + return wrap(delegate); +} diff --git a/packages/core/src/body/response-body-logging.test.ts b/packages/core/src/body/response-body-logging.test.ts new file mode 100644 index 0000000..9fe6b83 --- /dev/null +++ b/packages/core/src/body/response-body-logging.test.ts @@ -0,0 +1,397 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/response-body-logging.test.ts +// Exercises: BODY-22 (lazy, drain-once), BODY-23 (fits-cap: full capture, repeatable non-consuming +// reads), BODY-24 (exceeds-cap: prefix+tail once, second read fails), BODY-26 (drain failure cached, +// partial bytes retained, error() does not drain), BODY-27 (close-once shared guard), BODY-28 (captured +// buffer survives close), BODY-29 (reported length), BODY-32 (negative cap rejected), BODY-25 (a +// zero-length delegate chunk is a stream-contract violation, never end-of-stream), BODY-27/BODY-28 again +// (close() ends the drain rather than poisoning the wrapper: snapshot still serves the captured prefix, +// read() reports IO-42's state error, error() reports only a genuine drain failure) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; +import {withResponseLogging} from './response-body-logging.js'; + +function readableOf(...chunks: number[][]): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); +} + +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves instead. */ +async function rejection(promise: Promise<unknown>): Promise<Error> { + try { + await promise; + } catch (error: unknown) { + return error as Error; + } + throw new Error('expected a rejection, but the promise resolved'); +} + +async function readAll( + stream: ReadableStream<Uint8Array>, +): Promise<Uint8Array> { + const reader = stream.getReader(); + const chunks: Uint8Array[] = []; + let total = 0; + for (;;) { + const {done, value} = await reader.read(); + if (done) break; + chunks.push(value); + total += value.length; + } + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; +} + +describe('withResponseLogging regimes (BODY-22..24)', () => { + test('nothing is captured until read() is called (BODY-22 laziness)', () => { + expect( + withResponseLogging(readableOf([1, 2, 3]), 100).snapshot().length, + ).toBe(0); + }); + + test('fits-cap: fully captures, and every later read() is a fresh non-consuming view (BODY-23)', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); + }); + + test('exceeds-cap: replays the prefix then the live tail, consumer receives the complete body (BODY-24)', async () => { + const logged = withResponseLogging(readableOf([1, 2], [3, 4, 5]), 3); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3, 4, 5]); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); // only the prefix up to the cap is retained + }); + + test('exceeds-cap: a second read() throws (BODY-24)', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3, 4]), 1); + await logged.read(); + expect(logged.read()).rejects.toThrow(); + }); +}); + +describe('withResponseLogging lifecycle (BODY-27, 28)', () => { + test('the delegate is cancelled at most once however often close is called (BODY-27)', async () => { + // The exceeds-cap regime deliberately: on the fits path the delegate is already closed by the time + // the guard runs, so cancel() is a spec no-op and a counter there proves nothing -- and BODY-27 + // exists for the transports that are less forgiving than a spec-compliant ReadableStream. + const {stream, cancels} = countingStream([1, 2], [3, 4]); + const logged = withResponseLogging(stream, 1); + await logged.read(); + await logged.close(); + await logged.close(); + await logged.close(); + expect(cancels()).toBe(1); + }); + + test('the wrapper close and the tail stream share one guard (BODY-27)', async () => { + const {stream, cancels} = countingStream([1, 2], [3, 4]); + const logged = withResponseLogging(stream, 1); + const tail = await logged.read(); + await tail.cancel(); // tail path + await logged.close(); // wrapper path + expect(cancels()).toBe(1); + }); + + test('the captured buffer survives close -- snapshot still works after (BODY-28)', async () => { + const logged = withResponseLogging(readableOf([1, 2]), 100); + await readAll(await logged.read()); + await logged.close(); + expect([...logged.snapshot()]).toEqual([1, 2]); + }); + + test('teardown is close() only -- no [Symbol.asyncDispose] on the >=20.3 floor', () => { + // See Response's matching assertion: the symbol is undefined on the declared floor, so declaring + // it binds the method to the string "undefined". Absence is the assertion. + const logged = withResponseLogging(readableOf([1]), 100); + expect(Object.keys(logged)).not.toContain('undefined'); + expect(typeof logged.close).toBe('function'); + }); +}); + +describe('withResponseLogging error caching (BODY-26)', () => { + test('a drain failure is cached: read() re-throws it, snapshot keeps the partial bytes (BODY-26)', () => { + const boom = new Error('upstream reset'); + const failing = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + }, + pull(controller) { + controller.error(boom); + }, + }); + const logged = withResponseLogging(failing, 100); + + expect(logged.read()).rejects.toBe(boom); + expect(logged.read()).rejects.toBe(boom); // same cached error, upstream never re-read + expect([...logged.snapshot()]).toEqual([1, 2]); // partial capture retained, snapshot does not throw + expect(logged.error()).toBe(boom); + }); + + test('error() reports null without triggering a drain (BODY-26)', () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect(logged.error()).toBeNull(); + expect(logged.snapshot().length).toBe(0); // still undrained -- error() did not read anything + }); +}); + +describe('withResponseLogging properties and lengths (BODY-29..34)', () => { + test('contentLength is the captured size when it fits, the declared length when it does not (BODY-29)', async () => { + const fits = withResponseLogging(readableOf([1, 2, 3]), 100, 3); + await fits.read(); + expect(fits.contentLength).toBe(3); + + const exceeds = withResponseLogging(readableOf([1, 2, 3, 4]), 2, 4); + await exceeds.read(); + expect(exceeds.contentLength).toBe(4); // the delegate's true length, not the 2-byte prefix + }); + + test('a negative cap is rejected at construction (BODY-32)', () => { + expect(() => withResponseLogging(readableOf([1]), -1)).toThrow( + InvariantViolation, + ); + }); + + test('for any (cap, body) pair the consumer receives every byte and the tap stays bounded', async () => { + await fc.assert( + fc.asyncProperty( + fc.uint8Array({minLength: 0, maxLength: 512}), + fc.integer({min: 0, max: 600}), + async (payload, cap) => { + const source = new ReadableStream<Uint8Array>({ + start(controller) { + if (payload.length > 0) controller.enqueue(payload); + controller.close(); + }, + }); + const logged = withResponseLogging(source, cap); + + // BODY-34: the consumer gets the complete body whichever regime triggered. + expect([...(await readAll(await logged.read()))]).toEqual([ + ...payload, + ]); + // BODY-23/BODY-24: the capture is bounded by the cap either way. + expect(logged.snapshot().length).toBe(Math.min(payload.length, cap)); + }, + ), + {seed: 0x3b}, + ); + }); +}); + +/** + * A delegate that counts calls to `cancel()` and throws on the second, standing in for the transports + * BODY-27 names -- the ones that do not tolerate a double close. Counting the underlying source's + * `cancel` callback instead would prove nothing: the Streams spec makes a second `cancel()` on an + * already-cancelled stream a resolved no-op that never reaches the source. + */ +function countingStream(...chunks: number[][]): { + stream: ReadableStream<Uint8Array>; + cancels: () => number; +} { + let cancels = 0; + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); + const delegate = stream.cancel.bind(stream); + stream.cancel = async (reason?: unknown): Promise<void> => { + cancels += 1; + if (cancels > 1) + throw new Error('transport does not tolerate a double close'); + return delegate(reason); + }; + return {stream, cancels: () => cancels}; +} + +describe('close failures (BODY-28)', () => { + test('a non-TypeError from cancel() propagates rather than being swallowed', async () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + }, + }); + stream.cancel = (): Promise<void> => + Promise.reject(new Error('CONNECTION STUCK')); + const logged = withResponseLogging(stream, 1); + await logged.read(); // exceeds-cap regime leaves the delegate live, so close() really cancels + expect(logged.close()).rejects.toThrow('CONNECTION STUCK'); + }); +}); + +describe('delegate stream contract (BODY-25)', () => { + function emptyThenData(): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([7])); + controller.close(); + }, + }); + } + + test('a zero-length chunk is raised, never tolerated as a no-op', () => { + // Matches RetentionWindow under IO-17's identical rule: a response body reaches both this tee and + // BufferedSource, so the two layers must not disagree about the same upstream. + const logged = withResponseLogging(emptyThenData(), 100); + expect(logged.read()).rejects.toThrow(SourceContractViolationError); + }); + + test('the violation is cached like any other drain failure (BODY-26)', () => { + const logged = withResponseLogging(emptyThenData(), 100); + expect(logged.read()).rejects.toThrow(SourceContractViolationError); + expect(logged.error()).toBeInstanceOf(SourceContractViolationError); + expect(logged.read()).rejects.toThrow(SourceContractViolationError); + }); +}); + +describe('the tail path enforces the same chunk contract (BODY-25)', () => { + test('a zero-length chunk after the cap is raised, not enqueued', async () => { + // The drain stops at the cap, so a violating chunk arriving afterwards is read by tailStream, not + // drainOnce. A rule that holds in one regime and not the other makes the same upstream pass or + // fail depending only on how big the body happened to be. + let pulls = 0; + const delegate = new ReadableStream<Uint8Array>({ + pull(controller) { + pulls += 1; + if (pulls === 1) { + controller.enqueue(Uint8Array.from([1, 2, 3, 4])); + return; + } + controller.enqueue(new Uint8Array(0)); + }, + }); + const logged = withResponseLogging(delegate, 2); + const tail = await logged.read(); + + expect((await rejection(readAll(tail))).name).toBe( + 'SourceContractViolationError', + ); + // BODY-26: cached like any other delegate failure, so error() still reports it. + expect(logged.error()).toBeInstanceOf(SourceContractViolationError); + }); +}); + +describe('the tap is inert after close(), not poisoned by it (BODY-27, BODY-28)', () => { + // closeDelegate releases the reader. Every entry point that used to start a drain unconditionally then + // read from a detached reader, so `snapshot()` cached a raw `TypeError: Invalid state: The reader is + // not attached to a stream` as the wrapper's failure -- and `error()` reported that forever, over a + // capture that never failed. BODY-28 says the captured bytes survive close; they cannot survive it + // behind a fabricated error. + + test('close-then-snapshot returns the captured prefix and starts no drain', async () => { + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + await logged.read(); // exceeds-cap: the prefix is captured, the delegate stays live + await logged.close(); + + expect([...logged.snapshot()]).toEqual([1, 2]); + await new Promise(resolve => setTimeout(resolve, 0)); // a drain started here would settle by now + expect(logged.error()).toBeNull(); + }); + + test('close-before-any-read leaves snapshot empty and error() null', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + expect([...logged.snapshot()]).toEqual([]); + await new Promise(resolve => setTimeout(resolve, 0)); + expect(logged.error()).toBeNull(); + }); + + test('close-then-read rejects with ClosedResourceError, not a detached-reader TypeError', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + const error = await rejection(logged.read()); + expect(error).toBeInstanceOf(ClosedResourceError); + expect(error.message).toBe('LoggedResponseBody is closed'); + }); + + test('close-then-read in the exceeds-cap regime rejects too -- there is no live tail left', async () => { + // The drain stopped at the cap and nobody took the tail, so the captured prefix is NOT the whole + // body. Serving it would hand the consumer a silently truncated response; the delegate that held + // the rest is gone. + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + logged.snapshot(); // starts the drain without taking the tail + await new Promise(resolve => setTimeout(resolve, 0)); + expect([...logged.snapshot()]).toEqual([1, 2]); + await logged.close(); + + expect(await rejection(logged.read())).toBeInstanceOf(ClosedResourceError); + }); + + test('close-then-error reports only a genuine drain failure (BODY-26)', async () => { + const boom = new Error('upstream reset'); + const failing = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + }, + pull(controller) { + controller.error(boom); + }, + }); + const logged = withResponseLogging(failing, 100); + expect(await rejection(logged.read())).toBe(boom); + // `cancel()` on an errored stream rejects with that stream's own stored error, which the + // 'a non-TypeError from cancel() propagates' case above already pins. Not what this test is about. + await logged.close().catch(() => undefined); + + // The real failure is not displaced by a close-induced one, and snapshot still shows the partial + // capture BODY-26 asked to be retained. + expect(logged.error()).toBe(boom); + expect([...logged.snapshot()]).toEqual([1, 2]); + }); + + test('the fits-cap regime still serves repeatable reads after its own close (BODY-23, BODY-28)', async () => { + // The drain itself closes the delegate on this path, so "closed" must NOT mean "unreadable". + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + await logged.close(); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); + expect(logged.error()).toBeNull(); + }); +}); + +describe('snapshot is a drain trigger (BODY-22)', () => { + test('calling snapshot starts the drain, without a read()', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect([...logged.snapshot()]).toEqual([]); // synchronous: the drain has only just been started + await new Promise(resolve => setTimeout(resolve, 0)); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); + }); + + test('the drain still happens exactly once (BODY-22)', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + logged.snapshot(); + logged.snapshot(); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + }); + + test('a snapshot-triggered drain failure still reaches read(), and does not go unhandled', async () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.error(new Error('UPSTREAM GONE')); + }, + }); + const logged = withResponseLogging(stream, 100); + expect([...logged.snapshot()]).toEqual([]); + await new Promise(resolve => setTimeout(resolve, 0)); + expect(logged.read()).rejects.toThrow('UPSTREAM GONE'); + }); +}); diff --git a/packages/core/src/body/response-body-logging.ts b/packages/core/src/body/response-body-logging.ts new file mode 100644 index 0000000..0e23e01 --- /dev/null +++ b/packages/core/src/body/response-body-logging.ts @@ -0,0 +1,278 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/response-body-logging.ts +import {invariant} from '../invariant.js'; +import {ByteQueue} from '../io/byte-queue.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; +import {MAX_BYTE_ARRAY_LENGTH} from '../io/limits.js'; +import {ConsumedBodyError} from './errors.js'; + +/** + * A lazily-draining, bounded capture wrapper over a raw response body stream (BODY-22..29). + * + * Teardown is `close()` only. Revisit when a project-wide explicit resource management pass lands + * across all Phase 2/3a resource classes. + * + * @internal + */ +export interface LoggedResponseBody { + /** + * Returns a stream serving the body. Lazy -- nothing is read from the delegate until the first call + * (BODY-22). Fits-cap regime: every call, including calls after the first, returns a fresh + * non-consuming view over the captured bytes (BODY-23). Exceeds-cap regime: exactly one call is + * allowed; a second throws (BODY-24). If the drain failed, every call re-throws the cached error. + * After `close()` in any regime but fits-cap, throws `ClosedResourceError`: the delegate is gone and + * the captured prefix is not the whole body (BODY-27, BODY-28). + */ + read(): Promise<ReadableStream<Uint8Array>>; + /** + * Non-consuming; reflects whatever has been captured so far, even after a failed drain (BODY-26) and + * after `close()` (BODY-28), which it never restarts a drain past. + */ + snapshot(): Uint8Array; + /** + * The cached drain failure, or null. MUST NOT trigger a drain (BODY-26), and reports only a genuine + * upstream failure -- never one manufactured by reading past `close()`. + */ + error(): Error | null; + /** Captured size iff fully captured within the cap, else the delegate's declared length (BODY-29). */ + readonly contentLength: number; + /** + * Releases the delegate. Idempotent, and shared with the exceeds-cap tail stream's own completion so + * the delegate is cancelled at most once however close is reached (BODY-27). The captured bytes + * survive, so `snapshot()` still works afterwards (BODY-28). + */ + close(): Promise<void>; +} + +/** + * Mutable state for one wrapper instance. Extracted from the factory closure so the factory stays under + * the 70-line function cap and each step below is independently testable. + */ +interface DrainState { + readonly captured: ByteQueue; + readonly reader: ReadableStreamDefaultReader<Uint8Array>; + readonly delegate: ReadableStream<Uint8Array>; + readonly cap: number; + regime: 'undrained' | 'fits' | 'exceeds'; + tailConsumed: boolean; + pendingTailChunk: Uint8Array | undefined; + failure: Error | null; + closed: boolean; + started: Promise<void> | undefined; +} + +/** BODY-27: one close-once guard shared by the wrapper's close and the tail stream's completion. */ +async function closeDelegate(state: DrainState): Promise<void> { + if (state.closed) return; + state.closed = true; + // MUST precede cancel(): cancel() rejects with TypeError on a locked stream, and reading to done does + // not release the lock (see Response.bytes for the same trap). + state.reader.releaseLock(); + // BODY-28: on the fits-cap path the capture already succeeded, so a close failure is best-effort and + // must not surface as a drain error. Narrowed to the one thing cancel() reports here. + await state.delegate.cancel().catch((error: unknown) => { + if (!(error instanceof TypeError)) throw error; + }); +} + +/** + * BODY-25: a delegate chunk of zero bytes is a stream-contract violation, not a no-op and never EOF -- + * EOF is signalled only by `{done: true}`. `ReadableStreamDefaultReader.read()` carries no requested + * count, so the requirement's "for a positive requested count" has no literal analog, but the tolerant + * reading is the wrong one to pick: `RetentionWindow` raises on the same input under IO-17's identical + * rule, and a response body reaches both this tee and `BufferedSource`, so a divergence would make one + * upstream fail or succeed depending only on which wrapper it passed through. + * + * Applied on BOTH read paths -- `drainOnce` and the exceeds-cap tail -- because a rule that holds in one + * regime and not the other makes the same upstream pass or fail depending only on how big the body + * happened to be. + */ +function assertNonEmptyChunk(value: Uint8Array): void { + if (value.length === 0) { + throw new SourceContractViolationError( + 'source delivered 0 bytes without signalling end of stream', + ); + } +} + +/** + * Reads until EOF (fits regime) or until the cap is reached (exceeds regime, leaving the delegate open + * and the overflow chunk staged). BODY-26: a failure is cached, never allowed to truncate silently. + */ +async function drainOnce(state: DrainState): Promise<void> { + try { + for (;;) { + // Serial by necessity: each read depends on the previous one advancing the cursor. + const {done, value} = await state.reader.read(); + if (done) { + state.regime = 'fits'; + await closeDelegate(state); + return; + } + assertNonEmptyChunk(value); + if (state.captured.size + value.length <= state.cap) { + state.captured.writeBytes(value); + continue; + } + const room = state.cap - state.captured.size; + if (room > 0) state.captured.writeBytes(value.subarray(0, room)); + state.pendingTailChunk = value.subarray(room); + state.regime = 'exceeds'; + invariant( + state.captured.size <= state.cap, + `captured past the ${String(state.cap)}-byte cap`, + ); + return; + } + } catch (error: unknown) { + // BODY-26: retain what was read and cache the error rather than discarding a partial capture. + state.failure = error instanceof Error ? error : new Error(String(error)); + throw state.failure; + } +} + +/** + * BODY-22's once-only, lazily-started drain. Concurrent first accesses share the one in-flight promise. + * + * The detached `.catch` matters: a snapshot-triggered drain has no awaiter, so without it a drain failure + * becomes an unhandled rejection. Attaching a handler to a *copy* leaves the stored promise rejected, so + * `read()` still re-throws the cached failure on every call (BODY-26). + * + * BODY-28: after `close()` there is nothing left to drain -- `closeDelegate` released the reader, so + * starting one here reads from a detached reader, raises a raw `TypeError: Invalid state`, and + * `drainOnce`'s catch caches it as this wrapper's `failure`. `error()` would then report a fabricated + * upstream failure forever, over a capture that never failed, and the captured bytes BODY-28 promises + * survive close would be reachable only past that lie. A drain already in flight is left alone: on the + * fits-cap path the drain closes the delegate itself, and its own promise is what `read()` awaits. + */ +function startDrain(state: DrainState): Promise<void> { + if (state.closed && state.started === undefined) return Promise.resolve(); + state.started ??= drainOnce(state); + void state.started.catch(() => undefined); + return state.started; +} + +/** A fresh, non-consuming view over the fully-captured bytes. Repeatable (BODY-23). */ +function capturedStream(state: DrainState): ReadableStream<Uint8Array> { + const bytes = state.captured.snapshot(); + return new ReadableStream<Uint8Array>({ + start(controller) { + if (bytes.length > 0) controller.enqueue(bytes); + controller.close(); + }, + }); +} + +/** + * Replays the captured prefix, then continues from the still-live tail (BODY-24). Pull-driven, one + * chunk per pull: looping inside start() would eagerly materialize the whole remaining body in the + * controller's queue -- precisely the oversized payloads the cap exists to keep off the heap. + */ +function tailStream(state: DrainState): ReadableStream<Uint8Array> { + const prefix = state.captured.snapshot(); + let staged: Uint8Array | undefined = state.pendingTailChunk; + let prefixSent = false; + return new ReadableStream<Uint8Array>({ + async pull(controller) { + if (!prefixSent) { + prefixSent = true; + if (prefix.length > 0) { + controller.enqueue(prefix); + return; + } + } + if (staged !== undefined) { + const chunk = staged; + staged = undefined; + if (chunk.length > 0) { + controller.enqueue(chunk); + return; + } + } + const {done, value} = await state.reader.read(); + if (done) { + await closeDelegate(state); + controller.close(); + return; + } + try { + assertNonEmptyChunk(value); // BODY-25, same rule as the drain + } catch (error: unknown) { + // Cached like any other delegate failure so `error()` still reports it (BODY-26); the throw + // errors this stream, which is what the consumer of the tail actually observes. + state.failure = + error instanceof Error ? error : new Error(String(error)); + throw state.failure; + } + controller.enqueue(value); + }, + async cancel() { + await closeDelegate(state); + }, + }); +} + +/** + * Wraps a raw response body stream (BODY-22..29). + * + * @internal Unwired until Phase 7 supplies a Logger to drive it. + */ +export function withResponseLogging( + delegate: ReadableStream<Uint8Array>, + capBytes: number, + declaredLength = -1, +): LoggedResponseBody { + invariant( + capBytes >= 0, + `capBytes must be non-negative, got ${String(capBytes)}`, + ); // BODY-32 + const state: DrainState = { + captured: new ByteQueue(), + reader: delegate.getReader(), + delegate, + cap: Math.min(capBytes, MAX_BYTE_ARRAY_LENGTH), // BODY-32: clamp, do not attempt an impossible allocation + regime: 'undrained', + tailConsumed: false, + pendingTailChunk: undefined, + failure: null, + closed: false, + started: undefined, + }; + + return { + async read(): Promise<ReadableStream<Uint8Array>> { + await startDrain(state); // a cached failure re-throws here on every call (BODY-26) + // Ordered deliberately. `fits` first: on that path the drain closed the delegate itself, and + // BODY-23 still requires every later read to be a fresh non-consuming view -- "closed" there does + // not mean "unreadable" (BODY-28). + if (state.regime === 'fits') return capturedStream(state); + if (state.tailConsumed) { + throw new ConsumedBodyError('logged-response'); + } + // Anything else with the delegate gone: there is no live tail to continue from, and the captured + // prefix is not the whole body, so serving it would hand the consumer a silently truncated + // response. IO-42's state error, not the raw `TypeError` a detached reader throws. + if (state.closed) throw new ClosedResourceError('LoggedResponseBody'); + state.tailConsumed = true; + return tailStream(state); + }, + snapshot(): Uint8Array { + // BODY-22 lists snapshot in the drain's trigger set alongside read. The accessor is synchronous, + // so it starts the drain and returns what has been captured so far rather than awaiting it; a + // later read() awaits the very same in-flight promise, so the delegate is still read exactly once. + // (BODY-26's "snapshot returns the partial bytes without throwing" is why it cannot await here.) + // After close() `startDrain` is a no-op, so this is the post-mortem accessor BODY-28 asks for. + void startDrain(state); + return state.captured.snapshot(); + }, + error: () => state.failure, // deliberately does not drain (BODY-26) + get contentLength(): number { + // BODY-29: the capture is the true length only when the whole body fit within the cap. + return state.regime === 'fits' ? state.captured.size : declaredLength; + }, + close: () => closeDelegate(state), + }; +} diff --git a/packages/core/src/body/serde-body.test.ts b/packages/core/src/body/serde-body.test.ts new file mode 100644 index 0000000..01e3239 --- /dev/null +++ b/packages/core/src/body/serde-body.test.ts @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/serde-body.test.ts +// Exercises: SERDE-2 (serde's declared media type is the default Content-Type; never a format-agnostic +// constant), SERDE-9 (an encode failure surfaces as the SDK type with the original chained). +import {expect, test} from 'bun:test'; +import type {Serde} from '../seams/serde.js'; +import {SerializationError} from '../serde/errors.js'; +import {serdeBody} from './serde-body.js'; + +const unused = (): never => { + throw new Error('unused'); +}; + +const fakeSerde = ( + mediaType: string, + encode: (value: unknown) => Uint8Array, +): Serde => ({ + mediaType, + serializer: { + serialize: encode, + serializeToString: unused, + serializeInto: unused, + serializeTo: unused, + }, + deserializer: { + deserialize: unused, + deserializeFrom: unused, + }, +}); + +const encodeJson = (value: unknown): Uint8Array => + new TextEncoder().encode(JSON.stringify(value)); + +test("media type defaults to the serde's declared type", () => { + const body = serdeBody({a: 1}, fakeSerde('application/json', encodeJson)); + expect(body.mediaType).toBe('application/json'); +}); + +test('a non-JSON serde stamps its own type, never a format-agnostic constant', () => { + const body = serdeBody({a: 1}, fakeSerde('application/cbor', encodeJson)); + expect(body.mediaType).toBe('application/cbor'); + expect(body.mediaType).not.toBe('application/octet-stream'); +}); + +test('an explicit media type overrides the default', () => { + const body = serdeBody( + {a: 1}, + fakeSerde('application/json', encodeJson), + 'application/merge-patch+json', + ); + expect(body.mediaType).toBe('application/merge-patch+json'); +}); + +test('the body is eagerly encoded, so it is replayable and has a known length', () => { + const body = serdeBody({a: 1}, fakeSerde('application/json', encodeJson)); + expect(body.replayable).toBe(true); + expect(body.contentLength).toBe(new TextEncoder().encode('{"a":1}').length); +}); + +test('the encoded bytes reach the sink', async () => { + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(chunk); + }, + }); + await serdeBody({a: 1}, fakeSerde('application/json', encodeJson)).writeTo( + sink, + ); + const joined = chunks.reduce( + (acc, c) => acc + new TextDecoder().decode(c), + '', + ); + expect(joined).toBe('{"a":1}'); +}); + +test('an encode failure surfaces as SerializationError with the original chained', () => { + const boom = new Error('circular'); + const broken = fakeSerde('application/json', () => { + throw boom; + }); + let caught: unknown; + try { + serdeBody({a: 1}, broken); + } catch (e: unknown) { + caught = e; + } + expect(caught).toBeInstanceOf(SerializationError); + // `as SerializationError`: narrowed by the assertion above, which the compiler cannot follow. + expect((caught as SerializationError).cause).toBe(boom); +}); + +test('an already-typed SerializationError from the codec is not double-wrapped', () => { + const original = new SerializationError('codec said no'); + const broken = fakeSerde('application/json', () => { + throw original; + }); + let caught: unknown; + try { + serdeBody({a: 1}, broken); + } catch (e: unknown) { + caught = e; + } + expect(caught).toBe(original); +}); diff --git a/packages/core/src/body/serde-body.ts b/packages/core/src/body/serde-body.ts new file mode 100644 index 0000000..2597fdc --- /dev/null +++ b/packages/core/src/body/serde-body.ts @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/serde-body.ts +import type {Serde} from '../seams/serde.js'; +import {SerializationError} from '../serde/errors.js'; +import type {Body} from './body.js'; +import {byteArrayBody} from './simple-bodies.js'; + +/** + * Build a request body from a value plus a {@link Serde}, defaulting `Content-Type` to the serde's + * own declared wire media type (SERDE-2). + * + * There is deliberately **no** format-agnostic fallback on this path. `Serde.mediaType` is a + * required, non-optional field, so a serde cannot fail to declare one, and this function never + * substitutes `application/octet-stream` — a non-JSON serde silently stamping a JSON content type is + * exactly the failure SERDE-2 exists to prevent. + * + * Encoding is eager, which makes the body `replayable` (retry re-sends it) and gives it a known + * `contentLength`. A streaming, non-replayable variant is deliberately not offered: a body that + * cannot be replayed cannot survive a retry or a redirect, and every serde payload this SDK builds + * is small enough to buffer. + * + * @param value - the value to encode into the body. + * @param serde - the bundle whose serializer encodes it and whose media type labels it. + * @param mediaType - an explicit `Content-Type` override; defaults to `serde.mediaType`. + * @returns a replayable, frozen {@link Body} carrying the encoded bytes. + * @throws SerializationError when the serializer cannot encode `value`, with the backing failure + * chained as `cause` (SERDE-9). + * @throws MediaTypeParseError when the resolved media type contains a byte that would break out of + * the header it is rendered into (HTTP-26/HTTP-51). + * @public + */ +export function serdeBody( + value: unknown, + serde: Serde, + mediaType?: string, +): Body { + let bytes: Uint8Array; + try { + bytes = serde.serializer.serialize(value); + } catch (e: unknown) { + // Already the SDK's stable write-path type: rethrow rather than nest it under a second one, + // which would bury the codec's own message one `cause` deeper for no added information. + if (e instanceof SerializationError) throw e; + throw new SerializationError('failed to encode the request body', { + cause: e, + }); + } + return byteArrayBody(bytes, mediaType ?? serde.mediaType); +} diff --git a/packages/core/src/body/simple-bodies.test.ts b/packages/core/src/body/simple-bodies.test.ts new file mode 100644 index 0000000..68c0016 --- /dev/null +++ b/packages/core/src/body/simple-bodies.test.ts @@ -0,0 +1,233 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/simple-bodies.test.ts +// Exercises: HTTP-36/BODY-1 (mediaType, contentLength, replayable, writeTo), HTTP-38/BODY-35 (replayable +// by source; form-urlencoded uses "+" for space, distinct from RFC 3986 query encoding; a field value +// that cannot be rendered is raised, never dropped), HTTP-26/HTTP-51 (a media type is header-safe), +// RECOV-12 (a close failure never masks the primary write failure), HTTP-1/XCUT-15 (frozen at +// construction, so the declared length cannot be desynced from the bytes writeTo emits) +import {describe, expect, test} from 'bun:test'; +import {MediaTypeParseError} from '../http/errors.js'; +import {FormBodyValidationError} from './errors.js'; +import {multipartBody} from './multipart-body.js'; +import {streamBody} from './stream-body.js'; +import { + byteArrayBody, + formUrlEncodedBody, + stringBody, +} from './simple-bodies.js'; + +async function drain(body: { + writeTo: (sink: WritableStream<Uint8Array>) => Promise<void>; +}): Promise<Uint8Array> { + const chunks: Uint8Array[] = []; + await body.writeTo( + new WritableStream({write: chunk => void chunks.push(chunk)}), + ); + const total = chunks.reduce((sum, chunk) => sum + chunk.length, 0); + const result = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + result.set(chunk, offset); + offset += chunk.length; + } + return result; +} + +describe('ByteArrayBody', () => { + test('reports kind, mediaType, contentLength, and is always replayable', () => { + const body = byteArrayBody( + Uint8Array.from([1, 2, 3]), + 'application/octet-stream', + ); + expect(body.kind).toBe('byte-array'); + expect(body.mediaType).toBe('application/octet-stream'); + expect(body.contentLength).toBe(3); + expect(body.replayable).toBe(true); + }); + + test('defaults mediaType to undefined -- absence is undefined, never null', () => { + expect(byteArrayBody(Uint8Array.from([1])).mediaType).toBeUndefined(); + }); + + test('writeTo emits the exact bytes, twice, byte-for-byte identical (BODY-1)', async () => { + const body = byteArrayBody(Uint8Array.from([9, 8, 7])); + expect([...(await drain(body))]).toEqual([9, 8, 7]); + expect([...(await drain(body))]).toEqual([9, 8, 7]); + }); + + test('holds an independent copy -- mutating the caller array afterwards does not change it', async () => { + const input = Uint8Array.from([1, 2, 3]); + const body = byteArrayBody(input); + input[0] = 99; + expect([...(await drain(body))]).toEqual([1, 2, 3]); + }); +}); + +describe('StringBody', () => { + test('encodes UTF-8 and reports the byte length, not the character length', () => { + const body = stringBody('héllo'); + expect(body.contentLength).toBe(6); // "é" is 2 bytes in UTF-8 + expect(body.replayable).toBe(true); + }); + + test('writeTo emits the UTF-8 bytes', async () => { + expect(new TextDecoder().decode(await drain(stringBody('hi')))).toBe('hi'); + }); +}); + +describe('FormUrlEncodedBody (HTTP-38/BODY-35)', () => { + test('mediaType is fixed and the body is always replayable', () => { + const body = formUrlEncodedBody(new Map([['a', 'b']])); + expect(body.mediaType).toBe('application/x-www-form-urlencoded'); + expect(body.replayable).toBe(true); + }); + + test('encodes space as "+" rather than "%20"', async () => { + const body = formUrlEncodedBody(new Map([['q', 'a b']])); + expect(new TextDecoder().decode(await drain(body))).toBe('q=a+b'); + }); + + test('joins multiple params with "&", preserving insertion order', async () => { + const body = formUrlEncodedBody( + new Map([ + ['a', '1'], + ['b', '2'], + ]), + ); + expect(new TextDecoder().decode(await drain(body))).toBe('a=1&b=2'); + }); + + test('percent-encodes reserved characters in keys and values', async () => { + const body = formUrlEncodedBody(new Map([['a&b', 'c=d']])); + expect(new TextDecoder().decode(await drain(body))).toBe('a%26b=c%3Dd'); + }); +}); + +function failingSink(): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write: () => { + throw new Error('SOCKET GONE'); + }, + }); +} + +function decode(bytes: Uint8Array): string { + return new TextDecoder().decode(bytes); +} + +describe('media types are header-safe (HTTP-26/HTTP-51)', () => { + test('byteArrayBody rejects a media type carrying CR/LF', () => { + expect(() => + byteArrayBody(Uint8Array.from([1]), 'text/plain\r\nX-Injected: pwned'), + ).toThrow(MediaTypeParseError); + }); + + test('stringBody rejects a media type carrying a control character', () => { + expect(() => stringBody('x', 'text/plain\u0007')).toThrow( + MediaTypeParseError, + ); + }); + + test('a parameterised media type is still accepted', () => { + expect( + byteArrayBody(Uint8Array.from([1]), 'text/plain; charset=utf-8') + .mediaType, + ).toBe('text/plain; charset=utf-8'); + }); +}); + +describe('a write failure is never masked by the close (RECOV-12, RETRY-2)', () => { + test('ByteArrayBody surfaces the sink failure', () => { + expect( + byteArrayBody(Uint8Array.from([1, 2])).writeTo(failingSink()), + ).rejects.toThrow('SOCKET GONE'); + }); + + test('StringBody surfaces the sink failure', () => { + expect(stringBody('hi').writeTo(failingSink())).rejects.toThrow( + 'SOCKET GONE', + ); + }); + + test('FormUrlEncodedBody surfaces the sink failure', () => { + expect(formUrlEncodedBody({a: 'b'}).writeTo(failingSink())).rejects.toThrow( + 'SOCKET GONE', + ); + }); +}); + +describe('form field values (HTTP-38/BODY-35)', () => { + test('primitives are rendered rather than dropped', async () => { + const body = formUrlEncodedBody({count: 5, flag: true, big: 9n}); + expect(decode(await drain(body))).toBe('count=5&flag=true&big=9'); + }); + + test('null renders as a valueless parameter', async () => { + expect(decode(await drain(formUrlEncodedBody({empty: null})))).toBe( + 'empty=', + ); + }); + + test('array values render element-wise', async () => { + expect(decode(await drain(formUrlEncodedBody({tag: ['a', 2]})))).toBe( + 'tag=a&tag=2', + ); + }); + + test('a value that cannot be rendered throws naming the field', () => { + expect(() => formUrlEncodedBody({profile: {a: 1}} as never)).toThrow( + FormBodyValidationError, + ); + expect(() => formUrlEncodedBody({profile: {a: 1}} as never)).toThrow( + /"profile"/, + ); + }); + + test('undefined is rejected too -- an absent field is never guessed at', () => { + expect(() => formUrlEncodedBody({missing: undefined} as never)).toThrow( + FormBodyValidationError, + ); + }); +}); + +describe('every Body variant is frozen at construction (HTTP-1)', () => { + // `readonly` is erased at run time. Without the freeze a caller can reassign contentLength after + // construction and desync the value a transport stamps into Content-Length from the bytes writeTo + // emits -- the same drift HTTP-51 makes MultipartBody share one framing routine to prevent. + const variants = (): {name: string; body: object}[] => [ + {name: 'ByteArrayBody', body: byteArrayBody(Uint8Array.from([1, 2, 3]))}, + {name: 'StringBody', body: stringBody('abc')}, + {name: 'FormUrlEncodedBody', body: formUrlEncodedBody({a: 'b'})}, + { + name: 'StreamBody', + body: streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + ), + }, + { + name: 'MultipartBody', + body: multipartBody([{name: 'a', body: stringBody('x')}], 'B'), + }, + ]; + + for (const {name, body} of variants()) { + test(`${name} is frozen and refuses a contentLength reassignment`, () => { + expect(Object.isFrozen(body)).toBe(true); + expect(() => { + (body as {contentLength: number}).contentLength = 999; + }).toThrow(TypeError); + }); + } + + test('the declared length still matches the bytes written after a reassignment attempt', async () => { + const body = byteArrayBody(Uint8Array.from([1, 2, 3])); + expect(() => { + (body as {contentLength: number}).contentLength = 999; + }).toThrow(TypeError); + expect((await drain(body)).length).toBe(body.contentLength); + }); +}); diff --git a/packages/core/src/body/simple-bodies.ts b/packages/core/src/body/simple-bodies.ts new file mode 100644 index 0000000..db2c8af --- /dev/null +++ b/packages/core/src/body/simple-bodies.ts @@ -0,0 +1,236 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/simple-bodies.ts +import {QueryParams, type QueryParamsBuilder} from '../http/query-params.js'; +import {invariant} from '../invariant.js'; +import type {Body} from './body.js'; +import {FormBodyValidationError} from './errors.js'; +import {freezeBody} from './freeze-body.js'; +import {assertHeaderSafeMediaType} from './media-type-safety.js'; +import {withBodyWriter} from './write-body.js'; + +/** + * A body backed by an in-memory byte array (BODY-1). Always replayable. + * + * @public + */ +export class ByteArrayBody implements Body { + /** Discriminates this variant within the {@link Body} union. */ + readonly kind = 'byte-array' as const; + /** The declared media type, or `undefined` when the caller supplied none. */ + readonly mediaType: string | undefined; + /** The exact byte count `writeTo` will emit -- always known for an in-memory body. */ + readonly contentLength: number; + /** Always `true`: the bytes are held in memory, so every write is byte-for-byte identical. */ + readonly replayable = true; + readonly #bytes: Uint8Array; + + constructor(bytes: Uint8Array, mediaType?: string) { + assertHeaderSafeMediaType(mediaType); // HTTP-26/HTTP-51 + // Defensive copy: `bytes` caller passed might be mutated later (HTTP-1). Kept `#private` -- + // exposing this publicly would let a caller mutate a "replayable" body's contents after + // construction, silently breaking the byte-for-byte-identical guarantee BODY-1 requires. + this.#bytes = Uint8Array.from(bytes); + this.mediaType = mediaType; + this.contentLength = this.#bytes.length; + freezeBody(this); + } + + /** + * Writes the held bytes into `sink`, then closes it (BODY-1). Repeatable. + * + * @param sink - the destination; this body's to close, the caller's only to supply. + */ + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + await withBodyWriter(sink, async writer => { + if (this.#bytes.length > 0) await writer.write(this.#bytes); + }); + } +} + +/** + * Creates a replayable ByteArrayBody (BODY-1). + * + * @throws MediaTypeParseError when `mediaType` contains a control character or non-ASCII byte, which + * would let it break out of the header it is rendered into (HTTP-26/HTTP-51). + * @public + */ +export function byteArrayBody( + bytes: Uint8Array, + mediaType?: string, +): ByteArrayBody { + return new ByteArrayBody(bytes, mediaType); +} + +/** + * A body backed by an in-memory string (BODY-1). Always replayable. + * + * @public + */ +export class StringBody implements Body { + /** Discriminates this variant within the {@link Body} union. */ + readonly kind = 'string' as const; + /** Defaults to `text/plain; charset=utf-8`, matching the UTF-8 encoding `writeTo` emits. */ + readonly mediaType: string; + /** The UTF-8 byte count, which is not the character count for non-ASCII text. */ + readonly contentLength: number; + /** Always `true`: the text is held in memory, so every write is byte-for-byte identical. */ + readonly replayable = true; + /** The source text, exactly as supplied. */ + readonly text: string; + readonly #bytes: Uint8Array; + + constructor(text: string, mediaType = 'text/plain; charset=utf-8') { + assertHeaderSafeMediaType(mediaType); // HTTP-26/HTTP-51 + this.text = text; + this.mediaType = mediaType; + this.#bytes = new TextEncoder().encode(text); + this.contentLength = this.#bytes.length; + freezeBody(this); + } + + /** + * Writes the UTF-8 encoding of {@link StringBody.text} into `sink`, then closes it (BODY-1). + * Repeatable. + * + * @param sink - the destination; this body's to close, the caller's only to supply. + */ + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + await withBodyWriter(sink, async writer => { + if (this.#bytes.length > 0) await writer.write(this.#bytes); + }); + } +} + +/** + * Creates a replayable StringBody (BODY-1). + * + * @throws MediaTypeParseError when `mediaType` contains a control character or non-ASCII byte, which + * would let it break out of the header it is rendered into (HTTP-26/HTTP-51). + * @public + */ +export function stringBody( + text: string, + mediaType = 'text/plain; charset=utf-8', +): StringBody { + return new StringBody(text, mediaType); +} + +/** + * A form field value. Primitives are rendered with their standard string form; `null` produces a + * valueless parameter. Anything else is rejected rather than silently dropped. + * + * @public + */ +export type FormUrlEncodedValue = string | number | boolean | bigint | null; + +/** + * Accepted input shapes for {@link formUrlEncodedBody}. + * + * @public + */ +export type FormUrlEncodedInput = + | QueryParams + | ReadonlyMap<string, FormUrlEncodedValue | readonly FormUrlEncodedValue[]> + | Record<string, FormUrlEncodedValue | readonly FormUrlEncodedValue[]> + | readonly (readonly [string, FormUrlEncodedValue])[]; + +// BODY-35: a form field that is neither a primitive nor null cannot be rendered, and dropping it would +// put a silently incomplete body on the wire. Fail naming the key instead. +function toFieldValue(key: string, value: unknown): string | null { + if (typeof value === 'string' || value === null) return value; + if ( + typeof value === 'number' || + typeof value === 'boolean' || + typeof value === 'bigint' + ) { + return String(value); + } + throw new FormBodyValidationError(key, value); +} + +function addParamValue( + builder: QueryParamsBuilder, + key: string, + value: unknown, +): void { + if (Array.isArray(value)) { + for (const element of value as readonly unknown[]) { + builder.add(key, toFieldValue(key, element)); + } + return; + } + builder.add(key, toFieldValue(key, value)); +} + +function toQueryParams(input: FormUrlEncodedInput): QueryParams { + if (input instanceof QueryParams) return input; + const builder = QueryParams.newBuilder(); + const entries: readonly (readonly [unknown, unknown])[] = + input instanceof Map + ? [...input.entries()] + : Array.isArray(input) + ? (input as readonly (readonly [unknown, unknown])[]) + : Object.entries(input); + for (const [key, value] of entries) { + if (typeof key !== 'string') + throw new FormBodyValidationError(String(key), key); + addParamValue(builder, key, value); + } + return builder.build(); +} + +/** + * A body backed by URL-encoded form data (BODY-1, HTTP-38/BODY-35). Always replayable. + * + * @public + */ +export class FormUrlEncodedBody implements Body { + /** Discriminates this variant within the {@link Body} union. */ + readonly kind = 'form-urlencoded' as const; + /** Fixed at `application/x-www-form-urlencoded` -- the encoding defines the media type. */ + readonly mediaType = 'application/x-www-form-urlencoded'; + /** The byte count of the encoded form, always known. */ + readonly contentLength: number; + /** Always `true`: the encoded form is held in memory (BODY-35). */ + readonly replayable = true; + /** The normalized parameters, whatever input shape they were built from. */ + readonly params: QueryParams; + readonly #bytes: Uint8Array; + + constructor(input: FormUrlEncodedInput) { + this.params = toQueryParams(input); + // HTTP-38/BODY-35: x-www-form-urlencoded uses '+' for space, distinct from RFC 3986 query encoding. + const encoded = this.params.encode().replace(/%20/g, '+'); + invariant( + !encoded.includes(' '), + 'form-urlencoded encoding produced illegal space', + ); + this.#bytes = new TextEncoder().encode(encoded); + this.contentLength = this.#bytes.length; + freezeBody(this); + } + + /** + * Writes the encoded form into `sink`, then closes it (BODY-1). Repeatable. + * + * @param sink - the destination; this body's to close, the caller's only to supply. + */ + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + await withBodyWriter(sink, async writer => { + if (this.#bytes.length > 0) await writer.write(this.#bytes); + }); + } +} + +/** + * Creates a replayable FormUrlEncodedBody (BODY-1, HTTP-38/BODY-35). + * + * @throws FormBodyValidationError when a field name is not a string, or a field value is neither a + * primitive nor `null` -- such a field cannot be rendered and is never dropped silently. + * @public + */ +export function formUrlEncodedBody( + input: FormUrlEncodedInput, +): FormUrlEncodedBody { + return new FormUrlEncodedBody(input); +} diff --git a/packages/core/src/body/stream-body.test.ts b/packages/core/src/body/stream-body.test.ts new file mode 100644 index 0000000..b966275 --- /dev/null +++ b/packages/core/src/body/stream-body.test.ts @@ -0,0 +1,341 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/stream-body.test.ts +// Exercises: BODY-9 (always single-use -- no generic mark/reset on Node's ReadableStream), BODY-3 +// (second write fails loudly and is race-safe), BODY-8 (caller's stream is not force-closed -- read to +// natural exhaustion), HTTP-39/BODY-10 (declared length verified, short stream raises +// delivered-of-declared, and an overrunning stream is stopped BEFORE the extra bytes reach the sink), +// IO-3 (a contentLength below the -1 sentinel is rejected), HTTP-26/HTTP-51 (a media type is +// header-safe), RECOV-12 (a close failure never masks the primary write failure), HTTP-1 (frozen at +// construction so the declared length cannot be desynced from the written bytes), HTTP-39/BODY-10 again +// (a zero-length delivery during an exact-length copy is a source-contract violation, never a no-op and +// never spun on, and no empty chunk reaches the sink) +import {describe, expect, test} from 'bun:test'; +import {MediaTypeParseError} from '../http/errors.js'; +import {InvariantViolation} from '../invariant.js'; +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; +import {ConsumedBodyError} from './errors.js'; +import {streamBody} from './stream-body.js'; + +/** Sentinel distinguishing "cancel() never ran" from "cancel() ran with undefined". */ +const NOT_CANCELLED = Symbol('not-cancelled'); + +function readableOf(...chunks: number[][]): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); +} + +function collectingSink(): { + sink: WritableStream<Uint8Array>; + written: () => Uint8Array; +} { + const chunks: Uint8Array[] = []; + const sink = new WritableStream<Uint8Array>({ + write: chunk => void chunks.push(chunk), + }); + return { + sink, + written: () => { + const total = chunks.reduce((sum, c) => sum + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; + }, + }; +} + +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves instead. */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error: unknown) { + return error; + } + throw new Error('expected a rejection, but the promise resolved'); +} + +describe('caller stream ownership (BODY-8)', () => { + test('a sink failure does not cancel the caller stream on the unknown-length path', async () => { + // `pipeTo`'s default (`preventCancel: false`) cancels the SOURCE when the destination errors, + // which takes cancellation ownership away from the caller on exactly the failure path -- and + // disagrees with the declared-length path below, which only releases its reader. + let cancelReason: unknown = NOT_CANCELLED; + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + }, + cancel(reason) { + cancelReason = reason; + }, + }); + const failing = new WritableStream<Uint8Array>({ + write: () => { + throw new Error('SOCKET GONE'); + }, + }); + + expect(streamBody(source).writeTo(failing)).rejects.toThrow('SOCKET GONE'); + await Promise.resolve(); + expect(cancelReason).toBe(NOT_CANCELLED); + }); + + test('a sink failure does not cancel the caller stream on the declared-length path either', async () => { + let cancelReason: unknown = NOT_CANCELLED; + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + controller.close(); + }, + cancel(reason) { + cancelReason = reason; + }, + }); + const failing = new WritableStream<Uint8Array>({ + write: () => { + throw new Error('SOCKET GONE'); + }, + }); + + expect(streamBody(source, undefined, 3).writeTo(failing)).rejects.toThrow( + 'SOCKET GONE', + ); + await Promise.resolve(); + expect(cancelReason).toBe(NOT_CANCELLED); + }); +}); + +describe('StreamBody properties and writeTo (BODY-1, BODY-9)', () => { + test('is always single-use, regardless of declared length (BODY-9)', () => { + expect(streamBody(readableOf([1, 2]), undefined, 2).replayable).toBe(false); + }); + + test('reports the caller-supplied mediaType and contentLength', () => { + const body = streamBody(readableOf([1]), 'application/octet-stream', 1); + expect(body.mediaType).toBe('application/octet-stream'); + expect(body.contentLength).toBe(1); + }); + + test('defaults contentLength to -1 (unknown)', () => { + expect(streamBody(readableOf([1])).contentLength).toBe(-1); + }); + + test('writeTo forwards the exact bytes', async () => { + const {sink, written} = collectingSink(); + await streamBody(readableOf([1, 2], [3])).writeTo(sink); + expect([...written()]).toEqual([1, 2, 3]); + }); + + test('a second write throws ConsumedBodyError (BODY-3)', async () => { + const body = streamBody(readableOf([1])); + await body.writeTo(collectingSink().sink); + expect(body.writeTo(collectingSink().sink)).rejects.toThrow( + ConsumedBodyError, + ); + }); +}); + +describe('StreamBody declared length verification (HTTP-39, BODY-10, IO-3)', () => { + test('a declared length the stream cannot satisfy raises EndOfStreamError (HTTP-39/BODY-10)', () => { + const body = streamBody(readableOf([1, 2]), undefined, 5); + expect(body.writeTo(collectingSink().sink)).rejects.toThrow( + EndOfStreamError, + ); + }); + + test('a satisfied declared length writes exactly that many bytes (HTTP-39/BODY-10)', async () => { + const {sink, written} = collectingSink(); + await streamBody(readableOf([1, 2], [3]), undefined, 3).writeTo(sink); + expect([...written()]).toEqual([1, 2, 3]); + }); + + test('a declared length of 0 is a legitimate empty write (BODY-10)', () => { + const {sink, written} = collectingSink(); + void streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo(sink); + expect(written().length).toBe(0); + }); + + test('a contentLength below the -1 sentinel is rejected at construction (IO-3)', () => { + expect(() => streamBody(readableOf([1]), undefined, -2)).toThrow( + InvariantViolation, + ); + }); + + test('concurrent first writes: exactly one proceeds, the other rejects (BODY-3 race-safety)', async () => { + const body = streamBody(readableOf([1, 2, 3])); + const results = await Promise.allSettled([ + body.writeTo(collectingSink().sink), + body.writeTo(collectingSink().sink), + ]); + expect(results.filter(r => r.status === 'fulfilled').length).toBe(1); + expect(results.filter(r => r.status === 'rejected').length).toBe(1); + }); +}); + +interface SinkState { + written: number[]; + closed: boolean; + aborted: boolean; +} + +function probeSink(): {state: SinkState; sink: WritableStream<Uint8Array>} { + const state: SinkState = {written: [], closed: false, aborted: false}; + const sink = new WritableStream<Uint8Array>({ + write: chunk => void state.written.push(...chunk), + close: () => void (state.closed = true), + abort: () => void (state.aborted = true), + }); + return {state, sink}; +} + +describe('a mis-framed body never reaches the wire (HTTP-39/BODY-10)', () => { + test('an overrunning chunk is refused before any of it is written', () => { + const {state, sink} = probeSink(); + expect( + streamBody(readableOf([1, 2, 3, 4, 5, 6, 7, 8]), undefined, 3).writeTo( + sink, + ), + ).rejects.toThrow(EndOfStreamError); + // Not [1,2,3,4,5,6,7,8]: once a transport has stamped Content-Length: 3, the surplus sits on the + // socket where the peer reads it as the start of the next message. + expect(state.written).toEqual([]); + expect(state.aborted).toBe(true); + }); + + test('bytes written before the overrun stay written, the straddling chunk does not', () => { + const {state, sink} = probeSink(); + expect( + streamBody(readableOf([1, 2], [3, 4]), undefined, 3).writeTo(sink), + ).rejects.toThrow(EndOfStreamError); + expect(state.written).toEqual([1, 2]); + }); + + test('a short stream aborts the sink rather than closing it cleanly', () => { + const {state, sink} = probeSink(); + expect( + streamBody(readableOf([1]), undefined, 5).writeTo(sink), + ).rejects.toThrow(EndOfStreamError); + expect(state.aborted).toBe(true); + expect(state.closed).toBe(false); // a truncated body is never signalled as complete + }); + + test('an exact-length stream closes the sink cleanly', async () => { + const {state, sink} = probeSink(); + await streamBody(readableOf([1, 2, 3]), undefined, 3).writeTo(sink); + expect(state.written).toEqual([1, 2, 3]); + expect(state.closed).toBe(true); + expect(state.aborted).toBe(false); + }); +}); + +describe('a zero-length delivery is a source-contract violation (HTTP-39/BODY-10)', () => { + /** + * The auditor's probe, bounded so an unguarded run terminates instead of hanging the suite: a source + * that delivers nothing but empty chunks, then finally ends. Unbounded is the real-world shape, and + * the pull count below is what proves the copy did not spin on it. + */ + function emptyOnly(limit: number): { + stream: ReadableStream<Uint8Array>; + pulls: () => number; + } { + let pulls = 0; + const stream = new ReadableStream<Uint8Array>({ + pull(controller) { + pulls += 1; + if (pulls > limit) { + controller.close(); + return; + } + controller.enqueue(new Uint8Array(0)); + }, + }); + return {stream, pulls: () => pulls}; + } + + test('an empty-only source raises on the first empty chunk rather than spinning on it', async () => { + const {stream, pulls} = emptyOnly(1000); + const {state, sink} = probeSink(); + const error = await rejection( + streamBody(stream, undefined, 3).writeTo(sink), + ); + + // Not EndOfStreamError after a thousand futile reads: an unbounded source of empty chunks never + // ends, so nothing downstream can ever diagnose it, and every one of those chunks reaches the sink. + expect(error).toBeInstanceOf(SourceContractViolationError); + // Two, not one: the default queuing strategy reads one chunk ahead, so the source is pulled again + // the moment our read drains its queue. What matters is that it is not `limit`. + expect(pulls()).toBeLessThanOrEqual(2); + expect(state.written).toEqual([]); + expect(state.aborted).toBe(true); // a mis-framed body is never signalled as a clean close + }); + + test('an empty chunk between real chunks is raised, and never reaches the sink', async () => { + const emptyBetween = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths: number[] = []; + const sink = new WritableStream<Uint8Array>({ + write: chunk => void chunkLengths.push(chunk.length), + }); + + expect( + await rejection(streamBody(emptyBetween, undefined, 3).writeTo(sink)), + ).toBeInstanceOf(SourceContractViolationError); + // `io/buffered-sink.ts` writeString: a zero-length chunk is HTTP/1.1 chunked encoding's TERMINATING + // chunk, so forwarding one ends the request body early on the wire. + expect(chunkLengths).toEqual([2]); + }); + + test('a declared length of 0 over a source that just closes stays a legitimate empty write (BODY-10)', async () => { + const {state, sink} = probeSink(); + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo(sink); + expect(state.written).toEqual([]); + expect(state.closed).toBe(true); + }); +}); + +describe('StreamBody media type and failure propagation', () => { + test('rejects a media type carrying CR/LF (HTTP-26/HTTP-51)', () => { + expect(() => + streamBody(readableOf([1]), 'text/plain\r\nX-Injected: pwned'), + ).toThrow(MediaTypeParseError); + }); + + test('surfaces the sink failure, not a close TypeError (RECOV-12)', () => { + const sink = new WritableStream<Uint8Array>({ + write: () => { + throw new Error('SOCKET GONE'); + }, + }); + expect( + streamBody(readableOf([1, 2]), undefined, 2).writeTo(sink), + ).rejects.toThrow('SOCKET GONE'); + }); +}); diff --git a/packages/core/src/body/stream-body.ts b/packages/core/src/body/stream-body.ts new file mode 100644 index 0000000..093e0a7 --- /dev/null +++ b/packages/core/src/body/stream-body.ts @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/stream-body.ts +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; +import {invariant} from '../invariant.js'; +import type {Body} from './body.js'; +import {ConsumedBodyError} from './errors.js'; +import {freezeBody} from './freeze-body.js'; +import {assertHeaderSafeMediaType} from './media-type-safety.js'; +import {withBodyWriter} from './write-body.js'; + +/** + * HTTP-39/BODY-10: a zero-length delivery during an exact-length copy is a source-contract violation, + * never end-of-stream and never something to spin on -- `{done: true}` is the only end signal. + * + * Two failure modes, and neither one is diagnosable anywhere else. A source that only ever yields empty + * chunks never ends, so the delivered-of-declared check below is never reached; and every empty chunk + * that gets past here reaches the transport sink, where to an HTTP/1.1 chunked-encoding transport a + * zero-length chunk is the TERMINATING chunk (`io/buffered-sink.ts`'s `writeString`) -- so tolerating + * one ends the request body early on the wire while the copy still believes it is mid-body. + * + * The wording and the error type are `io/retention-window.ts`'s deliberately: `read()` carries no + * requested count, so the requirement's "for a positive requested count" has no literal analog, and a + * request body reaches both this copy and `BufferedSource` -- a divergence would make the same upstream + * fail or succeed depending only on which wrapper it passed through. `body/response-body-logging.ts` + * makes the same call for BODY-25 on the response side. + * + * A declared length of 0 is still a legitimate empty write (BODY-10): that is a source that signals + * `{done: true}` immediately, which never reaches this check. + */ +function assertNonEmptyChunk(value: Uint8Array): void { + if (value.length === 0) { + throw new SourceContractViolationError( + 'source delivered 0 bytes without signalling end of stream', + ); + } +} + +/** + * A single-use body backed by a caller-supplied stream. + * + * @public + */ +export class StreamBody implements Body { + /** Discriminates this variant within the {@link Body} union. */ + readonly kind = 'stream' as const; + /** The declared media type, or `undefined` when the caller supplied none. */ + readonly mediaType: string | undefined; + /** The caller-declared byte count, or -1 when unknown (BODY-10). */ + readonly contentLength: number; + /** Always `false` -- Node's `ReadableStream` has no generic mark/reset (BODY-9). */ + readonly replayable = false; + readonly #stream: ReadableStream<Uint8Array>; + // Not `readonly`, and deliberately unaffected by `freezeBody(this)` below: freeze never touches + // `#private` fields, so BODY-3's consumed-once guard still works on a frozen instance. + #consumed = false; + + constructor( + stream: ReadableStream<Uint8Array>, + mediaType?: string, + contentLength = -1, + ) { + assertHeaderSafeMediaType(mediaType); // HTTP-26/HTTP-51 + invariant( + contentLength >= -1, + `contentLength must be >= -1 (-1 = unknown), got ${String(contentLength)}`, + ); // IO-3 + this.#stream = stream; + this.mediaType = mediaType; + this.contentLength = contentLength; + freezeBody(this); // HTTP-1 + } + + /** + * Writes every byte of the wrapped stream into `sink`, then closes it (BODY-1). + * + * @param sink - the destination; this body's to close, the caller's only to supply. + * @throws {@link ConsumedBodyError} on a second call -- this body is single-use (BODY-3). + * @throws EndOfStreamError when a declared `contentLength` disagrees with the bytes the stream + * actually yields, in either direction (HTTP-39/BODY-10). + * @throws SourceContractViolationError when the stream delivers a zero-length chunk without + * signalling end of stream during a declared-length write (HTTP-39/BODY-10). + */ + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + if (this.#consumed) throw new ConsumedBodyError('stream'); + this.#consumed = true; // set before the first await -- BODY-3's race-safety guard + + if (this.contentLength < 0) { + // BODY-8: `preventCancel` is load-bearing, not a default worth inheriting. `pipeTo`'s default + // (`preventCancel: false`) cancels the SOURCE when the destination fails -- taking cancellation + // ownership away from the caller on exactly the failure path where they need it, and + // contradicting `#writeExactly` below, which only releases its reader. Without it one class + // has two opposite ownership rules depending on whether a length was declared. + await this.#stream.pipeTo(sink, {preventCancel: true}); + return; + } + await this.#writeExactly(sink, this.contentLength); + } + + /** HTTP-39/BODY-10: writes precisely `declared` bytes or raises naming delivered-of-declared. */ + async #writeExactly( + sink: WritableStream<Uint8Array>, + declared: number, + ): Promise<void> { + const reader = this.#stream.getReader(); + try { + await withBodyWriter(sink, async writer => { + let delivered = 0; + for (;;) { + // Serial by necessity: each read depends on the previous one advancing the cursor. + const {done, value} = await reader.read(); + if (done) break; + assertNonEmptyChunk(value); // HTTP-39/BODY-10 + // Checked BEFORE the write, not after the loop: once a transport has stamped the declared + // Content-Length, an overrun byte sits on the socket where the peer reads it as the start of + // the next message, and a thrown error cannot recall bytes already written (HTTP-39/BODY-10). + if (delivered + value.length > declared) { + throw new EndOfStreamError(delivered + value.length, declared); + } + delivered += value.length; + await writer.write(value); + } + // Raised inside the writer scope so withBodyWriter aborts: a truncated body must never be + // signalled to the sink as a clean close. + if (delivered !== declared) { + throw new EndOfStreamError(delivered, declared); + } + }); + } finally { + reader.releaseLock(); // BODY-8: release our handle, never cancel the caller's stream + } + } +} + +/** + * Creates a single-use StreamBody (BODY-9). + * + * @throws MediaTypeParseError when `mediaType` contains a control character or non-ASCII byte, which + * would let it break out of the header it is rendered into (HTTP-26/HTTP-51). + * @throws ConsumedBodyError from `writeTo` when the body has already been written once (BODY-3). + * @throws EndOfStreamError from `writeTo` when the stream yields a byte count other than the declared + * `contentLength` (HTTP-39/BODY-10). + * @throws SourceContractViolationError from `writeTo` when the stream delivers a zero-length chunk + * without signalling end of stream during a declared-length write (HTTP-39/BODY-10). + * @public + */ +export function streamBody( + stream: ReadableStream<Uint8Array>, + mediaType?: string, + contentLength = -1, +): StreamBody { + return new StreamBody(stream, mediaType, contentLength); +} diff --git a/packages/core/src/body/typed-response.test.ts b/packages/core/src/body/typed-response.test.ts new file mode 100644 index 0000000..c794015 --- /dev/null +++ b/packages/core/src/body/typed-response.test.ts @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/typed-response.test.ts +// Exercises: HTTP-44 (raw fields without touching the body, parse-once memoized including failure -- +// a synchronous throw included), HTTP-45 (concurrent first callers serialized to one parse run) +import {describe, expect, test} from 'bun:test'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {TypedResponse} from './typed-response.js'; + +function readableOf(text: string): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(c) { + c.enqueue(new TextEncoder().encode(text)); + c.close(); + }, + }); +} + +function baseResponse( + body: ReadableStream<Uint8Array> | null = null, +): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .reasonPhrase('OK') + .body(body) + .build(); +} + +describe('TypedResponse', () => { + test('exposes raw fields without touching the body (HTTP-44)', () => { + const response = baseResponse(readableOf('untouched')); + const typed = new TypedResponse(response, r => r.text()); + expect(typed.status.code).toBe(200); + expect(typed.headers).toBe(response.headers); + expect(typed.protocol).toBe('http/1.1'); + expect(typed.reason).toBe('OK'); + expect(typed.request).toBe(response.request); + expect(response.body?.locked).toBe(false); + }); + + test('parses on first value() call and memoizes the result', async () => { + let calls = 0; + const typed = new TypedResponse(baseResponse(readableOf('x')), () => { + calls += 1; + return Promise.resolve('parsed'); + }); + expect(await typed.value()).toBe('parsed'); + expect(await typed.value()).toBe('parsed'); + expect(calls).toBe(1); + }); + + test('memoizes a thrown failure -- every later call re-throws the same error, parse never re-runs', () => { + let calls = 0; + const failure = new Error('parse failed'); + const typed = new TypedResponse(baseResponse(readableOf('x')), () => { + calls += 1; + return Promise.reject(failure); + }); + expect(typed.value()).rejects.toBe(failure); + expect(typed.value()).rejects.toBe(failure); + expect(calls).toBe(1); + }); + + test('concurrent first callers share one in-flight parse (HTTP-45)', async () => { + let calls = 0; + const typed = new TypedResponse(baseResponse(readableOf('x')), async () => { + calls += 1; + await Promise.resolve(); + return 'value'; + }); + const [a, b] = await Promise.all([typed.value(), typed.value()]); + expect(a).toBe('value'); + expect(b).toBe('value'); + expect(calls).toBe(1); + }); +}); + +describe('memoization covers a synchronously-throwing parser (HTTP-44)', () => { + test('the handler runs once even when it throws before returning a promise', () => { + let calls = 0; + // Typed `=> Promise<T>` but not `async`: validating an argument before the first await is ordinary, + // and a bare `??=` never completes the assignment when the right-hand side throws. + const typed = new TypedResponse<string>( + baseResponse(readableOf('x')), + () => { + calls += 1; + throw new Error('sync boom'); + }, + ); + for (let attempt = 0; attempt < 3; attempt += 1) { + expect(typed.value()).rejects.toThrow('sync boom'); + } + expect(calls).toBe(1); + }); + + test('the same rejected promise is handed back, never a second body read', () => { + const typed = new TypedResponse<string>( + baseResponse(readableOf('x')), + () => { + throw new Error('sync boom'); + }, + ); + const first = typed.value(); + expect(typed.value()).toBe(first); + expect(first).rejects.toThrow('sync boom'); + }); +}); diff --git a/packages/core/src/body/typed-response.ts b/packages/core/src/body/typed-response.ts new file mode 100644 index 0000000..d4d6055 --- /dev/null +++ b/packages/core/src/body/typed-response.ts @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/typed-response.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; + +/** + * A typed view over an HTTP response (HTTP-44). Wraps an underlying raw Response and a parser function, + * materializing and parsing the response value lazily on the first call to `value()`. + * + * Deliberately does NOT expose the underlying `Response` itself (only its status/headers/protocol/ + * reason/request, per HTTP-44) -- doing so would let a caller read the single-use body directly, + * bypassing `value()`'s memoization and the HTTP-45 in-flight-promise serialization entirely. + * + * @public + */ +export class TypedResponse<T> { + readonly #response: Response; + readonly #parse: (response: Response) => Promise<T>; + #memoized: Promise<T> | undefined; + + constructor(response: Response, parse: (response: Response) => Promise<T>) { + this.#response = response; + this.#parse = parse; + } + + /** The response status, carrying HTTP-11's range classification. Never touches the body. */ + get status(): Response['status'] { + return this.#response.status; + } + + /** The response headers. Never touches the body. */ + get headers(): Response['headers'] { + return this.#response.headers; + } + + /** The negotiated protocol as its lower-case wire token, e.g. `http/1.1`. */ + get protocol(): string { + return this.#response.protocol.token; // lower-case token string (Protocol.token) + } + + /** + * The reason phrase as sent, or `undefined` when the transport supplied none -- following + * `Response.reasonPhrase` rather than re-converting absence to `null`. + */ + get reason(): string | undefined { + return this.#response.reasonPhrase; + } + + /** The originating request (HTTP-44). Accessing raw fields never consumes the body. */ + get request(): Request { + return this.#response.request; + } + + /** + * Lazily parses and returns the typed value. Memoized: the parser function runs at most once, and + * subsequent calls return the same parsed value (or re-throw the same error) without re-parsing or + * re-reading the body (HTTP-44). Concurrent first callers share the single in-flight parse (HTTP-45). + * + * @returns the parsed value. + * @throws Whatever the parser raises -- rethrown identically on every later call, never re-parsed. + */ + value(): Promise<T> { + // The `async` wrapper is load-bearing: a parser is typed `=> Promise<T>` but may still be a plain + // function that throws synchronously (validating an argument before the first await is ordinary). + // A bare `this.#memoized ??= this.#parse(...)` never completes the assignment in that case, so the + // handler re-runs on the next call and re-reads a single-use body whose bytes are already gone -- + // exactly what HTTP-44's "without re-running the handler or re-reading the body" forbids. + this.#memoized ??= (async () => this.#parse(this.#response))(); + return this.#memoized; + } +} diff --git a/packages/core/src/body/write-body.test.ts b/packages/core/src/body/write-body.test.ts new file mode 100644 index 0000000..5cb8d33 --- /dev/null +++ b/packages/core/src/body/write-body.test.ts @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/write-body.test.ts +// Exercises: RECOV-12 (a failure inside the writer scope is never masked by the teardown -- the sink is +// aborted, and an abort() that itself rejects does not displace the primary failure), RETRY-2 (the +// primary failure reaches the caller unwrapped so classification can walk its own cause chain) +import {describe, expect, test} from 'bun:test'; +import {rejection} from '../io/test-support/rejection.js'; +import {withBodyWriter} from './write-body.js'; + +interface SinkLog { + readonly chunks: Uint8Array[]; + closed: boolean; + abortReason: unknown; +} + +function recordingSink(overrides: UnderlyingSink<Uint8Array> = {}): { + stream: WritableStream<Uint8Array>; + log: SinkLog; +} { + const log: SinkLog = {chunks: [], closed: false, abortReason: undefined}; + const stream = new WritableStream<Uint8Array>({ + write: chunk => void log.chunks.push(chunk), + close: () => void (log.closed = true), + abort: reason => void (log.abortReason = reason), + ...overrides, + }); + return {stream, log}; +} + +describe('withBodyWriter success path', () => { + test('writes through and closes the sink', async () => { + const {stream, log} = recordingSink(); + + await withBodyWriter(stream, async writer => { + await writer.write(Uint8Array.from([1, 2])); + }); + + expect(log.chunks).toEqual([Uint8Array.from([1, 2])]); + expect(log.closed).toBe(true); + expect(log.abortReason).toBeUndefined(); + }); + + test('a close failure propagates unwrapped (RETRY-2)', async () => { + const {stream} = recordingSink({ + close: () => { + throw new Error('CLOSE FAILED'); + }, + }); + + const error = await rejection( + withBodyWriter(stream, () => Promise.resolve()), + ); + + expect(error.message).toBe('CLOSE FAILED'); + }); +}); + +describe('withBodyWriter failure path (RECOV-12, RETRY-2)', () => { + test('aborts the sink with the primary failure and rethrows it', async () => { + const {stream, log} = recordingSink(); + const primary = new Error('SOCKET GONE'); + + const error = await rejection( + withBodyWriter(stream, () => Promise.reject(primary)), + ); + + expect(error).toBe(primary); + expect(log.abortReason).toBe(primary); + expect(log.closed).toBe(false); + }); + + test('an abort() that itself rejects does not displace the primary failure', async () => { + const {stream} = recordingSink({ + abort: () => { + throw new Error('ABORT FAILED'); + }, + }); + const primary = new Error('SOCKET GONE'); + + const error = await rejection( + withBodyWriter(stream, () => Promise.reject(primary)), + ); + + expect(error).toBe(primary); + }); + + test('aborting an already-errored stream still surfaces the primary failure', async () => { + // The sink's own write() poisons the stream, so abort() runs against a stream that is already + // errored -- the case the naive `finally { close() }` shape turns into a bogus TypeError. + const {stream} = recordingSink({ + write: () => { + throw new Error('SINK EXPLODED'); + }, + }); + + const error = await rejection( + withBodyWriter(stream, async writer => { + await writer.write(Uint8Array.from([1])); + }), + ); + + expect(error.message).toBe('SINK EXPLODED'); + }); +}); diff --git a/packages/core/src/body/write-body.ts b/packages/core/src/body/write-body.ts new file mode 100644 index 0000000..535e320 --- /dev/null +++ b/packages/core/src/body/write-body.ts @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/body/write-body.ts + +/** + * Runs `write` against a fresh writer over `sink`, closing on success and aborting on failure. + * + * The naive shape -- `try { ... } finally { await writer.close(); }` -- is wrong twice over. Closing an + * already-errored writer rejects with a TypeError, and a throwing `finally` *replaces* the in-flight + * exception, so the real "connection died mid-upload" cause is destroyed rather than chained (RECOV-12). + * That is not merely a bad message: RETRY-2 classifies a failure by walking its cause chain, so an I/O + * failure surfacing as a TypeError about closing a stream is silently declassified as non-retryable. + * + * Aborting rather than closing on failure also tells the transport the message is broken; a clean close + * would signal a complete body that was never fully written. + */ +export async function withBodyWriter( + sink: WritableStream<Uint8Array>, + write: (writer: WritableStreamDefaultWriter<Uint8Array>) => Promise<void>, +): Promise<void> { + const writer = sink.getWriter(); + try { + await write(writer); + } catch (error: unknown) { + // Best-effort: abort() resolves on an already-errored stream, and a sink whose own abort() throws + // must not displace the primary failure either. + await writer.abort(error).catch(() => undefined); + throw error; + } + // On the success path a close failure IS the primary failure, so it propagates unwrapped. + await writer.close(); +} diff --git a/packages/core/src/cancellation.test.ts b/packages/core/src/cancellation.test.ts new file mode 100644 index 0000000..8a00439 --- /dev/null +++ b/packages/core/src/cancellation.test.ts @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/cancellation.test.ts +// Exercises: XCUT-1 (cancellation surfaces as one distinct terminal type wherever it was observed), +// XCUT-3 (a cancellation is told apart from a timeout by ambient state, not by a message string). +import {describe, expect, test} from 'bun:test'; +import {abortToSdkError} from './cancellation.js'; +import {TransportFailureError} from './io/errors.js'; +import {CancellationError} from './seams/transport.js'; + +describe('abortToSdkError (XCUT-1, XCUT-3)', () => { + test('a caller abort becomes a CancellationError carrying the reason', () => { + const controller = new AbortController(); + const reason = new Error('caller went away'); + controller.abort(reason); + + const mapped = abortToSdkError(controller.signal, controller.signal.reason); + + expect(mapped).toBeInstanceOf(CancellationError); + expect(mapped.cause).toBe(reason); + }); + + test('a timeout abort becomes a TransportFailureError, never a CancellationError', async () => { + const signal = AbortSignal.timeout(1); + await new Promise(resolve => { + signal.addEventListener('abort', resolve, {once: true}); + }); + + const mapped = abortToSdkError(signal, signal.reason); + + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(mapped).not.toBeInstanceOf(CancellationError); + }); +}); diff --git a/packages/core/src/cancellation.ts b/packages/core/src/cancellation.ts new file mode 100644 index 0000000..c673f93 --- /dev/null +++ b/packages/core/src/cancellation.ts @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/cancellation.ts +import type {DexpaceError} from './http/errors.js'; +import {TransportFailureError} from './io/errors.js'; +import {CancellationError, isTimeoutSignal} from './seams/transport.js'; + +/** + * Maps an aborted signal to this SDK's own terminal type, so `XCUT-1`'s "a distinct, terminal, + * NON-retryable signal" is one type wherever the abort was observed. + * + * Before 2026-09-02 only the transports did this (`@dexpace/transport-shared`'s `abortToSdkError`). + * Core's own cancellable waits -- the retry engine's backoff, the bearer cache's token fetch -- + * surfaced `signal.reason` verbatim, so a caller writing + * `catch (e) { if (e instanceof CancellationError) ... }` handled a cancelled dispatch and silently + * missed a cancelled backoff, which arrived as a bare `DOMException` named `AbortError`. + * + * `XCUT-3` is why this is not unconditionally a `CancellationError`: a cancellation must be + * distinguishable from a timeout, and `AbortSignal.timeout()` aborts with a `TimeoutError` reason. + * + * **Deliberately a second copy** of `@dexpace/transport-shared`'s function of the same name, not a + * shared one. That package peer-depends on core and could only reach this through core's PUBLIC + * barrel; publishing an internal mapper to widen a package boundary is the wrong trade for six + * lines. Both are pinned by tests asserting the same two branches, so a divergence surfaces as a + * failure rather than as silent drift -- the same disposition `docs/work/mvp/2026-09-04-open-items-dissolution.md` K18 records for + * `isHeaderSafe`. + * + * @param signal - the aborted signal. + * @param cause - the original abort reason, kept as the returned error's `cause`. + * @returns `TransportFailureError` when the abort was a timeout, `CancellationError` otherwise. + * + * @internal + */ +export function abortToSdkError( + signal: AbortSignal, + cause: unknown, +): DexpaceError { + return isTimeoutSignal(signal) + ? new TransportFailureError('operation timed out', {cause}) + : new CancellationError('operation cancelled', {cause}); +} diff --git a/packages/core/src/config/build-info.test.ts b/packages/core/src/config/build-info.test.ts new file mode 100644 index 0000000..92c69eb --- /dev/null +++ b/packages/core/src/config/build-info.test.ts @@ -0,0 +1,141 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/build-info.test.ts +// Exercises: CFG-36 (version and runtime identity resolved once at load, each falling back to a +// non-blank "unknown", plus a default ordered [sdkToken, runtimeToken] list with no blank entry), +// NFR-15 (the version is the real compiled-in one, never the placeholder, on any build whose +// codegen step ran). +import {describe, expect, test} from 'bun:test'; +import {SDK_VERSION} from '../generated/version.js'; +import type {RuntimeHost} from './build-info.js'; +import {detectRuntimeIdentity, getBuildInfo} from './build-info.js'; + +describe('getBuildInfo (CFG-36, NFR-15)', () => { + test('reports the generated build-time version, not the placeholder', () => { + // The first assertion pins only that nothing transforms the generated constant on its way into + // the descriptor: it borrows the implementation's own import, so it cannot carry NFR-15 alone. + // The second one does, as does the token-ordering test below, which spells the prefix out. + expect(getBuildInfo().sdkVersion).toBe(SDK_VERSION); + expect(getBuildInfo().sdkVersion).not.toBe('unknown'); + }); + + test('reports a non-blank runtime identity', () => { + expect(getBuildInfo().runtimeIdentity.trim()).not.toBe(''); + }); + + test('identifies this runtime as Node-compatible, since process.version is defined here', () => { + expect(getBuildInfo().runtimeIdentity).toMatch(/^node\//u); + }); + + test('orders identityTokens as [sdkToken, runtimeToken]', () => { + const {identityTokens, sdkVersion, runtimeIdentity} = getBuildInfo(); + + expect(identityTokens).toEqual([ + `dexpace-sdk/${sdkVersion}`, + runtimeIdentity, + ]); + }); + + test('emits no blank identity token', () => { + for (const token of getBuildInfo().identityTokens) { + expect(token.trim()).not.toBe(''); + } + }); + + test('resolves once, returning the same frozen instance on every call', () => { + expect(getBuildInfo()).toBe(getBuildInfo()); + expect(Object.isFrozen(getBuildInfo())).toBe(true); + expect(Object.isFrozen(getBuildInfo().identityTokens)).toBe(true); + }); +}); + +describe('detectRuntimeIdentity (CFG-36)', () => { + test('reports a Node-style token when process.version is present', () => { + const host: RuntimeHost = {process: {version: 'v20.11.0'}}; + + expect(detectRuntimeIdentity(host)).toBe('node/20.11.0'); + }); + + test('prefers process over Deno and navigator', () => { + const host: RuntimeHost = { + process: {version: 'v20.11.0'}, + Deno: {version: {deno: '1.44.0'}}, + navigator: {userAgent: 'Mozilla/5.0'}, + }; + + expect(detectRuntimeIdentity(host)).toBe('node/20.11.0'); + }); + + test('reports a Deno-style token when only Deno is present', () => { + const host: RuntimeHost = {Deno: {version: {deno: '1.44.0'}}}; + + expect(detectRuntimeIdentity(host)).toBe('deno/1.44.0'); + }); + + test('reports the user agent when only navigator is present', () => { + const host: RuntimeHost = {navigator: {userAgent: 'Mozilla/5.0'}}; + + expect(detectRuntimeIdentity(host)).toBe('Mozilla/5.0'); + }); + + test('falls back to the non-blank literal "unknown" when nothing is detectable', () => { + expect(detectRuntimeIdentity({})).toBe('unknown'); + }); + + test('treats a blank detected value as undetectable rather than emitting a blank token', () => { + const host: RuntimeHost = { + process: {version: ' '}, + navigator: {userAgent: ''}, + }; + + expect(detectRuntimeIdentity(host)).toBe('unknown'); + }); + + test('ignores a non-string detected value', () => { + const host: RuntimeHost = {process: {version: 20}}; + + expect(detectRuntimeIdentity(host)).toBe('unknown'); + }); +}); + +describe('detectRuntimeIdentity sanitization (CFG-36, RECOV-33)', () => { + test('trims a detected value rather than carrying its surrounding whitespace into the token', () => { + const host: RuntimeHost = {process: {version: ' v20.11.0 '}}; + + expect(detectRuntimeIdentity(host)).toBe('node/20.11.0'); + }); + + test('treats a version that strips to nothing as undetectable', () => { + // `'v'` alone leaves `node/` with no version behind it, which is a worse answer than saying so. + const host: RuntimeHost = {process: {version: 'v'}}; + + expect(detectRuntimeIdentity(host)).toBe('unknown'); + }); + + test('rejects a detected value that is not header-safe', () => { + // The value is ambient and unvalidated at its source, and RECOV-33 puts it straight into a + // header. One non-ASCII byte in `navigator.userAgent` used to make the default client-identity + // step reject every outbound request with a HeaderValidationError. + for (const host of [ + {navigator: {userAgent: 'Mozilla/5.0 (caf\u00e9)'}}, + {process: {version: 'v1.0\r\nX-Injected: yes'}}, + {Deno: {version: {deno: '1.0\nX: y'}}}, + {navigator: {userAgent: 'Mozilla/5.0 \u007f'}}, + ] satisfies RuntimeHost[]) { + expect(detectRuntimeIdentity(host)).toBe('unknown'); + } + }); + + test('emits only header-safe characters for every host shape it accepts', () => { + const headerSafe = /^[\t\u0020-\u007e]+$/u; + + for (const host of [ + {}, + {process: {version: 'v20.11.0'}}, + {Deno: {version: {deno: '1.44.0'}}}, + {navigator: {userAgent: 'Mozilla/5.0'}}, + {navigator: {userAgent: 'Mozilla/5.0 (caf\u00e9)'}}, + ] satisfies RuntimeHost[]) { + expect(detectRuntimeIdentity(host)).toMatch(headerSafe); + } + }); +}); diff --git a/packages/core/src/config/build-info.ts b/packages/core/src/config/build-info.ts new file mode 100644 index 0000000..86e0d60 --- /dev/null +++ b/packages/core/src/config/build-info.ts @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/build-info.ts +import {SDK_VERSION} from '../generated/version.js'; + +/** + * The build/runtime identity descriptor (CFG-36): the SDK's own version, the host runtime it is + * executing on, and the ordered token list a `User-Agent`-style header composes from. Every field is + * non-blank -- an undetectable value reads `unknown`, never an empty string. + * + * @public + */ +export interface BuildInfo { + /** This SDK's own published version, compiled in at build time; `'unknown'` if it was not. */ + readonly sdkVersion: string; + /** The host runtime, e.g. `'node/20.11.0'`; `'unknown'` when undetectable. */ + readonly runtimeIdentity: string; + /** `[sdkToken, runtimeToken]`, in that order; every entry is non-blank. */ + readonly identityTokens: readonly string[]; +} + +/** + * The shapes this module feature-detects on `globalThis`; none of them is imported, so the same + * source compiles and runs on every runtime in core's floor. + * + * @internal + */ +export interface RuntimeHost { + readonly process?: {readonly version?: unknown}; + readonly Deno?: {readonly version?: {readonly deno?: unknown}}; + readonly navigator?: {readonly userAgent?: unknown}; +} + +/** + * Printable ASCII plus HTAB -- the outbound header value grammar's character class, restated here. + * + * Deliberately *not* `hasForbiddenOutboundValueByte` from `http/ascii-validation.js`. `config/`'s + * outbound edges are already a live concern (`docs/work/mvp/2026-09-04-open-items-dissolution.md` K11), and adding a second one to + * reuse a four-line predicate is the wrong trade. `docs/work/mvp/2026-09-04-open-items-dissolution.md` K18 owns the duplication and + * names this as one of the call sites a consolidation would fold in. + */ +function isHeaderSafe(value: string): boolean { + for (const character of value) { + const code = character.codePointAt(0) ?? 0; + if (code !== 0x09 && (code < 0x20 || code > 0x7e)) return false; + } + return true; +} + +/** + * A detected value fit to become an identity token: trimmed, non-blank, and header-safe. + * + * Both halves earn their place (CFG-36, RECOV-33, NFR-15). The value is ambient -- `process.version`, + * `Deno.version.deno`, `navigator.userAgent` -- and nothing guarantees it is ASCII or tidy. It used + * to be returned *untrimmed* despite the blank test trimming, so `' v20.0.0 '` became + * `node/ v20.0.0 ` and the `^v` strip silently missed; and it was never validated, so a + * `navigator.userAgent` carrying one non-ASCII byte made the default `clientIdentityStep` reject + * every outbound request with a `HeaderValidationError`. An unusable value is undetectable, not + * fatal: the caller falls back to `unknown`, exactly as it does for an absent one. + */ +function toUsableToken(value: unknown): string | null { + if (typeof value !== 'string') return null; + const trimmed = value.trim(); + return trimmed !== '' && isHeaderSafe(trimmed) ? trimmed : null; +} + +/** + * Feature-detected, never throwing (CFG-36). Node and Bun report through `process`, Deno through + * `Deno.version`, browsers and Workers through `navigator.userAgent`; anything else falls back to + * the literal `unknown`, matching CFG-36's own "falls back to a non-blank `unknown`" wording. + * + * Takes the host explicitly rather than reading `globalThis` inline: the branches this phase cannot + * execute on its own runners are then reachable from a test without deleting a global, which no test + * may do (`docs/knowledge/harvested/testing.md:50` -- tests must survive parallel execution). + * + * @param host - the ambient global object to interrogate. + * @returns a non-blank runtime identity token. + * + * @internal + */ +export function detectRuntimeIdentity(host: RuntimeHost): string { + const nodeVersion = toUsableToken(host.process?.version); + // A `process.version` of exactly `'v'` strips to nothing; `node/` with no version behind it is a + // worse answer than saying so, and falling through gives the remaining probes their chance. + const stripped = nodeVersion?.replace(/^v/u, '') ?? ''; + if (stripped !== '') return `node/${stripped}`; + + const denoVersion = toUsableToken(host.Deno?.version?.deno); + if (denoVersion !== null) return `deno/${denoVersion}`; + + const userAgent = toUsableToken(host.navigator?.userAgent); + if (userAgent !== null) return userAgent; + + return 'unknown'; +} + +function resolveBuildInfo(): BuildInfo { + const sdkVersion = toUsableToken(SDK_VERSION) ?? 'unknown'; + const runtimeIdentity = detectRuntimeIdentity(globalThis); + return Object.freeze({ + sdkVersion, + runtimeIdentity, + identityTokens: Object.freeze([ + `dexpace-sdk/${sdkVersion}`, + runtimeIdentity, + ]), + }); +} + +/** + * Module-level mutable state, which `docs/knowledge/harvested/variables-and-declarations.md:22` bans outright. + * Deliberate: CFG-36's descriptor is resolved once per process, and the alternative -- re-running the + * feature detection per request -- is the cost the memo exists to avoid. Safe against the rule's + * stated hazard because `resolveBuildInfo` is deterministic within a process, so no test can observe + * a different value depending on which test ran first. There is deliberately no reset hook: nothing + * needs one, and adding one would publish a way to make the descriptor lie. + */ +let cachedBuildInfo: BuildInfo | undefined; + +/** + * The process-wide build/runtime descriptor (CFG-36), resolved on first access and cached thereafter + * so the runtime detection runs once rather than per request. + * + * @returns the frozen descriptor; the same instance on every call. + * + * @public + */ +export function getBuildInfo(): BuildInfo { + cachedBuildInfo ??= resolveBuildInfo(); + return cachedBuildInfo; +} diff --git a/packages/core/src/config/client-identity-step.test.ts b/packages/core/src/config/client-identity-step.test.ts new file mode 100644 index 0000000..2d789a1 --- /dev/null +++ b/packages/core/src/config/client-identity-step.test.ts @@ -0,0 +1,318 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/client-identity-step.test.ts +// Exercises: RECOV-33 (Append mode joins tokens with single spaces and composes onto the FIRST +// existing value while preserving every other pre-existing value; an empty first value is treated as +// absent so no leading space is emitted; Replace mode overwrites; an empty or blank-joining token +// list is a no-op that never emits a blank or whitespace-only header), NFR-15 (the default tokens +// carry the real compiled-in version). +import {describe, expect, test} from 'bun:test'; +import {HeaderValidationError} from '../http/errors.js'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import type {StepContext} from '../pipeline/step.js'; +import {detectRuntimeIdentity, getBuildInfo} from './build-info.js'; +import {clientIdentityStep} from './client-identity-step.js'; + +function requestWith(headers?: Headers): Request { + const builder = Request.newBuilder().url('https://example.com'); + return headers === undefined + ? builder.build() + : builder.headers(headers).build(); +} + +/** What one drive of the step observed. */ +interface StepRun { + /** What the step handed to `next`; `undefined` if it never called `next`, or called it with nothing. */ + readonly forwarded: Request | undefined; + readonly nextCalls: number; + readonly rejection: unknown; +} + +/** + * Drives the step with a hand-rolled `next` that records what it was handed, counts its invocations, + * and answers with a minimal 200 -- so a test can read the request the step actually forwarded and + * assert the negative space (that a rejecting step forwarded nothing at all). + * + * `next` takes a **required** `Request`, though `StepContext`'s `Next` declares the parameter + * optional. Deliberate: with an optional parameter and a `?? request` fallback, a step that called + * `ctx.next()` -- forwarding nothing, which is a legal step idiom -- was indistinguishable from one + * that forwarded the original instance, and the no-op test below could not tell them apart. + */ +async function driveStep( + descriptor: ReturnType<typeof clientIdentityStep>, + request: Request = requestWith(), +): Promise<StepRun> { + let forwarded: Request | undefined; + let nextCalls = 0; + const context = { + next: (handed: Request): Promise<Response> => { + forwarded = handed; + nextCalls += 1; + return Promise.resolve( + Response.newBuilder() + .request(handed) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .build(), + ); + }, + // `StepContext` also carries `context` and an optional `fork`; this step reads neither, so the + // fake declares only `next` and the widening bridges what is missing. + } as unknown as StepContext; + + try { + await descriptor.fn(request, context); + return {forwarded, nextCalls, rejection: undefined}; + } catch (reason: unknown) { + return {forwarded, nextCalls, rejection: reason}; + } +} + +/** The request the step forwarded, failing loudly if it rejected or forwarded nothing. */ +async function forwardedRequest( + descriptor: ReturnType<typeof clientIdentityStep>, + request?: Request, +): Promise<Request> { + const {forwarded, rejection} = await driveStep(descriptor, request); + expect(rejection).toBeUndefined(); + if (forwarded === undefined) { + throw new Error( + 'expected the step to forward a request, but it forwarded nothing', + ); + } + return forwarded; +} + +describe('clientIdentityStep on the failure path', () => { + test('rejects with a HeaderValidationError when a token is not header-safe', async () => { + const {rejection} = await driveStep( + clientIdentityStep({tokens: ['sdk/1.0\nX-Injected: evil']}), + ); + + expect(rejection).toBeInstanceOf(HeaderValidationError); + }); + + test('forwards nothing when composition fails', async () => { + const original = requestWith( + Headers.newBuilder().add('User-Agent', 'original').build(), + ); + + const {nextCalls} = await driveStep( + clientIdentityStep({tokens: ['sdk/1.0\nX-Injected: evil']}), + original, + ); + + expect(nextCalls).toBe(0); + }); + + test('leaves the inbound request untouched when composition fails', async () => { + const original = requestWith( + Headers.newBuilder().add('User-Agent', 'original').build(), + ); + + await driveStep( + clientIdentityStep({tokens: ['sdk/1.0\nX-Injected: evil']}), + original, + ); + + expect(original.headers.getAll('User-Agent')).toEqual(['original']); + }); + + test('composes cleanly from a runtime identity that could not be detected', async () => { + // The end-to-end shape of the build-info guard: an ambient value carrying a non-ASCII byte + // resolves to `unknown` at its source, so the step still emits a legal header rather than + // failing every request that passes through it. + const runtimeIdentity = detectRuntimeIdentity({ + navigator: {userAgent: 'Mozilla/5.0 (caf\u00e9)'}, + }); + + const forwarded = await forwardedRequest( + clientIdentityStep({tokens: ['dexpace-sdk/1.2.3', runtimeIdentity]}), + ); + + expect(forwarded.headers.getAll('User-Agent')).toEqual([ + 'dexpace-sdk/1.2.3 unknown', + ]); + }); +}); + +describe('clientIdentityStep placement', () => { + test('occupies the outermost non-pillar slot', () => { + expect(clientIdentityStep().stage).toBe('PRE_REDIRECT'); + }); + + test('carries a stable identity symbol across instances', () => { + expect(clientIdentityStep().type).toBe( + clientIdentityStep({tokens: ['x']}).type, + ); + }); +}); + +describe('clientIdentityStep append mode (RECOV-33)', () => { + test('sets the joined token line as the sole value when the header is absent', async () => { + const step = clientIdentityStep({tokens: ['sdk/1.0', 'node/20']}); + + const result = await forwardedRequest(step); + + expect(result.headers.getAll('User-Agent')).toEqual(['sdk/1.0 node/20']); + }); + + test('composes onto the first existing value and preserves every other value', async () => { + const headers = Headers.newBuilder() + .add('User-Agent', 'existing-agent') + .add('User-Agent', 'second') + .build(); + const step = clientIdentityStep({tokens: ['sdk/1.0']}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.getAll('User-Agent')).toEqual([ + 'existing-agent sdk/1.0', + 'second', + ]); + }); + + test('emits no leading space when the first existing value is empty', async () => { + const headers = Headers.newBuilder().add('User-Agent', '').build(); + const step = clientIdentityStep({tokens: ['sdk/1.0']}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.get('User-Agent')).toBe('sdk/1.0'); + }); +}); + +describe('clientIdentityStep append mode ordering (RECOV-33)', () => { + test('keeps the header in its original position among the other headers', async () => { + const headers = Headers.newBuilder() + .add('Accept', 'application/json') + .add('User-Agent', 'existing-agent') + .add('X-Trailing', 'yes') + .build(); + const step = clientIdentityStep({tokens: ['sdk/1.0']}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.names()).toEqual([ + 'Accept', + 'User-Agent', + 'X-Trailing', + ]); + }); + + test('matches the header name case-insensitively rather than adding a second one', async () => { + const headers = Headers.newBuilder() + .add('user-agent', 'existing-agent') + .build(); + const step = clientIdentityStep({tokens: ['sdk/1.0']}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.getAll('User-Agent')).toEqual([ + 'existing-agent sdk/1.0', + ]); + }); + + test('leaves other headers untouched', async () => { + const headers = Headers.newBuilder() + .add('Accept', 'application/json') + .build(); + const step = clientIdentityStep({tokens: ['sdk/1.0']}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.get('Accept')).toBe('application/json'); + }); +}); + +describe('clientIdentityStep replace mode (RECOV-33)', () => { + test('overwrites every existing value', async () => { + const headers = Headers.newBuilder() + .add('User-Agent', 'existing-agent') + .add('User-Agent', 'second') + .build(); + const step = clientIdentityStep({tokens: ['sdk/1.0'], mode: 'replace'}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.getAll('User-Agent')).toEqual(['sdk/1.0']); + }); +}); + +describe('clientIdentityStep no-op cases (RECOV-33)', () => { + test('emits no header for an empty token list', async () => { + const step = clientIdentityStep({tokens: []}); + + const result = await forwardedRequest(step); + + expect(result.headers.get('User-Agent')).toBeUndefined(); + }); + + test('emits no header when the tokens join to a blank line', async () => { + const step = clientIdentityStep({tokens: ['', ' ']}); + + const result = await forwardedRequest(step); + + expect(result.headers.get('User-Agent')).toBeUndefined(); + }); + + test('leaves an existing header untouched when the token list is blank', async () => { + const headers = Headers.newBuilder() + .add('User-Agent', 'existing-agent') + .build(); + const step = clientIdentityStep({tokens: []}); + + const result = await forwardedRequest(step, requestWith(headers)); + + expect(result.headers.getAll('User-Agent')).toEqual(['existing-agent']); + }); + + test('forwards the original request instance when it is a no-op', async () => { + const request = requestWith(); + const step = clientIdentityStep({tokens: []}); + + // Read straight off `driveStep`, whose `next` records exactly what it was handed. Going through + // `forwardedRequest` would be just as strict now, but this states the claim at its narrowest: + // the step passed *this instance*, not merely something that behaves like it. + const {forwarded} = await driveStep(step, request); + + expect(forwarded).toBe(request); + }); +}); + +describe('clientIdentityStep configuration', () => { + test('writes a caller-chosen header for a second identity line', async () => { + const step = clientIdentityStep({ + headerName: 'X-Client-Info', + tokens: ['app/2.0'], + }); + + const result = await forwardedRequest(step); + + expect(result.headers.get('X-Client-Info')).toBe('app/2.0'); + expect(result.headers.get('User-Agent')).toBeUndefined(); + }); + + test('defaults to User-Agent carrying the build and runtime identity tokens (NFR-15)', async () => { + const step = clientIdentityStep(); + + const result = await forwardedRequest(step); + + expect(result.headers.get('User-Agent')).toBe( + getBuildInfo().identityTokens.join(' '), + ); + }); + + test('does not alias the caller settings object after construction', async () => { + const tokens = ['sdk/1.0']; + const step = clientIdentityStep({tokens}); + + tokens.push('sneaked/9.9'); + const result = await forwardedRequest(step); + + expect(result.headers.get('User-Agent')).toBe('sdk/1.0'); + }); +}); diff --git a/packages/core/src/config/client-identity-step.ts b/packages/core/src/config/client-identity-step.ts new file mode 100644 index 0000000..21a1c1b --- /dev/null +++ b/packages/core/src/config/client-identity-step.ts @@ -0,0 +1,141 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/client-identity-step.ts +import type {Headers} from '../http/headers.js'; +import type {Request} from '../http/request.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {getBuildInfo} from './build-info.js'; + +/** + * How {@link clientIdentityStep} composes its tokens into the target header (RECOV-33). + * + * @public + */ +export interface ClientIdentitySettings { + /** + * The header to write. + * + * @defaultValue `'User-Agent'` + */ + readonly headerName?: string | undefined; + + /** + * The tokens to compose, joined with single spaces. + * + * @defaultValue `getBuildInfo().identityTokens` + */ + readonly tokens?: readonly string[] | undefined; + + /** + * `'append'` composes after the first existing value; `'replace'` overwrites every existing value. + * + * @defaultValue `'append'` + */ + readonly mode?: 'append' | 'replace' | undefined; +} + +/** Identity for pipeline anchoring; a fresh symbol per module, not per call. */ +const CLIENT_IDENTITY_STEP_TYPE = Symbol('dexpace.client-identity'); + +/** + * RECOV-33's Append composition: the token line joins onto the FIRST existing value, and an empty + * first value counts as absent so no leading space is emitted. + * + * Its own function despite having one caller: the empty-first-value rule is RECOV-33's, not an + * implementation detail of {@link composeHeaders}, and it earns a name and this note rather than an + * unexplained ternary inside the write. + */ +function composeFirstValue(existingFirst: string, tokenLine: string): string { + return existingFirst === '' ? tokenLine : `${existingFirst} ${tokenLine}`; +} + +/** + * The header name and mode {@link composeHeaders} needs, after {@link clientIdentityStep} has applied + * its defaults. One object rather than two more parameters, so `composeHeaders` stays inside the + * three-parameter cap (`docs/knowledge/harvested/function-design.md:22`). + */ +interface ResolvedComposition { + readonly headerName: string; + readonly mode: 'append' | 'replace'; +} + +/** + * Writes the composed header, preserving every pre-existing value RECOV-33 says must survive. + * + * `HeadersBuilder` offers only `set` (replace the whole value list) and `add` (append one more), so + * Append mode rewrites the list explicitly: `set` the composed first value -- which keeps the + * header's position in insertion order, unlike a remove-then-re-add -- then `add` each remaining + * original value back in order. Replace mode legitimately overwrites everything, so a plain `set` + * is the whole of it. + */ +function composeHeaders( + headers: Headers, + composition: ResolvedComposition, + tokenLine: string, +): Headers { + const {headerName, mode} = composition; + const builder = headers.newBuilder(); + const existing = mode === 'replace' ? [] : headers.getAll(headerName); + if (existing.length === 0) return builder.set(headerName, tokenLine).build(); + + // The `''` fallback is unreachable -- the guard above returned for an empty list -- and exists + // only because `noUncheckedIndexedAccess` cannot see that. The *reachable* empty-string case is a + // header whose first value really is empty, which `composeFirstValue` owns. + builder.set(headerName, composeFirstValue(existing[0] ?? '', tokenLine)); + for (const value of existing.slice(1)) builder.add(headerName, value); + return builder.build(); +} + +/** + * Builds the client-identity pipeline step (RECOV-33), which stamps the SDK's build and runtime + * identity onto every outbound request and so closes NFR-15's "report the real version" clause. + * + * Append mode (the default) joins the tokens with single spaces and appends them after the first + * existing header value, preserving every other pre-existing value untouched, or sets them as the + * sole value when the header is absent. Replace mode overwrites every existing value. A token list + * that is empty or joins to a blank line makes the step a no-op: it never emits a blank or + * whitespace-only header. + * + * Not a pillar step. It occupies `PRE_REDIRECT`, the outermost user-extensible slot, so it runs once + * per top-level call rather than once per redirect or retry attempt, and it is not installed by any + * preset -- a caller adds it to their own pipeline. + * + * @param settings - header name, tokens, and composition mode; every field is optional. + * @returns the step descriptor to install. + * @throws HeaderValidationError -- as a rejected promise -- when the composed value or the header + * name is not legal on the outbound path. + * + * @public + */ +export function clientIdentityStep( + settings: ClientIdentitySettings = {}, +): StepDescriptor { + const headerName = settings.headerName ?? 'User-Agent'; + const mode = settings.mode ?? 'append'; + // Copied and frozen here, never aliased: a caller that keeps its own array must not be able to + // rewrite what an already-installed step emits (the same defensive-copy discipline HTTP-3 puts on + // every model builder). + const tokens = + settings.tokens === undefined + ? undefined + : Object.freeze([...settings.tokens]); + + return { + type: CLIENT_IDENTITY_STEP_TYPE, + stage: 'PRE_REDIRECT', + fn: async (request: Request, ctx) => { + // The default resolves per invocation rather than per install, so a step built before anything + // has touched `getBuildInfo()` still stamps the resolved descriptor. + const tokenLine = (tokens ?? getBuildInfo().identityTokens) + .join(' ') + .trim(); + if (tokenLine === '') return ctx.next(request); + + const headers = composeHeaders( + request.headers, + {headerName, mode}, + tokenLine, + ); + return ctx.next(request.newBuilder().headers(headers).build()); + }, + }; +} diff --git a/packages/core/src/config/clock.test.ts b/packages/core/src/config/clock.test.ts new file mode 100644 index 0000000..f7d81a9 --- /dev/null +++ b/packages/core/src/config/clock.test.ts @@ -0,0 +1,244 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/clock.test.ts +// Exercises: CFG-15 (three operations, shared platform-backed default), CFG-16 (monotonic +// non-decreasing, meaningful only relative to itself), CFG-17 (sleep rejects negative, resolves +// promptly at zero, honors cancellation), and V13 (any finite duration is honored by chaining timers, +// so RETRY-18's 365-day pacing clamp -- ~14x what one setTimeout can carry -- is waitable). +import {describe, expect, test} from 'bun:test'; +import {CancellationError} from '../seams/transport.js'; +import {MAX_SLEEP_MS, defaultClock, sleepInChunks} from './clock.js'; + +/** + * Returns the reason a promise rejected with, failing loudly if it resolves instead. Awaiting the + * promise directly (rather than `expect(...).rejects`) keeps the rejection reason available for + * identity assertions -- CFG-17 is specifically about *which* value surfaces, not merely that one + * does. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (reason: unknown) { + return reason; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +describe('defaultClock.now (CFG-15, CFG-16)', () => { + test('returns a wall-clock epoch millisecond value bracketed by two Date.now readings', () => { + const before = Date.now(); + + const value = defaultClock.now(); + + expect(value).toBeGreaterThanOrEqual(before); + expect(value).toBeLessThanOrEqual(Date.now()); + }); +}); + +describe('defaultClock.monotonic (CFG-16)', () => { + test('is non-decreasing across two readings', () => { + const first = defaultClock.monotonic(); + + const second = defaultClock.monotonic(); + + expect(second).toBeGreaterThanOrEqual(first); + }); + + test('draws on a different source from now rather than the wall clock under another name', () => { + // CFG-16's whole point: `now` MAY step backwards, `monotonic` MUST NOT, which is why elapsed-time + // math rides `monotonic`. Wiring `monotonic` to `Date.now()` satisfies every other assertion in + // this file, so the distinction needs its own test. + // Read with `Reflect.get` rather than `defaultClock.monotonic`: a bare method reference trips + // `@typescript-eslint/unbound-method`, and the fact being asserted is about the two property + // *values*, not about calling either of them. + expect(Reflect.get(defaultClock, 'monotonic')).not.toBe( + Reflect.get(defaultClock, 'now'), + ); + }); + + test('reads on a process-relative scale, not an epoch scale', () => { + // The second, independent half: an implementation could still return a wall-clock reading from a + // distinct function. `performance.now()` is milliseconds since process start; `Date.now()` is a + // ~1.7e12 epoch offset. Anything within 1e9 ms (~11.6 days) of the epoch value is the wall clock. + const separation = Math.abs(defaultClock.monotonic() - defaultClock.now()); + + expect(separation).toBeGreaterThan(1e9); + }); +}); + +describe('defaultClock.sleep (CFG-17)', () => { + test('resolves promptly when the duration is zero', async () => { + const start = defaultClock.monotonic(); + + await defaultClock.sleep(0); + + expect(defaultClock.monotonic() - start).toBeLessThan(50); + }); + + test('waits at least the requested duration when not cancelled', async () => { + const start = defaultClock.monotonic(); + + await defaultClock.sleep(20); + + expect(defaultClock.monotonic() - start).toBeGreaterThanOrEqual(15); + }); + + test("rejects with the signal's own abort reason when the signal is already aborted", async () => { + const controller = new AbortController(); + const reason = new Error('cancelled'); + controller.abort(reason); + + const pending = defaultClock.sleep(60_000, controller.signal); + + const surfaced = await rejectionOf(pending); + expect(surfaced).toBeInstanceOf(CancellationError); + expect((surfaced as Error).cause).toBe(reason); + }); + + test('rejects with a CancellationError carrying the reason when cancelled mid-wait', async () => { + // Mapped rather than rethrown verbatim (N1/XCUT-1): one cancellation type wherever the abort was + // observed, with the caller's own reason kept as `cause`, so nothing is lost. CFG-17's + // "re-assert the cancellation status" clause is about the STATUS, not the object -- and + // `AbortSignal.aborted` is latched, so a downstream handler sees the cancelled state either way. + const controller = new AbortController(); + const reason = new Error('cancelled'); + const start = defaultClock.monotonic(); + + const pending = defaultClock.sleep(60_000, controller.signal); + queueMicrotask(() => { + controller.abort(reason); + }); + + const surfaced = await rejectionOf(pending); + expect(surfaced).toBeInstanceOf(CancellationError); + expect((surfaced as Error).cause).toBe(reason); + expect(defaultClock.monotonic() - start).toBeLessThan(50); + }); + + test('an already-aborted signal short-circuits before any timer is scheduled', async () => { + const controller = new AbortController(); + controller.abort(new Error('cancelled')); + const start = defaultClock.monotonic(); + + expect( + await rejectionOf(defaultClock.sleep(60_000, controller.signal)), + ).toBeDefined(); + + expect(defaultClock.monotonic() - start).toBeLessThan(50); + }); +}); + +describe('defaultClock.sleep duration bounds and scheduling (CFG-17, V13)', () => { + test('rejects a negative duration with a RangeError, as a rejection not a throw', async () => { + const reason = await rejectionOf(defaultClock.sleep(-1)); + + expect(reason).toBeInstanceOf(RangeError); + }); + + test('rejects a non-finite duration', async () => { + expect( + await rejectionOf(defaultClock.sleep(Number.POSITIVE_INFINITY)), + ).toBeInstanceOf(RangeError); + expect(await rejectionOf(defaultClock.sleep(Number.NaN))).toBeInstanceOf( + RangeError, + ); + }); + + test("a duration past one timer's reach is CHUNKED, never clamped to 1ms (V13)", async () => { + // The defect this replaces: `setTimeout` silently rewrites a delay above 2^31-1 to `1`, so + // `sleep(2 ** 31)` returned in ~1ms instead of waiting 24.8 days. Phase 7a repaired that by + // REJECTING the duration, which also made RETRY-18's 365-day pacing clamp unwaitable. + // + // Asserted through the injected chunk rather than by waiting: `2 ** 31` against a 1ms chunk + // would schedule two billion timers, so the scheduler count is checked with a chunk that + // divides a small duration. What this pins is that the SLICE handed to any single timer never + // exceeds the chunk -- which is exactly what stops the platform clamping. + const slices: number[] = []; + + await sleepInChunks(4, undefined, { + chunkMs: 1, + onChunk: sliceMs => slices.push(sliceMs), + }); + + expect(slices).toEqual([1, 1, 1, 1]); + }); + + test('3x the chunk plus one schedules four timers and resolves (V13)', async () => { + const slices: number[] = []; + + await sleepInChunks(3 * 2 + 1, undefined, { + chunkMs: 2, + onChunk: sliceMs => slices.push(sliceMs), + }); + + expect(slices).toHaveLength(4); + expect(slices).toEqual([2, 2, 2, 1]); + expect(slices.reduce((a, b) => a + b, 0)).toBe(7); + }); + + test('every slice is bounded by MAX_SLEEP_MS in production, with no chunking injected', () => { + // The production chunk is the platform's own limit, so a real oversized sleep slices at exactly + // the largest delay a timer can carry rather than at some smaller invented number. + expect(MAX_SLEEP_MS).toBe(2 ** 31 - 1); + }); +}); + +describe('defaultClock.sleep chunk-boundary cancellation (CFG-17, V13)', () => { + test('an abort BETWEEN chunks rejects with the mapped CancellationError', async () => { + const controller = new AbortController(); + const reason = new Error('gave up mid-wait'); + const slices: number[] = []; + + const pending = sleepInChunks(10, controller.signal, { + chunkMs: 1, + onChunk: sliceMs => { + slices.push(sliceMs); + if (slices.length === 3) controller.abort(reason); + }, + }); + + const surfaced = await rejectionOf(pending); + expect(surfaced).toBeInstanceOf(CancellationError); + expect((surfaced as Error).cause).toBe(reason); + // Stopped at the boundary rather than running all ten slices. + expect(slices.length).toBeLessThan(10); + }); + + test('a duration at exactly one chunk still takes a single timer', async () => { + const slices: number[] = []; + + await sleepInChunks(2, undefined, { + chunkMs: 2, + onChunk: sliceMs => slices.push(sliceMs), + }); + + expect(slices).toEqual([2]); + }); + + test('yields to the event loop at zero rather than only to the microtask queue', async () => { + // A `Promise.resolve()` short-circuit satisfies "returns promptly" while starving timers and + // I/O: a zero-backoff retry loop spun millions of times without ever letting a pending + // `setTimeout(fn, 0)` run. + let timerRan = false; + setTimeout(() => { + timerRan = true; + }, 0); + + await defaultClock.sleep(0); + + expect(timerRan).toBe(true); + }); + + test('honors an already-aborted signal ahead of the zero-duration path', async () => { + // The aborted check precedes the duration path, so cancellation wins even where the wait would + // have been instantaneous anyway (CFG-17). + const controller = new AbortController(); + const reason = new Error('cancelled'); + controller.abort(reason); + + const surfaced = await rejectionOf( + defaultClock.sleep(0, controller.signal), + ); + expect(surfaced).toBeInstanceOf(CancellationError); + expect((surfaced as Error).cause).toBe(reason); + }); +}); diff --git a/packages/core/src/config/clock.ts b/packages/core/src/config/clock.ts new file mode 100644 index 0000000..b2c1b6d --- /dev/null +++ b/packages/core/src/config/clock.ts @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/clock.ts +import {abortToSdkError} from '../cancellation.js'; + +/** + * An injectable seam for wall-clock instant, monotonic elapsed-time measurement, and a cancellable + * wait (CFG-15). Time-dependent logic routes through this seam so tests drive time deterministically + * instead of depending on real elapsed wall time. + * + * One primitive covers the reference's blocking-sleep/scheduled-async-delay pair (CFG-15/CFG-17 vs. + * CFG-18): Node has no carrier threads to distinguish "block this one" from "schedule that one" + * against, and every timer is already non-blocking. Recorded in the phase's Deviation Ledger. + * + * @public + */ +export interface Clock { + /** + * Wall-clock epoch milliseconds. MAY move backwards (clock adjustment, NTP step) and MUST NOT be + * used for elapsed-time math -- use {@link Clock.monotonic} for that (CFG-16). + */ + now(): number; + + /** + * A non-decreasing counter for measuring elapsed durations (CFG-16). Only the difference between + * two readings of the same clock is meaningful; the absolute value is not. + */ + monotonic(): number; + + /** + * Resolves after the requested delay, or rejects if `signal` fires first (CFG-17). + * + * `0` returns promptly as CFG-17 requires, but on the next turn of the *event loop* rather than the + * next microtask: a `Promise.resolve()` short-circuit satisfies "promptly" while starving timers + * and I/O, so a retry loop with a zero backoff spun 4.1 million times in 300ms without letting a + * pending `setTimeout(fn, 0)` run once. + * + * @remarks + * **Any finite, non-negative duration is honored**, including one longer than a single + * `setTimeout` delay can represent. `setTimeout` clamps a delay above 2^31 - 1 ms and *silently* + * rewrites it to `1`, so the default implementation chains timers in chunks rather than issuing + * one oversized delay. That matters because `RETRY-18`/`RECOV-26` mandate clamping a server pacing + * hint to a 365-day ceiling — roughly fourteen times what one timer can carry — so a conformant + * retry must be able to wait longer than one timer allows. + * + * `Clock` is a seam a consumer may implement. **A custom implementation must honor long durations + * too**: passing `durationMs` straight to `setTimeout` reintroduces the silent clamp, which turns + * an overflowed backoff into no backoff at all and a hot loop against the upstream. + * + * Cancellation is checked before the first timer and again between chunks, so a long wait aborts + * promptly rather than at the next chunk boundary only. The abort path clears the pending timer; + * the resolve path detaches the abort listener. Neither a pending timer nor a live listener + * outlives the wait. + * + * @param durationMs - how long to wait; any finite, non-negative number. + * @param signal - the caller's cancellation signal, if any. + * @returns a promise resolving when the full duration has elapsed. + * @throws CancellationError when `signal` aborts, carrying the caller's own abort reason as + * `cause` — or `TransportFailureError` when the abort was a timeout, so a cancellation stays + * distinguishable from a timeout (XCUT-1, XCUT-3). CFG-17's "re-assert the cancellation status" + * clause is satisfied structurally: `AbortSignal.aborted` is latched, so a downstream handler + * observes the cancelled state whatever object is thrown. + * @throws RangeError as a rejected promise, never synchronously, when `durationMs` is negative or + * not a finite number. + */ + sleep(durationMs: number, signal?: AbortSignal): Promise<void>; +} + +/** + * The longest delay ONE `setTimeout` call can carry. Above this the platform clamps to a 32-bit + * signed integer and *silently* rewrites the delay to `1` (Node prints a `TimeoutOverflowWarning` on + * stderr and continues), so a single `setTimeout(fn, 2 ** 31)` fires in about a millisecond instead + * of waiting 24.8 days. + * + * This is the CHUNK SIZE, not a ceiling on `sleep`. Until 2026-09-02 it was a ceiling: `sleep` + * rejected anything larger with an `InvariantViolation`. That repaired the silent clamp — which was + * the 2026-08-27 adversarial review's actual intent — but it also made a `RETRY-18`-conformant + * pacing wait impossible, since that requirement clamps a server hint to 365 days, roughly fourteen + * times this value. Chaining timers keeps the review's intent (never a silent clamp) and drops the + * premise it rested on (that one timer is all there is). + * + * @internal + */ +export const MAX_SLEEP_MS = 2 ** 31 - 1; + +/** + * Chunking parameters for {@link sleepInChunks}, injected only by tests. + * + * A multi-chunk wait is otherwise unobservable without waiting 24.8 real days or installing a fake + * timer. A tiny `chunkMs` makes the chunking testable on REAL timers, and `onChunk` counts the + * slices without the test having to infer them from elapsed time. + * + * @internal + */ +export interface SleepChunking { + /** The largest slice to hand a single timer. Defaults to {@link MAX_SLEEP_MS}. */ + readonly chunkMs: number; + /** Called once per slice, before its timer is scheduled, with the slice's length. */ + readonly onChunk?: ((sliceMs: number) => void) | undefined; +} + +/** + * One slice: a single timer raced against the signal. Never called with more than `MAX_SLEEP_MS`. + * + * The abort path clears the timer and the resolve path detaches the listener, so neither outlives + * the slice -- which is what keeps a chunked wait from accumulating one listener per chunk. + */ +function sleepOnce(sliceMs: number, signal?: AbortSignal): Promise<void> { + return new Promise<void>((resolve, reject) => { + // The no-signal case is split out rather than written with `?.` throughout, so the abort branch + // below can close over a NARROWED `signal` -- otherwise mapping the reason needs a non-null + // assertion, which this project's lint forbids and which would be load-bearing here. + if (signal === undefined) { + setTimeout(resolve, sliceMs); + return; + } + const onAbort = (): void => { + clearTimeout(timer); + reject(abortToSdkError(signal, signal.reason)); + }; + const settle = (): void => { + signal.removeEventListener('abort', onAbort); + resolve(); + }; + const timer = setTimeout(settle, sliceMs); + signal.addEventListener('abort', onAbort, {once: true}); + }); +} + +/** + * The one wait primitive (CFG-17), chaining timers so any finite duration is honored. + * + * Split out of the object literal below so it carries an explicit return type and a name that + * appears in stack traces, and exported so a test can drive the chunking with a tiny `chunkMs` on + * real timers instead of waiting 24.8 days or installing a fake clock. + * + * @param durationMs - how long to wait; any finite, non-negative number. + * @param signal - the caller's cancellation signal, if any. + * @param chunking - test-only slice control; production passes nothing. + * @returns a promise resolving when the full duration has elapsed. + * + * @internal + */ +export async function sleepInChunks( + durationMs: number, + signal?: AbortSignal, + chunking?: SleepChunking, +): Promise<void> { + if (!Number.isFinite(durationMs) || durationMs < 0) { + // A rejection, never a synchronous throw: `sleep` returns a promise on every other path, and a + // caller awaiting it must not have to also wrap the call site in a try/catch. A RangeError + // rather than the project's assertion signal -- `InvariantViolation` is `@internal`, so a + // `@throws` naming it on this `@public` method promised a class no consumer can catch. + return Promise.reject( + new RangeError( + `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, + ), + ); + } + const chunkMs = chunking?.chunkMs ?? MAX_SLEEP_MS; + let remaining = durationMs; + // A `do` rather than a `while`: zero must still go through ONE real timer. A resolved promise + // settles on the microtask queue, which never lets the event loop turn, so a zero-backoff loop + // would starve timers and I/O -- measured at 4.1 million iterations in 300ms with a pending + // `setTimeout(fn, 0)` never running. + do { + // At the loop HEAD, so this one statement is both CFG-17's "an already-fired signal is honored + // before any timer is scheduled" and the between-chunks check that lets a long wait abort at a + // slice boundary. An abort arriving mid-slice is rejected immediately by `sleepOnce`'s own + // listener, so no window is left uncovered. + if (signal?.aborted === true) { + throw abortToSdkError(signal, signal.reason); + } + const slice = Math.min(remaining, chunkMs); + chunking?.onChunk?.(slice); + await sleepOnce(slice, signal); + remaining -= slice; + } while (remaining > 0); +} + +/** + * The shared platform-backed default CFG-15 requires. Frozen: it is a process-wide singleton, and a + * caller swapping one method on it would silently retime every consumer that took the default. + * + * @public + */ +export const defaultClock: Clock = Object.freeze({ + now: (): number => Date.now(), + monotonic: (): number => globalThis.performance.now(), + // Bound without the chunking parameter, so the seam's public signature stays two parameters and + // production always takes MAX_SLEEP_MS slices. + sleep: (durationMs: number, signal?: AbortSignal): Promise<void> => + sleepInChunks(durationMs, signal), +}); diff --git a/packages/core/src/config/configuration.test.ts b/packages/core/src/config/configuration.test.ts new file mode 100644 index 0000000..26f1391 --- /dev/null +++ b/packages/core/src/config/configuration.test.ts @@ -0,0 +1,649 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/configuration.test.ts +// Exercises: CFG-1 (strict layered precedence), CFG-2 (an empty environment value is absent), CFG-3 +// (normalized property key), CFG-4 (raw property accessor, no normalization), CFG-5/CFG-6/CFG-7 +// (never-throw typed accessors; CFG-7's grammar itself is duration.test.ts's), CFG-8 (immutable, +// override map copied at build), CFG-9 (copy-on-write derive), CFG-10 (remove drops only the +// override layer), CFG-11 (substitutable env/property seams, production default delegates to the +// platform environment), CFG-13 (global slot, last-write-wins), CFG-37 (fail-fast when a required +// argument is not the shape its parameter names), CFG-38 (typed accessors resolve through the full +// layered lookup). +// CFG-12 is deliberately untested: `docs/work/mvp/2026-09-04-open-items-dissolution.md` K3 records that a single-threaded-use +// statement has no observable behavior in this runtime to assert. +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import {randomUuid} from './identifiers.js'; +import { + CFG_KEY_HTTPS_PROXY, + CFG_KEY_HTTP_PROXY, + CFG_KEY_LOG_LEVEL, + CFG_KEY_MAX_RETRY_ATTEMPTS, + CFG_KEY_NO_PROXY, + ConfigurationBuilder, + defaultConfiguration, + getGlobalConfiguration, + setGlobalConfiguration, +} from './configuration.js'; + +// Captured before any test writes the slot, so "defaults to an empty configuration" stays a real +// claim no matter what order the tests below run in. +const GLOBAL_AT_LOAD = getGlobalConfiguration(); + +function sourceOf( + entries: Record<string, string>, +): (key: string) => string | undefined { + return key => entries[key]; +} + +describe('layered precedence (CFG-1)', () => { + test('resolves the override ahead of the environment', () => { + const config = new ConfigurationBuilder() + .withEnvSource(sourceOf({X: 'from-env'})) + .put('X', 'from-override') + .build(); + + expect(config.getString('X', 'from-default')).toBe('from-override'); + }); + + test('resolves the environment ahead of the property layer', () => { + const config = new ConfigurationBuilder() + .withEnvSource(sourceOf({X: 'from-env'})) + .withPropertySource(sourceOf({x: 'from-property'})) + .build(); + + expect(config.getString('X', 'from-default')).toBe('from-env'); + }); + + test('resolves the property layer ahead of the default', () => { + const config = new ConfigurationBuilder() + .withPropertySource(sourceOf({x: 'from-property'})) + .build(); + + expect(config.getString('X', 'from-default')).toBe('from-property'); + }); + + test('resolves the default when no layer supplies a value', () => { + const config = new ConfigurationBuilder().build(); + + expect(config.getString('X', 'from-default')).toBe('from-default'); + }); + + test('resolves undefined when no layer supplies a value and no default is given', () => { + const config = new ConfigurationBuilder().build(); + + expect(config.getString('X')).toBeUndefined(); + }); +}); + +describe('empty environment values (CFG-2)', () => { + test('falls through to the property layer when the environment value is empty', () => { + const config = new ConfigurationBuilder() + .withEnvSource(() => '') + .withPropertySource(sourceOf({x: 'from-property'})) + .build(); + + expect(config.getString('X')).toBe('from-property'); + }); + + test('falls through to the default when the environment value is empty and no property exists', () => { + const config = new ConfigurationBuilder().withEnvSource(() => '').build(); + + expect(config.getString('X', 'from-default')).toBe('from-default'); + }); +}); + +describe('property key normalization (CFG-3, CFG-4)', () => { + test('queries the property layer under the lower-cased, dotted key', () => { + const config = new ConfigurationBuilder() + .withPropertySource(sourceOf({'max.retry.attempts': '5'})) + .build(); + + expect(config.getString('MAX_RETRY_ATTEMPTS')).toBe('5'); + }); + + test('resolves a camelCase property-only key through the raw accessor', () => { + const config = new ConfigurationBuilder() + .withPropertySource(sourceOf({'https.proxyHost': 'proxy.example.com'})) + .build(); + + expect(config.getRawProperty('https.proxyHost')).toBe('proxy.example.com'); + }); + + test('does not resolve a camelCase property-only key through the normalizing accessor', () => { + const config = new ConfigurationBuilder() + .withPropertySource(sourceOf({'https.proxyHost': 'proxy.example.com'})) + .build(); + + expect(config.getString('https.proxyHost')).toBeUndefined(); + }); + + test('falls back when the raw accessor finds nothing', () => { + const config = new ConfigurationBuilder().build(); + + expect(config.getRawProperty('https.proxyHost', 'fallback')).toBe( + 'fallback', + ); + }); +}); + +describe('getInt (CFG-5, CFG-38)', () => { + test('resolves the default when the key is absent', () => { + expect(new ConfigurationBuilder().build().getInt('X', 42)).toBe(42); + }); + + test('resolves the default when the value is not an integer', () => { + const config = new ConfigurationBuilder().put('X', 'not-a-number').build(); + + expect(config.getInt('X', 42)).toBe(42); + }); + + test('resolves the default for a value with a trailing non-digit tail', () => { + const config = new ConfigurationBuilder().put('X', '12abc').build(); + + expect(config.getInt('X', 42)).toBe(42); + }); + + test('resolves the default for a fractional value', () => { + const config = new ConfigurationBuilder().put('X', '1.5').build(); + + expect(config.getInt('X', 42)).toBe(42); + }); + + test('normalizes a negative zero onto positive zero', () => { + const config = new ConfigurationBuilder().put('X', '-0').build(); + + expect(Object.is(config.getInt('X', -1), 0)).toBe(true); + }); + + test('returns a negative integer as-is', () => { + const config = new ConfigurationBuilder().put('X', '-5').build(); + + expect(config.getInt('X', 0)).toBe(-5); + }); + + test('resolves through the layered lookup, not the override map alone', () => { + const config = new ConfigurationBuilder().withEnvSource(() => '7').build(); + + expect(config.getInt('X', 0)).toBe(7); + }); +}); + +describe('getBoolean (CFG-6, CFG-38)', () => { + test('accepts true and false case-insensitively', () => { + const yes = new ConfigurationBuilder().put('X', 'TRUE').build(); + const no = new ConfigurationBuilder().put('X', 'False').build(); + + expect(yes.getBoolean('X', false)).toBe(true); + expect(no.getBoolean('X', true)).toBe(false); + }); + + test('rejects a truthy-looking value that is not literally true', () => { + for (const value of ['1', 'yes', 'on']) { + const config = new ConfigurationBuilder().put('X', value).build(); + + expect(config.getBoolean('X', false)).toBe(false); + } + }); + + test('rejects a falsy-looking value that is not literally false', () => { + // Asserted against a `true` fallback deliberately. Under a `false` fallback a lenient parser that + // read `'0'`/`'no'`/`'off'` as `false` would return exactly what the fallback returns, so the test + // could not tell CFG-6's strict grammar from a permissive one. + for (const value of ['0', 'no', 'off']) { + const config = new ConfigurationBuilder().put('X', value).build(); + + expect(config.getBoolean('X', true)).toBe(true); + } + }); + + test('resolves through the layered lookup, not the override map alone', () => { + const config = new ConfigurationBuilder() + .withEnvSource(() => 'true') + .build(); + + expect(config.getBoolean('X', false)).toBe(true); + }); +}); + +describe('getDuration (CFG-7, CFG-38)', () => { + // The grammar itself is duration.test.ts's; what belongs here is the accessor's own contract -- + // the layered lookup runs first, and a rejected value becomes the caller's fallback, never a throw. + function durationOf(raw: string, fallback = 0): number { + return new ConfigurationBuilder() + .put('X', raw) + .build() + .getDuration('X', fallback); + } + + test('returns the parsed duration when the value is one', () => { + expect(durationOf('PT5S')).toBe(5000); + expect(durationOf('500ms')).toBe(500); + expect(durationOf('1000')).toBe(1000); + }); + + test('falls back to the caller default when the grammar rejects the value', () => { + expect(durationOf('PT-5S', 99)).toBe(99); + expect(durationOf('5x', 99)).toBe(99); + }); + + test('falls back to the caller default when no layer supplies a value', () => { + expect(new ConfigurationBuilder().build().getDuration('X', 99)).toBe(99); + }); + + test('resolves through the layered lookup, not the override map alone', () => { + const config = new ConfigurationBuilder() + .withEnvSource(() => 'PT2S') + .build(); + + expect(config.getDuration('X', 0)).toBe(2000); + }); + + test('never throws for an arbitrary string', () => { + fc.assert( + fc.property(fc.string(), value => { + const config = new ConfigurationBuilder().put('X', value).build(); + + expect(() => config.getDuration('X', 7)).not.toThrow(); + }), + ); + }); +}); + +describe('immutability (CFG-8)', () => { + test('is unaffected by builder mutation after build', () => { + const builder = new ConfigurationBuilder().put('X', 'original'); + const config = builder.build(); + + builder.put('X', 'mutated-after-build'); + + expect(config.getString('X')).toBe('original'); + }); + + test('is frozen', () => { + const config = new ConfigurationBuilder().build(); + + expect(Object.isFrozen(config)).toBe(true); + }); +}); + +describe('derive (CFG-9, CFG-10)', () => { + test('leaves the receiver unchanged', () => { + const base = new ConfigurationBuilder().put('X', 'base').build(); + + const derived = base.derive(builder => { + builder.put('X', 'derived'); + }); + + expect(base.getString('X')).toBe('base'); + expect(derived.getString('X')).toBe('derived'); + }); + + test('inherits the source seams by reference', () => { + const base = new ConfigurationBuilder() + .withEnvSource(sourceOf({Y: 'env-y'})) + .build(); + + const derived = base.derive(builder => { + builder.put('X', 'derived'); + }); + + expect(derived.getString('Y')).toBe('env-y'); + }); + + test('detaches only the copy when the mutator replaces a source', () => { + const base = new ConfigurationBuilder() + .withEnvSource(sourceOf({Y: 'env-y'})) + .build(); + + const derived = base.derive(builder => { + builder.withEnvSource(sourceOf({Y: 'replaced'})); + }); + + expect(base.getString('Y')).toBe('env-y'); + expect(derived.getString('Y')).toBe('replaced'); + }); + + test('drops only the override layer on remove, falling through to the environment', () => { + const base = new ConfigurationBuilder() + .withEnvSource(sourceOf({X: 'from-env'})) + .put('X', 'from-override') + .build(); + + const derived = base.derive(builder => { + builder.remove('X'); + }); + + expect(derived.getString('X', 'from-default')).toBe('from-env'); + }); + + test('treats removing a key with no override as a no-op', () => { + const base = new ConfigurationBuilder() + .withEnvSource(sourceOf({X: 'from-env'})) + .build(); + + const derived = base.derive(builder => { + builder.remove('X'); + }); + + expect(derived.getString('X')).toBe('from-env'); + }); +}); + +describe('substitutable seams (CFG-11)', () => { + test('routes the environment lookup through the injected function, verbatim', () => { + const seen: string[] = []; + const config = new ConfigurationBuilder() + .withEnvSource(key => { + seen.push(key); + return undefined; + }) + .build(); + + config.getString('SOME_KEY'); + + expect(seen).toEqual(['SOME_KEY']); + }); + + test('routes the property lookup through the injected function, normalized', () => { + const seen: string[] = []; + const config = new ConfigurationBuilder() + .withPropertySource(key => { + seen.push(key); + return undefined; + }) + .build(); + + config.getString('SOME_KEY'); + + expect(seen).toEqual(['some.key']); + }); + + test('delegates the production default to the ambient environment', () => { + // A fresh key per run, drawn from the package's own generator rather than the clock: two + // same-millisecond runs of this file would otherwise pick the same name and race on real + // `process.env` (`docs/knowledge/harvested/testing.md:36`, `:50`). + const key = `DEXPACE_TEST_${randomUuid().replaceAll('-', '')}`; + const host = globalThis as { + process?: {env?: Record<string, string | undefined>}; + }; + const environment = host.process?.env; + expect(environment).toBeDefined(); + if (environment === undefined) return; + environment[key] = 'from-real-env'; + + try { + expect(defaultConfiguration().getString(key)).toBe('from-real-env'); + } finally { + Reflect.deleteProperty(environment, key); + } + }); + + test('leaves the production property seam empty, since Node has no such store', () => { + expect( + defaultConfiguration().getRawProperty('https.proxyHost'), + ).toBeUndefined(); + }); +}); + +describe('the global slot (CFG-13)', () => { + test('defaults to an empty configuration', () => { + expect(GLOBAL_AT_LOAD.getString('X', 'from-default')).toBe('from-default'); + }); + + test('returns the instance most recently written, last-write-wins', () => { + const first = new ConfigurationBuilder().put('X', 'first').build(); + const second = new ConfigurationBuilder().put('X', 'second').build(); + + setGlobalConfiguration(first); + setGlobalConfiguration(second); + + try { + expect(getGlobalConfiguration()).toBe(second); + } finally { + setGlobalConfiguration(GLOBAL_AT_LOAD); + } + }); +}); + +describe('well-known keys (CFG-14)', () => { + test('exposes a stable constant per documented key', () => { + expect([ + CFG_KEY_MAX_RETRY_ATTEMPTS, + CFG_KEY_LOG_LEVEL, + CFG_KEY_HTTP_PROXY, + CFG_KEY_HTTPS_PROXY, + CFG_KEY_NO_PROXY, + ]).toEqual([ + 'DEXPACE_MAX_RETRY_ATTEMPTS', + 'DEXPACE_LOG_LEVEL', + 'HTTP_PROXY', + 'HTTPS_PROXY', + 'NO_PROXY', + ]); + }); +}); + +// Key names every `Record`-backed seam answers through `Object.prototype` rather than with a string: +// `process.env['constructor']` is a *function*, `process.env['__proto__']` is an object. +const PROTOTYPE_KEYS = ['__proto__', 'constructor', 'toString', 'valueOf']; + +describe('a total lookup whatever the seam answers (CFG-5, CFG-6, CFG-7, CFG-11)', () => { + test('resolves the default for a prototype-named key against a record-backed seam', () => { + const config = new ConfigurationBuilder() + .withEnvSource(sourceOf({A: 'a'})) + .build(); + + for (const key of PROTOTYPE_KEYS) { + expect(config.getString(key, 'fallback')).toBe('fallback'); + } + }); + + test('never throws from a typed accessor for a prototype-named key', () => { + const config = new ConfigurationBuilder() + .withEnvSource(sourceOf({A: 'a'})) + .build(); + + for (const key of PROTOTYPE_KEYS) { + expect(config.getInt(key, 7)).toBe(7); + expect(config.getBoolean(key, true)).toBe(true); + expect(config.getDuration(key, 9)).toBe(9); + } + }); + + test('resolves the default for a prototype-named key against the production environment seam', () => { + // The same hazard on the wiring `defaultConfiguration` actually ships: `getInt('constructor')` + // used to die on `TypeError: (...).trim is not a function` here, on real `process.env`. + const config = defaultConfiguration(); + + for (const key of PROTOTYPE_KEYS) { + expect(config.getString(key, 'fallback')).toBe('fallback'); + expect(config.getInt(key, 7)).toBe(7); + expect(config.getBoolean(key, true)).toBe(true); + expect(config.getDuration(key, 9)).toBe(9); + } + }); +}); + +describe('fail-fast validation (CFG-37)', () => { + test('rejects an absent override key', () => { + const builder = new ConfigurationBuilder(); + + expect(() => builder.put(undefined as unknown as string, 'v')).toThrow( + InvariantViolation, + ); + }); + + test('rejects an absent override value', () => { + const builder = new ConfigurationBuilder(); + + expect(() => builder.put('k', undefined as unknown as string)).toThrow( + InvariantViolation, + ); + }); + + test('rejects an absent removal key', () => { + const builder = new ConfigurationBuilder(); + + expect(() => builder.remove(undefined as unknown as string)).toThrow( + InvariantViolation, + ); + }); + + test('rejects an absent source function', () => { + const builder = new ConfigurationBuilder(); + + expect(() => + builder.withEnvSource(undefined as unknown as () => undefined), + ).toThrow(InvariantViolation); + expect(() => + builder.withPropertySource(undefined as unknown as () => undefined), + ).toThrow(InvariantViolation); + }); + + test('rejects an absent derive mutator', () => { + const config = new ConfigurationBuilder().build(); + + expect(() => config.derive(undefined as unknown as () => void)).toThrow( + InvariantViolation, + ); + }); + + test('rejects an absent global-configuration value', () => { + expect(() => { + setGlobalConfiguration( + undefined as unknown as ReturnType<typeof defaultConfiguration>, + ); + }).toThrow(InvariantViolation); + }); + + test('rejects a global-configuration value that is present but not a configuration', () => { + // Every other guard in this module checks the shape it needs; this one checked only for null, + // so a `42` reached the process-wide slot and surfaced far from the fault. + expect(() => { + setGlobalConfiguration( + 42 as unknown as ReturnType<typeof defaultConfiguration>, + ); + }).toThrow(InvariantViolation); + expect(getGlobalConfiguration()).not.toBe(42); + }); + + test('accepts an explicitly absent lookup default, which is documented-nullable', () => { + const config = new ConfigurationBuilder().build(); + + expect(() => config.getString('X', undefined)).not.toThrow(); + }); +}); + +describe('a total lookup when a seam fails (CFG-5, CFG-6, CFG-7, CFG-11)', () => { + test('falls through to the lower layers when the environment seam throws', () => { + // CFG-11 makes the seam caller-supplied, so it can be backed by a file or a remote store and + // fail like any I/O. CFG-5's never-throw clause is the stronger obligation. + const config = new ConfigurationBuilder() + .withEnvSource(() => { + throw new Error('env seam exploded'); + }) + .withPropertySource(sourceOf({x: 'from-property'})) + .build(); + + expect(config.getString('X', 'fallback')).toBe('from-property'); + }); + + test('resolves the caller default from every accessor when the environment seam throws', () => { + const config = new ConfigurationBuilder() + .withEnvSource(() => { + throw new Error('env seam exploded'); + }) + .build(); + + expect(config.getString('X', 'fallback')).toBe('fallback'); + expect(config.getInt('X', 7)).toBe(7); + expect(config.getBoolean('X', true)).toBe(true); + expect(config.getDuration('X', 9)).toBe(9); + }); + + test('resolves the caller default from every accessor when the property seam throws', () => { + const config = new ConfigurationBuilder() + .withPropertySource(() => { + throw new Error('property seam exploded'); + }) + .build(); + + expect(config.getString('X', 'fallback')).toBe('fallback'); + expect(config.getInt('X', 7)).toBe(7); + expect(config.getBoolean('X', true)).toBe(true); + expect(config.getDuration('X', 9)).toBe(9); + expect(config.getRawProperty('X', 'fallback')).toBe('fallback'); + }); + + test('treats a seam answering with a non-string as a layer that supplies nothing', () => { + const config = new ConfigurationBuilder() + .withEnvSource((() => 42) as unknown as (key: string) => undefined) + .build(); + + expect(config.getString('X', 'fallback')).toBe('fallback'); + expect(config.getInt('X', 7)).toBe(7); + }); +}); + +describe('K14: a failing seam is now audible (CFG-5, CFG-11, CFG-24 sibling)', () => { + test('a failing seam is warned about, naming the source (K14)', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + + try { + const config = new ConfigurationBuilder() + .withEnvSource(() => { + throw new Error('env seam exploded'); + }) + .build(); + + expect(config.getString('X', 'fallback')).toBe('fallback'); + + const failures = events.filter( + e => e.get('event') === 'config.sourceFailed', + ); + expect(failures).toHaveLength(1); + expect(failures[0]?.get('source')).toBe('environment'); + expect(failures[0]?.get('key')).toBe('X'); + expect(String(failures[0]?.get('cause'))).toContain('env seam exploded'); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); + + test('a failing property seam names the property source (K14)', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + + try { + const config = new ConfigurationBuilder() + .withPropertySource(() => { + throw new Error('property seam exploded'); + }) + .build(); + + expect(config.getRawProperty('X', 'fallback')).toBe('fallback'); + + const failures = events.filter( + e => e.get('event') === 'config.sourceFailed', + ); + expect(failures).toHaveLength(1); + expect(failures[0]?.get('source')).toBe('property'); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); diff --git a/packages/core/src/config/configuration.ts b/packages/core/src/config/configuration.ts new file mode 100644 index 0000000..a1f315f --- /dev/null +++ b/packages/core/src/config/configuration.ts @@ -0,0 +1,401 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/configuration.ts +import {invariant} from '../invariant.js'; +import {getGlobalLogger} from '../observability/logger.js'; +import {parseDurationMs} from './duration.js'; + +/** + * A substitutable lookup seam: a key name in, a value or `undefined` out (CFG-11). Both the + * environment and the property layer are one of these, so a test supplies hermetic lookups without + * touching the real environment. + * + * @public + */ +export type SourceFn = (key: string) => string | undefined; + +const STRICT_INTEGER = /^[+-]?\d+$/u; + +/** + * Reads one layer, total. + * + * CFG-11 makes both seams caller-supplied, which means a layer can do two things the never-throw + * accessors of CFG-5/6/7 must absorb rather than propagate: + * + * * **It can throw.** A seam backed by a file, a secrets store, or a remote key/value store fails + * like any I/O. That used to escape unwrapped out of `getString`/`getInt`/`getBoolean`/ + * `getDuration`, contradicting this interface's own "never a throw" contract. + * * **It can answer with a non-string.** A `Record`-backed seam -- `process.env` included -- + * resolves a key named `constructor`, `toString`, or `__proto__` through `Object.prototype` and + * returns a *function* or an object. `getString` then handed back a non-string typed as + * `string | undefined`, and each typed accessor died on `raw.trim()` with a raw `TypeError`. + * + * Both resolve to "this layer supplies nothing", so the lookup falls through exactly as it does for + * an absent key. The production seam additionally guards the prototype case at its own source; this + * guard is the one that holds for a seam this package did not write. + * + * The residue used to be that a seam failure was then silently INVISIBLE: an operator whose + * secrets-store seam was misconfigured saw the caller's default resolve, with nothing anywhere to + * say why. That half is closed as of 2026-09-02 -- the swallowed throw is warned about, naming the + * layer and the key. The value still resolves to the caller's default either way, because CFG-5's + * never-throw clause is the stronger obligation. + * + * @param source - the caller-supplied lookup seam for one layer. + * @param key - the key being looked up, already normalized for that layer. + * @param layer - which layer this is, for the diagnostic. + */ +function readLayer( + source: SourceFn, + key: string, + layer: 'environment' | 'property', +): string | undefined { + let value: unknown; + try { + value = source(key); + } catch (error) { + // Deliberately unnarrowed: the throw comes from caller-supplied code, so no error type can be + // predicted, and CFG-5 makes "the lookup never fails the caller" the stronger obligation. + warnSourceFailed(layer, key, error); + return undefined; + } + return typeof value === 'string' ? value : undefined; +} + +/** The diagnostic for a seam that threw. Never the value -- a configuration value can be a secret. */ +function warnSourceFailed(layer: string, key: string, error: unknown): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('config.sourceFailed') + .field('source', layer) + .field('key', key) + .cause(error) + .emit(); + } catch { + // OBS-20: logger failure must never fail a lookup CFG-5 makes total. + } +} + +/** CFG-3: the property layer is queried lower-cased with underscores replaced by dots. */ +function normalizePropertyKey(key: string): string { + return key.toLowerCase().replaceAll('_', '.'); +} + +/** + * A layered, immutable string-keyed configuration (CFG-1, CFG-8). + * + * Lookups resolve in strict order: an explicit override for the exact key, then the environment + * source under the exact key, then the property source under the *normalized* key, then the + * caller's default. Built instances are frozen and safe to share without synchronization; a + * reconfigured instance comes from {@link Configuration.derive}, copy-on-write. + * + * Every typed accessor is total -- a missing or unparseable value yields the caller's default, never + * a throw (CFG-5, CFG-6, CFG-7). + * + * Every CFG-37 guard on this type and on {@link ConfigurationBuilder} is a *shape* check, not a null + * check: it tests that the argument is the kind of value the parameter names (`string`, `function`, + * object), so a wrongly-typed argument from an untyped caller fails here rather than far downstream. + * + * @public + */ +export interface Configuration { + /** + * Resolves `key` through the full layered lookup (CFG-1). + * + * @param key - the key name; the override and environment layers use it verbatim, the property + * layer uses its normalized (lower-cased, dotted) form. + * @param fallback - the value to return when no layer supplies one; may be omitted. + */ + getString(key: string, fallback?: string): string | undefined; + + /** + * Resolves `key` against the property layer alone, by exact name with no normalization (CFG-4), + * so a camelCase property-only key such as `https.proxyHost` resolves with its casing preserved. + */ + getRawProperty(key: string, fallback?: string): string | undefined; + + /** + * Resolves `key` through the full layered lookup and parses it as a base-10 integer (CFG-5, + * CFG-38). Negative integers are valid and returned as-is; anything unparseable yields `fallback`. + */ + getInt(key: string, fallback: number): number; + + /** + * Resolves `key` through the full layered lookup and parses it strictly as a boolean (CFG-6, + * CFG-38): only case-insensitive `true`/`false` are recognized, and `1`/`0`/`yes`/`no`/`on`/`off` + * all yield `fallback`. + */ + getBoolean(key: string, fallback: boolean): boolean; + + /** + * Resolves `key` through the full layered lookup and parses it as a duration in milliseconds + * (CFG-7, CFG-38): ISO-8601, `<number><unit>` shorthand, or a bare number of milliseconds. A + * negative duration or an unknown unit yields `fallbackMs`. + */ + getDuration(key: string, fallbackMs: number): number; + + /** + * Produces a reconfigured copy, copy-on-write (CFG-9): the override map is copied before `mutate` + * runs and the source seams are inherited by reference, so this instance is left unchanged. + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `mutate` is not a function (CFG-37). + */ + derive(mutate: (builder: ConfigurationBuilder) => void): Configuration; +} + +class LayeredConfiguration implements Configuration { + readonly #overrides: ReadonlyMap<string, string>; + readonly #envSource: SourceFn; + readonly #propertySource: SourceFn; + + constructor( + overrides: ReadonlyMap<string, string>, + envSource: SourceFn, + propertySource: SourceFn, + ) { + this.#overrides = overrides; + this.#envSource = envSource; + this.#propertySource = propertySource; + Object.freeze(this); + } + + getString(key: string, fallback?: string): string | undefined { + const override = this.#overrides.get(key); + if (override !== undefined) return override; + const fromEnv = readLayer(this.#envSource, key, 'environment'); + // CFG-2: an environment value that is present but empty is absent, so the lookup falls through. + if (fromEnv !== undefined && fromEnv !== '') return fromEnv; + const fromProperty = readLayer( + this.#propertySource, + normalizePropertyKey(key), + 'property', + ); + if (fromProperty !== undefined) return fromProperty; + return fallback; + } + + getRawProperty(key: string, fallback?: string): string | undefined { + return readLayer(this.#propertySource, key, 'property') ?? fallback; + } + + getInt(key: string, fallback: number): number { + const raw = this.getString(key)?.trim(); + if (raw === undefined || !STRICT_INTEGER.test(raw)) return fallback; + const value = Number(raw); + // `+ 0` folds `-0` -- which `"-0"` parses to and `Number.isSafeInteger` accepts -- onto `0`, so a + // caller never receives a value that is `===` zero yet differs under `Object.is` or `1 / n`. + return Number.isSafeInteger(value) ? value + 0 : fallback; + } + + getBoolean(key: string, fallback: boolean): boolean { + const raw = this.getString(key)?.trim().toLowerCase(); + if (raw === 'true') return true; + if (raw === 'false') return false; + return fallback; + } + + getDuration(key: string, fallbackMs: number): number { + const raw = this.getString(key); + if (raw === undefined) return fallbackMs; + const parsed = parseDurationMs(raw); + if (parsed === null || !Number.isFinite(parsed) || parsed < 0) + return fallbackMs; + return parsed; + } + + derive(mutate: (builder: ConfigurationBuilder) => void): Configuration { + invariant( + typeof mutate === 'function', + 'Configuration.derive: mutate is required', + ); + const builder = new ConfigurationBuilder() + .withEnvSource(this.#envSource) + .withPropertySource(this.#propertySource); + // CFG-9: the override map is copied *before* the mutator runs, so a mutator that throws cannot + // leave this instance half-rewritten either. + for (const [key, value] of this.#overrides) builder.put(key, value); + mutate(builder); + return builder.build(); + } +} + +/** + * Accumulates overrides and source seams into a {@link Configuration} (CFG-11). + * + * Single-threaded use only (CFG-12): the immutability guarantee is about the built `Configuration`, + * not about an in-progress builder. + * + * @public + */ +export class ConfigurationBuilder { + readonly #overrides = new Map<string, string>(); + #envSource: SourceFn = () => undefined; + #propertySource: SourceFn = () => undefined; + + /** + * Sets an override for the exact key, the highest-precedence layer. + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `key` or `value` is not a string (CFG-37). + */ + put(key: string, value: string): this { + invariant( + typeof key === 'string', + 'ConfigurationBuilder.put: key is required', + ); + invariant( + typeof value === 'string', + 'ConfigurationBuilder.put: value is required', + ); + this.#overrides.set(key, value); + return this; + } + + /** + * Drops the override for `key`, leaving the lower layers to answer as if it had never been set + * (CFG-10). Removing a key with no override is a no-op. + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `key` is not a string (CFG-37). + */ + remove(key: string): this { + invariant( + typeof key === 'string', + 'ConfigurationBuilder.remove: key is required', + ); + this.#overrides.delete(key); + return this; + } + + /** + * Replaces the environment seam (CFG-11). + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `source` is not a function (CFG-37). + */ + withEnvSource(source: SourceFn): this { + invariant( + typeof source === 'function', + 'ConfigurationBuilder.withEnvSource: source is required', + ); + this.#envSource = source; + return this; + } + + /** + * Replaces the property seam (CFG-11). + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `source` is not a function (CFG-37). + */ + withPropertySource(source: SourceFn): this { + invariant( + typeof source === 'function', + 'ConfigurationBuilder.withPropertySource: source is required', + ); + this.#propertySource = source; + return this; + } + + /** + * Freezes the accumulated state into a {@link Configuration}. The override map is copied here + * (CFG-8), so later mutation of this builder cannot reach the returned instance. + */ + build(): Configuration { + return new LayeredConfiguration( + new Map(this.#overrides), + this.#envSource, + this.#propertySource, + ); + } +} + +/** The well-known key for the retry-attempt cap (CFG-14). @public */ +export const CFG_KEY_MAX_RETRY_ATTEMPTS = 'DEXPACE_MAX_RETRY_ATTEMPTS'; +/** The well-known key for the log level (CFG-14). @public */ +export const CFG_KEY_LOG_LEVEL = 'DEXPACE_LOG_LEVEL'; +/** The well-known key for the plain-HTTP proxy URL (CFG-14). @public */ +export const CFG_KEY_HTTP_PROXY = 'HTTP_PROXY'; +/** The well-known key for the HTTPS proxy URL (CFG-14). @public */ +export const CFG_KEY_HTTPS_PROXY = 'HTTPS_PROXY'; +/** The well-known key for the proxy-bypass host list (CFG-14). @public */ +export const CFG_KEY_NO_PROXY = 'NO_PROXY'; + +/** + * Reads the ambient environment without a `node:` import, so the same source compiles and runs on + * the browser/Workers half of core's runtime floor, where it simply finds nothing. + */ +function readEnvironmentRecord(): Record<string, string | undefined> { + const host = globalThis as { + process?: {env?: Record<string, string | undefined>}; + }; + return host.process?.env ?? {}; +} + +/** + * The production environment seam. `Object.hasOwn` rather than a bare index: `process.env` is an + * ordinary object, so `env['constructor']`, `env['toString']`, and `env['__proto__']` would + * otherwise resolve through `Object.prototype` and hand the layered lookup a function or an object + * where a string was promised (CFG-5). + */ +function readAmbientEnvironment(key: string): string | undefined { + const env = readEnvironmentRecord(); + return Object.hasOwn(env, key) ? env[key] : undefined; +} + +/** + * The production wiring CFG-11 requires: the environment seam delegates to the platform + * environment. + * + * The property seam is deliberately a function that always returns `undefined`. Node has no ambient + * key/value store distinct from `process.env`, and routing a synthetic "system property" back + * through `process.env` under a different key would invent a layer the platform does not have. The + * seam stays substitutable so a host that *does* have one can supply it. + * + * @returns a fresh `Configuration` reading the live environment on every lookup. + * + * @public + */ +export function defaultConfiguration(): Configuration { + return new ConfigurationBuilder() + .withEnvSource(readAmbientEnvironment) + .build(); +} + +/** + * Module-level mutable state, which `docs/knowledge/harvested/variables-and-declarations.md:22` bans outright. + * Deliberate: CFG-13 *specifies* a process-wide, last-write-wins slot, so the shared-by-every-importer + * property the rule warns about is the requirement rather than a side effect. The rule's real cost -- + * state carried between test cases in one process -- is live and unmitigated: there is no reset hook, + * so a test that writes this slot must restore it itself (`configuration.test.ts` captures the + * load-time value and restores it in a `finally`). + */ +let globalConfiguration: Configuration = new ConfigurationBuilder().build(); + +/** + * The process-wide configuration slot (CFG-13), defaulting to an empty `Configuration`. + * + * @returns the configuration most recently passed to {@link setGlobalConfiguration}. + * + * @public + */ +export function getGlobalConfiguration(): Configuration { + return globalConfiguration; +} + +/** + * Replaces the process-wide configuration slot, last-write-wins (CFG-13). + * + * @throws an assertion failure (a caller bug, not a catchable condition) when `config` is not an object (CFG-37). + * + * @public + */ +export function setGlobalConfiguration(config: Configuration): void { + // Typed as `unknown` first: CFG-37 exists for the untyped caller the compiler never sees, so the + // check has to survive a type that says it cannot happen. + const supplied: unknown = config; + // A `typeof` check, not just a null check: every other CFG-37 guard in this module tests the shape + // it needs (`string` for an override, `function` for a seam or a mutator). Accepting `42` here put + // a number in the process-wide slot, where it surfaced as a failure in an unrelated consumer far + // from the fault (`docs/knowledge/harvested/error-handling.md:36`). + invariant( + typeof supplied === 'object' && supplied !== null, + 'setGlobalConfiguration: config is required', + ); + globalConfiguration = config; +} diff --git a/packages/core/src/config/duration.test.ts b/packages/core/src/config/duration.test.ts new file mode 100644 index 0000000..f9c00fc --- /dev/null +++ b/packages/core/src/config/duration.test.ts @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/duration.test.ts +// Exercises: CFG-7 (the duration grammar itself -- ISO-8601 P/p-prefixed, shorthand <number><unit> +// over ms/s/m/h/d case-insensitively, a bare number as milliseconds; a negative duration and an +// unknown unit are rejected, and rejection is a null the caller turns into its own default). +// The accessor-level half of CFG-7 -- that Configuration.getDuration returns the caller's fallback +// on rejection and resolves through the full layered lookup first -- lives in configuration.test.ts. +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {parseDurationMs} from './duration.js'; + +describe('parseDurationMs (CFG-7)', () => { + test('accepts an ISO-8601 duration', () => { + expect(parseDurationMs('PT5S')).toBe(5000); + expect(parseDurationMs('P1DT2H3M4S')).toBe(93_784_000); + }); + + test('accepts a lower-case ISO-8601 duration', () => { + expect(parseDurationMs('pt5s')).toBe(5000); + }); + + test('accepts shorthand units case-insensitively', () => { + expect(parseDurationMs('500ms')).toBe(500); + expect(parseDurationMs('2S')).toBe(2000); + expect(parseDurationMs('3m')).toBe(180_000); + expect(parseDurationMs('4h')).toBe(14_400_000); + expect(parseDurationMs('5d')).toBe(432_000_000); + }); + + test('reads a bare number as milliseconds', () => { + expect(parseDurationMs('1000')).toBe(1000); + }); + + test('tolerates surrounding whitespace', () => { + expect(parseDurationMs(' PT5S ')).toBe(5000); + }); + + test('rejects a negative duration', () => { + expect(parseDurationMs('PT-5S')).toBeNull(); + expect(parseDurationMs('-500ms')).toBeNull(); + }); + + test('rejects an unknown unit', () => { + expect(parseDurationMs('5x')).toBeNull(); + }); + + test('rejects an ISO-8601 duration with no components at all', () => { + expect(parseDurationMs('P')).toBeNull(); + expect(parseDurationMs('PT')).toBeNull(); + }); + + test('rejects the ambiguous month designator rather than guessing', () => { + expect(parseDurationMs('P5M')).toBeNull(); + }); + + test('reads the three grammars onto one scale', () => { + // The canonical law CFG-7 implies (`docs/knowledge/harvested/testing.md:28`): ISO-8601, shorthand, and a + // bare number are three spellings of one duration, so for any whole number of seconds all three + // must land on the same milliseconds. A totality property cannot see the two scales drift apart. + fc.assert( + fc.property(fc.integer({min: 0, max: 100_000}), seconds => { + const iso = parseDurationMs(`PT${String(seconds)}S`); + + expect(iso).toBe(seconds * 1000); + expect(parseDurationMs(`${String(seconds)}s`)).toBe(iso); + expect(parseDurationMs(String(seconds * 1000))).toBe(iso); + }), + ); + }); + + test('never throws for an arbitrary string', () => { + fc.assert( + fc.property(fc.string(), value => { + expect(() => parseDurationMs(value)).not.toThrow(); + }), + ); + }); +}); diff --git a/packages/core/src/config/duration.ts b/packages/core/src/config/duration.ts new file mode 100644 index 0000000..c90b67c --- /dev/null +++ b/packages/core/src/config/duration.ts @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/duration.ts + +/** An ISO-8601 duration, months deliberately unsupported -- `P5M` is ambiguous, `PT5M` is not. */ +const ISO_DURATION = + /^p(?:(\d+(?:\.\d+)?)d)?(?:t(?:(\d+(?:\.\d+)?)h)?(?:(\d+(?:\.\d+)?)m)?(?:(\d+(?:\.\d+)?)s)?)?$/iu; +const SHORTHAND_DURATION = /^(\d+(?:\.\d+)?)(ms|s|m|h|d)$/iu; +const BARE_NUMBER = /^\d+(?:\.\d+)?$/u; + +const MS_PER_SECOND = 1000; +const MS_PER_MINUTE = 60_000; +const MS_PER_HOUR = 3_600_000; +const MS_PER_DAY = 86_400_000; + +/** + * The shorthand unit table. Built from the constants above rather than from its own literals: the + * ISO-8601 and shorthand grammars are two spellings of one scale, and CFG-7 requires them to agree. + */ +const MS_PER_UNIT: ReadonlyMap<string, number> = new Map([ + ['ms', 1], + ['s', MS_PER_SECOND], + ['m', MS_PER_MINUTE], + ['h', MS_PER_HOUR], + ['d', MS_PER_DAY], +]); + +function parseIsoDuration(raw: string): number | null { + const match = ISO_DURATION.exec(raw); + if (match === null) return null; + const [days, hours, minutes, seconds] = match.slice(1); + // `P` and `PT` match with every group absent; a duration with no component at all is not a + // duration, so it falls through to the caller's default rather than resolving to zero. + if ( + days === undefined && + hours === undefined && + minutes === undefined && + seconds === undefined + ) { + return null; + } + return ( + Number(days ?? 0) * MS_PER_DAY + + Number(hours ?? 0) * MS_PER_HOUR + + Number(minutes ?? 0) * MS_PER_MINUTE + + Number(seconds ?? 0) * MS_PER_SECOND + ); +} + +function parseShorthandDuration(raw: string): number | null { + const match = SHORTHAND_DURATION.exec(raw); + if (match === null) return null; + // Both groups are mandatory in `SHORTHAND_DURATION`, so a match guarantees both participated, and + // the unit is one of `MS_PER_UNIT`'s five keys by construction of the alternation. The compiler + // sees neither fact under `noUncheckedIndexedAccess`, so the destructuring defaults and the + // `undefined` check below are unreachable and exist only to satisfy the index type. + const [, amount = '', unit = ''] = match; + const perUnit = MS_PER_UNIT.get(unit.toLowerCase()); + if (perUnit === undefined) return null; + return Number(amount) * perUnit; +} + +/** + * CFG-7's grammar, total: ISO-8601 (`P`/`p`-prefixed), shorthand `<number><unit>` over ms/s/m/h/d + * case-insensitively, or a bare number read as milliseconds. Every unrecognized form -- an unknown + * unit, a negative, anything else -- yields `null` so the caller falls back to its default. + * + * Its own module rather than a helper inside `configuration.ts`: the grammar is a concept separate + * from the layered lookup that happens to consume it (`docs/knowledge/harvested/module-organization.md:42`). + * `Configuration.getDuration` is its only caller today. + * + * @param raw - the candidate duration; surrounding whitespace is tolerated. + * @returns the duration in milliseconds, or `null` when `raw` is not one. + * + * @internal + */ +export function parseDurationMs(raw: string): number | null { + const trimmed = raw.trim(); + if (BARE_NUMBER.test(trimmed)) return Number(trimmed); + return parseShorthandDuration(trimmed) ?? parseIsoDuration(trimmed); +} diff --git a/packages/core/src/config/equality.test.ts b/packages/core/src/config/equality.test.ts new file mode 100644 index 0000000..cc6a273 --- /dev/null +++ b/packages/core/src/config/equality.test.ts @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/equality.test.ts +// Exercises: CFG-33 (content-based array comparison recursing into nested arrays, null-safety, +// hash/equality consistency), CFG-34 (NaN equals NaN, +0 does not equal -0, a typed array is never +// equal to a plain array of the same numeric values). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {deepEqual, deepHash} from './equality.js'; + +describe('deepEqual (CFG-33)', () => { + test('compares primitives with Object.is', () => { + expect(deepEqual(1, 1)).toBe(true); + expect(deepEqual('a', 'b')).toBe(false); + }); + + test('compares arrays element by element', () => { + expect(deepEqual([1, 2, 3], [1, 2, 3])).toBe(true); + }); + + test('recurses into nested arrays', () => { + expect(deepEqual([1, [2, 3]], [1, [2, 3]])).toBe(true); + expect(deepEqual([1, [2, 3]], [1, [2, 4]])).toBe(false); + }); + + test('treats arrays of different lengths as unequal', () => { + expect(deepEqual([1, 2], [1, 2, 3])).toBe(false); + }); + + test('treats two empty arrays as equal', () => { + expect(deepEqual([], [])).toBe(true); + }); + + test('treats two nulls as equal and null as unequal to undefined', () => { + expect(deepEqual(null, null)).toBe(true); + expect(deepEqual(null, undefined)).toBe(false); + }); + + test('overflows the stack on a self-referential array rather than terminating', () => { + // Pinned, not fixed. Both helpers recurse without a cycle guard or a depth cap, and neither is + // exported from the package barrel or called by anything yet. `docs/work/mvp/2026-09-04-open-items-dissolution.md` K16 records + // the acyclic, bounded-depth precondition the first consumer inherits. + const cyclic: unknown[] = []; + cyclic.push(cyclic); + const other: unknown[] = []; + other.push(other); + + expect(() => deepEqual(cyclic, other)).toThrow(RangeError); + }); + + test('falls back to identity for non-array objects', () => { + const shared = {x: 2}; + + expect(deepEqual(shared, shared)).toBe(true); + expect(deepEqual({x: 2}, {x: 2})).toBe(false); + }); + + test('compares typed arrays by element value', () => { + expect(deepEqual(new Uint8Array([1, 2]), new Uint8Array([1, 2]))).toBe( + true, + ); + expect(deepEqual(new Uint8Array([1, 2]), new Uint8Array([1, 3]))).toBe( + false, + ); + }); +}); + +describe('deepEqual floating-point array semantics (CFG-34)', () => { + test('treats NaN as equal to NaN inside an array', () => { + expect(deepEqual([Number.NaN], [Number.NaN])).toBe(true); + }); + + test('treats +0 as unequal to -0 inside an array', () => { + expect(deepEqual([0], [-0])).toBe(false); + }); + + test('never treats a typed numeric array as equal to a plain array of the same values', () => { + expect(deepEqual(new Float64Array([1, 2]), [1, 2])).toBe(false); + }); + + test('never treats two differently-typed numeric arrays as equal', () => { + expect(deepEqual(new Float64Array([1, 2]), new Int32Array([1, 2]))).toBe( + false, + ); + }); + + test('treats a DataView as an opaque object rather than an indexed collection', () => { + const bytes = new Uint8Array([1, 2]).buffer; + + expect(deepEqual(new DataView(bytes), new DataView(bytes))).toBe(false); + }); +}); + +describe('deepHash recursion limits (CFG-33)', () => { + test('overflows the stack on a self-referential array rather than terminating', () => { + // The `deepHash` half of the G16 precondition; the `deepEqual` half is pinned above. + const cyclic: unknown[] = []; + cyclic.push(cyclic); + + expect(() => deepHash(cyclic)).toThrow(RangeError); + }); + + test('overflows the stack past its recursion depth rather than terminating', () => { + let deep: unknown[] = []; + for (let i = 0; i < 100_000; i += 1) deep = [deep]; + + expect(() => deepHash(deep)).toThrow(RangeError); + }); +}); + +describe('deepHash (CFG-33)', () => { + test('hashes null and undefined to zero', () => { + expect(deepHash(null)).toBe(0); + expect(deepHash(undefined)).toBe(0); + }); + + test('agrees with deepEqual for equal nested arrays', () => { + const left = [1, 'a', [3, 4]]; + const right = [1, 'a', [3, 4]]; + + expect(deepEqual(left, right)).toBe(true); + expect(deepHash(left)).toBe(deepHash(right)); + }); + + test('hashes NaN consistently, matching its self-equality inside an array', () => { + expect(deepHash([Number.NaN])).toBe(deepHash([Number.NaN])); + }); + + test('hashes +0 and -0 distinctly, matching their inequality', () => { + expect(deepHash([0])).not.toBe(deepHash([-0])); + }); + + test('distinguishes element order', () => { + expect(deepHash([1, 2])).not.toBe(deepHash([2, 1])); + }); + + test('hashes equal bigints to the same value', () => { + expect(deepHash(9_007_199_254_740_993n)).toBe( + deepHash(9_007_199_254_740_993n), + ); + expect(deepHash(1n)).not.toBe(deepHash(2n)); + }); + + test('hashes booleans distinctly', () => { + expect(deepHash(true)).not.toBe(deepHash(false)); + }); +}); + +describe('deepEqual and deepHash properties (CFG-33)', () => { + const tree = fc.letrec<{node: unknown}>(rec => ({ + node: fc.oneof( + {depthSize: 'small'}, + fc.integer(), + fc.string(), + fc.boolean(), + fc.constant(null), + fc.array(rec('node'), {maxLength: 4}), + ), + })).node; + + test('is reflexive', () => { + fc.assert( + fc.property(tree, value => { + expect(deepEqual(value, value)).toBe(true); + }), + ); + }); + + test('is symmetric', () => { + fc.assert( + fc.property(tree, tree, (left, right) => { + expect(deepEqual(left, right)).toBe(deepEqual(right, left)); + }), + ); + }); + + test('hashes every equal pair to the same value', () => { + fc.assert( + fc.property(tree, tree, (left, right) => { + if (deepEqual(left, right)) { + expect(deepHash(left)).toBe(deepHash(right)); + } + }), + ); + }); +}); diff --git a/packages/core/src/config/equality.ts b/packages/core/src/config/equality.ts new file mode 100644 index 0000000..7e175f7 --- /dev/null +++ b/packages/core/src/config/equality.ts @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/equality.ts + +/** + * Whether `value` is one of the indexed collections {@link deepEqual} compares by content: a plain + * array or a typed array. `DataView` is excluded -- it is an `ArrayBuffer` view with no indexed + * elements, so it falls through to identity comparison like any other object. + */ +function isIndexedCollection(value: unknown): value is ArrayLike<unknown> { + return ( + Array.isArray(value) || + (ArrayBuffer.isView(value) && !(value instanceof DataView)) + ); +} + +/** + * Whether two indexed collections are the same kind. CFG-34's "an object array and a primitive array + * of the same numeric values MUST NOT be equal" is, in this runtime, the plain-array/typed-array + * split -- and one `Float64Array` is not the same kind as one `Int32Array` either, so typed arrays + * compare prototypes. + */ +function isSameCollectionKind( + a: ArrayLike<unknown>, + b: ArrayLike<unknown>, +): boolean { + const isATyped = !Array.isArray(a); + const isBTyped = !Array.isArray(b); + if (isATyped !== isBTyped) return false; + if (!isATyped) return true; + return Object.getPrototypeOf(a) === Object.getPrototypeOf(b); +} + +/** + * Compares two values by content (CFG-33, CFG-34). + * + * Arrays -- plain or typed -- compare element by element, recursing into nested arrays; everything + * else compares with `Object.is`, never `===`. That is exactly CFG-34's floating-point semantics -- + * `NaN` equals `NaN`, and `+0` does not equal `-0` -- and it holds at the top level as well as + * per element, so `deepEqual(NaN, NaN)` is `true` and `deepEqual(0, -0)` is `false`. + * A typed array is never equal to a plain array of the same numeric values. Null-safe: two `null`s + * are equal, as are two `undefined`s. + * + * Consistent with {@link deepHash}: every pair this reports equal hashes to the same value. + * + * @param a - the left value. + * @param b - the right value. + * @returns whether the two values are deeply equal. + * + * @internal + */ +export function deepEqual(a: unknown, b: unknown): boolean { + if (a === null || a === undefined || b === null || b === undefined) + return a === b; + if (!isIndexedCollection(a) || !isIndexedCollection(b)) + return Object.is(a, b); + if (!isSameCollectionKind(a, b) || a.length !== b.length) return false; + + for (let i = 0; i < a.length; i += 1) { + if (!deepEqual(a[i], b[i])) return false; + } + return true; +} + +/** One bucket for every `NaN`, so `deepEqual([NaN], [NaN])` and their hashes agree. */ +const NAN_HASH = 0x7ff8; + +/** `-0` hashes distinctly from `+0`, which hashes to 0, because CFG-34 makes them unequal. */ +const NEGATIVE_ZERO_HASH = 1; + +function hashNumber(value: number): number { + if (Number.isNaN(value)) return NAN_HASH; + if (Object.is(value, -0)) return NEGATIVE_ZERO_HASH; + return Math.trunc(value) | 0; +} + +function hashString(value: string): number { + let hash = 0; + for (let i = 0; i < value.length; i += 1) { + hash = (hash * 31 + value.charCodeAt(i)) | 0; + } + return hash; +} + +/** + * Hashes a value consistently with {@link deepEqual} (CFG-33): every pair `deepEqual` reports equal + * hashes to the same number. Unequal values MAY collide -- that is what a hash is. + * + * Null-safe: `deepHash(null)` and `deepHash(undefined)` are both `0`. + * + * @param value - the value to hash. + * @returns a 32-bit signed hash. + * + * @internal + */ +export function deepHash(value: unknown): number { + if (value === null || value === undefined) return 0; + if (isIndexedCollection(value)) { + let hash = 17; + // eslint-disable-next-line @typescript-eslint/prefer-for-of -- `ArrayLike` covers typed and plain arrays alike but is not itself iterable, so `for-of` does not type-check here + for (let i = 0; i < value.length; i += 1) { + hash = (hash * 31 + deepHash(value[i])) | 0; + } + return hash; + } + if (typeof value === 'number') return hashNumber(value); + if (typeof value === 'string') return hashString(value); + if (typeof value === 'boolean') return value ? 1231 : 1237; + if (typeof value === 'bigint') return hashString(value.toString()); + // Objects, functions, and symbols compare by identity in `deepEqual`, so a single shared bucket + // keeps the two helpers consistent without inventing a structural hash `deepEqual` would not honor. + return 1; +} diff --git a/packages/core/src/config/http-date.test.ts b/packages/core/src/config/http-date.test.ts new file mode 100644 index 0000000..6015784 --- /dev/null +++ b/packages/core/src/config/http-date.test.ts @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/http-date.test.ts +// Exercises: CFG-29 (canonical formatting -- zero-padded day, literal GMT, UTC), CFG-30 (tolerant +// parsing -- case-insensitive month, zone aliases, informational weekday), CFG-31 (strict on the +// rest -- blank input and a missing post-weekday comma both fail; totality). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import {formatHttpDate, parseHttpDate} from './http-date.js'; + +describe('formatHttpDate (CFG-29)', () => { + test('renders the canonical form in UTC', () => { + const epochMs = Date.UTC(1994, 10, 6, 8, 49, 37); + + expect(formatHttpDate(epochMs)).toBe('Sun, 06 Nov 1994 08:49:37 GMT'); + }); + + test('zero-pads a single-digit day-of-month', () => { + const epochMs = Date.UTC(2026, 0, 1, 0, 0, 0); + + expect(formatHttpDate(epochMs)).toBe('Thu, 01 Jan 2026 00:00:00 GMT'); + }); + + test('rejects an instant outside the four-digit-year range RFC 1123 renders', () => { + // `padStart(4, '0')` cannot render a year outside 0000..9999: year -1 came out as `00-1` and + // year 275760 as `275760`, both malformed HTTP-dates emitted with no error at all. + expect(() => formatHttpDate(-62_198_755_200_000)).toThrow( + InvariantViolation, + ); + expect(() => formatHttpDate(253_402_300_800_000)).toThrow( + InvariantViolation, + ); + expect(() => formatHttpDate(8_640_000_000_000_000)).toThrow( + InvariantViolation, + ); + }); + + test('renders the outermost instants inside that range', () => { + expect(formatHttpDate(-62_167_219_200_000)).toBe( + 'Sat, 01 Jan 0000 00:00:00 GMT', + ); + expect(formatHttpDate(253_402_300_799_000)).toBe( + 'Fri, 31 Dec 9999 23:59:59 GMT', + ); + }); + + test('rejects a non-representable instant as a programmer error', () => { + expect(() => formatHttpDate(Number.NaN)).toThrow(InvariantViolation); + }); +}); + +describe('parseHttpDate tolerance (CFG-30)', () => { + const canonical = Date.UTC(2026, 0, 1, 0, 0, 10); + + test('accepts a canonical HTTP-date', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:10 GMT')).toBe(canonical); + }); + + test('accepts an upper-case month name', () => { + expect(parseHttpDate('Thu, 01 JAN 2026 00:00:10 GMT')).toBe(canonical); + }); + + test('accepts a lower-case month name', () => { + expect(parseHttpDate('Thu, 01 jan 2026 00:00:10 GMT')).toBe(canonical); + }); + + test('normalizes UTC to the same instant as GMT', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:10 UTC')).toBe(canonical); + }); + + test('normalizes +0000 to the same instant as GMT', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:10 +0000')).toBe(canonical); + }); + + test('normalizes +00:00 to the same instant as GMT', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:10 +00:00')).toBe(canonical); + }); + + test('treats the weekday token as informational, even when it contradicts the date', () => { + expect(parseHttpDate('Mon, 01 Jan 2026 00:00:10 GMT')).toBe(canonical); + }); + + test('accepts a single-digit day-of-month', () => { + expect(parseHttpDate('Thu, 1 Jan 2026 00:00:10 GMT')).toBe(canonical); + }); + + test('tolerates surrounding whitespace', () => { + expect(parseHttpDate(' Thu, 01 Jan 2026 00:00:10 GMT ')).toBe(canonical); + }); + + test('rolls a leap second into the following minute rather than rejecting it', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:60 GMT')).toBe( + Date.UTC(2026, 0, 1, 0, 1, 0), + ); + }); +}); + +describe('parseHttpDate strictness (CFG-31)', () => { + test('rejects blank input', () => { + expect(parseHttpDate('')).toBeNull(); + }); + + test('rejects whitespace-only input', () => { + expect(parseHttpDate(' ')).toBeNull(); + }); + + test('rejects a form missing the comma after the weekday', () => { + expect(parseHttpDate('Mon 01 Jan 2024 00:00:00 GMT')).toBeNull(); + }); + + test('rejects an out-of-range day rather than rolling it over', () => { + expect(parseHttpDate('Thu, 32 Jan 2026 00:00:10 GMT')).toBeNull(); + }); + + test('rejects a day that does not exist in the given month', () => { + expect(parseHttpDate('Thu, 31 Feb 2026 00:00:10 GMT')).toBeNull(); + }); + + test('rejects an out-of-range hour', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 24:00:10 GMT')).toBeNull(); + }); + + test('rejects an unknown month name', () => { + expect(parseHttpDate('Thu, 01 Foo 2026 00:00:10 GMT')).toBeNull(); + }); + + test('rejects a non-zero numeric offset', () => { + expect(parseHttpDate('Thu, 01 Jan 2026 00:00:10 +0100')).toBeNull(); + }); + + test('reads a two-digit-looking year literally rather than mapping it onto the 1900s', () => { + const parsed = parseHttpDate('Sun, 01 Jan 0026 00:00:00 GMT'); + + expect(parsed).not.toBeNull(); + expect(new Date(parsed ?? 0).getUTCFullYear()).toBe(26); + }); +}); + +describe('parseHttpDate properties', () => { + test('never throws for an arbitrary string', () => { + fc.assert( + fc.property(fc.string(), value => { + expect(() => parseHttpDate(value)).not.toThrow(); + }), + ); + }); + + test('round-trips every second-precision instant a canonical format produces', () => { + fc.assert( + // The full span `formatHttpDate` accepts, not just the modern slice: year 0000-01-01 through + // 9999-12-31. The old bound stopped at year 2100, so it never reached the negative epochs and + // out-of-range years where the round-trip actually broke. + fc.property( + fc.integer({min: -62_167_219_200_000, max: 253_402_300_799_000}), + epochMs => { + const truncated = Math.floor(epochMs / 1000) * 1000; + + expect(parseHttpDate(formatHttpDate(truncated))).toBe(truncated); + }, + ), + ); + }); +}); diff --git a/packages/core/src/config/http-date.ts b/packages/core/src/config/http-date.ts new file mode 100644 index 0000000..a14afe3 --- /dev/null +++ b/packages/core/src/config/http-date.ts @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/http-date.ts +import {invariant} from '../invariant.js'; + +/** Month abbreviations in canonical casing, indexed by `Date`'s zero-based UTC month. */ +const MONTH_NAMES = [ + 'Jan', + 'Feb', + 'Mar', + 'Apr', + 'May', + 'Jun', + 'Jul', + 'Aug', + 'Sep', + 'Oct', + 'Nov', + 'Dec', +] as const; + +/** Weekday abbreviations in canonical casing, indexed by `Date`'s zero-based UTC day. */ +const WEEKDAY_NAMES = [ + 'Sun', + 'Mon', + 'Tue', + 'Wed', + 'Thu', + 'Fri', + 'Sat', +] as const; + +/** + * The RFC 1123 grammar this parser accepts. The leading weekday group is optional *as a whole*, so + * dropping the comma after it does not degrade into "no weekday" -- the day-of-month group then has + * to match the weekday text and fails, which is exactly CFG-31's missing-comma rejection. + */ +const HTTP_DATE = + /^(?:[a-z]{3,9},\s+)?(\d{1,2})\s+([a-z]{3})\s+(\d{4})\s+(\d{2}):(\d{2}):(\d{2})\s+(?:GMT|UTC|\+00:?00)$/iu; + +/** RFC 1123's `date1` is a four-digit year, so these are the only instants CFG-29 can render. */ +const MIN_HTTP_DATE_YEAR = 0; +const MAX_HTTP_DATE_YEAR = 9999; + +function padTwoDigits(value: number): string { + return value < 10 ? `0${String(value)}` : String(value); +} + +/** + * Formats an instant as the canonical RFC 1123 HTTP-date (CFG-29): always UTC, a zero-padded + * two-digit day-of-month, and a literal `GMT` -- e.g. `Sun, 06 Nov 1994 08:49:37 GMT`. + * + * @param epochMs - the instant, in epoch milliseconds. + * @returns the canonical HTTP-date string. + * @throws an assertion failure (a caller bug, not a catchable condition) when `epochMs` is not a finite number in `Date`'s representable range, + * or falls outside the four-digit-year span RFC 1123 can render -- a programmer error, not a value + * any wire input can produce. + * + * @public + */ +export function formatHttpDate(epochMs: number): string { + const date = new Date(epochMs); + invariant( + !Number.isNaN(date.getTime()), + `formatHttpDate: epochMs must be a representable instant, got ${String(epochMs)}`, + ); + const year = date.getUTCFullYear(); + // A second, narrower bound than `Date`'s own. RFC 1123's `date1` carries a four-digit year, and + // `padStart(4, '0')` cannot render one outside 0000..9999: year -1 came out as `00-1` and year + // 275760 as `275760`, both malformed HTTP-dates emitted with no error at all, and neither + // survived a round-trip back through `parseHttpDate` (CFG-29). + invariant( + year >= MIN_HTTP_DATE_YEAR && year <= MAX_HTTP_DATE_YEAR, + `formatHttpDate: epochMs must fall in the four-digit-year range RFC 1123 renders, got year ${String(year)}`, + ); + // `getUTCDay()` is 0..6 and `getUTCMonth()` 0..11 for the representable date the invariant above + // guarantees, so both lookups always hit; `noUncheckedIndexedAccess` cannot see that. An + // `invariant` rather than a `?? ''` fallback, because a silent empty string here would emit a + // malformed HTTP-date with no error at all -- the same failure the year check above exists to stop. + const weekday = WEEKDAY_NAMES[date.getUTCDay()]; + const month = MONTH_NAMES[date.getUTCMonth()]; + invariant( + weekday !== undefined && month !== undefined, + `formatHttpDate: unreachable -- no weekday or month name for epochMs ${String(epochMs)}`, + ); + const day = padTwoDigits(date.getUTCDate()); + const yearText = String(year).padStart(4, '0'); + const hours = padTwoDigits(date.getUTCHours()); + const minutes = padTwoDigits(date.getUTCMinutes()); + const seconds = padTwoDigits(date.getUTCSeconds()); + return `${weekday}, ${day} ${month} ${yearText} ${hours}:${minutes}:${seconds} GMT`; +} + +/** The already-range-checked UTC fields {@link toEpochMs} assembles into an instant. */ +interface DateFields { + readonly year: number; + readonly month: number; + readonly day: number; + readonly hour: number; + readonly minute: number; + readonly second: number; +} + +/** + * Builds an instant from already-range-checked UTC fields, rejecting any combination `Date` would + * silently roll over (`31 Feb`, `31 Apr`). `Date.UTC` cannot be used directly: it maps a two-digit + * year onto 1900-1999, which would turn `0026` into 1926 without any error. + */ +function toEpochMs(fields: DateFields): number | null { + const {year, month, day, hour, minute, second} = fields; + const date = new Date(0); + date.setUTCFullYear(year, month, day); + const rolledOver = + date.getUTCFullYear() !== year || + date.getUTCMonth() !== month || + date.getUTCDate() !== day; + if (rolledOver) return null; + // Applied only after the calendar check, so a leap second (:60) rolls into the following minute + // without that rollover being misread as an out-of-range day. + date.setUTCHours(hour, minute, second, 0); + return date.getTime(); +} + +/** + * Parses an RFC 1123 HTTP-date, total: any string that is not a valid date yields `null` rather + * than throwing. + * + * Never `Date.parse`/`new Date(string)` -- JS date-string parsing is permissive and non-standardized + * across engines, the opposite of a total parser's contract. + * + * Tolerant (CFG-30) of an informational weekday, which is stripped and never validated against the + * date; a single-digit day; case-insensitive month names; and the zone tokens `GMT`, `UTC`, `+0000`, + * and `+00:00`, which all normalize to a zero offset. Strict on the rest (CFG-31): blank input and a + * missing comma after the weekday both fail, and every field is range-checked so an out-of-range + * value is rejected rather than silently rolled over into a valid but wrong instant. + * + * @param raw - the candidate HTTP-date; surrounding whitespace is tolerated. + * @returns the instant in epoch milliseconds, or `null` when `raw` is not a valid HTTP-date. + * + * @public + */ +export function parseHttpDate(raw: string): number | null { + const match = HTTP_DATE.exec(raw.trim()); + if (match === null) return null; + + const day = Number(match[1]); + const monthText = (match[2] ?? '').toLowerCase(); + const month = MONTH_NAMES.findIndex( + candidate => candidate.toLowerCase() === monthText, + ); + const year = Number(match[3]); + const hour = Number(match[4]); + const minute = Number(match[5]); + const second = Number(match[6]); + + // A leap second (:60) is accepted and normalizes into the following minute; every other field is + // a hard range check. + if (month < 0 || day < 1 || hour > 23 || minute > 59 || second > 60) + return null; + return toEpochMs({year, month, day, hour, minute, second}); +} diff --git a/packages/core/src/config/identifiers.test.ts b/packages/core/src/config/identifiers.test.ts new file mode 100644 index 0000000..f546504 --- /dev/null +++ b/packages/core/src/config/identifiers.test.ts @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/identifiers.test.ts +// Exercises: CFG-32 (type-4 UUID with the RFC 4122 version-4/IETF-variant layout, a large batch free +// of collisions, and a named failure when the runtime exposes no WebCrypto). +// CFG-32's concurrency clause has no test: `randomUuid` is synchronous and holds no state, so there +// is nothing to interleave and no assertion that would fail if there were. The argument is carried +// on `randomUuid`'s TSDoc instead. A `Promise.all` over synchronous calls, which is what a test here +// would be, asserts only that `Promise.all` works. +import {describe, expect, test} from 'bun:test'; +import {InvariantViolation} from '../invariant.js'; +import {randomUuid, randomUuidFrom} from './identifiers.js'; + +const UUID_V4 = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/u; + +describe('randomUuid (CFG-32)', () => { + test('produces the RFC 4122 version-4, IETF-variant layout', () => { + expect(randomUuid()).toMatch(UUID_V4); + }); + + test('produces 36 characters with hyphens at the canonical offsets', () => { + const id = randomUuid(); + + expect(id).toHaveLength(36); + expect([id[8], id[13], id[18], id[23]]).toEqual(['-', '-', '-', '-']); + }); + + test('produces no collisions across a large batch', () => { + const seen = new Set<string>(); + + for (let i = 0; i < 10_000; i += 1) seen.add(randomUuid()); + + expect(seen.size).toBe(10_000); + }); + + test('names the missing dependency when the runtime exposes no WebCrypto', () => { + // The random source is passed in rather than read off `globalThis`, so this branch is reachable + // without deleting or reassigning a global -- which would break parallel execution + // (`docs/knowledge/harvested/testing.md:50`). Reading `getRandomValues` off `undefined` would otherwise + // report only `TypeError: Cannot read properties of undefined`. + expect(() => randomUuidFrom(undefined)).toThrow(InvariantViolation); + expect(() => randomUuidFrom(undefined)).toThrow(/globalThis\.crypto/u); + expect(() => randomUuidFrom(undefined)).toThrow(/20\.3/u); + }); + + test('names the missing dependency when WebCrypto carries no getRandomValues', () => { + expect(() => randomUuidFrom({} as unknown as Crypto)).toThrow( + InvariantViolation, + ); + }); +}); diff --git a/packages/core/src/config/identifiers.ts b/packages/core/src/config/identifiers.ts new file mode 100644 index 0000000..a6948f5 --- /dev/null +++ b/packages/core/src/config/identifiers.ts @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/identifiers.ts +import {invariant} from '../invariant.js'; + +const HEX_BY_BYTE: readonly string[] = Array.from({length: 256}, (_, byte) => + byte.toString(16).padStart(2, '0'), +); + +/** + * {@link randomUuid}'s body, with the random source taken explicitly. + * + * Parameterized for the same reason `build-info.ts`'s `detectRuntimeIdentity(host)` is: the + * missing-WebCrypto branch is then reachable from a test without deleting or reassigning a global, + * which no test may do (`docs/knowledge/harvested/testing.md:50` -- every test must survive parallel + * execution). + * + * @param webCrypto - the WebCrypto implementation, or `undefined` on a runtime that exposes none. + * @returns a lower-case, hyphenated 36-character UUID. + * @throws InvariantViolation when `webCrypto` is absent or carries no `getRandomValues`. + * + * @internal + */ +export function randomUuidFrom(webCrypto: Crypto | undefined): string { + invariant( + typeof webCrypto?.getRandomValues === 'function', + 'randomUuid: globalThis.crypto.getRandomValues is unavailable. @dexpace/core needs WebCrypto exposed as a global -- Node >= 20.3 (see engines.node), or any browser/Workers runtime.', + ); + + const bytes = new Uint8Array(16); + webCrypto.getRandomValues(bytes); + bytes[6] = ((bytes[6] ?? 0) & 0x0f) | 0x40; + bytes[8] = ((bytes[8] ?? 0) & 0x3f) | 0x80; + + const hex = Array.from(bytes, byte => HEX_BY_BYTE[byte] ?? '00').join(''); + return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`; +} + +/** + * Generates a type-4 UUID with the RFC 4122 layout -- version 4 in the high nibble of byte 6, the + * IETF variant in the high bits of byte 8 (CFG-32). + * + * Callers MUST treat the output as **non-cryptographic** despite the CSPRNG source: it is an + * identifier for correlation, not a secret or a capability token. + * + * Concurrency-safe by construction -- no shared mutable state here, and none in + * `crypto.getRandomValues` either. `globalThis.crypto` is core's already-fixed cross-runtime + * primitive; a `node:crypto` import would break the browser/Workers half of the runtime floor. + * + * @returns a lower-case, hyphenated 36-character UUID. + * @throws an assertion failure (a caller bug, not a catchable condition) when the runtime exposes no global WebCrypto to draw from -- a + * deployment error, reported by name rather than as the bare `TypeError` that reading + * `getRandomValues` off `undefined` would otherwise produce. + * + * @public + */ +export function randomUuid(): string { + // The lib types declare `globalThis.crypto` non-nullable, but `randomUuidFrom`'s guard exists + // precisely for the runtime where it is not: a Node release below `engines.node`'s 20.3 floor, + // where WebCrypto is absent from ESM, or an embedder that withholds it. Declaring the parameter + // `Crypto | undefined` is what keeps that guard reachable -- the compiler cannot see that runtime, + // so the check has to survive a type saying it cannot happen. + return randomUuidFrom(globalThis.crypto); +} diff --git a/packages/core/src/config/proxy.test.ts b/packages/core/src/config/proxy.test.ts new file mode 100644 index 0000000..4781890 --- /dev/null +++ b/packages/core/src/config/proxy.test.ts @@ -0,0 +1,877 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/proxy.test.ts +// Exercises: CFG-22 (immutable model, credential-masking string form via formatProxyOptions and the +// own toString the factory attaches), CFG-23 (glob bypass -- full string, case-insensitive, +// metacharacters literal, compiled once at construction), CFG-24 (property layer ahead of +// environment, HTTPS ahead of HTTP, port from the host's own layer, https-only credentials, never +// throws), CFG-25 (explicit in-range port, no guessing), CFG-26 (separator escape and token order), +// CFG-27 (a bare "*" is bypass-all), CFG-28 (nothing reads the environment implicitly). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import type {Configuration} from './configuration.js'; +import {ConfigurationBuilder} from './configuration.js'; +import type {ProxyOptions} from './proxy.js'; +import { + compiledGlobs, + createProxyOptions, + formatProxyOptions, + globsFor, + resolveProxyOptions, + shouldBypassProxy, +} from './proxy.js'; + +/** + * `ProxyOptions` deliberately declares no `toString(): string` member -- every object satisfies one + * through `Object.prototype`, so the declaration could not have enforced CFG-22's masking anyway, and + * it would have forced `Omit`/`Pick` gymnastics through the public API. What the factory ships is an + * *own* `toString` delegating to `formatProxyOptions`. Reaching it therefore needs one explicit + * widening, kept here so the asymmetry is stated once rather than cast at six call sites. + * + * A pure type widening: it asserts nothing about the argument, and is applied below to a hand-built + * literal that provably has no own `toString`. {@link hasOwnToString} is the one that checks. + */ +function asStringable(options: ProxyOptions): ProxyOptions & { + toString(): string; +} { + return options; +} + +/** Whether `options` carries its own `toString` at all, as distinct from inheriting Object's. */ +function hasOwnToString(options: ProxyOptions): boolean { + return Object.hasOwn(options, 'toString'); +} + +/** + * A fresh bypass-options value per test. Never a shared `const` at describe scope: the array is + * mutable, and `proxy.ts` keys its compiled-glob cache by that exact array *identity*, so a shared + * fixture would be shared into module-level state that outlives the describe + * (`docs/knowledge/harvested/testing.md:52`, `:50`). + */ +function bypassOptions( + ...nonProxyHosts: string[] +): Pick<ProxyOptions, 'bypassAll' | 'nonProxyHosts'> { + return {bypassAll: false, nonProxyHosts}; +} + +function configWith( + env: Record<string, string> = {}, + properties: Record<string, string> = {}, +): Configuration { + return new ConfigurationBuilder() + .withEnvSource(key => env[key]) + .withPropertySource(key => properties[key]) + .build(); +} + +/** + * Resolves a configuration that must yield a proxy. A regression to `null` then names itself, rather + * than reaching the assertion as an empty string and reporting a confusing value mismatch. + */ +function resolvedProxy(config: Configuration): ProxyOptions { + const options = resolveProxyOptions(config); + if (options === null) { + throw new Error( + 'expected the proxy to resolve, but resolution returned null', + ); + } + return options; +} + +describe('createProxyOptions (CFG-22)', () => { + test('masks credentials in its string form', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + credentials: {username: 'user', password: 'secret'}, + }); + + expect(asStringable(options).toString()).toBe( + 'http://***:***@proxy.example.com:8080', + ); + }); + + test('leaks neither the username nor the password through string interpolation', () => { + const options = createProxyOptions({ + type: 'socks5', + host: 'proxy.example.com', + port: 1080, + credentials: {username: 'user', password: 'secret'}, + }); + + const rendered = String(asStringable(options)); + + expect(rendered).not.toContain('secret'); + expect(rendered).not.toContain('user'); + }); + + test('renders no credential segment when there are none', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + }); + + expect(asStringable(options).toString()).toBe( + 'http://proxy.example.com:8080', + ); + }); + + test('is frozen, with a frozen non-proxy host list', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + nonProxyHosts: ['*.internal'], + }); + + expect(Object.isFrozen(options)).toBe(true); + expect(Object.isFrozen(options.nonProxyHosts)).toBe(true); + }); + + test('copies the non-proxy host list rather than aliasing the caller array', () => { + const patterns = ['*.internal']; + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + nonProxyHosts: patterns, + }); + + patterns.push('*.other'); + + expect(options.nonProxyHosts).toEqual(['*.internal']); + }); +}); + +describe('createProxyOptions bypass-all (CFG-22, CFG-27)', () => { + test('carries a caller-set bypass-all flag through to the built value', () => { + // `resolveProxyOptions` always passes `false` (CFG-27 turns bypass-all into a `null` resolution), + // so without this the factory could ignore `init.bypassAll` entirely and stay green. + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + bypassAll: true, + }); + + expect(options.bypassAll).toBe(true); + expect(shouldBypassProxy(options, 'anything.example.com')).toBe(true); + }); + + test('defaults bypass-all to false when the caller omits it', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + }); + + expect(options.bypassAll).toBe(false); + }); +}); + +describe('formatProxyOptions (CFG-22)', () => { + test('agrees with the own toString a factory-built instance carries', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + credentials: {username: 'user', password: 'secret'}, + }); + + // One masking implementation: the instance's `toString` delegates here rather than restating it. + expect(hasOwnToString(options)).toBe(true); + expect(asStringable(options).toString()).toBe(formatProxyOptions(options)); + expect(formatProxyOptions(options)).toBe( + 'http://***:***@proxy.example.com:8080', + ); + }); + + test('renders the same masked form whatever the credentials contain', () => { + // CFG-22's masking law. The two fixed-credential tests above prove the shape; this proves the + // guarantee is about every username and password rather than about `user`/`secret`. Asserted as + // an exact match rather than `not.toContain`, which a one-character username would defeat -- + // `'p'` appears in `proxy.example.com` no matter how well the masking works. + fc.assert( + fc.property(fc.string(), fc.string(), (username, password) => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + credentials: {username, password}, + }); + + expect(formatProxyOptions(options)).toBe( + 'http://***:***@proxy.example.com:8080', + ); + }), + ); + }); + + test('is the masking contract for a hand-built options object, which carries no toString obligation', () => { + // `ProxyOptions` deliberately declares no `toString(): string` member -- every object satisfies + // one through `Object.prototype`, so the declaration could not have enforced masking anyway. + // A literal therefore type-checks, renders as an ordinary object, and leaks nothing; the masked + // form comes from the free function. + const options: ProxyOptions = { + type: 'socks5', + host: 'proxy.example.com', + port: 1080, + nonProxyHosts: [], + credentials: {username: 'user', password: 'secret'}, + bypassAll: false, + }; + + expect(hasOwnToString(options)).toBe(false); + expect(String(asStringable(options))).toBe('[object Object]'); + expect(String(asStringable(options))).not.toContain('secret'); + expect(formatProxyOptions(options)).toBe( + 'socks5://***:***@proxy.example.com:1080', + ); + }); +}); + +describe('shouldBypassProxy (CFG-23)', () => { + test('matches a subdomain case-insensitively', () => { + const options = bypassOptions('*.internal.example.com'); + + expect(shouldBypassProxy(options, 'API.internal.example.com')).toBe(true); + }); + + test('does not match the apex domain', () => { + const options = bypassOptions('*.internal.example.com'); + + expect(shouldBypassProxy(options, 'internal.example.com')).toBe(false); + }); + + test('requires a full-string match', () => { + const options = bypassOptions('*.internal.example.com'); + + expect(shouldBypassProxy(options, 'a.internal.example.com.evil.test')).toBe( + false, + ); + }); + + test('treats a dot in the pattern literally', () => { + expect(shouldBypassProxy(bypassOptions('a.b'), 'axb')).toBe(false); + }); + + test('treats ? as exactly one character', () => { + const single = bypassOptions('ho?t.example.com'); + + expect(shouldBypassProxy(single, 'host.example.com')).toBe(true); + expect(shouldBypassProxy(single, 'hoost.example.com')).toBe(false); + }); + + test('escapes regex metacharacters rather than honoring them', () => { + const metacharacters = bypassOptions('a+b'); + + expect(shouldBypassProxy(metacharacters, 'aab')).toBe(false); + expect(shouldBypassProxy(metacharacters, 'a+b')).toBe(true); + }); + + test('short-circuits on bypass-all regardless of the glob list', () => { + expect( + shouldBypassProxy( + {bypassAll: true, nonProxyHosts: []}, + 'anything.example.com', + ), + ).toBe(true); + }); + + test('returns false for an empty glob list', () => { + expect(shouldBypassProxy(bypassOptions(), 'example.com')).toBe(false); + }); +}); + +describe('bypass glob compilation (CFG-23)', () => { + test('compiles the glob list at construction, before any bypass decision', () => { + const options = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + nonProxyHosts: ['*.internal.example.com'], + }); + + // Read before any `shouldBypassProxy` call. That is the whole assertion: a lazy `globsFor` on + // first use produces identical bypass answers, so without reading the cache here the + // construction-time compile could be deleted outright and every other test would stay green. + expect(compiledGlobs.has(options.nonProxyHosts)).toBe(true); + }); + + test('returns the very same compiled list on a second lookup rather than recompiling', () => { + const patterns = Object.freeze(['*.internal.example.com']); + + const first = globsFor(patterns); + const second = globsFor(patterns); + + // Identity, not equality: `patterns.map(...)` twice is deeply equal but not the same array, so + // only `toBe` can tell a cache hit from a recompile. + expect(second).toBe(first); + }); + + test('answers bypass decisions consistently across repeated calls for one list', () => { + const shared = createProxyOptions({ + type: 'http', + host: 'proxy.example.com', + port: 8080, + nonProxyHosts: ['*.internal.example.com'], + }); + + expect(shouldBypassProxy(shared, 'a.internal.example.com')).toBe(true); + expect(shouldBypassProxy(shared, 'b.internal.example.com')).toBe(true); + }); +}); + +describe('shouldBypassProxy glob dialect edges (CFG-23)', () => { + test('treats a backslash as an ordinary literal, since the dialect defines no escape', () => { + const escaped = {bypassAll: false, nonProxyHosts: ['a\\*b']}; + + expect(shouldBypassProxy(escaped, 'a\\*b')).toBe(true); + expect(shouldBypassProxy(escaped, 'a*b')).toBe(false); + expect(shouldBypassProxy(escaped, 'aXXXb')).toBe(false); + }); + + test('matches a star-dense pattern in linear time rather than backtracking', () => { + // A legal `NO_PROXY` entry. Under the previous `*` -> `.*` regex this pattern against this host + // drove catastrophic backtracking for 38 seconds of blocked event loop; the two-pointer walk + // that replaced it settles in microseconds. + const dense = {bypassAll: false, nonProxyHosts: ['*a*a*a*a*a*a*a*a*a*b']}; + const started = performance.now(); + + expect(shouldBypassProxy(dense, 'a'.repeat(60))).toBe(false); + + expect(performance.now() - started).toBeLessThan(50); + }); +}); + +describe('resolveProxyOptions from the environment (CFG-24, CFG-25)', () => { + test('prefers HTTPS_PROXY over HTTP_PROXY', () => { + const options = resolveProxyOptions( + configWith({ + HTTPS_PROXY: 'https://secure.example.com:9090', + HTTP_PROXY: 'http://plain.example.com:8080', + }), + ); + + expect(options?.host).toBe('secure.example.com'); + expect(options?.port).toBe(9090); + }); + + test('falls back to HTTP_PROXY when HTTPS_PROXY is absent', () => { + const options = resolveProxyOptions( + configWith({HTTP_PROXY: 'http://plain.example.com:8080'}), + ); + + expect(options?.host).toBe('plain.example.com'); + }); + + test('reads percent-decoded credentials out of the URL', () => { + const options = resolveProxyOptions( + configWith({HTTPS_PROXY: 'http://us%40er:p%3Ass@proxy.example.com:8080'}), + ); + + expect(options?.credentials).toEqual({username: 'us@er', password: 'p:ss'}); + }); + + test('maps a socks5 scheme to the socks5 proxy type', () => { + const options = resolveProxyOptions( + configWith({HTTPS_PROXY: 'socks5://proxy.example.com:1080'}), + ); + + expect(options?.type).toBe('socks5'); + }); + + test('rejects an unknown scheme', () => { + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'ftp://proxy.example.com:21'}), + ), + ).toBeNull(); + }); + + test('rejects a proxy URL with no port, never guessing a default', () => { + expect( + resolveProxyOptions(configWith({HTTPS_PROXY: 'https://example.com'})), + ).toBeNull(); + }); + + test('rejects an out-of-range port', () => { + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'https://example.com:70000'}), + ), + ).toBeNull(); + }); +}); + +describe('resolveProxyOptions rejection paths (CFG-24, CFG-25)', () => { + test('resolves to null when no proxy is configured', () => { + expect(resolveProxyOptions(configWith())).toBeNull(); + }); + + test('resolves to null on malformed input, without throwing', () => { + const config = configWith({HTTPS_PROXY: 'not a url at all'}); + + expect(() => resolveProxyOptions(config)).not.toThrow(); + expect(resolveProxyOptions(config)).toBeNull(); + }); + + test('never throws for an arbitrary environment value', () => { + fc.assert( + fc.property(fc.string(), value => { + expect(() => + resolveProxyOptions(configWith({HTTPS_PROXY: value})), + ).not.toThrow(); + }), + ); + }); + + test('never throws for an arbitrary URL-shaped environment value', () => { + // A bare `fc.string()` essentially never produces something the URL parser accepts, so it could + // not reach the credential decode at all -- it missed a `URIError` escaping on a lone `%`. This + // generator assembles values that parse, then perturbs the pieces most likely to break. + const piece = fc.stringMatching(/^[\w%.:+~@-]{0,12}$/u); + const parts = fc.record({ + scheme: fc.constantFrom('http', 'https', 'socks5', 'ftp', 'zz'), + user: piece, + password: piece, + host: piece, + port: fc.constantFrom( + '', + ':0', + ':80', + ':443', + ':8080', + ':65535', + ':70000', + ':x', + ), + }); + + fc.assert( + fc.property(parts, ({scheme, user, password, host, port}) => { + const credentials = user === '' ? '' : `${user}:${password}@`; + const value = `${scheme}://${credentials}${host}${port}`; + + expect(() => + resolveProxyOptions(configWith({HTTPS_PROXY: value})), + ).not.toThrow(); + }), + ); + }); +}); + +describe("CFG-24's WARNING half: a rejected proxy URL is audible", () => { + async function collectWarnings( + run: () => void, + ): Promise<Map<string, unknown>[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + run(); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events.filter(e => e.get('event') === 'http.proxy.configRejected'); + } + + test('an unparseable URL warns, naming the variable and the reason', async () => { + const rejected = await collectWarnings(() => { + expect( + resolveProxyOptions(configWith({HTTPS_PROXY: 'not a url at all'})), + ).toBeNull(); + }); + expect(rejected).toHaveLength(1); + expect(rejected[0]?.get('source')).toBe('HTTPS_PROXY'); + expect(rejected[0]?.get('reason')).toBe('unparseable'); + }); + + test('an absent port warns, naming the variable and the reason (CFG-25)', async () => { + const rejected = await collectWarnings(() => { + expect( + resolveProxyOptions(configWith({HTTP_PROXY: 'http://p.example.com'})), + ).toBeNull(); + }); + expect(rejected).toHaveLength(1); + expect(rejected[0]?.get('source')).toBe('HTTP_PROXY'); + expect(rejected[0]?.get('reason')).toBe('port'); + }); + + test('an unsupported scheme warns, naming the variable and the reason', async () => { + const rejected = await collectWarnings(() => { + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'ftp://p.example.com:21'}), + ), + ).toBeNull(); + }); + expect(rejected).toHaveLength(1); + expect(rejected[0]?.get('reason')).toBe('scheme'); + }); + + test('a well-formed proxy URL warns about nothing', async () => { + const rejected = await collectWarnings(() => { + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'http://p.example.com:8080'}), + ), + ).not.toBeNull(); + }); + expect(rejected).toHaveLength(0); + }); +}); + +describe('resolveProxyOptions URL-layer edges (CFG-24, CFG-25)', () => { + test('honors an explicitly written default port that the URL parser normalizes away', () => { + // `new URL('http://p:80').port` is `''` -- the WHATWG parser folds a special scheme's default + // port away, making `http://p:80` indistinguishable from `http://p` by that field alone. CFG-25 + // bans *guessing* an absent port, not discarding one the operator wrote, and `:80` / `:443` are + // the two most common proxy configurations there are. + expect( + resolveProxyOptions(configWith({HTTP_PROXY: 'http://p.example.com:80'})) + ?.port, + ).toBe(80); + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'https://p.example.com:443'}), + )?.port, + ).toBe(443); + }); + + test('rejects a port above the 0..65535 range even when the URL parser rejects it first', () => { + expect( + resolveProxyOptions( + configWith({HTTP_PROXY: 'http://p.example.com:70000'}), + ), + ).toBeNull(); + }); + + test('resolves an IPv6 literal to a bare address, without the URL parser brackets', () => { + expect( + resolveProxyOptions(configWith({HTTP_PROXY: 'http://[2001:db8::1]:8080'})) + ?.host, + ).toBe('2001:db8::1'); + }); + + test('re-brackets an IPv6 host when rendering, so address and port stay separable', () => { + const options = resolvedProxy( + configWith({HTTP_PROXY: 'http://[2001:db8::1]:8080'}), + ); + + expect(formatProxyOptions(options)).toBe('http://[2001:db8::1]:8080'); + }); + + test('leaves a registered name unbracketed when rendering', () => { + const options = resolvedProxy( + configWith({HTTP_PROXY: 'http://p.example.com:8080'}), + ); + + expect(formatProxyOptions(options)).toBe('http://p.example.com:8080'); + }); + + test('treats an empty user name in the URL as no credentials at all', () => { + expect( + resolveProxyOptions( + configWith({HTTP_PROXY: 'http://:secret@p.example.com:8080'}), + )?.credentials, + ).toBeUndefined(); + }); + + test('resolves to null rather than throwing on an unescaped percent in the credentials', () => { + // `decodeURIComponent('pa%ss')` raises `URIError`, and a literal `%` in a proxy password is + // ordinary operator input, so CFG-24's never-throw clause has to cover the decode too. + const config = configWith({ + HTTP_PROXY: 'http://u:pa%ss@h.example.com:8080', + }); + + expect(() => resolveProxyOptions(config)).not.toThrow(); + expect(resolveProxyOptions(config)).toBeNull(); + }); +}); + +describe('resolveProxyOptions from the property layer (CFG-24)', () => { + test('prefers the property layer over the environment', () => { + const options = resolveProxyOptions( + configWith( + {HTTPS_PROXY: 'https://from-env.example.com:9090'}, + { + 'https.proxyHost': 'from-property.example.com', + 'https.proxyPort': '3128', + }, + ), + ); + + expect(options?.host).toBe('from-property.example.com'); + expect(options?.port).toBe(3128); + }); + + test('prefers https.proxyHost over http.proxyHost', () => { + const options = resolveProxyOptions( + configWith( + {}, + { + 'https.proxyHost': 'secure.example.com', + 'https.proxyPort': '3128', + 'http.proxyHost': 'plain.example.com', + 'http.proxyPort': '8080', + }, + ), + ); + + expect(options?.host).toBe('secure.example.com'); + expect(options?.port).toBe(3128); + }); + + test('takes the port from the same layer as the chosen host, never the other one', () => { + const options = resolveProxyOptions( + configWith( + {}, + {'https.proxyHost': 'secure.example.com', 'http.proxyPort': '8080'}, + ), + ); + + expect(options).toBeNull(); + }); +}); + +describe('resolveProxyOptions property credentials (CFG-24)', () => { + test('reads credentials only from the https properties, even when the host came from http', () => { + const options = resolveProxyOptions( + configWith( + {}, + { + 'http.proxyHost': 'plain.example.com', + 'http.proxyPort': '8080', + 'https.proxyUser': 'user', + 'https.proxyPassword': 'secret', + }, + ), + ); + + expect(options?.host).toBe('plain.example.com'); + expect(options?.credentials).toEqual({ + username: 'user', + password: 'secret', + }); + }); + + test('resolves to null when the property host has no port', () => { + expect( + resolveProxyOptions( + configWith({}, {'https.proxyHost': 'secure.example.com'}), + ), + ).toBeNull(); + }); +}); + +describe('resolveProxyOptions property-layer edges (CFG-24, CFG-25)', () => { + test('rejects every numeric-literal form that is not a bare run of digits', () => { + // `Number()` alone accepts all of these, each of which would silently connect to a port the + // operator never wrote (CFG-25's "non-numeric" clause, read the way `getInt` reads it). + for (const port of [ + '0x10', + '0b11', + '0o17', + '1e2', + '1e3', + '80.0', + '+80', + '-80', + '8_0', + '1.5', + '.5', + 'Infinity', + 'NaN', + '', + ' ', + '65536', + ]) { + expect( + resolveProxyOptions( + configWith( + {}, + {'http.proxyHost': 'p.example.com', 'http.proxyPort': port}, + ), + ), + ).toBeNull(); + } + }); + + test('accepts a bare run of digits, surrounding whitespace tolerated', () => { + for (const [port, expected] of [ + ['80', 80], + [' 8080 ', 8080], + ['0', 0], + ['65535', 65_535], + ] as const) { + expect( + resolveProxyOptions( + configWith( + {}, + {'http.proxyHost': 'p.example.com', 'http.proxyPort': port}, + ), + )?.port, + ).toBe(expected); + } + }); +}); + +describe('non-proxy host resolution (CFG-26, CFG-27)', () => { + test('splits NO_PROXY on commas', () => { + const options = resolveProxyOptions( + configWith({ + HTTPS_PROXY: 'https://example.com:8080', + NO_PROXY: 'a.test,b.test', + }), + ); + + expect(options?.nonProxyHosts).toEqual(['a.test', 'b.test']); + }); + + test('honors a backslash-escaped comma in NO_PROXY', () => { + const options = resolveProxyOptions( + configWith({ + HTTPS_PROXY: 'https://example.com:8080', + NO_PROXY: String.raw`a\,b,c`, + }), + ); + + expect(options?.nonProxyHosts).toEqual(['a,b', 'c']); + }); + + test('lets the pipe-separated property list win over NO_PROXY', () => { + const options = resolveProxyOptions( + configWith( + {HTTPS_PROXY: 'https://example.com:8080', NO_PROXY: 'from-env.test'}, + {'http.nonProxyHosts': String.raw`a\|b|c`}, + ), + ); + + expect(options?.nonProxyHosts).toEqual(['a|b', 'c']); + }); +}); + +describe('non-proxy host token order (CFG-26, CFG-27)', () => { + test('retains a whitespace-only fragment as an empty token, dropping only empty ones', () => { + const options = resolveProxyOptions( + configWith({HTTPS_PROXY: 'https://example.com:8080', NO_PROXY: 'a,, ,c'}), + ); + + expect(options?.nonProxyHosts).toEqual(['a', '', 'c']); + }); + + test('trims surrounding whitespace from each token', () => { + const options = resolveProxyOptions( + configWith({ + HTTPS_PROXY: 'https://example.com:8080', + NO_PROXY: ' a.test , b.test ', + }), + ); + + expect(options?.nonProxyHosts).toEqual(['a.test', 'b.test']); + }); + + test('treats a bare "*" as bypass-all, resolving to null so the caller routes directly', () => { + expect( + resolveProxyOptions( + configWith({HTTPS_PROXY: 'https://example.com:8080', NO_PROXY: '*'}), + ), + ).toBeNull(); + }); + + test('treats a "*" among several entries as an ordinary any-host glob', () => { + const options = resolveProxyOptions( + configWith({ + HTTPS_PROXY: 'https://example.com:8080', + NO_PROXY: '*,x.test', + }), + ); + + expect(options?.nonProxyHosts).toEqual(['*', 'x.test']); + }); +}); + +describe('non-proxy host escape round-trip (CFG-26)', () => { + test('returns every token whole, whatever separators it contains', () => { + // CFG-26's escape law: a token written with its separators escaped comes back exactly as written. + // The fixed cases above pin one escaped comma; this pins the round-trip over arbitrary tokens, + // including ones that are all separators. At least two tokens, so a lone `*` cannot turn the + // resolution into CFG-27's bypass-all and return `null`. + const token = fc.stringMatching(/^[a-z,.*-]{1,10}$/u); + + fc.assert( + fc.property(fc.array(token, {minLength: 2, maxLength: 4}), tokens => { + const encoded = tokens + .map(one => one.replaceAll(',', String.raw`\,`)) + .join(','); + + const options = resolvedProxy( + configWith({ + HTTPS_PROXY: 'https://example.com:8080', + NO_PROXY: encoded, + }), + ); + + expect(options.nonProxyHosts).toEqual(tokens); + }), + ); + }); +}); + +describe('implicit reads (CFG-28)', () => { + test('consults the environment seam only when the resolver is invoked', () => { + let reads = 0; + const config = new ConfigurationBuilder() + .withEnvSource(key => { + reads += 1; + return key === 'HTTPS_PROXY' ? 'https://example.com:8080' : undefined; + }) + .build(); + + expect(reads).toBe(0); + + resolveProxyOptions(config); + + expect(reads).toBeGreaterThan(0); + }); +}); + +describe('proxy endpoint normalization across both CFG-24 tiers', () => { + test('resolves an IPv6 literal to the same bare address the environment layer produces', () => { + const fromProperty = resolvedProxy( + configWith( + {}, + {'http.proxyHost': '[2001:db8::1]', 'http.proxyPort': '8080'}, + ), + ); + const fromEnvironment = resolvedProxy( + configWith({HTTP_PROXY: 'http://[2001:db8::1]:8080'}), + ); + + expect(fromProperty.host).toBe('2001:db8::1'); + expect(fromProperty.host).toBe(fromEnvironment.host); + }); + + test('treats an empty https.proxyUser as no credentials, matching the URL layer', () => { + expect( + resolveProxyOptions( + configWith( + {}, + { + 'http.proxyHost': 'p.example.com', + 'http.proxyPort': '8080', + 'https.proxyUser': '', + 'https.proxyPassword': 'secret', + }, + ), + )?.credentials, + ).toBeUndefined(); + }); +}); diff --git a/packages/core/src/config/proxy.ts b/packages/core/src/config/proxy.ts new file mode 100644 index 0000000..85ddd9b --- /dev/null +++ b/packages/core/src/config/proxy.ts @@ -0,0 +1,567 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/proxy.ts +import type {Configuration} from './configuration.js'; +import { + CFG_KEY_HTTPS_PROXY, + CFG_KEY_HTTP_PROXY, + CFG_KEY_NO_PROXY, +} from './configuration.js'; +import {getGlobalLogger} from '../observability/logger.js'; + +/** Property-layer keys, read raw so their camelCase survives (CFG-4, CFG-24, CFG-26). */ +const PROPERTY_KEYS = { + httpsHost: 'https.proxyHost', + httpsPort: 'https.proxyPort', + httpHost: 'http.proxyHost', + httpPort: 'http.proxyPort', + httpsUser: 'https.proxyUser', + httpsPassword: 'https.proxyPassword', + nonProxyHosts: 'http.nonProxyHosts', +} as const; + +const MAX_PORT = 65_535; + +/** + * A bare run of decimal digits. Bare `Number()` also accepts `0x10`, `0b11`, `0o17`, `1e2`, `80.0`, + * and `+80`, each of which would silently connect to a port the operator never wrote (CFG-25). + * + * Deliberately *stricter* than the integer grammar `configuration.ts` applies to `getInt`, which + * permits the leading `+`/`-` sign that a port cannot carry. + */ +const DECIMAL_DIGITS = /^\d+$/u; + +/** The proxy transports CFG-22 enumerates. @public */ +export type ProxyType = 'http' | 'socks4' | 'socks5'; + +/** Optional proxy credentials -- a documented-nullable slot, exempt from CFG-37. @public */ +export interface ProxyCredentials { + /** The proxy user name, already percent-decoded. */ + readonly username: string; + /** The proxy password, already percent-decoded. Never rendered by `toString`. */ + readonly password: string; +} + +/** + * An immutable proxy configuration (CFG-22): transport type, socket address, the ordered non-proxy + * host glob list, optional credentials, an optional challenge-handler slot, and an explicit + * bypass-all flag. + * + * `@dexpace/transport-undici` is the only consumer: it hands `credentials` to undici's `ProxyAgent` + * constructor and probes `challengeHandler` for the two TRANSPORT-30 warnings. `transport-fetch` + * ships no `proxy` option at all, because Node's bare global `fetch` exposes no proxy hook -- a + * deliberate scope boundary, audited as `docs/deviations.md` item 13. + * + * CFG-22's credential-masking string rendering is {@link formatProxyOptions}, a free function rather + * than a `toString(): string` member: every object satisfies such a member through + * `Object.prototype`, so declaring it here would state a contract the type cannot enforce while + * forcing every other signature in this module to `Omit` it back out. A `createProxyOptions` instance + * additionally carries an own `toString` that delegates to the same function, so interpolating one + * into a log masks; a hand-built object literal has no such obligation and renders as an ordinary + * object -- which leaks nothing either. + * + * @public + */ +export interface ProxyOptions { + /** The proxy transport. */ + readonly type: ProxyType; + /** + * The proxy host name or literal address, always **bare**: an IPv6 literal carries no `[...]` + * brackets whichever CFG-24 layer supplied it, so a transport joining `host` and `port` never has + * to know which one answered. `URL.hostname` brackets one and the `https.proxyHost` property does + * not; resolution normalizes both to this form. A consumer composing a URL authority re-adds them. + */ + readonly host: string; + /** The proxy port, always explicit and within 0..65535 -- never guessed from the scheme (CFG-25). */ + readonly port: number; + /** Ordered glob patterns for hosts that bypass this proxy (CFG-23). */ + readonly nonProxyHosts: readonly string[]; + /** Proxy credentials, when the configuration supplied them. */ + readonly credentials?: ProxyCredentials | undefined; + /** + * The optional challenge-handler slot CFG-22's field list requires (a MUST, + * `docs/product-spec/16-configuration.md`). **Nothing dispatches through it, and nothing is going + * to.** That disposition is settled. + * + * undici's `ProxyAgent` takes its credential solely from its own constructor and rejects a + * per-request `Proxy-Authorization` with `InvalidArgumentError`, a deliberate upstream security + * fix. The constructor runs before any challenge exists, so a handler-minted credential can never + * reach the exchange that provoked it. The full audit is `docs/deviations.md` item 13. + * + * `@dexpace/transport-undici` answers TRANSPORT-30's discoverability clause instead: a WARN at + * construction when a handler is configured, a second WARN on the first real 407, Basic proxy auth + * through `credentials` -- which that transport does pass to the `ProxyAgent` constructor -- and + * the 407 returned untouched for the caller's own auth layer. + * + * Typed `unknown` rather than a declared signature deliberately. The one concrete argument a + * handler could take is the native client's own response type -- undici's + * `Dispatcher.ResponseData`, which is what the Phase 8a plan sketched -- and SEAM-1's zero runtime + * dependencies forbid core from naming it. Inventing a transport-neutral challenge shape here + * would publish a contract with no implementation behind it and no way to validate one. `unknown` + * states honestly that the protocol is unspecified. The slot's only readers are the two + * `typeof x === 'function'` probes in `transport-undici`'s `challenge-handler.ts`, and those + * probes are what make both WARNs fire -- so the field is load-bearing for discoverability without + * anything dispatching through it. + * + * It stays for that reason and one more: removing it would break CFG-22's MUST and strand + * TRANSPORT-30's SHOULD-warn clause with no subject. + */ + readonly challengeHandler?: unknown; + /** When set, every host bypasses this proxy and is dialled directly (CFG-23, CFG-27). */ + readonly bypassAll: boolean; +} + +/** + * A glob or a candidate host reduced to its comparison form: lower-cased once, then split into code + * *points* rather than UTF-16 code units so `?` still means one character in the astral planes. + */ +type FoldedText = readonly string[]; + +/** + * Compiled bypass globs, keyed by the very array they were compiled from, so CFG-23's "compiled once + * at construction" holds for a factory-built `ProxyOptions` without putting a cache field on the + * public shape -- and still holds for a hand-built one, which compiles on its first use and reuses + * the result thereafter. + * + * The consequence for a hand-built one: `createProxyOptions` freezes the list it stores, but an + * object literal typed as `ProxyOptions` may carry a mutable array, and that array's first compile is + * cached against it permanently. A pattern pushed onto it afterwards is silently ignored by + * {@link shouldBypassProxy}. Build through the factory, or treat the array as frozen by convention. + * + * Exported so `proxy.test.ts` can read the cache *before* any bypass decision. Without that, + * CFG-23's "compiled once at construction" is indistinguishable from "compiled on first use", and + * {@link createProxyOptions}'s {@link compileGlobs} call could be deleted with every test still + * green. Not re-exported from the package barrel. + * + * @internal + */ +export const compiledGlobs = new WeakMap< + readonly string[], + readonly FoldedText[] +>(); + +function foldToCodePoints(value: string): FoldedText { + return Array.from(value.toLowerCase()); +} + +/** + * CFG-23's glob dialect: `*` is any run, `?` is exactly one character, everything else is a literal, + * and the match is full-string and case-insensitive. + * + * **There is no escape character.** CFG-23 defines none, so a backslash is an ordinary literal: the + * pattern `a\*b` matches the three-character host `a\*b` and nothing else -- neither `a*b` nor + * `aXXXb`. + * + * Deliberately a hand-written walk rather than a `RegExp`. Translating `*` to `.*` produced adjacent + * unanchored runs, and a non-matching host then drove catastrophic backtracking: the pattern + * `*a*a*a*a*a*a*a*a*a*b` against a 60-character host took 38 seconds of blocked event loop. Patterns + * are operator-supplied through `NO_PROXY` / `http.nonProxyHosts` and the host is often a redirect + * target, so that exponent was reachable from ordinary configuration. This walk keeps exactly one + * backtrack anchor per `*` instead of a stack of them, so it is O(pattern x text) at worst. + */ +function globMatches(pattern: FoldedText, text: FoldedText): boolean { + let patternAt = 0; + let textAt = 0; + let starAt = -1; + let resumeAt = 0; + while (textAt < text.length) { + const expected = pattern[patternAt]; + if ( + expected === '?' || + (expected !== undefined && expected === text[textAt]) + ) { + patternAt += 1; + textAt += 1; + } else if (expected === '*') { + starAt = patternAt; + resumeAt = textAt; + patternAt += 1; + } else if (starAt === -1) { + return false; + } else { + // The most recent `*` swallows one more character and the walk resumes just after it. + resumeAt += 1; + patternAt = starAt + 1; + textAt = resumeAt; + } + } + while (pattern[patternAt] === '*') patternAt += 1; + return patternAt === pattern.length; +} + +/** + * The compiled form of `patterns`, compiling and caching on first use. Exported for the identity + * assertion in `proxy.test.ts` that pins the reuse; not re-exported from the package barrel. + * + * @internal + */ +export function globsFor(patterns: readonly string[]): readonly FoldedText[] { + const cached = compiledGlobs.get(patterns); + if (cached !== undefined) return cached; + const compiled = patterns.map(foldToCodePoints); + compiledGlobs.set(patterns, compiled); + return compiled; +} + +/** + * Fills {@link compiledGlobs} for `patterns`, so a later {@link globsFor} on the very same array is a + * lookup rather than a compile. This is CFG-23's "compiled once at construction". + * + * A separate name and a `void` return because {@link createProxyOptions} is the one caller that wants + * the effect without the value: `compileGlobs(hosts);` reads as the effect it is, where + * `globsFor(hosts);` read as a discarded result and looked deletable. + */ +function compileGlobs(patterns: readonly string[]): void { + globsFor(patterns); +} + +/** + * Whether `host` should bypass the proxy and be dialled directly (CFG-23). + * + * @param options - the proxy configuration; only the bypass flag and glob list are read. + * @param host - the destination host name. + * @returns `true` when bypass-all is set, or when `host` matches any configured glob. + * + * @public + */ +export function shouldBypassProxy( + options: Pick<ProxyOptions, 'bypassAll' | 'nonProxyHosts'>, + host: string, +): boolean { + if (options.bypassAll) return true; + const candidate = foldToCodePoints(host); + return globsFor(options.nonProxyHosts).some(glob => + globMatches(glob, candidate), + ); +} + +/** + * Renders a proxy for logs with any credentials masked (CFG-22). The single masking implementation: + * the own `toString` a {@link createProxyOptions} instance carries delegates here. + * + * @param options - the proxy to render. + * @returns `type://host:port` -- the proxy type stands in as the scheme, an IPv6 host is + * re-bracketed, and `***:***@` stands in for any credentials. + * + * @public + */ +export function formatProxyOptions(options: ProxyOptions): string { + const credentials = options.credentials === undefined ? '' : '***:***@'; + // `host` is stored bare, so an IPv6 literal has to be re-bracketed here or the rendering is + // ambiguous: `2001:db8::1:8080` cannot be read back as an address plus a port. A colon in the host + // is the only case that needs it -- no registered name or IPv4 literal can contain one. + const host = options.host.includes(':') ? `[${options.host}]` : options.host; + return `${options.type}://${credentials}${host}:${String(options.port)}`; +} + +/** + * What {@link createProxyOptions} accepts: the three fields a proxy cannot be described without, plus + * the four the factory defaults or leaves empty. + * + * @public + */ +export interface ProxyOptionsInit { + /** The proxy transport. */ + readonly type: ProxyType; + /** The proxy host name or literal address. */ + readonly host: string; + /** The proxy port, explicit and within 0..65535 -- never guessed from the scheme (CFG-25). */ + readonly port: number; + /** Ordered glob patterns for hosts that bypass this proxy; defaults to empty (CFG-23). */ + readonly nonProxyHosts?: readonly string[] | undefined; + /** Proxy credentials, when there are any. */ + readonly credentials?: ProxyCredentials | undefined; + /** + * The optional challenge-handler slot CFG-22's field list requires, copied through to the built + * {@link ProxyOptions} unchanged. Nothing dispatches through it, by settled disposition rather + * than by omission; the reasoning is on {@link ProxyOptions.challengeHandler}. + */ + readonly challengeHandler?: unknown; + /** Whether every host bypasses this proxy; defaults to `false` (CFG-27). */ + readonly bypassAll?: boolean | undefined; +} + +/** + * Builds a frozen {@link ProxyOptions} carrying an own `toString` that delegates to + * {@link formatProxyOptions}, so interpolating the result into a log masks credentials (CFG-22), and + * whose bypass globs are compiled once, here, rather than on every bypass decision (CFG-23). + * + * @param init - the proxy's fields; `nonProxyHosts` defaults to empty and `bypassAll` to `false`. + * @returns the frozen proxy configuration. + * + * @public + */ +export function createProxyOptions(init: ProxyOptionsInit): ProxyOptions { + const nonProxyHosts = Object.freeze([...(init.nonProxyHosts ?? [])]); + compileGlobs(nonProxyHosts); + const fields = { + type: init.type, + host: init.host, + port: init.port, + nonProxyHosts, + credentials: init.credentials, + challengeHandler: init.challengeHandler, + bypassAll: init.bypassAll ?? false, + }; + return Object.freeze({...fields, toString: () => formatProxyOptions(fields)}); +} + +/** + * Splits on unescaped occurrences of `separator`, preserving a backslash-escaped separator inside the + * token it belongs to (CFG-26). + */ +function splitEscaped(raw: string, separator: string): string[] { + const tokens: string[] = []; + let current = ''; + for (let i = 0; i < raw.length; i += 1) { + const character = raw[i] ?? ''; + if (character === '\\' && raw[i + 1] === separator) { + current += `\\${separator}`; + i += 1; + } else if (character === separator) { + tokens.push(current); + current = ''; + } else { + current += character; + } + } + tokens.push(current); + return tokens; +} + +/** + * CFG-26's observable order, exactly: split, drop empty, unescape, trim. Trimming last is what makes + * a whitespace-only fragment survive the drop and land as an empty token. + */ +function parseNonProxyHosts(raw: string, separator: string): readonly string[] { + return splitEscaped(raw, separator) + .filter(token => token !== '') + .map(token => token.replaceAll(`\\${separator}`, separator)) + .map(token => token.trim()); +} + +interface NonProxyResolution { + readonly bypassAll: boolean; + readonly hosts: readonly string[]; +} + +/** + * CFG-26: the property layer (pipe-separated) wins over the environment variable (comma-separated). + * CFG-27: a resolved list of exactly one bare `*` is bypass-all, represented by the flag rather than + * as a literal glob entry; a `*` among several entries stays an ordinary any-host glob. + */ +function resolveNonProxyHosts(config: Configuration): NonProxyResolution { + const fromProperty = config.getRawProperty(PROPERTY_KEYS.nonProxyHosts); + const hosts = + fromProperty === undefined + ? parseNonProxyHosts(config.getString(CFG_KEY_NO_PROXY) ?? '', ',') + : parseNonProxyHosts(fromProperty, '|'); + if (hosts.length === 1 && hosts[0] === '*') + return {bypassAll: true, hosts: []}; + return {bypassAll: false, hosts}; +} + +/** CFG-25: explicit, numeric, and in range -- no default-port guessing, ever. */ +function parsePort(raw: string | undefined): number | null { + if (raw === undefined) return null; + const trimmed = raw.trim(); + if (!DECIMAL_DIGITS.test(trimmed)) return null; + const port = Number(trimmed); + return port <= MAX_PORT ? port : null; +} + +const PROXY_TYPE_BY_SCHEME: ReadonlyMap<string, ProxyType> = new Map([ + ['http:', 'http'], + ['https:', 'http'], + ['socks:', 'socks5'], + ['socks5:', 'socks5'], + ['socks5h:', 'socks5'], + ['socks4:', 'socks4'], + ['socks4a:', 'socks4'], +]); + +/** + * What both CFG-24 tiers produce: the endpoint fields only. Deliberately not + * `Omit<ProxyOptionsInit, 'challengeHandler'>` -- that also admits `nonProxyHosts` and `bypassAll`, + * which neither producer sets and which {@link resolveProxyOptions} supplies from the separate + * CFG-26/CFG-27 resolution, making the spread order there silently load-bearing. + */ +type ProxyEndpoint = Pick< + ProxyOptionsInit, + 'type' | 'host' | 'port' | 'credentials' +>; + +/** + * CFG-22's `host` is the bare address, so an IPv6 literal loses the brackets `URL.hostname` puts back + * on it and the two CFG-24 layers agree on one representation (see {@link ProxyOptions.host}). + */ +function unbracketHost(host: string): string { + return host.startsWith('[') && host.endsWith(']') ? host.slice(1, -1) : host; +} + +/** + * The port the operator actually wrote, which is not always `url.port`. + * + * The WHATWG parser normalizes a *special* scheme's default port to the empty string, so `url.port` + * cannot tell `http://p:80` from `http://p` and `https://p:443` from `https://p`. CFG-25 bans + * *guessing* an absent port; it does not license discarding one the operator wrote. Re-reading under + * a scheme the parser does not treat as special leaves the port verbatim. + * + * Only the port is taken from the probe: a non-special scheme also skips host lower-casing, so every + * other field still comes from the real `url`. + */ +function explicitPort(raw: string, url: URL): string | undefined { + if (url.port !== '') return url.port; + try { + const probe = new URL( + raw.replace(/^[a-z][a-z\d+.-]*:/iu, 'x-dexpace-probe:'), + ); + return probe.port === '' ? undefined : probe.port; + } catch { + // Parsed under its real scheme but not under the probe: not a port question, so CFG-25's + // "absent port is invalid" result stands. + return undefined; + } +} + +/** + * CFG-24's `user:pass@` segment, percent-decoded. An empty user name is no credentials at all -- the + * same rule the property layer applies -- so `http://:secret@host:8080` carries none. + */ +function readUrlCredentials(url: URL): ProxyCredentials | undefined { + if (url.username === '') return undefined; + return { + username: decodeURIComponent(url.username), + password: decodeURIComponent(url.password), + }; +} + +/** + * CFG-24's WARNING half: "invalid config -> null + warning". The null half was always here; until + * 2026-09-02 the warning had nowhere to go, because no `Logger` seam existed when 7a shipped this + * module. The consequence was that a typo'd `HTTPS_PROXY` routed every request DIRECT with + * nothing to read anywhere. + * + * Never the URL itself: a proxy URL carries `user:pass@`, and CFG-22 masks credentials in every + * rendering. The variable that supplied it and the reason it was rejected are enough to act on. + */ +function warnProxyRejected(source: string, reason: string): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('http.proxy.configRejected') + .field('source', source) + .field('reason', reason) + .emit(); + } catch { + // OBS-20: logger failure must never fail resolution, which CFG-24 makes total. + } +} + +/** + * CFG-24's environment form: `scheme://user:pass@host:port`. Total -- malformed input is `null`, + * and every null path warns through {@link warnProxyRejected} naming `source`. + */ +function parseProxyUrl(raw: string, source: string): ProxyEndpoint | null { + const trimmed = raw.trim(); + let url: URL; + let port: number | null; + let credentials: ProxyCredentials | undefined; + try { + url = new URL(trimmed); + port = parsePort(explicitPort(trimmed, url)); + // Inside the `try` on purpose: `decodeURIComponent` raises `URIError` on a lone `%`, which an + // un-encoded proxy password legitimately contains, and CFG-24 requires null rather than a throw. + credentials = readUrlCredentials(url); + } catch { + // A malformed proxy URL is ordinary bad configuration, not a programmer error: CFG-24 requires + // resolution to return null rather than throw. + warnProxyRejected(source, 'unparseable'); + return null; + } + // Three gates, three distinct warnings. Kept as separate `if`s rather than the single conjunction + // they were, because "the proxy was rejected" is not actionable and "the port is unusable" is. + const type = PROXY_TYPE_BY_SCHEME.get(url.protocol); + if (type === undefined) { + warnProxyRejected(source, 'scheme'); + return null; + } + if (port === null) { + warnProxyRejected(source, 'port'); + return null; + } + if (url.hostname === '') { + warnProxyRejected(source, 'host'); + return null; + } + return {type, host: unbracketHost(url.hostname), port, credentials}; +} + +/** + * CFG-24's property form. Host is `https.proxyHost` preferred over `http.proxyHost`, and the port + * MUST come from the same layer as the chosen host -- an `https` host never borrows an `http` port. + * Credentials read only from `https.proxyUser`/`https.proxyPassword`, with no `http.*` fallback, + * even when the host came from the `http` layer. + */ +function resolveFromProperties(config: Configuration): ProxyEndpoint | null { + const httpsHost = config.getRawProperty(PROPERTY_KEYS.httpsHost); + const host = httpsHost ?? config.getRawProperty(PROPERTY_KEYS.httpHost); + if (host === undefined || host === '') return null; + + const portKey = + httpsHost === undefined ? PROPERTY_KEYS.httpPort : PROPERTY_KEYS.httpsPort; + const port = parsePort(config.getRawProperty(portKey)); + if (port === null) return null; + + const username = config.getRawProperty(PROPERTY_KEYS.httpsUser); + const password = config.getRawProperty(PROPERTY_KEYS.httpsPassword); + // An empty user name is no credentials, the same rule the URL layer applies, so a blank + // `https.proxyUser` does not fabricate a masked `***:***@` for a proxy that has none. + const credentials = + username === undefined || username === '' + ? undefined + : {username, password: password ?? ''}; + return {type: 'http', host: unbracketHost(host), port, credentials}; +} + +/** CFG-24's environment form, HTTPS_PROXY preferred over HTTP_PROXY. */ +function resolveFromEnvironment(config: Configuration): ProxyEndpoint | null { + const https = config.getString(CFG_KEY_HTTPS_PROXY); + const source = https === undefined ? CFG_KEY_HTTP_PROXY : CFG_KEY_HTTPS_PROXY; + const raw = https ?? config.getString(CFG_KEY_HTTP_PROXY); + return raw === undefined ? null : parseProxyUrl(raw, source); +} + +/** + * Resolves proxy settings from configuration (CFG-24), or `null` when none is configured, the + * configuration is invalid, or bypass-all is in force. + * + * Total: malformed input never throws (CFG-24, CFG-25). The property layer is consulted first and + * the environment second; in the default Node wiring the property seam is empty, so this is + * effectively environment-only there -- the precedence exists for a host that supplies a real + * property store through the seam. + * + * Nothing here runs implicitly: the environment is read only because a caller invoked this (CFG-28). + * + * @param config - the configuration to resolve against. + * @returns the resolved proxy, or `null`. + * + * @public + */ +export function resolveProxyOptions( + config: Configuration, +): ProxyOptions | null { + const {bypassAll, hosts} = resolveNonProxyHosts(config); + if (bypassAll) return null; + + const endpoint = + resolveFromProperties(config) ?? resolveFromEnvironment(config); + if (endpoint === null) return null; + + // `bypassAll` is stated rather than left to the factory default: CFG-27 makes bypass-all resolve to + // `null` above, so a `ProxyOptions` that came out of *resolution* can never carry `true`. + return createProxyOptions({ + ...endpoint, + nonProxyHosts: hosts, + bypassAll: false, + }); +} diff --git a/packages/core/src/config/retryable.test.ts b/packages/core/src/config/retryable.test.ts new file mode 100644 index 0000000..b5e6ae4 --- /dev/null +++ b/packages/core/src/config/retryable.test.ts @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/retryable.test.ts +// Exercises: CFG-35 (exactly 408, 429, and 5xx except 501/505 are retryable; where implemented, +// this exact set is a hard contract). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {InvariantViolation} from '../invariant.js'; +import {RETRYABLE_STATUSES, isRetryableStatus} from './retryable.js'; + +/** + * CFG-35's membership rule restated from the requirement's prose, so a property can compare the + * implementation against the requirement rather than against itself. + */ +function isRetryableByRequirement(code: number): boolean { + if (code === 408 || code === 429) return true; + return code >= 500 && code <= 599 && code !== 501 && code !== 505; +} + +/** 408 and 429, plus the hundred 5xx codes less 501 and 505 (CFG-35). */ +const RETRYABLE_STATUS_COUNT = 2 + 100 - 2; + +describe('RETRYABLE_STATUSES immutability (CFG-35)', () => { + test('refuses an add, so the hard contract cannot be rewritten by a consumer', () => { + // The `ReadonlySet` type is compile-time only, and `Object.freeze` does not seal a `Set`'s + // internal slots. This binding leaves through the package barrel, so `add(418)` used to succeed + // and permanently change the process-wide classifier for everyone. + expect(() => (RETRYABLE_STATUSES as Set<number>).add(418)).toThrow( + InvariantViolation, + ); + expect(isRetryableStatus(418)).toBe(false); + }); + + test('refuses a delete, so a retryable status cannot be removed by a consumer', () => { + expect(() => (RETRYABLE_STATUSES as Set<number>).delete(500)).toThrow( + InvariantViolation, + ); + expect(isRetryableStatus(500)).toBe(true); + }); + + test('refuses a clear', () => { + expect(() => { + (RETRYABLE_STATUSES as Set<number>).clear(); + }).toThrow(InvariantViolation); + expect(RETRYABLE_STATUSES.size).toBe(RETRYABLE_STATUS_COUNT); + }); + + test('is frozen, so the refusing mutators cannot be defined away', () => { + expect(Object.isFrozen(RETRYABLE_STATUSES)).toBe(true); + }); +}); + +describe('isRetryableStatus (CFG-35)', () => { + test('treats 408 Request Timeout as retryable', () => { + expect(isRetryableStatus(408)).toBe(true); + }); + + test('treats 429 Too Many Requests as retryable', () => { + expect(isRetryableStatus(429)).toBe(true); + }); + + test('treats the 5xx range as retryable', () => { + for (const code of [500, 502, 503, 504, 599]) { + expect(isRetryableStatus(code)).toBe(true); + } + }); + + test('excludes 501 Not Implemented from the retryable 5xx range', () => { + expect(isRetryableStatus(501)).toBe(false); + }); + + test('excludes 505 HTTP Version Not Supported from the retryable 5xx range', () => { + expect(isRetryableStatus(505)).toBe(false); + }); + + test('treats every other status as not retryable', () => { + for (const code of [200, 201, 301, 400, 401, 404, 409, 418, 499, 600]) { + expect(isRetryableStatus(code)).toBe(false); + } + }); +}); + +describe('RETRYABLE_STATUSES (CFG-35)', () => { + // Both properties compare against `isRetryableByRequirement`, never against each other. + // `isRetryableStatus`'s body *is* `RETRYABLE_STATUSES.has(code)`, so a property asserting those two + // agree compares an expression with itself and survives replacing the whole set with `{418}`. + test('holds exactly the codes CFG-35 names, across the whole status range', () => { + fc.assert( + fc.property(fc.integer({min: 100, max: 700}), code => { + expect(RETRYABLE_STATUSES.has(code)).toBe( + isRetryableByRequirement(code), + ); + }), + ); + }); + + test('exposes a predicate that answers what CFG-35 names, across the whole status range', () => { + fc.assert( + fc.property(fc.integer({min: 100, max: 700}), code => { + expect(isRetryableStatus(code)).toBe(isRetryableByRequirement(code)); + }), + ); + }); + + test('holds exactly 100 codes -- 408, 429, and the hundred 5xx less 501 and 505', () => { + expect(RETRYABLE_STATUSES.size).toBe(RETRYABLE_STATUS_COUNT); + }); +}); diff --git a/packages/core/src/config/retryable.ts b/packages/core/src/config/retryable.ts new file mode 100644 index 0000000..9dcf26c --- /dev/null +++ b/packages/core/src/config/retryable.ts @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/config/retryable.ts +import {InvariantViolation} from '../invariant.js'; + +function buildRetryableStatuses(): Set<number> { + const codes = new Set<number>([408, 429]); + for (let code = 500; code <= 599; code += 1) { + // 501 Not Implemented and 505 HTTP Version Not Supported both say the server cannot fulfill the + // request in the form it was asked, no matter how many times it is asked again. + if (code !== 501 && code !== 505) codes.add(code); + } + return codes; +} + +/** Refuses a mutation of the set the barrel publishes as a contract (CFG-35). */ +function denyMutation(operation: string): () => never { + return () => { + throw new InvariantViolation( + `RETRYABLE_STATUSES is immutable: ${operation} is not permitted (CFG-35)`, + ); + }; +} + +/** + * The single retryable-status definition (CFG-35): exactly 408, 429, and every 5xx except 501 and + * 505. Where implemented, this exact set is a hard contract, so it lives in one place and every + * consumer -- the retry engine's `RETRY-1` classifier included -- re-exports it rather than + * restating it. + * + * The `ReadonlySet` *type* is not enough on its own, and neither is `Object.freeze`, which does not + * seal a `Set`'s internal slots -- `add`/`delete`/`clear` go straight past it. Phase 1's + * `IDEMPOTENT_METHODS` can live with that because it is module-private and unreachable; this binding + * leaves through the package barrel, where `(RETRYABLE_STATUSES as Set<number>).add(418)` used to + * succeed and permanently rewrite the process-wide classifier. So the three mutators become own + * properties that throw, and the freeze is what stops them being defined back. + * + * @public + */ +export const RETRYABLE_STATUSES: ReadonlySet<number> = Object.freeze( + Object.defineProperties(buildRetryableStatuses(), { + add: {value: denyMutation('add')}, + delete: {value: denyMutation('delete')}, + clear: {value: denyMutation('clear')}, + }), +); + +/** + * Whether a response status is retryable (CFG-35). + * + * @param code - the HTTP status code. + * @returns `true` for 408, 429, and 5xx other than 501 and 505; `false` otherwise. + * + * @public + */ +export function isRetryableStatus(code: number): boolean { + return RETRYABLE_STATUSES.has(code); +} diff --git a/packages/core/src/context/context.test.ts b/packages/core/src/context/context.test.ts new file mode 100644 index 0000000..2f10708 --- /dev/null +++ b/packages/core/src/context/context.test.ts @@ -0,0 +1,286 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/context.test.ts +// Exercises: CTX-1 (one-way promotion, incl. the compile-time no-promote-back check), CTX-2 (additive, +// non-mutating, carries forward instrumentation + key), CTX-3 (one shared call key across the whole +// chain), CTX-5/CTX-6 (off-chain construction, fresh key per default call at population scale, explicit +// key pinning), CTX-7 (immutable), CTX-15 (keys stay call-unique though every bundle field is identical), +// CTX-16 (operationName absent at dispatch, introduced at request, carried forward, never keyed on) +import {describe, expect, test} from 'bun:test'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import { + type DispatchContext, + createDispatchContext, + createExchangeContext, + createRequestContext, + promoteToExchange, + promoteToRequest, +} from './context.js'; +import {noopInstrumentationBundle} from './instrumentation.js'; + +function aRequest(): Request { + return Request.newBuilder().url('https://example.com').build(); +} + +function aResponse(request: Request): Response { + return Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .build(); +} + +describe('promotion chain (CTX-1, CTX-2, CTX-3)', () => { + test('dispatch exposes exactly its expected artifacts', () => { + const dispatch = createDispatchContext(); + expect(dispatch.kind).toBe('dispatch'); + expect(dispatch.key).toBeDefined(); + expect(dispatch.instrumentation).toBe(noopInstrumentationBundle); + }); + + test('promoting dispatch to request adds exactly the request, carrying key and instrumentation forward by reference', () => { + const dispatch = createDispatchContext(); + const request = aRequest(); + const requestCtx = promoteToRequest(dispatch, request, 'GetWidget'); + + expect(requestCtx.kind).toBe('request'); + expect(requestCtx.key).toBe(dispatch.key); + expect(requestCtx.instrumentation).toBe(dispatch.instrumentation); + expect(requestCtx.request).toBe(request); + expect(requestCtx.operationName).toBe('GetWidget'); + }); + + test('the source context is unchanged by promotion', () => { + const dispatch = createDispatchContext(); + const before = {...dispatch}; + promoteToRequest(dispatch, aRequest()); + expect(dispatch).toEqual(before); + }); + + test('promoting request to exchange adds exactly the response, carrying everything else forward', () => { + const request = aRequest(); + const requestCtx = promoteToRequest( + createDispatchContext(), + request, + 'GetWidget', + ); + const response = aResponse(request); + const exchangeCtx = promoteToExchange(requestCtx, response); + + expect(exchangeCtx.kind).toBe('exchange'); + expect(exchangeCtx.key).toBe(requestCtx.key); + expect(exchangeCtx.instrumentation).toBe(requestCtx.instrumentation); + expect(exchangeCtx.operationName).toBe('GetWidget'); + expect(exchangeCtx.request).toBe(request); + expect(exchangeCtx.response).toBe(response); + }); + + test('the whole chain shares one call key across all three flavors', () => { + const dispatch = createDispatchContext(); + const requestCtx = promoteToRequest(dispatch, aRequest()); + const exchangeCtx = promoteToExchange( + requestCtx, + aResponse(requestCtx.request), + ); + expect(requestCtx.key).toBe(dispatch.key); + expect(exchangeCtx.key).toBe(dispatch.key); + }); +}); + +describe('promotion is one-way (CTX-1)', () => { + test('no promotion function accepts an ExchangeContext, so there is no way back', () => { + const requestCtx = promoteToRequest(createDispatchContext(), aRequest()); + const exchangeCtx = promoteToExchange( + requestCtx, + aResponse(requestCtx.request), + ); + + // CTX-1's "the exchange type exposes no method promoting back" is a compile-time guarantee in this + // design, not a runtime one: promoteToRequest/promoteToExchange are free functions typed to accept + // only DispatchContext/RequestContext respectively, and there is no third promotion function. These + // two @ts-expect-error lines are the assertion -- `bun run typecheck` FAILS if either promotion ever + // widens to accept a terminal context, which a prose-only comment would not catch. + // @ts-expect-error -- ExchangeContext is terminal; it is not a DispatchContext + promoteToRequest(exchangeCtx, aRequest()); + // @ts-expect-error -- ExchangeContext is terminal; it is not a RequestContext + promoteToExchange(exchangeCtx, aResponse(requestCtx.request)); + + expect(exchangeCtx.kind).toBe('exchange'); + }); +}); + +describe('off-chain construction (CTX-5, CTX-6)', () => { + test('default construction mints a fresh, distinct key every call', () => { + const a = createDispatchContext(); + const b = createDispatchContext(); + expect(a.key).not.toBe(b.key); + }); + + test('N default-constructed contexts across all three flavors are pairwise key-distinct', () => { + // CTX-5's "globally distinct across the whole process and all three flavors" is a property over the + // whole population, not just a pair -- a keying scheme that collided every Nth call would pass the + // pairwise test above. Every bundle field is identical here (all use noopInstrumentationBundle), so + // this is also CTX-15's "call-key derivation MUST remain call-unique even when every bundle field is + // identical" at scale. + const request = aRequest(); + const keys = new Set<symbol>(); + for (let i = 0; i < 1000; i += 1) { + keys.add(createDispatchContext().key); + keys.add(createRequestContext(request).key); + keys.add(createExchangeContext(request, aResponse(request)).key); + } + expect(keys.size).toBe(3000); + }); +}); + +describe('default key RENDERING (CTX-8)', () => { + test('two default keys of one flavor RENDER differently, not just compare so', () => { + // Appendix C states CTX-8 as "an error whose MESSAGE identifies the key", and + // `DuplicateContextKeyError`'s message is `String(key)`. Until 2026-09-02 every default key of a + // flavor rendered as the identical `Symbol(dispatch-context)`, so the message named the flavor + // and not the key. + const a = String(createDispatchContext().key); + const b = String(createDispatchContext().key); + expect(a).not.toBe(b); + expect(a).toMatch(/^Symbol\(dispatch-context#\d+\)$/u); + }); + + test('the three flavors stay distinguishable in the rendering', () => { + const request = aRequest(); + expect(String(createRequestContext(request).key)).toMatch( + /^Symbol\(request-context#\d+\)$/u, + ); + expect( + String(createExchangeContext(request, aResponse(request)).key), + ).toMatch(/^Symbol\(exchange-context#\d+\)$/u); + }); +}); + +describe('off-chain construction, continued (CTX-5, CTX-6)', () => { + test('an explicit key can be pinned so two contexts share one slot', () => { + const key = Symbol('shared'); + const a = createDispatchContext({key}); + const b = createDispatchContext({key}); + expect(a.key).toBe(b.key); + }); + + test('an explicit instrumentation bundle is carried onto the context verbatim', () => { + const instrumentation = { + ...noopInstrumentationBundle, + traceId: 'a'.repeat(32), + isValid: true, + }; + expect(createDispatchContext({instrumentation}).instrumentation).toBe( + instrumentation, + ); + }); + + test('a caller-supplied instrumentation bundle is frozen by the factory (CTX-7)', () => { + // Object.freeze on the context is shallow, so without this the bundle behind `instrumentation` stays + // writable and the caller can mutate a "immutable" context out from under the whole chain. + const instrumentation = { + ...noopInstrumentationBundle, + traceId: 'a'.repeat(32), + }; + const dispatch = createDispatchContext({instrumentation}); + + expect(Object.isFrozen(dispatch.instrumentation)).toBe(true); + expect(Object.isFrozen(instrumentation)).toBe(true); // frozen in place, so the reference stays shared + }); + + test('createRequestContext and createExchangeContext also default to a fresh key per call', () => { + const request = aRequest(); + const a = createRequestContext(request); + const b = createRequestContext(request); + expect(a.key).not.toBe(b.key); + + const c = createExchangeContext(request, aResponse(request)); + const d = createExchangeContext(request, aResponse(request)); + expect(c.key).not.toBe(d.key); + }); +}); + +describe('operationName (CTX-16)', () => { + test('is absent at the dispatch stage', () => { + expect('operationName' in createDispatchContext()).toBe(false); + }); + + test('defaults to undefined when not supplied at promotion', () => { + const requestCtx = promoteToRequest(createDispatchContext(), aRequest()); + expect(requestCtx.operationName).toBeUndefined(); + }); + + test('is carried forward unchanged across the request-to-exchange promotion', () => { + const requestCtx = promoteToRequest( + createDispatchContext(), + aRequest(), + 'GetWidget', + ); + const exchangeCtx = promoteToExchange( + requestCtx, + aResponse(requestCtx.request), + ); + expect(exchangeCtx.operationName).toBe('GetWidget'); + }); + + test('is advisory only -- it never influences the call key', () => { + // CTX-16: "never influencing the request, dispatch decision, or store key." Two otherwise-identical + // promotions differing only in operationName keep their source keys; and pinning one key across two + // different operation names still yields one slot, proving the name is not folded into it. + const key = Symbol('shared'); + const a = promoteToRequest( + createDispatchContext({key}), + aRequest(), + 'GetWidget', + ); + const b = promoteToRequest( + createDispatchContext({key}), + aRequest(), + 'DeleteWidget', + ); + expect(a.key).toBe(b.key); + expect(a.operationName).not.toBe(b.operationName); + }); +}); + +describe('immutability (CTX-7)', () => { + test('a promotion freezes a bundle that never passed through a factory', () => { + // The context flavors are interfaces, not classes, so 4b/4c can hand a promotion a + // literal-constructed context whose bundle was never frozen. Without this the promoted context is + // "immutable" in name only: the caller keeps a writable reference to its trace state. + const instrumentation = { + ...noopInstrumentationBundle, + traceId: 'a'.repeat(32), + }; + const forged: DispatchContext = { + kind: 'dispatch', + key: Symbol('forged'), + instrumentation, + }; + + const requestCtx = promoteToRequest(forged, aRequest()); + + expect(Object.isFrozen(requestCtx.instrumentation)).toBe(true); + expect(requestCtx.instrumentation).toBe(instrumentation); // frozen in place -- CTX-2 still holds + expect( + Object.isFrozen( + promoteToExchange(requestCtx, aResponse(requestCtx.request)) + .instrumentation, + ), + ).toBe(true); + }); + + test('every context flavor is frozen', () => { + const dispatch = createDispatchContext(); + expect(Object.isFrozen(dispatch)).toBe(true); + const requestCtx = promoteToRequest(dispatch, aRequest()); + expect(Object.isFrozen(requestCtx)).toBe(true); + expect( + Object.isFrozen( + promoteToExchange(requestCtx, aResponse(requestCtx.request)), + ), + ).toBe(true); + }); +}); diff --git a/packages/core/src/context/context.ts b/packages/core/src/context/context.ts new file mode 100644 index 0000000..3a1bf6c --- /dev/null +++ b/packages/core/src/context/context.ts @@ -0,0 +1,236 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/context.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import { + noopInstrumentationBundle, + type InstrumentationBundle, +} from './instrumentation.js'; + +/** + * Before any request (CTX-1). No `operationName` — CTX-16 introduces it at the request stage. + * + * @public + */ +export interface DispatchContext { + /** The discriminant. Branch on it to tell which promotion stage a step is observing. */ + readonly kind: 'dispatch'; + /** This call's identity, unique per `Runtime.send()` and stable across every promotion (CTX-4/CTX-6). */ + readonly key: symbol; + /** Correlation and tracing for this call, shared by reference across every promotion (CTX-2/CTX-3). */ + readonly instrumentation: InstrumentationBundle; +} + +/** + * An outgoing request assembled (CTX-1). + * + * @public + */ +export interface RequestContext { + /** The discriminant. Branch on it to tell which promotion stage a step is observing. */ + readonly kind: 'request'; + /** This call's identity, carried unchanged from the dispatch stage (CTX-4/CTX-6). */ + readonly key: symbol; + /** Correlation and tracing for this call, carried by reference from the dispatch stage (CTX-2/CTX-3). */ + readonly instrumentation: InstrumentationBundle; + /** The operation this call belongs to, or `undefined` when the caller named none (CTX-16). */ + readonly operationName: string | undefined; + /** The assembled outbound request. Immutable; a step substitutes by passing a new one to `ctx.next`. */ + readonly request: Request; +} + +/** + * A response arrived; terminal — no further promotion exists (CTX-1). + * + * @public + */ +export interface ExchangeContext { + /** The discriminant. Branch on it to tell which promotion stage a step is observing. */ + readonly kind: 'exchange'; + /** This call's identity, carried unchanged from the dispatch stage (CTX-4/CTX-6). */ + readonly key: symbol; + /** Correlation and tracing for this call, carried by reference from the dispatch stage (CTX-2/CTX-3). */ + readonly instrumentation: InstrumentationBundle; + /** The operation this call belongs to, or `undefined` when the caller named none (CTX-16). */ + readonly operationName: string | undefined; + /** The request that actually went on the wire — the substituted one, if a step replaced it (CTX-1). */ + readonly request: Request; + /** The response that arrived. OPEN: whoever owns the drive owns closing its body. */ + readonly response: Response; +} + +/** + * The three promotion-chain stages as one discriminated union, branched on `kind`. + * + * @public + */ +export type ExecutionContext = + DispatchContext | RequestContext | ExchangeContext; + +/** + * Optional inputs shared by the three off-chain `create*` factories. One options object rather than + * positional parameters: `createExchangeContext` would otherwise take five, and ESLint's `max-params` is 3 + * and counts optional parameters. Every field is spelled `?: T | undefined` for + * `exactOptionalPropertyTypes`. + * + * @internal + */ +export interface ContextInit { + /** Advisory operation label (CTX-16); never influences the request, dispatch, or store key. */ + readonly operationName?: string | undefined; + /** @defaultValue `noopInstrumentationBundle` */ + readonly instrumentation?: InstrumentationBundle | undefined; + /** + * Pin to make two contexts share one store slot (CTX-5). + * + * @defaultValue a fresh `Symbol()` per call, described `<flavor>#<n>` so CTX-8's duplicate-key + * message names a distinct key rather than a flavor + */ + readonly key?: symbol | undefined; +} + +/** + * Serial number for a default-constructed context key's DESCRIPTION (CTX-8). + * + * `Symbol()` is already the identity, and CTX-4/5/6's uniqueness never depended on the description. + * What did depend on it is CTX-8's message clause, which appendix C states as "an error **whose + * message** identifies the key": every default-constructed context of a flavor rendered as the + * identical `Symbol(dispatch-context)`, so `DuplicateContextKeyError`'s message named the KIND of + * key and not WHICH key. One counter across all three flavors, so no two default keys anywhere in + * the process render alike. + * + * A second module-level mutable binding, on top of the `contextStore` singleton that already takes + * that deviation (`docs/knowledge/harvested/variables-and-declarations.md:22`). It is write-only + * from outside: nothing reads it, nothing resets it, and no behaviour keys off its value -- it + * labels a string. + */ +let nextKeySerial = 0; + +/** A distinctly-described default key: `flavor#N`. Never called when the caller pinned one. */ +function defaultKey(flavor: string): symbol { + nextKeySerial += 1; + return Symbol(`${flavor}#${String(nextKeySerial)}`); +} + +/** + * Off-chain construction (CTX-5): `key` defaults to a fresh Symbol() per call unless pinned, which is also + * what makes default keys globally distinct across the process and all three flavors (CTX-6). Takes + * `Omit<ContextInit, 'operationName'>` — CTX-16 introduces the operation name at the request stage, so the + * dispatch factory does not offer it. + * + * @internal + */ +export function createDispatchContext( + init: Omit<ContextInit, 'operationName'> = {}, +): DispatchContext { + const { + instrumentation = noopInstrumentationBundle, + key = defaultKey('dispatch-context'), + } = init; + return Object.freeze({ + kind: 'dispatch', + key, + instrumentation: freezeBundle(instrumentation), + }); +} + +/** + * Off-chain construction (CTX-5/6) — see `promoteToRequest` for the normal promotion path. + * + * @internal + */ +export function createRequestContext( + request: Request, + init: ContextInit = {}, +): RequestContext { + const { + operationName, + instrumentation = noopInstrumentationBundle, + key = defaultKey('request-context'), + } = init; + return Object.freeze({ + kind: 'request', + key, + instrumentation: freezeBundle(instrumentation), + operationName, + request, + }); +} + +/** + * Off-chain construction (CTX-5/6) — see `promoteToExchange` for the normal promotion path. + * + * @internal + */ +export function createExchangeContext( + request: Request, + response: Response, + init: ContextInit = {}, +): ExchangeContext { + const { + operationName, + instrumentation = noopInstrumentationBundle, + key = defaultKey('exchange-context'), + } = init; + return Object.freeze({ + kind: 'exchange', + key, + instrumentation: freezeBundle(instrumentation), + operationName, + request, + response, + }); +} + +/** + * dispatch -\> request (CTX-1/2/3): adds the request, carries key + instrumentation forward verbatim — + * `freezeBundle` is idempotent and freezes in place, so the bundle reference CTX-2 carries forward is + * unchanged; it is re-run because `DispatchContext` is an interface, so a caller can hand a + * literal-constructed context whose bundle never passed through a `create*` factory. + * + * @internal + */ +export function promoteToRequest( + context: DispatchContext, + request: Request, + operationName?: string, +): RequestContext { + return Object.freeze({ + kind: 'request', + key: context.key, + instrumentation: freezeBundle(context.instrumentation), + operationName, + request, + }); +} + +/** + * request -\> exchange (CTX-1/2/3): adds the response, carries everything else forward verbatim; the + * bundle is re-frozen for the same reason as `promoteToRequest`. + * + * @internal + */ +export function promoteToExchange( + context: RequestContext, + response: Response, +): ExchangeContext { + return Object.freeze({ + kind: 'exchange', + key: context.key, + instrumentation: freezeBundle(context.instrumentation), + operationName: context.operationName, + request: context.request, + response, + }); +} + +/** + * CTX-7: a context must be immutable, but `Object.freeze` on the context object is shallow, so a + * caller-supplied bundle would stay writable behind the `instrumentation` slot. Frozen in place rather than + * copied, so the reference the promotions carry forward (CTX-2) is the one the caller handed in. + * `noopInstrumentationBundle` is already frozen, so the default path costs nothing. Idempotent, which is + * what lets both the factories and the two promotions call it unconditionally. + */ +function freezeBundle(bundle: InstrumentationBundle): InstrumentationBundle { + return Object.isFrozen(bundle) ? bundle : Object.freeze(bundle); +} diff --git a/packages/core/src/context/errors.test.ts b/packages/core/src/context/errors.test.ts new file mode 100644 index 0000000..efea243 --- /dev/null +++ b/packages/core/src/context/errors.test.ts @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/errors.test.ts +// Exercises: CTX-8 (reject-on-duplicate insert failure, naming the key) +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import {DuplicateContextKeyError} from './errors.js'; + +describe('DuplicateContextKeyError', () => { + test('descends from DexpaceError and names the offending key', () => { + const key = Symbol('call-1'); + const error = new DuplicateContextKeyError(key); + expect(error).toBeInstanceOf(DexpaceError); + expect(error.key).toBe(key); + expect(error.message).toContain('call-1'); + }); + + test('sets name from its own constructor', () => { + expect(new DuplicateContextKeyError(Symbol('x')).name).toBe( + 'DuplicateContextKeyError', + ); + }); + + test('cause chains through', () => { + const cause = new Error('boom'); + expect(new DuplicateContextKeyError(Symbol('x'), {cause}).cause).toBe( + cause, + ); + }); +}); diff --git a/packages/core/src/context/errors.ts b/packages/core/src/context/errors.ts new file mode 100644 index 0000000..eacab55 --- /dev/null +++ b/packages/core/src/context/errors.ts @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * `installIfAbsent` found the key already occupied (CTX-8). + * + * @internal + */ +export class DuplicateContextKeyError extends DexpaceError { + readonly key: symbol; + + constructor(key: symbol, options?: ErrorOptions) { + super(`context key already registered: ${String(key)}`, options); + this.key = key; + } +} diff --git a/packages/core/src/context/instrumentation.test.ts b/packages/core/src/context/instrumentation.test.ts new file mode 100644 index 0000000..d647d4d --- /dev/null +++ b/packages/core/src/context/instrumentation.test.ts @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/instrumentation.test.ts +// Exercises: CTX-14 (bundle shape), CTX-15 (no-op default: invalid sentinels, isValid/isRemote false, +// no-op span/tracer factory), CTX-20 (tracer factory safe to invoke concurrently, emits nothing) +import {describe, expect, test} from 'bun:test'; +import {NOOP_SPAN} from '../observability/span.js'; +import {noopInstrumentationBundle} from './instrumentation.js'; + +describe('noopInstrumentationBundle (CTX-15)', () => { + test('reserves all-zero trace/span ids and zero flags', () => { + expect(noopInstrumentationBundle.traceId).toBe( + '00000000000000000000000000000000', + ); + expect(noopInstrumentationBundle.spanId).toBe('0000000000000000'); + expect(noopInstrumentationBundle.traceFlags).toBe(0); + expect(noopInstrumentationBundle.traceState).toBe(''); + }); + + test('names its trace-id encoding flavor', () => { + // CTX-14 requires the flavor field; CTX-15 fixes no sentinel for it, so the disabled bundle says + // 'none' rather than claiming an encoding it never produced ids in. + expect(noopInstrumentationBundle.traceIdEncoding).toBe('none'); + }); + + test('is invalid and not remote', () => { + expect(noopInstrumentationBundle.isValid).toBe(false); + expect(noopInstrumentationBundle.isRemote).toBe(false); + }); + + // CTX-15 says "a no-op span", and it means an object: the requirement lists the no-op span beside the + // no-op tracer factory, and `createInstrumentationBundle` has always used `NOOP_SPAN` for the ENABLED + // bundle (`observability/tracing.ts`). Phase 4a shipped `undefined` here because no `Span` type existed + // yet and recorded the gap as a partial deviation; `Span`/`NOOP_SPAN` landed in Phase 7b, so the reason + // expired and the two bundles no longer disagree. Identity, not shape: a caller narrowing `unknown` may + // compare against the exported singleton. + test('carries the no-op span singleton, not undefined', () => { + expect(noopInstrumentationBundle.activeSpan).toBe(NOOP_SPAN); + }); + + test('tracerFactory emits nothing and is safe to invoke repeatedly (CTX-20)', () => { + expect(noopInstrumentationBundle.tracerFactory('op-a')).toBeUndefined(); + expect(noopInstrumentationBundle.tracerFactory('op-b')).toBeUndefined(); + }); + + test('is frozen', () => { + expect(Object.isFrozen(noopInstrumentationBundle)).toBe(true); + }); +}); diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts new file mode 100644 index 0000000..5eb7b08 --- /dev/null +++ b/packages/core/src/context/instrumentation.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/instrumentation.ts +import {NOOP_SPAN} from '../observability/span.js'; + +/** + * Correlation/instrumentation bundle every execution context carries (CTX-14), reachable from a + * custom step as `ctx.context.instrumentation`. + * + * **Two members are still typed `unknown`, and the reason this note used to give is no longer true.** + * `activeSpan` and `tracerFactory` are `unknown` rather than a `Span`/`Tracer`. The original reason — + * "nothing in the package consumes either yet, pending Phase 7a" — expired when Phase 7a landed + * (`bd37a08`) and shipped `Span` and `Tracer` in `observability/tracing.ts`. That phase did not narrow + * these two: `tracerFactory` is consumed, by `pipeline/runtime.ts:60-65` and + * `observability/logging-step.ts:305-312`, both of which reach a `Tracer` through a cast; + * `createInstrumentationBundle` fills `activeSpan` (`observability/tracing.ts:191`), and nothing in + * this package reads it back. + * + * They stay `unknown` because narrowing a published member is a breaking change rather than a + * maintenance edit — it widens what a caller may pass and narrows what they receive — so it belongs + * to a deliberate version bump. Treat either as opaque until then. Every other member below is stable. + * + * @public + */ +export interface InstrumentationBundle { + /** W3C trace-id, 32 lower-case hex characters. All-zero when tracing is disabled (CTX-15). */ + readonly traceId: string; + /** W3C span-id, 16 lower-case hex characters. All-zero when tracing is disabled (CTX-15). */ + readonly spanId: string; + /** W3C trace-flags byte; bit 0 is the sampled flag. `0` when tracing is disabled. */ + readonly traceFlags: number; + /** W3C tracestate header value, verbatim. Empty when tracing is disabled. */ + readonly traceState: string; + /** How `traceId`/`spanId` are encoded; `'none'` when tracing is disabled (CTX-15). */ + readonly traceIdEncoding: string; + /** Whether this bundle carries a usable trace context. `false` for the disabled default (CTX-15). */ + readonly isValid: boolean; + /** Whether the trace context was propagated in from a caller rather than started locally. */ + readonly isRemote: boolean; + /** + * The span this call runs inside. Never absent: the disabled-tracing default carries the inert + * `NOOP_SPAN` singleton, which is what CTX-15's "a no-op span" asks for. + * + * PROVISIONAL: typed `unknown` pending Phase 7a's tracing adapter — see this interface's own note. + */ + readonly activeSpan: unknown; + /** + * Returns the tracer to open `operationName`'s span from — a **tracer**, not a started span. Every + * consumer in this package narrows the result and calls `startSpan()` on it itself + * (`packages/core/src/pipeline/runtime.ts:60-65`, `observability/logging-step.ts:305-312`), and + * `createInstrumentationBundle` supplies a `(operationName: string) => Tracer` + * (`observability/tracing.ts:180,191`). A no-op returning `undefined` when tracing is disabled; + * both consumers substitute `NOOP_TRACER` for that `undefined`. + * + * **It is asked twice per call, for two different scopes.** `Runtime.send()` asks for + * `'http.client.operation'`'s tracer once per call and opens `OBS-29`'s one-per-operation span from + * it, outside every pillar, so a retry attempt and a redirect hop stay inside it. The LOGGING pillar + * step asks again per transmission, under `CTX-16`'s operation name when the pipeline was built with + * one and `'http.client.request'` otherwise, and its spans are children of the first. Returning one + * shared tracer for both is fine; `OBS-29`'s 1:1 clause is about the operation *span*, which + * `send()` opens exactly once whatever this returns. + * + * A consumer reaches this by building a bundle with `createInstrumentationBundle(tracerFactory)` and + * passing it as `PipelineOptions.instrumentation` — to `new PipelineBuilder(transport, options)` or + * to `standardResilience(transport, options)`. Before 2026-09-05 there was no public route: every + * pipeline a consumer could build carried the no-op bundle (audit #67 / #80). + * + * PROVISIONAL: the return type is `unknown`, and narrowing it is a version-bump decision — see this + * interface's own note for why Phase 7a landing did not settle it. + * + * @param operationName - the operation whose span the returned tracer will be asked to start. + * @returns the tracer for that operation, or `undefined` when tracing is disabled. + */ + readonly tracerFactory: (operationName: string) => unknown; +} + +/** + * The disabled-tracing default (CTX-15): reserved invalid sentinels, no-op span and tracer factory. Every + * field is constant, so call-key uniqueness (CTX-4) must not depend on any of them — see `context.ts`'s + * `Symbol()`-based keys. + * + * @internal + */ +export const noopInstrumentationBundle: InstrumentationBundle = Object.freeze({ + traceId: '00000000000000000000000000000000', + spanId: '0000000000000000', + traceFlags: 0, + traceState: '', + traceIdEncoding: 'none', + isValid: false, + isRemote: false, + activeSpan: NOOP_SPAN, + tracerFactory: () => undefined, +}); diff --git a/packages/core/src/context/store.test.ts b/packages/core/src/context/store.test.ts new file mode 100644 index 0000000..fdf3ee3 --- /dev/null +++ b/packages/core/src/context/store.test.ts @@ -0,0 +1,272 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/store.test.ts +// Exercises: XCUT-14 (a process-lived map whose key space callers influence carries a hard cap and +// drains back under it with a LOOP after each insert, so an insert burst converges to the bound +// instead of sitting above it -- see the burst test near the end of this file), +// CTX-3 (all three flavors collapse to one slot, successive promotions overwriting it), +// CTX-4 (two contexts sharing identical trace AND span id get distinct keys and both +// register), CTX-8 (install-or-replace never throws; reject-on-duplicate fails naming the key), +// CTX-9/CTX-10 (identity-conditional close, intermediate-link close is a no-op), CTX-11/CTX-12 (bounded, +// post-insert drain loop), CTX-17 (a never-promoted dispatch context leaves no entry; its close is a +// harmless no-op), CTX-13 (arbitrary victim; no entry is promised to survive), CTX-18 (unknown-key +// lookup/close are well-defined no-ops), CTX-19 (strong refs), +// XCUT-14 (a caller-keyed process-lived map -- "context registries" is the requirement's own first +// example -- carries a hard cap and a post-insert drain loop, and a burst never leaves it stuck above) +// +// Every test builds its own `new ContextStore()`. The exported `contextStore` singleton is module-level +// mutable state shared by every test file in a `bun test` run -- 4c's runtime.test.ts installs into that +// same object -- so an absolute `size` assertion against it reads a counter a sibling file can move, and a +// blanket clear() wipes a sibling's entries. docs/knowledge/harvested/testing.md:50,52. The singleton gets exactly +// one assertion here: that it is a ContextStore. +import {describe, expect, test} from 'bun:test'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {InvariantViolation} from '../invariant.js'; +import { + createDispatchContext, + promoteToExchange, + promoteToRequest, +} from './context.js'; +import {DuplicateContextKeyError} from './errors.js'; +import {ContextStore, contextStore} from './store.js'; + +function aRequest(): Request { + return Request.newBuilder().url('https://example.com').build(); +} + +function aResponse(request: Request): Response { + return Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .build(); +} + +describe('install / installIfAbsent (CTX-8)', () => { + test('install never throws and is retrievable by key', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + store.install(context); + expect(store.get(context.key)).toBe(context); + }); + + test('install unconditionally overwrites an existing occupant', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + store.install(context); + const promoted = promoteToRequest(context, aRequest()); + store.install(promoted); + expect(store.get(context.key)).toBe(promoted); + }); + + test('installIfAbsent succeeds when the key is free', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + store.installIfAbsent(context); + expect(store.get(context.key)).toBe(context); + }); + + test('installIfAbsent on an occupied key throws DuplicateContextKeyError naming the key', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + store.installIfAbsent(context); + const other = createDispatchContext({ + instrumentation: context.instrumentation, + key: context.key, + }); + + let caught: unknown; + try { + store.installIfAbsent(other); + } catch (error) { + caught = error; + } + + // CTX-8 says the error names the key, so assert the field, not only the class -- the store passing + // the wrong symbol through would otherwise be invisible here, and a symbol does not survive the + // message-substring check the rest of the suite uses for named-field errors. + expect(caught).toBeInstanceOf(DuplicateContextKeyError); + expect((caught as DuplicateContextKeyError).key).toBe(context.key); + }); + + test('a rejected installIfAbsent leaves the incumbent in the slot', () => { + // CTX-8's "admits exactly one winner": the loser must not have displaced or corrupted the winner. + const store = new ContextStore(); + const winner = createDispatchContext(); + store.installIfAbsent(winner); + const loser = createDispatchContext({key: winner.key}); + + expect(() => { + store.installIfAbsent(loser); + }).toThrow(DuplicateContextKeyError); + + expect(store.get(winner.key)).toBe(winner); + expect(store.size).toBe(1); + }); +}); + +describe('call-key uniqueness under an identical bundle (CTX-4)', () => { + test('two contexts sharing identical trace AND span id get distinct keys and both register', () => { + // §7's own Conformance clause for CTX-4, transcribed. Both contexts carry the very same + // noopInstrumentationBundle -- identical traceId, spanId, flags, state -- which is exactly the + // disabled-tracing case CTX-15 warns about. Symbol() keys make them distinct anyway, so neither + // evicts the other. + const store = new ContextStore(); + const a = createDispatchContext(); + const b = createDispatchContext(); + expect(a.instrumentation).toBe(b.instrumentation); + expect(a.key).not.toBe(b.key); + + store.install(a); + store.install(b); + expect(store.get(a.key)).toBe(a); + expect(store.get(b.key)).toBe(b); + expect(store.size).toBe(2); + }); +}); + +describe('one slot for the whole chain (CTX-3)', () => { + test('all three flavors register under the identical slot, each promotion overwriting the last', () => { + // CTX-3's store-level clause: "all three flavors register under the identical store slot and + // successive promotions overwrite one entry." Asserted here rather than in context.test.ts, which + // can only show the keys match -- that they collapse to ONE entry needs a store. + const store = new ContextStore(); + const dispatch = createDispatchContext(); + const request = aRequest(); + const requestCtx = promoteToRequest(dispatch, request, 'GetWidget'); + const exchangeCtx = promoteToExchange(requestCtx, aResponse(request)); + + store.install(dispatch); + store.install(requestCtx); + store.install(exchangeCtx); + + expect(store.size).toBe(1); + expect(store.get(dispatch.key)).toBe(exchangeCtx); + }); +}); + +describe('no auto-registration at construction (CTX-17)', () => { + test('a freshly constructed dispatch context is not in the store', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + expect(store.get(context.key)).toBeUndefined(); + expect(store.size).toBe(0); + }); + + test('promoting registers nothing either, and closing the unregistered source is a harmless no-op', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + promoteToRequest(context, aRequest()); // promotion alone registers nothing in 4a -- see below + expect(store.size).toBe(0); + expect(() => { + store.close(context); + }).not.toThrow(); + }); + + // CTX-17's other half -- "the first store entry is installed by the first promotion" -- is NOT + // satisfied here: promoteToRequest/promoteToExchange are pure and never touch the store, so an + // explicit store.install(...) is what registers anything. That call belongs to 4c's pipeline, + // which owns the store handle. Tracked as a deferral in this plan's Self-Review, not an omission. +}); + +describe('close (CTX-9, CTX-10)', () => { + test('evicts when the closing context is the current occupant', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + store.install(context); + store.close(context); + expect(store.get(context.key)).toBeUndefined(); + }); + + test('closing an intermediate link already superseded by promotion is a no-op', () => { + const store = new ContextStore(); + const dispatch = createDispatchContext(); + store.install(dispatch); + const promoted = promoteToRequest(dispatch, aRequest()); + store.install(promoted); // furthest-reached link now occupies the slot + + store.close(dispatch); // intermediate link -- must not evict the live promoted occupant + expect(store.get(dispatch.key)).toBe(promoted); + }); + + test('closing an unknown or already-removed key is a well-defined no-op (CTX-18)', () => { + const store = new ContextStore(); + const context = createDispatchContext(); + expect(() => { + store.close(context); + }).not.toThrow(); + store.install(context); + store.close(context); + expect(() => { + store.close(context); + }).not.toThrow(); + }); +}); + +describe('lookup (CTX-18)', () => { + test('an unknown key returns undefined, never throws', () => { + expect(new ContextStore().get(Symbol('unknown'))).toBeUndefined(); + }); +}); + +describe('bounded drain (CTX-11, CTX-12, CTX-13)', () => { + // These pin the BOUND, not the drain's shape. `install`/`installIfAbsent` each set one key before + // draining, so the map is never more than one over the cap and a single check-then-evict would pass + // every assertion here -- verified by mutation. CTX-12/XCUT-14's loop is retained for runtimes where + // concurrent inserts stack overshoots; see the note on `#drain`. + + test('a burst of inserts past the cap converges the store to at or under the cap', () => { + const store = new ContextStore(5); + for (let i = 0; i < 50; i += 1) { + store.install(createDispatchContext()); + expect(store.size).toBeLessThanOrEqual(5); // drains after every single insert, never overshoots + } + // Negative space: bounding only from above would also pass for a store that retained nothing at all. + // 50 distinct keys against a cap of 5 must leave the store saturated, not empty. + expect(store.size).toBe(5); + }); + + test('installIfAbsent also drains after a successful insert', () => { + const store = new ContextStore(2); + for (let i = 0; i < 10; i += 1) { + store.installIfAbsent(createDispatchContext()); + } + expect(store.size).toBe(2); + }); + + test('a cap below 1 is rejected at construction', () => { + // The constructor is the only place this is checked, which is what lets #drain skip an unreachable + // in-loop undefined guard. A bad cap is a violated precondition -- a programmer error -- so it fails + // through invariant (assertions.md:4, error-handling.md:36), not an ad-hoc throw. + expect(() => new ContextStore(0)).toThrow(InvariantViolation); + expect(() => new ContextStore(-1)).toThrow(InvariantViolation); + expect(() => new ContextStore(1.5)).toThrow(InvariantViolation); + expect(() => new ContextStore(1)).not.toThrow(); + }); +}); + +describe('clear', () => { + test('drops every entry, leaving the store reusable', () => { + const store = new ContextStore(); + const kept = createDispatchContext(); + store.install(kept); + store.install(createDispatchContext()); + + store.clear(); + + expect(store.size).toBe(0); + expect(store.get(kept.key)).toBeUndefined(); + store.install(kept); // still usable afterwards -- clear() resets entries, not the cap + expect(store.get(kept.key)).toBe(kept); + }); +}); + +describe('the process-wide singleton', () => { + test('is a real ContextStore instance', () => { + // The only assertion this file makes against the singleton: it is shared with every other test file + // in the run, so nothing behavioural may be asserted through it. + expect(contextStore).toBeInstanceOf(ContextStore); + }); +}); diff --git a/packages/core/src/context/store.ts b/packages/core/src/context/store.ts new file mode 100644 index 0000000..49ab4a4 --- /dev/null +++ b/packages/core/src/context/store.ts @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/context/store.ts +import {invariant} from '../invariant.js'; +import type {ExecutionContext} from './context.js'; +import {DuplicateContextKeyError} from './errors.js'; + +// Backstop cap (CTX-11, XCUT-14); a leaked context pins its whole request/response graph, including a +// possibly unread body holding a connection. +const DEFAULT_MAX_ENTRIES = 10_000; + +/** + * A bounded, keyed store of in-flight execution contexts (CTX-7..13, CTX-18, CTX-19). Also the textbook + * subject of `XCUT-14`, which names "context registries" first among the caller-keyed process-lived maps + * that MUST carry a hard cap and drain back under it in a loop after each insert — an unbounded one is a + * memory-exhaustion vector, not merely a leak. Thread-safety is + * satisfied by construction: Node's single-threaded event loop means no two synchronous Map mutations + * ever interleave, collapsing the reference's concurrent-map requirement into a plain Map. The Map holds + * strong references — never WeakRef/WeakMap — so a registered context keeps its whole Request+Response + * graph reachable and the cap, not the collector, is the leak backstop (CTX-19). + * + * @internal + */ +export class ContextStore { + readonly #entries = new Map<symbol, ExecutionContext>(); + readonly #maxEntries: number; + + /** + * @throws InvariantViolation when `maxEntries` is not a positive integer — a violated precondition, + * never an operational failure a caller recovers from. + */ + constructor(maxEntries: number = DEFAULT_MAX_ENTRIES) { + // A bad cap is a violated precondition — a programmer error — so it crashes at the fault via the + // project's one assertion primitive rather than an ad-hoc `if (!x) throw` + // (docs/knowledge/harvested/assertions.md:4, docs/knowledge/harvested/error-handling.md:36). + invariant( + Number.isInteger(maxEntries) && maxEntries >= 1, + `maxEntries must be a positive integer, got ${String(maxEntries)}`, + ); + this.#maxEntries = maxEntries; + } + + /** + * Install-or-replace; never throws (CTX-8). Nothing in 4a calls this — the promotion functions are + * pure and never touch a store (CTX-17's negative half); 4c's pipeline is the first caller. + */ + install(context: ExecutionContext): void { + this.#entries.set(context.key, context); + this.#drain(); + invariant( + this.#entries.size <= this.#maxEntries, + 'context store above its cap after a drain', + ); + } + + /** + * Install only if absent; every other concurrent caller fails (CTX-8). + * + * @throws DuplicateContextKeyError when the key is already occupied. The error carries the offending + * `key` as a field — the symbol itself, not just its rendering in the message. + */ + installIfAbsent(context: ExecutionContext): void { + if (this.#entries.has(context.key)) { + throw new DuplicateContextKeyError(context.key); + } + this.#entries.set(context.key, context); + this.#drain(); + invariant( + this.#entries.size <= this.#maxEntries, + 'context store above its cap after a drain', + ); + } + + /** Absent key returns undefined, never throws (CTX-18). */ + get(key: symbol): ExecutionContext | undefined { + return this.#entries.get(key); + } + + /** + * Evicts the slot only when the current occupant IS `context` (reference identity, CTX-9). Closing an + * intermediate link already superseded by a later promotion, or an unknown/already-removed key, is a + * well-defined no-op (CTX-10, CTX-18). + */ + close(context: ExecutionContext): void { + if (this.#entries.get(context.key) === context) { + this.#entries.delete(context.key); + } + } + + /** + * Drops every entry. Not part of `§7`'s contract — it exists so a test that must observe the shared + * singleton (4c's runtime tests) can reset it. Prefer constructing an isolated `ContextStore`. + */ + clear(): void { + this.#entries.clear(); + } + + /** Entries currently tracked; at or below the cap once inserts quiesce (CTX-11, CTX-13). */ + get size(): number { + return this.#entries.size; + } + + #drain(): void { + // CTX-12 / XCUT-14: a loop, not a single check-then-evict, so an insert burst converges to the cap. + // + // DO NOT "simplify" this loop into an `if`. On this runtime the two are behaviorally identical and + // no test can tell them apart: both callers set exactly one key before draining, so the map is never + // more than one over the cap at entry and the loop never needs a second pass. The loop survives + // because CTX-12 and XCUT-14 mandate the shape for runtimes where concurrent inserts can stack + // several overshoots before any drain runs — the burst test below pins the bound, not the shape. + // + // CTX-13: victim selection is arbitrary — oldest-inserted (Map iteration order) is the cheapest + // choice, not a retention promise; callers must not rely on any particular entry surviving. + // + // No undefined-guard inside the loop: the constructor rejects maxEntries < 1, so `size > maxEntries` + // proves size >= 2 and the iterator always yields. A guard here would be unreachable code the + // coverage gate could never exercise. + for (const oldestKey of this.#entries.keys()) { + if (this.#entries.size <= this.#maxEntries) return; + this.#entries.delete(oldestKey); + } + } +} + +/** + * The one registry 4c's `Runtime.send()` installs into. Module-level mutable state, which + * `docs/knowledge/harvested/variables-and-declarations.md:22` bans — accepted here because threading a store handle + * through builder → runtime → every step would be a wide API change for no observable gain, and logged in + * the design's Deviation Ledger for Phase 10. Tests must build their own `new ContextStore()` rather than + * asserting through this one: it is shared by every test file in a `bun test` run. + * + * @internal + */ +export const contextStore = new ContextStore(); diff --git a/packages/core/src/generated/version.ts b/packages/core/src/generated/version.ts new file mode 100644 index 0000000..1615b3a --- /dev/null +++ b/packages/core/src/generated/version.ts @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/generated/version.ts +// Generated by scripts/gen-version.mjs from package.json -- do not edit by hand. + +/** The published version of `@dexpace/core`, compiled in at build time (NFR-15). @internal */ +export const SDK_VERSION = '0.0.0'; diff --git a/packages/core/src/http/ascii-validation.test.ts b/packages/core/src/http/ascii-validation.test.ts index 33511aa..ae90b81 100644 --- a/packages/core/src/http/ascii-validation.test.ts +++ b/packages/core/src/http/ascii-validation.test.ts @@ -3,24 +3,26 @@ // Exercises: HTTP-18 (outbound value grammar: HTAB + printable ASCII 0x20-0x7E only) import {describe, expect, test} from 'bun:test'; import { - hasForbiddenOutboundByte, + hasForbiddenOutboundValueByte, hasForbiddenNameByte, hasForbiddenInboundValueByte, } from './ascii-validation.js'; -describe('hasForbiddenOutboundByte', () => { +describe('hasForbiddenOutboundValueByte', () => { test('accepts HTAB and printable ASCII', () => { - expect(hasForbiddenOutboundByte('a\tb')).toBe(false); - expect(hasForbiddenOutboundByte('printable ASCII 0x20-0x7E')).toBe(false); + expect(hasForbiddenOutboundValueByte('a\tb')).toBe(false); + expect(hasForbiddenOutboundValueByte('printable ASCII 0x20-0x7E')).toBe( + false, + ); }); test('rejects CR/LF and other control characters', () => { - expect(hasForbiddenOutboundByte('a\r\nb')).toBe(true); - expect(hasForbiddenOutboundByte('a\0b')).toBe(true); + expect(hasForbiddenOutboundValueByte('a\r\nb')).toBe(true); + expect(hasForbiddenOutboundValueByte('a\0b')).toBe(true); }); test('rejects non-ASCII bytes', () => { - expect(hasForbiddenOutboundByte('vålue')).toBe(true); + expect(hasForbiddenOutboundValueByte('vålue')).toBe(true); }); }); diff --git a/packages/core/src/http/ascii-validation.ts b/packages/core/src/http/ascii-validation.ts index ff650c4..f5811a2 100644 --- a/packages/core/src/http/ascii-validation.ts +++ b/packages/core/src/http/ascii-validation.ts @@ -10,7 +10,7 @@ * @param value - the text to inspect. * @returns `true` when at least one byte is forbidden. */ -export function hasForbiddenOutboundByte(value: string): boolean { +export function hasForbiddenOutboundValueByte(value: string): boolean { for (const ch of value) { const code = ch.codePointAt(0) ?? 0; const allowed = code === 0x09 || (code >= 0x20 && code <= 0x7e); @@ -38,7 +38,7 @@ export function hasForbiddenNameByte(value: string): boolean { * Reports whether `value` contains a byte forbidden in an *inbound* header value: control * characters (C0 except HTAB, plus DEL) only. * - * Deliberately laxer than {@link hasForbiddenOutboundByte} — RFC 7230 permits obs-text (≥ 0x80) in + * Deliberately laxer than {@link hasForbiddenOutboundValueByte} — RFC 7230 permits obs-text (≥ 0x80) in * a response field value, and applying the outbound grammar inbound would silently drop legitimate * headers such as a Latin-1 `Content-Disposition` filename (HTTP-19). * diff --git a/packages/core/src/http/builder.ts b/packages/core/src/http/builder.ts index 5411659..c4bc1e4 100644 --- a/packages/core/src/http/builder.ts +++ b/packages/core/src/http/builder.ts @@ -20,6 +20,22 @@ export interface Builder<T> { build(): T; } +/** + * The one frozen empty list every multi-value accessor returns for an absent name. + * + * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: + * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and the + * TSDoc on `Headers.getAll` and `QueryParams.getAll` promises a frozen list on every path. Both + * returned a fresh `[]` on a miss, which was neither (audit #67 / #76). Sharing one instance is + * safe precisely because it is frozen — there is no state in it to alias, and no caller can add + * any. + * + * Lives here rather than in either model because both need it and the two models deliberately + * import nothing from each other; this module is already the shared construction helper they both + * import from. + */ +export const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); + /** * Returns `value` when present, throwing a field-named error when it is `null` or `undefined`. * diff --git a/packages/core/src/http/charset.ts b/packages/core/src/http/charset.ts new file mode 100644 index 0000000..a1910b2 --- /dev/null +++ b/packages/core/src/http/charset.ts @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/http/charset.ts +import {MediaType} from './media-type.js'; + +/** + * HTTP-42's charset resolution: the media type's declared `charset`, falling back to UTF-8 when the + * media type is absent or unparseable. Never throws. + */ +export function resolveCharset(mediaType: string | undefined): string { + if (mediaType === undefined) return 'utf-8'; + try { + return MediaType.parse(mediaType).charset ?? 'utf-8'; + } catch { + return 'utf-8'; + } +} + +/** + * Decodes a whole message body with `charset`, falling back to UTF-8 when the label is unknown + * (HTTP-42). `TextDecoder` throws a RangeError on an unrecognized label, which callers on an error path + * are least able to handle. + * + * NOT interchangeable with `io/text-codec.ts`'s `decodeText`, despite the similar shape -- the two + * disagree by design and the name says so: + * + * - This one is whole-body decoding at the HTTP layer. It delegates every label to `TextDecoder`, so + * `iso-8859-1` follows the WHATWG Encoding Standard's mapping onto windows-1252 (0x80 decodes to + * U+20AC), and it consumes a leading BOM, which is what a caller of `Response.text()` expects. + * - `io/text-codec.decodeText` is per-FRAGMENT decoding at the byte layer. It implements true + * ISO-8859-1 so that IO-13's write/read round-trip holds against `encodeText`, and sets + * `ignoreBOM` so a U+FEFF appearing mid-stream survives as ordinary data (SSE-12). + * + * Reaching for the wrong one silently changes bytes. Pick by layer: message bodies here, stream + * fragments there. + */ +export function decodeBodyText(bytes: Uint8Array, charset: string): string { + try { + return new TextDecoder(charset).decode(bytes); + } catch { + return new TextDecoder('utf-8').decode(bytes); + } +} diff --git a/packages/core/src/http/errors.test.ts b/packages/core/src/http/errors.test.ts index 6e799ad..494d2d6 100644 --- a/packages/core/src/http/errors.test.ts +++ b/packages/core/src/http/errors.test.ts @@ -1,12 +1,20 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/errors.test.ts -// Exercises: HTTP-4 (field-named errors), HTTP-20 (no value echo, escaped name) +// Exercises: HTTP-4 (field-named errors), HTTP-7 (body on a body-forbidding method), +// HTTP-20 (no value echo, escaped name) import {describe, expect, test} from 'bun:test'; import { DexpaceError, - DomainModelError, + isDomainModelError, RequiredFieldError, HeaderValidationError, + MediaTypeParseError, + ProtocolParseError, + UrlConstructionError, + RequestOptionsValidationError, + EtagParseError, + HttpRangeValidationError, + RequestConditionsValidationError, toError, RequestBodyNotAllowedError, } from './errors.js'; @@ -55,16 +63,40 @@ describe('RequestBodyNotAllowedError', () => { }); }); -// Exercises: the Phase 2 retrofit — DexpaceError as the taxonomy root above DomainModelError +// Exercises: the flattened taxonomy — DexpaceError is the single root, and `isDomainModelError` is +// the group check that replaced the removed `DomainModelError` class tier describe('DexpaceError', () => { test('sets name to the concrete subclass name', () => { const error = new DexpaceError('boom'); expect(error.name).toBe('DexpaceError'); }); - test('DomainModelError is a DexpaceError, and every existing leaf still narrows by DomainModelError', () => { - const error = new RequiredFieldError('url'); - expect(error).toBeInstanceOf(DomainModelError); - expect(error).toBeInstanceOf(DexpaceError); + test('every domain-model leaf sits two levels down and isDomainModelError matches it', () => { + const leaves = [ + new RequiredFieldError('url'), + new HeaderValidationError('name', 'X-Trace', undefined), + new MediaTypeParseError('bad media type'), + new ProtocolParseError('bad protocol'), + new UrlConstructionError('bad url'), + new RequestOptionsValidationError('bad options'), + new EtagParseError('bad etag'), + new HttpRangeValidationError('bad range'), + new RequestConditionsValidationError('bad conditions'), + new RequestBodyNotAllowedError('GET'), + ]; + expect(leaves).toHaveLength(10); + for (const leaf of leaves) { + expect(leaf).toBeInstanceOf(DexpaceError); + expect(isDomainModelError(leaf)).toBe(true); + // Two levels, not three: the leaf's own superclass is DexpaceError itself, so a + // reintroduced tier fails here rather than passing silently through `instanceof`. + expect(Object.getPrototypeOf(leaf.constructor)).toBe(DexpaceError); + } + }); + + test('an error outside the domain model is not matched', () => { + expect(isDomainModelError(new DexpaceError('boom'))).toBe(false); + expect(isDomainModelError(new Error('boom'))).toBe(false); + expect(isDomainModelError(undefined)).toBe(false); }); }); diff --git a/packages/core/src/http/errors.ts b/packages/core/src/http/errors.ts index 1bbee1a..bc8e01a 100644 --- a/packages/core/src/http/errors.ts +++ b/packages/core/src/http/errors.ts @@ -18,18 +18,6 @@ export class DexpaceError extends Error { } } -/** - * The root of every error the HTTP domain model throws. - * - * Catch this to handle any construction, validation, or parse failure from the model uniformly; - * catch a leaf subclass to distinguish a specific failure. A sibling of the seam layer's - * {@link DexpaceError}-rooted errors (`CancellationError`, `OperationAssemblyError`) — a cancelled - * transport call or an unassembled operation is not itself a domain-model construction failure. - * - * @public - */ -export class DomainModelError extends DexpaceError {} - // Narrows a caught `unknown` into an Error (styleguide 8.4). Defined once, imported everywhere a caught // value becomes a `cause`. Must never itself throw from inside a catch — String() can throw on a // null-prototype object or a hostile toString, hence the inner try. @@ -59,7 +47,7 @@ export function toError(value: unknown): Error { * * @public */ -export class RequiredFieldError extends DomainModelError { +export class RequiredFieldError extends DexpaceError { /** The name of the field that was missing. */ readonly fieldName: string; @@ -94,7 +82,7 @@ function escapeControlChars(input: string): string { * * @public */ -export class HeaderValidationError extends DomainModelError { +export class HeaderValidationError extends DexpaceError { /** Whether the header's name or its value failed validation. */ readonly kind: 'name' | 'value'; /** @@ -128,7 +116,7 @@ export class HeaderValidationError extends DomainModelError { * * @public */ -export class MediaTypeParseError extends DomainModelError {} +export class MediaTypeParseError extends DexpaceError {} /** * Thrown when a protocol identifier is not one of the recognized HTTP versions or their aliases @@ -136,23 +124,50 @@ export class MediaTypeParseError extends DomainModelError {} * * @public */ -export class ProtocolParseError extends DomainModelError {} +export class ProtocolParseError extends DexpaceError {} /** - * Thrown when a request URL is malformed or not absolute; the message carries the offending input - * and the underlying parse failure is chained as `cause` (HTTP-47). + * Thrown when a URL cannot be constructed from what a caller supplied. + * + * Three cases, all of them "this input has no URL form": + * + * - A request URL that is malformed or not absolute (HTTP-47). The message carries the offending + * input and the underlying parse failure is chained as `cause`. + * - A base URL handed to `buildRequest()` that is malformed, not absolute, or carries a fragment + * (SEAM-27). + * - A query-parameter name or value carrying an unpaired surrogate. Such a string has no UTF-8 + * form, so RFC 3986 percent-encoding is undefined for it and `QueryParams.encode()` could only + * fail; `QueryParamsBuilder.add` rejects it at the call that supplied it instead. `cause` is not + * set on this path — nothing was caught, the input was inspected (HTTP-29, audit #67 / #76). + * `QueryParams.parse` does NOT throw it: HTTP-31 makes parsing lenient, so it substitutes U+FFFD. + * Pagination's query splice — `spliceQueryParam` and `readQueryParam`, behind `cursorStrategy` + * and `pageNumberStrategy` — rejects the same input for the same reason, through the same + * predicate (PAGE-22, audit #67 / #79). That one is reachable without any caller mistake, since + * the cursor is server-supplied; its message names the parameter and never echoes the value. * * @public */ -export class UrlConstructionError extends DomainModelError {} +export class UrlConstructionError extends DexpaceError {} /** - * Thrown when a per-call operational override is out of range — a non-null timeout that is zero or - * negative, or a negative max-retries (HTTP-35). + * Thrown when a per-call operational override is out of range (HTTP-35). + * + * The ranges checked are the FULL ranges, not the lower bounds the requirement's own wording names. + * HTTP-35's point is that an out-of-range override is a loud error at the call site that supplied + * it, never a value reinterpreted downstream, and a value that only *some* consumer refuses is the + * same failure moved one seam away: + * + * - `timeoutMs` must be an integer in `1 .. 2**32 - 1` — the range `AbortSignal.timeout()` accepts, + * which is the only one a transport can honour. Zero, negatives, `Infinity`, `NaN`, a fractional + * millisecond and anything above the ceiling are all rejected. Zero is rejected rather than + * reinterpreted: it means "no timeout" in one transport and is an error in another. + * - `maxRetries` must be a non-negative integer. `0` is accepted and means "disable retries for + * this call", distinct from `undefined`; `Infinity` and `NaN` are rejected because they make a + * retry driver's ceiling test permanently false and its loop unbounded. * * @public */ -export class RequestOptionsValidationError extends DomainModelError {} +export class RequestOptionsValidationError extends DexpaceError {} /** * Thrown when an ETag is unterminated, has an empty strong opaque tag, or contains a character @@ -160,7 +175,7 @@ export class RequestOptionsValidationError extends DomainModelError {} * * @public */ -export class EtagParseError extends DomainModelError {} +export class EtagParseError extends DexpaceError {} /** * Thrown when a byte range is invalid — a negative offset, a non-positive length, an overflowing @@ -168,7 +183,7 @@ export class EtagParseError extends DomainModelError {} * * @public */ -export class HttpRangeValidationError extends DomainModelError {} +export class HttpRangeValidationError extends DexpaceError {} /** * Thrown when conditional-request state is contradictory — mixing the any-tag (`*`) with a concrete @@ -176,7 +191,7 @@ export class HttpRangeValidationError extends DomainModelError {} * * @public */ -export class RequestConditionsValidationError extends DomainModelError {} +export class RequestConditionsValidationError extends DexpaceError {} /** * Thrown when a request carries a body on a method whose classification forbids one — GET, HEAD, @@ -184,7 +199,7 @@ export class RequestConditionsValidationError extends DomainModelError {} * * @public */ -export class RequestBodyNotAllowedError extends DomainModelError { +export class RequestBodyNotAllowedError extends DexpaceError { /** * @param method - the method that forbids a body, named in the message. */ @@ -192,3 +207,47 @@ export class RequestBodyNotAllowedError extends DomainModelError { super(`method ${method} does not allow a request body`); } } + +/** + * Groups every error the HTTP domain model throws, with no class tier between those leaves and + * {@link DexpaceError} — the corpus caps custom error hierarchies at two levels. This is the + * replacement for the `DomainModelError` class, which was removed: a `catch` that read + * `error instanceof DomainModelError` reads `isDomainModelError(error)` instead, and narrows to the + * same union. + * + * Matches any construction, validation, or parse failure the model raises; test against a leaf class + * to distinguish a specific one. Deliberately excludes the seam layer's own {@link DexpaceError} + * leaves (`CancellationError`, `OperationAssemblyError`) — a cancelled transport call or an + * unassembled operation is not itself a domain-model construction failure. + * + * @param error - the caught value. + * @returns whether `error` is one of the ten domain-model error classes. + * + * @public + */ +export function isDomainModelError( + error: unknown, +): error is + | RequiredFieldError + | HeaderValidationError + | MediaTypeParseError + | ProtocolParseError + | UrlConstructionError + | RequestOptionsValidationError + | EtagParseError + | HttpRangeValidationError + | RequestConditionsValidationError + | RequestBodyNotAllowedError { + return ( + error instanceof RequiredFieldError || + error instanceof HeaderValidationError || + error instanceof MediaTypeParseError || + error instanceof ProtocolParseError || + error instanceof UrlConstructionError || + error instanceof RequestOptionsValidationError || + error instanceof EtagParseError || + error instanceof HttpRangeValidationError || + error instanceof RequestConditionsValidationError || + error instanceof RequestBodyNotAllowedError + ); +} diff --git a/packages/core/src/http/headers.test.ts b/packages/core/src/http/headers.test.ts index 08cde02..618e6bb 100644 --- a/packages/core/src/http/headers.test.ts +++ b/packages/core/src/http/headers.test.ts @@ -1,9 +1,17 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/headers.test.ts -// Exercises: HTTP-13 (case-insensitive storage), HTTP-14 (multi-value add/set), HTTP-15 (null removes), +// Exercises: XCUT-18 (header name/value validation is the request-splitting defense, and it lives at the +// transport-agnostic model layer so no transport can be reached with a CR/LF-bearing header: names reject +// every C0 control byte INCLUDING HTAB plus DEL and non-ASCII; outbound values reject the same set EXCEPT +// HTAB; inbound values are lenient about obs-text but still reject control bytes), +// HTTP-13 (case-insensitive storage), HTTP-14 (multi-value add/set), HTTP-15 (null removes), // HTTP-16 (insertion order), HTTP-3 (newBuilder derivation doesn't alias), HTTP-5 (no live-builder leak), +// XCUT-15's ingested-collection clause (a builder defensively copies what it is handed, so mutating that +// collection after build() cannot alter the built model, and a derived builder never aliases its source), // HTTP-17 (outbound name validation + trim), HTTP-18 (outbound value validation), HTTP-19 (inbound leniency), -// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop) +// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop), +// HTTP-5 again (getAll returns a FROZEN list on both the present-name and the absent-name path), +// HTTP-13 once more (Headers.equals asserted directly, not only through Request.equals) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {Headers, HeaderName} from './headers.js'; @@ -245,3 +253,119 @@ describe('HeaderName.lowerCased (HTTP-21)', () => { expect(name.raw).toBe('Content-Type'); }); }); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const headers = Headers.newBuilder() + .add('X-Tag', 'a') + .add('X-Tag', 'b') + .build(); + + test('the present-name list is frozen', () => { + // Asserted directly for the first time by audit #67 / #76. `build()` freezes each value list, + // and `getAll` returns that same reference rather than a copy, so the freeze is the whole of + // HTTP-5's "cannot mutate the model through the returned value" on this accessor — if the + // freeze were ever dropped, nothing else would notice. + const values = headers.getAll('X-Tag'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('c')).toThrow(TypeError); + expect(headers.getAll('X-Tag')).toEqual(['a', 'b']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + // It was a fresh `[]` — unfrozen, against the TSDoc's promise of a frozen list, and a fresh + // allocation on every miss. + const first = headers.getAll('nope'); + const second = headers.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); + +// Reached only through `Request.equals` until audit #67 / #76. Each case below is a way the +// comparison could be wrong that a Request-level test would not isolate. +function buildHeaders(pairs: readonly (readonly [string, string])[]): Headers { + const builder = Headers.newBuilder(); + for (const [name, value] of pairs) builder.add(name, value); + return builder.build(); +} + +describe('Headers.equals directly (HTTP-13): names and casing', () => { + test('is reflexive and true for an identical construction', () => { + const a = buildHeaders([['X-A', '1']]); + expect(a.equals(a)).toBe(true); + expect(a.equals(buildHeaders([['X-A', '1']]))).toBe(true); + }); + + test('name casing does not participate — HTTP-13 folds names', () => { + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['x-a', '1']])), + ).toBe(true); + }); + + test('value casing DOES participate — only names are folded', () => { + expect( + buildHeaders([['X-A', 'v']]).equals(buildHeaders([['X-A', 'V']])), + ).toBe(false); + }); + + test('the order of distinct NAMES does not matter', () => { + const ab = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + const ba = buildHeaders([ + ['X-B', '2'], + ['X-A', '1'], + ]); + expect(ab.equals(ba)).toBe(true); + expect(ba.equals(ab)).toBe(true); + }); +}); + +describe('Headers.equals directly (HTTP-13): values, order and subsets', () => { + test('the order of VALUES under one name does matter (HTTP-14)', () => { + const ab = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'b'], + ]); + const ba = buildHeaders([ + ['X-T', 'b'], + ['X-T', 'a'], + ]); + expect(ab.equals(ba)).toBe(false); + }); + + test('a strict subset is not equal, in either direction', () => { + const one = buildHeaders([['X-A', '1']]); + const two = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('same name count, disjoint names, is not equal', () => { + // The length pre-check passes here, so this is the case that proves the per-name lookup runs. + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['X-B', '1']])), + ).toBe(false); + }); + + test('same names with different value COUNTS is not equal', () => { + const one = buildHeaders([['X-T', 'a']]); + const two = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'a'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('two empty instances are equal', () => { + expect( + Headers.newBuilder().build().equals(Headers.newBuilder().build()), + ).toBe(true); + }); +}); diff --git a/packages/core/src/http/headers.ts b/packages/core/src/http/headers.ts index 31ff0f1..f452d7e 100644 --- a/packages/core/src/http/headers.ts +++ b/packages/core/src/http/headers.ts @@ -1,10 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/headers.ts -import type {Builder} from './builder.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; import {HeaderValidationError} from './errors.js'; import { hasForbiddenNameByte, - hasForbiddenOutboundByte, + hasForbiddenOutboundValueByte, hasForbiddenInboundValueByte, } from './ascii-validation.js'; @@ -17,7 +17,7 @@ function validateName(name: string): string { } function validateOutboundValue(name: string, value: string): void { - if (hasForbiddenOutboundByte(value)) { + if (hasForbiddenOutboundValueByte(value)) { throw new HeaderValidationError('value', name, value); } } @@ -130,10 +130,14 @@ export class Headers { * Returns every value stored under `name`, in insertion order. * * @param name - the header name, as a string or a {@link HeaderName}. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string | HeaderName): readonly string[] { - return this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? []; + return ( + this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? + EMPTY_VALUE_LIST + ); } /** diff --git a/packages/core/src/http/index.ts b/packages/core/src/http/index.ts index 4ae37a7..be882e6 100644 --- a/packages/core/src/http/index.ts +++ b/packages/core/src/http/index.ts @@ -7,7 +7,7 @@ export type {Builder} from './builder.js'; export { DexpaceError, - DomainModelError, + isDomainModelError, RequiredFieldError, HeaderValidationError, MediaTypeParseError, diff --git a/packages/core/src/http/media-type.test.ts b/packages/core/src/http/media-type.test.ts index d4c8246..c327d0f 100644 --- a/packages/core/src/http/media-type.test.ts +++ b/packages/core/src/http/media-type.test.ts @@ -48,6 +48,34 @@ describe('charset', () => { test('is undefined, never throws, when absent or unknown', () => { expect(MediaType.parse('text/plain').charset).toBeUndefined(); }); + + test('HTTP-24: an UNKNOWN encoding label is null, not the label verbatim', () => { + // The requirement's own conformance text: `charset=utf-8` -> UTF-8; `charset=bogus` -> null; + // no charset -> null. "Unknown" is resolved against the runtime's WHATWG Encoding registry, + // which is what `new TextDecoder(label)` accepts. + expect(MediaType.parse('text/plain;charset=bogus').charset).toBeUndefined(); + expect( + MediaType.parse('text/plain;charset="not an encoding"').charset, + ).toBeUndefined(); + }); + + test('HTTP-24: every recognized label still comes back with its original case', () => { + expect(MediaType.parse('text/plain;charset=UTF-8').charset).toBe('UTF-8'); + expect(MediaType.parse('text/plain;charset=iso-8859-1').charset).toBe( + 'iso-8859-1', + ); + expect(MediaType.parse('text/plain;charset=Shift_JIS').charset).toBe( + 'Shift_JIS', + ); + }); + + test('HTTP-24: the raw parameter is still reachable verbatim', () => { + // `charset` answers "which encoding", and an unknown one is no encoding. `parameter('charset')` + // answers "what did the wire say", and still says it. + expect( + MediaType.parse('text/plain;charset=bogus').parameter('charset'), + ).toBe('bogus'); + }); }); describe('construction rejects forbidden bytes (HTTP-26)', () => { diff --git a/packages/core/src/http/media-type.ts b/packages/core/src/http/media-type.ts index b2afabf..d5b9393 100644 --- a/packages/core/src/http/media-type.ts +++ b/packages/core/src/http/media-type.ts @@ -1,7 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/media-type.ts import {MediaTypeParseError} from './errors.js'; -import {hasForbiddenOutboundByte} from './ascii-validation.js'; +import {hasForbiddenOutboundValueByte} from './ascii-validation.js'; const TOKEN_RE = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/; @@ -32,7 +32,7 @@ function splitRespectingQuotes(input: string, separator: string): string[] { } function validateNoForbiddenBytes(value: string): void { - if (hasForbiddenOutboundByte(value)) { + if (hasForbiddenOutboundValueByte(value)) { throw new MediaTypeParseError( `media type contains a forbidden character (${String(value.length)} chars)`, ); @@ -202,14 +202,23 @@ export class MediaType { } /** - * The `charset` parameter, resolved case-insensitively — `undefined` when absent, never throwing, - * so callers fall back to their own default (HTTP-24). + * The `charset` parameter, resolved case-insensitively — `undefined` when absent **or unknown**, + * never throwing, so callers fall back to their own default (HTTP-24). * - * The value is returned verbatim and is not checked against a registry of known encodings; an - * unrecognized name surfaces as-is rather than as `undefined`. + * "Unknown" is the runtime's own WHATWG Encoding registry: a label `TextDecoder` refuses is a + * label nothing in this SDK could decode with, so reporting it as a charset would only move the + * failure to whoever tried to use it. The label's original case is preserved for the ones it + * accepts (HTTP-23). + * + * The raw parameter stays reachable through {@link MediaType.parameter} — that getter answers + * "what did the wire say", this one answers "which encoding", and an unusable label is no + * encoding. {@link MediaType.render} likewise round-trips the parameter verbatim (HTTP-25), so + * the resolution here never rewrites the value. */ get charset(): string | undefined { - return this.parameter('charset'); + const declared = this.parameter('charset'); + if (declared === undefined) return undefined; + return isKnownEncoding(declared) ? declared : undefined; } /** @@ -259,3 +268,23 @@ export class MediaType { return true; } } + +/** + * Whether the runtime recognizes `label` as an encoding (HTTP-24's "unknown" half). + * + * `TextDecoder`'s constructor is the registry: it throws `RangeError` for a label outside the + * WHATWG Encoding Standard's table and accepts every alias inside it, which is a strictly better + * answer than any list this package could hand-maintain -- and it is the same resolution + * `decodeBodyText` performs one layer down, so the two cannot disagree about what is decodable. + * + * Not memoized. A caller-influenced label-to-boolean map is exactly the unbounded, process-lived + * cache XCUT-14 forbids, and construction is cheap next to the parse that produced the label. + */ +function isKnownEncoding(label: string): boolean { + try { + new TextDecoder(label); + return true; + } catch { + return false; + } +} diff --git a/packages/core/src/http/query-params.test.ts b/packages/core/src/http/query-params.test.ts index 7a71853..fb51ebf 100644 --- a/packages/core/src/http/query-params.test.ts +++ b/packages/core/src/http/query-params.test.ts @@ -1,10 +1,31 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.test.ts // Exercises: HTTP-28 (case-sensitive, multi-value, value-less param), HTTP-29/32 (RFC 3986 encoding), -// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse) +// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse), +// HTTP-5 (getAll returns a frozen list on every path, present name or absent) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; -import {QueryParams} from './query-params.js'; +import { + QueryParams, + decodeQueryComponent, + encodeQueryComponent, +} from './query-params.js'; +import {UrlConstructionError} from './errors.js'; + +/** + * Strings that mix ordinary text with UNPAIRED surrogate code units. `fc.string()` alone never + * produces one — its default unit is printable ASCII — so the URIError path it is here to cover + * would go ungenerated. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); describe('case-sensitive names and multi-value (HTTP-28)', () => { test('page and Page are distinct names', () => { @@ -124,3 +145,117 @@ describe('newBuilder derivation (HTTP-3)', () => { expect(original.getAll('x')).toEqual(['1', '2']); }); }); + +// PAGE-22 restates HTTP-29's rule. These assertions pin the shared function so the pagination splice can rely +// on it instead of restating the rule and drifting. +test('the component encoder is directly reachable and follows RFC 3986 (HTTP-29, reused by PAGE-22)', () => { + expect(encodeQueryComponent('a b')).toBe('a%20b'); + expect(encodeQueryComponent('a+b')).toBe('a%2Bb'); + expect(encodeQueryComponent('a/b')).toBe('a%2Fb'); + expect(encodeQueryComponent('a=b')).toBe('a%3Db'); + expect(encodeQueryComponent('a*b')).toBe('a%2Ab'); + expect(encodeQueryComponent('AZaz09-._~')).toBe('AZaz09-._~'); +}); + +test('the component decoder treats a literal + as data, not a space (HTTP-29, PAGE-22)', () => { + expect(decodeQueryComponent('a+b')).toBe('a+b'); + expect(decodeQueryComponent('a%20b')).toBe('a b'); + expect(decodeQueryComponent('a%2Bb')).toBe('a+b'); +}); + +test('the decoder falls back to raw text on malformed percent-encoding (HTTP-31)', () => { + expect(decodeQueryComponent('a%zzb')).toBe('a%zzb'); +}); + +describe('lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string carrying an unpaired surrogate. Before audit #67 / #76 that escaped the + // `DexpaceError` tree entirely, and it escaped from `encode()` and `equals()` — accessors whose + // TSDoc documents no throw at all — rather than from the `add()` that accepted the value. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate value', 'c', LONE_HIGH], + ['a lone low surrogate value', 'c', LONE_LOW], + ['a lone surrogate name', LONE_HIGH, 'v'], + ['a lone surrogate inside a longer value', 'c', `ok${LONE_HIGH}ok`], + ])('add() rejects %s with UrlConstructionError', (_label, name, value) => { + expect(() => QueryParams.newBuilder().add(name, value)).toThrow( + UrlConstructionError, + ); + }); + + test('a well-formed surrogate PAIR is accepted and encoded as UTF-8', () => { + // U+1F600, one code point spelled with two code units. Rejecting this too would make the check + // "no astral characters", which HTTP-29 does not say. + expect( + QueryParams.newBuilder().add('emoji', '\u{1F600}').build().encode(), + ).toBe('emoji=%F0%9F%98%80'); + }); + + test('parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing', () => { + // HTTP-31 is MUST-level about `parse` never throwing, so the strict `add()` path cannot be the + // one `parse` uses for a value it did not choose. Replacement matches what the platform's own + // query serializer does with the same input: `new URL('https://x/?a=\uD800').search` is + // `?a=%EF%BF%BD` (measured 2026-09-05). This is `Headers`' outbound/inbound split, applied to + // the query model. + const parsed = QueryParams.parse(`a=${LONE_HIGH}`); + expect(parsed.get('a')).toBe('�'); + expect(parsed.encode()).toBe('a=%EF%BF%BD'); + }); + + test('parse() sanitizes the NAME as well as the value', () => { + const parsed = QueryParams.parse(`${LONE_HIGH}=v`); + expect(parsed.has('�')).toBe(true); + expect(parsed.encode()).toBe('%EF%BF%BD=v'); + }); + + test('no URIError escapes encode() or equals(), whatever the builder admitted (property)', () => { + fc.assert( + fc.property(surrogateBearingString, surrogateBearingString, (n, v) => { + let params: QueryParams; + try { + params = QueryParams.newBuilder().add(n, v).build(); + } catch (e: unknown) { + expect(e).toBeInstanceOf(UrlConstructionError); + return; + } + expect(() => params.encode()).not.toThrow(); + expect(() => + params.equals(QueryParams.newBuilder().build()), + ).not.toThrow(); + }), + {numRuns: 500}, + ); + }); + + test('no anything escapes parse(), whatever it is handed (property, HTTP-31)', () => { + fc.assert( + fc.property(surrogateBearingString, raw => { + const parsed = QueryParams.parse(raw); + expect(() => parsed.encode()).not.toThrow(); + }), + {numRuns: 500}, + ); + }); +}); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const params = QueryParams.newBuilder().add('x', '1').add('x', '2').build(); + + test('the present-name list is frozen', () => { + const values = params.getAll('x'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('3')).toThrow(TypeError); + expect(params.getAll('x')).toEqual(['1', '2']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + const first = params.getAll('nope'); + const second = params.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); diff --git a/packages/core/src/http/query-params.ts b/packages/core/src/http/query-params.ts index 0510f3f..0166b9b 100644 --- a/packages/core/src/http/query-params.ts +++ b/packages/core/src/http/query-params.ts @@ -1,9 +1,31 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.ts -import type {Builder} from './builder.js'; -import {encodeRfc3986Component} from './rfc3986.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; +import {UrlConstructionError} from './errors.js'; +import { + encodeRfc3986Component, + hasLoneSurrogate, + toWellFormed, +} from './rfc3986.js'; -function safeDecodeComponent(value: string): string { +/** + * @internal + * RFC 3986 component encoding (HTTP-29): space → `%20` (never `+`), literal `+` → `%2B`, everything outside + * the unreserved set `A–Z a–z 0–9 - . _ ~` percent-encoded. + * + * Exported so `src/pagination/query-splice.ts` can reuse it. `PAGE-22` restates this exact rule, and two + * encoders in one codebase is a drift bug waiting to happen — there is exactly one. + */ +export function encodeQueryComponent(value: string): string { + return encodeRfc3986Component(value); +} + +/** + * @internal + * RFC 3986 component decoding (HTTP-29/HTTP-31): a literal `+` reads back as `+`, `%20` as a space, and + * malformed percent-encoding falls back to raw text rather than throwing. + */ +export function decodeQueryComponent(value: string): string { try { return decodeURIComponent(value); } catch { @@ -11,6 +33,22 @@ function safeDecodeComponent(value: string): string { } } +/** + * The strict half of the surrogate rule, applied by `add()`. + * + * `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + * on an unpaired surrogate — outside the `DexpaceError` tree, and out of `encode()` or `equals()` + * rather than out of the call that supplied the value. Rejecting here puts the failure at the call + * site, which is the same place HTTP-35 and HTTP-17/18 put theirs (audit #67 / #76). + */ +function requireWellFormed(kind: 'name' | 'value', text: string): void { + if (hasLoneSurrogate(text)) { + throw new UrlConstructionError( + `query parameter ${kind} contains an unpaired surrogate and cannot be percent-encoded`, + ); + } +} + let createQueryParams: ( valuesByName: ReadonlyMap<string, readonly string[]>, insertionOrder: readonly string[], @@ -26,7 +64,9 @@ let createQueryParams: ( * Encoding and parsing are deliberately asymmetric and kept as separate operations. * {@link QueryParams.encode} is strict RFC 3986 percent-encoding — not * `application/x-www-form-urlencoded`, so a space is `%20` and never `+` (HTTP-29/32). - * {@link QueryParams.parse} is lenient and never throws (HTTP-31). + * {@link QueryParams.parse} is lenient and never throws (HTTP-31). The asymmetry extends to + * unpaired surrogates: {@link QueryParamsBuilder.add} rejects one, while + * {@link QueryParams.parse} substitutes U+FFFD for it. * * @example * ```ts @@ -82,8 +122,10 @@ export class QueryParams { * (HTTP-31). * * A `null`, `undefined`, or blank input yields empty parameters; a leading `?` is tolerated; a - * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; and - * malformed percent-encoding falls back to the raw text rather than failing. + * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; + * malformed percent-encoding falls back to the raw text rather than failing; and an unpaired + * surrogate is replaced with U+FFFD rather than rejected the way + * {@link QueryParamsBuilder.add} rejects one, so the result is always encodable. * * @param raw - the query string, with or without its leading `?`. * @returns the parsed, frozen parameters. @@ -99,7 +141,13 @@ export class QueryParams { const eqIndex = segment.indexOf('='); const rawName = eqIndex === -1 ? segment : segment.slice(0, eqIndex); const rawValue = eqIndex === -1 ? '' : segment.slice(eqIndex + 1); - builder.add(safeDecodeComponent(rawName), safeDecodeComponent(rawValue)); + // HTTP-31 is MUST-level that parsing never throws, so the strict `add()` path above cannot be + // the one `parse` uses on text it did not choose — exactly the split `Headers` draws between + // its outbound (`add`) and inbound (`addInbound`) methods for HTTP-18 against HTTP-19. + builder.add( + toWellFormed(decodeQueryComponent(rawName)), + toWellFormed(decodeQueryComponent(rawValue)), + ); } return builder.build(); } @@ -118,10 +166,11 @@ export class QueryParams { * Returns every value stored under `name`, in insertion order. * * @param name - the parameter name. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string): readonly string[] { - return this.#valuesByName.get(name) ?? []; + return this.#valuesByName.get(name) ?? EMPTY_VALUE_LIST; } /** @@ -181,9 +230,15 @@ export class QueryParamsBuilder implements Builder<QueryParams> { * @param value - the value; `null` records a value-less parameter as a single empty string * (HTTP-28). * @returns this builder, for chaining. + * @throws {@link UrlConstructionError} when the name or the value carries an unpaired surrogate. + * Such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined for it and + * {@link QueryParams.encode} could only fail — it is rejected here, at the call that supplied it. + * A well-formed surrogate pair is ordinary text and is accepted (HTTP-29). */ add(name: string, value: string | null): this { const actualValue = value ?? ''; + requireWellFormed('name', name); + requireWellFormed('value', actualValue); if (!this.#valuesByName.has(name)) { this.#insertionOrder.push(name); this.#valuesByName.set(name, []); diff --git a/packages/core/src/http/request-conditions.test.ts b/packages/core/src/http/request-conditions.test.ts index 4382f64..b89c3bb 100644 --- a/packages/core/src/http/request-conditions.test.ts +++ b/packages/core/src/http/request-conditions.test.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request-conditions.test.ts -// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag exclusivity) +// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag +// exclusivity, and an invalid Date rejected at the setter rather than emitted as `Invalid Date`) import {describe, expect, test} from 'bun:test'; import { RequestConditions, @@ -134,3 +135,53 @@ describe('ifUnmodifiedSince (HTTP-50)', () => { ]); }); }); + +describe('an invalid Date is rejected at the setter (HTTP-50)', () => { + // `toRfc1123` is `date.toUTCString()`, which renders the literal string `Invalid Date` for a NaN + // time value. `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar + // waves it through and it reaches the wire as `If-Modified-Since: Invalid Date` — a header no + // server can evaluate, produced from a caller mistake made several frames earlier. HTTP-50's + // "emit RFC 1123 dates" is not satisfiable from a NaN instant, so the setter is where it fails. + // Measured on the pre-fix tree, audit #67 / #76. + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifModifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifUnmodifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test('the message names the setter, so the caller knows which field to fix', () => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(new Date('nope')), + ).toThrow(/If-Modified-Since/); + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(new Date('nope')), + ).toThrow(/If-Unmodified-Since/); + }); + + test('no invalid instant can reach applyTo, so no header renders "Invalid Date"', () => { + const builder = RequestConditions.newBuilder(); + expect(() => builder.ifModifiedSince(new Date('nope'))).toThrow( + RequestConditionsValidationError, + ); + const headers = builder.build().applyTo(Headers.newBuilder().build()); + expect(headers.has('If-Modified-Since')).toBe(false); + }); +}); diff --git a/packages/core/src/http/request-conditions.ts b/packages/core/src/http/request-conditions.ts index 2af7ec9..db1d564 100644 --- a/packages/core/src/http/request-conditions.ts +++ b/packages/core/src/http/request-conditions.ts @@ -30,6 +30,25 @@ function toRfc1123(date: Date): string { return date.toUTCString(); } +/** + * Copies `date` after rejecting a NaN time value. + * + * `toUTCString()` is total — it renders the literal string `Invalid Date` rather than throwing — + * and `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar accepts it + * and it reaches the wire as `If-Modified-Since: Invalid Date`. HTTP-50 requires an RFC 1123 date, + * which a NaN instant cannot produce, so the failure belongs at the setter that was handed the bad + * `Date` rather than several frames downstream (audit #67 / #76). + */ +function copyValidInstant(date: Date, headerName: string): Date { + const time = date.getTime(); + if (Number.isNaN(time)) { + throw new RequestConditionsValidationError( + `${headerName}: date must be a valid instant, got an invalid Date`, + ); + } + return new Date(time); +} + // eslint-disable-next-line max-params -- private, builder-internal plumbing; field count fixed by HTTP-50 let createRequestConditions: ( ifMatch: readonly ETag[], @@ -45,7 +64,9 @@ let createRequestConditions: ( * Multiple entity-tags emit as one comma-separated header; dates emit in RFC 1123 form. * {@link RequestConditions.applyTo} uses `set`, never `add`, so applying the same conditions twice * cannot duplicate a header. The any-tag (`*`) is mutually exclusive with concrete entity-tags, and - * repeated `*` collapses to one — enforced when the tag is added, not at emission. + * repeated `*` collapses to one — enforced when the tag is added, not at emission. An invalid + * `Date` is likewise rejected by the setter that was handed it, so no instance can emit the literal + * header value `Invalid Date`. * * @example * ```ts @@ -118,8 +139,12 @@ export class RequestConditions { * untouched. * * Emission goes through the strict outbound header path, which rejects obs-text. An ETag whose - * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here; reconciling that - * against HTTP-18 is left to a later phase rather than guessed at now. + * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here, so a server-issued + * one does not round-trip. That is decided, not open: HTTP-18 is MUST-level and reinforced by + * XCUT-18's splitting defense, HTTP-48's permission is SHOULD-level, and no relaxed emit path is + * added — see item 15 of + * `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` and its + * as-built audit in `docs/deviations.md`. * * @param headers - the headers to derive from; not modified. * @returns a new {@link Headers} carrying the preconditions. @@ -200,9 +225,12 @@ export class RequestConditionsBuilder implements Builder<RequestConditions> { * @param date - the instant; copied, not aliased, so a caller mutating its own `Date` after * `build()` cannot change what {@link RequestConditions.applyTo} emits. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value. An + * invalid `Date` renders as the literal `Invalid Date`, which the outbound header grammar accepts + * and no server can evaluate, so it is rejected here rather than emitted (HTTP-50). */ ifModifiedSince(date: Date): this { - this.#ifModifiedSince = new Date(date.getTime()); + this.#ifModifiedSince = copyValidInstant(date, 'If-Modified-Since'); return this; } @@ -212,9 +240,11 @@ export class RequestConditionsBuilder implements Builder<RequestConditions> { * @param date - the instant; copied, not aliased, exactly as in * {@link RequestConditionsBuilder.ifModifiedSince}. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value, for the + * reason given on {@link RequestConditionsBuilder.ifModifiedSince} (HTTP-50). */ ifUnmodifiedSince(date: Date): this { - this.#ifUnmodifiedSince = new Date(date.getTime()); + this.#ifUnmodifiedSince = copyValidInstant(date, 'If-Unmodified-Since'); return this; } diff --git a/packages/core/src/http/request-options.test.ts b/packages/core/src/http/request-options.test.ts index aef49e0..bcfc28f 100644 --- a/packages/core/src/http/request-options.test.ts +++ b/packages/core/src/http/request-options.test.ts @@ -1,7 +1,11 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request-options.test.ts -// Exercises: HTTP-34 (EMPTY sentinel, defensive tag copy), HTTP-35 (timeout/maxRetries validation) +// Exercises: HTTP-34 (EMPTY sentinel, defensive tag copy), HTTP-35 (timeout/maxRetries validation), +// AUTH-4 (the per-call auth descriptor tier, added in Phase 5c) import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {createAuthDescriptor} from '../auth/descriptor.js'; +import {createAuthRequirement} from '../auth/requirement.js'; import {RequestOptions} from './request-options.js'; import {RequestOptionsValidationError} from './errors.js'; @@ -13,6 +17,89 @@ describe('RequestOptions.EMPTY', () => { }); }); +describe('per-call auth descriptor (AUTH-4)', () => { + test('EMPTY carries no auth descriptor', () => { + expect(RequestOptions.EMPTY.auth).toBeUndefined(); + }); + + test('the builder stores and the accessor returns the same descriptor instance', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('NO_AUTH')]); + expect(RequestOptions.newBuilder().auth(descriptor).build().auth).toBe( + descriptor, + ); + }); + + test('an explicit undefined clears the override', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('NO_AUTH')]); + const builder = RequestOptions.newBuilder().auth(descriptor); + expect(builder.auth(undefined).build().auth).toBeUndefined(); + }); + + test('a derived builder carries the descriptor forward (HTTP-3)', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('BASIC')]); + const original = RequestOptions.newBuilder().auth(descriptor).build(); + expect(original.newBuilder().build().auth).toBe(descriptor); + }); +}); + +describe('operation auth descriptor (AUTH-4, docs/work/mvp/2026-09-04-open-items-dissolution.md W1)', () => { + test('EMPTY carries no operation descriptor', () => { + expect(RequestOptions.EMPTY.operationAuth).toBeUndefined(); + }); + + test('the builder round-trips the descriptor by reference', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('BASIC')]); + expect( + RequestOptions.newBuilder().operationAuth(descriptor).build() + .operationAuth, + ).toBe(descriptor); + }); + + test('a derived builder carries the descriptor forward (HTTP-3)', () => { + const descriptor = createAuthDescriptor([createAuthRequirement('BASIC')]); + const original = RequestOptions.newBuilder() + .operationAuth(descriptor) + .build(); + expect(original.newBuilder().build().operationAuth).toBe(descriptor); + }); + + test('the two slots are independent — filling one leaves the other unset', () => { + const perCall = createAuthDescriptor([createAuthRequirement('BASIC')]); + const operation = createAuthDescriptor([createAuthRequirement('API_KEY')]); + const options = RequestOptions.newBuilder() + .auth(perCall) + .operationAuth(operation) + .build(); + expect(options.auth).toBe(perCall); + expect(options.operationAuth).toBe(operation); + }); +}); + +const timeoutCandidate = fc.oneof( + fc.double({noNaN: false}), + fc.integer({min: -10, max: 10}), + fc.constantFrom(2 ** 32 - 1, 2 ** 32, Number.MAX_SAFE_INTEGER), +); + +/** + * Either the builder refuses `candidate` with the typed error, or it admits a value inside + * `AbortSignal.timeout()`'s range. There is no third outcome — that is the whole HTTP-35 claim. + */ +function expectAdmittedTimeoutInRange(candidate: number): void { + let accepted: number | undefined; + try { + accepted = RequestOptions.newBuilder() + .timeoutMs(candidate) + .build().timeoutMs; + } catch (e: unknown) { + expect(e).toBeInstanceOf(RequestOptionsValidationError); + return; + } + expect(Number.isInteger(accepted)).toBe(true); + expect(accepted).toBeGreaterThanOrEqual(1); + expect(accepted).toBeLessThanOrEqual(2 ** 32 - 1); +} + describe('timeout validation (HTTP-35)', () => { test('rejects zero or negative timeout', () => { expect(() => RequestOptions.newBuilder().timeoutMs(0)).toThrow( @@ -23,6 +110,59 @@ describe('timeout validation (HTTP-35)', () => { ); }); + test('rejects a non-finite timeout, the same way maxRetries does (P2)', () => { + expect(() => + RequestOptions.newBuilder().timeoutMs(Number.POSITIVE_INFINITY), + ).toThrow(RequestOptionsValidationError); + expect(() => RequestOptions.newBuilder().timeoutMs(Number.NaN)).toThrow( + RequestOptionsValidationError, + ); + }); + + // Flipped by audit #67 / #76. The old case pinned `timeoutMs(1.5)` as accepted "because a deadline + // can honor a fractional millisecond". Nothing downstream can: the only consumer is + // `composeSignal`, which hands the value to `AbortSignal.timeout()`, and that throws + // `RangeError: The value of "delay" is out of range. It must be an integer.` — inside the + // transport, one seam away from the setter that accepted it. HTTP-35 puts the range check at the + // setter, so the range checked is `AbortSignal.timeout()`'s, the only one a transport can honor. + test('rejects a fractional timeout, which no transport deadline can honor', () => { + expect(() => RequestOptions.newBuilder().timeoutMs(1.5)).toThrow( + RequestOptionsValidationError, + ); + }); + + test("rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1", () => { + expect(() => RequestOptions.newBuilder().timeoutMs(2 ** 32)).toThrow( + RequestOptionsValidationError, + ); + expect(() => + RequestOptions.newBuilder().timeoutMs(Number.MAX_SAFE_INTEGER), + ).toThrow(RequestOptionsValidationError); + }); + + test('accepts the ceiling itself, so the boundary is inclusive', () => { + expect( + RequestOptions.newBuilder() + .timeoutMs(2 ** 32 - 1) + .build().timeoutMs, + ).toBe(2 ** 32 - 1); + }); + + // The invariant, stated runtime-independently: a value this setter accepts is inside + // `AbortSignal.timeout()`'s documented range, and anything else fails here. It is asserted as a + // property rather than against `AbortSignal.timeout()` itself because the two runtimes disagree — + // Bun accepts `1.5` and `2**32` where Node raises `RangeError` — which is precisely why the range + // is checked in the model instead of left to whichever runtime the caller happens to be on. + // `tests/node-conformance/seams.test.mjs` closes the Node half. + test('every accepted timeout is an integer in 1..2**32 - 1 (property)', () => { + fc.assert( + fc.property(timeoutCandidate, candidate => { + expectAdmittedTimeoutInRange(candidate); + }), + {numRuns: 500}, + ); + }); + test('accepts a null (undefined) timeout — no override', () => { expect(() => RequestOptions.newBuilder().timeoutMs(undefined).build(), @@ -43,11 +183,33 @@ describe('maxRetries validation (HTTP-35)', () => { ); }); + test('rejects a non-finite maxRetries, which would make a retry loop unbounded', () => { + // Worse in effect than a negative value: a negative one still fails a downstream `>= 1` guard, + // while Infinity/NaN make an "attempt >= ceiling" test permanently false and the loop endless. + for (const value of [Number.POSITIVE_INFINITY, Number.NaN]) { + expect(() => RequestOptions.newBuilder().maxRetries(value)).toThrow( + RequestOptionsValidationError, + ); + } + }); + + test('rejects a fractional maxRetries, which is not a count of wire sends', () => { + expect(() => RequestOptions.newBuilder().maxRetries(1.5)).toThrow( + RequestOptionsValidationError, + ); + }); + test('accepts 0, meaning "disable retries for this call"', () => { expect(RequestOptions.newBuilder().maxRetries(0).build().maxRetries).toBe( 0, ); }); + + test('accepts a positive integer', () => { + expect(RequestOptions.newBuilder().maxRetries(3).build().maxRetries).toBe( + 3, + ); + }); }); describe('tags are defensively copied at build (HTTP-34)', () => { diff --git a/packages/core/src/http/request-options.ts b/packages/core/src/http/request-options.ts index a961ede..b0258dd 100644 --- a/packages/core/src/http/request-options.ts +++ b/packages/core/src/http/request-options.ts @@ -1,20 +1,33 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request-options.ts +import type {AuthDescriptor} from '../auth/descriptor.js'; import type {Builder} from './builder.js'; import {RequestOptionsValidationError} from './errors.js'; +/** + * The largest timeout `AbortSignal.timeout()` accepts, and therefore the largest one this model + * will hold: the platform rejects anything above it with `RangeError: The value of "delay" is out + * of range. It must be >= 0 && <= 4294967295.` (HTTP-35, audit #67 / #76). + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + +// eslint-disable-next-line max-params -- private, builder-internal plumbing; one parameter per HTTP-34 field let createRequestOptions: ( timeoutMs: number | undefined, maxRetries: number | undefined, tags: ReadonlyMap<string, string>, + auth: AuthDescriptor | undefined, + operationAuth: AuthDescriptor | undefined, ) => RequestOptions; /** * Immutable per-call operational overrides that are deliberately *not* part of the wire form: a - * timeout, a max-retries count, and opaque string-keyed tags (HTTP-34). + * timeout, a max-retries count, opaque string-keyed tags (HTTP-34), and a per-call auth descriptor + * (AUTH-4). * - * Every field defaults to a "use the configured default" sentinel of `undefined`, and - * {@link RequestOptions.EMPTY} is the canonical override-nothing instance. + * Every scalar field defaults to a "use the configured default" sentinel of `undefined`; tags default + * to an empty map, which means the same thing. {@link RequestOptions.EMPTY} is the canonical + * override-nothing instance. * * `undefined` and `0` are different states for max-retries: `undefined` means "use the default", * while `0` means "disable retries for this call" (HTTP-35). @@ -25,28 +38,41 @@ export class RequestOptions { readonly #timeoutMs: number | undefined; readonly #maxRetries: number | undefined; readonly #tags: ReadonlyMap<string, string>; + readonly #auth: AuthDescriptor | undefined; + readonly #operationAuth: AuthDescriptor | undefined; + // eslint-disable-next-line max-params -- private, builder-internal; one parameter per HTTP-34 field private constructor( timeoutMs: number | undefined, maxRetries: number | undefined, tags: ReadonlyMap<string, string>, + auth: AuthDescriptor | undefined, + operationAuth: AuthDescriptor | undefined, ) { this.#timeoutMs = timeoutMs; this.#maxRetries = maxRetries; this.#tags = tags; + this.#auth = auth; + this.#operationAuth = operationAuth; Object.freeze(this); } static { - createRequestOptions = (timeoutMs, maxRetries, tags) => - new RequestOptions(timeoutMs, maxRetries, tags); + // eslint-disable-next-line max-params -- private, builder-internal plumbing; one parameter per HTTP-34 field + createRequestOptions = (timeoutMs, maxRetries, tags, auth, operationAuth) => + new RequestOptions(timeoutMs, maxRetries, tags, auth, operationAuth); } - /** The canonical "override nothing" instance: no timeout, no retry override, no tags. */ + /** + * The canonical "override nothing" instance: no timeout, no retry override, no tags, and neither + * auth descriptor. + */ static readonly EMPTY = new RequestOptions( undefined, undefined, Object.freeze(new Map()), + undefined, + undefined, ); /** @@ -68,10 +94,16 @@ export class RequestOptions { return new RequestOptionsBuilder() .timeoutMs(this.#timeoutMs) .maxRetries(this.#maxRetries) - .tags(this.#tags); + .tags(this.#tags) + .auth(this.#auth) + .operationAuth(this.#operationAuth); } - /** The per-call timeout in milliseconds, or `undefined` to use the configured default. */ + /** + * The per-call timeout in milliseconds, or `undefined` to use the configured default. Always an + * integer in `1 .. 2**32 - 1` when defined — {@link RequestOptionsBuilder.timeoutMs} admits + * nothing else, so a transport can pass it to `AbortSignal.timeout()` unchecked. + */ get timeoutMs(): number | undefined { return this.#timeoutMs; } @@ -93,6 +125,41 @@ export class RequestOptions { tag(key: string): string | undefined { return this.#tags.get(key); } + + /** + * The per-call auth descriptor, or `undefined` to use the configured tiers. + * + * Fills AUTH-4's most-specific `perCall` tier: when present it wins over any `perCall`, `operation`, + * or `client` descriptor the AUTH pillar step was constructed with, and a tier below it is never + * consulted even if this one turns out to be unsatisfiable. + * + * Returned by reference: an {@link AuthDescriptor} is frozen at construction, so there is nothing to + * copy defensively. + */ + get auth(): AuthDescriptor | undefined { + return this.#auth; + } + + /** + * The operation's declared auth descriptor, or `undefined` when the operation declares none. + * + * Fills AUTH-4's middle `operation` tier. Selection is `perCall ?? operation ?? client`, so + * {@link RequestOptions.auth} still wins over this, and this still wins over whatever descriptor + * the AUTH pillar step was constructed with. A tier below the selected one is never consulted even + * if the selected one turns out to be unsatisfiable. + * + * This slot exists for a generated client, not for a hand-written call: it is where an operation + * table's static `auth` declaration goes, so the caller's genuine per-call override stays + * distinguishable from it. Without it a generator has to fold the two together itself — which + * reimplements this very precedence rule outside core and leaves core unable to tell which tier + * won. `examples/petstore/FINDINGS.md` §4 measures that cost; `docs/work/mvp/2026-09-04-open-items-dissolution.md` W1 records it. + * + * Returned by reference: an {@link AuthDescriptor} is frozen at construction, so there is nothing + * to copy defensively. + */ + get operationAuth(): AuthDescriptor | undefined { + return this.#operationAuth; + } } /** @@ -107,20 +174,40 @@ export class RequestOptionsBuilder implements Builder<RequestOptions> { #timeoutMs: number | undefined; #maxRetries: number | undefined; readonly #tags = new Map<string, string>(); + #auth: AuthDescriptor | undefined; + #operationAuth: AuthDescriptor | undefined; /** * Sets the per-call timeout. * + * The range check is the FULL range, not merely its lower bound. `Infinity` and `NaN` are as out + * of range as `-1`: a non-finite deadline is one no clock can compare against, so it degrades to + * "no deadline" silently rather than failing at the call site that supplied it, which is exactly + * what HTTP-35 exists to prevent. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range a transport can honor. `composeSignal` is the one consumer of this value and it + * hands it straight to `AbortSignal.timeout()`, which throws a `RangeError` on a fractional + * millisecond and on anything above `4294967295`. A timeout this setter accepted and a transport + * then refused is the failure HTTP-35 exists to move to the call site, so integrality is checked + * here and not treated as an implementation detail of one transport (audit #67 / #76 flipped the + * earlier reading, which accepted `1.5` on the argument that a duration may be fractional; no + * consumer of this field can express one). + * * @param value - the timeout in milliseconds, or `undefined` for no override. Zero is rejected * rather than reinterpreted: it means "no timeout" in one transport and is an error in another * (HTTP-35). * @returns this builder, for chaining. - * @throws {@link RequestOptionsValidationError} when a defined value is zero or negative. + * @throws {@link RequestOptionsValidationError} when a defined value is zero, negative, not + * finite, not an integer, or greater than `2**32 - 1`. */ timeoutMs(value: number | undefined): this { - if (value !== undefined && value <= 0) { + if ( + value !== undefined && + !(Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { throw new RequestOptionsValidationError( - `timeout must be positive, got ${String(value)}`, + `timeout must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, got ${String(value)}`, ); } this.#timeoutMs = value; @@ -130,16 +217,24 @@ export class RequestOptionsBuilder implements Builder<RequestOptions> { /** * Sets the per-call retry ceiling. * + * The range check is deliberately wider than "not negative". A retry ceiling is a count of wire + * sends, so `Infinity` and `NaN` are as out-of-range as `-1` -- and they are worse in effect: a + * negative value at least fails a downstream lower-bound guard, while a non-finite one makes a + * retry driver's "have I reached the ceiling" test permanently false and its loop unbounded. + * HTTP-35's point is that an out-of-range retry count is a loud error at the call site that + * supplied it, never a value reinterpreted somewhere downstream. + * * @param value - the maximum retries, or `undefined` for no override. `0` is accepted and means - * "disable retries for this call"; a negative count is rejected rather than silently - * reinterpreted (HTTP-35). + * "disable retries for this call"; anything that is not a non-negative integer is rejected rather + * than silently reinterpreted (HTTP-35). * @returns this builder, for chaining. - * @throws {@link RequestOptionsValidationError} when a defined value is negative. + * @throws {@link RequestOptionsValidationError} when a defined value is negative, fractional, or + * not finite. */ maxRetries(value: number | undefined): this { - if (value !== undefined && value < 0) { + if (value !== undefined && !(Number.isInteger(value) && value >= 0)) { throw new RequestOptionsValidationError( - `maxRetries must not be negative, got ${String(value)}`, + `maxRetries must be a non-negative integer, got ${String(value)}`, ); } this.#maxRetries = value; @@ -157,6 +252,36 @@ export class RequestOptionsBuilder implements Builder<RequestOptions> { return this; } + /** + * Sets the per-call auth descriptor, filling AUTH-4's `perCall` tier for this call only. + * + * No validation beyond the type: {@link createAuthDescriptor} already rejects an empty requirement + * list and freezes the result (AUTH-3), so any constructed descriptor is valid by construction. + * + * @param descriptor - the descriptor, or `undefined` for no override. + * @returns this builder, for chaining. + */ + auth(descriptor: AuthDescriptor | undefined): this { + this.#auth = descriptor; + return this; + } + + /** + * Sets the operation's declared auth descriptor, filling AUTH-4's `operation` tier for this call. + * + * Independent of {@link RequestOptionsBuilder.auth}: filling both is the normal case for a + * generated client whose caller also passed an override, and `perCall ?? operation ?? client` + * resolves them. No validation beyond the type, for the same reason + * {@link RequestOptionsBuilder.auth} needs none. + * + * @param descriptor - the operation's descriptor, or `undefined` when it declares none. + * @returns this builder, for chaining. + */ + operationAuth(descriptor: AuthDescriptor | undefined): this { + this.#operationAuth = descriptor; + return this; + } + /** * Copies and freezes the accumulated state into an immutable {@link RequestOptions}. * @@ -170,6 +295,8 @@ export class RequestOptionsBuilder implements Builder<RequestOptions> { this.#timeoutMs, this.#maxRetries, Object.freeze(new Map(this.#tags)), + this.#auth, + this.#operationAuth, ); } } diff --git a/packages/core/src/http/request.test.ts b/packages/core/src/http/request.test.ts index 568ff7c..216b150 100644 --- a/packages/core/src/http/request.test.ts +++ b/packages/core/src/http/request.test.ts @@ -1,10 +1,15 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request.test.ts -// Exercises: HTTP-6 (required fields), HTTP-7 (body/method legality), HTTP-8 (GET default / missing method), +// Exercises: XCUT-15's alias and new-instance clauses (a model retains no alias to externally-mutable +// state -- the returned URL is cloned per access, so mutating it cannot reach the request -- and every +// "setter" yields a NEW instance rather than mutating in place: the HTTP-3/5 rows below. The +// ingested-collection clause is asserted in headers.test.ts and query-params.test.ts), +// HTTP-6 (required fields), HTTP-7 (body/method legality), HTTP-8 (GET default / missing method), // HTTP-9 (method), HTTP-46 (textual URL equality, no DNS), HTTP-47 (malformed URL), HTTP-3/5 (derivation, // immutability) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; +import {stringBody} from '../body/simple-bodies.js'; import {Request} from './request.js'; import {Headers} from './headers.js'; import { @@ -31,7 +36,7 @@ describe('method/body legality (HTTP-7)', () => { Request.newBuilder() .method(method) .url('https://example.com') - .body('x') + .body(stringBody('x')) .build(), ).toThrow(RequestBodyNotAllowedError); } @@ -42,7 +47,7 @@ describe('method/body legality (HTTP-7)', () => { Request.newBuilder() .method('POST') .url('https://example.com') - .body('x') + .body(stringBody('x')) .build(), ).not.toThrow(); }); @@ -51,21 +56,11 @@ describe('method/body legality (HTTP-7)', () => { const request = Request.newBuilder() .method('GET') .url('https://example.com') - .body('x') + .body(stringBody('x')) .body(undefined) .build(); expect(request.body).toBeUndefined(); }); - - test('a null body clears like undefined — HTTP-7 rejects only a non-null body', () => { - const request = Request.newBuilder() - .method('GET') - .url('https://example.com') - .body('x') - .body(null) - .build(); - expect(request.body).toBeUndefined(); - }); }); describe('method defaulting (HTTP-8)', () => { @@ -76,7 +71,10 @@ describe('method defaulting (HTTP-8)', () => { test('fails naming the missing method when a body is set with no method', () => { expect(() => - Request.newBuilder().url('https://example.com').body('x').build(), + Request.newBuilder() + .url('https://example.com') + .body(stringBody('x')) + .build(), ).toThrow('method is required'); }); }); diff --git a/packages/core/src/http/request.ts b/packages/core/src/http/request.ts index 2610f29..9105de4 100644 --- a/packages/core/src/http/request.ts +++ b/packages/core/src/http/request.ts @@ -1,5 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request.ts +import type {Body} from '../body/body.js'; import type {Builder} from './builder.js'; import {requireField} from './builder.js'; import {UrlConstructionError, RequestBodyNotAllowedError} from './errors.js'; @@ -21,7 +22,7 @@ let createRequest: ( method: Method, url: URL, headers: Headers, - body: unknown, + body: Body | undefined, ) => Request; /** @@ -39,7 +40,7 @@ let createRequest: ( * const request = Request.newBuilder() * .method('POST') * .url('https://example.com/items') - * .body('payload') + * .body(stringBody('payload')) * .build(); * ``` * @@ -49,14 +50,14 @@ export class Request { readonly #method: Method; readonly #url: URL; readonly #headers: Headers; - readonly #body: unknown; + readonly #body: Body | undefined; // eslint-disable-next-line max-params -- private, builder-internal; field count fixed by the wire model (HTTP-6) private constructor( method: Method, url: URL, headers: Headers, - body: unknown, + body: Body | undefined, ) { this.#method = method; this.#url = url; @@ -113,13 +114,8 @@ export class Request { return this.#headers; } - /** - * The request body, or `undefined` when absent. - * - * Typed `unknown` on purpose: this phase only needs presence or absence to enforce HTTP-7/8. The - * body lifecycle — streaming, replayability, charset — is owned by a later phase. - */ - get body(): unknown { + /** The request body, or `undefined` when absent. */ + get body(): Body | undefined { return this.#body; } @@ -128,8 +124,7 @@ export class Request { * * The URL is compared by textual external form only, never by resolving the host — native URL * equality on some platforms resolves DNS, which blocks and is wrong for virtual hosts sharing an - * IP (HTTP-46). The body is compared by reference for now; value equality arrives with the real - * body model in a later phase. + * IP (HTTP-46). * * @param other - the request to compare against. * @returns `true` when every compared facet is equal. @@ -153,7 +148,7 @@ export class RequestBuilder implements Builder<Request> { #method: Method | undefined; #url: URL | undefined; #headers: Headers = Headers.newBuilder().build(); - #body: unknown; + #body: Body | undefined; /** * Sets the request method. @@ -194,12 +189,11 @@ export class RequestBuilder implements Builder<Request> { /** * Sets or clears the request body. * - * @param body - the body, or `null`/`undefined` to clear it. `null` normalizes to `undefined`: - * HTTP-7 rejects only a *non-null* body, so passing `null` clears exactly like `undefined`. + * @param body - the body, or `undefined` to clear it. * @returns this builder, for chaining. */ - body(body: unknown): this { - this.#body = body ?? undefined; + body(body: Body | undefined): this { + this.#body = body; return this; } diff --git a/packages/core/src/http/response.test.ts b/packages/core/src/http/response.test.ts index d3d8b46..3e06b76 100644 --- a/packages/core/src/http/response.test.ts +++ b/packages/core/src/http/response.test.ts @@ -1,17 +1,52 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/response.test.ts -// Exercises: HTTP-6 (response's required fields: request, protocol, status) +// Exercises: HTTP-6 (required fields), HTTP-41/BODY-14 (single-use body, same reference on repeat +// access), HTTP-41/BODY-15, HTTP-43 (idempotent close, releases the connection whether or not the body +// was read), HTTP-41/BODY-16 (convenience readers close in a finally-style guarantee), HTTP-42 +// (charset default and UTF-8 fallback) import {describe, expect, test} from 'bun:test'; -import {Response} from './response.js'; -import {Request} from './request.js'; +import {Headers} from './headers.js'; import {Protocol} from './protocol.js'; +import {Request} from './request.js'; +import {Response} from './response.js'; import {Status} from './status.js'; -import {Headers} from './headers.js'; + +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves. */ +async function rejection(promise: Promise<unknown>): Promise<Error> { + try { + await promise; + } catch (error: unknown) { + return error as Error; + } + throw new Error('expected the promise to reject, but it resolved'); +} function baseRequest(): Request { return Request.newBuilder().url('https://example.com').build(); } +function readableOf(text: string): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); +} + +function baseResponse( + body: ReadableStream<Uint8Array> | null = null, + headers: Headers = Headers.newBuilder().build(), +): Response { + return Response.newBuilder() + .request(baseRequest()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers(headers) + .body(body) + .build(); +} + describe('required fields', () => { test('throws naming request when missing', () => { expect(() => @@ -42,60 +77,227 @@ describe('required fields', () => { }); describe('construction', () => { - test('carries the originating request, protocol, status, headers, and an optional reason phrase/body', () => { + test('carries the originating request, protocol, status, headers, and an optional reason phrase', () => { const request = baseRequest(); const response = Response.newBuilder() .request(request) .protocol(Protocol.HTTP_1_1) .status(Status.of(200)) .reasonPhrase('OK') - .body('payload') .build(); expect(response.request.equals(request)).toBe(true); expect(response.protocol.equals(Protocol.HTTP_1_1)).toBe(true); expect(response.status.equals(Status.of(200))).toBe(true); expect(response.reasonPhrase).toBe('OK'); - expect(response.body).toBe('payload'); }); - test('reason phrase and body are optional', () => { + test('reason phrase is optional, body defaults to null', () => { const response = Response.newBuilder() .request(baseRequest()) .protocol(Protocol.HTTP_1_1) .status(Status.of(204)) .build(); expect(response.reasonPhrase).toBeUndefined(); - expect(response.body).toBeUndefined(); + expect(response.body).toBeNull(); }); }); describe('newBuilder derivation', () => { test('deriving a builder and rebuilding does not affect the original', () => { - const original = Response.newBuilder() - .request(baseRequest()) - .protocol(Protocol.HTTP_1_1) - .status(Status.of(200)) - .build(); + const original = baseResponse(); original.newBuilder().status(Status.of(500)).build(); expect(original.status.code).toBe(200); }); }); -describe('headers (HTTP-6)', () => { - test('defaults to empty headers and carries what the builder was given', () => { - const bare = Response.newBuilder() - .request(baseRequest()) - .protocol(Protocol.HTTP_1_1) - .status(Status.of(204)) +describe('body (HTTP-41/BODY-14)', () => { + test('repeated access returns the same reference, not a replay', () => { + const stream = readableOf('x'); + const response = baseResponse(stream); + expect(response.body).toBe(stream); + expect(response.body).toBe(response.body); + }); +}); + +describe('bytes/text (BODY-16, HTTP-42)', () => { + test('bytes() reads the whole body', async () => { + const response = baseResponse(readableOf('hello')); + expect(new TextDecoder().decode(await response.bytes())).toBe('hello'); + }); + + test('bytes() on a null body returns empty', async () => { + expect(await baseResponse(null).bytes()).toEqual(new Uint8Array(0)); + }); + + test('text() defaults to UTF-8 when no content-type is declared', async () => { + expect(await baseResponse(readableOf('héllo')).text()).toBe('héllo'); + }); + + test('text() uses the declared charset', async () => { + const bytes = Uint8Array.from([0x68, 0xe9]); // "hé" in ISO-8859-1 + const stream = new ReadableStream<Uint8Array>({ + start: c => { + c.enqueue(bytes); + c.close(); + }, + }); + const headers = Headers.newBuilder() + .add('content-type', 'text/plain;charset=iso-8859-1') .build(); - expect(bare.headers.names()).toEqual([]); + expect(await baseResponse(stream, headers).text()).toBe('hé'); + }); - const response = bare - .newBuilder() - .headers(Headers.newBuilder().add('Content-Type', 'text/plain').build()) + test('text() falls back to UTF-8 when the content-type itself is unparseable', async () => { + // Distinct from an unrecognized *charset* below: here MediaType.parse throws before any charset + // is read. HTTP-42's fallback has to cover absent, unparseable, and unrecognized alike. + const headers = Headers.newBuilder() + .add('content-type', 'not a media type at all') .build(); - expect(response.headers.get('content-type')).toBe('text/plain'); - expect(bare.headers.has('content-type')).toBe(false); + expect(await baseResponse(readableOf('ok'), headers).text()).toBe('ok'); + }); + + test('text() falls back to UTF-8 when the declared charset is unrecognized', async () => { + const headers = Headers.newBuilder() + .add('content-type', 'text/plain;charset=bogus-charset') + .build(); + expect(await baseResponse(readableOf('ok'), headers).text()).toBe('ok'); + }); + + test('bytes() closes the response even though the read succeeded', async () => { + const response = baseResponse(readableOf('x')); + await response.bytes(); + expect(response.close()).resolves.toBeUndefined(); // idempotent, already closed + }); +}); + +describe('close (HTTP-41/BODY-15, HTTP-43)', () => { + test('cancels the body at most once however often close is called (BODY-15, HTTP-43)', async () => { + let cancels = 0; + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + }); + // Counting calls to cancel(), not the source's cancel callback: the Streams spec makes a second + // cancel() on an already-cancelled stream a resolved no-op that never reaches the source, so only + // the call count can show the guard working -- and the throw stands in for a transport whose + // cancel is not re-entrant, which is why HTTP-43 asks for at-most-once in the first place. + const delegate = stream.cancel.bind(stream); + stream.cancel = async (reason?: unknown): Promise<void> => { + cancels += 1; + if (cancels > 1) + throw new Error('transport does not tolerate a double close'); + return delegate(reason); + }; + const response = baseResponse(stream); + await response.close(); + await response.close(); + await response.close(); + // Counted, not merely "did not throw": cancel() on an already-cancelled ReadableStream resolves + // quietly, so idempotence observed only as the absence of a throw tests nothing. The guard exists + // for transports whose cancel is not re-entrant. + expect(cancels).toBe(1); + }); + + test('a failed release is not remembered as a successful close', () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw new Error('CONNECTION STUCK'); + }, + }); + const response = baseResponse(stream); + // Every caller sees the failure -- marking the response closed before awaiting would report a + // connection that was never released as released. + expect(response.close()).rejects.toThrow('CONNECTION STUCK'); + expect(response.close()).rejects.toThrow('CONNECTION STUCK'); + }); + + test('releases the connection even when the body was never read', async () => { + let cancelled = false; + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + cancelled = true; + }, + }); + await baseResponse(stream).close(); + expect(cancelled).toBe(true); + }); + + test('teardown is close() only -- no [Symbol.asyncDispose] on the >=20.3 floor', () => { + // The symbol postdates engines.node ">=20.3" (it arrived in Node 20.4), where the computed key evaluates to `undefined` + // and binds the method to the string "undefined" instead. Asserting its ABSENCE is what keeps it + // from being reintroduced ahead of the floor bump that would make it real on every resource owner. + const response = baseResponse(readableOf('x')); + expect( + Object.getOwnPropertyNames(Object.getPrototypeOf(response)), + ).not.toContain('undefined'); + expect(typeof response.close).toBe('function'); + }); +}); + +describe('the close guarantee survives a locked body (BODY-16)', () => { + // `getReader()` itself throws when an external consumer already holds the lock, and BODY-15 + // forbids assuming the body was never touched. Acquiring the reader above the try meant the one + // failure BODY-16's guarantee most needs to cover was the one that skipped close entirely. + function lockedResponse(): {response: Response; released: () => boolean} { + let released = false; + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + released = true; + }, + }); + const response = baseResponse(stream); + stream.getReader(); // an external consumer takes the lock + return {response, released: () => released}; + } + + test('bytes() still closes the response when the body is already locked', async () => { + const {response, released} = lockedResponse(); + expect((await rejection(response.bytes())).name).toBe('TypeError'); + // The connection is released as far as this response can release it; the external lock holder's + // own close finishes the job, exactly as close() already documents. + expect(released()).toBe(false); + // Idempotent and already-closed: a second close is a no-op rather than a second cancel attempt. + await response.close(); + }); + + test('text() inherits the same guarantee', async () => { + const {response} = lockedResponse(); + expect((await rejection(response.text())).name).toBe('TypeError'); + await response.close(); + }); +}); + +describe('construction is builder-only (HTTP-2)', () => { + test('the constructor is unreachable from outside the module', () => { + // `Response` is exported as a VALUE, so a public field-wise constructor would let a caller skip + // build()'s required-field validation and would appear in the emitted .d.ts. The private + // constructor plus the createResponse friend hook is what prevents both. + // + // The assertion is the @ts-expect-error itself: every argument below is well-typed and the arity + // is right, so privacy is the ONLY reason this line errors. If the private constructor is ever + // lost, the suppression becomes unused and `tsc` fails the build. + const args = [ + baseRequest(), + Protocol.HTTP_1_1, + Status.of(200), + undefined, + Headers.newBuilder().build(), + null, + ] as const; + const construct = (): unknown => + // @ts-expect-error -- HTTP-2: constructible only through ResponseBuilder, never directly. + new Response(...args); + expect(construct()).toBeInstanceOf(Response); }); }); diff --git a/packages/core/src/http/response.ts b/packages/core/src/http/response.ts index 8c9c68e..1f3b95e 100644 --- a/packages/core/src/http/response.ts +++ b/packages/core/src/http/response.ts @@ -2,10 +2,11 @@ // packages/core/src/http/response.ts import type {Builder} from './builder.js'; import {requireField} from './builder.js'; -import type {Request} from './request.js'; +import {decodeBodyText, resolveCharset} from './charset.js'; +import {Headers} from './headers.js'; import type {Protocol} from './protocol.js'; +import type {Request} from './request.js'; import type {Status} from './status.js'; -import {Headers} from './headers.js'; // eslint-disable-next-line max-params -- private, builder-internal plumbing; field count fixed by HTTP-6 let createResponse: ( @@ -14,16 +15,19 @@ let createResponse: ( status: Status, reasonPhrase: string | undefined, headers: Headers, - body: unknown, + body: ReadableStream<Uint8Array> | null, ) => Response; /** * An immutable HTTP response: the originating request, the negotiated protocol, the status, an - * optional reason phrase, headers, and an optional body (HTTP-6). + * optional reason phrase, headers, and a single-use body stream (HTTP-6). * * Status-range classification is reached through {@link Response.status} — `response.status.isSuccess`, * `response.status.isError`, and the rest (HTTP-11). * + * Owns the body's connection, released by {@link Response.close}. Teardown is `close()` only. + * Revisit when a project-wide explicit resource management pass lands across all Phase 2/3a resource classes. + * * @public */ export class Response { @@ -32,7 +36,10 @@ export class Response { readonly #status: Status; readonly #reasonPhrase: string | undefined; readonly #headers: Headers; - readonly #body: unknown; + readonly #body: ReadableStream<Uint8Array> | null; + // Not `readonly` -- Object.freeze(this) below only freezes normal properties, never #private fields, + // so this can still track close state after construction (BODY-15, HTTP-43). + #closing: Promise<void> | undefined; // eslint-disable-next-line max-params -- private, builder-internal; field count fixed by the wire model (HTTP-6) private constructor( @@ -41,7 +48,7 @@ export class Response { status: Status, reasonPhrase: string | undefined, headers: Headers, - body: unknown, + body: ReadableStream<Uint8Array> | null, ) { this.#request = request; this.#protocol = protocol; @@ -53,6 +60,9 @@ export class Response { } static { + // TypeScript has no friend classes, so ResponseBuilder reaches the private constructor through this + // module-scoped hook, assigned exactly once. HTTP-2: no public field-wise constructor may appear in + // the emitted `.d.ts`, or a consumer can construct around build()'s required-field validation. // eslint-disable-next-line max-params -- private, builder-internal plumbing; field count fixed by HTTP-6 createResponse = (request, protocol, status, reasonPhrase, headers, body) => new Response(request, protocol, status, reasonPhrase, headers, body); @@ -72,7 +82,8 @@ export class Response { * * Every field it carries is itself immutable — `Request` freezes and defensively clones its URL, * and `Headers`, `Status`, and `Protocol` are frozen values — so sharing them cannot leak - * mutability back into either instance. + * mutability back into either instance. The body stream is shared by reference, since it is + * single-use by definition (BODY-14) and a copy would be a replay. * * @returns a {@link ResponseBuilder} holding this response's state. */ @@ -111,12 +122,98 @@ export class Response { return this.#headers; } + /** Single-use (BODY-14) -- the same reference every call, never a replay. */ + get body(): ReadableStream<Uint8Array> | null { + return this.#body; + } + /** - * The response body, or `undefined` when absent. Typed `unknown` until the body lifecycle lands - * in a later phase. + * Reads the whole body as bytes, closing the response whether or not the read succeeds (BODY-16). + * + * @returns every byte of the body, or an empty array when there is no body. + * @throws Whatever the body stream raises mid-read, and a `TypeError` when an external consumer + * already holds the body's reader lock. The connection is released in every case. */ - get body(): unknown { - return this.#body; + async bytes(): Promise<Uint8Array> { + if (this.#body === null) { + await this.close(); + return new Uint8Array(0); + } + const chunks: Uint8Array[] = []; + let total = 0; + // Acquired INSIDE the try. `getReader()` itself throws a TypeError when an external consumer + // already holds the lock, and BODY-15 forbids assuming the body was never touched -- so acquiring + // it above the try meant the one failure BODY-16's guarantee most needs to cover was the one that + // skipped the close entirely, leaving the connection held. + let reader: ReadableStreamDefaultReader<Uint8Array> | undefined; + try { + reader = this.#body.getReader(); + for (;;) { + // Serial by necessity: each read depends on the previous one advancing the cursor. + const {done, value} = await reader.read(); + if (done) break; + chunks.push(value); + total += value.length; + } + } finally { + // MUST precede close(): ReadableStream.cancel() rejects with TypeError on a locked stream, and + // reading to done does NOT release the lock. Without this the finally replaces the read value + // with a TypeError and bytes()/text() never succeed. + reader?.releaseLock(); + await this.close(); + } + const result = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + result.set(chunk, offset); + offset += chunk.length; + } + return result; + } + + /** + * Reads the whole body as text, closing the response the same way {@link Response.bytes} does. + * + * Decodes with the charset declared by `content-type`, falling back to UTF-8 when it is absent, + * unparseable, or an unrecognized label (HTTP-42). + * + * @returns the decoded body, or the empty string when there is no body. + * @throws Whatever {@link Response.bytes} throws, which this delegates to -- including the + * `TypeError` an externally locked body produces. The connection is released in every case. + */ + async text(): Promise<string> { + const bytes = await this.bytes(); + return decodeBodyText( + bytes, + resolveCharset(this.#headers.get('content-type')), + ); + } + + /** + * Releases the underlying connection whether or not the body was ever read (BODY-15, HTTP-43). + * + * Idempotent, and safe to call while an external consumer still holds the body's reader lock. + * + * @throws Whatever cancelling the body stream raises, other than the `TypeError` a locked stream + * reports — that one is expected here and swallowed. + */ + async close(): Promise<void> { + // Memoized rather than flag-guarded, the same shape BufferedSink.close settled on for IO-5/IO-41: + // a `#closed = true` set before the await reports a FAILED release as success to every later caller, + // over a connection that was never released. Handing every caller the same promise propagates the + // failure on every path while still cancelling at most once. + this.#closing ??= this.#release(); + return this.#closing; + } + + async #release(): Promise<void> { + if (this.#body === null) return; + // BODY-15 forbids assuming the body was read, so an external consumer may still hold the reader + // lock -- cancel() rejects with TypeError in that case. Swallow only that: the caller asked to + // release the connection, and the lock holder's own close will finish the job. + await this.#body.cancel().catch((error: unknown) => { + if (!(error instanceof TypeError)) throw error; + }); } } @@ -131,7 +228,7 @@ export class ResponseBuilder implements Builder<Response> { #status: Status | undefined; #reasonPhrase: string | undefined; #headers: Headers = Headers.newBuilder().build(); - #body: unknown; + #body: ReadableStream<Uint8Array> | null = null; /** * Sets the originating request. Required. @@ -191,10 +288,12 @@ export class ResponseBuilder implements Builder<Response> { /** * Sets the response body. * - * @param body - the body, or `undefined` when absent. + * @param body - the single-use body stream, or `null` when the response carries none. + * `null` rather than `undefined` here mirrors WHATWG `fetch`'s `Response.body` deliberately; + * `Request.body` keeps the domain model's `undefined` convention. * @returns this builder, for chaining. */ - body(body: unknown): this { + body(body: ReadableStream<Uint8Array> | null): this { this.#body = body; return this; } diff --git a/packages/core/src/http/rfc3986.ts b/packages/core/src/http/rfc3986.ts index a2d6db5..c96955a 100644 --- a/packages/core/src/http/rfc3986.ts +++ b/packages/core/src/http/rfc3986.ts @@ -1,13 +1,66 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/rfc3986.ts +/** + * Matches an UNPAIRED surrogate code unit, and only an unpaired one. + * + * In a `u`-mode pattern the engine works in code points, so a well-formed surrogate pair is one + * non-surrogate code point and does not match, while a lone high or low unit stays a surrogate code + * point and does. Equivalent to `!String.prototype.isWellFormed()`, which is ES2024 and so outside + * this repo's declared `lib` (`tsconfig.base.json` pins `ES2023`) even though the + * `engines.node >= 20.3` runtime has it — raising `lib` for one predicate is a wider change than + * the predicate is worth. + * + * Two patterns, one rule: `.test()` must not carry `lastIndex` between calls, and `.replace()` must + * be global. Neither is exported; the two functions below are, so no caller can pick the wrong one. + */ +const LONE_SURROGATE = /\p{Surrogate}/u; +const LONE_SURROGATE_GLOBAL = /\p{Surrogate}/gu; + +/** Unicode's replacement character, what a lenient repair substitutes for an unpaired surrogate. */ +const REPLACEMENT_CHARACTER = '\uFFFD'; + +/** + * Whether `value` carries an unpaired surrogate, and so has no UTF-8 form and cannot be + * percent-encoded. The strict half of the rule: a call site that was HANDED such a string rejects + * it (audit #67 / #76). + * + * @param value - the string to inspect. + * @returns `true` when at least one surrogate code unit is unpaired. + */ +export function hasLoneSurrogate(value: string): boolean { + return LONE_SURROGATE.test(value); +} + +/** + * `value` with every unpaired surrogate replaced by U+FFFD. The lenient half, for a call site that + * MUST NOT throw — `QueryParams.parse` under HTTP-31. Matches what the platform's own query + * serializer does with the same input: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` + * (measured 2026-09-05). + * + * @param value - the string to repair. + * @returns `value` with unpaired surrogates replaced; the same string when there are none. + */ +export function toWellFormed(value: string): string { + return value.replace(LONE_SURROGATE_GLOBAL, REPLACEMENT_CHARACTER); +} + /** * Percent-encodes a single URL component per RFC 3986, patching `encodeURIComponent`'s divergence: * `encodeURIComponent` leaves `! * ' ( )` unescaped, but none of them are in RFC 3986's unreserved * set (HTTP-29). * - * @param value - the raw component value. + * Deliberately NOT total, and deliberately not guarded here. `encodeURIComponent` throws + * `URIError: URI malformed` on a string carrying an unpaired surrogate, because such a string has + * no UTF-8 form. Every caller in this package rejects or repairs that input BEFORE reaching here — + * `QueryParamsBuilder.add`, `QueryParams.parse` and `substitutePathParams` each do, and each throws + * the error class its own call site already throws — so a second, silent guard inside the encoder + * would only move the failure back off the call site (audit #67 / #76). + * + * @param value - the raw component value; must not carry an unpaired surrogate. * @returns the percent-encoded component. + * @throws A platform `URIError` when `value` carries an unpaired surrogate. Callers validate first; + * see the note above. */ export function encodeRfc3986Component(value: string): string { return encodeURIComponent(value).replace( diff --git a/packages/core/src/index.public.test.ts b/packages/core/src/index.public.test.ts new file mode 100644 index 0000000..a020427 --- /dev/null +++ b/packages/core/src/index.public.test.ts @@ -0,0 +1,126 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/index.public.test.ts +// Exercises: SEAM-21's closure — the reshaped serde seam is reachable from the package's public entry +// point, which is what a separate `@dexpace/codec-json` package needs and what promotes it (SERDE-1/5). +import {expect, test} from 'bun:test'; + +test('the serde seam is publicly importable, because a separate codec package must reach it', async () => { + const barrel = await import('./index.js'); + for (const name of [ + 'absent', + 'nullValue', + 'present', + 'ofNullable', + 'foldTristate', + 'valueOrNull', + 'isAbsent', + 'isNull', + 'isPresent', + 'isTristate', + 'tristateToString', + 'TRISTATE_BRAND', + 'SerializationError', + 'DeserializationError', + 'isSerdeError', + 'serdeBody', + 'decodeResponse', + 'decodeSuccessResponse', + ]) { + expect(barrel).toHaveProperty(name); + } +}); + +test('io/ is still not public — 3b froze that decision and 6a does not reopen it', async () => { + const barrel = await import('./index.js'); + for (const name of [ + 'ByteQueue', + 'BufferedSource', + 'BufferedSink', + 'TeeSink', + ]) { + expect(barrel).not.toHaveProperty(name); + } +}); + +test('the SSE surface is publicly importable', async () => { + const barrel = await import('./index.js'); + for (const name of [ + 'sseStreamFrom', + 'SseStream', + 'typedSseStream', + 'mapperValue', + 'MAPPER_SKIP', + 'MAPPER_DONE', + 'SseStreamError', + 'SseLineTooLongError', + 'makeSseEvent', + 'sseEventsEqual', + 'isSseEventEmpty', + 'sseEventToString', + ]) { + expect(barrel).toHaveProperty(name); + } +}); + +test('the pagination surface is publicly importable', async () => { + const barrel = await import('./index.js'); + for (const name of [ + 'Paginator', + 'Page', + 'pageInfo', + 'cursorStrategy', + 'pageNumberStrategy', + 'linkHeaderStrategy', + 'paginateWithFetchers', + 'PaginationError', + ]) { + expect(barrel).toHaveProperty(name); + } +}); + +test('the SSE parser internals stay private — publishing them would publish a way to break SSE-17', async () => { + const barrel = await import('./index.js'); + for (const name of ['SseParser', 'SseLineReader']) { + expect(barrel).not.toHaveProperty(name); + } +}); + +test('the URL-manipulation internals stay private — one public query surface, not two', async () => { + const barrel = await import('./index.js'); + for (const name of [ + 'spliceQueryParam', + 'readQueryParam', + 'parseLinkHeader', + 'findNextLink', + ]) { + expect(barrel).not.toHaveProperty(name); + } +}); + +test('nothing under src/pagination/ imports serde (§12: the engine is serde-agnostic)', async () => { + const {readdirSync, readFileSync} = await import('node:fs'); + const paginationDir = new URL('./pagination/', import.meta.url); + const names = readdirSync(paginationDir).filter( + f => f.endsWith('.ts') && !f.endsWith('.test.ts'), + ); + const sourceOf = (name: string): string => + readFileSync(new URL(name, paginationDir), 'utf8'); + for (const name of names) { + const code = sourceOf(name).replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, ''); + expect(code).not.toMatch(/from\s+['"].*(serde|codec-json)/); + } +}); + +test('nothing under src/pagination/ uses URLSearchParams (PAGE-21)', async () => { + const {readdirSync, readFileSync} = await import('node:fs'); + const paginationDir = new URL('./pagination/', import.meta.url); + const names = readdirSync(paginationDir).filter( + f => f.endsWith('.ts') && !f.endsWith('.test.ts'), + ); + const sourceOf = (name: string): string => + readFileSync(new URL(name, paginationDir), 'utf8'); + for (const name of names) { + const code = sourceOf(name).replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, ''); + expect(code).not.toContain('URLSearchParams'); + } +}); diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 9ae93a0..49eadf2 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,11 +1,18 @@ // SPDX-License-Identifier: MIT // packages/core/src/index.ts /** - * The immutable, transport-agnostic HTTP domain model at the heart of `@dexpace/core`. + * The transport-agnostic HTTP core of `@dexpace/core`: an immutable domain model, and the pipeline + * that drives it. * - * Every type here is frozen at construction and built through a builder or a static factory, so - * case-insensitivity, multi-value semantics, ordering, header-injection defenses, method/body - * legality, and total status handling are fixed once and behave identically under every transport. + * Every DOMAIN MODEL type — requests, responses, headers, bodies, status — is frozen at construction + * and reachable only through a builder or a static factory, so case-insensitivity, multi-value + * semantics, ordering, header-injection defenses, method/body legality, and total status handling are + * fixed once and behave identically under every transport. + * + * The PIPELINE surface promoted in Phase 5c is deliberately not held to that rule. `PipelineBuilder` + * is mutable by design and freezes only at `build()`; `Stage`, `Step`, `Next`, `StepContext`, and the + * settings records are plain types a caller writes literals for; `authStep`, `retryStep`, + * `redirectStep`, and `standardResilience` are factories returning descriptors and runtimes. * * The package has zero runtime dependencies. * @@ -13,14 +20,384 @@ */ export * from './http/index.js'; -// Deliberately NOT `export * from './seams/index.js';` — that barrel also carries the internal-only, -// provisional Serde<T> (SEAM-21 will reshape it in Phase 6). Naming each public export here instead keeps -// Serde<T> unreachable from the package's public entry point and out of the api-extractor surface. +// `packages/core/src/seams/` deliberately has NO folder-level barrel. It carried one from Phase 2 +// until 2026-09-02, when it was deleted: docs/knowledge/harvested/module-organization.md:18 bans +// internal folder-level barrels outright and api-design.md:6 makes this file the package's sole +// barrel, and nothing had ever imported it -- its only reference in the workspace was the comment +// here explaining why it was not re-exported. Naming each public export below keeps the package's +// surface a decision made in one place rather than a consequence of what a folder re-exports. export type {Transport} from './seams/transport.js'; export { composeSignal, isTimeoutSignal, CancellationError, } from './seams/transport.js'; +// The four flat leaves plus `isIoError`. `decodeResponse`'s guard passes anything already in this +// SDK's typed tree through untouched, so a caller genuinely receives `ClosedResourceError` and its +// siblings today and until 2026-09-04 had no name to catch them by. The guard is the category catch +// that a flat tree cannot offer through `instanceof` (docs/work/mvp/2026-09-04-open-items-dissolution.md H8). +export { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, + TransportFailureError, +} from './io/errors.js'; +// The shape of a suppressed-error pair, type-only. `instanceof SuppressedError` is NOT a valid test +// on the declared `engines.node >=20.3` floor, where the global is absent, so a caller that wants to +// narrow one needs this interface rather than the class (docs/work/mvp/2026-09-04-open-items-dissolution.md H8). +export type {SuppressedErrorLike} from './suppress.js'; export type {OperationDescriptor} from './seams/operation.js'; export {buildRequest, OperationAssemblyError} from './seams/operation.js'; + +// Deliberately NOT `export * from './body/index.js';` — that barrel also carries withRequestLogging/ +// withResponseLogging, internal until Phase 7 supplies a Logger to drive them. Naming each public export +// here instead keeps that boundary enforced at the barrel, not by convention. +// The concrete body classes are exported as TYPES ONLY. Exporting the class as a value publishes +// `new ByteArrayBody(...)` as a field-wise constructor, which HTTP-2 forbids ("constructible only +// through their builder or dedicated factory") and which duplicates the factory functions for no +// stated need (NFR-3). Callers construct via the factories and annotate with the types. +export type {Body, FileBodyDescriptor} from './body/body.js'; +export { + ConsumedBodyError, + FormBodyValidationError, + HttpStatusValidationError, + isBodyError, + MultipartBoundaryError, +} from './body/errors.js'; +export {HttpStatusError, toHttpError} from './body/http-status-error.js'; +export {materialize} from './body/materialize.js'; +export { + multipartBody, + type MultipartBody, + MultipartBodyBuilder, + type MultipartPart, +} from './body/multipart-body.js'; +export { + byteArrayBody, + type ByteArrayBody, + formUrlEncodedBody, + type FormUrlEncodedBody, + type FormUrlEncodedInput, + type FormUrlEncodedValue, + stringBody, + type StringBody, +} from './body/simple-bodies.js'; +export {streamBody, type StreamBody} from './body/stream-body.js'; +export {TypedResponse} from './body/typed-response.js'; + +// --------------------------------------------------------------------------------------------- +// The pillar-authoring surface, promoted in Phase 5c. +// +// 5c is the first point a caller can assemble a genuinely working pipeline -- all three resilience +// pillars plus the preset now exist. Promoting any earlier would have frozen shapes 5c still had +// latitude to reshape, which is why every prior phase deliberately exported nothing from here. +// --------------------------------------------------------------------------------------------- + +// Group 1: the authoring surface itself. +export type {Stage} from './pipeline/stage.js'; +export {PILLAR_STAGES, STAGE_ORDER} from './pipeline/stage.js'; +export type {Next, Step, StepContext, StepDescriptor} from './pipeline/step.js'; +export {PipelineBuilder} from './pipeline/builder.js'; +export {Runtime} from './pipeline/runtime.js'; +// The five errors a hand-built pipeline can actually provoke. Every one is the subject of a +// `@throws` tag on a symbol above, and until 2026-09-02 none was exported, so a consumer read the +// tag, reached for `instanceof`, and had nothing to reach for. +export { + AnchorNotFoundError, + CrossStageEditError, + CursorAlreadyAdvancedError, + PillarCollisionError, + ReservedStageError, +} from './pipeline/errors.js'; +export {retryStep} from './retry/retry-step.js'; +// RETRY-34's read side. The retry pillar surfaces the FINAL attempt's own error, so `instanceof` +// against it does not depend on how many attempts ran; this is how the earlier ones are reached. +// Exported alongside `retryStep` because the two are one contract: nothing else in the barrel can +// tell a caller that the error they caught is the third of three. +export {retryAttempts} from './retry/attempt-trail.js'; +// The trail entry for a response the engine discarded whose status is outside 400-599 — reachable +// only through a caller-widened `retryableStatuses`. +export {RetryDiscardedResponseError} from './retry/errors.js'; +export {redirectStep} from './redirect/redirect-step.js'; +// `withRedirect` installs `redirectStep` together with the REDIR-11(c) guard that keeps the internal +// cross-origin marker off the wire. Publishing the pillar without them made `withRedirect`'s own +// instruction -- "a caller who installs redirectStep() directly is responsible for installing the +// guard too" -- name an obligation no consumer could discharge. +export { + stripCrossOriginMarkerStep, + withRedirect, +} from './redirect/strip-marker-step.js'; +export { + NonReplayableBodyError, + SchemeDowngradeError, +} from './redirect/errors.js'; +export {authStep} from './auth/auth-step.js'; +export {standardResilience} from './auth/preset.js'; + +// Group 2: everything Group 1's signatures name. A promoted function whose parameter type is +// internal-only is an API a caller cannot call, and api-extractor reports each omission as +// `ae-forgotten-export`. +// +// The word "internal-only" above is deliberate and must not be spelled as the TSDoc tag: gts turns +// `stripInternal` on, and TypeScript tests the WHOLE leading comment range of a declaration for that +// tag as a substring -- so writing it in prose here silently deletes the export below from the +// emitted `.d.ts`. It did, for one commit. `api-extractor.json` now fails `api:ci` on the resulting +// `ae-forgotten-export`, and `verify:consumer-types` compiles these four names from the built +// package, so the same slip cannot ship twice. +// The whole context family, not just `ExecutionContext`: it is a union alias, and `StepContext.context` +// makes every member reachable from a promoted signature. A caller writing a custom step reads +// `ctx.context.kind` to tell which promotion stage it is in. +export type { + DispatchContext, + ExchangeContext, + ExecutionContext, + RequestContext, +} from './context/context.js'; +export type {InstrumentationBundle} from './context/instrumentation.js'; +// `PipelineOptions` is what a caller passes to `new PipelineBuilder(transport, options)` and, by +// extension, to `standardResilience`; it is the only public route to `OBS-29`'s per-operation span +// and `CTX-16`'s operation name, so it is exported beside the bundle it carries. +export type {PipelineOptions} from './pipeline/builder.js'; +export type {BackoffSettings} from './retry/backoff.js'; +export type {RetrySettings} from './retry/settings.js'; +export type {RetryStepOptions} from './retry/retry-step.js'; +export type { + RedirectCondition, + RedirectPredicate, + RedirectSettings, +} from './redirect/settings.js'; +export type {StandardResilienceOptions} from './auth/preset.js'; +export type { + ApiKeyCredentialConfig, + AuthCredentialSet, + AuthStepSettings, + BearerCredential, + ChallengeHook, +} from './auth/auth-step.js'; +export type {AuthTiers} from './auth/resolve.js'; +export type {AuthScheme} from './auth/scheme.js'; +export type {DigestAlgorithm} from './auth/digest.js'; + +// Factories, not bare interfaces: AUTH-3 validates and freezes inside `createAuthDescriptor`, and +// every credential type is NOMINAL -- each carries a `#` field, so no caller-side object literal is +// assignable and the AUTH-9 validation in each factory cannot be routed around. Without these, +// API_KEY, OAUTH2, BASIC and DIGEST auth are unreachable from outside the package. +// All five are VALUE exports, not type-only ones: they are classes, `TokenProvider` returns a +// `BearerToken`, and `BasicCredential`/`DigestCredential` became classes on 2026-09-04 so AUTH-8's +// redaction covers their passwords too (audit #67 / #71). +export type {AuthDescriptor} from './auth/descriptor.js'; +export {createAuthDescriptor} from './auth/descriptor.js'; +export type {AuthRequirement} from './auth/requirement.js'; +export { + authRequirementsEqual, + createAuthRequirement, +} from './auth/requirement.js'; +export type {TokenProvider} from './auth/credential.js'; +export { + ApiKeyCredential, + BasicCredential, + BearerToken, + DigestCredential, + NameKeyCredential, + bearerTokensEqual, + createBearerToken, +} from './auth/credential.js'; +export {AuthResolutionError, PlaintextCredentialError} from './auth/errors.js'; + +// --------------------------------------------------------------------------------------------- +// The serde seam, promoted in Phase 6a. +// +// Public because `@dexpace/codec-json` is a SEPARATE PACKAGE and can reach core only through this +// entry point — which is what settles the promotion question by force. Phase 2 kept `Serde<T>` +// package-private precisely so SEAM-21's reshape would not be a breaking change; the reshape has +// landed, so that marking comes off here. +// +// The phrase "package-private" is deliberate: `stripInternal` is on, and TypeScript tests a +// declaration's WHOLE leading comment range for the release tag as a SUBSTRING — spelling that tag +// out in prose here silently deletes the export below from the emitted `.d.ts`. It did, once. +// --------------------------------------------------------------------------------------------- +export type {Deserializer, Schema, Serde, Serializer} from './seams/serde.js'; +export { + DeserializationError, + isSerdeError, + SerializationError, +} from './serde/errors.js'; +export type { + DeserializationErrorOptions, + SerdeErrorOptions, +} from './serde/errors.js'; +export { + absent, + foldTristate, + isAbsent, + isNull, + isPresent, + isTristate, + nullValue, + ofNullable, + present, + TRISTATE_BRAND, + tristateToString, + valueOrNull, +} from './serde/tristate.js'; +export type {Tristate, TristateBranches} from './serde/tristate.js'; +export { + decodeResponse, + decodeSuccessResponse, +} from './serde/response-handlers.js'; +export type {DecodeTarget} from './serde/response-handlers.js'; +export {serdeBody} from './body/serde-body.js'; + +// SSE (Phase 6b). The parser and line reader stay internal: they are driven only through the facade, and +// exposing them would expose a way to violate SSE-17's non-ownership contract by accident. +export type {SseEvent, SseEventFields} from './sse/event.js'; +export { + isSseEventEmpty, + makeSseEvent, + sseEventToString, + sseEventsEqual, +} from './sse/event.js'; +export {SseLineTooLongError} from './sse/line-reader.js'; +export {SseStreamError} from './sse/errors.js'; +export {SseStream, sseStreamFrom} from './sse/stream.js'; +export type {SseStreamFromOptions, SseStreamOptions} from './sse/stream.js'; +export { + MAPPER_DONE, + MAPPER_SKIP, + mapperValue, + typedSseStream, +} from './sse/typed.js'; +export type {MapperOutcome, SseMapper} from './sse/typed.js'; + +// Pagination (Phase 6c). The query splice and link tokenizer stay internal: publishing them would put a second +// URL-manipulation surface next to Phase 1's QueryParams, which is the confusion the one-encoder rule avoids. +export {Page, pageInfo} from './pagination/page.js'; +export type {PageInfo} from './pagination/page.js'; +export type {PaginationStrategy} from './pagination/strategy.js'; +export {Paginator} from './pagination/paginator.js'; +export type {PaginatorInit} from './pagination/paginator.js'; +export { + cursorStrategy, + linkHeaderStrategy, + pageNumberStrategy, +} from './pagination/strategies.js'; +export {paginateWithFetchers} from './pagination/fetchers.js'; +export type { + FetcherPage, + FetcherPaginationInit, + PagingOptions, +} from './pagination/fetchers.js'; +export {PaginationError} from './pagination/errors.js'; + +// Phase 4b — the recovery-chain execution model (RECOV-*). Published 2026-09-04: `DispatchConfig` +// requires a `requestChain` and a `responseChain`, nothing in this package constructs one, and the +// whole folder was internal-only — so the execution model the RECOV requirements describe had no +// entry point at all. The chains stay CLASSES rather than plain data plus free functions +// (docs/knowledge/harvested/data-modeling.md:10 would prefer the latter): RECOV-14's text is written +// about chain and step *instances*, and the defensive copy wants a construction boundary. +export type {RequestStep} from './recovery/request-chain.js'; +export {RequestRecoveryChain} from './recovery/request-chain.js'; +export type {RecoveryStep, ResponseStep} from './recovery/response-chain.js'; +export {ResponseRecoveryChain} from './recovery/response-chain.js'; +export type {Outcome} from './recovery/outcome.js'; +export {failure, fold, success} from './recovery/outcome.js'; +export type {DispatchConfig} from './recovery/orchestrator.js'; +export {dispatchWithRecovery} from './recovery/orchestrator.js'; +export {statusMappingStep} from './recovery/status-mapping.js'; +export {wrapCancellation} from './recovery/cancellation.js'; +// RECOV-32's idempotency-key step. A `RequestStep`, not a `StepDescriptor` — it composes into a +// `RequestRecoveryChain` rather than into a pipeline stage, which is why it is here and +// `clientIdentityStep` is with the Phase 7a block below. +export type {IdempotencyKeyOptions} from './recovery/idempotency-key.js'; +export {idempotencyKeyStep} from './recovery/idempotency-key.js'; + +// Phase 7a — configuration and platform primitives. There is no `./config/index.js`, because 7a's +// design doc rules one out by name. Not because the question is settled: this repo carries both +// patterns — `http/`, `body/`, `io/`, and `seams/` each have an internal barrel, while `pipeline/`, +// `context/`, and `config/` do not — and so does the knowledge corpus, where +// docs/knowledge/harvested/module-organization.md:18 bans internal barrels outright and +// docs/knowledge/harvested/api-design.md:8 endorses one per feature folder, with no entry in the corpus's +// `--section conflicts` reconciling them. 7a followed its design doc and names each symbol here +// against its own file. That stays the shape, and `client-identity-step.ts` stays in `config/`: +// once a symbol is `@public` and named here against its own module path, its folder is invisible to +// every consumer, and moving it to `recovery/` would only trade its one outbound `→ pipeline/` edge +// for a new `→ config/` one for `./build-info.js` (docs/work/mvp/2026-09-04-open-items-dissolution.md K11, closed 2026-09-04). +// Deliberately NOT exported: `config/equality.js`'s deepEqual/deepHash — no requirement gives a +// caller direct access to them, and they have no in-package caller either as of 2026-08-27, so the +// module is reachable only from its own test (docs/work/mvp/2026-09-04-open-items-dissolution.md K16 owns the first real consumer). +export type {Clock} from './config/clock.js'; +export {defaultClock} from './config/clock.js'; +export type {BuildInfo} from './config/build-info.js'; +export {getBuildInfo} from './config/build-info.js'; +// RECOV-33's identity-stamping step. Public because a caller installs it in their own pipeline — +// `standardResilience` does not install it — so an unexported factory satisfies nothing. Promoted +// 2026-09-04; every other step factory (`authStep`, `retryStep`, `redirectStep`, `loggingStep`, +// `stripCrossOriginMarkerStep`) was already here (docs/work/mvp/2026-09-04-open-items-dissolution.md K1). +export type {ClientIdentitySettings} from './config/client-identity-step.js'; +export {clientIdentityStep} from './config/client-identity-step.js'; +export type {Configuration, SourceFn} from './config/configuration.js'; +export { + CFG_KEY_HTTPS_PROXY, + CFG_KEY_HTTP_PROXY, + CFG_KEY_LOG_LEVEL, + CFG_KEY_MAX_RETRY_ATTEMPTS, + CFG_KEY_NO_PROXY, + ConfigurationBuilder, + defaultConfiguration, + getGlobalConfiguration, + setGlobalConfiguration, +} from './config/configuration.js'; +export {formatHttpDate, parseHttpDate} from './config/http-date.js'; +export {randomUuid} from './config/identifiers.js'; +export type { + ProxyCredentials, + ProxyOptions, + ProxyOptionsInit, + ProxyType, +} from './config/proxy.js'; +export { + createProxyOptions, + formatProxyOptions, + resolveProxyOptions, + shouldBypassProxy, +} from './config/proxy.js'; +export {RETRYABLE_STATUSES, isRetryableStatus} from './config/retryable.js'; + +// Phase 7b — Observability and instrumentation. +export type { + CreateLoggerOptions, + LogEvent, + LogLevel, + Logger, +} from './observability/logger.js'; +export { + NOOP_LOGGER, + createLogger, + getGlobalLogger, + setGlobalLogger, +} from './observability/logger.js'; +export type { + Scope, + Span, + SpanContext, + Tracer, +} from './observability/tracing.js'; +export { + NOOP_SPAN, + NOOP_TRACER, + activateSpan, + activateSpanForCorrelation, + createInstrumentationBundle, + getActiveSpan, +} from './observability/tracing.js'; +export type {Counter, Histogram, Meter} from './observability/metrics.js'; +export {NOOP_METER} from './observability/metrics.js'; +export type {DroppedHeaderPolicy} from './observability/redaction.js'; +export type { + LoggingGranularity, + LoggingStepSettings, +} from './observability/logging-step.js'; +export {LOGGING_STEP_TYPE, loggingStep} from './observability/logging-step.js'; diff --git a/packages/core/src/invariant.test.ts b/packages/core/src/invariant.test.ts new file mode 100644 index 0000000..7b55da9 --- /dev/null +++ b/packages/core/src/invariant.test.ts @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/invariant.test.ts +// Exercises: the project's sole assertion primitive (styleguide 5.6), its error class, and the +// discriminated-union exhaustiveness helper docs/knowledge/harvested/data-modeling.md requires every switch to +// close with. +import {describe, expect, test} from 'bun:test'; +import {assertNever, invariant, InvariantViolation} from './invariant.js'; + +describe('invariant', () => { + test('does not throw when the condition is truthy', () => { + expect(() => { + invariant(true, 'unreachable'); + }).not.toThrow(); + }); + + test('throws InvariantViolation with the given message when the condition is falsy', () => { + expect(() => { + invariant(false, 'broken precondition'); + }).toThrow(InvariantViolation); + expect(() => { + invariant(false, 'broken precondition'); + }).toThrow('broken precondition'); + }); + + test('InvariantViolation sets its name and descends from Error', () => { + const error = new InvariantViolation('boom'); + expect(error).toBeInstanceOf(Error); + expect(error.name).toBe('InvariantViolation'); + expect(error.message).toBe('boom'); + }); +}); + +describe('assertNever', () => { + test('throws InvariantViolation naming the unreachable value', () => { + expect(() => { + // @ts-expect-error -- deliberately calling with a value that is not `never`, to exercise the + // runtime path a newly-added union variant would reach. + assertNever('unexpected-variant'); + }).toThrow(InvariantViolation); + expect(() => { + // @ts-expect-error -- same as above. + assertNever('unexpected-variant'); + }).toThrow('unexpected-variant'); + }); + + test('does not throw from its own message construction on an unstringifiable value', () => { + // `String()` is not total: a null-prototype object has no `toString` to reach, and a value + // whose `toString` throws propagates that throw. An assertion helper that reported THOSE + // instead of the invariant violation would name the wrong failure at the worst moment. + expect(() => { + assertNever(Object.create(null) as never); + }).toThrow(InvariantViolation); + expect(() => { + assertNever({ + toString() { + throw new Error('boom'); + }, + } as never); + }).toThrow(InvariantViolation); + }); + + test('accepts a custom message', () => { + expect(() => { + // @ts-expect-error -- same as above. + assertNever('x', 'custom message'); + }).toThrow('custom message'); + }); +}); diff --git a/packages/core/src/invariant.ts b/packages/core/src/invariant.ts new file mode 100644 index 0000000..39a427e --- /dev/null +++ b/packages/core/src/invariant.ts @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/invariant.ts + +/** + * Thrown by {@link invariant} when a broken precondition or postcondition is detected. + * + * Its own class distinguishes a programmer error — a violated invariant — from an operational + * failure a caller might recover from (styleguide 5.6, 8.7). + * + * @internal + */ +export class InvariantViolation extends Error { + constructor(msg: string) { + super(msg); + this.name = 'InvariantViolation'; + } +} + +/** + * The project's single sanctioned assertion primitive (styleguide 5.6). + * + * A TypeScript assertion function: after `invariant(x !== undefined, msg)`, `x` narrows to exclude + * `undefined` for the rest of the scope. Used for preconditions and postconditions — broken + * invariants, never operational failures a caller might recover from, which go through the typed + * error tree instead. + * + * @internal + */ +export function invariant(cond: unknown, msg: string): asserts cond { + if (!cond) throw new InvariantViolation(msg); +} + +/** + * Closes an exhaustive discriminated-union `switch`'s `default` case + * (`docs/knowledge/harvested/data-modeling.md`). If a new union variant is ever added without a matching + * `case`, the call stops type-checking; if one reaches this at runtime anyway — a value crossing a + * seam that the type says cannot exist — it crashes loudly rather than falling through silently. + * + * @internal + */ +export function assertNever(value: never, message?: string): never { + throw new InvariantViolation( + message ?? `unreachable case: ${describe(value)}`, + ); +} + +/** + * `String(value)` is not total: it throws on a null-prototype object (no `toString` to reach) and + * on any value whose `toString`/`Symbol.toPrimitive` throws — the same hazard + * `docs/knowledge/harvested/error-handling.md:18` makes `toError` guard. An assertion helper that throws from + * its own message construction reports the wrong failure at the worst moment, so the fallback is a + * fixed string. + */ +function describe(value: unknown): string { + try { + return String(value); + } catch { + return 'an unstringifiable value'; + } +} diff --git a/packages/core/src/io/buffered-sink.test.ts b/packages/core/src/io/buffered-sink.test.ts new file mode 100644 index 0000000..de5fa4e --- /dev/null +++ b/packages/core/src/io/buffered-sink.test.ts @@ -0,0 +1,314 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-sink.test.ts +// Exercises: IO-4 (exact head removal, no partial write), IO-5 (flush, closeable), +// IO-13 (symmetric write-side encodings), IO-18 (emit vs flush), IO-41 (idempotent close), +// IO-42 (rejects after close), IO-6 (wrapper owns the caller's stream), +// IO-16 (writable bridge: close closes, abort aborts) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSink} from './buffered-sink.js'; +import {BufferedSource} from './buffered-source.js'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, EndOfStreamError} from './errors.js'; +import { + collectingWritableStream, + failingCloseWritableStream, + failingWritableStream, + gatedWritableStream, +} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const queueOf = (...values: number[]): ByteQueue => { + const queue = new ByteQueue(); + queue.writeBytes(Uint8Array.from(values)); + return queue; +}; + +describe('BufferedSink', () => { + test('IO-4: write removes exactly the requested count from the source head', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const source = queueOf(1, 2, 3, 4); + await sink.write(source, 3); + await sink.close(); + expect([...written()]).toEqual([1, 2, 3]); + expect(source.size).toBe(1); + }); + + test('IO-4: writing more than the source holds throws and transfers nothing', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const source = queueOf(1, 2); + expect(await rejection(sink.write(source, 3))).toBeInstanceOf( + EndOfStreamError, + ); + await sink.close(); + expect([...written()]).toEqual([]); + expect(source.size).toBe(2); + }); + + test('IO-13: writeUtf8 encodes non-ASCII text symmetrically with the read side', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.writeUtf8('héllo ☃'); + await sink.close(); + expect(new TextDecoder('utf-8').decode(written())).toBe('héllo ☃'); + }); + + test('IO-13: writeString encodes ISO-8859-1', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.writeString('hé', 'iso-8859-1'); + await sink.close(); + expect([...written()]).toEqual([0x68, 0xe9]); + }); + + test('IO-13: writeString rejects a code point ISO-8859-1 cannot represent', async () => { + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + expect( + (await rejection(sink.writeString('☃', 'iso-8859-1'))).message, + ).toContain('code point 9731 is not representable in iso-8859-1'); + }); + + test('IO-13: writeString rejects a charset the write side cannot encode', async () => { + // TextEncoder is UTF-8-only and SEAM-1 forbids an encoding dependency, so the write side covers + // exactly UTF-8 and ISO-8859-1. Anything else throws rather than silently re-encoding as UTF-8. + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + expect( + (await rejection(sink.writeString('x', 'shift_jis'))).message, + ).toContain( + 'unsupported write charset: shift_jis (only utf-8 and iso-8859-1 can be encoded)', + ); + }); +}); + +describe('BufferedSink charset round-trips (IO-13)', () => { + /** + * The FULL 0x00–0xFF range, C1 controls included. An earlier version of this generator carved out + * 0x80–0x9F, which is exactly the band where the platform diverges: the WHATWG Encoding Standard maps + * the label `iso-8859-1` onto windows-1252, so `TextDecoder` turns 0x80 into U+20AC. Excluding the + * divergent band made the property pass over a bug rather than find it — which is why decoding + * ISO-8859-1 is now this package's own job (see `decodeText`). + */ + const latin1Codes = fc.array(fc.integer({min: 0x00, max: 0xff}), { + maxLength: 64, + }); + + test('property: arbitrary text round-trips through the sink and back as UTF-8', async () => { + // Styleguide 11.5 names codecs explicitly, and IO-13's whole claim is that the write side is + // symmetric with the read side — a claim only a round-trip can check. + await fc.assert( + fc.asyncProperty( + fc.string({unit: 'grapheme', maxLength: 64}), + async text => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.writeString(text, 'utf-8'); + await sink.close(); + const source = BufferedSource.overBytes(written()); + expect(await source.readString('utf-8')).toBe(text); + }, + ), + ); + }); + + test('property: arbitrary ISO-8859-1 text round-trips as one byte per code point', async () => { + // IO-13's own conformance note names ISO-8859-1 as the non-UTF-8 charset to round-trip. The + // one-byte-per-code-point assertion is what distinguishes an honored charset from a silent + // UTF-8 re-encoding, which would widen every code point above 0x7F to two bytes. + await fc.assert( + fc.asyncProperty(latin1Codes, async codes => { + const text = codes.map(code => String.fromCharCode(code)).join(''); + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.writeString(text, 'iso-8859-1'); + await sink.close(); + expect([...written()]).toEqual(codes); + const source = BufferedSource.overBytes(written()); + expect(await source.readString('iso-8859-1')).toBe(text); + }), + ); + }); + + test('IO-13: the C1 band round-trips instead of becoming windows-1252 typography', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const text = '\u0080\u0091\u009f'; + await sink.writeString(text, 'iso-8859-1'); + await sink.close(); + expect([...written()]).toEqual([0x80, 0x91, 0x9f]); + const source = BufferedSource.overBytes(written()); + const decoded = await source.readString('iso-8859-1'); + // windows-1252 would give [0x20ac, 0x2018, 0x178] — EUR, curly quote, Y-diaeresis — none of which + // can be re-encoded, so the inverse direction breaks too. + expect(Array.from(decoded, c => c.codePointAt(0))).toEqual([ + 0x80, 0x91, 0x9f, + ]); + expect(decoded).toBe(text); + }); +}); + +describe('BufferedSink lifecycle (IO-18, IO-41, IO-42, IO-6)', () => { + test('IO-18: flush and emit both return the sink for chaining', async () => { + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + expect(await sink.emit()).toBe(sink); + expect(await sink.flush()).toBe(sink); + await sink.close(); + }); + + test('IO-18: flush waits for the destination to drain; emit does not', async () => { + // Identity assertions alone would pass with emit and flush sharing one body, which is precisely + // what IO-18 forbids: the requirement is that the two be DISTINGUISHABLE. + const {stream, delivered, release} = gatedWritableStream(); + const sink = BufferedSink.overStream(stream); + + void sink.write(queueOf(1, 2, 3, 4), 4); + const flushed = sink.flush(); + let flushSettled = false; + void flushed.then(() => { + flushSettled = true; + }); + + await Promise.resolve(); + expect(delivered()).toBe(0); + expect(flushSettled).toBe(false); + + release(); + await flushed; + expect(delivered()).toBe(4); + }); + + test('IO-18: emit surfaces a failure the underlying stream has already suffered', async () => { + // An emit that never touches the writer reports success on a dead stream, handing a caller that + // uses it as a handoff checkpoint a green light on a body that never left. + const sink = BufferedSink.overStream(failingWritableStream('boom')); + expect( + (await rejection(sink.write(queueOf(1, 2, 3, 4), 4))).message, + ).toContain('boom'); + expect((await rejection(sink.emit())).message).toContain('boom'); + expect((await rejection(sink.flush())).message).toContain('boom'); + }); + + test('IO-41: a close that FAILS reports the failure on every later call, never a silent success', async () => { + // Setting the closed flag and early-returning on it makes the retry resolve, so a destination that + // was never released is reported as closed and healthy (BODY-27 wants the failure surfaced). + const sink = BufferedSink.overStream( + failingCloseWritableStream('close failed'), + ); + expect((await rejection(sink.close())).message).toContain('close failed'); + expect((await rejection(sink.close())).message).toContain('close failed'); + expect(sink.closed).toBe(true); + }); +}); + +describe('BufferedSink write failure and empty payloads (IO-4, IO-25)', () => { + test('IO-4: a failed write leaves the caller its bytes to retry', async () => { + // Consuming from `src` before the downstream write is known to succeed destroys the payload: the + // caller catches the rejection holding nothing, and nothing reached the wire either. + const sink = BufferedSink.overStream(failingWritableStream('boom')); + const source = queueOf(1, 2, 3, 4); + expect((await rejection(sink.write(source, 4))).message).toContain('boom'); + expect(source.size).toBe(4); + expect([...source.snapshot()]).toEqual([1, 2, 3, 4]); + }); + + test('IO-4: a short source is refused before anything reaches the wire', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const source = queueOf(1, 2); + expect(await rejection(sink.write(source, 3))).toBeInstanceOf( + EndOfStreamError, + ); + expect(source.size).toBe(2); + expect(written().length).toBe(0); + }); + + test('an empty payload writes no chunk at all, matching the tee and the bridge', async () => { + // A zero-length chunk is the terminating chunk to an HTTP/1.1 chunked-encoding transport, so + // emitting one for `writeUtf8('')` can end a request body early. + const {stream, chunkSizes} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.writeUtf8(''); + await sink.writeString('', 'iso-8859-1'); + expect(chunkSizes()).toEqual([]); + }); +}); + +describe('BufferedSink bridge lifecycle (IO-16)', () => { + test('IO-16: aborting the bridge aborts the sink and carries the reason', async () => { + // Collapsing an abort into a graceful close commits a cancelled body downstream as a well-formed + // complete one, so the peer cannot tell an aborted upload from a successful short one. + const {stream, isClosed, wasAborted, abortReason} = + collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const writer = sink.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + const reason = new Error('user cancelled'); + await writer.abort(reason); + expect(wasAborted()).toBe(true); + expect(abortReason()).toBe(reason); + expect(isClosed()).toBe(false); + expect(sink.closed).toBe(true); + }); + + test('IO-16: closing the bridge closes the sink gracefully', async () => { + const {stream, isClosed, wasAborted} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const writable = sink.toWritableStream(); + const writer = writable.getWriter(); + await writer.write(Uint8Array.from([1])); + await writer.close(); + expect(isClosed()).toBe(true); + expect(wasAborted()).toBe(false); + expect(sink.closed).toBe(true); + }); + + test('IO-41: close is idempotent', async () => { + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.close(); + await sink.close(); + expect(sink.closed).toBe(true); + }); + + test('IO-42: write, flush, and emit all reject after close', async () => { + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.close(); + expect(await rejection(sink.write(queueOf(1), 1))).toBeInstanceOf( + ClosedResourceError, + ); + expect(await rejection(sink.flush())).toBeInstanceOf(ClosedResourceError); + expect(await rejection(sink.emit())).toBeInstanceOf(ClosedResourceError); + }); + + test('IO-6: closing the sink closes the caller stream it took ownership of', async () => { + const {stream, isClosed} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + await sink.close(); + expect(isClosed()).toBe(true); + }); +}); + +describe('BufferedSink host-native bridge (IO-16)', () => { + test('toWritableStream forwards written chunks', async () => { + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const bridge = sink.toWritableStream(); + const writer = bridge.getWriter(); + await writer.write(Uint8Array.from([1, 2])); + await writer.close(); + expect([...written()]).toEqual([1, 2]); + }); + + test('closing the bridge closes the sink', async () => { + const {stream} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + const writer = sink.toWritableStream().getWriter(); + await writer.close(); + expect(sink.closed).toBe(true); + }); +}); diff --git a/packages/core/src/io/buffered-sink.ts b/packages/core/src/io/buffered-sink.ts new file mode 100644 index 0000000..d389788 --- /dev/null +++ b/packages/core/src/io/buffered-sink.ts @@ -0,0 +1,165 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-sink.ts +import type {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, EndOfStreamError} from './errors.js'; +import {assertCount} from './limits.js'; +import type {Sink} from './sink.js'; +import {encodeText} from './text-codec.js'; + +/** + * A buffered byte sink over a `WritableStream<Uint8Array>` (IO-4, IO-5, IO-13, IO-18). + * + * Takes no `AbortSignal` and imposes no timeout (IO-40). Not safe for concurrent use (IO-37). + * + * @internal + */ +export class BufferedSink implements Sink { + readonly #writer: WritableStreamDefaultWriter<Uint8Array>; + #closed = false; + #closing: Promise<void> | undefined; + /** + * The most recent downstream write, settled or not. + * + * `emit()` and `flush()` await this rather than returning unconditionally. Without it neither method + * observes the writer at all: both report success on a stream that has already errored, and `flush()` + * resolves while a write started with `void sink.write(...)` is still outstanding — so IO-18's + * emit/flush distinction is unobservable in the only direction that matters. + */ + #lastWrite: Promise<void> = Promise.resolve(); + + private constructor(writer: WritableStreamDefaultWriter<Uint8Array>) { + this.#writer = writer; + } + + /** Wrap a caller-supplied stream (IO-30). */ + static overStream(stream: WritableStream<Uint8Array>): BufferedSink { + return new BufferedSink(stream.getWriter()); + } + + get closed(): boolean { + return this.#closed; + } + + /** + * Remove exactly `count` bytes from `src`'s head and push them downstream (IO-4). Fails rather than + * writing a partial amount when `src` holds fewer. + * + * `src` is drained only AFTER the downstream write resolves. Consuming first — the obvious reading of + * "remove, then push" — destroys the payload when the write fails, leaving the caller that catches the + * rejection with nothing to retry and nothing on the wire. + */ + async write(src: ByteQueue, count: number): Promise<void> { + assertCount(count); + this.#assertOpen(); + if (count === 0) return; + if (src.size < count) throw new EndOfStreamError(src.size, count); + await this.#push(src.copyOut(0, count)); + src.skip(count); + } + + /** Encode and write UTF-8 text (IO-13). */ + async writeUtf8(text: string): Promise<void> { + return this.writeString(text, 'utf-8'); + } + + /** + * Encode and write text with an explicit charset (IO-13). + * + * An empty payload writes NOTHING rather than a zero-length chunk, matching `write(src, 0)`, the tee, + * and the bridge. A zero-length chunk is not inert on the wire: to an HTTP/1.1 chunked-encoding + * transport it is the terminating chunk, so emitting one for `writeUtf8('')` can end a request body + * early. + */ + async writeString(text: string, charset: string): Promise<void> { + this.#assertOpen(); + const encoded = encodeText(text, charset); + if (encoded.length === 0) return; + await this.#push(encoded); + } + + /** + * IO-18: a full force-out toward the destination — the outstanding write must reach the destination + * AND the destination must have drained. + */ + async flush(): Promise<BufferedSink> { + this.#assertOpen(); + await this.#lastWrite; + await this.#writer.ready; + return this; + } + + /** + * IO-18: a cheap one-level handoff — hand the buffered bytes to the underlying stream and surface any + * failure, without waiting for the destination to drain. + */ + async emit(): Promise<BufferedSink> { + this.#assertOpen(); + await this.#lastWrite; + return this; + } + + /** + * IO-5, IO-41: closeable and idempotent, and the underlying resource is released at most once. + * + * Memoized rather than flag-guarded. Setting `#closed` before awaiting and early-returning on it means + * a close that FAILS is reported as a success to every later caller — `sink.closed` reads `true` for a + * destination that was never released, and the retry silently resolves. Handing every caller the same + * promise makes the failure propagate on every path (BODY-27) while still closing at most once. + */ + async close(): Promise<void> { + this.#closing ??= this.#release(async () => this.#writer.close()); + return this.#closing; + } + + /** + * IO-42: discard the destination with a reason rather than committing what was written. Shares the + * close latch, so a sink is torn down exactly once whichever path gets there first. + */ + async abort(reason?: unknown): Promise<void> { + this.#closing ??= this.#release(async () => this.#writer.abort(reason)); + return this.#closing; + } + + /** + * A writable host-native byte-stream bridge (IO-16). Closing the bridge closes the sink; ABORTING it + * aborts the sink, carrying the reason through. + */ + toWritableStream(): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write: async (chunk): Promise<void> => { + if (chunk.length === 0) return; + await this.#push(chunk); + }, + close: async (): Promise<void> => { + await this.close(); + }, + // Forwarding the reason matters: collapsing an abort into a graceful close commits a cancelled + // request body downstream as a well-formed complete one, so the peer cannot tell an aborted upload + // from a successful short one. + abort: async (reason: unknown): Promise<void> => { + await this.abort(reason); + }, + }); + } + + /** Track the in-flight write so `emit`/`flush` can observe it, without leaking an unhandled rejection. */ + async #push(payload: Uint8Array): Promise<void> { + const pending = this.#writer.write(payload); + this.#lastWrite = pending; + // A caller may start a write with `void sink.write(...)` and only learn of the failure at the next + // `emit()`/`flush()`. Marking the promise handled here keeps that from surfacing as an unhandled + // rejection first; `pending` itself still rejects for everyone awaiting it. + pending.catch(() => undefined); + await pending; + } + + async #release(teardown: () => Promise<void>): Promise<void> { + this.#closed = true; + await teardown(); + } + + /** IO-42: a stream-backed sink rejects writes, flushes, and emits after close. */ + #assertOpen(): void { + if (this.#closed) throw new ClosedResourceError('BufferedSink'); + } +} diff --git a/packages/core/src/io/buffered-source.test.ts b/packages/core/src/io/buffered-source.test.ts new file mode 100644 index 0000000..73e3d87 --- /dev/null +++ b/packages/core/src/io/buffered-source.test.ts @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-source.test.ts +// Exercises: IO-1 (read protocol), IO-2 (zero-count read), IO-3 (negative count), +// IO-11 (exhausted, single-byte read, remaining-bytes read), IO-12 (exact-count read), +// IO-15 (skip), IO-41 (idempotent close), IO-42 (stream-backed rejects after close), +// IO-6 (wrapper owns the caller's stream) +import {describe, expect, test} from 'bun:test'; +import {BufferedSource} from './buffered-source.js'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, EndOfStreamError} from './errors.js'; +import {END_OF_STREAM} from './limits.js'; +import {fakeReadableStream} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const bytes = (...values: number[]): Uint8Array => Uint8Array.from(values); + +const sourceOver = (...chunks: Uint8Array[]): BufferedSource => + BufferedSource.overStream(fakeReadableStream(chunks)); + +describe('BufferedSource core reads', () => { + test('IO-1: read appends to the destination tail and returns the transferred count', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const dest = new ByteQueue(); + dest.writeBytes(bytes(9)); + expect(await source.read(dest, 2)).toBe(2); + expect([...dest.snapshot()]).toEqual([9, 1, 2]); + }); + + test('IO-1: read returns END_OF_STREAM once exhausted', async () => { + const source = sourceOver(bytes(1)); + const dest = new ByteQueue(); + expect(await source.read(dest, 4)).toBe(1); + expect(await source.read(dest, 4)).toBe(END_OF_STREAM); + }); + + test('IO-2: a zero-count read returns 0 on a fresh source', async () => { + const source = sourceOver(bytes(1)); + expect(await source.read(new ByteQueue(), 0)).toBe(0); + }); + + test('IO-2: a zero-count read returns 0 — not END_OF_STREAM — on an exhausted source', async () => { + const source = sourceOver(); + expect(await source.read(new ByteQueue(), 4)).toBe(END_OF_STREAM); + expect(await source.read(new ByteQueue(), 0)).toBe(0); + }); + + test('IO-3: a negative count is rejected before any I/O', async () => { + const source = sourceOver(bytes(1, 2)); + expect( + (await rejection(source.read(new ByteQueue(), -1))).message, + ).toContain('count must be a non-negative integer, got -1'); + }); + + test('IO-11: exhausted() is false while bytes remain and true once they do not', async () => { + const source = sourceOver(bytes(1)); + expect(await source.exhausted()).toBe(false); + await source.readBytes(); + expect(await source.exhausted()).toBe(true); + }); + + test('IO-11: readByte returns the next byte, then fails at end', async () => { + const source = sourceOver(bytes(7)); + expect(await source.readByte()).toBe(7); + expect(await rejection(source.readByte())).toBeInstanceOf(EndOfStreamError); + }); + + test('IO-11: readBytes returns all remaining bytes, and empty when already exhausted', async () => { + const source = sourceOver(bytes(1, 2), bytes(3)); + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + expect([...(await source.readBytes())]).toEqual([]); + }); + + test('IO-12: readExactly returns exactly the requested count across chunk boundaries', async () => { + const source = sourceOver(bytes(1), bytes(2, 3), bytes(4)); + expect([...(await source.readExactly(3))]).toEqual([1, 2, 3]); + }); + + test('IO-12: readExactly fails rather than returning a short result', async () => { + const source = sourceOver(bytes(1, 2)); + expect(await rejection(source.readExactly(3))).toBeInstanceOf( + EndOfStreamError, + ); + }); +}); + +describe('BufferedSource skip and lifecycle (IO-15, IO-41, IO-42, IO-6)', () => { + test('IO-15: skip advances past exactly the requested count', async () => { + const source = sourceOver(bytes(1, 2, 3, 4)); + await source.skip(2); + expect([...(await source.readBytes())]).toEqual([3, 4]); + }); + + test('IO-15: skip fails when fewer bytes remain', async () => { + const source = sourceOver(bytes(1, 2)); + expect(await rejection(source.skip(3))).toBeInstanceOf(EndOfStreamError); + }); + + test('IO-15: skip(0) is a no-op, even at and after end of stream', async () => { + const source = sourceOver(bytes(1)); + await source.skip(0); + await source.readBytes(); + await source.skip(0); + expect(await source.exhausted()).toBe(true); + }); + + test('IO-41: close is idempotent', async () => { + const source = sourceOver(bytes(1)); + await source.close(); + await source.close(); + expect(source.closed).toBe(true); + }); + + test('IO-42: a stream-backed source REJECTS reads after close', async () => { + // The opposite direction from ByteQueue, which stays readable. IO-42 names both as the + // inconsistency porters get wrong; both directions are asserted, here and in Task 4. + const source = sourceOver(bytes(1, 2)); + await source.close(); + expect(await rejection(source.read(new ByteQueue(), 1))).toBeInstanceOf( + ClosedResourceError, + ); + expect(await rejection(source.readBytes())).toBeInstanceOf( + ClosedResourceError, + ); + }); + + test('overBytes wraps a byte array as an independent copy', async () => { + const input = bytes(1, 2, 3); + const source = BufferedSource.overBytes(input); + input[0] = 99; + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('IO-6: closing the source cancels the caller stream it took ownership of', async () => { + let cancelled = false; + const source = BufferedSource.overStream( + fakeReadableStream([bytes(1)], () => { + cancelled = true; + }), + ); + await source.close(); + expect(cancelled).toBe(true); + }); +}); + +describe('BufferedSource host-native bridge (IO-16)', () => { + test('toReadableStream yields the remaining bytes', async () => { + const source = sourceOver(bytes(1, 2), bytes(3)); + const collected: number[] = []; + for await (const chunk of source.toReadableStream()) + collected.push(...chunk); + expect(collected).toEqual([1, 2, 3]); + }); + + test('closing the bridge closes the owning source', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const stream = source.toReadableStream(); + await stream.cancel(); + expect(source.closed).toBe(true); + }); +}); diff --git a/packages/core/src/io/buffered-source.text.test.ts b/packages/core/src/io/buffered-source.text.test.ts new file mode 100644 index 0000000..ed558c1 --- /dev/null +++ b/packages/core/src/io/buffered-source.text.test.ts @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-source.text.test.ts +// Exercises: IO-13 (UTF-8 and explicit-charset decode), IO-14 (line reads: \n and \r\n terminators, +// lone \r stays content, final unterminated line returned as-is, undefined when exhausted first) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSource} from './buffered-source.js'; +import {fakeReadableStream} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const utf8 = (text: string): Uint8Array => new TextEncoder().encode(text); + +const sourceOver = (...chunks: Uint8Array[]): BufferedSource => + BufferedSource.overStream(fakeReadableStream(chunks)); + +/** Split `bytes` at the given cut points, so a terminator can straddle a chunk boundary. */ +function chunkAt(bytes: Uint8Array, cuts: readonly number[]): Uint8Array[] { + const bounded = [ + ...new Set(cuts.filter(c => c > 0 && c < bytes.length)), + ].sort((a, b) => a - b); + const out: Uint8Array[] = []; + let previous = 0; + for (const cut of bounded) { + out.push(bytes.subarray(previous, cut)); + previous = cut; + } + // An empty trailing subarray (only possible when `bytes` itself is empty) would enqueue a zero-length + // chunk, which RetentionWindow correctly rejects as an IO-17 protocol violation — a stream signals + // end-of-stream via `done`, never via a 0-byte delivery. Omitting it here keeps the fixture itself + // protocol-clean. + const last = bytes.subarray(previous); + if (last.length > 0) out.push(last); + return out; +} + +describe('BufferedSource text reads (IO-13)', () => { + test('readUtf8 decodes non-ASCII text', async () => { + expect(await sourceOver(utf8('héllo ☃')).readUtf8()).toBe('héllo ☃'); + }); + + test('readUtf8 decodes across a chunk boundary that splits a multi-byte character', async () => { + const encoded = utf8('☃'); + const source = sourceOver(encoded.subarray(0, 1), encoded.subarray(1)); + expect(await source.readUtf8()).toBe('☃'); + }); + + test('readString decodes an explicit non-UTF-8 charset', async () => { + // 0xE9 is é in ISO-8859-1 and invalid alone in UTF-8 — so this only passes if the charset is honored. + const source = sourceOver(Uint8Array.from([0x68, 0xe9])); + expect(await source.readString('iso-8859-1')).toBe('hé'); + }); + + test('readString rejects an unknown charset label', async () => { + expect( + (await rejection(sourceOver(utf8('x')).readString('not-a-charset'))) + .message, + ).toContain('unsupported charset: not-a-charset'); + }); +}); + +describe('BufferedSource line reads (IO-14)', () => { + test('splits on \\n and consumes the terminator', async () => { + const source = sourceOver(utf8('one\ntwo\n')); + expect(await source.readUtf8Line()).toBe('one'); + expect(await source.readUtf8Line()).toBe('two'); + expect(await source.readUtf8Line()).toBeUndefined(); + }); + + test('treats \\r\\n as a terminator and strips both bytes', async () => { + const source = sourceOver(utf8('one\r\ntwo\r\n')); + expect(await source.readUtf8Line()).toBe('one'); + expect(await source.readUtf8Line()).toBe('two'); + }); + + test('keeps a lone \\r not followed by \\n as line content', async () => { + const source = sourceOver(utf8('a\rb\n')); + expect(await source.readUtf8Line()).toBe('a\rb'); + }); + + test('returns a final unterminated line as-is', async () => { + const source = sourceOver(utf8('one\ntwo')); + expect(await source.readUtf8Line()).toBe('one'); + expect(await source.readUtf8Line()).toBe('two'); + expect(await source.readUtf8Line()).toBeUndefined(); + }); + + test('returns undefined when exhausted before any byte', async () => { + expect(await sourceOver().readUtf8Line()).toBeUndefined(); + }); + + test('returns an empty string for an empty line', async () => { + const source = sourceOver(utf8('\nx\n')); + expect(await source.readUtf8Line()).toBe(''); + expect(await source.readUtf8Line()).toBe('x'); + }); + + test('property: lines round-trip across adversarial chunk boundaries', async () => { + // IO-14's rationale calls out surviving slice-window boundaries; hand-picked examples miss the case + // where \r and \n land in different chunks, so the cut points are generated. + await fc.assert( + fc.asyncProperty( + fc.array(fc.stringMatching(/^[a-z \r]*$/), {maxLength: 8}), + fc.constantFrom('\n', '\r\n'), + fc.array(fc.integer({min: 0, max: 64}), {maxLength: 8}), + async (lines, terminator, cuts) => { + const encoded = utf8( + lines.map(line => `${line}${terminator}`).join(''), + ); + const source = BufferedSource.overStream( + fakeReadableStream(chunkAt(encoded, cuts)), + ); + + const read: string[] = []; + for (;;) { + const line = await source.readUtf8Line(); + if (line === undefined) break; + read.push(line); + } + // A line-content trailing \r merges with an appended \n into \r\n and is stripped by the + // reader; with a \r\n terminator only the terminator's own \r is stripped, so a content \r + // survives. The oracle mirrors exactly that rule. + const expected = lines.map(line => + terminator === '\n' ? line.replace(/\r$/, '') : line, + ); + expect(read).toEqual(expected); + }, + ), + ); + }); +}); + +describe('BufferedSource text decoding fidelity (IO-13, IO-14)', () => { + test('a BOM is preserved on every line, not silently deleted', async () => { + // A fresh TextDecoder per fragment with the default `ignoreBOM: false` strips U+FEFF wherever a + // fragment happens to begin. SSE-12 requires a mid-stream BOM to survive as ordinary data, so + // losing it here would make that requirement unimplementable in Phase 6b — the byte is gone before + // the SSE parser ever sees the line. + const source = BufferedSource.overBytes(utf8('a\n\ufeffb\n\ufeffc')); + const lines: (string | undefined)[] = []; + for (;;) { + const line = await source.readUtf8Line(); + if (line === undefined) break; + lines.push(line); + } + expect(lines).toEqual(['a', '\ufeffb', '\ufeffc']); + }); + + test('a leading BOM survives a whole-body read', async () => { + // Dropping it would silently remove a body's first three bytes, breaking content hashing, + // signature verification and exact-length assertions. + const source = BufferedSource.overBytes(utf8('\ufeffpayload')); + const text = await source.readUtf8(); + expect(text).toBe('\ufeffpayload'); + expect(text.length).toBe(8); + }); + + test('a leading BOM survives a counted read', async () => { + const source = BufferedSource.overBytes(utf8('\ufeffab')); + expect(await source.readUtf8(5)).toBe('\ufeffab'); + }); + + test('an unusable charset is refused before any byte is consumed', async () => { + const source = BufferedSource.overBytes(utf8('hello')); + expect( + (await rejection(source.readString('no-such-charset'))).message, + ).toContain('unsupported charset'); + expect(await source.readUtf8()).toBe('hello'); + }); + + test('readUtf8Line stays linear in the length of the line', async () => { + // Re-peeking the whole scanned prefix on every pulled chunk makes this quadratic in bytes copied + // with no line-length bound — and this is the primitive header and chunked-encoding parsing run + // over attacker-controlled bytes, so a peer dribbling a long newline-free line pins a CPU core. + const measure = async (length: number): Promise<number> => { + const source = BufferedSource.overStream(dribbledLine(length)); + const started = performance.now(); + await source.readUtf8Line(); + return performance.now() - started; + }; + await measure(2000); // warm the JIT so the ratio reflects the algorithm, not compilation + const small = await measure(4000); + const large = await measure(16000); + // Quadratic would be ~16x for 4x the input. Linear is ~4x; the ceiling is loose so the test does + // not go flaky on a noisy machine, but it is far under what a quadratic scan would produce. + expect(large).toBeLessThan(Math.max(small, 1) * 10); + }); + + /** One byte per chunk, so every byte forces another scan pass. */ + function dribbledLine(length: number): ReadableStream<Uint8Array> { + let at = 0; + return new ReadableStream<Uint8Array>({ + pull(controller): void { + if (at < length) { + controller.enqueue(Uint8Array.from([0x61])); + at += 1; + return; + } + controller.enqueue(Uint8Array.from([0x0a])); + controller.close(); + }, + }); + } +}); diff --git a/packages/core/src/io/buffered-source.ts b/packages/core/src/io/buffered-source.ts new file mode 100644 index 0000000..4545a95 --- /dev/null +++ b/packages/core/src/io/buffered-source.ts @@ -0,0 +1,348 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-source.ts +import {invariant} from '../invariant.js'; +import {ByteQueue, copyBytes} from './byte-queue.js'; +import {ClosedResourceError, EndOfStreamError} from './errors.js'; +import {assertAllocatable, assertCount, END_OF_STREAM} from './limits.js'; +import {RetentionWindow, type Cursor} from './retention-window.js'; +import {assertDecodable, decodeText} from './text-codec.js'; + +/** + * A buffered, non-blocking byte source over a `ReadableStream<Uint8Array>` (IO-11–IO-24). + * + * Peek and slice views are instances of this same class over the same `RetentionWindow`, differing only in + * their cursor, their byte budget, and whether they own the window. A second class would need either + * inheritance — which styleguide 6.4 reserves for `Error` hierarchies — or ten duplicated delegating + * methods. + * + * Takes no `AbortSignal` and imposes no timeout: IO-40 assigns deadlines and prompt cancellation of + * blocked I/O to the transport that owns the real socket. Not safe for concurrent use (IO-37). + * + * @internal + */ +export class BufferedSource { + readonly #window: RetentionWindow; + readonly #cursor: Cursor; + readonly #ownsWindow: boolean; + readonly #limit: number; + readonly #startedAt: number; + #closed = false; + + // eslint-disable-next-line max-params -- private, view-internal plumbing; peek()/slice() are the public entry points (0-2 params each) + private constructor( + window: RetentionWindow, + cursor: Cursor, + ownsWindow: boolean, + limit: number, + ) { + this.#window = window; + this.#cursor = cursor; + this.#ownsWindow = ownsWindow; + this.#limit = limit; + this.#startedAt = cursor.at; + } + + /** Wrap a caller-supplied stream (IO-30). */ + static overStream(stream: ReadableStream<Uint8Array>): BufferedSource { + const window = new RetentionWindow(stream.getReader()); + return new BufferedSource( + window, + window.register(0), + true, + Number.POSITIVE_INFINITY, + ); + } + + /** Wrap a byte array as an independent copy (IO-30). */ + static overBytes(bytes: Uint8Array): BufferedSource { + const copy = copyBytes(bytes); + return BufferedSource.overStream( + new ReadableStream<Uint8Array>({ + start(controller): void { + if (copy.length > 0) controller.enqueue(copy); + controller.close(); + }, + }), + ); + } + + /** + * Whether this source can still be read. + * + * Must consider the window, not just this instance's own flag: a peek/slice view is invalidated when + * its parent closes the window (IO-22) without anything touching the view's flag, so reading `#closed` + * alone reports an unusable view as open — making the natural guard `if (!view.closed) …` take the + * throwing branch every time, which is the opposite of what exposing the flag is for. + */ + get closed(): boolean { + return this.#closed || this.#window.closed; + } + + /** Read up to `count` bytes onto `dest`'s tail (IO-1, IO-2, IO-3). */ + async read(dest: ByteQueue, count: number): Promise<number> { + assertCount(count); + this.#assertOpen(); + // IO-2 before any exhaustion determination — a zero-count read is 0, never END_OF_STREAM. + if (count === 0) return 0; + const want = Math.min(count, this.#remainingBudget()); + if (want <= 0) return END_OF_STREAM; + const available = await this.#window.pullThrough(this.#cursor.at + 1); + if (!available) return END_OF_STREAM; + return this.#window.readInto(this.#cursor, dest, want); + } + + /** True exactly when no more bytes are available (IO-11). */ + async exhausted(): Promise<boolean> { + this.#assertOpen(); + if (this.#remainingBudget() <= 0) return true; + return !(await this.#window.pullThrough(this.#cursor.at + 1)); + } + + /** The next byte, or a failure at end of stream (IO-11). */ + async readByte(): Promise<number> { + const [value] = await this.readExactly(1); + invariant(value !== undefined, 'readExactly(1) returned an empty array'); + return value; + } + + /** Every remaining byte; empty when already exhausted (IO-11). */ + async readBytes(): Promise<Uint8Array> { + this.#assertOpen(); + const staging = new ByteQueue(); + while ((await this.read(staging, READ_CHUNK)) !== END_OF_STREAM) { + // IO-9: check as we go, not at the `snapshot()` at the end. A count-less read cannot know the + // total up front, but deferring the check until materialization means a multi-gigabyte body is + // fully buffered first — so the process is far likelier to die of a low-level allocation failure + // than to reach the actionable refusal IO-9 exists to produce. + assertAllocatable(staging.size); + } + return staging.snapshot(); + } + + /** Exactly `count` bytes, or a failure — never a short result (IO-12). */ + async readExactly(count: number): Promise<Uint8Array> { + assertCount(count); + this.#assertOpen(); + // IO-9: refuse eagerly with an actionable error. Routing this through ByteQueue would raise + // EndOfStreamError instead, since takeBytes checks its size before it ever tries to allocate. + assertAllocatable(count); + const staging = new ByteQueue(); + while (staging.size < count) { + const read = await this.read(staging, count - staging.size); + if (read === END_OF_STREAM) + throw new EndOfStreamError(staging.size, count); + } + return staging.takeBytes(count); + } + + /** Decode `count` bytes (or every remaining byte) as UTF-8 (IO-13). */ + async readUtf8(count?: number): Promise<string> { + return this.readString('utf-8', count); + } + + /** Decode `count` bytes (or every remaining byte) with an explicit charset (IO-13). */ + async readString(charset: string, count?: number): Promise<string> { + this.#assertOpen(); + // Reject an unusable label BEFORE consuming bytes, so a bad charset does not also destroy the body. + assertDecodable(charset); + const raw = + count === undefined + ? await this.readBytes() + : await this.readExactly(count); + return decodeText(raw, charset); + } + + /** + * The next line as UTF-8, with its terminator consumed (IO-14). + * + * Both `\n` and `\r\n` terminate. A lone `\r` not followed by `\n` stays line content, which falls out + * of scanning only for `\n`. Returns the final unterminated line as-is, and `undefined` when the source + * is exhausted before any byte — `undefined` rather than the spec's language-agnostic "null", per + * styleguide 3.5. + * + * Scans with a NON-CONSUMING peek before reading, deliberately. Reading first and pushing back the + * over-read cannot work: every read advances this cursor and `RetentionWindow.readInto` then trims the + * queue head to the slowest cursor, so the bytes past the terminator are already discarded by the time + * anything could rewind over them. Peeking leaves the cursor still, so the bytes stay retained, and the + * subsequent `readExactly` consumes exactly the line plus its terminator. + */ + async readUtf8Line(): Promise<string | undefined> { + this.#assertOpen(); + const at = await this.#scanForNewline(); + if (at === END_OF_STREAM) { + const rest = await this.readBytes(); + return rest.length === 0 ? undefined : decodeText(rest, 'utf-8'); + } + const line = await this.readExactly(at + 1); + const end = at > 0 && line[at - 1] === CARRIAGE_RETURN ? at - 1 : at; + // Decoding goes through `decodeText`, which sets `ignoreBOM`. A per-line decoder with the default + // would strip U+FEFF from the front of EVERY line, not just the stream's first — see the note on + // `decodeText`, and SSE-12, which requires a mid-stream BOM to survive as ordinary data. + return decodeText(line.subarray(0, end), 'utf-8'); + } + + /** + * Offset of the next `\n` relative to this cursor, or `END_OF_STREAM` if the source ends first. + * Never advances the cursor. Retention grows by one line's length, which is what IO-14 requires and + * all it requires. + */ + async #scanForNewline(): Promise<number> { + let searched = 0; + for (;;) { + const available = Math.min( + this.#window.availableFrom(this.#cursor), + this.#remainingBudget(), + ); + if (available > searched) { + // Peek ONLY the bytes pulled since the last pass. Re-peeking the whole scanned prefix each time + // makes this quadratic in bytes copied, with no line-length bound — and this is the primitive + // header and chunked-encoding parsing run over attacker-controlled bytes, so a peer that + // dribbles a long newline-free line would pin a CPU core. + const tail = this.#window.peekBytes( + this.#cursor, + searched, + available - searched, + ); + const found = tail.indexOf(NEWLINE); + if (found >= 0) return searched + found; + searched = available; + } + if (searched >= this.#remainingBudget()) return END_OF_STREAM; + if (!(await this.#window.pullThrough(this.#cursor.at + searched + 1))) + return END_OF_STREAM; + } + } + + /** + * A non-consuming view over the whole remaining source (IO-19). Reads from it never advance this + * source's cursor. + * + * Deliberately uncapped: §5 bounds nothing, and every buffering cap the product spec mandates lives in + * §6 (Phase 3b). See `RetentionWindow` for why a cap here would partially fail IO-19. + */ + peek(): BufferedSource { + this.#assertOpen(); + return new BufferedSource( + this.#window, + this.#window.register(this.#cursor.at), + false, + this.#remainingBudget(), + ); + } + + /** + * A non-consuming, length-bounded view exposing at most `count` bytes starting `offset` ahead of this + * cursor (IO-20). + * + * Offset overflow is detected LAZILY — an offset past the source size constructs fine and surfaces as + * an empty read (IO-21) — because callers may slice speculatively before the body length is known. A + * negative offset or count is rejected eagerly. A slice of a slice composes additively and caps at the + * outer slice's remaining budget (IO-23). + */ + slice(offset: number, count: number): BufferedSource { + invariant( + Number.isInteger(offset) && offset >= 0, + `offset must be a non-negative integer, got ${String(offset)}`, + ); + assertCount(count); + this.#assertOpen(); + const budget = Math.max( + 0, + Math.min(count, this.#remainingBudget() - offset), + ); + return new BufferedSource( + this.#window, + this.#window.register(this.#cursor.at + offset), + false, + budget, + ); + } + + /** Advance past exactly `count` bytes; `skip(0)` is a no-op even at end of stream (IO-15). */ + async skip(count: number): Promise<void> { + assertCount(count); + this.#assertOpen(); + if (count === 0) return; + const staging = new ByteQueue(); + let skipped = 0; + while (skipped < count) { + const read = await this.read(staging, count - skipped); + if (read === END_OF_STREAM) throw new EndOfStreamError(skipped, count); + skipped += read; + staging.clear(); + } + } + + /** + * IO-41: idempotent. A view releases only its own cursor and never closes its parent or moves the + * parent's cursor (IO-22); the owning source closes the window, which invalidates every outstanding + * view. + */ + async close(): Promise<void> { + if (this.#closed) return; + this.#closed = true; + if (!this.#ownsWindow) { + this.#window.release(this.#cursor); + return; + } + // Awaited, so the promise settles only once the underlying reader is really cancelled and its lock + // released — and rejects if that teardown fails, rather than reporting a success that never happened. + await this.#window.close(); + } + + /** IO-42: a stream-backed source rejects reads after close, unlike an in-memory `ByteQueue`. */ + #assertOpen(): void { + if (this.#closed) throw new ClosedResourceError('BufferedSource'); + this.#window.assertUsable(); + } + + /** + * A read-only host-native byte-stream bridge (IO-16). Closing the bridge closes the owning source. + * + * For this port the host-native byte stream IS `ReadableStream` — that is `sdk-design/03` §3.1's whole + * premise, and it keeps core free of any `node:` import. A consumer wanting a Node `Readable` calls + * `Readable.fromWeb()` at their own edge. + */ + toReadableStream(): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + pull: async (controller): Promise<void> => { + const staging = new ByteQueue(); + let read: number; + try { + read = await this.read(staging, BRIDGE_CHUNK); + } catch (e: unknown) { + // A mid-stream read failure errors the bridge, and `cancel` is NOT invoked on an errored + // stream — so without this the reader lock and the retention window are both stranded on + // exactly the failure path that matters most for a connection-backed source. + await this.close().catch(() => undefined); + throw e; + } + if (read === END_OF_STREAM) { + // Close the BRIDGE only. IO-16 requires that closing the bridge close the owning source, and + // `cancel` below does that; auto-closing at natural EOF is an extra step that would tear down + // the whole RetentionWindow and invalidate every outstanding peek/slice view — defeating + // IO-19's stated purpose (previews, replay) for its most natural usage: take a preview, hand + // the bridge to the transport, read the preview afterwards. + controller.close(); + return; + } + controller.enqueue(staging.snapshot()); + }, + cancel: async (): Promise<void> => { + await this.close(); + }, + }); + } + + #remainingBudget(): number { + if (this.#limit === Number.POSITIVE_INFINITY) + return Number.POSITIVE_INFINITY; + return Math.max(0, this.#limit - (this.#cursor.at - this.#startedAt)); + } +} + +/** How much a bulk drain asks for per iteration. Not a retention bound — `read` transfers, never buffers. */ +const READ_CHUNK = 16 * 1024; +const BRIDGE_CHUNK = 16 * 1024; +const NEWLINE = 0x0a; +const CARRIAGE_RETURN = 0x0d; diff --git a/packages/core/src/io/buffered-source.views.test.ts b/packages/core/src/io/buffered-source.views.test.ts new file mode 100644 index 0000000..5886a78 --- /dev/null +++ b/packages/core/src/io/buffered-source.views.test.ts @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/buffered-source.views.test.ts +// Exercises: IO-19 (peek is non-consuming over the whole remaining source), IO-20 (bounded slice), +// IO-21 (lazy offset overflow, eager negative rejection), IO-22 (closing a slice does not close the +// parent; closing the parent invalidates slices), IO-23 (independence, additive composition), +// IO-24 (reading a closed slice is a state error, distinct from EOF), +// IO-16 (the readable bridge: cancel closes the source, EOF does not, a failed read does) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSource} from './buffered-source.js'; +import {ClosedResourceError} from './errors.js'; +import {drainStream, fakeReadableStream} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const bytes = (...values: number[]): Uint8Array => Uint8Array.from(values); + +const sourceOver = (...chunks: Uint8Array[]): BufferedSource => + BufferedSource.overStream(fakeReadableStream(chunks)); + +describe('BufferedSource views', () => { + test('IO-19: reads from a peek do not advance the original cursor', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const peek = source.peek(); + expect([...(await peek.readBytes())]).toEqual([1, 2, 3]); + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('IO-20: a slice exposes at most count bytes starting offset ahead', async () => { + const source = sourceOver(bytes(1, 2, 3, 4, 5)); + const slice = source.slice(1, 3); + expect([...(await slice.readBytes())]).toEqual([2, 3, 4]); + }); + + test('IO-20: reading past the window behaves as end-of-window, and never advances the parent', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const slice = source.slice(0, 2); + expect([...(await slice.readBytes())]).toEqual([1, 2]); + expect([...(await slice.readBytes())]).toEqual([]); + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('IO-21: an offset past the source size succeeds at construction and reads as empty', async () => { + const source = sourceOver(bytes(1, 2)); + const slice = source.slice(100, 4); + expect([...(await slice.readBytes())]).toEqual([]); + }); + + test('IO-21: a negative offset or count is rejected eagerly at construction', () => { + const source = sourceOver(bytes(1, 2)); + expect(() => source.slice(-1, 2)).toThrow( + 'offset must be a non-negative integer, got -1', + ); + expect(() => source.slice(0, -2)).toThrow( + 'count must be a non-negative integer, got -2', + ); + }); +}); + +describe('BufferedSource view lifecycle and independence (IO-22, IO-23, IO-24)', () => { + test('IO-22: closing a slice neither closes the parent nor advances its cursor', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const slice = source.slice(0, 2); + await slice.readBytes(); + await slice.close(); + expect(source.closed).toBe(false); + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('IO-22: closing the parent invalidates outstanding slices', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const slice = source.slice(0, 2); + await source.close(); + expect(await rejection(slice.readBytes())).toBeInstanceOf( + ClosedResourceError, + ); + }); + + test('IO-24: reading an explicitly closed slice fails loudly, distinct from a normal EOF', async () => { + const source = sourceOver(bytes(1, 2, 3)); + const slice = source.slice(0, 2); + await slice.close(); + expect(await rejection(slice.readBytes())).toBeInstanceOf( + ClosedResourceError, + ); + }); + + test('IO-23: two slices of one source have independent cursors and budgets', async () => { + const source = sourceOver(bytes(1, 2, 3, 4)); + const first = source.slice(0, 2); + const second = source.slice(2, 2); + expect([...(await second.readBytes())]).toEqual([3, 4]); + expect([...(await first.readBytes())]).toEqual([1, 2]); + }); + + test('IO-23: a slice of a slice composes offsets additively and caps at the outer remainder', async () => { + const source = sourceOver(bytes(1, 2, 3, 4, 5, 6)); + const outer = source.slice(1, 4); // 2,3,4,5 + const inner = outer.slice(1, 10); // starts at 3, capped to 3 bytes: 3,4,5 + expect([...(await inner.readBytes())]).toEqual([3, 4, 5]); + }); +}); + +describe('BufferedSource view properties', () => { + test('property: an arbitrary slice reads exactly the bytes at its window', async () => { + await fc.assert( + fc.asyncProperty( + fc.uint8Array({minLength: 1, maxLength: 64}), + fc.integer({min: 0, max: 64}), + fc.integer({min: 0, max: 64}), + async (data, offset, count) => { + const source = BufferedSource.overStream(fakeReadableStream([data])); + const slice = source.slice(offset, count); + const expected = [...data.subarray(offset, offset + count)]; + expect([...(await slice.readBytes())]).toEqual(expected); + }, + ), + ); + }); + + test('property: no view read advances any other view or the parent', async () => { + await fc.assert( + fc.asyncProperty( + fc.uint8Array({minLength: 1, maxLength: 32}), + async data => { + const source = BufferedSource.overStream(fakeReadableStream([data])); + const first = source.peek(); + const second = source.peek(); + await first.readBytes(); + expect([...(await second.readBytes())]).toEqual([...data]); + expect([...(await source.readBytes())]).toEqual([...data]); + }, + ), + ); + }); +}); + +describe('BufferedSource closed reporting and bridge lifecycle (IO-16, IO-19, IO-22)', () => { + const sourceOf = (...values: number[]): BufferedSource => + BufferedSource.overBytes(bytes(...values)); + + test('IO-22: a view invalidated by its parent reports itself closed', async () => { + // Reading only the view's own flag makes the natural guard `if (!view.closed) await view.read()` + // take the throwing branch every time, which defeats the point of exposing the flag. + const source = sourceOf(1, 2, 3); + const view = source.peek(); + await source.close(); + expect(source.closed).toBe(true); + expect(view.closed).toBe(true); + }); + + test('IO-19: draining the bridge to EOF leaves outstanding previews readable', async () => { + // IO-16 requires that closing the BRIDGE close the source; auto-closing at natural EOF is an extra + // step that tears down the whole window and invalidates every peek — defeating IO-19's stated + // rationale (previews, replay) for its most natural usage. + const source = sourceOf(1, 2, 3); + const preview = source.peek(); + await drainStream(source.toReadableStream()); + expect([...(await preview.readBytes())]).toEqual([1, 2, 3]); + expect(source.closed).toBe(false); + await source.close(); + }); + + test('IO-16: cancelling the bridge closes the owning source', async () => { + const source = sourceOf(1, 2, 3); + const bridge = source.toReadableStream(); + await bridge.cancel(); + expect(source.closed).toBe(true); + }); + + test('IO-16: a mid-stream read failure closes the source instead of stranding it', async () => { + // `cancel` is NOT invoked on an errored stream, so without an explicit close here the reader lock + // and the retention window are both stranded on the failure path that matters most for a + // connection-backed source. + const stream = new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(bytes(1, 2, 3)); + }, + pull(): never { + throw new Error('mid-stream read failure'); + }, + }); + const source = BufferedSource.overStream(stream); + expect(stream.locked).toBe(true); + expect( + (await rejection(drainStream(source.toReadableStream()))).message, + ).toContain('mid-stream read failure'); + expect(source.closed).toBe(true); + // The lock is what actually leaks. The underlying `cancel` callback is deliberately NOT asserted: + // per the Streams spec, cancelling an ALREADY-ERRORED stream rejects with the stored error without + // ever invoking the underlying source's cancel, so there is nothing left to observe there. + expect(stream.locked).toBe(false); + }); +}); diff --git a/packages/core/src/io/byte-queue.bench.ts b/packages/core/src/io/byte-queue.bench.ts new file mode 100644 index 0000000..5519f36 --- /dev/null +++ b/packages/core/src/io/byte-queue.bench.ts @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/byte-queue.bench.ts +// Baseline only — no optimization has been applied and none is justified yet (styleguide 15.1, 15.6: +// do not tune ahead of a profile). This exists so Phases 6 and 8 inherit a regression floor on the +// SDK's hottest data structure. mitata measures a warm JIT in isolation, not end-to-end throughput. +import {bench, run} from 'mitata'; +import {ByteQueue} from './byte-queue.js'; + +const SMALL = new Uint8Array(64).fill(1); +const LARGE = new Uint8Array(64 * 1024).fill(1); + +bench( + 'ByteQueue writeBytes x1000 small chunks (warm-JIT, not end-to-end)', + () => { + const queue = new ByteQueue(); + for (let i = 0; i < 1000; i += 1) queue.writeBytes(SMALL); + }, +); + +// A pool of pre-filled queues, so the two benches below measure only the operation they are named +// after. `writeBytes` COPIES, so a 64 KiB fill inside the timed closure costs about as much as the read +// it is setting up — roughly halving the sensitivity of the regression floor Phases 6 and 8 diff +// against. mitata has no per-iteration setup hook, so the fill is hoisted and the pool re-primed in +// batches instead. Only the first bench measures `writeBytes`, deliberately. +const POOL_SIZE = 256; + +function primedPool(): ByteQueue[] { + return Array.from({length: POOL_SIZE}, () => { + const queue = new ByteQueue(); + queue.writeBytes(LARGE); + return queue; + }); +} + +let readPool = primedPool(); +let readAt = 0; +const sinkQueue = new ByteQueue(); + +bench('ByteQueue read of 64 KiB, pre-filled (warm-JIT, not end-to-end)', () => { + if (readAt >= POOL_SIZE) { + readPool = primedPool(); + readAt = 0; + } + const source = readPool[readAt]; + readAt += 1; + if (source === undefined) return; + sinkQueue.clear(); + source.read(sinkQueue, source.size); +}); + +const snapshotQueue = new ByteQueue(); +snapshotQueue.writeBytes(LARGE); + +bench( + 'ByteQueue snapshot of 64 KiB, pre-filled (warm-JIT, not end-to-end)', + () => { + snapshotQueue.snapshot(); + }, +); + +await run(); diff --git a/packages/core/src/io/byte-queue.property.test.ts b/packages/core/src/io/byte-queue.property.test.ts new file mode 100644 index 0000000..d6ea9e4 --- /dev/null +++ b/packages/core/src/io/byte-queue.property.test.ts @@ -0,0 +1,77 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/byte-queue.property.test.ts +// Exercises: IO-7 (FIFO order across arbitrary chunk splits), IO-8 (snapshot independence), +// IO-10 (copyTo is non-consuming) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {ByteQueue} from './byte-queue.js'; + +const chunks = fc.array(fc.uint8Array({maxLength: 32}), {maxLength: 16}); + +describe('ByteQueue properties', () => { + test('IO-7: writing arbitrary chunks then reading back preserves byte order exactly', () => { + fc.assert( + fc.property(chunks, input => { + const queue = new ByteQueue(); + for (const chunk of input) queue.writeBytes(chunk); + const expected = input.flatMap(chunk => [...chunk]); + expect(queue.size).toBe(expected.length); + expect([...queue.snapshot()]).toEqual(expected); + }), + ); + }); + + test('IO-7: reading in arbitrary increments yields the same bytes as reading all at once', () => { + fc.assert( + fc.property( + chunks, + fc.array(fc.integer({min: 0, max: 8}), {maxLength: 32}), + (input, steps) => { + const source = new ByteQueue(); + for (const chunk of input) source.writeBytes(chunk); + const expected = input.flatMap(chunk => [...chunk]); + + const dest = new ByteQueue(); + for (const step of steps) source.read(dest, step); + source.read(dest, source.size); + + expect([...dest.snapshot()]).toEqual(expected); + }, + ), + ); + }); + + test('IO-8: a snapshot is every written byte, leaves size alone, and survives later writes', () => { + fc.assert( + fc.property(chunks, fc.uint8Array({maxLength: 16}), (input, later) => { + const queue = new ByteQueue(); + for (const chunk of input) queue.writeBytes(chunk); + const before = queue.snapshot(); + const sizeBefore = queue.size; + + // Comparing `before` against a copy of ITSELF is the trap here: both sides derive from the same + // array, so the assertion holds for any implementation — a `snapshot()` returning an empty array + // passes it. Pin the CONTENT against the input instead, so the property can actually fail. + expect([...before]).toEqual(input.flatMap(chunk => [...chunk])); + expect(queue.size).toBe(sizeBefore); + + queue.writeBytes(later); + expect([...before]).toEqual(input.flatMap(chunk => [...chunk])); + expect(queue.size).toBe(sizeBefore + later.length); + }), + ); + }); + + test('IO-10: copyTo never changes the source size', () => { + fc.assert( + fc.property(chunks, fc.integer({min: 0, max: 16}), (input, offset) => { + const source = new ByteQueue(); + for (const chunk of input) source.writeBytes(chunk); + fc.pre(offset <= source.size); + const sizeBefore = source.size; + source.copyTo(new ByteQueue(), offset); + expect(source.size).toBe(sizeBefore); + }), + ); + }); +}); diff --git a/packages/core/src/io/byte-queue.test.ts b/packages/core/src/io/byte-queue.test.ts new file mode 100644 index 0000000..2027c29 --- /dev/null +++ b/packages/core/src/io/byte-queue.test.ts @@ -0,0 +1,280 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/byte-queue.test.ts +// Exercises: IO-1 (tail-append, transferred count, EOF sentinel), IO-2 (zero-count read), +// IO-3 (negative count rejected before any I/O), IO-4 (exact head removal, no partial write), +// IO-7 (FIFO buffer that is simultaneously source and sink), IO-8 (snapshot/copyOut independence), +// IO-30 (a wrapped byte array is an independent copy) +import {describe, expect, test} from 'bun:test'; +import {ByteQueue} from './byte-queue.js'; +import {AllocationLimitError, EndOfStreamError} from './errors.js'; +import {END_OF_STREAM, MAX_BYTE_ARRAY_LENGTH} from './limits.js'; + +const bytes = (...values: number[]): Uint8Array => Uint8Array.from(values); + +const drain = (queue: ByteQueue): number[] => [...queue.snapshot()]; + +describe('ByteQueue read (IO-1, IO-2, IO-7)', () => { + test('starts empty', () => { + expect(new ByteQueue().size).toBe(0); + }); + + test('IO-7: bytes written through the sink surface read back through the source surface in order', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3)); + source.writeBytes(bytes(4, 5)); + const dest = new ByteQueue(); + expect(source.read(dest, 5)).toBe(5); + expect(drain(dest)).toEqual([1, 2, 3, 4, 5]); + expect(source.size).toBe(0); + }); + + test('IO-1: read appends to the TAIL of a non-empty destination, never overwriting', () => { + const dest = new ByteQueue(); + dest.writeBytes(bytes(9, 9)); + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2)); + expect(source.read(dest, 2)).toBe(2); + expect(drain(dest)).toEqual([9, 9, 1, 2]); + }); + + test('IO-1: read never returns more than requested, and returns at least 1 when not exhausted', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3, 4)); + const dest = new ByteQueue(); + expect(source.read(dest, 2)).toBe(2); + expect(source.size).toBe(2); + }); + + test('IO-1: read of a partial source returns what it has, then END_OF_STREAM', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2)); + const dest = new ByteQueue(); + expect(source.read(dest, 8)).toBe(2); + expect(source.read(dest, 8)).toBe(END_OF_STREAM); + }); + + test('IO-2: a zero-count read returns 0 on a non-empty source', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1)); + expect(source.read(new ByteQueue(), 0)).toBe(0); + }); + + test('IO-2: a zero-count read returns 0 — NOT end-of-stream — on an exhausted source', () => { + expect(new ByteQueue().read(new ByteQueue(), 0)).toBe(0); + }); + + test('IO-3: a negative count is rejected before any transfer', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3)); + const dest = new ByteQueue(); + expect(() => source.read(dest, -1)).toThrow( + 'count must be a non-negative integer, got -1', + ); + expect(source.size).toBe(3); + expect(dest.size).toBe(0); + }); +}); + +describe('ByteQueue write (IO-3, IO-4)', () => { + test('IO-4: write removes exactly the requested count from the source HEAD, in order', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3)); + const dest = new ByteQueue(); + dest.write(source, 3); + expect(source.size).toBe(0); + expect(drain(dest)).toEqual([1, 2, 3]); + }); + + test('IO-4: writing more than the source holds throws instead of writing partially', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3)); + const dest = new ByteQueue(); + expect(() => { + dest.write(source, 4); + }).toThrow(EndOfStreamError); + expect(source.size).toBe(3); + expect(dest.size).toBe(0); + }); + + test('IO-3: write rejects a negative count', () => { + expect(() => { + new ByteQueue().write(new ByteQueue(), -2); + }).toThrow('count must be a non-negative integer, got -2'); + }); + + test('writeBytes copies, so mutating the caller input afterwards does not change the queue', () => { + const input = bytes(1, 2, 3); + const queue = new ByteQueue(); + queue.writeBytes(input); + input[0] = 99; + expect(drain(queue)).toEqual([1, 2, 3]); + }); + + test('a transfer that straddles chunk boundaries preserves order', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2)); + source.writeBytes(bytes(3, 4)); + source.writeBytes(bytes(5, 6)); + const dest = new ByteQueue(); + expect(source.read(dest, 3)).toBe(3); + expect(drain(dest)).toEqual([1, 2, 3]); + expect(source.size).toBe(3); + const rest = new ByteQueue(); + expect(source.read(rest, 3)).toBe(3); + expect(drain(rest)).toEqual([4, 5, 6]); + }); +}); + +describe('ByteQueue snapshot and copyTo (IO-8, IO-9, IO-10)', () => { + test('IO-8: snapshot does not consume or mutate', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + expect([...queue.snapshot()]).toEqual([1, 2, 3]); + expect(queue.size).toBe(3); + }); + + test('IO-8: a snapshot is independent of later mutations, in both directions', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + const first = queue.snapshot(); + queue.writeBytes(bytes(4)); + expect([...first]).toEqual([1, 2, 3]); + first[0] = 99; + expect([...queue.snapshot()]).toEqual([1, 2, 3, 4]); + }); + + test('IO-9: materializing past the limit fails with an actionable error, not an allocation crash', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + expect(() => queue.takeBytes(MAX_BYTE_ARRAY_LENGTH + 1)).toThrow( + AllocationLimitError, + ); + }); + + test('IO-10: copyTo copies a window without consuming or mutating the source', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3, 4, 5)); + const dest = new ByteQueue(); + source.copyTo(dest, 1, 3); + expect([...dest.snapshot()]).toEqual([2, 3, 4]); + expect(source.size).toBe(5); + }); + + test('IO-10: copyTo defaults to offset-through-end', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3, 4)); + const dest = new ByteQueue(); + source.copyTo(dest, 2); + expect([...dest.snapshot()]).toEqual([3, 4]); + }); + + test('IO-10: copyTo rejects an out-of-range window', () => { + const source = new ByteQueue(); + source.writeBytes(bytes(1, 2, 3)); + expect(() => { + source.copyTo(new ByteQueue(), 2, 5); + }).toThrow('copy window 2..7 exceeds size 3'); + expect(() => { + source.copyTo(new ByteQueue(), -1); + }).toThrow('offset must be a non-negative integer, got -1'); + }); + + test('IO-10: clear discards every byte', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + queue.clear(); + expect(queue.size).toBe(0); + expect([...queue.snapshot()]).toEqual([]); + }); +}); + +describe('ByteQueue takeBytes and skip', () => { + test('takeBytes consumes exactly the requested count', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3, 4)); + expect([...queue.takeBytes(2)]).toEqual([1, 2]); + expect(queue.size).toBe(2); + }); + + test('takeBytes past the end throws rather than returning short', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2)); + expect(() => queue.takeBytes(3)).toThrow(EndOfStreamError); + }); + + test('skip discards from the head and returns how many it discarded', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3, 4)); + expect(queue.skip(2)).toBe(2); + expect([...queue.snapshot()]).toEqual([3, 4]); + expect(queue.skip(9)).toBe(2); + expect(queue.size).toBe(0); + }); +}); + +describe('ByteQueue close (IO-41, IO-42)', () => { + test('IO-41: close is idempotent — a second close does not throw', () => { + const queue = new ByteQueue(); + queue.close(); + expect(() => { + queue.close(); + }).not.toThrow(); + expect(queue.closed).toBe(true); + }); + + test('IO-42: a purely in-memory buffer stays readable and writable after close', () => { + // IO-42 carves this out explicitly, and Phase 3b depends on it: snapshot-after-close is how + // post-mortem body logging works. Making an in-memory buffer throw here is one of the two + // directions IO-42 names as the porter's trap; the other is Task 6's stream-backed source, which + // MUST reject after close. + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + queue.close(); + expect([...queue.snapshot()]).toEqual([1, 2, 3]); + expect(() => { + queue.writeBytes(bytes(4)); + }).not.toThrow(); + expect(queue.read(new ByteQueue(), 1)).toBe(1); + }); +}); + +describe('ByteQueue input independence (IO-8, IO-30)', () => { + test('IO-30: a Node Buffer is COPIED, not aliased, so mutating it afterwards changes nothing', () => { + // `Buffer.prototype.slice` is an alias for `subarray`, so `bytes.slice()` would hand the queue a + // view over the caller's memory. A Buffer — very often a pooled one from a socket read — is the + // most likely input type in a Node SDK, which makes this the case IO-30 most needs to hold. + const buffer = Buffer.from('SECRET'); + const queue = new ByteQueue(); + queue.writeBytes(buffer); + buffer.fill(0x58); + expect(new TextDecoder().decode(queue.snapshot())).toBe('SECRET'); + }); + + test('IO-30: a retained chunk survives its pooled backing buffer being reused', () => { + const pool = Buffer.allocUnsafe(32); + pool.write('FIRSTCHUNK', 0, 'latin1'); + const queue = new ByteQueue(); + queue.writeBytes(pool.subarray(0, 10)); + pool.write('OVERWRITTEN', 0, 'latin1'); + expect(new TextDecoder().decode(queue.snapshot())).toBe('FIRSTCHUNK'); + }); + + test('copyOut returns an independent window without consuming', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2, 3)); + queue.writeBytes(bytes(4, 5)); + expect([...queue.copyOut(1, 3)]).toEqual([2, 3, 4]); + expect([...queue.copyOut(3)]).toEqual([4, 5]); + expect(queue.size).toBe(5); + const window = queue.copyOut(0, 2); + window[0] = 99; + expect([...queue.copyOut(0, 2)]).toEqual([1, 2]); + }); + + test('copyOut rejects a window past the end', () => { + const queue = new ByteQueue(); + queue.writeBytes(bytes(1, 2)); + expect(() => queue.copyOut(0, 3)).toThrow(); + expect(() => queue.copyOut(-1, 1)).toThrow(); + }); +}); diff --git a/packages/core/src/io/byte-queue.ts b/packages/core/src/io/byte-queue.ts new file mode 100644 index 0000000..75fbd5a --- /dev/null +++ b/packages/core/src/io/byte-queue.ts @@ -0,0 +1,315 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/byte-queue.ts +import {invariant} from '../invariant.js'; +import {AllocationLimitError, EndOfStreamError} from './errors.js'; +import { + assertAllocatable, + assertCount, + END_OF_STREAM, + MAX_BYTE_ARRAY_LENGTH, +} from './limits.js'; + +/** + * One node in the queue's chunk list. `bytes` is never mutated after the node is linked in, which is what + * makes zero-copy `subarray` transfers between queues safe; `start` is the first byte not yet consumed. + */ +interface Chunk { + readonly bytes: Uint8Array; + start: number; + next: Chunk | undefined; +} + +/** + * A FIFO byte queue that is simultaneously a source and a sink (IO-7). + * + * Synchronous throughout: pure memory has nothing to wait for, so making it async would allocate a Promise + * on the SDK's hottest data structure (styleguide 15.4) and force every downstream synchronous consumer to + * become async for no I/O reason. `BufferedSource`/`BufferedSink` are the async surfaces. + * + * Not safe for concurrent use (IO-37); callers serialize access. + * + * @internal + */ +export class ByteQueue { + #head: Chunk | undefined = undefined; + #tail: Chunk | undefined = undefined; + #size = 0; + #closed = false; + /** Bumped whenever bytes leave the head, invalidating the memoized seek position. */ + #generation = 0; + #seekChunk: Chunk | undefined = undefined; + #seekStart = 0; + #seekGeneration = -1; + + /** Bytes currently held (IO-7). */ + get size(): number { + return this.#size; + } + + /** Whether `close()` has been called. */ + get closed(): boolean { + return this.#closed; + } + + /** + * Append an independent copy of `bytes` to the tail. The copy is what lets IO-30's byte-array-wrapping + * factory promise that mutating the caller's input afterwards does not change the source, and it is + * what makes the `Chunk.bytes` "never mutated after linked in" invariant — and therefore `#moveTo`'s + * zero-copy `subarray` transfers — safe. + */ + writeBytes(bytes: Uint8Array): void { + if (bytes.length === 0) return; + this.#append(copyBytes(bytes)); + } + + /** + * Move up to `count` bytes from this queue's head onto `dest`'s tail (IO-1). + * + * Returns the number transferred: at least 1 when `count` is positive and the queue is not exhausted, + * exactly 0 when `count` is 0, `END_OF_STREAM` at end, and never more than requested. + */ + read(dest: ByteQueue, count: number): number { + assertCount(count); + // IO-2 is checked BEFORE exhaustion, deliberately: a zero-count read returns 0 even on an exhausted + // queue, and must never collapse to END_OF_STREAM. Reordering these two lines breaks IO-2. + if (count === 0) return 0; + if (this.#size === 0) return END_OF_STREAM; + const take = Math.min(count, this.#size); + this.#moveTo(dest, take); + return take; + } + + /** + * Move exactly `count` bytes from `src`'s head onto this queue's tail (IO-4). Fails rather than + * transferring a partial amount when `src` holds fewer. + */ + write(src: ByteQueue, count: number): void { + assertCount(count); + if (src.#size < count) throw new EndOfStreamError(src.#size, count); + src.#moveTo(this, count); + } + + /** + * A fresh, independent copy of the current contents, without consuming or mutating (IO-8). Later + * mutations do not affect a returned snapshot, and vice versa. + */ + snapshot(): Uint8Array { + return this.#materialize(0, this.#size); + } + + /** + * A fresh, independent copy of the window `[offset, offset + count)`, without consuming or mutating. + * + * The single-copy counterpart of `copyTo` for callers that want bytes rather than another queue: + * `copyTo(staging, ...)` followed by `staging.snapshot()` materializes the same window twice, which is + * what made `BufferedSource`'s newline scan quadratic. + */ + copyOut(offset: number, count?: number): Uint8Array { + invariant( + Number.isInteger(offset) && offset >= 0, + `offset must be a non-negative integer, got ${String(offset)}`, + ); + const length = count ?? this.#size - offset; + assertCount(length); + invariant( + offset + length <= this.#size, + `copy window ${String(offset)}..${String(offset + length)} exceeds size ${String(this.#size)}`, + ); + return this.#materialize(offset, length); + } + + /** + * Copy the window `[offset, offset + count)` into `dest` WITHOUT consuming or mutating this queue + * (IO-10). `count` defaults to "from offset through end". An out-of-range window is rejected. + */ + copyTo(dest: ByteQueue, offset: number, count?: number): void { + invariant( + Number.isInteger(offset) && offset >= 0, + `offset must be a non-negative integer, got ${String(offset)}`, + ); + const length = count ?? this.#size - offset; + assertCount(length); + invariant( + offset + length <= this.#size, + `copy window ${String(offset)}..${String(offset + length)} exceeds size ${String(this.#size)}`, + ); + if (length === 0) return; + dest.#append(this.#materialize(offset, length)); + } + + /** Consume and return exactly `count` bytes, failing rather than returning short. */ + takeBytes(count: number): Uint8Array { + assertCount(count); + // IO-9 before IO-4/IO-12's short-source check: an over-limit request is refused with an actionable + // AllocationLimitError even when the queue also happens to be short, rather than surfacing as an + // ordinary EndOfStreamError that hides the real problem. + assertAllocatable(count); + if (count > this.#size) throw new EndOfStreamError(this.#size, count); + const out = this.#materialize(0, count); + this.#discard(count); + return out; + } + + /** Discard up to `count` bytes from the head; returns how many were actually discarded. */ + skip(count: number): number { + assertCount(count); + const dropped = Math.min(count, this.#size); + this.#discard(dropped); + return dropped; + } + + /** Discard every byte (IO-10). */ + clear(): void { + this.#head = undefined; + this.#tail = undefined; + this.#size = 0; + this.#generation += 1; + } + + /** + * Copy `count` bytes starting `offset` from the head into one contiguous array (IO-9-bounded). + * + * Parameter order matches `copyTo(dest, offset, count)` deliberately: two adjacent `number`s in + * opposite orders across two methods is exactly the transposition hazard styleguide 5.5 names. + */ + #materialize(offset: number, count: number): Uint8Array { + assertAllocatable(count); + const out = allocate(count); + const seek = this.#seek(offset); + let chunk = seek.chunk; + let from = offset - seek.chunkStart; + let at = 0; + while (chunk !== undefined && at < count) { + const available = chunk.bytes.length - chunk.start; + const take = Math.min(available - from, count - at); + const start = chunk.start + from; + out.set(chunk.bytes.subarray(start, start + take), at); + at += take; + from = 0; + chunk = chunk.next; + } + return out; + } + + /** + * Position at the chunk holding logical `offset`, resuming from the last seek when it is still valid. + * + * Walking from the head every time makes a SEQUENTIAL scan quadratic in the number of chunks, which is + * how `readUtf8Line` stayed quadratic even after it was changed to peek only the newly pulled tail: a + * peer dribbling one byte per chunk produces one chunk per byte, and each peek re-walked all of them. + * Bytes are only ever appended at the tail, so a remembered position stays correct until something is + * removed from the head — which `#generation` detects. + */ + #seek(offset: number): {chunk: Chunk | undefined; chunkStart: number} { + let chunk = this.#head; + let chunkStart = 0; + if ( + this.#seekChunk !== undefined && + this.#seekGeneration === this.#generation && + this.#seekStart <= offset + ) { + chunk = this.#seekChunk; + chunkStart = this.#seekStart; + } + while (chunk !== undefined) { + const available = chunk.bytes.length - chunk.start; + if (offset < chunkStart + available) break; + chunkStart += available; + chunk = chunk.next; + } + this.#seekChunk = chunk; + this.#seekStart = chunkStart; + this.#seekGeneration = this.#generation; + return {chunk, chunkStart}; + } + + #discard(count: number): void { + if (count > 0) this.#generation += 1; + let remaining = count; + while (remaining > 0) { + const head = this.#head; + invariant(head !== undefined, 'byte-queue underflow during discard'); + const take = Math.min(head.bytes.length - head.start, remaining); + head.start += take; + remaining -= take; + if (head.start === head.bytes.length) this.#dropHead(); + } + this.#size -= count; + } + + /** + * Mark this queue closed (IO-41 — idempotent, the underlying resource released at most once). + * + * Deliberately leaves the read/write surface usable: IO-42 exempts a purely in-memory buffer so that + * snapshot-after-close body logging still works. A queue owns no external resource, so there is nothing + * else to release here. Invalidating derived views is `RetentionWindow`'s job, not this class's — views + * are cursors over a window, never over a bare queue. + */ + close(): void { + this.#closed = true; + } + + /** Caller owns the source-side size accounting; `#dropHead` deliberately does not touch `#size`. */ + #moveTo(dest: ByteQueue, count: number): void { + if (count > 0) this.#generation += 1; + let remaining = count; + while (remaining > 0) { + const head = this.#head; + invariant(head !== undefined, 'byte-queue underflow during move'); + const take = Math.min(head.bytes.length - head.start, remaining); + dest.#append(head.bytes.subarray(head.start, head.start + take)); + head.start += take; + remaining -= take; + if (head.start === head.bytes.length) this.#dropHead(); + } + this.#size -= count; + } + + #append(bytes: Uint8Array): void { + const chunk: Chunk = {bytes, start: 0, next: undefined}; + if (this.#tail === undefined) this.#head = chunk; + else this.#tail.next = chunk; + this.#tail = chunk; + this.#size += bytes.length; + } + + #dropHead(): void { + const head = this.#head; + invariant(head !== undefined, 'byte-queue drop with no head'); + this.#head = head.next; + if (this.#head === undefined) this.#tail = undefined; + } +} + +/** + * A genuinely independent copy of `bytes`. + * + * `bytes.slice()` is NOT sufficient: a Node `Buffer` is a `Uint8Array` subclass whose own + * `slice` is an alias for `subarray`, so it returns an aliasing view over the caller's memory. Since a + * `Buffer` — very often a pooled one handed over by a socket read — is the single most likely input type + * in a Node SDK, that would silently break IO-30's independence guarantee. Allocating + * a fresh array and `set` copies into it unconditionally, whatever the argument's subclass does. + * + * @internal + */ +export function copyBytes(bytes: Uint8Array): Uint8Array { + const out = new Uint8Array(bytes.length); + out.set(bytes); + return out; +} + +/** + * IO-9's backstop. The eager `MAX_BYTE_ARRAY_LENGTH` check is deliberately conservative, so a host whose + * real ceiling is lower would otherwise surface a raw `RangeError` — exactly the "low-level allocation + * crash" IO-9 exists to prevent. + */ +function allocate(count: number): Uint8Array { + try { + return new Uint8Array(count); + } catch (e: unknown) { + if (e instanceof RangeError) { + throw new AllocationLimitError(count, MAX_BYTE_ARRAY_LENGTH, {cause: e}); + } + throw e; + } +} diff --git a/packages/core/src/io/errors.test.ts b/packages/core/src/io/errors.test.ts new file mode 100644 index 0000000..bc4a453 --- /dev/null +++ b/packages/core/src/io/errors.test.ts @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/errors.test.ts +// Exercises: IO-4/IO-11/IO-12/IO-15 (EndOfStreamError), IO-17 (SourceContractViolationError), +// IO-24/IO-42 (ClosedResourceError), IO-9 (AllocationLimitError) +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, + TransportFailureError, +} from './errors.js'; + +describe('IoError tree', () => { + test('IoError descends from DexpaceError', () => { + expect(new IoError('boom')).toBeInstanceOf(DexpaceError); + }); + + test('every leaf descends from DexpaceError directly, not through IoError (Phase 3b retrofit)', () => { + expect(new EndOfStreamError(3, 8)).toBeInstanceOf(DexpaceError); + expect(new EndOfStreamError(3, 8)).not.toBeInstanceOf(IoError); + expect(new SourceContractViolationError('zero read')).toBeInstanceOf( + DexpaceError, + ); + expect(new ClosedResourceError('BufferedSource')).toBeInstanceOf( + DexpaceError, + ); + expect(new AllocationLimitError(9, 8)).toBeInstanceOf(DexpaceError); + }); + + test('each error sets name from its own constructor', () => { + expect(new EndOfStreamError(3, 8).name).toBe('EndOfStreamError'); + expect(new ClosedResourceError('ByteQueue').name).toBe( + 'ClosedResourceError', + ); + }); + + test('EndOfStreamError names delivered-of-requested as typed fields and in the message', () => { + const error = new EndOfStreamError(3, 8); + expect(error.delivered).toBe(3); + expect(error.requested).toBe(8); + expect(error.message).toBe('end of stream: delivered 3 of 8 bytes'); + }); + + test('ClosedResourceError names the resource and is distinct from end-of-stream', () => { + const error = new ClosedResourceError('BufferedSource'); + expect(error.message).toBe('BufferedSource is closed'); + expect(error).not.toBeInstanceOf(EndOfStreamError); + }); + + test('AllocationLimitError points at streaming alternatives', () => { + const error = new AllocationLimitError(5_000, 4_000); + expect(error.requested).toBe(5_000); + expect(error.limit).toBe(4_000); + expect(error.message).toBe( + 'cannot materialize 5000 bytes as one array (limit 4000); stream the body instead', + ); + }); + + test('cause chains through', () => { + const cause = new RangeError('array too large'); + expect(new AllocationLimitError(5, 4, {cause}).cause).toBe(cause); + }); + + test('isIoError groups every leaf, including bare IoError, without a class tier', () => { + expect(isIoError(new IoError('x'))).toBe(true); + expect(isIoError(new EndOfStreamError(1, 2))).toBe(true); + expect(isIoError(new SourceContractViolationError('x'))).toBe(true); + expect(isIoError(new ClosedResourceError('x'))).toBe(true); + expect(isIoError(new AllocationLimitError(1, 2))).toBe(true); + expect(isIoError(new DexpaceError('other'))).toBe(false); + expect(isIoError(new Error('plain'))).toBe(false); + }); +}); + +describe('TransportFailureError (TRANSPORT-20)', () => { + test('is an IoError subtype', () => { + const error = new TransportFailureError('connect ECONNREFUSED'); + expect(error).toBeInstanceOf(IoError); + expect(error.name).toBe('TransportFailureError'); + }); + + test('carries an optional cause', () => { + const cause = new Error('ECONNREFUSED'); + const error = new TransportFailureError('connect failed', {cause}); + expect(error.cause).toBe(cause); + }); +}); diff --git a/packages/core/src/io/errors.ts b/packages/core/src/io/errors.ts new file mode 100644 index 0000000..8c2f6b5 --- /dev/null +++ b/packages/core/src/io/errors.ts @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * Root of the I/O error tree (product-spec §5). + * + * Error messages in this tree carry counts and limits, never buffer contents — these buffers hold request + * and response bodies, which routinely contain credentials and PII (styleguide 8.8). + * + * @public + */ +export class IoError extends DexpaceError { + // bun's coverage tool never marks a bodiless subclass's implicit constructor as covered + // (undercounts function coverage); an explicit forwarding constructor is instrumented + // correctly and keeps the file above the 80% function-coverage floor without changing behavior. + // eslint-disable-next-line @typescript-eslint/no-useless-constructor -- see comment above + constructor(message: string, options?: ErrorOptions) { + super(message, options); + } +} + +/** + * A source ended before delivering the requested number of bytes (IO-11, IO-12, IO-15), or a sink write + * found fewer bytes in its source buffer than requested (IO-4). + * + * @public + */ +export class EndOfStreamError extends DexpaceError { + /** How many bytes the source actually delivered before ending. */ + readonly delivered: number; + /** How many bytes the caller required. */ + readonly requested: number; + + constructor(delivered: number, requested: number, options?: ErrorOptions) { + super( + `end of stream: delivered ${String(delivered)} of ${String(requested)} bytes`, + options, + ); + this.delivered = delivered; + this.requested = requested; + } +} + +/** + * A foreign source violated the read protocol — most commonly by returning zero bytes for a positive + * requested count, which IO-17 requires be raised rather than tolerated as end-of-stream or spun on. + * + * @public + */ +export class SourceContractViolationError extends DexpaceError { + // See IoError's constructor above: keeps this bodiless subclass registered for bun's + // function coverage. + // eslint-disable-next-line @typescript-eslint/no-useless-constructor -- see comment above + constructor(message: string, options?: ErrorOptions) { + super(message, options); + } +} + +/** + * A closed source, sink, buffer, or view was used (IO-42), or a view outlived the parent that invalidated + * it (IO-22). Distinct from `EndOfStreamError` by requirement — IO-24 demands a closed view fail loudly + * with a state error rather than looking like a normal exhaustion. + * + * @public + */ +export class ClosedResourceError extends DexpaceError { + /** The resource that was already closed, as it appears in the message. */ + readonly resource: string; + + constructor(resource: string, options?: ErrorOptions) { + super(`${resource} is closed`, options); + this.resource = resource; + } +} + +/** + * A materialization would exceed the maximum single-array allocation (IO-9). The message points at the + * streaming alternative, as IO-9 requires. + * + * @public + */ +export class AllocationLimitError extends DexpaceError { + /** The byte count the materialization asked for. */ + readonly requested: number; + /** The maximum single-array allocation this runtime permits (IO-9). */ + readonly limit: number; + + constructor(requested: number, limit: number, options?: ErrorOptions) { + super( + `cannot materialize ${String(requested)} bytes as one array (limit ${String(limit)}); stream the body instead`, + options, + ); + this.requested = requested; + this.limit = limit; + } +} + +/** + * Groups every leaf in this file, including bare `IoError`, without reintroducing a class tier between + * them and `DexpaceError` — the corpus caps custom error hierarchies at two levels. Retrofits Phase 3a's + * shape, where the four leaves extended `IoError` (a 3-tier chain). `http/errors.ts` carried an + * identically-shaped tier for longer; `isDomainModelError` is this guard's counterpart there, added + * when that one was flattened. + * + * @public + */ +export function isIoError( + error: unknown, +): error is + | IoError + | EndOfStreamError + | SourceContractViolationError + | ClosedResourceError + | AllocationLimitError { + return ( + error instanceof IoError || + error instanceof EndOfStreamError || + error instanceof SourceContractViolationError || + error instanceof ClosedResourceError || + error instanceof AllocationLimitError + ); +} + +/** + * The canonical retryable transport-failure exception (TRANSPORT-20): any send that produced no HTTP + * response — connection refused, DNS/TLS failure, peer reset, connect/read timeout. A subtype of IoError + * so 5a's `classify.ts` cause-walk already treats it as always-retryable with no change to that file. + * + * @public + */ +export class TransportFailureError extends IoError { + // eslint-disable-next-line @typescript-eslint/no-useless-constructor -- load-bearing for Bun function coverage (see IoError) + constructor(message: string, options?: ErrorOptions) { + super(message, options); + } +} diff --git a/packages/core/src/io/factories.test.ts b/packages/core/src/io/factories.test.ts new file mode 100644 index 0000000..3f993be --- /dev/null +++ b/packages/core/src/io/factories.test.ts @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/factories.test.ts +// Exercises: IO-30 (factory half — fresh, independent, empty buffers; stream, byte-array, and +// foreign-primitive wrapping; the byte-array source is an independent copy), IO-17 (a primitive +// source returning 0 for a positive request fails loudly, and one that misreports its transferred +// count in either direction is a contract violation rather than an exhausted stream) +import {describe, expect, test} from 'bun:test'; +import {ByteQueue} from './byte-queue.js'; +import {SourceContractViolationError} from './errors.js'; +import { + bufferedSinkOverPrimitive, + bufferedSinkOverStream, + bufferedSourceOverBytes, + bufferedSourceOverPrimitive, + bufferedSourceOverStream, + newByteQueue, +} from './factories.js'; +import {END_OF_STREAM} from './limits.js'; +import { + collectingWritableStream, + fakeReadableStream, +} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +describe('IO-30 factories', () => { + test('two buffers are distinct and both empty', () => { + const first = newByteQueue(); + const second = newByteQueue(); + expect(first).not.toBe(second); + expect(first.size).toBe(0); + expect(second.size).toBe(0); + }); + + test('buffers are independent — writing to one does not affect the other', () => { + const first = newByteQueue(); + const second = newByteQueue(); + first.writeBytes(Uint8Array.from([1, 2])); + expect(second.size).toBe(0); + }); + + test('wrapping a byte array then mutating the input leaves the source unchanged', async () => { + const input = Uint8Array.from([1, 2, 3]); + const source = bufferedSourceOverBytes(input); + input[0] = 99; + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('wrapping a caller stream produces a readable source', async () => { + const source = bufferedSourceOverStream( + fakeReadableStream([Uint8Array.from([7, 8])]), + ); + expect([...(await source.readBytes())]).toEqual([7, 8]); + }); + + test('wrapping a caller stream produces a writable sink', async () => { + const {stream, written} = collectingWritableStream(); + const sink = bufferedSinkOverStream(stream); + await sink.writeUtf8('hi'); + await sink.close(); + expect(new TextDecoder().decode(written())).toBe('hi'); + }); + + test('wrapping a foreign primitive source supplies the typed reads', async () => { + const backing = newByteQueue(); + backing.writeBytes(Uint8Array.from([1, 2, 3])); + const source = bufferedSourceOverPrimitive({ + read: (dest, count) => backing.read(dest, count), + }); + expect([...(await source.readBytes())]).toEqual([1, 2, 3]); + }); + + test('IO-17: a primitive source returning 0 for a positive request fails loudly', async () => { + const source = bufferedSourceOverPrimitive({read: () => 0}); + expect(await rejection(source.readBytes())).toBeInstanceOf( + SourceContractViolationError, + ); + }); + + test('wrapping a foreign primitive sink supplies the typed writes', async () => { + const collected = newByteQueue(); + const sink = bufferedSinkOverPrimitive({ + write: (src, count) => { + collected.write(src, count); + }, + }); + await sink.writeUtf8('hi'); + await sink.close(); + expect(new TextDecoder().decode(collected.snapshot())).toBe('hi'); + }); +}); + +describe('bufferedSourceOverPrimitive residue handling (IO-1, IO-17)', () => { + test('a primitive that appends more than it reports fails loudly instead of losing bytes', async () => { + // A staging queue hoisted into the closure and drained by `read` rather than by its size leaves the + // excess at the head, where it is both dropped from its own pull and re-emitted out of order on the + // next one — 8 bytes in, 4 out, no error at all. + let call = 0; + const source = bufferedSourceOverPrimitive({ + read(dest): number { + call += 1; + if (call > 2) return END_OF_STREAM; + const base = call === 1 ? 10 : 20; + dest.writeBytes(Uint8Array.from([base, base + 1, base + 2, base + 3])); + return 2; + }, + }); + expect(await rejection(source.readBytes())).toBeInstanceOf( + SourceContractViolationError, + ); + }); + + test('a primitive that appends bytes at end of stream fails loudly', async () => { + const source = bufferedSourceOverPrimitive({ + read(dest): number { + dest.writeBytes(Uint8Array.from([1, 2])); + return END_OF_STREAM; + }, + }); + expect(await rejection(source.readBytes())).toBeInstanceOf( + SourceContractViolationError, + ); + }); + + test('each pull gets a fresh staging queue, so nothing carries between them', async () => { + let call = 0; + const source = bufferedSourceOverPrimitive({ + read(dest): number { + call += 1; + if (call > 2) return END_OF_STREAM; + const base = call === 1 ? 10 : 20; + dest.writeBytes(Uint8Array.from([base, base + 1])); + return 2; + }, + }); + expect([...(await source.readBytes())]).toEqual([10, 11, 20, 21]); + }); +}); + +describe('a foreign primitive source that misreports its count (IO-17)', () => { + test('over-reporting is a contract violation, not an exhausted stream', async () => { + // Left to `takeBytes` this surfaced as `EndOfStreamError: delivered 2 of 99 bytes` -- reporting a + // foreign source's broken accounting as end-of-stream, the exact confusion IO-17 forbids. + const source = bufferedSourceOverPrimitive({ + read(dest: ByteQueue): number { + dest.writeBytes(Uint8Array.from([1, 2])); + return 99; + }, + }); + const error = await rejection(source.readBytes()); + expect(error).toBeInstanceOf(SourceContractViolationError); + expect(error.message).toContain('appended only 2'); + // No close(): the pull failure already errored the stream, so cancel() would reject with the very + // same error. Matches the zero-read case above. + }); + + test('under-reporting is a contract violation too', async () => { + const source = bufferedSourceOverPrimitive({ + read(dest: ByteQueue): number { + dest.writeBytes(Uint8Array.from([1, 2, 3])); + return 1; + }, + }); + expect(await rejection(source.readBytes())).toBeInstanceOf( + SourceContractViolationError, + ); + }); +}); diff --git a/packages/core/src/io/factories.ts b/packages/core/src/io/factories.ts new file mode 100644 index 0000000..5dc6cbd --- /dev/null +++ b/packages/core/src/io/factories.ts @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/factories.ts +import {BufferedSink} from './buffered-sink.js'; +import {BufferedSource} from './buffered-source.js'; +import {ByteQueue} from './byte-queue.js'; +import {SourceContractViolationError} from './errors.js'; +import {END_OF_STREAM} from './limits.js'; + +/** + * IO-30's factory half. Named free functions rather than a namespace object, so the module stays + * tree-shakeable (styleguide 10.1, 15.9). + * + * IO-30's provider-*resolution* half — install precedence, idempotent install, caching, warning, + * de-duplication, and the IO-31–IO-36 rules it defers to — is deliberately not built. There is one + * implementation, always present, requiring no installation call; `sdk-design/03` §3.1 derives this in + * full, and it is the same permanent simplification as SEAM-5–SEAM-10. + * + * @internal + */ + +/** A fresh, independent, empty buffer (IO-30). */ +export function newByteQueue(): ByteQueue { + return new ByteQueue(); +} + +/** Wrap a caller stream as a buffered source (IO-30). */ +export function bufferedSourceOverStream( + stream: ReadableStream<Uint8Array>, +): BufferedSource { + return BufferedSource.overStream(stream); +} + +/** Wrap a byte array as a buffered source over an independent copy (IO-30). */ +export function bufferedSourceOverBytes(bytes: Uint8Array): BufferedSource { + return BufferedSource.overBytes(bytes); +} + +/** Wrap a caller stream as a buffered sink (IO-30). */ +export function bufferedSinkOverStream( + stream: WritableStream<Uint8Array>, +): BufferedSink { + return BufferedSink.overStream(stream); +} + +/** + * The raw read protocol of IO-1 — append up to `count` bytes to `dest`'s tail, return the number + * transferred or `END_OF_STREAM` — with none of the typed reads, views, or line semantics. What a + * "foreign primitive" source implements. + */ +export interface PrimitiveSource { + read(dest: ByteQueue, count: number): Promise<number> | number; +} + +/** The raw write protocol of IO-4 — remove exactly `count` bytes from `src`'s head, push downstream. */ +export interface PrimitiveSink { + write(src: ByteQueue, count: number): Promise<void> | void; +} + +/** How much the primitive-source adapter asks for per pull. */ +const PRIMITIVE_CHUNK = 16 * 1024; + +/** Wrap a foreign primitive source with the typed buffered surface (IO-30). */ +export function bufferedSourceOverPrimitive( + source: PrimitiveSource, +): BufferedSource { + return BufferedSource.overStream( + new ReadableStream<Uint8Array>({ + async pull(controller): Promise<void> { + // A FRESH queue per pull, matching `bufferedSinkOverPrimitive`. Hoisting one into the closure + // and draining only `read` of it leaves any excess the primitive appended sitting at the head, + // where it is both lost from its own pull and re-emitted out of order on the next one. + const staging = new ByteQueue(); + const read = await source.read(staging, PRIMITIVE_CHUNK); + if (read === END_OF_STREAM) { + assertDrained( + staging, + 'foreign source appended bytes at end of stream', + ); + controller.close(); + return; + } + if (read === 0) { + // IO-17: a zero-byte read for a positive request is a source-contract violation — never + // tolerated as end-of-stream, never spun on. + throw new SourceContractViolationError( + 'foreign source returned 0 bytes for a positive request', + ); + } + // IO-17: over-reporting is a contract violation too, and must say so. Left to `takeBytes` it + // surfaced as `EndOfStreamError: delivered 2 of 99 bytes` -- reporting a foreign source's + // broken accounting as an exhausted stream, which is the exact confusion IO-17 forbids. + if (staging.size < read) { + throw new SourceContractViolationError( + `foreign source reported ${String(read)} bytes but appended only ${String(staging.size)}`, + ); + } + const chunk = staging.takeBytes(read); + // IO-17: appending more than it reported is a contract violation too. Silently dropping the + // excess is how bytes go missing with no error at all. + assertDrained( + staging, + `foreign source reported ${String(read)} bytes but appended more`, + ); + controller.enqueue(chunk); + }, + }), + ); +} + +function assertDrained(staging: ByteQueue, message: string): void { + if (staging.size > 0) throw new SourceContractViolationError(message); +} + +/** Wrap a foreign primitive sink with the typed buffered surface (IO-30). */ +export function bufferedSinkOverPrimitive(sink: PrimitiveSink): BufferedSink { + return BufferedSink.overStream( + new WritableStream<Uint8Array>({ + async write(chunk): Promise<void> { + const staging = new ByteQueue(); + staging.writeBytes(chunk); + await sink.write(staging, staging.size); + }, + }), + ); +} diff --git a/packages/core/src/io/index.ts b/packages/core/src/io/index.ts new file mode 100644 index 0000000..1358a5a --- /dev/null +++ b/packages/core/src/io/index.ts @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/index.ts +// Internal barrel for product-spec §5 (IO-1–IO-42). +// +// The PROVIDER types here — BufferedSource, BufferedSink, ByteQueue, TeeSink and their factories — +// are @internal and are not re-exported from packages/core/src/index.ts, kept out of the +// api-extractor surface so a later phase can promote them deliberately (styleguide 10.3), or not at +// all: 3b shaped BODY-1's write-to-sink around the platform's WritableStream rather than BufferedSink. +// +// The ERROR leaves are a different case, and this comment has twice claimed otherwise. Phase 8a +// promoted `IoError` and `TransportFailureError` to the public barrel because TRANSPORT-20 makes the +// subtyping a requirement and `retry/classify.ts`'s cause-walk is load-bearing on it, and Phase 9's +// U9 pass promoted `EndOfStreamError` — it was the subject of four `@throws` tags on public symbols +// with no class a caller could catch. `1f48926` finished the set: `isIoError`, +// `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` are exported as +// well, so every error symbol re-exported below is also on `packages/core/src/index.ts:39-48` and in +// `packages/core/etc/core.api.md`. Nothing in this file's error block is internal any more +// (docs/work/mvp/2026-09-04-open-items-dissolution.md H8, whose remaining sub-item was the category +// catch `isIoError` now provides). +// +// "Load-bearing on it" is narrower than it sounds, and the difference is a decision rather than an +// accident. The cause-walk at `../retry/classify.ts:90` tests `instanceof IoError`, so it matches +// `IoError` and `TransportFailureError` — and NOT the four leaves below, which extend `DexpaceError` +// directly and are grouped only by `isIoError`. That branch means "the wire failed": a send that +// produced no response is retryable (RETRY-4, TRANSPORT-20), while a violated source contract, a +// closed resource, an allocation cap and a short exact-length copy are this package's own failures +// and repeat identically on the next attempt. Audit #67 / #78 decided it; `docs/deviations.md` +// item 17 carries the rationale and `../retry/classify.test.ts` pins one answer per class. Do not +// re-parent a leaf under `IoError` to tidy the tree — that silently makes it retryable. +export {BufferedSink} from './buffered-sink.js'; +export {BufferedSource} from './buffered-source.js'; +export {ByteQueue, copyBytes} from './byte-queue.js'; +export { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, +} from './errors.js'; +export { + bufferedSinkOverPrimitive, + bufferedSinkOverStream, + bufferedSourceOverBytes, + bufferedSourceOverPrimitive, + bufferedSourceOverStream, + newByteQueue, + type PrimitiveSink, + type PrimitiveSource, +} from './factories.js'; +export { + assertAllocatable, + assertCount, + END_OF_STREAM, + MAX_BYTE_ARRAY_LENGTH, +} from './limits.js'; +export {writeAll} from './pump.js'; +export {RetentionWindow, type Cursor} from './retention-window.js'; +export type {Sink} from './sink.js'; +export {TeeSink} from './tee-sink.js'; +export {decodeText, encodeText} from './text-codec.js'; diff --git a/packages/core/src/io/limits.test.ts b/packages/core/src/io/limits.test.ts new file mode 100644 index 0000000..47c6ccb --- /dev/null +++ b/packages/core/src/io/limits.test.ts @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/limits.test.ts +// Exercises: IO-1 (the end-of-stream sentinel), IO-9 (maximum single-array allocation) +import {describe, expect, test} from 'bun:test'; +import {AllocationLimitError} from './errors.js'; +import { + assertAllocatable, + END_OF_STREAM, + MAX_BYTE_ARRAY_LENGTH, +} from './limits.js'; + +describe('limits', () => { + test('END_OF_STREAM is the -1 sentinel IO-1 specifies', () => { + expect(END_OF_STREAM).toBe(-1); + }); + + test('MAX_BYTE_ARRAY_LENGTH is a positive safe integer', () => { + expect(Number.isSafeInteger(MAX_BYTE_ARRAY_LENGTH)).toBe(true); + expect(MAX_BYTE_ARRAY_LENGTH).toBeGreaterThan(0); + }); + + // There is deliberately NO test that a Uint8Array of MAX_BYTE_ARRAY_LENGTH actually allocates. + // Honest verification means allocating 2 GiB, which is far too heavy for the default suite, and the + // cheap stand-in — comparing the constant against another compile-time constant — cannot fail for any + // value the constant could plausibly hold, so it reads as coverage while asserting nothing. The + // guarantee is carried instead by the RangeError backstop in `ByteQueue.allocate`, which converts a + // host whose real ceiling is lower into an AllocationLimitError (see byte-queue.test.ts). + test('MAX_BYTE_ARRAY_LENGTH stays under the 2 GiB the docs promise', () => { + expect(MAX_BYTE_ARRAY_LENGTH).toBe(2 ** 31 - 1); + }); + + test('IO-9: assertAllocatable refuses over the ceiling and permits everything at or under it', () => { + expect(() => { + assertAllocatable(MAX_BYTE_ARRAY_LENGTH + 1); + }).toThrow(AllocationLimitError); + expect(() => { + assertAllocatable(MAX_BYTE_ARRAY_LENGTH); + }).not.toThrow(); + expect(() => { + assertAllocatable(0); + }).not.toThrow(); + }); + + test('IO-9: the refusal names the limit and points at streaming alternatives', () => { + // This is the guard the count-less read path applies incrementally, so it stands in for the + // multi-gigabyte case the suite cannot afford to allocate. + const error = new AllocationLimitError( + MAX_BYTE_ARRAY_LENGTH + 1, + MAX_BYTE_ARRAY_LENGTH, + ); + expect(error.message).toContain(String(MAX_BYTE_ARRAY_LENGTH)); + }); +}); diff --git a/packages/core/src/io/limits.ts b/packages/core/src/io/limits.ts new file mode 100644 index 0000000..7c00049 --- /dev/null +++ b/packages/core/src/io/limits.ts @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/limits.ts +import {invariant} from '../invariant.js'; +import {AllocationLimitError} from './errors.js'; + +/** + * End-of-stream sentinel returned by every read (IO-1). + * + * The numeric protocol is kept spec-literal rather than modelled as `number | undefined`, because IO-2 + * (a zero-count read returns 0 and must NOT report end-of-stream) and, later, BODY-25 ("EOF is signaled + * only by the explicit sentinel") both reason over it. + * + * @internal + */ +export const END_OF_STREAM = -1; + +/** + * Largest byte count this package will attempt to materialize as one contiguous `Uint8Array` (IO-9). + * + * Deliberately conservative. Core is runtime-agnostic, so `node:buffer`'s constant is unavailable; V8 and + * JavaScriptCore disagree on the real ceiling and both have moved it, and rule 12.6 forbids probing at + * import time. 2 GiB − 1 is at or below every supported host's limit. Callers that exceed it get an + * actionable `AllocationLimitError` rather than a low-level allocation crash; a `RangeError` backstop at + * the allocation site covers any host whose real ceiling is lower still. + * + * @internal + */ +export const MAX_BYTE_ARRAY_LENGTH = 2 ** 31 - 1; + +/** + * IO-9's eager guard: refuse a materialization that would exceed the ceiling, with an actionable error + * that points at streaming alternatives, BEFORE any allocation is attempted. + * + * A named function rather than an inlined `if` at each site because the count-less read path has to + * apply it incrementally — it cannot know the total up front — and a rule applied in two shapes is a + * rule that drifts. + */ +export function assertAllocatable(count: number): void { + if (count > MAX_BYTE_ARRAY_LENGTH) { + throw new AllocationLimitError(count, MAX_BYTE_ARRAY_LENGTH); + } +} + +/** + * IO-3's eager guard: a negative or non-integer count is an argument error, rejected BEFORE any I/O so + * neither the source nor the destination is touched. + * + * Single-sourced here for the same reason `assertAllocatable` is. It previously existed as three + * byte-for-byte copies (`byte-queue.ts`, `buffered-source.ts`, `buffered-sink.ts`) and `TeeSink` — the + * fourth size-taking surface — had none at all, so a negative count reached it and was rejected only + * indirectly, by whichever `ByteQueue` call happened to run first. That is exactly the drift the + * "a rule applied in two shapes is a rule that drifts" note above warns about. + */ +export function assertCount(count: number): void { + invariant( + Number.isInteger(count) && count >= 0, + `count must be a non-negative integer, got ${String(count)}`, + ); +} diff --git a/packages/core/src/io/pump.test.ts b/packages/core/src/io/pump.test.ts new file mode 100644 index 0000000..80edb15 --- /dev/null +++ b/packages/core/src/io/pump.test.ts @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/pump.test.ts +// Exercises: IO-17 (pump to exhaustion, terminate only on the EOF sentinel, raise a zero-read for a +// positive request as a source-contract violation) +import {describe, expect, test} from 'bun:test'; +import {BufferedSink} from './buffered-sink.js'; +import {BufferedSource} from './buffered-source.js'; +import {SourceContractViolationError} from './errors.js'; +import {writeAll} from './pump.js'; +import { + collectingWritableStream, + fakeReadableStream, + protocolViolatingStream, +} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +describe('writeAll (IO-17)', () => { + test('pumps the source to exhaustion and returns the total transferred', async () => { + const source = BufferedSource.overStream( + fakeReadableStream([Uint8Array.from([1, 2]), Uint8Array.from([3, 4, 5])]), + ); + const {stream, written} = collectingWritableStream(); + const sink = BufferedSink.overStream(stream); + + expect(await writeAll(source, sink)).toBe(5); + await sink.close(); + expect([...written()]).toEqual([1, 2, 3, 4, 5]); + }); + + test('an already-exhausted source transfers zero and does not hang', async () => { + const source = BufferedSource.overStream(fakeReadableStream([])); + const {stream} = collectingWritableStream(); + expect(await writeAll(source, BufferedSink.overStream(stream))).toBe(0); + }); + + test('a source returning zero bytes for a positive request is a contract violation', async () => { + // Never tolerated as end-of-stream, and never spun on forever — a misbehaving foreign source must + // fail loudly rather than hang or truncate a body. + const source = BufferedSource.overStream(protocolViolatingStream()); + const {stream} = collectingWritableStream(); + expect( + await rejection(writeAll(source, BufferedSink.overStream(stream))), + ).toBeInstanceOf(SourceContractViolationError); + }); +}); diff --git a/packages/core/src/io/pump.ts b/packages/core/src/io/pump.ts new file mode 100644 index 0000000..50db09c --- /dev/null +++ b/packages/core/src/io/pump.ts @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/pump.ts +import type {BufferedSource} from './buffered-source.js'; +import {ByteQueue} from './byte-queue.js'; +import type {Sink} from './sink.js'; +import {END_OF_STREAM} from './limits.js'; + +/** How much the pump asks for per iteration. */ +const PUMP_CHUNK = 16 * 1024; + +/** + * Pump `source` to exhaustion into `sink` and return the total bytes transferred (IO-17). + * + * Terminates only on the end-of-stream sentinel. A zero-byte read for a non-zero requested count is a + * source-contract violation raised by the source itself — never tolerated here as end-of-stream, and + * never spun on. + * + * @internal + */ +export async function writeAll( + source: BufferedSource, + sink: Sink, +): Promise<number> { + const staging = new ByteQueue(); + let total = 0; + for (;;) { + const read = await source.read(staging, PUMP_CHUNK); + if (read === END_OF_STREAM) return total; + await sink.write(staging, staging.size); + total += read; + } +} diff --git a/packages/core/src/io/retention-window.test.ts b/packages/core/src/io/retention-window.test.ts new file mode 100644 index 0000000..dfa8e77 --- /dev/null +++ b/packages/core/src/io/retention-window.test.ts @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/retention-window.test.ts +// Exercises: IO-19/IO-20 (non-consuming views), IO-22 (parent close invalidates views), +// IO-23 (mutually independent cursors), IO-24 (closed view fails loudly, distinct from EOF), +// IO-17 (a chunk that is not a Uint8Array is a source-contract violation), +// IO-41 (teardown is awaited, releases the reader lock, and surfaces its failure) +import {describe, expect, test} from 'bun:test'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, SourceContractViolationError} from './errors.js'; +import {RetentionWindow} from './retention-window.js'; +import {fakeReadableStream} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const bytes = (...values: number[]): Uint8Array => Uint8Array.from(values); + +const windowOver = (...chunks: Uint8Array[]): RetentionWindow => + new RetentionWindow(fakeReadableStream(chunks).getReader()); + +describe('RetentionWindow', () => { + test('pullThrough pulls until the requested logical offset is available', async () => { + const window = windowOver(bytes(1, 2), bytes(3, 4)); + expect(await window.pullThrough(3)).toBe(true); + expect(window.pulledThrough).toBeGreaterThanOrEqual(3); + }); + + test('pullThrough returns false once the stream is exhausted', async () => { + const window = windowOver(bytes(1, 2)); + expect(await window.pullThrough(5)).toBe(false); + expect(window.pulledThrough).toBe(2); + }); + + test('readInto advances only the cursor it is given', async () => { + const window = windowOver(bytes(1, 2, 3, 4)); + const first = window.register(0); + const second = window.register(0); + await window.pullThrough(4); + + const dest = new ByteQueue(); + expect(window.readInto(first, dest, 2)).toBe(2); + expect(first.at).toBe(2); + expect(second.at).toBe(0); + }); + + test('IO-23: two cursors read the same bytes independently', async () => { + const window = windowOver(bytes(1, 2, 3)); + const first = window.register(0); + const second = window.register(0); + await window.pullThrough(3); + + const a = new ByteQueue(); + const b = new ByteQueue(); + window.readInto(first, a, 3); + window.readInto(second, b, 3); + expect([...a.snapshot()]).toEqual([1, 2, 3]); + expect([...b.snapshot()]).toEqual([1, 2, 3]); + }); + + test('bytes behind the slowest cursor are trimmed, bytes at or ahead of it are retained', async () => { + const window = windowOver(bytes(1, 2, 3, 4)); + const fast = window.register(0); + const slow = window.register(0); + await window.pullThrough(4); + + window.readInto(fast, new ByteQueue(), 4); + expect(window.retainedBytes).toBe(4); // slow still needs all four + + window.readInto(slow, new ByteQueue(), 4); + expect(window.retainedBytes).toBe(0); // nobody needs them now + }); +}); + +describe('RetentionWindow trim, peek, and close', () => { + test('releasing a cursor lets the head trim forward', async () => { + const window = windowOver(bytes(1, 2, 3, 4)); + const fast = window.register(0); + const slow = window.register(0); + await window.pullThrough(4); + window.readInto(fast, new ByteQueue(), 4); + + window.release(slow); + expect(window.retainedBytes).toBe(0); + }); + + test('peekBytes materializes without advancing the cursor', async () => { + const window = windowOver(bytes(1, 2, 3)); + const cursor = window.register(0); + await window.pullThrough(3); + expect([...window.peekBytes(cursor, 0, 2)]).toEqual([1, 2]); + expect(cursor.at).toBe(0); + }); + + test('peekBytes reads a window starting `offset` ahead of the cursor', async () => { + const window = windowOver(bytes(1, 2, 3, 4, 5)); + const cursor = window.register(0); + await window.pullThrough(5); + expect([...window.peekBytes(cursor, 2, 2)]).toEqual([3, 4]); + // Clamped to what has been pulled, never over-reading past the end. + expect([...window.peekBytes(cursor, 3, 99)]).toEqual([4, 5]); + expect([...window.peekBytes(cursor, 5, 1)]).toEqual([]); + expect(cursor.at).toBe(0); + }); + + test('IO-22/IO-24: after close, any cursor use throws ClosedResourceError, not an EOF', async () => { + const window = windowOver(bytes(1, 2, 3)); + const cursor = window.register(0); + await window.pullThrough(3); + await window.close(); + + expect(() => { + window.assertUsable(); + }).toThrow(ClosedResourceError); + expect(() => window.readInto(cursor, new ByteQueue(), 1)).toThrow( + ClosedResourceError, + ); + }); + + test('IO-41: close is idempotent', async () => { + const window = windowOver(bytes(1)); + await window.close(); + await window.close(); + expect(window.closed).toBe(true); + }); +}); + +describe('RetentionWindow source-contract guards (IO-17)', () => { + /** + * A stream that yields a chunk the TYPE system says cannot occur. The cast is the point: these values + * arrive from a caller-supplied stream, so the compile-time narrowing guarantees nothing at runtime. + */ + const windowOverRaw = (chunk: unknown): RetentionWindow => + new RetentionWindow( + new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(chunk as Uint8Array); + controller.close(); + }, + }).getReader(), + ); + + test('an undefined chunk is an IoError, not a raw TypeError', async () => { + // The `done: false` narrowing is a COMPILE-time guarantee about a value that arrives from a + // caller-supplied stream, so without a runtime check this escapes the IoError tree entirely. + const window = windowOverRaw(undefined); + expect(await rejection(window.pullThrough(1))).toBeInstanceOf( + SourceContractViolationError, + ); + }); + + test('a string chunk is rejected at the boundary, not left to corrupt the queue', async () => { + // What `Readable.toWeb()` yields when the Node stream has an encoding set. `'abc'.length` is 3, so + // a length-only check waves it through and it detonates much later, far from its cause. + const window = windowOverRaw('abc'); + expect(await rejection(window.pullThrough(1))).toBeInstanceOf( + SourceContractViolationError, + ); + }); +}); + +describe('RetentionWindow teardown (IO-41)', () => { + test('close resolves only after the underlying cancel has finished', async () => { + // Detaching the cancel lets `close()` resolve ahead of the real release, racing anything a caller + // sequences on it — connection reuse, shutdown. + const order: string[] = []; + const stream = new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(bytes(1)); + }, + async cancel(): Promise<void> { + await Bun.sleep(10); + order.push('underlying-cancel-finished'); + }, + }); + const window = new RetentionWindow(stream.getReader()); + await window.close(); + order.push('close-returned'); + expect(order).toEqual(['underlying-cancel-finished', 'close-returned']); + }); + + test('a cancel failure propagates instead of being swallowed', async () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(bytes(1)); + }, + cancel(): never { + throw new Error('socket teardown failed'); + }, + }); + const window = new RetentionWindow(stream.getReader()); + expect((await rejection(window.close())).message).toContain( + 'socket teardown failed', + ); + }); + + test('close releases the reader lock, which cancel alone never does', async () => { + const stream = new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(bytes(1)); + }, + }); + const window = new RetentionWindow(stream.getReader()); + expect(stream.locked).toBe(true); + await window.close(); + expect(stream.locked).toBe(false); + }); + + test('overlapping closes share one teardown and one outcome', async () => { + let cancels = 0; + const stream = new ReadableStream<Uint8Array>({ + start(controller): void { + controller.enqueue(bytes(1)); + }, + cancel(): void { + cancels += 1; + }, + }); + const window = new RetentionWindow(stream.getReader()); + await Promise.all([window.close(), window.close()]); + await window.close(); + expect(cancels).toBe(1); + }); +}); diff --git a/packages/core/src/io/retention-window.ts b/packages/core/src/io/retention-window.ts new file mode 100644 index 0000000..c62cb98 --- /dev/null +++ b/packages/core/src/io/retention-window.ts @@ -0,0 +1,202 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/retention-window.ts +import {invariant} from '../invariant.js'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, SourceContractViolationError} from './errors.js'; + +/** + * A reader's position, as a logical offset into the whole stream. Two cursors over one window are + * mutually independent (IO-23): advancing one never moves another. + * + * @internal + */ +export interface Cursor { + at: number; +} + +/** + * The shared buffer behind a `BufferedSource` and all of its peek/slice views. + * + * Bytes are retained from `min(all live cursors)` forward and trimmed as the slowest cursor advances, so + * with no views outstanding retention collapses to the read size. There is deliberately **no cap** here: + * §5 bounds nothing, and every cap the product spec mandates (BODY-19, BODY-30/HTTP-52, BODY-34) sits in + * §6 and belongs to Phase 3b. A cap at this layer would bound the spread between the fastest and slowest + * cursor, which in the divergent case stops a view reaching the end and partially fails IO-19's MUST. + * + * Owns the stream reader, so a view — which owns no reader — can still pull through its parent's source. + * + * @internal + */ +export class RetentionWindow { + readonly #queue = new ByteQueue(); + readonly #cursors = new Set<Cursor>(); + readonly #reader: ReadableStreamDefaultReader<Uint8Array> | undefined; + #retainedFrom = 0; + #pulledThrough = 0; + #exhausted = false; + #closed = false; + #closing: Promise<void> | undefined; + + constructor(reader: ReadableStreamDefaultReader<Uint8Array> | undefined) { + this.#reader = reader; + this.#exhausted = reader === undefined; + } + + /** Logical offset one past the last byte pulled from the stream. */ + get pulledThrough(): number { + return this.#pulledThrough; + } + + /** Bytes currently held because some cursor may still need them. */ + get retainedBytes(): number { + return this.#queue.size; + } + + get closed(): boolean { + return this.#closed; + } + + /** Register a new cursor at a logical offset (IO-23 — its own cursor, independent of every other). */ + register(at: number): Cursor { + this.assertUsable(); + const cursor: Cursor = {at}; + this.#cursors.add(cursor); + return cursor; + } + + /** + * Drop a cursor and let the retained head trim forward (IO-22 — releasing a view neither closes the + * parent nor moves the parent's cursor). + */ + release(cursor: Cursor): void { + this.#cursors.delete(cursor); + if (!this.#closed) this.#trim(); + } + + /** + * Pull from the stream until `offset` bytes are available, or the stream ends. Returns false at end. + */ + async pullThrough(offset: number): Promise<boolean> { + this.assertUsable(); + while (this.#pulledThrough < offset && !this.#exhausted) { + await this.#pullOnce(); + this.assertUsable(); + } + return this.#pulledThrough >= offset; + } + + /** Move up to `count` already-pulled bytes onto `dest`, advancing only `cursor`. */ + readInto(cursor: Cursor, dest: ByteQueue, count: number): number { + this.assertUsable(); + const take = Math.min(count, this.#pulledThrough - cursor.at); + if (take <= 0) return 0; + this.#queue.copyTo(dest, cursor.at - this.#retainedFrom, take); + cursor.at += take; + this.#trim(); + return take; + } + + /** + * Materialize up to `count` already-pulled bytes starting `offset` ahead of `cursor`, without + * advancing it (IO-19, IO-20). + * + * The offset exists so an incremental scanner can re-peek only the tail it has not seen. Without it + * every caller re-materializes the whole scanned prefix on each pull, which is quadratic. + */ + peekBytes(cursor: Cursor, offset: number, count: number): Uint8Array { + this.assertUsable(); + const from = cursor.at + offset; + const take = Math.min(count, this.#pulledThrough - from); + if (take <= 0) return new Uint8Array(0); + return this.#queue.copyOut(from - this.#retainedFrom, take); + } + + /** How many pulled bytes sit at or ahead of `cursor`. Does not pull and does not advance. */ + availableFrom(cursor: Cursor): number { + this.assertUsable(); + return Math.max(0, this.#pulledThrough - cursor.at); + } + + /** IO-24: a closed window fails loudly with a state error, never as a normal EOF. */ + assertUsable(): void { + if (this.#closed) throw new ClosedResourceError('BufferedSource'); + } + + /** + * IO-41: idempotent. IO-22: invalidates every outstanding view, so a later read from one fails loudly + * rather than returning stale bytes. + * + * The returned promise settles only once the underlying reader has actually been cancelled and its + * lock released, and it REJECTS when that teardown fails. Detaching the cancel (`void reader.cancel()`) + * would let `await source.close()` resolve ahead of the real release — racing anything a caller + * sequences on it, such as connection reuse — and would swallow a teardown failure entirely. + * + * Memoized rather than early-returned on a flag: two overlapping `close()` calls must await the SAME + * teardown and observe the SAME outcome, instead of the second resolving while the first is still in + * flight or has already failed. + */ + close(): Promise<void> { + this.#closing ??= this.#teardown(); + return this.#closing; + } + + async #teardown(): Promise<void> { + this.#closed = true; + this.#cursors.clear(); + this.#queue.clear(); + this.#queue.close(); + const reader = this.#reader; + if (reader === undefined) return; + try { + await reader.cancel(); + } finally { + // `cancel()` cancels the STREAM; it never releases the reader's lock — only `releaseLock()` does, + // and without it the caller's ReadableStream stays locked forever. Runs even when the cancel + // rejects, because a stream that failed to cancel is exactly the one whose lock must not leak. + reader.releaseLock(); + } + } + + async #pullOnce(): Promise<void> { + invariant(this.#reader !== undefined, 'pull on a window with no reader'); + const {done, value} = await this.#reader.read(); + if (done) { + this.#exhausted = true; + return; + } + // IO-17: the `done: false` narrowing is a COMPILE-time guarantee about a value that arrives from a + // caller-supplied stream, so it guarantees nothing at runtime. Without this check `undefined` + // escapes as a raw `TypeError` outside the IoError tree, and a string chunk — what + // `Readable.toWeb()` yields when the Node stream has an encoding set — passes the length test and + // corrupts the queue, detonating much later and far from its cause. + if (!(value instanceof Uint8Array)) { + throw new SourceContractViolationError( + `source delivered a non-Uint8Array chunk (${typeof value})`, + ); + } + if (value.length === 0) { + // IO-17: a zero-length delivery for an outstanding read is a source-contract violation, never + // end-of-stream and never something to spin on. + throw new SourceContractViolationError( + 'source delivered 0 bytes without signalling end of stream', + ); + } + this.#queue.writeBytes(value); + this.#pulledThrough += value.length; + } + + /** Drop everything no live cursor can still reach. */ + #trim(): void { + const low = this.#lowestCursor(); + const drop = low - this.#retainedFrom; + if (drop <= 0) return; + this.#queue.skip(drop); + this.#retainedFrom = low; + } + + #lowestCursor(): number { + let low = this.#pulledThrough; + for (const cursor of this.#cursors) low = Math.min(low, cursor.at); + return low; + } +} diff --git a/packages/core/src/io/sink.ts b/packages/core/src/io/sink.ts new file mode 100644 index 0000000..e69b5fa --- /dev/null +++ b/packages/core/src/io/sink.ts @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/sink.ts +import type {ByteQueue} from './byte-queue.js'; + +/** + * The write surface of §5 — what `BufferedSink` and `TeeSink` both are. + * + * This interface exists because `TeeSink` is a `BufferedSink` DECORATOR, and TypeScript has no structural + * escape hatch for that: `BufferedSink` carries `#private` fields, which make it a nominal type, so a + * decorator can never be assignable to it no matter how faithfully it mirrors the API. Without a shared + * interface `writeAll` — the only pump in the package — cannot accept a tee, and tees cannot nest, which + * defeats the body-capture use case IO-25 exists for and silently invites callers to bypass the tap by + * reaching for the primary's bridge instead. + * + * `flush`/`emit` return `Promise<Sink>` rather than `this` because IO-18 only asks that they be chainable; + * each implementation narrows the return to its own type. + * + * @internal + */ +export interface Sink { + /** Whether the sink has been closed or aborted (IO-42). */ + readonly closed: boolean; + + /** Remove exactly `count` bytes from `src`'s head and push them downstream (IO-4). */ + write(src: ByteQueue, count: number): Promise<void>; + + /** Encode and write UTF-8 text (IO-13). */ + writeUtf8(text: string): Promise<void>; + + /** Encode and write text with an explicit charset (IO-13). */ + writeString(text: string, charset: string): Promise<void>; + + /** IO-18: force buffered bytes all the way out toward the destination. */ + flush(): Promise<Sink>; + + /** IO-18: a cheap one-level handoff. */ + emit(): Promise<Sink>; + + /** IO-5, IO-41: closeable and idempotent. */ + close(): Promise<void>; + + /** Discard the destination with a reason rather than committing what was written (IO-42). */ + abort(reason?: unknown): Promise<void>; + + /** A writable host-native byte-stream bridge (IO-16). */ + toWritableStream(): WritableStream<Uint8Array>; +} diff --git a/packages/core/src/io/tee-sink.test.ts b/packages/core/src/io/tee-sink.test.ts new file mode 100644 index 0000000..0ac8c76 --- /dev/null +++ b/packages/core/src/io/tee-sink.test.ts @@ -0,0 +1,346 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/tee-sink.test.ts +// Exercises: IO-25 (mirror into a tap AND forward the full untruncated payload), +// IO-26 (tap capacity limit; unbounded default; a limit of 0 mirrors nothing), +// IO-27 (mirror BEFORE forwarding; staging cleared even on a failed write), +// IO-28 (no direct backing-buffer handle), IO-29 (flush/close/emit forward to the primary only), +// IO-42 (write after close rejects with the source intact), +// IO-13 (the tap mirrors the primary's exact encoded bytes, and refuses a label identically), +// IO-16 (the tee's own writable bridge still feeds the tap), +// IO-3 (a negative count is an argument error, rejected before any transfer; and a non-integral +// tapLimit is one too, rejected at the constructor rather than at the first write) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSink} from './buffered-sink.js'; +import {BufferedSource} from './buffered-source.js'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError} from './errors.js'; +import {TeeSink} from './tee-sink.js'; +import {InvariantViolation} from '../invariant.js'; +import {writeAll} from './pump.js'; +import { + collectingWritableStream, + failingWritableStream, +} from './test-support/fake-stream.js'; +import {rejection} from './test-support/rejection.js'; + +const queueOf = (bytes: Uint8Array): ByteQueue => { + const queue = new ByteQueue(); + queue.writeBytes(bytes); + return queue; +}; + +describe('TeeSink', () => { + test('IO-25: the primary receives the full payload and the tap mirrors it', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.write(queueOf(Uint8Array.from([1, 2, 3])), 3); + await tee.close(); + expect([...written()]).toEqual([1, 2, 3]); + expect([...tee.snapshot()]).toEqual([1, 2, 3]); + }); + + test('IO-26: past the tap limit the tap stops copying but the primary still gets everything', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 2); + await tee.write(queueOf(Uint8Array.from([1, 2, 3, 4, 5])), 5); + await tee.close(); + expect([...written()]).toEqual([1, 2, 3, 4, 5]); + expect([...tee.snapshot()]).toEqual([1, 2]); + }); + + test('IO-26: a limit of 0 mirrors nothing and forwards everything', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 0); + await tee.write(queueOf(Uint8Array.from([1, 2, 3])), 3); + await tee.close(); + expect([...written()]).toEqual([1, 2, 3]); + expect([...tee.snapshot()]).toEqual([]); + }); + + test('IO-26: the default limit mirrors everything', async () => { + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.write(queueOf(new Uint8Array(10_000).fill(7)), 10_000); + await tee.close(); + expect(tee.snapshot().length).toBe(10_000); + }); +}); + +describe('TeeSink mirror-before-forward and lifecycle (IO-27, IO-28, IO-29, IO-42)', () => { + test('IO-27: a failed primary write still captures the attempted bytes in the tap', async () => { + const tee = new TeeSink( + BufferedSink.overStream(failingWritableStream('primary down')), + ); + expect( + (await rejection(tee.write(queueOf(Uint8Array.from([1, 2, 3])), 3))) + .message, + ).toContain('primary down'); + await Promise.resolve(); + expect([...tee.snapshot()]).toEqual([1, 2, 3]); + }); + + test("IO-27: a write following a FAILED write does not prepend the failed write's bytes", async () => { + // The staging buffer is per-call, so this holds structurally — but the assertion has to actually + // drive the failure path to prove it, which is why the first sink is the failing one. + const failing = new TeeSink( + BufferedSink.overStream(failingWritableStream('primary down')), + ); + expect( + (await rejection(failing.write(queueOf(Uint8Array.from([1, 2])), 2))) + .message, + ).toContain('primary down'); + + const {stream, written} = collectingWritableStream(); + const good = new TeeSink(BufferedSink.overStream(stream)); + await good.write(queueOf(Uint8Array.from([3])), 1); + await good.close(); + expect([...written()]).toEqual([3]); + }); + + test('IO-27: consecutive successful writes concatenate without duplication or reordering', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.write(queueOf(Uint8Array.from([1, 2])), 2); + await tee.write(queueOf(Uint8Array.from([3])), 1); + await tee.close(); + expect([...written()]).toEqual([1, 2, 3]); + }); +}); + +describe('TeeSink no-raw-buffer and close (IO-28, IO-29, IO-42)', () => { + test('IO-28: there is no direct backing-buffer handle', () => { + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect(() => tee.buffer).toThrow( + 'TeeSink exposes no backing buffer; use the typed write methods', + ); + }); + + test('IO-29: close forwards to the primary and leaves the tap intact for later snapshotting', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.write(queueOf(Uint8Array.from([1, 2])), 2); + await tee.close(); + expect([...written()]).toEqual([1, 2]); + expect([...tee.snapshot()]).toEqual([1, 2]); + }); + + test('IO-29: flush and emit return the tee and leave the tap intact', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.write(queueOf(Uint8Array.from([1, 2])), 2); + expect(await tee.flush()).toBe(tee); + expect(await tee.emit()).toBe(tee); + expect([...tee.snapshot()]).toEqual([1, 2]); + await tee.close(); + expect([...written()]).toEqual([1, 2]); + }); + + test('IO-29: flush and emit really reach the primary — a closed primary makes both reject', async () => { + // The observable proof that neither is swallowed by the decorator: BufferedSink rejects a flush + // or emit after close (IO-42), so the rejection can only have come from the primary. + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.close(); + expect(await rejection(tee.flush())).toBeInstanceOf(ClosedResourceError); + expect(await rejection(tee.emit())).toBeInstanceOf(ClosedResourceError); + }); +}); + +describe('TeeSink text writes (IO-13, IO-25)', () => { + test('IO-25: writeUtf8 forwards the encoded bytes and mirrors exactly those bytes', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.writeUtf8('héllo ☃'); + await tee.close(); + expect([...tee.snapshot()]).toEqual([...written()]); + expect(new TextDecoder('utf-8').decode(written())).toBe('héllo ☃'); + }); + + test('IO-13: writeString mirrors the charset-encoded bytes, not a UTF-8 re-encoding', async () => { + // 'é' is one byte in ISO-8859-1 and two in UTF-8, so a tap that re-encoded would differ from the + // wire body — the exact divergence the single shared `encodeText` exists to prevent. + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.writeString('hé', 'iso-8859-1'); + await tee.close(); + expect([...written()]).toEqual([0x68, 0xe9]); + expect([...tee.snapshot()]).toEqual([0x68, 0xe9]); + }); + + test('IO-13: an unsupported charset is refused before anything is mirrored or forwarded', async () => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect( + (await rejection(tee.writeString('x', 'shift_jis'))).message, + ).toContain( + 'unsupported write charset: shift_jis (only utf-8 and iso-8859-1 can be encoded)', + ); + await tee.close(); + expect([...tee.snapshot()]).toEqual([]); + expect([...written()]).toEqual([]); + }); + + test('IO-42: write after close rejects and leaves the source intact', async () => { + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.close(); + const source = queueOf(Uint8Array.from([1, 2])); + expect(await rejection(tee.write(source, 2))).toBeInstanceOf( + ClosedResourceError, + ); + expect(source.size).toBe(2); + expect([...tee.snapshot()]).toEqual([]); + }); + + test('IO-25 property: the primary always receives the exact concatenation of every written byte', async () => { + // The single most important property in §5: logging never reduces the wire body, whatever the cap. + await fc.assert( + fc.asyncProperty( + fc.array(fc.uint8Array({maxLength: 32}), {maxLength: 8}), + fc.integer({min: 0, max: 64}), + async (writes, tapLimit) => { + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), tapLimit); + for (const chunk of writes) + await tee.write(queueOf(chunk), chunk.length); + await tee.close(); + + const expected = writes.flatMap(chunk => [...chunk]); + expect([...written()]).toEqual(expected); + expect(tee.snapshot().length).toBe( + Math.min(tapLimit, expected.length), + ); + }, + ), + ); + }); +}); + +describe('TeeSink as a first-class sink (IO-16, IO-25)', () => { + test('a tee is accepted anywhere a sink is, including by the pump and by another tee', async () => { + // `BufferedSink`'s #private fields make it a NOMINAL type, so a decorator can never be assignable + // to it. Without a shared interface the only pump in the package cannot take a tee and tees cannot + // nest — which defeats the body capture IO-25 exists for. + const {stream, written} = collectingWritableStream(); + const inner = new TeeSink(BufferedSink.overStream(stream)); + const outer = new TeeSink(inner); + const total = await writeAll( + BufferedSource.overBytes(Uint8Array.from([1, 2, 3])), + outer, + ); + expect(total).toBe(3); + expect([...written()]).toEqual([1, 2, 3]); + expect([...outer.snapshot()]).toEqual([1, 2, 3]); + expect([...inner.snapshot()]).toEqual([1, 2, 3]); + }); + + test('IO-16: the tee exposes its own bridge, so bridged bytes still reach the tap', async () => { + // Handing callers the primary's bridge instead would route every byte written through it past the + // tap, silently producing an empty capture. + const {stream, written} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + const writer = tee.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([7, 8])); + await writer.close(); + expect([...written()]).toEqual([7, 8]); + expect([...tee.snapshot()]).toEqual([7, 8]); + }); + + test('IO-16: aborting the tee bridge aborts the primary with the reason', async () => { + const {stream, wasAborted, abortReason} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + const reason = new Error('cancelled'); + await tee.toWritableStream().abort(reason); + expect(wasAborted()).toBe(true); + expect(abortReason()).toBe(reason); + }); + + test('a failed primary write leaves the caller its bytes, and the tap records the attempt', async () => { + // IO-27 requires the tap capture the ATTEMPTED bytes, so it keeps them either way; what must not + // happen is `src` being drained by a write that never reached the wire. + const tee = new TeeSink( + BufferedSink.overStream(failingWritableStream('boom')), + ); + const source = queueOf(Uint8Array.from([1, 2, 3])); + expect((await rejection(tee.write(source, 3))).message).toContain('boom'); + expect(source.size).toBe(3); + expect([...source.snapshot()]).toEqual([1, 2, 3]); + expect([...tee.snapshot()]).toEqual([1, 2, 3]); + }); + + test('an empty payload produces the same chunk sequence as the sink and the bridge', async () => { + const {stream, chunkSizes} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + await tee.writeUtf8(''); + expect(chunkSizes()).toEqual([]); + expect(tee.snapshot().length).toBe(0); + }); +}); + +describe('argument validation (IO-3)', () => { + test('a negative count is rejected before the source or the tap is touched', async () => { + // Previously reached the tee unchecked and was rejected only indirectly, by whichever ByteQueue + // call happened to run first -- and not at all on the count === 0 and short-source early returns. + const {stream, chunkSizes} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + const source = queueOf(Uint8Array.from([1, 2, 3])); + + expect((await rejection(tee.write(source, -1))).name).toBe( + 'InvariantViolation', + ); + expect(source.size).toBe(3); + expect(tee.snapshot().length).toBe(0); + expect(chunkSizes()).toEqual([]); + }); + + test('a negative count is rejected even when the source is empty', async () => { + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect((await rejection(tee.write(new ByteQueue(), -1))).name).toBe( + 'InvariantViolation', + ); + }); + + // The constructor checked `tapLimit >= 0` only, so a fractional cap was accepted and the fault + // surfaced at the FIRST write instead: `#mirror` computes `room = tapLimit - tap.size`, hands it + // to `ByteQueue.copyTo`, and `assertCount` rejects it as `count must be a non-negative integer, + // got 2.5` — a message about the wrong parameter, at the wrong call, on a path where the primary + // write has not run yet. A byte count is integral for the same reason `count` is (IO-3, IO-26). + // Measured on the pre-fix tree, audit #67 / #76. + test.each([2.5, 0.5, -0.5, Number.NaN, Number.NEGATIVE_INFINITY])( + 'a tapLimit of %p is rejected at the constructor', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).toThrow(InvariantViolation); + }, + ); + + test('the message names tapLimit, not count', () => { + const {stream} = collectingWritableStream(); + expect(() => new TeeSink(BufferedSink.overStream(stream), 2.5)).toThrow( + /tapLimit/, + ); + }); + + test.each([0, 1, 4096, Number.POSITIVE_INFINITY])( + 'a tapLimit of %p is still accepted', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).not.toThrow(); + }, + ); + + test('the unbounded default stays Infinity, which is not an integer', () => { + // `Number.isInteger(Infinity)` is false, so the check has to admit it explicitly — it is the + // documented default and the cap `#mirror` reads as "no cap" (IO-26). + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect(tee.snapshot().length).toBe(0); + }); +}); diff --git a/packages/core/src/io/tee-sink.ts b/packages/core/src/io/tee-sink.ts new file mode 100644 index 0000000..83b1d15 --- /dev/null +++ b/packages/core/src/io/tee-sink.ts @@ -0,0 +1,169 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/tee-sink.ts +import {invariant} from '../invariant.js'; +import {ByteQueue} from './byte-queue.js'; +import {ClosedResourceError, EndOfStreamError, IoError} from './errors.js'; +import {assertCount} from './limits.js'; +import type {Sink} from './sink.js'; +import {encodeText} from './text-codec.js'; + +/** + * A sink that mirrors written bytes into a bounded in-memory tap while forwarding the full, untruncated + * payload to its primary (IO-25–IO-29). + * + * Built as a plain `Sink` decorator rather than on `TransformStream`, which `sdk-design/03` §3.1 + * sketches: a `TransformStream`'s own queueing and backpressure semantics muddy IO-27's + * mirror-before-forward ordering, the clause most easily gotten wrong. §3.1's substantive point — that + * the platform's `ReadableStream.tee()` solves a different problem (duplicating a *readable* for two + * consumers, not mirroring a *sink's* writes) — is why no platform primitive is used at all. + * + * Decorates the `Sink` INTERFACE, not `BufferedSink` itself, so a tee is usable everywhere a sink is — + * `writeAll`, another tee — and so it can offer its own bridge instead of forcing callers to reach for + * the primary's, which would route every byte past the tap. + * + * The tap has no cap by default. §5 bounds nothing; BODY-19 and BODY-34 set the real cap in Phase 3b. + * + * @internal + */ +export class TeeSink implements Sink { + readonly #primary: Sink; + readonly #tap = new ByteQueue(); + readonly #tapLimit: number; + + constructor(primary: Sink, tapLimit: number = Number.POSITIVE_INFINITY) { + // Integrality, not merely `>= 0`. A tap cap is a byte count, so IO-3's rule for `count` applies + // to it — and a fractional cap was not merely odd, it was deferred: `#mirror` computes + // `room = tapLimit - tap.size` and hands it to `ByteQueue.copyTo`, where `assertCount` rejects + // it as "count must be a non-negative integer, got 2.5". That names the wrong parameter, fires + // at the first write rather than at the construction that supplied it, and does so before the + // primary write on a path that has already taken bytes from the caller. `Infinity` is admitted + // explicitly: it is not an integer, and it is the documented unbounded default (IO-26). + // Audit #67 / #76. + invariant( + (Number.isInteger(tapLimit) && tapLimit >= 0) || + tapLimit === Number.POSITIVE_INFINITY, + `tapLimit must be a non-negative integer or Infinity, got ${String(tapLimit)}`, + ); + this.#primary = primary; + this.#tapLimit = tapLimit; + } + + /** Tracks the primary, which owns the real destination. */ + get closed(): boolean { + return this.#primary.closed; + } + + /** + * IO-28: a raw buffer write would reach only the tap or only the primary and silently corrupt the wire + * body, so no such handle exists. + */ + get buffer(): never { + throw new IoError( + 'TeeSink exposes no backing buffer; use the typed write methods', + ); + } + + /** + * Mirror into the tap, then forward the full payload to the primary (IO-25, IO-27). + * + * `src` is drained only after the primary write resolves, so a caller that catches a failed write + * still holds its bytes. The tap deliberately keeps the attempted bytes either way — IO-27 requires + * exactly that, so the tap records what was ATTEMPTED, not what reached the wire. + */ + async write(src: ByteQueue, count: number): Promise<void> { + // IO-3: a negative or non-integer count is an argument error, rejected here rather than left to + // whichever `ByteQueue` call happens to run first -- which reported it only as a side effect of + // `copyTo`, and not at all on the `count === 0` and short-source paths that return early. + assertCount(count); + // IO-42: reject before consuming from `src` or touching the tap. + if (this.#primary.closed) throw new ClosedResourceError('TeeSink'); + if (src.size < count) throw new EndOfStreamError(src.size, count); + if (count === 0) return; + const staging = new ByteQueue(); + src.copyTo(staging, 0, count); + // IO-27: mirror BEFORE forwarding, so a failed primary write still captures the attempted bytes. + this.#mirror(staging); + try { + // IO-27: the staging buffer is cleared even on a failed primary write, so a later write cannot + // prepend stale bytes. + await this.#primary.write(staging, count); + } finally { + staging.clear(); + } + src.skip(count); + } + + /** Mirror and forward UTF-8 text (IO-25). */ + async writeUtf8(text: string): Promise<void> { + return this.writeString(text, 'utf-8'); + } + + /** + * Mirror and forward text with an explicit charset (IO-25). + * + * Encodes once, through the shared `encodeText`, then routes the bytes down the normal `write` path. + * That guarantees the tap mirrors exactly the bytes the primary emits — not a UTF-8 re-encoding of + * them — and that an unsupported charset is refused identically on both sides. + */ + async writeString(text: string, charset: string): Promise<void> { + const encoded = new ByteQueue(); + encoded.writeBytes(encodeText(text, charset)); + return this.write(encoded, encoded.size); + } + + /** A non-consuming copy of the tap's contents. */ + snapshot(): Uint8Array { + return this.#tap.snapshot(); + } + + /** IO-29: forwards to the PRIMARY only, leaving the tap intact. */ + async flush(): Promise<TeeSink> { + await this.#primary.flush(); + return this; + } + + /** IO-29: forwards to the PRIMARY only, leaving the tap intact. */ + async emit(): Promise<TeeSink> { + await this.#primary.emit(); + return this; + } + + /** IO-29: forwards to the PRIMARY only; the tap survives for later snapshotting. */ + async close(): Promise<void> { + await this.#primary.close(); + } + + /** IO-29: forwards to the PRIMARY only; the tap survives, recording what was attempted. */ + async abort(reason?: unknown): Promise<void> { + await this.#primary.abort(reason); + } + + /** + * A writable host-native byte-stream bridge (IO-16) that still feeds the tap. + * + * Routed through this tee's own `write`, not the primary's bridge — handing callers the primary's + * would mean every byte written through it bypasses the tap, silently producing an empty capture. + */ + toWritableStream(): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write: async (chunk): Promise<void> => { + const staging = new ByteQueue(); + staging.writeBytes(chunk); + await this.write(staging, staging.size); + }, + close: async (): Promise<void> => { + await this.close(); + }, + abort: async (reason: unknown): Promise<void> => { + await this.abort(reason); + }, + }); + } + + /** IO-26: copy until the cap is reached, then stop copying while the payload still forwards. */ + #mirror(staging: ByteQueue): void { + const room = this.#tapLimit - this.#tap.size; + if (room <= 0) return; + staging.copyTo(this.#tap, 0, Math.min(room, staging.size)); + } +} diff --git a/packages/core/src/io/test-support/fake-stream.ts b/packages/core/src/io/test-support/fake-stream.ts new file mode 100644 index 0000000..3d208b6 --- /dev/null +++ b/packages/core/src/io/test-support/fake-stream.ts @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/test-support/fake-stream.ts +// Test-only. Excluded from the build (tsconfig.build.json) and never exported from any barrel. +// Styleguide 11.3: fake your own interfaces rather than reaching for mock.module. + +/** A readable stream that yields exactly the chunks given, at exactly those boundaries. */ +export function fakeReadableStream( + chunks: readonly Uint8Array[], + onCancel?: () => void, +): ReadableStream<Uint8Array> { + let index = 0; + return new ReadableStream<Uint8Array>({ + cancel(): void { + onCancel?.(); + }, + pull(controller): void { + if (index >= chunks.length) { + controller.close(); + return; + } + const chunk = chunks[index]; + index += 1; + if (chunk !== undefined) controller.enqueue(chunk); + }, + }); +} + +/** A readable stream that violates the read protocol by yielding an empty chunk (drives IO-17). */ +export function protocolViolatingStream(): ReadableStream<Uint8Array> { + return fakeReadableStream([new Uint8Array(0)]); +} + +/** A writable stream that accumulates everything written, for asserting the wire payload. */ +export function collectingWritableStream(): { + stream: WritableStream<Uint8Array>; + written: () => Uint8Array; + chunkSizes: () => number[]; + isClosed: () => boolean; + abortReason: () => unknown; + wasAborted: () => boolean; +} { + const parts: Uint8Array[] = []; + let closed = false; + let aborted = false; + let abortReason: unknown = undefined; + const stream = new WritableStream<Uint8Array>({ + write(chunk): void { + parts.push(chunk.slice()); + }, + close(): void { + closed = true; + }, + abort(reason: unknown): void { + aborted = true; + abortReason = reason; + }, + }); + const written = (): Uint8Array => { + const total = parts.reduce((sum, part) => sum + part.length, 0); + const out = new Uint8Array(total); + let at = 0; + for (const part of parts) { + out.set(part, at); + at += part.length; + } + return out; + }; + return { + stream, + written, + chunkSizes: () => parts.map(part => part.length), + isClosed: () => closed, + abortReason: () => abortReason, + wasAborted: () => aborted, + }; +} + +/** A writable stream whose writes stay pending until released, for observing emit/flush ordering. */ +export function gatedWritableStream(): { + stream: WritableStream<Uint8Array>; + delivered: () => number; + release: () => void; +} { + let delivered = 0; + let open: (() => void) | undefined; + const gate = new Promise<void>(resolve => { + open = resolve; + }); + const stream = new WritableStream<Uint8Array>({ + async write(chunk): Promise<void> { + await gate; + delivered += chunk.length; + }, + }); + return { + stream, + delivered: () => delivered, + release: () => open?.(), + }; +} + +/** A writable stream whose `close` rejects, for asserting teardown-failure behavior. */ +export function failingCloseWritableStream( + message: string, +): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + close(): never { + throw new Error(message); + }, + }); +} + +/** A writable stream whose first write rejects, for asserting failure-path behavior. */ +export function failingWritableStream( + message: string, +): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write(): never { + throw new Error(message); + }, + }); +} + +/** + * Read a stream to completion and return everything it yielded. + * + * Hand-rolled rather than `new Response(stream).arrayBuffer()`: `Response` is a restricted global here, + * since in this package the name belongs to the SDK's own HTTP model. + */ +export async function drainStream( + stream: ReadableStream<Uint8Array>, +): Promise<Uint8Array> { + const reader = stream.getReader(); + const parts: Uint8Array[] = []; + let total = 0; + for (;;) { + const {done, value} = await reader.read(); + if (done) break; + parts.push(value); + total += value.length; + } + const out = new Uint8Array(total); + let at = 0; + for (const part of parts) { + out.set(part, at); + at += part.length; + } + return out; +} diff --git a/packages/core/src/io/test-support/rejection.test.ts b/packages/core/src/io/test-support/rejection.test.ts new file mode 100644 index 0000000..5c83f30 --- /dev/null +++ b/packages/core/src/io/test-support/rejection.test.ts @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/test-support/rejection.test.ts +// Exercises the `rejection()` test helper's own failure paths, not covered by its many callers +// (which all reject with a real Error). +import {describe, expect, test} from 'bun:test'; +import {rejection} from './rejection.js'; + +describe('rejection', () => { + test('returns the rejection reason when the promise rejects with an Error', async () => { + const error = new Error('boom'); + expect(await rejection(Promise.reject(error))).toBe(error); + }); + + test('throws when the promise rejects with a non-Error value', async () => { + let caught: unknown; + try { + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- exercising rejection()'s non-Error branch + await rejection(Promise.reject('not an error')); + } catch (e: unknown) { + caught = e; + } + expect(caught).toBeInstanceOf(Error); + expect((caught as Error).message).toBe( + 'expected an Error rejection, got string', + ); + }); + + test('throws when the promise resolves instead of rejecting', async () => { + let caught: unknown; + try { + await rejection(Promise.resolve('fine')); + } catch (e: unknown) { + caught = e; + } + expect(caught).toBeInstanceOf(Error); + expect((caught as Error).message).toBe( + 'expected the promise to reject, but it resolved', + ); + }); +}); diff --git a/packages/core/src/io/test-support/rejection.ts b/packages/core/src/io/test-support/rejection.ts new file mode 100644 index 0000000..94dfb38 --- /dev/null +++ b/packages/core/src/io/test-support/rejection.ts @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/test-support/rejection.ts +// Test-only. Excluded from the build (tsconfig.build.json) and never exported from any barrel. + +/** + * Await a promise that must reject, and return the rejection reason. + * + * Why this exists rather than `expect(promise).rejects.toThrow(...)`: bun types `rejects` as + * `Matchers<unknown>`, whose `toThrow()` returns `void` even though at run time it returns a promise. + * So `await`ing it fails `@typescript-eslint/await-thenable`, and omitting the `await` leaves the + * assertion racing test teardown — bun still fails the run, but the failure can attribute to a later + * test. Capturing the rejection keeps every failure awaited and attributable, with no lint suppression. + */ +export async function rejection(promise: Promise<unknown>): Promise<Error> { + try { + await promise; + } catch (e: unknown) { + if (e instanceof Error) return e; + throw new Error(`expected an Error rejection, got ${typeof e}`); + } + throw new Error('expected the promise to reject, but it resolved'); +} diff --git a/packages/core/src/io/text-codec.ts b/packages/core/src/io/text-codec.ts new file mode 100644 index 0000000..43df212 --- /dev/null +++ b/packages/core/src/io/text-codec.ts @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/io/text-codec.ts +import {IoError} from './errors.js'; + +/** + * The single source of truth for both directions of IO-13's text encoding. + * + * Lives in its own module because encode and decode must agree byte for byte: IO-13 requires the two to + * round-trip, and the only way to guarantee that is to derive them from one table. `BufferedSink`, + * `TeeSink` and `BufferedSource` all route through here rather than each reaching for the platform. + * + * @internal + */ + +/** Charset labels this package encodes and decodes itself rather than delegating to the platform. */ +const LATIN1_LABELS = new Set([ + 'iso-8859-1', + 'latin1', + 'iso8859-1', + 'iso_8859-1', +]); +const UTF8_LABELS = new Set(['utf-8', 'utf8', 'unicode-1-1-utf-8']); + +/** + * Encode `text` for the wire (IO-13). + * + * ISO-8859-1 is a direct code-point-to-byte map for 0–255; anything above is not representable. + * `TextEncoder` is UTF-8-only — there is no `TextEncoder('iso-8859-1')` — and SEAM-1 forbids an encoding + * dependency, so any other label throws rather than silently re-encoding as UTF-8, which would corrupt + * the bytes on the wire. + */ +export function encodeText(text: string, charset: string): Uint8Array { + const normalized = charset.toLowerCase(); + if (UTF8_LABELS.has(normalized)) return new TextEncoder().encode(text); + if (!LATIN1_LABELS.has(normalized)) { + throw new IoError( + `unsupported write charset: ${charset} (only utf-8 and iso-8859-1 can be encoded)`, + ); + } + const out = new Uint8Array(text.length); + for (let i = 0; i < text.length; i += 1) { + const code = text.charCodeAt(i); + if (code > 0xff) { + throw new IoError( + `code point ${String(code)} is not representable in ${charset}`, + ); + } + out[i] = code; + } + return out; +} + +/** + * Decode `bytes` that arrived from the wire (IO-13). + * + * ISO-8859-1 is decoded HERE rather than through `TextDecoder`, deliberately. The WHATWG Encoding + * Standard maps the labels `iso-8859-1` and `latin1` onto windows-1252, so + * `new TextDecoder('iso-8859-1').encoding === 'windows-1252'` — which reinterprets 0x80–0x9F as typographic + * characters (0x80 becomes U+20AC EUR). That breaks IO-13's mandated symmetry in both directions: bytes + * written by `encodeText` do not come back, and text decoded that way cannot be re-encoded at all, + * because the substituted code points are above 0xFF. The direct byte-to-code-point map is the actual + * ISO-8859-1 the write side implements. + * + * Every other label goes to `TextDecoder`, which is correct for them. + * + * NOT interchangeable with `http/charset.ts`'s `decodeBodyText`, which decodes a whole message body at + * the HTTP layer, delegates `iso-8859-1` to `TextDecoder`'s windows-1252 mapping, and consumes a leading + * BOM. This one is per-fragment decoding at the byte layer. See that function's note for the full split. + * + * `ignoreBOM: true` is REQUIRED, not incidental. The decoder is applied per fragment — per line, per + * counted read — so the default (strip a leading U+FEFF) deletes a BOM anywhere a fragment happens to + * begin, not just at the start of a stream. That silently drops the first three bytes of a body, breaking + * content hashing and signature verification, and it makes SSE-12 ("any BOM later in the stream MUST be + * preserved as ordinary data") unimplementable in Phase 6b, because the byte is gone before the SSE + * parser ever sees the line. Consuming a single start-of-stream BOM belongs to whoever knows where the + * stream starts; it is not this function's business. Do not turn this flag off. + */ +export function decodeText(bytes: Uint8Array, charset: string): string { + const normalized = charset.toLowerCase(); + if (LATIN1_LABELS.has(normalized)) { + // Chunked because `String.fromCharCode(...bytes)` overflows the call stack on a large body. + let out = ''; + for (let at = 0; at < bytes.length; at += LATIN1_CHUNK) { + out += String.fromCharCode(...bytes.subarray(at, at + LATIN1_CHUNK)); + } + return out; + } + return decoderFor(charset).decode(bytes); +} + +const LATIN1_CHUNK = 8192; + +function decoderFor(charset: string): TextDecoder { + try { + return new TextDecoder(charset, {ignoreBOM: true}); + } catch (e: unknown) { + // A charset label reaching this layer is internal, so this is an argument error, not boundary + // data. Phase 3b's HTTP-42 owns the "unknown declared charset falls back to UTF-8" rule. + throw new IoError(`unsupported charset: ${charset}`, {cause: e}); + } +} + +/** Whether `charset` can be decoded at all — used to reject a bad label before any bytes are consumed. */ +export function assertDecodable(charset: string): void { + if (LATIN1_LABELS.has(charset.toLowerCase())) return; + decoderFor(charset); +} diff --git a/packages/core/src/observability/diagnostic-context.test.ts b/packages/core/src/observability/diagnostic-context.test.ts new file mode 100644 index 0000000..ba80ff5 --- /dev/null +++ b/packages/core/src/observability/diagnostic-context.test.ts @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/diagnostic-context.test.ts +// Exercises: OBS-10 (default allow-list {trace.id, span.id}, null allow-list folds all, null values skipped), +// OBS-24 (immutable snapshot bridge: capture, reinstall, restore including on throw), OBS-22/OBS-23's +// scoping mechanics (AsyncScopedStore.run unwinds across an await and over an enter() its callback made, +// where the enter() handle alone does not). +import {describe, expect, test} from 'bun:test'; +import { + captureDiagnosticSnapshot, + createAsyncScopedStore, + getDiagnosticContext, + pushDiagnosticFields, + runWithSnapshot, + withDiagnosticFields, +} from './diagnostic-context.js'; + +describe('allow-list folding (OBS-10)', () => { + test('only trace.id/span.id fold by default', () => { + withDiagnosticFields( + {'trace.id': 't1', 'span.id': 's1', 'app.custom': 'x'}, + () => { + const folded = getDiagnosticContext(['trace.id', 'span.id']); + expect(folded).toEqual({'trace.id': 't1', 'span.id': 's1'}); + }, + ); + }); + + test('a null allow-list folds every present key', () => { + withDiagnosticFields({'trace.id': 't1', 'app.custom': 'x'}, () => { + const folded = getDiagnosticContext(null); + expect(folded).toEqual({'trace.id': 't1', 'app.custom': 'x'}); + }); + }); + + test('keys with null or undefined values are skipped (OBS-10)', () => { + withDiagnosticFields( + { + 'trace.id': 't1', + 'span.id': null as unknown as string, + 'app.null': null as unknown as string, + }, + () => { + const folded = getDiagnosticContext(null); + expect(folded).toEqual({'trace.id': 't1'}); + expect('span.id' in folded).toBe(false); + expect('app.null' in folded).toBe(false); + }, + ); + }); + + test('prototype keys do not cause prototype pollution', () => { + withDiagnosticFields( + {['__proto__']: 'polluted', constructor: 'hacked'}, + () => { + const folded = getDiagnosticContext(null); + expect(Object.getPrototypeOf(folded)).toBe(Object.prototype); + expect(Object.hasOwn(Object.prototype, 'polluted')).toBe(false); + expect(Object.hasOwn(folded, '__proto__')).toBe(true); + expect(folded.__proto__).toBe('polluted'); + }, + ); + }); + + test('withDiagnosticFields and pushDiagnosticFields reject non-object inputs', () => { + expect(() => { + withDiagnosticFields( + null as unknown as Record<string, string>, + () => undefined, + ); + }).toThrow(); + expect(() => + pushDiagnosticFields(null as unknown as Record<string, string>), + ).toThrow(); + }); + + test('outside any withDiagnosticFields scope, the context is empty', () => { + expect(getDiagnosticContext(null)).toEqual({}); + }); +}); + +describe('async propagation', () => { + test('the context is visible after an await inside the scope', async () => { + await withDiagnosticFields({'trace.id': 't1'}, async () => { + await Promise.resolve(); + expect(getDiagnosticContext(null)['trace.id']).toBe('t1'); + }); + }); + + test('nested scopes restore the outer context on exit', () => { + withDiagnosticFields({'trace.id': 'outer'}, () => { + withDiagnosticFields({'trace.id': 'inner'}, () => { + expect(getDiagnosticContext(null)['trace.id']).toBe('inner'); + }); + expect(getDiagnosticContext(null)['trace.id']).toBe('outer'); + }); + }); +}); + +describe('snapshot bridge (OBS-24)', () => { + test('captures on the originating call and reinstalls on a detached callback', () => { + let capturedInsideBridge: string | undefined; + withDiagnosticFields({'trace.id': 'bridged'}, () => { + const snapshot = captureDiagnosticSnapshot(); + // Simulate a callback invoked outside the tracked continuation (e.g. a raw event-emitter callback). + setImmediate(() => { + runWithSnapshot(snapshot, () => { + capturedInsideBridge = getDiagnosticContext(null)['trace.id']; + }); + }); + }); + return new Promise<void>(resolve => { + setImmediate(() => { + expect(capturedInsideBridge).toBe('bridged'); + resolve(); + }); + }); + }); + + test('restores the prior context after the bridge, including when the guarded block throws', () => { + withDiagnosticFields({'trace.id': 'prior'}, () => { + const snapshot = captureDiagnosticSnapshot(); + expect(() => { + runWithSnapshot(snapshot, () => { + throw new Error('boom'); + }); + }).toThrow('boom'); + expect(getDiagnosticContext(null)['trace.id']).toBe('prior'); + }); + }); +}); + +describe('pushDiagnosticFields', () => { + test('pushes fields into scope and restores prior store on call handle', () => { + withDiagnosticFields({'trace.id': 'prior'}, () => { + const restore = pushDiagnosticFields({ + 'trace.id': 'pushed', + 'span.id': 's1', + }); + expect(getDiagnosticContext(null)).toEqual({ + 'trace.id': 'pushed', + 'span.id': 's1', + }); + restore(); + expect(getDiagnosticContext(null)).toEqual({'trace.id': 'prior'}); + // restore is idempotent + restore(); + expect(getDiagnosticContext(null)).toEqual({'trace.id': 'prior'}); + }); + }); + + test('top-level pushDiagnosticFields restores empty context after handle is closed', () => { + expect(getDiagnosticContext(null)).toEqual({}); + const restore = pushDiagnosticFields({'trace.id': 't-top'}); + expect(getDiagnosticContext(null)['trace.id']).toBe('t-top'); + restore(); + expect(getDiagnosticContext(null)).toEqual({}); + }); +}); + +describe('createAsyncScopedStore', () => { + test('run installs the value for the callback and restores across an await', async () => { + const store = createAsyncScopedStore<string>(); + expect(store.get()).toBeUndefined(); + + await store.run('scoped', async () => { + expect(store.get()).toBe('scoped'); + await Promise.resolve(); + expect(store.get()).toBe('scoped'); + }); + + expect(store.get()).toBeUndefined(); + }); + + test('run unwinds an enter() its callback left open, and restores on a throw', () => { + const store = createAsyncScopedStore<string>(); + store.run('outer', () => { + // The handle form, deliberately never closed -- what the LOGGING pillar's correlation scope + // effectively does once its close() lands in a later continuation. + store.enter('inner'); + expect(store.get()).toBe('inner'); + }); + expect(store.get()).toBeUndefined(); + + expect(() => { + store.run('outer', () => { + store.enter('inner'); + throw new Error('boom'); + }); + }).toThrow('boom'); + expect(store.get()).toBeUndefined(); + }); + + test('enter installs value and returned restore function resets prior value', () => { + const store = createAsyncScopedStore<string>(); + expect(store.get()).toBeUndefined(); + const restore = store.enter('val1'); + expect(store.get()).toBe('val1'); + restore(); + expect(store.get()).toBeUndefined(); + // restore is idempotent + restore(); + expect(store.get()).toBeUndefined(); + }); +}); diff --git a/packages/core/src/observability/diagnostic-context.ts b/packages/core/src/observability/diagnostic-context.ts new file mode 100644 index 0000000..4c0ecf5 --- /dev/null +++ b/packages/core/src/observability/diagnostic-context.ts @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/diagnostic-context.ts +// The one sanctioned node: import in this phase (see Global Constraints) -- AsyncLocalStorage has no +// cross-runtime equivalent, and this is the only mechanism Node offers for OBS-24's async-boundary bridge. +import {AsyncLocalStorage} from 'node:async_hooks'; +import {invariant} from '../invariant.js'; + +type DiagnosticStore = ReadonlyMap<string, string>; + +const storage = new AsyncLocalStorage<DiagnosticStore>(); + +/** OBS-24: an immutable, shareable snapshot of the diagnostic context at capture time. */ +export interface DiagnosticSnapshot { + readonly store: DiagnosticStore; +} + +/** + * OBS-24 (partial, by construction): AsyncLocalStorage already auto-propagates its store across `await`, + * promise chains, and timers via async_hooks, covering most of the reference's manual thread-local-bridge + * requirement for free. Pushes `fields` for the duration of `fn`, restoring the prior store afterward + * (including on throw, via AsyncLocalStorage.run's own guarantee). + */ +export function withDiagnosticFields<T>( + fields: Readonly<Record<string, string>>, + fn: () => T, +): T { + invariant( + typeof fields === 'object' && (fields as unknown) !== null, + 'withDiagnosticFields: fields must be an object', + ); + const current = storage.getStore() ?? new Map<string, string>(); + const next = new Map(current); + for (const [key, value] of Object.entries(fields)) next.set(key, value); + return storage.run(next, fn); +} + +/** OBS-10: default allow-list is exactly {trace.id, span.id}; null allow-list folds every present key. */ +export function getDiagnosticContext( + allowList: readonly string[] | null, +): Readonly<Record<string, string>> { + const store = storage.getStore(); + if (store === undefined) return {}; + const keys = allowList ?? [...store.keys()]; + const result: Record<string, string> = {}; + for (const key of keys) { + const value = store.get(key); + if (value !== undefined && (value as unknown) !== null) { + // Use defineProperty to safely set keys (including __proto__) without mutating Object.prototype. + Object.defineProperty(result, key, { + value, + enumerable: true, + writable: true, + configurable: true, + }); + } + } + return result; +} + +/** + * OBS-24's explicit bridge, for the residual case AsyncLocalStorage's automatic propagation doesn't cover: a + * callback invoked from outside the tracked continuation chain entirely (e.g. a raw event-emitter callback, + * or `setImmediate`/a third-party callback API that isn't `await`ed from the capturing scope). + */ +export function captureDiagnosticSnapshot(): DiagnosticSnapshot { + return {store: storage.getStore() ?? new Map()}; +} + +export function runWithSnapshot<T>( + snapshot: DiagnosticSnapshot, + fn: () => T, +): T { + return storage.run(snapshot.store, fn); +} + +/** + * The scope-handle form of `withDiagnosticFields`, for callers that cannot express their scope as a single + * callback -- OBS-23's span-correlation scope is one: a pipeline step pushes before `await next(...)` and + * restores after, with the two halves in different statements. Returns the restore function. + * + * **The restore reaches only the continuation that called it.** `enterWith` installs the store on the + * *current* async resource and every resource created from it; the returned function does the same with + * the previous store. Call it after an `await` and it runs on a different resource, so the caller that + * pushed keeps the pushed fields for the rest of its own continuation -- which is a leak when that caller + * is a library entry point and the continuation is the application's. Use `withDiagnosticFields` whenever + * the scope CAN be written as one callback; that form is `AsyncLocalStorage.run`, which restores on exit + * by construction. `Runtime.send` used this handle until 2026-09-05 and leaked `trace.id`/`span.id` into + * every subsequent application log (audit #67 / #80). + */ +export function pushDiagnosticFields( + fields: Readonly<Record<string, string>>, +): () => void { + invariant( + typeof fields === 'object' && (fields as unknown) !== null, + 'pushDiagnosticFields: fields must be an object', + ); + const previous = storage.getStore(); + const next = new Map(previous ?? []); + for (const [key, value] of Object.entries(fields)) next.set(key, value); + storage.enterWith(next); + + let restored = false; + return (): void => { + if (restored) return; + restored = true; + storage.enterWith(previous as unknown as DiagnosticStore); + }; +} + +/** + * `node:async_hooks` is confined to this file (see Global Constraints), so any other module needing + * async-scoped storage — `tracing.ts`'s current-span slot is the only one this phase adds — takes it from + * here rather than importing `AsyncLocalStorage` a second time. + */ +export interface AsyncScopedStore<T> { + get(): T | undefined; + /** + * Installs `value` for the rest of this async context; the returned function restores the prior value. + * + * Carries `pushDiagnosticFields`' caveat verbatim: the restore is an `enterWith` of its own, so it takes + * effect only on the async resource that runs it. A handle closed after an `await` leaves `value` + * installed on the resource that entered it. Prefer {@link AsyncScopedStore.run} for any scope that can + * be written as one callback. + */ + enter(value: T): () => void; + /** + * Runs `fn` with `value` installed, restoring whatever was installed before when `fn` returns -- + * including on a throw, and including for anything `fn` itself entered with the handle form. This is + * `AsyncLocalStorage.run`, so the restore is structural rather than a call a later continuation has to + * remember to make, and it is what a library entry point must use if the caller's context is to survive + * the call. + * + * `fn`'s return value is passed through untouched: an `async` callback hands back its promise, and the + * store is restored when the callback's synchronous prefix returns, not when the promise settles. That + * is the intended scoping -- everything the promise chain does inherits the store from the resource it + * was created on. + */ + run<R>(value: T, fn: () => R): R; +} + +export function createAsyncScopedStore<T>(): AsyncScopedStore<T> { + const scoped = new AsyncLocalStorage<T>(); + return { + get: () => scoped.getStore(), + run<R>(value: T, fn: () => R): R { + return scoped.run(value, fn); + }, + enter(value: T): () => void { + const previous = scoped.getStore(); + scoped.enterWith(value); + let restored = false; + return (): void => { + if (restored) return; + restored = true; + // `as`: enterWith's signature is `(store: T)`, but restoring "there was nothing here before" is + // exactly `undefined`, and getStore() returning undefined afterwards is the correct observable state. + scoped.enterWith(previous as T); + }; + }, + }; +} diff --git a/packages/core/src/observability/logger.test.ts b/packages/core/src/observability/logger.test.ts new file mode 100644 index 0000000..95da8bc --- /dev/null +++ b/packages/core/src/observability/logger.test.ts @@ -0,0 +1,424 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/logger.test.ts +// Exercises: OBS-1 (disabled path allocates/emits nothing, shared singleton event), OBS-3 (empty key +// rejected, null value emitted as literal "null"), OBS-4 (event() reserved-key precedence and single +// occurrence), OBS-5 (per-event > global > diagnostic-context precedence, actually wired through +// createLogger), OBS-6 (total field rendering), OBS-7 (truncation), OBS-8 (single-emit guard), OBS-9 (global +// context on every event), OBS-40 (once-per-logger reserved-key-collision warning, gated on verbose enabled, +// never fired for an ambient collision arriving via diagnostic context). +import {afterEach, describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {withDiagnosticFields} from './diagnostic-context.js'; +import { + NOOP_LOGGER, + createLogger, + getGlobalLogger, + setGlobalLogger, + type Logger, + type LogEvent, +} from './logger.js'; + +function collectingLogger(): { + logger: Logger; + emitted: Record<string, unknown>[]; +} { + const emitted: Record<string, unknown>[] = []; + function makeEvent( + globalFields: Readonly<Record<string, unknown>>, + ): LogEvent { + const fields: Record<string, unknown> = {...globalFields}; + let emittedOnce = false; + return { + field(key, value) { + if (key === '') throw new RangeError('field key must not be empty'); + fields[key] = value === null ? 'null' : value; + return this; + }, + event(name) { + if (name === '') delete fields.event; + else fields.event = name; + return this; + }, + cause(error) { + fields.cause = error; + return this; + }, + emit() { + if (emittedOnce) return; + emittedOnce = true; + emitted.push({...fields}); + }, + }; + } + const logger: Logger = { + atLevel: () => makeEvent({}), + withContext(context) { + return { + atLevel: () => makeEvent(context), + withContext: (ctx: Readonly<Record<string, unknown>>) => + logger.withContext(ctx), + }; + }, + }; + return {logger, emitted}; +} + +describe('the no-op default (OBS-1)', () => { + test('the disabled event is a shared singleton across calls', () => { + const noop = getGlobalLogger(); + setGlobalLogger(noop); // ensure default state for this test + const first = getGlobalLogger().atLevel('verbose'); + const second = getGlobalLogger().atLevel('verbose'); + expect(first).toBe(second); + }); + + test('every builder method returns the same event and emit is a no-op', () => { + const event = getGlobalLogger().atLevel('info'); + expect(event.field('k', 'v')).toBe(event); + expect(event.event('x')).toBe(event); + expect(event.cause(new Error('x'))).toBe(event); + expect(() => { + event.emit(); + }).not.toThrow(); + expect(NOOP_LOGGER.withContext({k: 'v'})).toBe(NOOP_LOGGER); + }); +}); + +describe('global logger slot (mirrors CFG-13)', () => { + // This block is the only one that mutates the module-level global slot -- restore the no-op default after + // every test so no later test file (or a later test in this one, if execution order ever changes) observes + // a logger some earlier test installed. The Global Constraints section requires exactly this discipline. + afterEach(() => { + setGlobalLogger(NOOP_LOGGER); + }); + + test('last-write-wins: getGlobalLogger returns the same instance after set', () => { + const {logger} = collectingLogger(); + setGlobalLogger(logger); + expect(getGlobalLogger()).toBe(logger); + }); + + test('defaults to the no-op logger when nothing has been set', () => { + expect(getGlobalLogger()).toBe(NOOP_LOGGER); + }); + + test('rejects null or non-logger input', () => { + expect(() => { + setGlobalLogger(null as unknown as Logger); + }).toThrow(); + expect(() => { + setGlobalLogger({} as unknown as Logger); + }).toThrow(); + }); +}); + +describe('field/event semantics (OBS-3, OBS-4)', () => { + test('an empty field key throws', () => { + const {logger} = collectingLogger(); + expect(() => logger.atLevel('info').field('', 'x')).toThrow(); + }); + + test('a null field value is emitted as the literal string "null"', () => { + const {logger, emitted} = collectingLogger(); + logger.atLevel('info').field('k', null).emit(); + expect(emitted[0]?.k).toBe('null'); + }); + + test('event(name) sets the reserved key exactly once; an empty name clears it', () => { + const {logger, emitted} = collectingLogger(); + logger.atLevel('info').event('x').emit(); + expect(emitted[0]?.event).toBe('x'); + + const {logger: logger2, emitted: emitted2} = collectingLogger(); + logger2.atLevel('info').event('x').event('').emit(); + expect(emitted2[0]?.event).toBeUndefined(); + }); +}); + +describe('single-emit guard (OBS-8)', () => { + test('a second terminal emit is a no-op', () => { + const {logger, emitted} = collectingLogger(); + const event = logger.atLevel('info').field('k', 1); + event.emit(); + event.emit(); + expect(emitted).toHaveLength(1); + }); +}); + +describe('global context (OBS-9)', () => { + test('a global field configured via withContext attaches to every event', () => { + const {logger, emitted} = collectingLogger(); + const withGlobal = logger.withContext({service: 'dexpace'}); + withGlobal.atLevel('info').emit(); + withGlobal.atLevel('info').field('extra', 1).emit(); + expect(emitted[0]?.service).toBe('dexpace'); + expect(emitted[1]?.service).toBe('dexpace'); + }); + + test('withContext rejects null or empty field key', () => { + const logger = createLogger(() => undefined); + expect(() => + logger.withContext(null as unknown as Record<string, unknown>), + ).toThrow(); + expect(() => logger.withContext({'': 'bad'})).toThrow(); + }); +}); + +describe('createLogger: diagnostic-context folding and full precedence (OBS-5)', () => { + function recordingLogger(options?: Parameters<typeof createLogger>[1]): { + logger: Logger; + emitted: ReadonlyMap<string, unknown>[]; + } { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger( + (_level, fields) => emitted.push(fields), + options, + ); + return {logger, emitted}; + } + + test('folds diagnostic-context fields when nothing else overrides them', () => { + const {logger, emitted} = recordingLogger(); + withDiagnosticFields({'trace.id': 't1', 'span.id': 's1'}, () => { + logger.atLevel('info').emit(); + }); + expect(emitted[0]?.get('trace.id')).toBe('t1'); + expect(emitted[0]?.get('span.id')).toBe('s1'); + }); + + test('global context (withContext) wins over folded diagnostic context for the same key', () => { + const {logger, emitted} = recordingLogger(); + withDiagnosticFields({'trace.id': 'from-diagnostic'}, () => { + logger.withContext({'trace.id': 'from-global'}).atLevel('info').emit(); + }); + expect(emitted[0]?.get('trace.id')).toBe('from-global'); + }); + + test('a per-event field wins over both global context and folded diagnostic context', () => { + const {logger, emitted} = recordingLogger(); + withDiagnosticFields({'trace.id': 'from-diagnostic'}, () => { + logger + .withContext({'trace.id': 'from-global'}) + .atLevel('info') + .field('trace.id', 'from-event') + .emit(); + }); + expect(emitted[0]?.get('trace.id')).toBe('from-event'); + }); + + test('outside any diagnostic-context scope, no diagnostic fields are folded', () => { + const {logger, emitted} = recordingLogger(); + logger.atLevel('info').emit(); + expect(emitted[0]?.has('trace.id')).toBe(false); + }); +}); + +describe('createLogger: reserved-key collision warning (OBS-40)', () => { + test('warns exactly once per logger, ambient keys never trigger it', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields)); + + logger.atLevel('info').field('event', 'x').emit(); // explicit collision, attempt 1 + logger.atLevel('info').field('event', 'y').emit(); // explicit collision, attempt 2 -- must not re-warn + + const warnings = emitted.filter( + f => f.get('event') === 'dexpace.logger.reservedKeyCollision', + ); + expect(warnings).toHaveLength(1); + }); + + test('never warns when verbose is disabled for this logger', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields), { + isLevelEnabled: level => level !== 'verbose', + }); + + logger.atLevel('info').field('event', 'x').emit(); + + expect( + emitted.some( + f => f.get('event') === 'dexpace.logger.reservedKeyCollision', + ), + ).toBe(false); + }); + + test('an ambient "event" key folded from diagnostic context is never warned about', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields)); + + withDiagnosticFields({event: 'ambient-value'}, () => { + logger.atLevel('info').emit(); + }); + + expect( + emitted.some( + f => f.get('event') === 'dexpace.logger.reservedKeyCollision', + ), + ).toBe(false); + }); +}); + +describe('createLogger: total field rendering (OBS-6)', () => { + function render(value: unknown): unknown { + const emitted: ReadonlyMap<string, unknown>[] = []; + createLogger((_level, fields) => emitted.push(fields)) + .atLevel('info') + .field('k', value) + .emit(); + return emitted[0]?.get('k'); + } + + test('an Error renders as "Name: message"', () => { + expect(render(new TypeError('boom'))).toBe('TypeError: boom'); + }); + + test('numeric and boolean primitives pass through type-preserving', () => { + expect(render(42)).toBe(42); + expect(render(false)).toBe(false); + expect(render(100n)).toBe(100n); + }); + + test('an array, a Map, and a Set each render to a bracketed form carrying their entries', () => { + expect(render([1, 2])).toBe('[1, 2]'); + expect(render(new Set(['a']))).toBe('[a]'); + expect(render(new Map([['a', 1]]))).toBe('[a=1]'); + expect(render({a: 1})).toBe('[a=1]'); + }); + + test('a value whose toString throws renders as the placeholder rather than propagating', () => { + const hostile = { + toString(): string { + throw new Error('nope'); + }, + }; + expect(render(hostile)).toBe('[unrenderable value]'); + }); +}); + +describe('createLogger: truncation and robust rendering (OBS-7)', () => { + function render(value: unknown): unknown { + const emitted: ReadonlyMap<string, unknown>[] = []; + createLogger((_level, fields) => emitted.push(fields)) + .atLevel('info') + .field('k', value) + .emit(); + return emitted[0]?.get('k'); + } + + test('an oversized string is truncated with a marker (OBS-7)', () => { + const rendered = String(render('x'.repeat(10_000))); + expect(rendered.length).toBeLessThan(10_000); + expect(rendered.endsWith('…[truncated]')).toBe(true); + }); + + test('property: rendering never throws for any value', () => { + fc.assert( + fc.property(fc.anything(), value => { + expect(() => render(value)).not.toThrow(); + }), + ); + }); + + test('unicode surrogate pair at truncation boundary is not sliced in half', () => { + const emoji = '😀'; // \uD83D\uDE00 + const str = 'a'.repeat(8191) + emoji + 'b'.repeat(100); + const rendered = String(render(str)); + expect(rendered.endsWith('…[truncated]')).toBe(true); + const beforeMarker = rendered.slice(0, -'…[truncated]'.length); + const lastCode = beforeMarker.charCodeAt(beforeMarker.length - 1); + expect(lastCode >= 0xd800 && lastCode <= 0xdbff).toBe(false); + }); +}); + +describe('createLogger: hostile and edge-case rendering (OBS-6)', () => { + function render(value: unknown): unknown { + const emitted: ReadonlyMap<string, unknown>[] = []; + createLogger((_level, fields) => emitted.push(fields)) + .atLevel('info') + .field('k', value) + .emit(); + return emitted[0]?.get('k'); + } + + test('a global-context value is rendered too, not passed raw to the sink', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const hostile = { + toString(): string { + throw new Error('nope'); + }, + }; + createLogger((_level, fields) => emitted.push(fields)) + .withContext({k: hostile}) + .atLevel('info') + .emit(); + expect(emitted[0]?.get('k')).toBe('[unrenderable value]'); + }); + + test('hostile toString returning non-string is rendered safely as placeholder (OBS-6)', () => { + expect(render({toString: () => undefined})).toBe('[unrenderable value]'); + expect(render({toString: () => null})).toBe('[unrenderable value]'); + expect(render({toString: () => ({nested: 1})})).toBe( + '[unrenderable value]', + ); + expect(render({[Symbol.toPrimitive]: () => null})).toBe( + '[unrenderable value]', + ); + }); + + test('LogEvent.field and event reject non-string keys/names', () => { + const logger = createLogger(() => undefined); + const event = logger.atLevel('info'); + expect(() => event.field(null as unknown as string, 'val')).toThrow(); + expect(() => event.field(123 as unknown as string, 'val')).toThrow(); + expect(() => event.event(null as unknown as string)).toThrow(); + }); + + test('cause sets the cause field', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields)); + const err = new Error('test-err'); + logger.atLevel('error').cause(err).emit(); + expect(emitted[0]?.get('cause')).toBe('Error: test-err'); + }); +}); + +describe('createLogger: the reserved key survives when no tag is set (OBS-4, OBS-9)', () => { + test('an ambient global-context "event" key is emitted when event() was never called', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields)); + + logger.withContext({event: 'app.ambient'}).atLevel('info').emit(); + + expect(emitted[0]?.get('event')).toBe('app.ambient'); + }); + + test('a set tag suppresses the ambient key, carrying "event" exactly once', () => { + const emitted: ReadonlyMap<string, unknown>[] = []; + const logger = createLogger((_level, fields) => emitted.push(fields)); + + logger + .withContext({event: 'app.ambient'}) + .atLevel('info') + .event('http.request') + .emit(); + + expect(emitted[0]?.get('event')).toBe('http.request'); + }); +}); + +describe('createLogger: disabled levels allocate nothing (OBS-1)', () => { + test('atLevel returns the shared NOOP_EVENT-equivalent when the level is disabled, sink is never called', () => { + const logger = createLogger( + () => { + throw new Error('sink must not be called for a disabled level'); + }, + {isLevelEnabled: level => level !== 'verbose'}, + ); + + const event = logger.atLevel('verbose'); + expect(event.field('k', 'v')).toBe(event); + expect(() => { + event.emit(); + }).not.toThrow(); + }); +}); diff --git a/packages/core/src/observability/logger.ts b/packages/core/src/observability/logger.ts new file mode 100644 index 0000000..ca4f167 --- /dev/null +++ b/packages/core/src/observability/logger.ts @@ -0,0 +1,393 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/logger.ts +import {invariant} from '../invariant.js'; +import {getDiagnosticContext} from './diagnostic-context.js'; + +/** + * The four log severity levels supported by the facade (OBS-2). + * + * @public + */ +export type LogLevel = 'error' | 'warning' | 'info' | 'verbose'; + +const RESERVED_EVENT_KEY = 'event'; +const COLLISION_WARNING_EVENT = 'dexpace.logger.reservedKeyCollision'; +const MAX_FIELD_LENGTH = 8192; +const TRUNCATION_MARKER = '…[truncated]'; +const UNRENDERABLE_PLACEHOLDER = '[unrenderable value]'; +const DEFAULT_DIAGNOSTIC_ALLOW_LIST: readonly string[] = Object.freeze([ + 'trace.id', + 'span.id', +]); + +/** + * Fluent builder for constructing and emitting structured log events (OBS-3, OBS-4, OBS-8). + * + * @public + */ +export interface LogEvent { + /** OBS-3: an empty key MUST be rejected. A null value is emitted as the literal string "null". */ + field(key: string, value: unknown): this; + /** OBS-4: sets the reserved "event" tag exclusively; an empty name clears it. */ + event(name: string): this; + /** Sets the cause field of the event. */ + cause(error: unknown): this; + /** OBS-8: at most once; a second call is a no-op, safe under concurrent invocation. */ + emit(): void; +} + +/** + * Logging facade interface (OBS-1, OBS-9). + * + * @public + */ +export interface Logger { + /** OBS-1: enabled/disabled is decided once, here. The disabled path allocates and emits nothing. */ + atLevel(level: LogLevel): LogEvent; + /** OBS-9: attaches a global key/value context to every event this returns. */ + withContext(fields: Readonly<Record<string, unknown>>): Logger; +} + +/** + * OBS-6/OBS-7: total field-value rendering -- never throws, for any input. + */ +function renderField(value: unknown): unknown { + try { + if (value === null || value === undefined) return 'null'; + // OBS-6: numeric/boolean/bigint primitives pass through type-preserving and are exempt from truncation. + if ( + typeof value === 'number' || + typeof value === 'boolean' || + typeof value === 'bigint' + ) { + return value; + } + return truncate(renderNonPrimitive(value)); + } catch { + return UNRENDERABLE_PLACEHOLDER; + } +} + +function renderNonPrimitive(value: unknown): string { + try { + if (typeof value === 'string') return value; + if (value instanceof Error) return `${value.name}: ${value.message}`; + if (Array.isArray(value)) { + return `[${value.map(renderScalar).join(', ')}]`; + } + if (value instanceof Set) { + return `[${[...value].map(renderScalar).join(', ')}]`; + } + if (value instanceof Map) return renderPairs([...value]); + if (typeof value === 'object' && value !== null) { + if ( + typeof (value as {toString?: unknown}).toString === 'function' && + (value as {toString: unknown}).toString !== Object.prototype.toString + ) { + const custom = (value as {toString(): unknown}).toString(); + return typeof custom === 'string' ? custom : UNRENDERABLE_PLACEHOLDER; + } + if (Symbol.toPrimitive in value) { + const prim = (value as {[Symbol.toPrimitive](hint: string): unknown})[ + Symbol.toPrimitive + ]('string'); + return typeof prim === 'string' ? prim : UNRENDERABLE_PLACEHOLDER; + } + return renderPairs(Object.entries(value)); + } + if (typeof value === 'symbol') return value.toString(); + if (typeof value === 'function') return value.name || '[Function]'; + if ( + typeof value === 'number' || + typeof value === 'boolean' || + typeof value === 'bigint' + ) { + return value.toString(); + } + return UNRENDERABLE_PLACEHOLDER; + } catch { + return UNRENDERABLE_PLACEHOLDER; + } +} + +/** Formats shallow scalar representations for elements inside collections to prevent unbounded recursion. */ +function renderScalar(value: unknown): string { + try { + if (value === null || value === undefined) return 'null'; + if (value instanceof Error) return `${value.name}: ${value.message}`; + if (typeof value === 'object') return Array.isArray(value) ? '[…]' : '{…}'; + if (typeof value === 'symbol') return value.toString(); + if (typeof value === 'function') return value.name || '[Function]'; + if (typeof value === 'string') return value; + if ( + typeof value === 'number' || + typeof value === 'boolean' || + typeof value === 'bigint' + ) { + return value.toString(); + } + return UNRENDERABLE_PLACEHOLDER; + } catch { + return UNRENDERABLE_PLACEHOLDER; + } +} + +function renderPairs(pairs: readonly (readonly [unknown, unknown])[]): string { + const rendered = pairs.map( + ([key, entry]) => `${renderScalar(key)}=${renderScalar(entry)}`, + ); + return `[${rendered.join(', ')}]`; +} + +/** OBS-7: bounded to 8 KiB with a marker. Primitives never reach here (see renderField). */ +function truncate(rendered: string): string { + if (rendered.length <= MAX_FIELD_LENGTH) return rendered; + let sliceEnd = MAX_FIELD_LENGTH; + const lastCode = rendered.charCodeAt(sliceEnd - 1); + // Avoid splitting a UTF-16 surrogate pair across the truncation boundary + if (lastCode >= 0xd800 && lastCode <= 0xdbff) { + sliceEnd -= 1; + } + return rendered.slice(0, sliceEnd) + TRUNCATION_MARKER; +} + +/** OBS-40: throttles the reserved-key-collision warning to at most one emission per Logger instance. */ +class CollisionWarningGate { + private warned = false; + public shouldWarn(): boolean { + if (this.warned) return false; + this.warned = true; + return true; + } +} + +/** One object rather than six positional arguments -- `max-params: 3` applies to constructors too. */ +interface LogEventInit { + readonly level: LogLevel; + readonly wiring: LoggerWiring; + readonly diagnosticFields: Readonly<Record<string, string>>; + readonly verboseEnabled: boolean; +} + +class RealLogEvent implements LogEvent { + private readonly level: LogLevel; + private readonly sink: ( + level: LogLevel, + fields: ReadonlyMap<string, unknown>, + ) => void; + private readonly collisionGate: CollisionWarningGate; + private readonly verboseEnabled: boolean; + private readonly fields: Map<string, unknown>; + private eventTag: string | undefined; + private emitted = false; + + public constructor(init: LogEventInit) { + const {level, wiring, diagnosticFields, verboseEnabled} = init; + this.level = level; + this.sink = wiring.sink; + this.collisionGate = wiring.collisionGate; + this.verboseEnabled = verboseEnabled; + const globalFields = wiring.globalFields; + + // Field precedence: diagnostic context < global context < per-event field < explicit event tag. + this.fields = new Map(); + for (const [key, value] of Object.entries(diagnosticFields)) { + this.fields.set(key, renderField(value)); + } + for (const [key, value] of Object.entries(globalFields)) { + this.fields.set(key, renderField(value)); + } + } + + public field(key: string, value: unknown): this { + invariant( + typeof key === 'string' && key !== '', + 'LogEvent.field: key must not be empty', + ); + if (key === RESERVED_EVENT_KEY) { + if (this.verboseEnabled && this.collisionGate.shouldWarn()) { + this.sink( + 'verbose', + new Map<string, unknown>([ + [RESERVED_EVENT_KEY, COLLISION_WARNING_EVENT], + [ + 'message', + 'LogEvent.field: "event" is reserved; use event() to set it instead.', + ], + ]), + ); + } + return this; + } + this.fields.set(key, renderField(value)); + return this; + } + + public event(name: string): this { + invariant( + typeof name === 'string', + 'LogEvent.event: name must be a string', + ); + this.eventTag = name === '' ? undefined : name; + return this; + } + + public cause(error: unknown): this { + this.fields.set('cause', renderField(error)); + return this; + } + + public emit(): void { + if (this.emitted) return; + this.emitted = true; + + const withTag = new Map(this.fields); + if (this.eventTag !== undefined) { + withTag.set(RESERVED_EVENT_KEY, this.eventTag); + } + + this.sink(this.level, withTag); + } +} + +/** OBS-1: one shared, allocation-minimal inert event -- every builder method returns `this`, emit() is a no-op. */ +const NOOP_EVENT: LogEvent = Object.freeze({ + field(): LogEvent { + return NOOP_EVENT; + }, + event(): LogEvent { + return NOOP_EVENT; + }, + cause(): LogEvent { + return NOOP_EVENT; + }, + emit(): void { + return; + }, +}); + +/** + * The no-op default (OBS-1), installed process-wide until a consumer supplies a real one. + * + * @public + */ +export const NOOP_LOGGER: Logger = Object.freeze({ + atLevel(): LogEvent { + return NOOP_EVENT; + }, + withContext(): Logger { + return NOOP_LOGGER; + }, +}); + +/** + * Configuration options for {@link createLogger}. + * + * @public + */ +export interface CreateLoggerOptions { + readonly globalFields?: Readonly<Record<string, unknown>> | undefined; + /** OBS-10: default is trace.id and span.id; null folds every present diagnostic-context key. */ + readonly diagnosticAllowList?: readonly string[] | null | undefined; + /** OBS-1: gates atLevel's allocation -- a disabled level returns NOOP_EVENT without building a real one. */ + readonly isLevelEnabled?: ((level: LogLevel) => boolean) | undefined; +} + +/** Every field the built Logger closes over. One object, so no function here exceeds `max-params: 3`. */ +interface LoggerWiring { + readonly sink: ( + level: LogLevel, + fields: ReadonlyMap<string, unknown>, + ) => void; + readonly globalFields: Readonly<Record<string, unknown>>; + readonly diagnosticAllowList: readonly string[] | null; + readonly isLevelEnabled: (level: LogLevel) => boolean; + readonly collisionGate: CollisionWarningGate; +} + +function buildLogger(wiring: LoggerWiring): Logger { + return { + atLevel(level: LogLevel): LogEvent { + if (!wiring.isLevelEnabled(level)) return NOOP_EVENT; + return new RealLogEvent({ + level, + wiring, + diagnosticFields: getDiagnosticContext(wiring.diagnosticAllowList), + verboseEnabled: wiring.isLevelEnabled('verbose'), + }); + }, + withContext(fields: Readonly<Record<string, unknown>>): Logger { + invariant( + typeof fields === 'object' && (fields as unknown) !== null, + 'Logger.withContext: fields must be an object', + ); + for (const key of Object.keys(fields)) { + invariant(key !== '', 'Logger.withContext: key must not be empty'); + } + return buildLogger({ + ...wiring, + globalFields: {...wiring.globalFields, ...fields}, + }); + }, + }; +} + +/** + * The single concrete `Logger` builder every real backend constructs through. + * + * @param sink - callback receiving log events. + * @param options - logger creation options. + * @returns a configured {@link Logger}. + * + * @public + */ +export function createLogger( + sink: (level: LogLevel, fields: ReadonlyMap<string, unknown>) => void, + options: CreateLoggerOptions = {}, +): Logger { + invariant( + typeof sink === 'function', + 'createLogger: sink must be a function', + ); + + return buildLogger({ + sink, + globalFields: options.globalFields ?? {}, + diagnosticAllowList: + options.diagnosticAllowList === undefined + ? DEFAULT_DIAGNOSTIC_ALLOW_LIST + : options.diagnosticAllowList, + isLevelEnabled: options.isLevelEnabled ?? ((): boolean => true), + collisionGate: new CollisionWarningGate(), + }); +} + +let globalLogger: Logger = NOOP_LOGGER; + +/** + * Returns the process-wide global logger instance. + * + * @public + */ +export function getGlobalLogger(): Logger { + return globalLogger; +} + +/** + * Sets the process-wide global logger instance. + * + * @param logger - the logger to install. + * + * @public + */ +export function setGlobalLogger(logger: Logger): void { + invariant( + (logger as unknown) !== null && (logger as unknown) !== undefined, + 'setGlobalLogger: logger is required', + ); + invariant( + typeof logger.atLevel === 'function', + 'setGlobalLogger: logger.atLevel must be a function', + ); + globalLogger = logger; +} diff --git a/packages/core/src/observability/logging-step.test.ts b/packages/core/src/observability/logging-step.test.ts new file mode 100644 index 0000000..ba7e12d --- /dev/null +++ b/packages/core/src/observability/logging-step.test.ts @@ -0,0 +1,780 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/logging-step.test.ts +// Exercises: XCUT-20 (observability code paths NEVER throw into the caller's request path -- a failing +// sink degrades to a self-describing http.instrumentation.* event and the request still completes), +// XCUT-24 (diagnostic body previews are byte-capped and non-consuming: OBS-36/37/38 below), +// OBS-34 (granularity gates log events, not span/metrics), OBS-35 (level resolves from +// Configuration, tolerant/case-insensitive), OBS-39 (stable http.request/http.response event names/keys, +// url.full always redacted), OBS-20 (a throwing Logger is caught and re-surfaced as http.instrumentation.*; +// a throwing tracer/meter propagates, NOT caught), OBS-36, OBS-37, OBS-38 (body previews), OBS-20's +// body-drain half (a failed request-side probe or response-side drain emits +// http.instrumentation.bodyCaptureFailed and the request still completes), OBS-35's "MUST NOT bake in a +// default config key name" (the configKey setting), OBS-21 (the per-request span ends exactly once, +// even when end() itself throws). +import {afterEach, describe, expect, test} from 'bun:test'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {Cursor} from '../pipeline/cursor.js'; +import {createRequestContext} from '../context/context.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import { + CFG_KEY_LOG_LEVEL, + ConfigurationBuilder, + setGlobalConfiguration, +} from '../config/configuration.js'; +import type {Body} from '../body/body.js'; +import {stringBody} from '../body/simple-bodies.js'; +import type {Logger, LogEvent} from './logger.js'; +import type {Meter} from './metrics.js'; +import type {Tracer} from './tracing.js'; +import {loggingStep} from './logging-step.js'; + +function spyLogger(): {logger: Logger; events: Record<string, unknown>[]} { + const events: Record<string, unknown>[] = []; + function event(): LogEvent { + const fields: Record<string, unknown> = {}; + const self: LogEvent = { + field(key, value) { + fields[key] = value; + return self; + }, + event(name) { + fields.event = name; + return self; + }, + cause(error) { + fields.cause = error; + return self; + }, + emit() { + events.push({...fields}); + }, + }; + return self; + } + return { + logger: { + atLevel: () => event(), + withContext: () => ({ + atLevel: () => event(), + withContext: () => ({}) as Logger, + }), + }, + events, + }; +} + +function textResponse( + status: number, + text: string, + contentType = 'text/plain', +): Response { + const bytes = new TextEncoder().encode(text); + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(bytes); + controller.close(); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers( + Headers.newBuilder() + .set('content-type', contentType) + .set('content-length', String(bytes.length)) + .build(), + ) + .body(stream) + .build(); +} + +function chunkedResponse(status: number): Response { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new Uint8Array([1, 2, 3])); + controller.close(); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(Headers.newBuilder().set('transfer-encoding', 'chunked').build()) + .body(stream) + .build(); +} + +function binaryResponse(status: number, bytes: Uint8Array): Response { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(bytes); + controller.close(); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers( + Headers.newBuilder() + .set('content-type', 'application/octet-stream') + .set('content-length', String(bytes.length)) + .build(), + ) + .body(stream) + .build(); +} + +async function send( + step: ReturnType<typeof loggingStep>, + transport: FakeTransport, + request = Request.newBuilder().url('https://example.com/test').build(), +): Promise<Response> { + return new Cursor({ + steps: [step], + transport, + request, + context: createRequestContext(request), + }).advance(); +} + +afterEach(() => { + setGlobalConfiguration( + new ConfigurationBuilder().remove(CFG_KEY_LOG_LEVEL).build(), + ); +}); + +describe('granularity controls (OBS-34)', () => { + test('at granularity: none, no log events are emitted but span & metrics run', async () => { + const {logger, events} = spyLogger(); + let spanStarted = false; + let spanEnded = false; + const fakeTracer: Tracer = { + startSpan: () => { + spanStarted = true; + return { + isRecording: true, + setAttribute: () => fakeTracer.startSpan('child'), + recordException: () => fakeTracer.startSpan('child'), + end: () => { + spanEnded = true; + }, + }; + }, + }; + let counterCalls = 0; + const fakeMeter: Meter = { + createCounter: () => ({ + add: () => { + counterCalls += 1; + }, + }), + createHistogram: () => ({ + record: () => undefined, + }), + }; + + const transport = new FakeTransport([countingResponse(200).response]); + await send( + loggingStep({ + logger, + granularity: 'none', + tracerFactory: () => fakeTracer, + meter: fakeMeter, + }), + transport, + ); + + expect(events).toHaveLength(0); + expect(spanStarted).toBe(true); + expect(spanEnded).toBe(true); + expect(counterCalls).toBe(1); + }); +}); + +describe('ambient configuration resolution (OBS-35)', () => { + test('resolves headers granularity from CFG_KEY_LOG_LEVEL', async () => { + setGlobalConfiguration( + new ConfigurationBuilder().put(CFG_KEY_LOG_LEVEL, 'HEADERS').build(), + ); + const {logger, events} = spyLogger(); + const transport = new FakeTransport([countingResponse(200).response]); + + await send(loggingStep({logger}), transport); + + const eventNames = events.map(e => e.event); + expect(eventNames).toContain('http.request'); + expect(eventNames).toContain('http.response'); + }); + + test('falls back to none when config is unset or invalid', async () => { + setGlobalConfiguration( + new ConfigurationBuilder().put(CFG_KEY_LOG_LEVEL, 'INVALID').build(), + ); + const {logger, events} = spyLogger(); + const transport = new FakeTransport([countingResponse(200).response]); + + await send(loggingStep({logger}), transport); + expect(events).toHaveLength(0); + }); +}); + +describe('structured events: request and response (OBS-39)', () => { + test('emits standard field hierarchy for successful request', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([countingResponse(200).response]); + const req = Request.newBuilder() + .url('https://user:secret@example.com/api?token=secret#token=hash') + .method('GET') + .build(); + + await send(loggingStep({logger, granularity: 'headers'}), transport, req); + + expect(events).toHaveLength(2); + const [requestEvent, responseEvent] = events; + + expect(requestEvent?.event).toBe('http.request'); + expect(requestEvent?.['http.request.method']).toBe('GET'); + expect(requestEvent?.['url.full']).not.toContain('secret'); + expect(requestEvent?.['url.full']).toContain('***'); + + expect(responseEvent?.event).toBe('http.response'); + expect(responseEvent?.['http.response.status_code']).toBe(200); + expect(typeof responseEvent?.['http.response.duration_ms']).toBe('number'); + }); + + test('emits error response event on failure', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([new Error('network failure')]); + + let caught: unknown; + try { + await send(loggingStep({logger, granularity: 'headers'}), transport); + } catch (e) { + caught = e; + } + expect(caught).toBeDefined(); + + expect(events).toHaveLength(2); + const failureEvent = events[1]; + expect(failureEvent?.event).toBe('http.response'); + expect(failureEvent?.['error.type']).toBe('Error'); + expect(failureEvent?.cause).toBeDefined(); + }); +}); + +describe('header redaction policies (OBS-17, OBS-18)', () => { + test('redacts non-allow-listed headers per DroppedHeaderPolicy', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: { + 'content-type': 'application/json', + authorization: 'Bearer secret', + }, + }), + ]); + + await send( + loggingStep({ + logger, + granularity: 'headers', + droppedHeaderPolicy: 'mark', + }), + transport, + ); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.header.content-type']).toBe( + 'application/json', + ); + expect(responseEvent?.['http.response.header.authorization']).toBe( + 'REDACTED', + ); + }); + + test('omits non-allow-listed headers when policy is omit', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: { + 'content-type': 'application/json', + 'set-cookie': 'session=abc', + }, + }), + ]); + + await send( + loggingStep({ + logger, + granularity: 'headers', + droppedHeaderPolicy: 'omit', + }), + transport, + ); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect('http.response.header.set-cookie' in (responseEvent ?? {})).toBe( + false, + ); + }); +}); + +describe('location header redaction (OBS-17)', () => { + test('a Location header is redacted through the URL-value redactor', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([ + countingResponse({ + status: 302, + headers: {location: 'https://other.example/cb?code=SECRET'}, + }), + ]); + + await send(loggingStep({logger, granularity: 'headers'}), transport); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect( + String(responseEvent?.['http.response.header.location']), + ).not.toContain('SECRET'); + }); +}); + +describe('failure containment: logger safety (OBS-20)', () => { + test('a throwing Logger is caught; the request still completes and emits logFailure', async () => { + const emittedEvents: Record<string, unknown>[] = []; + let shouldThrow = true; + const throwingLogger: Logger = { + atLevel(level) { + return { + field() { + return this; + }, + event(name) { + emittedEvents.push({level, event: name}); + return this; + }, + cause() { + return this; + }, + emit(): void { + if (shouldThrow) { + shouldThrow = false; + throw new Error('logger exploded'); + } + }, + }; + }, + withContext(): Logger { + return throwingLogger; + }, + }; + const transport = new FakeTransport([countingResponse(200).response]); + const res = await send( + loggingStep({logger: throwingLogger, granularity: 'headers'}), + transport, + ); + expect(res).toBeDefined(); + expect( + emittedEvents.some(e => e.event === 'http.instrumentation.logFailure'), + ).toBe(true); + }); + + test('a failing response body drain does not fail the request (OBS-20)', async () => { + const {logger} = spyLogger(); + const failingStream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.error(new Error('stream broke')); + }, + }); + const brokenResponse = Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers( + Headers.newBuilder() + .set('content-type', 'text/plain') + .set('content-length', '100') + .build(), + ) + .body(failingStream) + .build(); + const transport = new FakeTransport([brokenResponse]); + const res = await send( + loggingStep({logger, granularity: 'body'}), + transport, + ); + expect(res).toBeDefined(); + expect(res.status.code).toBe(200); + }); +}); + +describe('failure containment: tracer and meter propagation (OBS-20, OBS-30)', () => { + test('a throwing tracer is NOT caught -- it propagates and fails the request', async () => { + const {logger} = spyLogger(); + const explodingTracer: Tracer = { + startSpan(): never { + throw new Error('tracer exploded'); + }, + }; + const transport = new FakeTransport([countingResponse(200).response]); + + let caughtErr: unknown; + try { + await send( + loggingStep({ + logger, + granularity: 'headers', + tracerFactory: () => explodingTracer, + }), + transport, + ); + } catch (e) { + caughtErr = e; + } + expect(caughtErr).toBeDefined(); + expect((caughtErr as Error).message).toBe('tracer exploded'); + }); + + test('a throwing meter is NOT caught -- it propagates and fails the request', async () => { + const explodingMeter: Meter = { + createCounter() { + return { + add() { + throw new Error('meter counter exploded'); + }, + }; + }, + createHistogram() { + return { + record() { + throw new Error('meter histogram exploded'); + }, + }; + }, + }; + const transport = new FakeTransport([countingResponse(200).response]); + let caughtErr: unknown; + try { + await send( + loggingStep({meter: explodingMeter, granularity: 'none'}), + transport, + ); + } catch (e) { + caughtErr = e; + } + expect(caughtErr).toBeDefined(); + expect((caughtErr as Error).message).toBe('meter counter exploded'); + }); +}); + +describe('failure containment: non-error thrown values (XCUT-20)', () => { + test('a non-Error thrown value does not make the logging step throw its own TypeError', async () => { + const {logger, events} = spyLogger(); + const rejectingTransport = { + send: () => + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- testing non-Error rejection (XCUT-20) + Promise.reject('string rejection'), + close: () => Promise.resolve(), + }; + const req = Request.newBuilder().url('https://example.com/test').build(); + const cursor = new Cursor({ + steps: [loggingStep({logger, granularity: 'headers'})], + transport: rejectingTransport, + request: req, + context: createRequestContext(req), + }); + + let caughtErr: unknown; + try { + await cursor.advance(); + } catch (e) { + caughtErr = e; + } + expect(caughtErr).toBe('string rejection'); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['error.type']).toBe('Error'); + }); +}); + +describe('response body preview (OBS-36, OBS-37, OBS-38)', () => { + test('a body larger than previewSizeBytes still reaches caller in full, preview is capped', async () => { + const {logger, events} = spyLogger(); + const payload = 'x'.repeat(50_000); + const transport = new FakeTransport([textResponse(200, payload)]); + + const response = await send( + loggingStep({logger, granularity: 'body', previewSizeBytes: 128}), + transport, + ); + + expect(await response.text()).toHaveLength(50_000); + const responseEvent = events.find(e => e.event === 'http.response'); + expect(String(responseEvent?.['http.response.body.preview'])).toHaveLength( + 128, + ); + expect(responseEvent?.['http.response.body.size']).toBe(128); + }); + + test('an unknown-length response body skips capture entirely (OBS-37)', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([chunkedResponse(200)]); + + await send(loggingStep({logger, granularity: 'body'}), transport); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); + }); + + test('a binary body renders as a size-only marker, never decoded (OBS-38)', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([ + binaryResponse(200, new Uint8Array([0xff, 0xfe, 0x00])), + ]); + + await send(loggingStep({logger, granularity: 'body'}), transport); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBe( + '[binary 3 bytes captured]', + ); + expect(responseEvent?.['http.response.body.size']).toBe(3); + }); + + test('a truncated multi-byte sequence decodes to a replacement character, never throwing (OBS-38)', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([ + textResponse(200, '€€€', 'text/plain; charset=utf-8'), + ]); + + const res = await send( + loggingStep({logger, granularity: 'body', previewSizeBytes: 2}), + transport, + ); + expect(res).toBeDefined(); + + const responseEvent = events.find(e => e.event === 'http.response'); + expect(String(responseEvent?.['http.response.body.preview'])).toContain( + '\uFFFD', + ); + }); +}); + +describe('request body preview and validation (OBS-36, OBS-38)', () => { + test('request body preview is captured under granularity: body', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([textResponse(200, 'ok')]); + const req = Request.newBuilder() + .url('https://example.com/submit') + .method('POST') + .body(stringBody('hello payload', 'text/plain')) + .build(); + + await send(loggingStep({logger, granularity: 'body'}), transport, req); + + const requestEvent = events.find(e => e.event === 'http.request'); + expect(requestEvent?.['http.request.body.preview']).toBe('hello payload'); + expect(requestEvent?.['http.request.body.size']).toBe(13); + }); + + test('validates previewSizeBytes is positive and finite', () => { + expect(() => loggingStep({previewSizeBytes: -1})).toThrow(); + expect(() => loggingStep({previewSizeBytes: Number.NaN})).toThrow(); + }); + + test('tolerant granularity setting parsing', async () => { + const {logger, events} = spyLogger(); + const transport = new FakeTransport([countingResponse(200).response]); + await send( + loggingStep({logger, granularity: ' HEADERS ' as never}), + transport, + ); + expect(events.map(e => e.event)).toEqual(['http.request', 'http.response']); + }); + + test('negative or invalid content-length header skips body capture', async () => { + const {logger, events} = spyLogger(); + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new Uint8Array([1, 2, 3])); + controller.close(); + }, + }); + const resp = Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers(Headers.newBuilder().set('content-length', '-1').build()) + .body(stream) + .build(); + const transport = new FakeTransport([resp]); + + await send(loggingStep({logger, granularity: 'body'}), transport); + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); + }); +}); + +/** A declared-length response whose body errors on the first read -- a drain that cannot finish. */ +function failingResponse(error: Error): Response { + const failingStream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.error(error); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers( + Headers.newBuilder() + .set('content-type', 'text/plain') + .set('content-length', '100') + .build(), + ) + .body(failingStream) + .build(); +} + +describe('body-drain diagnostics (OBS-20)', () => { + test('a failing response drain emits http.instrumentation.bodyCaptureFailed with the cause', async () => { + const {logger, events} = spyLogger(); + const broke = new Error('stream broke'); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([failingResponse(broke)]), + ); + + expect(response.status.code).toBe(200); // OBS-20: the request still completes + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('response'); + expect(diagnostic?.cause).toBe(broke); + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); + }); + + test('a failing request-body probe emits the same diagnostic and still sends the request', async () => { + const {logger, events} = spyLogger(); + const gone = new Error('ENOENT: the file went away'); + // What `fileBody()` over a deleted file looks like to the step: replayable, so the tap probes + // it, and the probe is the thing that fails. + const brokenBody: Body = { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: 3, + replayable: true, + writeTo: () => Promise.reject(gone), + }; + const request = Request.newBuilder() + .url('https://example.com/upload') + .method('POST') + .body(brokenBody) + .build(); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([countingResponse(200).response]), + request, + ); + + expect(response.status.code).toBe(200); + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('request'); + expect(diagnostic?.cause).toBe(gone); + // The empty capture still ships, as it did before: the tap saw no bytes, and the preview field is + // what the tap holds. What changed is that the empty string is no longer the ONLY evidence -- the + // diagnostic above says which direction failed and why. + const requestEvent = events.find(e => e.event === 'http.request'); + expect(requestEvent?.['http.request.body.preview']).toBe(''); + }); +}); + +describe('the config key the ambient granularity is read from (OBS-35)', () => { + test('configKey names the key, so the baked-in default is not the only one', async () => { + setGlobalConfiguration( + new ConfigurationBuilder() + .put('ACME_SDK_LOG_LEVEL', 'headers') + .put(CFG_KEY_LOG_LEVEL, 'none') + .build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({logger, configKey: 'ACME_SDK_LOG_LEVEL'}), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events.map(e => e.event)).toEqual(['http.request', 'http.response']); + }); + + test('an explicit granularity still wins over the configured key (OBS-34)', async () => { + setGlobalConfiguration( + new ConfigurationBuilder().put('ACME_SDK_LOG_LEVEL', 'headers').build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({ + logger, + configKey: 'ACME_SDK_LOG_LEVEL', + granularity: 'none', + }), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events).toHaveLength(0); + }); +}); + +describe('the per-request span ends exactly once (OBS-21, OBS-29)', () => { + test('an end() that throws on the success path is not called a second time', async () => { + const {logger} = spyLogger(); + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span = { + isRecording: true, + setAttribute: () => span, + recordException: (error: unknown) => { + exceptions.push(error); + return span; + }, + end: (): void => { + ends += 1; + throw endFailed; + }, + }; + + let caught: unknown; + try { + await send( + loggingStep({ + logger, + granularity: 'headers', + tracerFactory: () => ({startSpan: () => span}), + }), + new FakeTransport([countingResponse(200).response]), + ); + } catch (error) { + caught = error; + } + + expect(caught).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([]); + }); +}); diff --git a/packages/core/src/observability/logging-step.ts b/packages/core/src/observability/logging-step.ts new file mode 100644 index 0000000..dfd5cd2 --- /dev/null +++ b/packages/core/src/observability/logging-step.ts @@ -0,0 +1,557 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/logging-step.ts +import type {StepDescriptor, StepContext} from '../pipeline/step.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {withRequestLogging} from '../body/request-body-logging.js'; +import { + withResponseLogging, + type LoggedResponseBody, +} from '../body/response-body-logging.js'; +import { + CFG_KEY_LOG_LEVEL, + getGlobalConfiguration, +} from '../config/configuration.js'; +import {defaultClock, type Clock} from '../config/clock.js'; +import {toError} from '../http/errors.js'; +import {decodeBodyText, resolveCharset} from '../http/charset.js'; +import {invariant} from '../invariant.js'; +import { + getGlobalLogger, + type LogEvent, + type LogLevel, + type Logger, +} from './logger.js'; +import { + redactHeaderValue, + redactUrl, + type DroppedHeaderPolicy, +} from './redaction.js'; +import { + NOOP_METER, + type Counter, + type Histogram, + type Meter, +} from './metrics.js'; +import { + NOOP_TRACER, + activateSpanForCorrelation, + type Tracer, +} from './tracing.js'; + +/** + * Logging granularity levels for HTTP request/response logging (OBS-34). + * + * @public + */ +export type LoggingGranularity = 'none' | 'headers' | 'body'; + +/** + * Settings for configuring the {@link loggingStep} (OBS-34..39). + * + * @public + */ +export interface LoggingStepSettings { + /** The logger to emit events to (default: getGlobalLogger()). */ + readonly logger?: Logger | undefined; + /** + * Severity the http.request/http.response events emit at (OBS-2's axis). + * Default: 'info' (failures always emit at 'error'). + */ + readonly severity?: LogLevel | undefined; + /** Granularity of logging (default: resolved from Configuration via {@link LoggingStepSettings.configKey}, fallback 'none'). */ + readonly granularity?: LoggingGranularity | undefined; + /** + * The configuration key the ambient granularity is read from when `granularity` is omitted (OBS-35). + * + * OBS-35 says the SDK MUST NOT bake in a default key name, and `CFG_KEY_LOG_LEVEL` + * (`DEXPACE_LOG_LEVEL`) is exactly that — CFG-14's well-known key, kept as the default because a + * required key would mean no caller gets ambient logging without naming one first. Set this to read + * a host application's own key instead; the resolution is the same layered, tolerant one either way. + * + * Nothing installs a configuration that reads the process environment by default: the global slot + * starts empty (CFG-13), so `setGlobalConfiguration(defaultConfiguration())` is the wiring that + * makes any environment variable — this one included — reachable. See + * `docs/sdk-documentation/pipelines.md`, "Turning logging on from the environment". + * + * @defaultValue {@link CFG_KEY_LOG_LEVEL} + */ + readonly configKey?: string | undefined; + /** Byte limit for request/response body previews (default: 8192). */ + readonly previewSizeBytes?: number | undefined; + /** Optional custom tracer factory. */ + readonly tracerFactory?: (() => Tracer) | undefined; + /** Metrics meter instance (default: NOOP_METER). */ + readonly meter?: Meter | undefined; + /** Policy for non-allow-listed headers (default: 'mark'). */ + readonly droppedHeaderPolicy?: DroppedHeaderPolicy | undefined; + /** Injected clock seam for duration measurement (default: defaultClock). */ + readonly clock?: Clock | undefined; +} + +const DEFAULT_PREVIEW_SIZE_BYTES = 8192; + +function resolveGranularity(settings: LoggingStepSettings): LoggingGranularity { + if (settings.granularity !== undefined) { + const rawSetting = settings.granularity.trim().toLowerCase(); + if (rawSetting === 'headers' || rawSetting === 'body') return rawSetting; + return 'none'; + } + const raw = getGlobalConfiguration() + .getString(settings.configKey ?? CFG_KEY_LOG_LEVEL, 'none') + ?.trim() + .toLowerCase(); + if (raw === 'headers' || raw === 'body') return raw; + return 'none'; +} + +function isTextMediaType(contentType: string | undefined): boolean { + if (contentType === undefined) return true; + const raw = contentType.toLowerCase().split(';')[0]?.trim() ?? ''; + if (raw === '') return true; + if ( + raw.startsWith('text/') || + raw === 'application/json' || + raw === 'application/xml' || + raw === 'application/javascript' || + raw === 'application/x-www-form-urlencoded' || + raw.endsWith('+json') || + raw.endsWith('+xml') || + raw.endsWith('+text') + ) { + return true; + } + return false; +} + +function renderBodyPreview( + bytes: Uint8Array | undefined, + contentType: string | undefined, +): string | undefined { + if (bytes === undefined || !(bytes instanceof Uint8Array)) return undefined; + if (bytes.length === 0) return ''; + try { + if (isTextMediaType(contentType)) { + return decodeBodyText(bytes, resolveCharset(contentType)); + } + return `[binary ${String(bytes.length)} bytes captured]`; + } catch { + return `[binary ${String(bytes.length)} bytes captured]`; + } +} + +interface LogEventBuilder { + readonly target: LogEvent; + readonly prefix: 'http.request' | 'http.response'; + readonly policy: DroppedHeaderPolicy; +} + +function addHeaderFields( + event: LogEventBuilder, + headers: Iterable<readonly [string, string]>, +): void { + for (const [name, value] of headers) { + const redacted = redactHeaderValue(name, value, event.policy); + if (redacted !== undefined) { + event.target.field( + `${event.prefix}.header.${name.toLowerCase()}`, + redacted, + ); + } + } +} + +/** Containment: logging failures must never cause the HTTP request pipeline to throw or fail (OBS-20). */ +function safeEmit(logger: Logger, build: () => void): void { + try { + build(); + } catch (error) { + try { + logger + .atLevel('verbose') + .event('http.instrumentation.logFailure') + .cause(error) + .emit(); + } catch { + // swallowed per OBS-20 + } + } +} + +/** + * OBS-20's body-drain clause. A capture failure is contained -- the request completes and the caller's + * body is untouched -- but containment is not silence: the failure re-surfaces as a best-effort + * `http.instrumentation.*` diagnostic, through the same {@link safeEmit} every other emission uses, so a + * secondary failure while reporting it is swallowed in turn. + * + * `verbose`, the level its sibling `http.instrumentation.logFailure` already emits at: nothing about the + * request changed, and what was lost is a diagnostic preview. Before 2026-09-05 both catches returned an + * empty capture and emitted nothing at all, so a `fileBody()` over a deleted file logged + * `"http.request.body.preview": ""` with no trace of why (audit #67 / #80). + */ +function emitBodyCaptureFailure( + logger: Logger, + direction: 'request' | 'response', + error: unknown, +): void { + safeEmit(logger, () => { + logger + .atLevel('verbose') + .event('http.instrumentation.bodyCaptureFailed') + .field('http.message.direction', direction) + .cause(error) + .emit(); + }); +} + +/** + * Stable identity symbol for the LOGGING pillar step. + * + * @public + */ +export const LOGGING_STEP_TYPE: unique symbol = Symbol('dexpace.logging'); + +interface EmitContext { + readonly logger: Logger; + readonly severity: LogLevel; + readonly granularity: LoggingGranularity; + readonly policy: DroppedHeaderPolicy; + readonly previewSizeBytes: number; +} + +interface BodyPreviewResult { + readonly preview: string | undefined; + readonly size: number | undefined; +} + +function emitRequestEvent( + context: EmitContext, + request: Request, + bodyInfo?: BodyPreviewResult, +): void { + if (context.granularity === 'none') return; + safeEmit(context.logger, () => { + const event = context.logger + .atLevel(context.severity) + .event('http.request') + .field('http.request.method', request.method) + .field('url.full', redactUrl(request.url)); + addHeaderFields( + {target: event, prefix: 'http.request', policy: context.policy}, + request.headers.entries(), + ); + if (bodyInfo?.preview !== undefined) { + event.field('http.request.body.preview', bodyInfo.preview); + if (bodyInfo.size !== undefined) { + event.field('http.request.body.size', bodyInfo.size); + } + } + event.emit(); + }); +} + +function emitResponseEvent( + context: EmitContext, + outcome: { + response: Response; + elapsedMs: number; + preview: string | undefined; + size: number | undefined; + }, +): void { + if (context.granularity === 'none') return; + safeEmit(context.logger, () => { + const event = context.logger + .atLevel(context.severity) + .event('http.response') + .field('http.response.status_code', outcome.response.status.code) + .field('http.response.duration_ms', outcome.elapsedMs); + addHeaderFields( + {target: event, prefix: 'http.response', policy: context.policy}, + outcome.response.headers.entries(), + ); + if (outcome.preview !== undefined) { + event.field('http.response.body.preview', outcome.preview); + if (outcome.size !== undefined) { + event.field('http.response.body.size', outcome.size); + } + } + event.emit(); + }); +} + +function emitFailureEvent( + context: EmitContext, + outcome: {error: Error; elapsedMs: number}, +): void { + if (context.granularity === 'none') return; + safeEmit(context.logger, () => { + context.logger + .atLevel('error') + .event('http.response') + .field('error.type', outcome.error.name) + .field('http.response.duration_ms', outcome.elapsedMs) + .cause(outcome.error) + .emit(); + }); +} + +function resolveTracer( + settings: LoggingStepSettings, + ctx: StepContext, +): Tracer { + if (settings.tracerFactory !== undefined) return settings.tracerFactory(); + + const factory = ctx.context.instrumentation.tracerFactory as + ((operationName: string) => Tracer | undefined) | undefined; + if (typeof factory !== 'function') return NOOP_TRACER; + + const opName = + 'operationName' in ctx.context ? ctx.context.operationName : undefined; + const created = factory(opName ?? 'http.client.request'); + return created ?? NOOP_TRACER; +} + +/** Captures response body preview safely when content-length is declared (OBS-36, OBS-37). */ +async function captureResponseBody( + response: Response, + context: EmitContext, +): Promise<{ + readonly response: Response; + readonly preview: string | undefined; + readonly size: number | undefined; +}> { + const previewSizeBytes = context.previewSizeBytes; + try { + const hasContentLength = response.headers.has('content-length'); + if (response.body === null || !hasContentLength) { + return {response, preview: undefined, size: undefined}; + } + const rawLen = Number.parseInt( + response.headers.get('content-length') ?? '-1', + 10, + ); + const declaredLen = Number.isFinite(rawLen) ? rawLen : -1; + if (declaredLen < 0) { + return {response, preview: undefined, size: undefined}; + } + const loggedResponse: LoggedResponseBody = withResponseLogging( + response.body, + previewSizeBytes, + declaredLen, + ); + const loggedStream = await loggedResponse.read(); + const captured = response.newBuilder().body(loggedStream).build(); + const snap = loggedResponse.snapshot(); + const preview = renderBodyPreview( + snap, + response.headers.get('content-type'), + ); + const size = snap.length > 0 ? snap.length : undefined; + return {response: captured, preview, size}; + } catch (error) { + // OBS-20: a body-drain failure must never fail the request -- and must not vanish either. + emitBodyCaptureFailure(context.logger, 'response', error); + return {response, preview: undefined, size: undefined}; + } +} + +async function prepareRequestBody( + request: Request, + context: EmitContext, +): Promise<{ + readonly outbound: Request; + readonly preview: string | undefined; + readonly size: number | undefined; +}> { + if (context.granularity !== 'body' || request.body === undefined) { + return {outbound: request, preview: undefined, size: undefined}; + } + const logged = withRequestLogging(request.body, context.previewSizeBytes); + if (logged.replayable) { + const probeSink = new WritableStream<Uint8Array>({ + write: () => undefined, + }); + try { + await logged.writeTo(probeSink); + } catch (error) { + // OBS-20: the probe is diagnostic-only, so its failure is contained -- and reported. + emitBodyCaptureFailure(context.logger, 'request', error); + } + } + const snap = logged.snapshot(); + const preview = renderBodyPreview( + snap, + request.headers.get('content-type') ?? request.body.mediaType, + ); + const size = snap.length > 0 ? snap.length : undefined; + const outbound = request.newBuilder().body(logged).build(); + return {outbound, preview, size}; +} + +interface StepInstruments { + readonly requestCounter: Counter; + readonly requestDuration: Histogram; + readonly clock: Clock; +} + +interface ExecutionPlan { + readonly settings: LoggingStepSettings; + readonly emitContext: EmitContext; + readonly instruments: StepInstruments; + readonly previewSizeBytes: number; +} + +interface PipelineExecutionArgs { + readonly ctx: StepContext; + readonly plan: ExecutionPlan; + readonly outbound: Request; + readonly startedAt: number; + readonly span: ReturnType<Tracer['startSpan']>; +} + +async function executePipeline(args: PipelineExecutionArgs): Promise<Response> { + const {ctx, plan, outbound, startedAt, span} = args; + const {emitContext, instruments} = plan; + try { + const response = await ctx.next(outbound); + const { + response: captured, + preview, + size, + } = emitContext.granularity === 'body' + ? await captureResponseBody(response, emitContext) + : {response, preview: undefined, size: undefined}; + + const elapsedMs = instruments.clock.monotonic() - startedAt; + instruments.requestCounter.add(1, { + method: outbound.method, + status: captured.status.code, + }); + instruments.requestDuration.record(elapsedMs, { + method: outbound.method, + status: captured.status.code, + }); + emitResponseEvent(emitContext, { + response: captured, + elapsedMs, + preview, + size, + }); + + return captured; + } catch (caught) { + const error = toError(caught); + const elapsedMs = instruments.clock.monotonic() - startedAt; + + instruments.requestCounter.add(1, { + method: outbound.method, + errorType: error.name, + }); + instruments.requestDuration.record(elapsedMs, { + method: outbound.method, + errorType: error.name, + }); + emitFailureEvent(emitContext, {error, elapsedMs}); + + span.recordException(error); + throw caught; + } +} + +async function handleRequestExecution( + request: Request, + ctx: StepContext, + plan: ExecutionPlan, +): Promise<Response> { + const {settings, emitContext, instruments} = plan; + const tracer = resolveTracer(settings, ctx); + const span = tracer.startSpan('http.client.request'); + const scope = activateSpanForCorrelation(span); + const startedAt = instruments.clock.monotonic(); + + try { + const {outbound, preview, size} = await prepareRequestBody( + request, + emitContext, + ); + emitRequestEvent(emitContext, outbound, {preview, size}); + return await executePipeline({ + ctx, + plan, + outbound, + startedAt, + span, + }); + } finally { + // ONE exit for `end()`, and it is here rather than on each path inside `executePipeline`: an + // `end()` that threw on the success path used to land in that function's own `catch`, which + // recorded the exception and called `end()` a second time on a span the tracer had already + // closed (OBS-21's idempotent-end clause is the tracer's promise, not this step's licence). + // Nested rather than sequential so a throwing `end()` -- which OBS-20 deliberately does not + // catch, because OBS-30 makes it the SPI's promise not to -- still cannot leak the scope. + try { + span.end(); + } finally { + scope.close(); + } + } +} + +/** + * Creates the LOGGING pillar step descriptor (OBS-34..39). + * + * @param settings - optional logging step settings. + * @returns the step descriptor. + * + * @public + */ +export function loggingStep( + settings: LoggingStepSettings = {}, +): StepDescriptor { + const meter = settings.meter ?? NOOP_METER; + const clock = settings.clock ?? defaultClock; + const policy = settings.droppedHeaderPolicy ?? 'mark'; + const severity: LogLevel = settings.severity ?? 'info'; + const previewSizeBytes = + settings.previewSizeBytes ?? DEFAULT_PREVIEW_SIZE_BYTES; + + invariant( + previewSizeBytes > 0, + 'loggingStep: previewSizeBytes must be positive', + ); + invariant( + Number.isFinite(previewSizeBytes), + 'loggingStep: previewSizeBytes must be finite', + ); + + const instruments: StepInstruments = { + requestCounter: meter.createCounter('http.client.request.count', { + unit: '{request}', + }), + requestDuration: meter.createHistogram('http.client.request.duration', { + unit: 'ms', + }), + clock, + }; + + return { + type: LOGGING_STEP_TYPE, + stage: 'LOGGING', + fn: (request, ctx) => { + const emitContext: EmitContext = { + logger: settings.logger ?? getGlobalLogger(), + severity, + granularity: resolveGranularity(settings), + policy, + previewSizeBytes, + }; + return handleRequestExecution(request, ctx, { + settings, + emitContext, + instruments, + previewSizeBytes, + }); + }, + }; +} diff --git a/packages/core/src/observability/metrics.test.ts b/packages/core/src/observability/metrics.test.ts new file mode 100644 index 0000000..2139b8b --- /dev/null +++ b/packages/core/src/observability/metrics.test.ts @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/metrics.test.ts +// Exercises: OBS-31 (no-op default discards every measurement, returns shared instrument singletons), OBS-33 +// (a histogram tolerates NaN/Infinity without throwing). +import {describe, expect, test} from 'bun:test'; +import {NOOP_METER} from './metrics.js'; + +describe('NOOP_METER (OBS-31)', () => { + test('createCounter returns the same shared instrument regardless of name', () => { + expect(NOOP_METER.createCounter('a')).toBe(NOOP_METER.createCounter('b')); + }); + + test('createHistogram returns the same shared instrument regardless of name', () => { + expect(NOOP_METER.createHistogram('a')).toBe( + NOOP_METER.createHistogram('b'), + ); + }); + + test('recording into the no-op instruments never throws, including non-finite values (OBS-33)', () => { + const counter = NOOP_METER.createCounter('http.client.request.count', { + unit: '{request}', + }); + const histogram = NOOP_METER.createHistogram( + 'http.client.request.duration', + {unit: 'ms'}, + ); + expect(() => { + counter.add(1, {method: 'GET'}); + }).not.toThrow(); + expect(() => { + histogram.record(Number.NaN); + }).not.toThrow(); + expect(() => { + histogram.record(Number.POSITIVE_INFINITY); + }).not.toThrow(); + }); +}); diff --git a/packages/core/src/observability/metrics.ts b/packages/core/src/observability/metrics.ts new file mode 100644 index 0000000..53d38e3 --- /dev/null +++ b/packages/core/src/observability/metrics.ts @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/metrics.ts + +/** + * Counter metric instrument (OBS-31, OBS-33). + * + * @public + */ +export interface Counter { + /** OBS-33: only non-negative increments are valid. */ + add(delta: number, attributes?: Readonly<Record<string, unknown>>): void; +} + +/** + * Histogram metric instrument (OBS-31, OBS-33). + * + * @public + */ +export interface Histogram { + /** OBS-33: tolerates any input, including non-finite values, without throwing. */ + record(value: number, attributes?: Readonly<Record<string, unknown>>): void; +} + +/** + * Metrics provider interface (OBS-31). + * + * @public + */ +export interface Meter { + createCounter( + name: string, + options?: {readonly unit?: string; readonly description?: string}, + ): Counter; + createHistogram( + name: string, + options?: {readonly unit?: string; readonly description?: string}, + ): Histogram; +} + +const NOOP_COUNTER: Counter = Object.freeze({ + add(): void { + return; + }, +}); +const NOOP_HISTOGRAM: Histogram = Object.freeze({ + record(): void { + return; + }, +}); + +/** + * Inert no-op {@link Meter} singleton (OBS-31). + * + * @public + */ +export const NOOP_METER: Meter = Object.freeze({ + createCounter(): Counter { + return NOOP_COUNTER; + }, + createHistogram(): Histogram { + return NOOP_HISTOGRAM; + }, +}); diff --git a/packages/core/src/observability/redaction.test.ts b/packages/core/src/observability/redaction.test.ts new file mode 100644 index 0000000..8ce88ff --- /dev/null +++ b/packages/core/src/observability/redaction.test.ts @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/redaction.test.ts +// Exercises: XCUT-19 (logging/telemetry redacts secrets BY DEFAULT: userinfo is never allow-listable, +// query and fragment key=value tokens are redacted unless explicitly allow-listed, and header logging is +// default-deny -- the whole clause is asserted across this file and credential.test.ts's no-secret-in- +// string-form rows), +// OBS-11 (userinfo always redacted), OBS-12 (query allow-list, default {api-version}), OBS-13 +// (fragment key=value tokens redacted the same way, plain fragment preserved), OBS-14 (scheme/host/port/path +// untouched, no spurious "?"), OBS-15 (malformed URL -> fixed sentinel, never throws), OBS-16 (header-value +// URL: absolute redacted like a request URL, relative keeps path + "?***" marker), OBS-18 (header-name +// allow-list, default-deny). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {redactHeaderValue, redactUrl} from './redaction.js'; + +describe('redactUrl: components and allow-lists (OBS-11..14)', () => { + test('userinfo is always redacted', () => { + const redacted = redactUrl('https://user:secret@example.com/path'); + expect(redacted).not.toContain('user'); + expect(redacted).not.toContain('secret'); + expect(redacted).toContain('***:***@'); + }); + + test('query values are redacted unless allow-listed (default {api-version})', () => { + const redacted = redactUrl('https://example.com/p?api-version=1&token=abc'); + expect(redacted).toContain('api-version=1'); + expect(redacted).toContain('token=***'); + }); + + test('empty query allow-list redacts every query param', () => { + const redacted = redactUrl( + 'https://example.com/p?api-version=1&token=abc', + new Set(), + ); + expect(redacted).toContain('api-version=***'); + expect(redacted).toContain('token=***'); + }); + + test('a fragment key=value token is redacted; a plain fragment is preserved', () => { + expect(redactUrl('https://example.com/p#access_token=SECRET')).toContain( + 'access_token=***', + ); + expect(redactUrl('https://example.com/p#section')).toContain('#section'); + expect(redactUrl('https://example.com/p#')).toBe('https://example.com/p#'); + expect(redactUrl('https://example.com/p')).toBe('https://example.com/p'); + }); + + test('scheme, host, port, and path are never altered', () => { + const redacted = redactUrl('https://example.com:8443/a/b?token=x'); + expect(redacted).toContain('https://example.com:8443/a/b'); + }); + + test('preserves encoded query parameter names containing spaces (OBS-12)', () => { + expect(redactUrl('https://example.com/p?a%20b=1')).toBe( + 'https://example.com/p?a%20b=***', + ); + }); + + test('opaque URLs like mailto are not altered to contain double slashes', () => { + expect(redactUrl('mailto:a@b.com')).toBe('mailto:a@b.com'); + }); +}); + +describe('redactUrl: delimiters and total safety (OBS-14..15)', () => { + test('a present-but-empty query keeps its trailing "?" (OBS-14)', () => { + expect(redactUrl('https://example.com/p?')).toBe('https://example.com/p?'); + }); + + test('a URL with no query gains no spurious "?" (OBS-14)', () => { + expect(redactUrl('https://example.com/p')).toBe('https://example.com/p'); + }); + + test('a "?" inside the fragment is not treated as a query delimiter (OBS-14)', () => { + expect(redactUrl('https://example.com/p#a?b')).toBe( + 'https://example.com/p#a?b', + ); + }); + + test('handles URL object input', () => { + const urlObj = new URL('https://example.com/p?api-version=2&secret=123'); + expect(redactUrl(urlObj)).toBe( + 'https://example.com/p?api-version=2&secret=***', + ); + const emptyQueryUrl = new URL('https://example.com/p'); + expect(redactUrl(emptyQueryUrl)).toBe('https://example.com/p'); + + const emptyTrailingQueryUrl = new URL('https://example.com/p?'); + expect(redactUrl(emptyTrailingQueryUrl)).toBe('https://example.com/p?'); + + const emptyTrailingHashUrl = new URL('https://example.com/p#'); + expect(redactUrl(emptyTrailingHashUrl)).toBe('https://example.com/p#'); + }); + + test('protocol-relative header URLs redact properly', () => { + expect(redactHeaderValue('Location', '//example.com/path?secret=123')).toBe( + '//example.com/path?***', + ); + }); + + test('a malformed URL redacts to the fixed sentinel, never throwing', () => { + expect(() => redactUrl('not a url at all ###')).not.toThrow(); + expect(redactUrl('not a url at all ###')).toBe('[malformed url]'); + }); + + test('the output is re-rendered from the parsed URL, so WHATWG normalisation shows (OBS-14)', () => { + // Pinned, not fixed: the result is assembled from `URL`'s components, so host case, a default + // port and an empty path normalise on the way through. Documented on `redactUrl` as inherent to + // parsing, and left as is by audit #67 / #80 -- re-rendering the caller's authority by hand would + // be a second URL renderer for no gain in what OBS-11..15 asks for. + expect(redactUrl('https://EXAMPLE.com:443')).toBe('https://example.com/'); + expect(redactUrl('http://Example.COM:80/p')).toBe('http://example.com/p'); + }); + + test('property: never throws for any string', () => { + fc.assert( + fc.property(fc.string(), value => { + expect(() => redactUrl(value)).not.toThrow(); + }), + ); + }); +}); + +describe('redactHeaderValue (OBS-16, OBS-17, OBS-18)', () => { + test('an allow-listed header name passes its value through', () => { + expect(redactHeaderValue('Content-Type', 'application/json')).toBe( + 'application/json', + ); + }); + + test('a non-allow-listed header name is marked, not passed through (default-deny, "mark" policy)', () => { + expect(redactHeaderValue('Authorization', 'Bearer secret')).toBe( + 'REDACTED', + ); + }); + + test('the "omit" policy drops a non-allow-listed header entirely (OBS-18)', () => { + expect( + redactHeaderValue('Authorization', 'Bearer secret', 'omit'), + ).toBeUndefined(); + }); + + test('a Location header carrying a query is redacted through the URL-value redactor', () => { + const value = redactHeaderValue('Location', '/callback?code=SECRET'); + expect(value).toContain('/callback?***'); + expect(value).not.toContain('SECRET'); + }); + + test('a Content-Location header carrying an absolute URL is redacted through the URL-value redactor', () => { + const value = redactHeaderValue( + 'Content-Location', + 'https://example.com/cb?token=SECRET', + ); + expect(value).toContain('https://example.com/cb?token=***'); + expect(value).not.toContain('SECRET'); + }); + + test('a relative path with no query/fragment passes through unchanged', () => { + expect(redactHeaderValue('Location', '/plain/path')).toBe('/plain/path'); + }); + + test('custom query allow-list is case-insensitive for param names', () => { + const redacted = redactUrl( + 'https://example.com/p?Custom-Param=val&secret=123', + new Set(['custom-PARAM']), + ); + expect(redacted).toContain('Custom-Param=val'); + expect(redacted).toContain('secret=***'); + }); + + test('rejects non-string header name or value', () => { + expect(() => redactHeaderValue(null as unknown as string, 'val')).toThrow(); + expect(() => + redactHeaderValue('Location', null as unknown as string), + ).toThrow(); + }); +}); diff --git a/packages/core/src/observability/redaction.ts b/packages/core/src/observability/redaction.ts new file mode 100644 index 0000000..b398840 --- /dev/null +++ b/packages/core/src/observability/redaction.ts @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/redaction.ts +// Exercises: OBS-11..18 +import {invariant} from '../invariant.js'; + +const DEFAULT_QUERY_ALLOW_LIST = new Set(['api-version']); +const DEFAULT_HEADER_ALLOW_LIST = new Set([ + 'content-type', + 'content-length', + 'accept', + 'user-agent', +]); +const MALFORMED_URL_SENTINEL = '[malformed url]'; + +function redactQueryString( + search: URLSearchParams, + allowList: ReadonlySet<string>, +): string { + const pairs: string[] = []; + for (const [key, value] of search) { + const val = allowList.has(key.toLowerCase()) + ? encodeURIComponent(value) + : '***'; + pairs.push(`${encodeURIComponent(key)}=${val}`); + } + return pairs.join('&'); +} + +function redactFragment(hash: string, allowList: ReadonlySet<string>): string { + if (hash === '' || hash === '#') return hash; + const raw = hash.slice(1); + if (!raw.includes('=')) return hash; + const tokens = raw.split('&').map(token => { + const [key, ...rest] = token.split('='); + if (key === undefined || rest.length === 0) return token; + return allowList.has(key.toLowerCase()) ? token : `${key}=***`; + }); + return `#${tokens.join('&')}`; +} + +function hasQueryDelimiter(input: URL | string): boolean { + const raw = typeof input === 'string' ? input : input.href; + const beforeFragment = raw.split('#')[0] ?? raw; + return beforeFragment.includes('?'); +} + +function hasHashDelimiter(input: URL | string): boolean { + const raw = typeof input === 'string' ? input : input.href; + return raw.includes('#'); +} + +/** + * Redacts sensitive components from a URL according to spec rules (OBS-11..15). + * + * **The result is a re-rendered URL, not the caller's string with holes in it.** Every input goes + * through WHATWG `URL`, and the output is assembled from its parsed components, so the normalisations + * parsing performs come with it: the host is lower-cased, a default port for the scheme + * (`https://h:443/`) is dropped, a missing path becomes `/`, and percent-encoding is canonicalised. + * A log line therefore need not match the request line byte for byte. That is inherent to parsing and + * is left as is deliberately (audit #67 / #80): re-rendering the original authority by hand would mean + * a second URL renderer in this package, maintained against WHATWG, for no gain in what OBS-11..15 + * actually asks for — that userinfo, non-allow-listed query values and fragment values do not reach a + * log. Compare identity elsewhere; this is for humans and log pipelines. + * + * @param input - the URL or string to redact. + * @param queryAllowList - set of allowed query parameter names (default: \{api-version\}). + * @returns the redacted URL string, or '[malformed url]' if parsing fails. + * + * @internal + */ +export function redactUrl( + input: URL | string, + queryAllowList: ReadonlySet<string> = DEFAULT_QUERY_ALLOW_LIST, +): string { + try { + const rawInput = typeof input === 'string' ? input : input.href; + const url = typeof input === 'string' ? new URL(input) : input; + const userinfo = + url.username !== '' || url.password !== '' ? '***:***@' : ''; + const normalizedAllowList = + queryAllowList === DEFAULT_QUERY_ALLOW_LIST + ? DEFAULT_QUERY_ALLOW_LIST + : new Set(Array.from(queryAllowList, k => k.toLowerCase())); + const query = redactQueryString(url.searchParams, normalizedAllowList); + let fragment = redactFragment(url.hash, normalizedAllowList); + // WHATWG URL normalizes empty query '?' and hash '#' to empty strings; preserve original delimiter presence. + if (fragment === '' && hasHashDelimiter(input)) { + fragment = '#'; + } + const separator = query !== '' || hasQueryDelimiter(input) ? '?' : ''; + const slashes = + url.host !== '' || rawInput.startsWith(`${url.protocol}//`) ? '//' : ''; + return `${url.protocol}${slashes}${userinfo}${url.host}${url.pathname}${separator}${query}${fragment}`; + } catch { + return MALFORMED_URL_SENTINEL; + } +} + +/** Handles header values (like Location or Content-Location) that may be absolute URLs or relative paths. */ +function redactAbsoluteOrRelativeUrl(value: string): string { + try { + return redactUrl(new URL(value)); + } catch { + const hasQueryOrFragment = value.includes('?') || value.includes('#'); + if (!hasQueryOrFragment) return value; + const path = value.split(/[?#]/u)[0] ?? value; + return `${path}?***`; + } +} + +/** + * Policy for handling non-allow-listed headers (OBS-18). + * + * @public + */ +export type DroppedHeaderPolicy = 'mark' | 'omit'; + +const REDACTED_MARKER = 'REDACTED'; + +/** + * Redacts a header value based on header name allow-list and policy (OBS-16..18). + * + * @param name - the header name. + * @param value - the header value. + * @param policy - whether to mark with REDACTED or omit (default: 'mark'). + * @returns the redacted value, or undefined if omitted. + * + * @internal + */ +export function redactHeaderValue( + name: string, + value: string, + policy: DroppedHeaderPolicy = 'mark', +): string | undefined { + invariant( + typeof name === 'string', + 'redactHeaderValue: name must be a string', + ); + invariant( + typeof value === 'string', + 'redactHeaderValue: value must be a string', + ); + + const lowerName = name.toLowerCase(); + if (lowerName === 'location' || lowerName === 'content-location') { + return redactAbsoluteOrRelativeUrl(value); + } + if (DEFAULT_HEADER_ALLOW_LIST.has(lowerName)) return value; + return policy === 'mark' ? REDACTED_MARKER : undefined; +} diff --git a/packages/core/src/observability/span.ts b/packages/core/src/observability/span.ts new file mode 100644 index 0000000..9b45f33 --- /dev/null +++ b/packages/core/src/observability/span.ts @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/span.ts + +// The inert half of the tracing surface: the span/tracer shapes (OBS-21, OBS-23) and the two no-op +// singletons (OBS-25) that need nothing else in the package to exist. +// +// This module exists to break an import cycle, not to introduce a layer. `tracing.ts` needs +// `InstrumentationBundle` from `context/instrumentation.ts` to type `createInstrumentationBundle`, and +// `context/instrumentation.ts` needs `NOOP_SPAN` for CTX-15's disabled-tracing default. Holding both in +// `tracing.ts` closes a cycle that `bun run verify:import-cycles` rejects -- type-only edges count there, +// deliberately -- and that gate's own message prescribes this fix: move the shared declaration into a +// module both sides can import. So this file imports nothing, and `tracing.ts` re-exports every name +// below, which is why no existing import path changed. +// +// Line comments, and deliberately none of them writes out the internal-marker JSDoc tag. Two traps stack +// at the top of a module: a leading `/** */` block binds to the first declaration below it rather than to +// the file, and gts turns on `stripInternal`, whose test is a plain substring scan of EVERY leading comment +// range of a declaration -- line comments included, doc block or not. So a module header that merely +// MENTIONS that tag deletes `SpanContext` from the emitted `.d.ts` while `tsc` stays silent; the failure +// surfaces one package later, as an unresolved name inside core's own `dist/`. Measured twice on the way +// to this wording. If this note is ever reworded, rebuild and check `dist/observability/span.d.ts` still +// declares all four names. + +/** + * Contextual metadata identifying a trace and span in distributed tracing (OBS-23, OBS-26). + * + * @public + */ +export interface SpanContext { + readonly traceId: string; + readonly spanId: string; + readonly traceFlags?: number | undefined; + readonly traceState?: string | undefined; +} + +/** + * A structural subset of `@opentelemetry/api`'s own `Span` shape (OBS-21, OBS-23). + * + * @public + */ +export interface Span { + readonly isRecording: boolean; + setAttribute(key: string, value: unknown): this; + recordException(error: unknown): this; + end(): void; + spanContext?(): SpanContext | undefined; +} + +/** + * Tracing facade interface for creating spans. + * + * @public + */ +export interface Tracer { + startSpan(name: string): Span; +} + +/** + * Inert no-op {@link Span} singleton (OBS-21, OBS-25). + * + * @public + */ +export const NOOP_SPAN: Span = Object.freeze({ + isRecording: false, + setAttribute(): Span { + return NOOP_SPAN; + }, + recordException(): Span { + return NOOP_SPAN; + }, + end(): void { + return; + }, + spanContext(): SpanContext { + return {traceId: '0'.repeat(32), spanId: '0'.repeat(16)}; + }, +}); + +/** + * Inert no-op {@link Tracer} singleton (OBS-25). + * + * @public + */ +export const NOOP_TRACER: Tracer = Object.freeze({ + startSpan(): Span { + return NOOP_SPAN; + }, +}); diff --git a/packages/core/src/observability/tracing.test.ts b/packages/core/src/observability/tracing.test.ts new file mode 100644 index 0000000..1c9b96e --- /dev/null +++ b/packages/core/src/observability/tracing.test.ts @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/tracing.test.ts +// Exercises: OBS-21 (non-recording span: inert mutators, idempotent end), OBS-22 (activation scope restores +// the prior span, including on throw), OBS-23 (correlation push/restore, skipped for a non-recording span), +// OBS-25 (allocation-free no-op singletons), OBS-26/27 (W3C 32-hex trace id / 16-hex span id, never +// all-zero; Datadog 64-bit decimal). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {getDiagnosticContext} from './diagnostic-context.js'; +import { + NOOP_SPAN, + NOOP_TRACER, + activateSpan, + activateSpanForCorrelation, + createInstrumentationBundle, + generateSpanId, + generateTraceId, + getActiveSpan, + type Span, +} from './tracing.js'; + +describe('NOOP_SPAN (OBS-21, OBS-25)', () => { + test('is non-recording and every mutator is inert, returning the same instance', () => { + expect(NOOP_SPAN.isRecording).toBe(false); + expect(NOOP_SPAN.setAttribute('k', 'v')).toBe(NOOP_SPAN); + expect(NOOP_SPAN.recordException(new Error('x'))).toBe(NOOP_SPAN); + }); + + test('end() is idempotent', () => { + expect(() => { + NOOP_SPAN.end(); + NOOP_SPAN.end(); + }).not.toThrow(); + }); +}); + +describe('NOOP_TRACER (OBS-25)', () => { + test('startSpan returns the shared NOOP_SPAN singleton, allocating nothing new', () => { + expect(NOOP_TRACER.startSpan('op-a')).toBe(NOOP_SPAN); + expect(NOOP_TRACER.startSpan('op-b')).toBe(NOOP_SPAN); + }); +}); + +describe('trace/span id generation (OBS-26, OBS-27)', () => { + test('W3C trace ids are 32 lowercase hex chars, never all-zero', () => { + for (let i = 0; i < 1000; i += 1) { + const id = generateTraceId('w3c'); + expect(id).toMatch(/^[0-9a-f]{32}$/u); + expect(id).not.toBe('0'.repeat(32)); + } + }); + + test('span ids are 16 lowercase hex chars, never all-zero', () => { + for (let i = 0; i < 1000; i += 1) { + const id = generateSpanId(); + expect(id).toMatch(/^[0-9a-f]{16}$/u); + expect(id).not.toBe('0'.repeat(16)); + } + }); + + test('Datadog trace ids are decimal, non-zero, within the 64-bit unsigned range', () => { + const id = generateTraceId('datadog'); + expect(id).toMatch(/^\d+$/u); + expect(BigInt(id)).toBeGreaterThan(0n); + expect(BigInt(id)).toBeLessThan(2n ** 64n); + }); + + test('the no-op flavor always yields the invalid all-zero sentinel', () => { + expect(generateTraceId('none')).toBe('0'.repeat(32)); + }); + + test('property: never produces the all-zero id across many draws', () => { + fc.assert( + fc.property(fc.constant(null), () => { + expect(generateTraceId('w3c')).not.toBe('0'.repeat(32)); + }), + {numRuns: 500}, + ); + }); +}); + +describe('span activation and restoration (OBS-22)', () => { + function recordingSpan(traceId: string, spanId: string): Span { + return { + isRecording: true, + setAttribute(): Span { + return this; + }, + recordException(): Span { + return this; + }, + end(): void { + return; + }, + spanContext: () => ({traceId, spanId}), + }; + } + + test('rejects invalid span input', () => { + expect(() => activateSpan(null as unknown as Span)).toThrow(); + expect(() => activateSpan({} as unknown as Span)).toThrow(); + }); + + test('close() restores the previously-active span', () => { + const outer = recordingSpan('a'.repeat(32), 'b'.repeat(16)); + const inner = recordingSpan('c'.repeat(32), 'd'.repeat(16)); + + const outerScope = activateSpan(outer); + const innerScope = activateSpan(inner); + expect(getActiveSpan()).toBe(inner); + innerScope.close(); + expect(getActiveSpan()).toBe(outer); + outerScope.close(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('close() restores even when the guarded code throws', () => { + const span = recordingSpan('a'.repeat(32), 'b'.repeat(16)); + const scope = activateSpan(span); + expect(() => { + try { + throw new Error('boom'); + } finally { + scope.close(); + } + }).toThrow('boom'); + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('close() is idempotent', () => { + const scope = activateSpan(recordingSpan('a'.repeat(32), 'b'.repeat(16))); + scope.close(); + expect(() => { + scope.close(); + }).not.toThrow(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); +}); + +describe('span log correlation: standard behavior (OBS-23)', () => { + function recordingSpan(traceId: string, spanId: string): Span { + return { + isRecording: true, + setAttribute(): Span { + return this; + }, + recordException(): Span { + return this; + }, + end(): void { + return; + }, + spanContext: () => ({traceId, spanId}), + }; + } + + test('a recording span pushes trace.id/span.id and restores them on close (OBS-23)', () => { + const span = recordingSpan('e'.repeat(32), 'f'.repeat(16)); + + const scope = activateSpanForCorrelation(span); + expect(getDiagnosticContext(null)['trace.id']).toBe('e'.repeat(32)); + expect(getDiagnosticContext(null)['span.id']).toBe('f'.repeat(16)); + scope.close(); + + expect(getDiagnosticContext(null)['trace.id']).toBeUndefined(); + }); + + test('a non-recording span pushes nothing and delegates to plain activation (OBS-23)', () => { + const scope = activateSpanForCorrelation(NOOP_SPAN); + expect(getDiagnosticContext(null)['trace.id']).toBeUndefined(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + scope.close(); + expect(() => { + scope.close(); + }).not.toThrow(); + }); +}); + +describe('span log correlation: edge cases (OBS-23)', () => { + test('a recording span with missing or throwing spanContext handles gracefully', () => { + const spanWithoutCtx: Span = { + isRecording: true, + setAttribute(): Span { + return this; + }, + recordException(): Span { + return this; + }, + end(): void { + return; + }, + }; + const scope1 = activateSpanForCorrelation(spanWithoutCtx); + expect(getDiagnosticContext(null)['trace.id']).toBeUndefined(); + scope1.close(); + + const throwingSpan = { + isRecording: true, + setAttribute(): Span { + return this as unknown as Span; + }, + recordException(): Span { + return this as unknown as Span; + }, + end(): void { + return; + }, + spanContext: () => { + throw new Error('hostile'); + }, + } as unknown as Span; + const scope2 = activateSpanForCorrelation(throwingSpan); + expect(getDiagnosticContext(null)['trace.id']).toBeUndefined(); + scope2.close(); + }); +}); + +describe('createInstrumentationBundle', () => { + test('generates valid W3C ids and marks the bundle valid', () => { + const bundle = createInstrumentationBundle(); + expect(bundle.traceId).toMatch(/^[0-9a-f]{32}$/u); + expect(bundle.spanId).toMatch(/^[0-9a-f]{16}$/u); + expect(bundle.isValid).toBe(true); + expect((bundle.tracerFactory as () => typeof NOOP_TRACER)()).toBe( + NOOP_TRACER, + ); + }); + + test('a supplied tracerFactory is reachable through the bundle', () => { + const spans: string[] = []; + const tracer = { + startSpan: (name: string) => { + spans.push(name); + return NOOP_SPAN; + }, + }; + const bundle = createInstrumentationBundle(() => tracer); + // The bundle's tracerFactory field is `unknown` per 4a's frozen shape; a real consumer (Task 6) casts it. + (bundle.tracerFactory as () => typeof tracer)(); + expect(spans).toHaveLength(0); // factory itself doesn't start a span; startSpan is called by a consumer + }); +}); diff --git a/packages/core/src/observability/tracing.ts b/packages/core/src/observability/tracing.ts new file mode 100644 index 0000000..c98d692 --- /dev/null +++ b/packages/core/src/observability/tracing.ts @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/tracing.ts +import {invariant} from '../invariant.js'; +import { + createAsyncScopedStore, + pushDiagnosticFields, +} from './diagnostic-context.js'; +import {NOOP_SPAN, NOOP_TRACER} from './span.js'; +import type {Span, Tracer} from './span.js'; +import type {InstrumentationBundle} from '../context/instrumentation.js'; + +// The span/tracer shapes and the two inert singletons live in `span.js` so that +// `context/instrumentation.ts` can reach `NOOP_SPAN` for CTX-15's disabled-tracing default without +// closing an import cycle with this module's `InstrumentationBundle` type edge. They are re-exported +// verbatim here, which is this module's published surface and every existing import path. +export type {Span, SpanContext, Tracer} from './span.js'; +export {NOOP_SPAN, NOOP_TRACER} from './span.js'; + +function randomHex(byteLength: number): string { + const bytes = new Uint8Array(byteLength); + globalThis.crypto.getRandomValues(bytes); + const allZero = bytes.every(b => b === 0); + if (allZero) bytes[byteLength - 1] = 1; + return Array.from(bytes, b => b.toString(16).padStart(2, '0')).join(''); +} + +/** + * Generates a trace identifier for the requested flavor (OBS-26, OBS-27). + * + * @param flavor - 'w3c', 'datadog', or 'none'. + * @returns the formatted trace id. + * + * @internal + */ +export function generateTraceId(flavor: 'w3c' | 'datadog' | 'none'): string { + if (flavor === 'none') return '0'.repeat(32); + if (flavor === 'datadog') { + const bytes = new Uint8Array(8); + globalThis.crypto.getRandomValues(bytes); + let value = 0n; + for (const byte of bytes) value = (value << 8n) | BigInt(byte); + return String(value === 0n ? 1n : value); + } + return randomHex(16); +} + +/** + * Generates a W3C span identifier (16 lowercase hex characters) (OBS-26, OBS-27). + * + * @returns 16-hex span id. + * + * @internal + */ +export function generateSpanId(): string { + return randomHex(8); +} + +/** + * Handle representing an active span scope (OBS-22). + * + * @public + */ +export interface Scope { + close(): void; +} + +const spanStorage = createAsyncScopedStore<Span>(); + +/** + * Returns the currently active span, or {@link NOOP_SPAN} if none is active. + * + * @public + */ +export function getActiveSpan(): Span { + return spanStorage.get() ?? NOOP_SPAN; +} + +/** + * Activates `span` for the current async context (OBS-22). + * + * @param span - the span to activate. + * @returns a {@link Scope} handle whose `close()` restores the previous span. + * + * @public + */ +export function activateSpan(span: Span): Scope { + requireSpan(span, 'activateSpan'); + + const restore = spanStorage.enter(span); + return {close: restore}; +} + +function requireSpan(span: Span, caller: string): void { + invariant( + (span as unknown) !== null && (span as unknown) !== undefined, + `${caller}: span is required`, + ); + invariant( + typeof span.end === 'function', + `${caller}: span must implement end()`, + ); +} + +/** + * The callback form of {@link activateSpan}, for a scope that can be written as one function: `span` is + * active for the whole of `fn`, and whatever was active before is active again the moment `fn` returns. + * + * `activateSpan`'s handle cannot make that promise across an `await`. Its `close()` is an `enterWith` on + * whichever async resource happens to run it, so a scope opened before an `await` and closed after one + * leaves the span installed on the resource that opened it -- the caller's, when the opener is + * `Runtime.send`. This form is `AsyncLocalStorage.run`, which unwinds structurally instead. The handle + * stays because OBS-22 specifies one; this is what the runtime uses. + * + * @param span - the span to activate for the extent of `fn`. + * @param fn - the work to run with `span` active. Its result is passed through untouched. + * @returns whatever `fn` returned. + * + * @internal + */ +export function runWithActiveSpan<T>(span: Span, fn: () => T): T { + requireSpan(span, 'runWithActiveSpan'); + return spanStorage.run(span, fn); +} + +/** Extracts trace.id and span.id from OpenTelemetry-compatible spanContext() if present. */ +function readCorrelationIds( + span: Span, +): Readonly<Record<string, string>> | undefined { + try { + const context = span.spanContext?.(); + if ( + context === undefined || + typeof context.traceId !== 'string' || + typeof context.spanId !== 'string' + ) { + return undefined; + } + return {'trace.id': context.traceId, 'span.id': context.spanId}; + } catch { + return undefined; + } +} + +/** + * Activates `span` and correlates it with the diagnostic context if recording (OBS-23). + * + * @param span - the span to activate and correlate. + * @returns a {@link Scope} handle. + * + * @public + */ +export function activateSpanForCorrelation(span: Span): Scope { + const scope = activateSpan(span); + if (!span.isRecording) return scope; + + const correlation = readCorrelationIds(span); + if (correlation === undefined) return scope; + + const restore = pushDiagnosticFields(correlation); + let closed = false; + return { + close(): void { + if (closed) return; + closed = true; + restore(); + scope.close(); + }, + }; +} + +/** + * Builds a populated {@link InstrumentationBundle} with generated W3C IDs and tracer factory. + * + * @param tracerFactory - optional custom tracer factory. + * @returns a valid {@link InstrumentationBundle}. + * + * @public + */ +export function createInstrumentationBundle( + tracerFactory?: (operationName: string) => Tracer, +): InstrumentationBundle { + return { + traceId: generateTraceId('w3c'), + spanId: generateSpanId(), + traceFlags: 1, + traceState: '', + traceIdEncoding: 'w3c', + isValid: true, + isRemote: false, + activeSpan: NOOP_SPAN, + tracerFactory: tracerFactory ?? ((): Tracer => NOOP_TRACER), + }; +} diff --git a/packages/core/src/pagination/cancellation.test.ts b/packages/core/src/pagination/cancellation.test.ts new file mode 100644 index 0000000..d2e86d8 --- /dev/null +++ b/packages/core/src/pagination/cancellation.test.ts @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/cancellation.test.ts +// Exercises: PAGE-25 (the signal reaches every exchange and halts the walk), PAGE-26 (page-granular +// cancellation; a fetched-but-undelivered page is dropped AND closed), PAGE-31 (no per-page recursion), +// PAGE-33 (a response that arrives after the abort is closed and discarded). +import {expect, test} from 'bun:test'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {pageInfo} from './page.js'; +import {Paginator} from './paginator.js'; +import type {PaginationStrategy} from './strategy.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; + +const initialRequest = (): Request => + ({ + method: 'GET', + url: new URL('https://api.test/items'), + headers: {get: () => undefined}, + }) as unknown as Request; + +const transportOf = ( + pages: number, + onClose?: (index: number) => void, +): FakeTransport => + new FakeTransport( + Array.from({length: pages}, (_, i) => + countingResponse({ + body: `page-${String(i)}`, + onCancel: () => onClose?.(i), + }), + ), + ); + +const endless = (): PaginationStrategy<string> => ({ + parse: (response: Response, template: Request) => + Promise.resolve( + pageInfo(['x'], { + method: template.method, + url: new URL(`${template.url.pathname}?page=next`, template.url), + } as Request), + ), +}); + +test('the signal is threaded into every page exchange (PAGE-25)', async () => { + const controller = new AbortController(); + const transport = transportOf(5); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: endless(), + maxPages: 3, + signal: controller.signal, + }); + for await (const page of paginator.pages()) { + void page; + } + expect(transport.sentSignals).toHaveLength(3); + expect(transport.sentSignals.every(s => s === controller.signal)).toBe(true); +}); + +test('aborting mid-walk stops the walk at the next page boundary (PAGE-25, PAGE-26)', async () => { + const controller = new AbortController(); + const transport = transportOf(10); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: endless(), + signal: controller.signal, + }); + + let delivered = 0; + for await (const page of paginator.pages()) { + void page; + delivered += 1; + if (delivered === 2) controller.abort(); + } + + // Pre-dispatch abort check ensures Page 3 is never dispatched. + expect(delivered).toBe(2); + expect(transport.sendCount).toBe(2); +}); + +test('a page fetched while abort is in flight is closed and discarded, never yielded (PAGE-26, PAGE-33)', async () => { + const controller = new AbortController(); + const closed: number[] = []; + let sendCount = 0; + const transport = { + send: () => { + sendCount += 1; + if (sendCount === 2) { + controller.abort(); + } + return Promise.resolve( + countingResponse({ + body: `page-${String(sendCount)}`, + onCancel: () => closed.push(sendCount), + }), + ); + }, + close: () => Promise.resolve(), + } as unknown as FakeTransport; + + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: endless(), + signal: controller.signal, + }); + + const delivered = []; + for await (const page of paginator.pages()) { + delivered.push(page); + } + + expect(delivered).toHaveLength(1); + expect(sendCount).toBe(2); + // Page 1 closed at exit, Page 2 closed by post-fetch abort check (PAGE-33). + expect(closed.sort()).toEqual([1, 2]); +}); + +test('thousands of immediately-resolved pages complete without stack growth (PAGE-31)', async () => { + const PAGES = 5000; + const transport = transportOf(PAGES); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: endless(), + maxPages: PAGES, + }); + + let count = 0; + for await (const page of paginator.pages()) { + void page; + count += 1; + } + + // A `for await` loop is iterative by construction — the engine structurally cannot recurse per page, which + // is PAGE-31's own sanctioned escape from building a trampoline. + expect(count).toBe(PAGES); +}); diff --git a/packages/core/src/pagination/errors.test.ts b/packages/core/src/pagination/errors.test.ts new file mode 100644 index 0000000..c03affa --- /dev/null +++ b/packages/core/src/pagination/errors.test.ts @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/errors.test.ts +// Exercises: PAGE-9 (cap validated at construction), PAGE-14 (re-iteration fails loudly). +import {expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import {PaginationError} from './errors.js'; + +test('sits directly under DexpaceError (two-level tree)', () => { + expect(new PaginationError('x')).toBeInstanceOf(DexpaceError); +}); + +test('name identifies the leaf in a stack trace', () => { + expect(new PaginationError('x').name).toBe('PaginationError'); +}); + +test('chains a cause when given one', () => { + const backing = new Error('root'); + expect(new PaginationError('x', {cause: backing}).cause).toBe(backing); +}); diff --git a/packages/core/src/pagination/errors.ts b/packages/core/src/pagination/errors.ts new file mode 100644 index 0000000..b02f1ac --- /dev/null +++ b/packages/core/src/pagination/errors.ts @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * A misuse or precondition failure of the pagination engine: a non-positive page cap at construction + * (PAGE-9), or a second iterator on the single-use page-level view (PAGE-14). + * + * Not used for transport, parse, or close failures — those propagate as whatever the underlying layer raised, + * because PAGE-28 requires the *original* cause to surface rather than a pagination-flavored wrapper. + * + * @public + */ +export class PaginationError extends DexpaceError { + // bun's coverage tool never marks a bodiless subclass's implicit constructor as covered + // (undercounts function coverage); an explicit forwarding constructor is instrumented + // correctly and keeps the file above the 80% function-coverage floor without changing behavior. + // eslint-disable-next-line @typescript-eslint/no-useless-constructor -- see comment above + constructor(message: string, options?: ErrorOptions) { + super(message, options); + } +} diff --git a/packages/core/src/pagination/fetchers.test.ts b/packages/core/src/pagination/fetchers.test.ts new file mode 100644 index 0000000..28c5797 --- /dev/null +++ b/packages/core/src/pagination/fetchers.test.ts @@ -0,0 +1,333 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/fetchers.test.ts +// Exercises: PAGE-34 (first fetcher runs once; next keys off nextLink with token fallback; blank link or +// undefined page terminates; a fetcher builds a page it does not close), PAGE-35 (one shared mutable options +// instance threaded through every call). +import {expect, test} from 'bun:test'; +import {Page} from './page.js'; +import {PaginationError} from './errors.js'; +import {paginateWithFetchers, type PagingOptions} from './fetchers.js'; +import type {Response} from '../http/response.js'; + +function fakePage<T>(items: readonly T[], onClose: () => void): Page<T> { + let closed = false; + const response = { + status: {code: 200}, + headers: {get: () => undefined}, + request: {}, + close(): Promise<void> { + if (!closed) { + closed = true; + onClose(); + } + return Promise.resolve(); + }, + } as unknown as Response; + return new Page(response, items); +} + +test('the first fetcher runs exactly once and the next keys off nextLink (PAGE-34)', async () => { + let firstCalls = 0; + const nextLinks: string[] = []; + const iterable = paginateWithFetchers<string>({ + first: () => { + firstCalls += 1; + return Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: '/p2', + }); + }, + next: link => { + nextLinks.push(link); + return Promise.resolve({page: fakePage(['b'], () => undefined)}); + }, + }); + + const seen = []; + for await (const page of iterable) seen.push(page.items[0]); + + expect(firstCalls).toBe(1); + expect(nextLinks).toEqual(['/p2']); + expect(seen).toEqual(['a', 'b']); +}); + +test('the continuation token is used only when no next link is present — link wins (PAGE-34)', async () => { + const keys: string[] = []; + let nextCalls = 0; + const iterable = paginateWithFetchers<string>({ + // Page 1 offers BOTH a link and a token; the link must win. + first: () => + Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: '/p2', + continuationToken: 'tok-2', + }), + next: key => { + keys.push(key); + nextCalls += 1; + return nextCalls === 1 + ? Promise.resolve({ + page: fakePage(['b'], () => undefined), + // Page 2 offers ONLY a token; now the token must be used. + continuationToken: 'tok-3', + }) + : Promise.resolve(undefined); + }, + }); + + const seen = []; + for await (const page of iterable) seen.push(page.items[0]); + + expect(seen).toEqual(['a', 'b']); + expect(keys).toEqual(['/p2', 'tok-3']); +}); + +test('a blank, whitespace-only, or undefined nextLink ends the stream (PAGE-34)', async () => { + let calls = 0; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: ' ', + }), + next: () => { + calls += 1; + return Promise.resolve({page: fakePage(['b'], () => undefined)}); + }, + }); + + const seen = []; + for await (const page of iterable) seen.push(page.items[0]); + + expect(seen).toEqual(['a']); + expect(calls).toBe(0); +}); + +test('an undefined first-page result yields an empty stream (PAGE-34)', async () => { + const iterable = paginateWithFetchers<string>({ + first: () => Promise.resolve(undefined), + next: () => Promise.resolve(undefined), + }); + + const seen = []; + for await (const page of iterable) seen.push(page); + + expect(seen).toEqual([]); +}); + +test('an undefined page from the next fetcher ends the stream (PAGE-34)', async () => { + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: '/p2', + }), + next: () => Promise.resolve(undefined), + }); + const seen = []; + for await (const page of iterable) seen.push(page.items[0]); + expect(seen).toEqual(['a']); +}); + +test('options bag is passed to first and next as the identical mutable instance (PAGE-35)', async () => { + const received: (PagingOptions | undefined)[] = []; + const iterable = paginateWithFetchers<string>({ + first: options => { + received.push(options); + options.custom = 'stashed'; + return Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: '/p2', + }); + }, + next: (_link, options) => { + received.push(options); + return Promise.resolve({page: fakePage(['b'], () => undefined)}); + }, + }); + + for await (const page of iterable) { + void page; + } + + expect(received[0]).toBe(received[1]); + expect(received[1]?.custom).toBe('stashed'); +}); + +test('options.nextLink and options.continuationToken are populated before next() call (PAGE-34, PAGE-35)', async () => { + const optionsSeen: PagingOptions[] = []; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => undefined), + nextLink: '/p2', + continuationToken: 'tok-1', + }), + next: (_key, options) => { + optionsSeen.push({...options}); + return Promise.resolve(undefined); + }, + }); + + for await (const page of iterable) { + void page; + } + + expect(optionsSeen).toHaveLength(1); + expect(optionsSeen[0]?.nextLink).toBe('/p2'); + expect(optionsSeen[0]?.continuationToken).toBe('tok-1'); +}); + +test('pages are closed as the consumer advances and at exhaustion (PAGE-3, PAGE-12)', async () => { + const closed: string[] = []; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => closed.push('a')), + nextLink: '/p2', + }), + next: () => + Promise.resolve({page: fakePage(['b'], () => closed.push('b'))}), + }); + + for await (const page of iterable) { + void page; + } + + expect(closed).toEqual(['a', 'b']); +}); + +test('the cap bounds a fetcher pair that never terminates, fetching nothing extra (PAGE-9)', async () => { + const closed: string[] = []; + let calls = 0; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => closed.push('a')), + nextLink: '/loop', + }), + next: () => { + calls += 1; + const label = `x${String(calls)}`; + return Promise.resolve({ + page: fakePage([label], () => closed.push(label)), + nextLink: '/loop', + }); + }, + maxPages: 3, + }); + + let delivered = 0; + for await (const page of iterable) { + void page; + delivered += 1; + } + + expect(delivered).toBe(3); + // Three pages delivered means the fetcher ran twice, not three times: the third call would produce a fourth + // page the cap forbids delivering, and a page fetched but never delivered is a page nobody closes. + expect(calls).toBe(2); + // Every page that was fetched was also closed — no leak on the capped path. + expect(closed.sort()).toEqual(['a', 'x1', 'x2']); +}); + +test.each([0, -1, 1.5, NaN])( + 'non-positive integer maxPages throws PaginationError (PAGE-9)', + maxPages => { + expect(() => + paginateWithFetchers<string>({ + first: () => Promise.resolve(undefined), + next: () => Promise.resolve(undefined), + maxPages, + }), + ).toThrow(PaginationError); + }, +); + +test('the fetcher view is single-use at the iterator level, and does not re-run first() (PAGE-14, PAGE-34)', async () => { + let firstCalls = 0; + const iterable = paginateWithFetchers<string>({ + first: () => { + firstCalls += 1; + return Promise.resolve({page: fakePage(['a'], () => undefined)}); + }, + next: () => { + throw new Error('must not be called'); + }, + }); + + for await (const page of iterable) { + void page; + } + + let caughtView: unknown; + try { + for await (const page of iterable) { + void page; + } + } catch (e: unknown) { + caughtView = e; + } + expect(caughtView).toBeInstanceOf(PaginationError); + // PAGE-34 says the first-page fetcher runs exactly once. Without the guard a second loop would run it again. + expect(firstCalls).toBe(1); +}); + +test('a throwing next fetcher propagates its failure and closes previous page (PAGE-15, PAGE-28)', async () => { + const fetcherFailure = new Error('page 2 fetch blew up'); + let page1Closed = false; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => { + page1Closed = true; + }), + nextLink: '/p2', + }), + next: () => Promise.reject(fetcherFailure), + }); + + let caught: unknown; + try { + for await (const page of iterable) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(fetcherFailure); + expect(page1Closed).toBe(true); +}); + +test('a close failure while advancing stops the walk before next fetch (PAGE-12, PAGE-15, PAGE-27)', async () => { + const closePreviousFailure = new Error('previous page close failed'); + let nextCalls = 0; + const iterable = paginateWithFetchers<string>({ + first: () => + Promise.resolve({ + page: fakePage(['a'], () => { + throw closePreviousFailure; + }), + nextLink: '/p2', + }), + next: () => { + nextCalls += 1; + return Promise.resolve({ + page: fakePage(['b'], () => undefined), + }); + }, + }); + + let caught: unknown; + try { + for await (const page of iterable) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(closePreviousFailure); + expect(nextCalls).toBe(0); +}); diff --git a/packages/core/src/pagination/fetchers.ts b/packages/core/src/pagination/fetchers.ts new file mode 100644 index 0000000..f73b3eb --- /dev/null +++ b/packages/core/src/pagination/fetchers.ts @@ -0,0 +1,154 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/fetchers.ts +import {invariant} from '../invariant.js'; +import {PaginationError} from './errors.js'; +import type {Page} from './page.js'; + +/** + * A mutable bag threaded through **every** fetcher call in one walk (PAGE-35). + * + * The same instance is passed each time, so a custom retriever can stash cursor or auth state between pages. + * Cross-call mutation visibility is the *point*, not a hazard to defend against — it is documented here rather + * than designed away. Single-consumer; needs no synchronization. + * + * @public + */ +export interface PagingOptions { + /** The RFC 8288 link target for the next page (PAGE-34). */ + nextLink?: string | undefined; + /** The continuation/cursor token fallback for the next page (PAGE-34). */ + continuationToken?: string | undefined; + /** Custom per-walk state stashed by callers across page requests (PAGE-35). */ + [key: string]: unknown; +} + +/** + * What a fetcher returns: a page it built and does not close, plus how to reach the one after it. + * + * @public + */ +export interface FetcherPage<T> { + /** The constructed page, which owns its underlying response and will be closed by the engine (PAGE-34). */ + readonly page: Page<T>; + /** The RFC 8288 next link, taking priority over continuationToken (PAGE-34). */ + readonly nextLink?: string | undefined; + /** The continuation token fallback used when nextLink is absent (PAGE-34). */ + readonly continuationToken?: string | undefined; +} + +/** + * Initialization options for {@link paginateWithFetchers}. + * + * @public + */ +export interface FetcherPaginationInit<T> { + /** Called exactly once, at the start of the walk. Return `undefined` for an empty stream. */ + first: (options: PagingOptions) => Promise<FetcherPage<T> | undefined>; + /** + * Called with the previous page's next link, or — only when no link was present — its continuation token. + * Return `undefined` to end the stream. + */ + next: ( + key: string, + options: PagingOptions, + ) => Promise<FetcherPage<T> | undefined>; + /** Maximum pages delivered. Unbounded when omitted. */ + maxPages?: number | undefined; +} + +/** + * Drive pagination from caller-supplied per-page fetchers instead of a strategy (PAGE-34). + * + * **Ownership**: each fetcher builds a {@link (Page:class)} that owns its response and must **not** close it — + * ownership transfers to the page, and this engine closes it as the consumer advances and at exhaustion. A + * fetcher that throws *before* building the page still owns whatever response it opened; this engine never saw + * it and has no handle with which to close it. + * + * **Next link wins** over the continuation token. A blank or whitespace-only link with no fallback token ends + * the stream, as does an `undefined` return from either fetcher — an `undefined` first page yields an empty + * stream rather than an error. + * + * **Single-use** (PAGE-14). This is a page-level view, so its iterator may be obtained at most once; a second + * `for await` over the same returned value throws rather than silently restarting. Without the guard a second + * loop would re-run `first()`, breaking PAGE-34's "exactly once" and double-consuming the walk. Call + * `paginateWithFetchers()` again for a fresh walk — the same restart path `Paginator.pages()` offers. + * + * @public + */ +export function paginateWithFetchers<T>( + init: FetcherPaginationInit<T>, +): AsyncIterable<Page<T>> { + if ( + init.maxPages !== undefined && + (!Number.isInteger(init.maxPages) || init.maxPages <= 0) + ) { + throw new PaginationError( + `maxPages must be a positive integer; received ${String(init.maxPages)}`, + ); + } + invariant( + typeof init.first === 'function', + 'paginateWithFetchers requires a first-page fetcher', + ); + invariant( + typeof init.next === 'function', + 'paginateWithFetchers requires a next-page fetcher', + ); + + let iteratorTaken = false; + + return { + async *[Symbol.asyncIterator](): AsyncGenerator<Page<T>> { + if (iteratorTaken) { + throw new PaginationError( + 'the fetcher pagination view is single-use; its iterator may be obtained at most once', + ); + } + iteratorTaken = true; + yield* driveFetchers(init); + }, + }; +} + +async function* driveFetchers<T>( + init: FetcherPaginationInit<T>, +): AsyncGenerator<Page<T>> { + const options: PagingOptions = {}; + let held: Page<T> | undefined; + let delivered = 0; + + try { + let current: FetcherPage<T> | undefined = await init.first(options); + + while (current !== undefined) { + const page: Page<T> = current.page; + held = page; + delivered += 1; + yield page; + + // PAGE-9: stop *before* fetching the page that would exceed the cap. + if (init.maxPages !== undefined && delivered >= init.maxPages) return; + + const key: string | undefined = nextKey(current); + if (key === undefined) return; + + // PAGE-12: release previous page before calling the next fetcher to eliminate connection overlap. + held = undefined; + await page.close(); + + options.nextLink = current.nextLink; + options.continuationToken = current.continuationToken; + current = await init.next(key, options); + } + } finally { + if (held !== undefined) await held.close(); + } +} + +/** PAGE-34: the next link wins; the continuation token is a fallback only when no usable link is present. */ +function nextKey<T>(page: FetcherPage<T>): string | undefined { + const link = page.nextLink?.trim(); + if (link !== undefined && link.length > 0) return link; + const token = page.continuationToken?.trim(); + return token !== undefined && token.length > 0 ? token : undefined; +} diff --git a/packages/core/src/pagination/lifecycle.test.ts b/packages/core/src/pagination/lifecycle.test.ts new file mode 100644 index 0000000..9a5cb27 --- /dev/null +++ b/packages/core/src/pagination/lifecycle.test.ts @@ -0,0 +1,529 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/lifecycle.test.ts +// Exercises: PAGE-4 (a malformed parse result closes the response and names the invariant), PAGE-11 (close +// BEFORE yielding items — the assertion appendix B does not make), PAGE-12 (close-on-abandon), PAGE-13 (parse +// failure closes inline, close error suppressed), PAGE-14 (single-use page view), PAGE-15 (close errors +// surface), PAGE-27 (exactly once on every path), PAGE-32 (consumer throw keeps consumer error primary, +// discarding return-phase close error). +import {expect, test} from 'bun:test'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {IoError} from '../io/errors.js'; +import {PaginationError} from './errors.js'; +import {pageInfo, type PageInfo} from './page.js'; +import {Paginator} from './paginator.js'; +import type {PaginationStrategy} from './strategy.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import type {SuppressedErrorLike} from '../suppress.js'; + +const requestAt = (href: string): Request => + ({ + url: new URL(href), + newBuilder() { + let target = new URL(href); + return { + url(next: URL) { + target = next; + return this; + }, + build: () => requestAt(target.href), + }; + }, + }) as unknown as Request; + +const initialRequest = (): Request => + requestAt('https://api.test/items?page=1'); + +function twoPageStrategy(): PaginationStrategy<string> { + return { + parse(_response: Response, template: Request): Promise<PageInfo<string>> { + const page = Number(template.url.searchParams.get('page') ?? '1'); + const items = [ + `p${String(page)}a`, + `p${String(page)}b`, + `p${String(page)}c`, + ]; + if (page >= 2) return Promise.resolve(pageInfo(items)); + const next = template + .newBuilder() + .url(new URL(`https://api.test/items?page=${String(page + 1)}`)) + .build(); + return Promise.resolve(pageInfo(items, next)); + }, + }; +} + +function transportOf( + pages: number, + onClose?: (index: number) => void, +): FakeTransport { + return new FakeTransport( + Array.from({length: pages}, (_unused, index) => + countingResponse({ + status: 200, + headers: {'X-Page': String(index + 1)}, + body: '{}', + onCancel: () => onClose?.(index), + }), + ), + ); +} + +test('the item view closes a page BEFORE yielding any of its items (PAGE-11)', async () => { + const events: string[] = []; + const transport = transportOf(2, index => + events.push(`close:${String(index)}`), + ); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + for await (const item of paginator.items()) { + events.push(`item:${item}`); + } + + // The close for page 0 must precede every one of its items. Under the design doc's illustrative snippet + // (close in a `finally`, after `yield*`) this assertion fails while PAGE-11's own checklist test still passes. + expect(events.indexOf('close:0')).toBeLessThan(events.indexOf('item:p1a')); + expect(events.indexOf('close:1')).toBeLessThan(events.indexOf('item:p2a')); +}); + +test('taking one item and stopping closes that page and fetches no second page (PAGE-11)', async () => { + const closed: number[] = []; + const transport = transportOf(2, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + for await (const item of paginator.items()) { + void item; + break; + } + + expect(closed).toEqual([0]); + expect(transport.sendCount).toBe(1); +}); + +test('breaking out of the page view closes the held page (PAGE-12)', async () => { + const closed: number[] = []; + const transport = transportOf(2, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + for await (const page of paginator.pages()) { + void page; + break; + } + + expect(closed).toEqual([0]); +}); + +test('advancing the page view closes the previous page (PAGE-12)', async () => { + const closed: number[] = []; + const transport = transportOf(2, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + const seen = []; + for await (const page of paginator.pages()) seen.push(page); + + expect(seen).toHaveLength(2); + expect(closed).toEqual([0, 1]); +}); + +test('a second pages() call returns a fresh, independent view — the restart PAGE-14 names', async () => { + // PAGE-14's own recovery clause: "a caller restarts pagination by requesting a fresh view from the engine." + // Guarding pages() itself would block that path AND make the engine stateful, which PAGE-8 forbids. + const transport = transportOf(4); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + const first = []; + for await (const page of paginator.pages()) first.push(page.items[0]); + const second = []; + for await (const page of paginator.pages()) second.push(page.items[0]); + + expect(second).toEqual(first); + expect(transport.sendCount).toBe(4); +}); + +test('the page view is single-use at the ITERATOR level too (PAGE-14)', async () => { + // The guard that matters. `for await` calls Symbol.asyncIterator afresh every time, so a view guarded only at + // the pages() level would let a second loop over the *same* view silently restart the whole walk — the exact + // "silently restart" PAGE-14 forbids, and invisible to the test above. + const transport = transportOf(2); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + const view = paginator.pages(); + + for await (const page of view) { + void page; + } + const sendsAfterFirstPass = transport.sendCount; + + let caught: unknown; + try { + for await (const page of view) { + void page; + } + } catch (e: unknown) { + caught = e; + } + expect(caught).toBeInstanceOf(PaginationError); + expect(transport.sendCount).toBe(sendsAfterFirstPass); +}); + +test('a transport failure surfaces the original cause, unwrapped (PAGE-28)', async () => { + const transportFailure = new IoError('connection reset'); + const transport = { + send: () => Promise.reject(transportFailure), + } as unknown as FakeTransport; + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + // No pagination-flavored wrapper: PAGE-28 wants the cause the caller can actually act on. + expect(caught).toBe(transportFailure); + expect(caught).not.toBeInstanceOf(PaginationError); +}); + +test('advancing closes the held page before next send, and surfaces close error (PAGE-12, PAGE-15)', async () => { + const closeFailure = new IoError('close failed'); + let sends = 0; + const transport = { + send: () => { + sends += 1; + return Promise.resolve( + countingResponse({ + status: 200, + headers: {'X-Page': '1'}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ); + }, + } as unknown as FakeTransport; + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(closeFailure); + expect(sends).toBe(1); +}); + +test('a parse failure closes the response inline and propagates the parse error (PAGE-13)', async () => { + const boom = new Error('malformed page'); + const closed: number[] = []; + const transport = transportOf(1, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: { + parse(): Promise<never> { + return Promise.reject(boom); + }, + }, + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(boom); + expect(closed).toEqual([0]); +}); + +test('a close failure during a parse failure is suppressed, not masking (PAGE-13)', async () => { + const parseFailure = new Error('malformed page'); + const closeFailure = new IoError('close failed'); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: { + parse(): Promise<never> { + return Promise.reject(parseFailure); + }, + }, + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect((caught as SuppressedErrorLike).name).toBe('SuppressedError'); + expect((caught as SuppressedErrorLike).error).toBe(parseFailure); + expect((caught as SuppressedErrorLike).suppressed).toBe(closeFailure); +}); + +test('a close error while releasing a held page surfaces rather than being swallowed (PAGE-15)', async () => { + const closeFailure = new IoError('close failed'); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {'X-Page': '1'}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: { + parse: () => Promise.resolve(pageInfo(['only'])), + }, + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(closeFailure); +}); + +test('a close failure while advancing stops walk before dispatching next request (PAGE-12, PAGE-15, PAGE-27)', async () => { + const closePreviousFailure = new IoError('previous page close failed'); + const secondResponseClosed: number[] = []; + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {'X-Page': '1'}, + body: '{}', + onCancel: () => { + throw closePreviousFailure; + }, + }), + countingResponse({ + status: 200, + headers: {'X-Page': '2'}, + body: '{}', + onCancel: () => { + secondResponseClosed.push(2); + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBe(closePreviousFailure); + expect(transport.sendCount).toBe(1); + expect(secondResponseClosed).toEqual([]); +}); + +test.each([ + [ + 'full drain of the page view', + 2, + async (paginator: Paginator<string>) => { + for await (const page of paginator.pages()) { + void page; + } + }, + ], + [ + 'full drain of the item view', + 2, + async (paginator: Paginator<string>) => { + for await (const item of paginator.items()) { + void item; + } + }, + ], + [ + 'early break from the page view', + 1, + async (paginator: Paginator<string>) => { + for await (const page of paginator.pages()) { + void page; + break; + } + }, + ], + [ + 'consumer throws mid-iteration', + 1, + async (paginator: Paginator<string>) => { + try { + for await (const page of paginator.pages()) { + void page; + throw new Error('consumer blew up'); + } + } catch { + /* expected */ + } + }, + ], +])( + 'every response closes exactly once: %s (PAGE-27)', + async (_name, expectedFetches, drive) => { + const closeCounts = new Map<number, number>(); + const transport = transportOf(2, index => { + closeCounts.set(index, (closeCounts.get(index) ?? 0) + 1); + }); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: twoPageStrategy(), + }); + + await drive(paginator); + + expect(closeCounts.size).toBe(expectedFetches); + for (let i = 0; i < expectedFetches; i++) { + expect(closeCounts.get(i)).toBe(1); + } + }, +); + +// A strategy is caller code, and `parse`'s declared return type does not survive the seam: an +// `any`-typed JSON decode, a forgotten `return`, or a server field the caller trusted all land here +// as a shape the engine's own types say cannot exist (PAGE-4). The casts below ARE the test — they +// reproduce the four values that reach `#walk` in practice. +function malformedStrategy(result: unknown): PaginationStrategy<string> { + return {parse: () => Promise.resolve(result as PageInfo<string>)}; +} + +test.each([ + ['undefined', undefined, /never null or undefined/], + ['null', null, /never null or undefined/], + ['{items: null}', {items: null, nextRequest: undefined}, /PageInfo\.items/], + ['{items: undefined}', {nextRequest: undefined}, /PageInfo\.items/], +])( + 'a strategy that returns %s closes the response exactly once and names the invariant (PAGE-4, PAGE-27)', + async (_name, result, message) => { + const closed: number[] = []; + const transport = transportOf(1, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(result), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + // Before the fix `items: null` reached the spread in `Page`'s constructor and surfaced as a + // bare `TypeError` from array iteration, which names nothing a caller can act on. + expect((caught as Error).message).toMatch(message); + expect(closed).toEqual([0]); + }, +); + +test('a close failure while rejecting a malformed PageInfo is suppressed, not masking (PAGE-4, PAGE-13)', async () => { + const closeFailure = new IoError('close failed'); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(undefined), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect((suppressed.error as Error).message).toMatch( + /never null or undefined/, + ); + expect(suppressed.suppressed).toBe(closeFailure); +}); diff --git a/packages/core/src/pagination/link-header.test.ts b/packages/core/src/pagination/link-header.test.ts new file mode 100644 index 0000000..8a169bd --- /dev/null +++ b/packages/core/src/pagination/link-header.test.ts @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/link-header.test.ts +// Exercises: PAGE-18 (RFC 5988/8288 link-value parsing — commas inside <> or quotes do not split, quoted-pair +// escapes, quoted/unquoted rel, multi-token rel, case-insensitive `next`), PAGE-20 (multiple header instances). +import {expect, test} from 'bun:test'; +import {findNextLink, parseLinkHeader} from './link-header.js'; + +test('a simple rel=next is found', () => { + expect(findNextLink(['</p?page=2>; rel="next"'])).toBe('/p?page=2'); +}); + +test('an unquoted rel works (PAGE-18)', () => { + expect(findNextLink(['</p?page=2>; rel=next'])).toBe('/p?page=2'); +}); + +test('rel matching is case-insensitive (PAGE-18)', () => { + expect(findNextLink(['</p?page=2>; rel="NEXT"'])).toBe('/p?page=2'); +}); + +test('a multi-token rel containing next matches (PAGE-18)', () => { + expect(findNextLink(['</p?page=2>; rel="prev next last"'])).toBe('/p?page=2'); +}); + +test('a tab-separated multi-token rel matches (PAGE-18)', () => { + expect(findNextLink(['</p?page=2>; rel="prev\tnext"'])).toBe('/p?page=2'); +}); + +test('the FIRST link-value whose rel contains next wins (PAGE-18)', () => { + expect(findNextLink(['</a>; rel="next", </b>; rel="next"'])).toBe('/a'); +}); + +test('rel=prev and rel=last decoys are skipped (PAGE-18)', () => { + expect( + findNextLink(['</a>; rel="prev", </b>; rel="last", </c>; rel="next"']), + ).toBe('/c'); +}); + +test('a comma inside the angle-bracketed URL does not split link-values (PAGE-18)', () => { + expect(findNextLink(['</p?ids=1,2,3>; rel="next"'])).toBe('/p?ids=1,2,3'); +}); + +test('a comma inside a quoted parameter value does not split link-values (PAGE-18)', () => { + const parsed = parseLinkHeader( + '</a>; title="one, two"; rel="next", </b>; rel="prev"', + ); + expect(parsed).toHaveLength(2); + expect(parsed[0]?.target).toBe('/a'); +}); + +test('a quoted-pair escape is honored (PAGE-18)', () => { + const parsed = parseLinkHeader('</a>; title="say \\"hi\\", ok"; rel="next"'); + expect(parsed).toHaveLength(1); + expect(parsed[0]?.rel).toContain('next'); +}); + +test('no Link header means end of stream (PAGE-18)', () => { + expect(findNextLink([])).toBeUndefined(); +}); + +test('a Link header with no rel=next means end of stream (PAGE-18)', () => { + expect(findNextLink(['</a>; rel="prev"'])).toBeUndefined(); +}); + +test('multiple separate Link header instances are normalized by concatenation (PAGE-20)', () => { + expect(findNextLink(['</a>; rel="last"', '</b>; rel="next"'])).toBe('/b'); +}); + +test('an empty header set maps to no next link (PAGE-20)', () => { + expect(findNextLink([''])).toBeUndefined(); +}); + +test('surrounding whitespace is tolerated', () => { + expect( + findNextLink([ + ' < /p?page=2 > ; rel = "next" '.replace(/ (?=[/>])|(?<=<) /g, ''), + ]), + ).toBe('/p?page=2'); +}); diff --git a/packages/core/src/pagination/link-header.ts b/packages/core/src/pagination/link-header.ts new file mode 100644 index 0000000..52950f4 --- /dev/null +++ b/packages/core/src/pagination/link-header.ts @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/link-header.ts + +export interface LinkValue { + /** The raw target inside the angle brackets, unresolved. */ + readonly target: string; + /** The `rel` tokens, lowercased and split on whitespace. Empty when the link-value carried no `rel`. */ + readonly rel: readonly string[]; +} + +/** + * Parse an RFC 5988/8288 `Link` header into its link-values (PAGE-18). + * + * A regular expression is the wrong tool here and a hand-rolled scanner is the right one, because the + * separator rules are context-sensitive in two directions at once: a comma splits link-values **only** outside + * both angle brackets and quoted strings, and a semicolon splits parameters under the same condition. A quoted + * string additionally supports quoted-pair escapes (`\"`), so quote tracking cannot be a simple toggle. + * + * @internal + */ +export function parseLinkHeader(value: string): readonly LinkValue[] { + const out: LinkValue[] = []; + for (const raw of splitTopLevel(value, ',')) { + const parsed = parseOne(raw); + if (parsed !== undefined) out.push(parsed); + } + return out; +} + +/** + * The first target whose `rel` contains the token `next`, case-insensitively (PAGE-18). + * + * Multiple `Link` header instances are normalized by concatenation before parsing (PAGE-20), which is exactly + * what the RFC's own list semantics allow. An empty header set maps to no next link. + * + * @internal + */ +export function findNextLink( + headerValues: readonly string[], +): string | undefined { + const combined = headerValues.filter(v => v.trim().length > 0).join(', '); + if (combined.length === 0) return undefined; + for (const link of parseLinkHeader(combined)) { + if (link.rel.includes('next')) return link.target; + } + return undefined; +} + +function parseOne(raw: string): LinkValue | undefined { + const trimmed = raw.trim(); + const open = trimmed.indexOf('<'); + const close = trimmed.indexOf('>', open + 1); + if (open === -1 || close === -1) return undefined; + + const target = trimmed.slice(open + 1, close).trim(); + const rel: string[] = []; + + for (const parameter of splitTopLevel(trimmed.slice(close + 1), ';')) { + const eq = parameter.indexOf('='); + if (eq === -1) continue; + if (parameter.slice(0, eq).trim().toLowerCase() !== 'rel') continue; + // `rel` may be quoted or unquoted, and a quoted value may list several whitespace-separated types. + rel.push( + ...unquote(parameter.slice(eq + 1).trim()) + .toLowerCase() + .split(/[\s]+/) + .filter(t => t.length > 0), + ); + } + + return {target, rel: Object.freeze(rel)}; +} + +/** Split on `separator` only at depth zero — outside `<...>` and outside a quoted string. */ +function splitTopLevel(input: string, separator: string): string[] { + const parts: string[] = []; + let current = ''; + let inAngle = false; + let inQuotes = false; + let escaped = false; + + for (const char of input) { + if (escaped) { + current += char; + escaped = false; + continue; + } + if (inQuotes && char === '\\') { + current += char; + escaped = true; + continue; + } + if (char === '"') { + inQuotes = !inQuotes; + current += char; + continue; + } + if (!inQuotes && char === '<') inAngle = true; + else if (!inQuotes && char === '>') inAngle = false; + + if (char === separator && !inAngle && !inQuotes) { + parts.push(current); + current = ''; + continue; + } + current += char; + } + parts.push(current); + return parts.filter(part => part.trim().length > 0); +} + +/** Strip surrounding double quotes and unescape quoted pairs. */ +function unquote(value: string): string { + if (!value.startsWith('"') || !value.endsWith('"') || value.length < 2) + return value; + return value.slice(1, -1).replace(/\\(.)/g, '$1'); +} diff --git a/packages/core/src/pagination/page.test.ts b/packages/core/src/pagination/page.test.ts new file mode 100644 index 0000000..3c4901a --- /dev/null +++ b/packages/core/src/pagination/page.test.ts @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/page.test.ts +// Exercises: PAGE-2 (items and metadata survive close; items never null, at construction too), PAGE-3 (one +// owned response, never null, closed exactly once), PAGE-4 (PageInfo shape, undefined next-request is the end +// signal). +import {expect, test} from 'bun:test'; +import {Page, pageInfo} from './page.js'; + +function fakeResponse(): { + response: Parameters<typeof makePage>[0]; + closes: () => number; +} { + let closeCount = 0; + let closing: Promise<void> | undefined; + const response = { + status: {code: 200}, + headers: { + get: (n: string) => (n.toLowerCase() === 'x-total' ? '42' : undefined), + }, + request: {method: 'GET'}, + async close(): Promise<void> { + closing ??= Promise.resolve().then(() => { + closeCount += 1; + }); + return closing; + }, + } as unknown as Parameters<typeof makePage>[0]; + return {response, closes: () => closeCount}; +} + +const makePage = <T>( + response: ConstructorParameters<typeof Page<T>>[0], + items: readonly T[], +): Page<T> => new Page(response, items); + +test('items and derived metadata remain readable after close (PAGE-2)', async () => { + const {response} = fakeResponse(); + const page = makePage(response, [1, 2, 3]); + await page.close(); + expect(page.items).toEqual([1, 2, 3]); + expect(page.status.code).toBe(200); + expect(page.headers.get('X-Total')).toBe('42'); + expect(page.request).toBeDefined(); +}); + +test('items are never null and are frozen (PAGE-2)', () => { + const {response} = fakeResponse(); + const page = makePage(response, []); + expect(page.items).toEqual([]); + expect(Object.isFrozen(page.items)).toBe(true); +}); + +test('the items list is defensively copied from the caller (PAGE-2)', () => { + const {response} = fakeResponse(); + const supplied = [1, 2]; + const page = makePage(response, supplied); + supplied.push(3); + expect(page.items).toEqual([1, 2]); +}); + +test('close releases the owned response exactly once, and is idempotent (PAGE-3)', async () => { + const {response, closes} = fakeResponse(); + const page = makePage(response, [1]); + await page.close(); + await page.close(); + await page.close(); + expect(closes()).toBe(1); +}); + +test('pageInfo with no next request signals end of stream (PAGE-4)', () => { + expect(pageInfo([1, 2]).nextRequest).toBeUndefined(); +}); + +test('pageInfo carries items plus a next request, both frozen (PAGE-4)', () => { + const next = {method: 'GET'} as never; + const info = pageInfo([1], next); + expect(info.items).toEqual([1]); + expect(info.nextRequest).toBe(next); + expect(Object.isFrozen(info)).toBe(true); +}); + +test('an empty items list with a next request is a valid non-terminal page (PAGE-4)', () => { + const next = {method: 'GET'} as never; + const info = pageInfo([], next); + expect(info.items).toEqual([]); + expect(info.nextRequest).toBe(next); +}); + +test('the disposal member releases the page exactly once where the runtime has it (PAGE-3, PAGE-12)', async () => { + const {response, closes} = fakeResponse(); + const page = makePage(response, [1]); + // Read through a cast rather than `Symbol.asyncDispose` directly: on the pinned floor (Node 20.3, + // which predates the symbol's 20.4 arrival) it is `undefined`, and a bare index would silently read + // the string key `"undefined"` instead. Same shape as `sse/stream.test.ts`. + const asyncDispose = (Symbol as {asyncDispose?: symbol}).asyncDispose; + + if (typeof asyncDispose !== 'symbol') { + // The guarded install is correctly a no-op here; close() is the whole teardown surface. + await page.close(); + expect(closes()).toBe(1); + return; + } + + const dispose = ( + page as unknown as Record<symbol, (() => Promise<void>) | undefined> + )[asyncDispose]; + expect(dispose).toBeDefined(); + expect(page.items).toEqual([1]); + expect(closes()).toBe(0); + + await dispose?.call(page); + expect(closes()).toBe(1); + + // Dispose delegates to close, so it inherits Response.close()'s idempotence rather than adding a second guard. + await page.close(); + expect(closes()).toBe(1); +}); + +test('no "undefined" prototype key survives the guarded install (PAGE-12)', () => { + // The regression this pins: an unguarded `async [Symbol.asyncDispose]()` class member binds to the + // string key "undefined" on the >=20.3 floor, leaving junk on the prototype and no working disposal. + // `http/response.test.ts` carries the same assertion for `Response`. + const {response} = fakeResponse(); + const page = makePage(response, [1]); + expect(Object.getOwnPropertyNames(Object.getPrototypeOf(page))).not.toContain( + 'undefined', + ); +}); + +// `Page` is `@public`, so these guards are reachable from consumer code, not only from the walk — +// and until audit #67 / #79 their messages said "never null" while the check tested `!== undefined`. +// A `null` therefore reached the item copy and surfaced as a bare `TypeError` from spread. +test.each([ + ['null items', null, /items must never be null/], + ['undefined items', undefined, /items must never be null/], +])('a Page rejects %s at construction (PAGE-2)', (_name, items, message) => { + const {response} = fakeResponse(); + expect(() => + makePage(response, items as unknown as readonly number[]), + ).toThrow(message); +}); + +test('a Page rejects a null response at construction (PAGE-3)', () => { + expect(() => + makePage( + null as unknown as ConstructorParameters<typeof Page<number>>[0], + [1], + ), + ).toThrow(/must own a response/); +}); diff --git a/packages/core/src/pagination/page.ts b/packages/core/src/pagination/page.ts new file mode 100644 index 0000000..0391257 --- /dev/null +++ b/packages/core/src/pagination/page.ts @@ -0,0 +1,124 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/page.ts +import type {Headers} from '../http/headers.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import type {Status} from '../http/status.js'; +import {invariant} from '../invariant.js'; + +/** + * A pagination strategy's parse output: the items on this page plus the request that fetches the next one + * (PAGE-4). + * + * `nextRequest === undefined` is the **single, exclusive** end-of-stream signal the engine recognizes. A + * strategy must never signal termination by throwing or through a side channel, and an empty `items` list + * paired with a defined `nextRequest` is a perfectly valid non-terminal page. + * + * @public + */ +export interface PageInfo<T> { + /** The materialized items on this page (PAGE-2). */ + readonly items: readonly T[]; + /** The request that fetches the next page, or `undefined` to signal end of stream (PAGE-4). */ + readonly nextRequest: Request | undefined; +} + +/** + * Construct a frozen {@link PageInfo}. Omit `nextRequest` to signal end of stream. + * + * @public + */ +export function pageInfo<T>( + items: readonly T[], + nextRequest?: Request, +): PageInfo<T> { + return Object.freeze({items: Object.freeze([...items]), nextRequest}); +} + +/** + * One page of results, owning exactly one underlying response (PAGE-2, PAGE-3). + * + * State is split by lifetime, which is the whole point of the type: the materialized item list and the derived + * status, headers, and originating request are captured at construction and **remain readable after close** — + * only the raw body and its connection become invalid. Reading `page.status` after `await page.close()` is + * supported, not a bug. + * + * Whoever pulls a page owns closing it. A component that hands a caller a live page — a first-page fetcher, for + * instance — must **not** close the response itself; ownership transfers to the page. + * + * A class rather than a frozen object because it owns a resource with a lifecycle and an idempotent close, + * which is `styleguide/typescript/06` §6.3's test for a class. + * + * @public + */ +export class Page<T> { + /** Materialized, frozen items that remain readable after close (PAGE-2). */ + readonly items: readonly T[]; + /** The HTTP response status code and reason phrase (PAGE-1). */ + readonly status: Status; + /** The HTTP response headers (PAGE-1). */ + readonly headers: Headers; + /** The executed request that produced this page (PAGE-1). */ + readonly request: Request; + // `#private` rather than the styleguide's default `private` (styleguide 6.6): `Page` is a *published* type + // holding a live connection, and `private` is compile-time-only — a consumer could reach the response through + // bracket access and close or re-read it behind the engine's back, breaking PAGE-3's single-owner rule and + // PAGE-27's close-exactly-once. Runtime unreachability is the requirement here, not just encapsulation. + readonly #response: Response; + + constructor(response: Response, items: readonly T[]) { + // `!== null` as well as `!== undefined`: both messages have always said "null", and testing only + // for `undefined` let a `null` through to the item copy below, where it surfaced as a bare + // `TypeError` from spread — outside the error tree and naming neither field (audit #67 / #79). + invariant( + (response as unknown) !== undefined && (response as unknown) !== null, + 'a Page must own a response (PAGE-3)', + ); + invariant( + (items as unknown) !== undefined && (items as unknown) !== null, + 'a Page’s items must never be null (PAGE-2)', + ); + + this.#response = response; + // Captured now, so they outlive the response (PAGE-2). Copied so a caller's later mutation cannot reach in. + this.items = Object.freeze([...items]); + this.status = response.status; + this.headers = response.headers; + this.request = response.request; + } + + /** + * Release the underlying response's body and connection (PAGE-3). + * + * Idempotent, by delegation: Phase 3b's `Response.close()` is already close-once, so this adds no second + * guard that could disagree with it. + */ + async close(): Promise<void> { + await this.#response.close(); + } +} + +// PAGE-12's scoped teardown, installed at run time only when the symbol exists — the same guarded +// shape `SseStream` uses. `Response` ships no disposal member at all (HTTP-38), and +// `http/response.test.ts` pins the absence of the junk key this guard exists to prevent. +// +// Because the install is conditional, this class deliberately does NOT declare `implements +// AsyncDisposable`: `await using page` therefore does not type-check on the declared floor, where the +// method is genuinely absent. `close()` is the supported teardown path — see `Paginator.pages()`, +// which tells consumers which scoped constructs actually give PAGE-12's guarantee. +// +// DO NOT restore this as a plain `async [Symbol.asyncDispose]()` class member. Node 20.3 is this +// package's declared floor (`engines.node`, checked by verify:runtime-floor) and predates the symbol, +// which arrived in 20.4. On the floor the computed key evaluates to `undefined` and binds the method +// to the string key `"undefined"` — a junk prototype entry, and no working disposal. Declaring it on +// the class would also emit it into the `.d.ts` unconditionally, promising consumers on the floor a +// method that is not there. +if (typeof Symbol.asyncDispose === 'symbol') { + Object.defineProperty(Page.prototype, Symbol.asyncDispose, { + value: function asyncDispose<T>(this: Page<T>): Promise<void> { + return this.close(); + }, + writable: true, + configurable: true, + }); +} diff --git a/packages/core/src/pagination/paginator.test.ts b/packages/core/src/pagination/paginator.test.ts new file mode 100644 index 0000000..7a29831 --- /dev/null +++ b/packages/core/src/pagination/paginator.test.ts @@ -0,0 +1,226 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/paginator.test.ts +// Exercises: PAGE-1 (both views over one walk, server order across boundaries), PAGE-6 (page-lazy, zero +// exchanges before the first probe), PAGE-7 (forward-only, idempotent end probes), PAGE-8 (independent +// iterations), PAGE-9/PAGE-10 (cap), PAGE-36 (options on every page). +import {expect, test} from 'bun:test'; +import {RequestOptions} from '../http/request-options.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {pageInfo, type PageInfo} from './page.js'; +import {Paginator} from './paginator.js'; +import {PaginationError} from './errors.js'; +import type {PaginationStrategy} from './strategy.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; + +/** Three pages of two items each, then end. Reads a page number off a header the FakeTransport stamps. */ +function threePageStrategy(): PaginationStrategy<string> { + return { + parse(_response: Response, template: Request): Promise<PageInfo<string>> { + const page = Number(template.url.searchParams.get('page') ?? '1'); + const items = [`p${String(page)}i1`, `p${String(page)}i2`]; + if (page >= 3) return Promise.resolve(pageInfo(items)); + const next = template + .newBuilder() + .url(new URL(`https://api.test/items?page=${String(page + 1)}`)) + .build(); + return Promise.resolve(pageInfo(items, next)); + }, + }; +} + +/** A server that never advances: every page reports another page after it. */ +function neverEndingStrategy(): PaginationStrategy<string> { + return { + parse(_response: Response, template: Request): Promise<PageInfo<string>> { + return Promise.resolve(pageInfo(['x'], template)); + }, + }; +} + +function transportOf(pages: number): FakeTransport { + return new FakeTransport( + Array.from({length: pages}, (_unused, index) => + countingResponse({ + status: 200, + headers: {'X-Page': String(index + 1)}, + body: '{}', + }), + ), + ); +} + +/** + * A `Request` stand-in carrying the two members the engine and the strategies actually touch: `url`, and a + * `newBuilder()` chain for `PAGE-23`'s swap-only-the-URL rewrite. + * + * `newBuilder()` is not optional here — `threePageStrategy` below calls it on every non-terminal page, so a bare + * `{url}` cast would fail on the first parse with `template.newBuilder is not a function`, in every test in this + * file that walks more than one page. + */ +const requestAt = (href: string): Request => + ({ + url: new URL(href), + newBuilder() { + let target = new URL(href); + return { + url(next: URL) { + target = next; + return this; + }, + build: () => requestAt(target.href), + }; + }, + }) as unknown as Request; + +const initialRequest = (): Request => + requestAt('https://api.test/items?page=1'); + +test('the item view flattens all pages in server order (PAGE-1)', async () => { + const paginator = new Paginator({ + transport: transportOf(3), + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + const seen: string[] = []; + for await (const item of paginator.items()) seen.push(item); + expect(seen).toEqual(['p1i1', 'p1i2', 'p2i1', 'p2i2', 'p3i1', 'p3i2']); +}); + +test('the page view yields exactly three pages with their own status and headers (PAGE-1)', async () => { + const paginator = new Paginator({ + transport: transportOf(3), + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + const pages = []; + for await (const page of paginator.pages()) pages.push(page); + expect(pages).toHaveLength(3); + expect(pages.map(p => p.headers.get('X-Page'))).toEqual(['1', '2', '3']); + expect(pages[0]?.status.code).toBe(200); +}); + +test('constructing the paginator and obtaining the iterator trigger zero exchanges (PAGE-6)', async () => { + const transport = transportOf(3); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + const iterable = paginator.items(); + const iterator = iterable[Symbol.asyncIterator](); + expect(transport.sendCount).toBe(0); + + await iterator.next(); + expect(transport.sendCount).toBe(1); +}); + +test('exactly one exchange occurs per page consumed (PAGE-6)', async () => { + const transport = transportOf(3); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + for await (const page of paginator.pages()) { + void page; + } + expect(transport.sendCount).toBe(3); +}); + +test('no exchange happens past the terminal page, and end probes are idempotent (PAGE-7)', async () => { + const transport = transportOf(3); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + const iterator = paginator.items()[Symbol.asyncIterator](); + while (!(await iterator.next()).done) { + /* drain */ + } + expect((await iterator.next()).done).toBe(true); + expect((await iterator.next()).done).toBe(true); + expect(transport.sendCount).toBe(3); +}); + +test('two independent iterations each drive a full fetch sequence with equal results (PAGE-8)', async () => { + const transport = transportOf(6); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: threePageStrategy(), + }); + const first: string[] = []; + for await (const item of paginator.items()) first.push(item); + const second: string[] = []; + for await (const item of paginator.items()) second.push(item); + expect(second).toEqual(first); + expect(transport.sendCount).toBe(6); +}); + +test('the cap stops a non-advancing server at exactly N exchanges (PAGE-9)', async () => { + const transport = transportOf(10); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: neverEndingStrategy(), + maxPages: 4, + }); + for await (const page of paginator.pages()) { + void page; + } + expect(transport.sendCount).toBe(4); +}); + +test.each([0, -1, 1.5, Number.NaN])( + 'a cap of %p is rejected at construction, not lazily (PAGE-9)', + maxPages => { + expect( + () => + new Paginator({ + transport: transportOf(1), + initialRequest: initialRequest(), + strategy: threePageStrategy(), + maxPages: maxPages, + }), + ).toThrow(PaginationError); + }, +); + +test('the default cap is effectively unbounded (PAGE-10)', async () => { + const transport = transportOf(500); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: neverEndingStrategy(), + }); + let count = 0; + for await (const page of paginator.pages()) { + void page; + count += 1; + if (count === 400) break; + } + expect(count).toBe(400); +}); + +test('per-call options reach every page exchange, not just the first (PAGE-36)', async () => { + const transport = transportOf(3); + // Deliberately NOT `RequestOptions.EMPTY`. The failure PAGE-36 guards is an engine that honours the caller's + // options on page 1 and falls back to the default on pages 2..N — and against `EMPTY` that bug is invisible, + // because the substituted default IS `EMPTY`. A distinctive instance makes the identity assertion bite. + // (Use whichever HTTP-3 `newBuilder()` setter Phase 1 actually shipped; the only thing that matters here is + // that `options !== RequestOptions.EMPTY`.) + const options = RequestOptions.EMPTY.newBuilder().timeoutMs(1_234).build(); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: threePageStrategy(), + options, + }); + for await (const page of paginator.pages()) { + void page; + } + expect(transport.sentOptions).toHaveLength(3); + expect(transport.sentOptions.every(o => o === options)).toBe(true); +}); diff --git a/packages/core/src/pagination/paginator.ts b/packages/core/src/pagination/paginator.ts new file mode 100644 index 0000000..2a41f91 --- /dev/null +++ b/packages/core/src/pagination/paginator.ts @@ -0,0 +1,300 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/paginator.ts +import type {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import type {Transport} from '../seams/transport.js'; +import {invariant} from '../invariant.js'; +import {PaginationError} from './errors.js'; +import {Page, type PageInfo} from './page.js'; +import type {PaginationStrategy} from './strategy.js'; +import {suppress} from '../suppress.js'; + +/** + * Initialization options for {@link Paginator}. + * + * @public + */ +export interface PaginatorInit<T> { + /** + * Any `Transport` — a raw one, a `FakeTransport`, or 4c's `Runtime` (which implements `Transport`, so a + * full resilience pipeline drops in unchanged). The engine is transport-agnostic by §12's own mandate. + */ + readonly transport: Transport; + /** The request template that fetches the initial page (page 1) (PAGE-1). */ + readonly initialRequest: Request; + /** The pagination strategy that parses each response into items and the next request (PAGE-5). */ + readonly strategy: PaginationStrategy<T>; + /** Maximum exchanges. Counts pages, not items. Unbounded when omitted (PAGE-10). */ + readonly maxPages?: number | undefined; + /** Applied to **every** page exchange, not just the first (PAGE-36). */ + readonly options?: RequestOptions | undefined; + /** Optional abort signal applied to every page exchange (PAGE-25). */ + readonly signal?: AbortSignal | undefined; +} + +/** + * Turns a paginated endpoint into a lazy stream of items or of whole pages (PAGE-1). + * + * Holds only frozen configuration and is safe to share; each call to {@link Paginator.items} or {@link Paginator.pages} builds a + * fresh walk with its own counter and cursor, so two iterations drive two full fetch sequences (PAGE-8). That + * is a property of generators, not bookkeeping performed here. + * + * **Laziness is free** (PAGE-6): a generator body does not run until its first `.next()`, so constructing this + * object, calling `items()`, and taking its iterator all trigger zero exchanges. + * + * **Cancellation race** (PAGE-33), inherent and worth knowing: if `signal` aborts *before* the transport + * delivers a response, that response never reaches this engine and releasing it is the transport's + * responsibility — cancelling a walk cannot reach into a response it was never handed. Conversely, a page + * request already dispatched may still complete after the abort; when it does, this engine closes and discards + * that response rather than yielding it. + * + * @public + */ +export class Paginator<T> { + // `#private` rather than `private` (styleguide 6.6 defaults to `private`): this is a published class, so the + // config bag must be unreachable via bracket access from consumer code, not merely compile-time-hidden. + // It is the class's ONLY field — PAGE-8 requires the engine to hold immutable configuration and nothing else. + readonly #init: PaginatorInit<T>; + + constructor(init: PaginatorInit<T>) { + // PAGE-9: fail fast at construction, not lazily on the first fetch, so a misconfiguration surfaces at the + // call site that caused it. + if ( + init.maxPages !== undefined && + (!Number.isInteger(init.maxPages) || init.maxPages <= 0) + ) { + throw new PaginationError( + `maxPages must be a positive integer; received ${String(init.maxPages)}`, + ); + } + this.#init = Object.freeze({...init}); + + invariant( + (this.#init.transport as unknown) !== undefined, + 'Paginator requires a transport', + ); + invariant( + (this.#init.initialRequest as unknown) !== undefined, + 'Paginator requires an initialRequest', + ); + invariant( + (this.#init.strategy as unknown) !== undefined, + 'Paginator requires a strategy', + ); + } + + /** + * Every item across every page, flattened in server order (PAGE-1). + * + * **Each page is closed before any of its items are yielded** (PAGE-11), after the items are copied. The + * items survive close (PAGE-2), so this costs nothing — and it means abandoning iteration mid-page can never + * strand a response, regardless of how long the consumer takes. + * + * Note this is deliberately *not* the ordering `sdk-design-nodejs/07` §7.1's illustrative snippet shows. That + * snippet closes in a `finally` after yielding, which holds the response open for the whole item walk; it + * passes PAGE-11's stated conformance test anyway, which is exactly why the ordering is called out here. + * + * Re-iterable, unlike {@link Paginator.pages}: PAGE-14 scopes single-use to the page-level view, and PAGE-8 requires + * independent iterations to work. + */ + items(): AsyncIterable<T> { + const walk = this.#walk.bind(this); + return { + async *[Symbol.asyncIterator](): AsyncGenerator<T> { + for await (const page of walk()) { + const items = page.items; + await page.close(); + yield* items; + } + }, + }; + } + + /** + * Whole pages, each exposing per-page status, headers, and originating request (PAGE-1). + * + * Auto-closing (PAGE-12): the previous page is closed as the consumer advances, and the currently held page + * is closed at exhaustion or on abandonment via the generator's `finally` — which the runtime drives + * automatically when a `for await` loop exits early through `break`, `return`, or a throw. + * + * **Consume this inside a scoped construct** (PAGE-12, MUST). A `for await` loop is one — it drives + * `.return()` on every exit path, including `break` and `throw`, so the held page is always released. Driving + * the iterator by hand is the case to be careful with: if you call `[Symbol.asyncIterator]()` yourself and + * then abandon it without calling `.return()`, the generator never resumes, its `finally` never runs, and the + * page it is holding stays open until the process exits. Two constructs give you the guarantee: stay inside a + * `for await`, or, when you drive the iterator yourself, call `.return()` on it from a `finally`. + * + * `await using` is deliberately **not** a third. {@link (Page:class)} installs `[Symbol.asyncDispose]` at run + * time only where the runtime has it, so it does not declare `AsyncDisposable` and `await using page` does not + * type-check against this package's `engines.node >=20.3` floor — the symbol arrived in Node 20.4. Every page + * this view yields is closed for you as the walk advances; `Page.close()` is the manual counterpart, and is + * idempotent. + * + * Single-use (PAGE-14) — **per view, not per paginator**. A second `[Symbol.asyncIterator]()` on *this* + * returned view fails loudly rather than silently restarting the walk. Calling `pages()` again is the + * sanctioned recovery path PAGE-14 itself names ("a caller restarts pagination by requesting a fresh view + * from the engine"), so it returns a new, independent view. Guarding `pages()` too would make the engine + * stateful, which PAGE-8 forbids ("the engine itself MUST hold only immutable configuration and be safe to + * share") and would break two concurrent callers sharing one `Paginator`. + */ + pages(): AsyncIterable<Page<T>> { + const walk = this.#walk.bind(this); + let iteratorTaken = false; + return { + [Symbol.asyncIterator]: (): AsyncIterator<Page<T>> => { + // PAGE-14 governs obtaining the *iterator*, not calling pages(), and guarding only the method would + // leave the exact hole the requirement names: `for await` calls Symbol.asyncIterator afresh each time, + // so iterating one returned view twice would silently restart the entire walk. 6b's SseStream guards at + // this same level, for the same reason. + if (iteratorTaken) { + throw new PaginationError( + 'the page-level view is single-use; its iterator may be obtained at most once', + ); + } + iteratorTaken = true; + return walk()[Symbol.asyncIterator](); + }, + }; + } + + /** The one drive routine both views share. */ + async *#walk(): AsyncGenerator<Page<T>> { + const {transport, strategy, initialRequest, maxPages, options, signal} = + this.#init; + let request: Request | undefined = initialRequest; + let fetched = 0; + let held: Page<T> | undefined; + + try { + while (request !== undefined) { + // PAGE-25/PAGE-26: check abort before dispatch so no extra request is sent past the abort boundary. + if (isAborted(signal)) return; + // PAGE-9: cap stops the walk even when the strategy still reports a next request. + if (maxPages !== undefined && fetched >= maxPages) return; + + // PAGE-12: release the previous page before dispatching the next request, so the two-page window + // does not hold connections open across subsequent network exchanges. + if (held !== undefined) { + const previous = held; + held = undefined; + await previous.close(); + } + + const response = await transport.send(request, options, signal); + fetched += 1; + + // PAGE-26/PAGE-33: an abort that landed while this exchange was in flight means the page must be + // dropped AND closed rather than delivered. + if (isAborted(signal)) { + await closeQuietly(response); + return; + } + + const info: PageInfo<T> = await parseOrClose( + strategy, + response, + request, + ); + held = await pageOrClose(response, info); + request = info.nextRequest; + yield held; + } + } finally { + // Covers exhaustion, an early `break`, and a consumer throw — a `for await` loop drives `.return()` on the + // generator (AsyncIteratorClose), which executes this `finally` block (PAGE-12, PAGE-27, PAGE-32). If + // `held.close()` throws during an active `.return()` unwind, the ECMAScript specification discards the close + // error and propagates the consumer's original error, satisfying PAGE-32's requirement. + if (held !== undefined) await held.close(); + } + } +} + +/** + * PAGE-13: if `parse` rejects, the page was never constructed, so nothing else will close this response — do it + * inline on the exceptional path. A close failure must not mask the parse failure: parse error primary, close + * error suppressed. + */ +async function parseOrClose<T>( + strategy: PaginationStrategy<T>, + response: Response, + template: Request, +): Promise<PageInfo<T>> { + try { + return await strategy.parse(response, template); + } catch (parseError: unknown) { + return closeThenRethrow(response, parseError, 'pagination parse failed'); + } +} + +/** + * PAGE-4: `parse` must always return a well-formed result, and must never signal termination through a side + * channel. A strategy that returns nothing is a programmer error, so the walk crashes at the fault rather than + * silently ending as if the server had run out of pages. + * + * PAGE-27: and it crashes *after* releasing the response. `parse` returning a malformed value is the one exit + * from this loop the `finally` in `#walk` cannot cover — `held` is still `undefined` there, because assigning it + * is precisely what failed — so, like PAGE-13's parse rejection, the release happens inline (audit #67 / #79). + * + * Both checks reject `null` as well as `undefined`, which is what their messages have always claimed. Testing + * only for `undefined` let `{items: null}` through to `Page`'s constructor, where the item copy surfaced as a + * bare `TypeError` from spread — naming nothing a caller could act on, and leaking the response on the way. + * + * Not async: the only asynchrony here is the close, and only on the failure path. + */ +function pageOrClose<T>( + response: Response, + info: PageInfo<T>, +): Promise<Page<T>> { + try { + invariant( + (info as unknown) !== undefined && (info as unknown) !== null, + 'PaginationStrategy.parse must return a PageInfo, never null or undefined', + ); + invariant( + (info.items as unknown) !== undefined && (info.items as unknown) !== null, + 'PageInfo.items must never be null or absent (PAGE-2)', + ); + return Promise.resolve(new Page(response, info.items)); + } catch (buildError: unknown) { + return closeThenRethrow( + response, + buildError, + 'the pagination strategy returned a malformed PageInfo', + ); + } +} + +/** + * Release `response`, then rethrow `primary`. Shared by the two inline-close paths so they cannot drift: a close + * failure is attached as suppressed and never masks the failure that got here first (PAGE-13, PAGE-15). + */ +async function closeThenRethrow( + response: Response, + primary: unknown, + context: string, +): Promise<never> { + try { + await response.close(); + } catch (closeError: unknown) { + throw suppress( + primary, + closeError, + `${context} and releasing the response also failed`, + ); + } + throw primary; +} + +/** PAGE-26: on an already-settled cancellation path, a close error is swallowed — nothing is left to report to. */ +async function closeQuietly(response: Response): Promise<void> { + try { + await response.close(); + } catch { + // Deliberately swallowed: the walk has already ended and there is no in-flight result to attach this to. + } +} + +function isAborted(signal?: AbortSignal): boolean { + return signal?.aborted === true; +} diff --git a/packages/core/src/pagination/query-splice.property.test.ts b/packages/core/src/pagination/query-splice.property.test.ts new file mode 100644 index 0000000..56a51af --- /dev/null +++ b/packages/core/src/pagination/query-splice.property.test.ts @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/query-splice.property.test.ts +import {expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {UrlConstructionError} from '../http/errors.js'; +import {readQueryParam, spliceQueryParam} from './query-splice.js'; + +/** + * Segment names and values drawn from characters a real server actually sends, including hostile ones. + * + * The alphabet deliberately excludes `"`, `#`, `<`, `>`, and `'`: those sit in the WHATWG query percent-encode + * set, so assigning to `URL.search` rewrites them and byte-for-byte preservation genuinely does not hold. That + * boundary is pinned by its own named test above rather than being smuggled into a property that would then + * fail for a reason unrelated to the splice logic this property exists to check. + */ +const rawSegment = fc + .tuple( + fc.stringMatching(/^[a-z]{1,6}$/), + fc.stringMatching(/^[a-zA-Z0-9:%._~-]{0,10}$/), + ) + .map(([name, value]) => `${name}=${value}`); + +test('every untargeted segment survives the splice byte-for-byte (PAGE-21)', () => { + fc.assert( + fc.property( + fc.array(rawSegment, {maxLength: 6}), + fc.string({minLength: 1}), + (segments, newValue) => { + const untargeted = segments.filter(s => !s.startsWith('page=')); + const url = new URL( + `https://h/p?${[...untargeted, 'page=1'].join('&')}`, + ); + const out = spliceQueryParam(url, 'page', newValue); + const outSegments = out.search.replace(/^\?/, '').split('&'); + return untargeted.every( + (segment, index) => outSegments[index] === segment, + ); + }, + ), + ); +}); + +test('write-then-read is the identity for any value (PAGE-22)', () => { + fc.assert( + fc.property(fc.string(), value => { + const url = spliceQueryParam( + new URL('https://h/p?a=1&b=2'), + 'cursor', + value, + ); + return readQueryParam(url, 'cursor') === value; + }), + ); +}); + +/** + * Strings that mix ordinary query text with UNPAIRED surrogate code units — the same generator + * `http/query-params.test.ts` uses, for the same reason: `fc.string()`'s default unit is printable + * ASCII, so the `URIError` path would otherwise go ungenerated. That is exactly why the identity + * property above never caught it. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); + +test('no URIError escapes the splice or the read, whatever a server sent (PAGE-22)', () => { + fc.assert( + fc.property( + surrogateBearingString, + surrogateBearingString, + (name, value) => { + const url = new URL('https://h/p?a=1'); + try { + const out = spliceQueryParam(url, name, value); + expect(readQueryParam(out, name)).toBe(value); + } catch (e: unknown) { + // The one sanctioned failure: inside the error tree, from the call that was handed the + // value. A `URIError` here means the guard was bypassed. + expect(e).toBeInstanceOf(UrlConstructionError); + } + }, + ), + {numRuns: 500}, + ); +}); diff --git a/packages/core/src/pagination/query-splice.test.ts b/packages/core/src/pagination/query-splice.test.ts new file mode 100644 index 0000000..ca546a7 --- /dev/null +++ b/packages/core/src/pagination/query-splice.test.ts @@ -0,0 +1,196 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/query-splice.test.ts +// Exercises: PAGE-21 (verbatim splice, untargeted params byte-for-byte), PAGE-22 (RFC 3986 component encoding, +// literal + is data; a component with no UTF-8 form is rejected as UrlConstructionError), PAGE-23 +// (replace-first / append / remove, order preserved), PAGE-24 (non-query components preserved exactly). +import {describe, expect, test} from 'bun:test'; +import {UrlConstructionError} from '../http/errors.js'; +import {readQueryParam, spliceQueryParam} from './query-splice.js'; + +const at = (href: string): URL => new URL(href); +const query = (url: URL): string => url.search.replace(/^\?/, ''); + +test('untargeted parameters survive byte-for-byte, order preserved (PAGE-21)', () => { + const out = spliceQueryParam( + at('https://h/p?flag&filter=a:b&page=1'), + 'page', + '2', + ); + expect(query(out)).toBe('flag&filter=a:b&page=2'); +}); + +test('a value-less flag stays value-less', () => { + const out = spliceQueryParam(at('https://h/p?flag&page=1'), 'page', '2'); + expect(query(out)).toContain('flag&'); + expect(query(out)).not.toContain('flag='); +}); + +test('reserved characters in untargeted values are not rewritten (PAGE-21)', () => { + const out = spliceQueryParam(at('https://h/p?a=x:y/z&page=1'), 'page', '2'); + expect(query(out)).toBe('a=x:y/z&page=2'); +}); + +test('a newly-set value uses RFC 3986 component encoding (PAGE-22)', () => { + expect(query(spliceQueryParam(at('https://h/p'), 'q', 'a b'))).toBe( + 'q=a%20b', + ); + expect(query(spliceQueryParam(at('https://h/p'), 'token', 'a+b/c='))).toBe( + 'token=a%2Bb%2Fc%3D', + ); +}); + +test('reading decodes with the same semantics — a literal + reads back as + (PAGE-22)', () => { + expect(readQueryParam(at('https://h/p?q=a+b'), 'q')).toBe('a+b'); + expect(readQueryParam(at('https://h/p?q=a%20b'), 'q')).toBe('a b'); +}); + +test('a value-less flag reads as the empty string, an absent name as undefined (PAGE-22)', () => { + expect(readQueryParam(at('https://h/p?flag'), 'flag')).toBe(''); + expect(readQueryParam(at('https://h/p?flag'), 'other')).toBeUndefined(); +}); + +test('reading takes the first match when a name repeats (PAGE-22)', () => { + expect(readQueryParam(at('https://h/p?page=1&page=9'), 'page')).toBe('1'); +}); + +test('setting replaces the first occurrence in place and drops later duplicates (PAGE-23)', () => { + expect( + query( + spliceQueryParam(at('https://h/p?page=1&sort=asc&page=9'), 'page', '2'), + ), + ).toBe('page=2&sort=asc'); +}); + +test('setting an absent parameter appends it (PAGE-23)', () => { + expect(query(spliceQueryParam(at('https://h/p?sort=asc'), 'page', '2'))).toBe( + 'sort=asc&page=2', + ); +}); + +test('setting undefined removes the parameter entirely (PAGE-23)', () => { + expect( + query( + spliceQueryParam(at('https://h/p?page=1&sort=asc'), 'page', undefined), + ), + ).toBe('sort=asc'); +}); + +test('removing the only parameter leaves an empty query', () => { + expect( + query(spliceQueryParam(at('https://h/p?page=1'), 'page', undefined)), + ).toBe(''); +}); + +test('setting on a URL with no query at all creates one', () => { + expect(query(spliceQueryParam(at('https://h/p'), 'page', '2'))).toBe( + 'page=2', + ); +}); + +test('every non-query component is preserved exactly (PAGE-24)', () => { + const source = at( + 'https://user:pw@host.example:8443/deep/path?page=1&keep=yes#frag', + ); + const out = spliceQueryParam(source, 'page', '2'); + expect(out.protocol).toBe(source.protocol); + expect(out.username).toBe(source.username); + expect(out.password).toBe(source.password); + expect(out.hostname).toBe(source.hostname); + expect(out.port).toBe(source.port); + expect(out.pathname).toBe(source.pathname); + expect(out.hash).toBe(source.hash); + expect(query(out)).toBe('page=2&keep=yes'); +}); + +test('the input URL is not mutated', () => { + const source = at('https://h/p?page=1'); + spliceQueryParam(source, 'page', '2'); + expect(query(source)).toBe('page=1'); +}); + +test('URLSearchParams-style canonicalization does NOT happen (PAGE-21)', () => { + // URLSearchParams would rewrite `a b` to `a+b` and re-encode `:`; the splice leaves both alone. + const out = spliceQueryParam( + at('https://h/p?msg=a%20b&path=x:y&page=1'), + 'page', + '2', + ); + expect(query(out)).toBe('msg=a%20b&path=x:y&page=2'); +}); + +test('the WHATWG query encode set is the one boundary of byte-for-byte preservation (PAGE-21)', () => { + // Assigning to `URL.search` percent-encodes C0 controls, space, " # < > and (on special schemes) ' — so an + // untargeted segment carrying one of those raw is rewritten. Every such character is one RFC 3986 already + // requires to be encoded in a query, so the only inputs affected were already non-conformant. Pinned here so + // the boundary is known rather than discovered, and recorded in the Deviation Ledger. + const out = spliceQueryParam(at('https://h/p?tag=<raw>&page=1'), 'page', '2'); + expect(query(out)).toBe('tag=%3Craw%3E&page=2'); + + // Everything RFC 3986 permits raw in a query survives untouched — which is the case that actually matters. + const safe = spliceQueryParam( + at('https://h/p?f=a:b/c!d$e(f)*g,h;i@j&page=1'), + 'page', + '2', + ); + expect(query(safe)).toBe('f=a:b/c!d$e(f)*g,h;i@j&page=2'); +}); + +test('stray empty segments are skipped, matching HTTP-31 query parsing', () => { + expect( + query(spliceQueryParam(at('https://h/p?a=1&&b=2&page=1'), 'page', '2')), + ).toBe('a=1&b=2&page=2'); +}); + +describe('a cursor with no UTF-8 form is rejected here, not inside encodeURIComponent (PAGE-22)', () => { + // The splice shares `HTTP-29`'s component encoder, and `encodeURIComponent` throws a bare + // `URIError: URI malformed` on a string carrying an unpaired surrogate. A cursor is SERVER + // -supplied — `{"next":"\ud800"}` is well-formed JSON — so this is reachable without any caller + // mistake, and until audit #67 / #79 it escaped the `DexpaceError` tree entirely. #76 closed the + // same hole at `QueryParamsBuilder.add` and left this one named. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate', LONE_HIGH], + ['a lone low surrogate', LONE_LOW], + ['a lone surrogate inside a longer cursor', `ok${LONE_HIGH}ok`], + ])('spliceQueryParam rejects %s as a value', (_label, value) => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', value), + ).toThrow(UrlConstructionError); + }); + + test('the message names the parameter and never echoes the value', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).toThrow(/value of query parameter "cursor"/); + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).not.toThrow(/secret/); + }); + + test('spliceQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, '2'), + ).toThrow(UrlConstructionError); + }); + + test('readQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => readQueryParam(at('https://h/p?a=1'), LONE_HIGH)).toThrow( + UrlConstructionError, + ); + }); + + test('removing a parameter still validates the name', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, undefined), + ).toThrow(UrlConstructionError); + }); + + test('a well-formed surrogate PAIR is ordinary text and splices normally', () => { + // Rejecting this too would make the rule "no astral characters", which PAGE-22 does not say. + const out = spliceQueryParam(at('https://h/p?a=1'), 'cursor', '\u{1F600}'); + expect(query(out)).toBe('a=1&cursor=%F0%9F%98%80'); + expect(readQueryParam(out, 'cursor')).toBe('\u{1F600}'); + }); +}); diff --git a/packages/core/src/pagination/query-splice.ts b/packages/core/src/pagination/query-splice.ts new file mode 100644 index 0000000..754cc6a --- /dev/null +++ b/packages/core/src/pagination/query-splice.ts @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/query-splice.ts +import {UrlConstructionError} from '../http/errors.js'; +import { + decodeQueryComponent, + encodeQueryComponent, +} from '../http/query-params.js'; +import {hasLoneSurrogate} from '../http/rfc3986.js'; + +/** + * `encodeQueryComponent` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` on a string + * carrying an unpaired surrogate — such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined + * for it (PAGE-22, HTTP-29). + * + * Reachable here without any caller mistake, which is what separates this call site from the others #76 closed: + * a cursor is SERVER-supplied, and `{"next":"\ud800"}` is well-formed JSON that `JSON.parse` hands back + * verbatim. The failure was a bare `URIError` from inside a strategy's `parse`, outside the `DexpaceError` tree + * and naming neither the parameter nor the page it came from (audit #67 / #79). + * + * The same class `QueryParamsBuilder.add` throws for the same input, since it is the same defect in the same + * encoder; `hasLoneSurrogate` is the single-sourced predicate, so the two cannot drift. The value itself is + * never echoed — a cursor is opaque server state and can carry a session token. + */ +function requireEncodable( + what: 'name' | 'value', + parameterName: string, + text: string, +): void { + if (!hasLoneSurrogate(text)) return; + const subject = + what === 'name' + ? 'a query parameter name' + : `the value of query parameter "${parameterName}"`; + throw new UrlConstructionError( + `${subject} contains an unpaired surrogate and cannot be percent-encoded`, + ); +} + +/** + * Rewrite one query parameter, splicing the raw query string rather than re-rendering it (PAGE-21–PAGE-24). + * + * **Why not `URLSearchParams`.** It re-serializes the *entire* query through its own canonical encoding on + * every mutation: untouched parameters get reordered and re-encoded (against PAGE-21's byte-for-byte rule), and + * a space becomes `+` rather than the `%20` this port standardizes on (HTTP-29). + * + * **Why not `QueryParams`.** Same problem in a different costume — `encode()` re-renders the whole query from + * the parsed model. `QueryParams` is the right tool for *building* a query and the wrong one for *splicing* one. + * Only the component *encoder* is shared, which is the part PAGE-22 and HTTP-29 genuinely agree on. + * + * Passing `undefined` removes the parameter. Setting replaces the first occurrence in place and drops later + * duplicates — the single-value convention paging parameters follow. Everything else is copied byte-for-byte. + * + * @throws UrlConstructionError when `name` or `value` carries an unpaired surrogate, and so has no + * percent-encoded form. + * + * @internal + */ +export function spliceQueryParam( + url: URL, + name: string, + value: string | undefined, +): URL { + requireEncodable('name', name, name); + if (value !== undefined) requireEncodable('value', name, value); + const encodedName = encodeQueryComponent(name); + const segments = splitQuery(url.search); + + const out: string[] = []; + let replaced = false; + + for (const segment of segments) { + if (nameOf(segment) !== encodedName) { + out.push(segment); // byte-for-byte, untouched + continue; + } + if (replaced) continue; // PAGE-23: later duplicates are dropped + replaced = true; + if (value !== undefined) + out.push(`${encodedName}=${encodeQueryComponent(value)}`); + } + + if (!replaced && value !== undefined) { + out.push(`${encodedName}=${encodeQueryComponent(value)}`); + } + + // Rebuilding through `URL` preserves scheme, userinfo, host, port, path, and fragment exactly (PAGE-24); + // only `search` is assigned. + const next = new URL(url.href); + next.search = out.length === 0 ? '' : `?${out.join('&')}`; + return next; +} + +/** + * Read one query parameter with the same RFC 3986 semantics the splice writes (PAGE-22). + * + * A literal `+` reads back as `+`, `%20` as a space, a value-less flag as the empty string, and an absent name + * as `undefined`. First match wins. + * + * @throws UrlConstructionError when `name` carries an unpaired surrogate, and so has no percent-encoded form. + * + * @internal + */ +export function readQueryParam(url: URL, name: string): string | undefined { + requireEncodable('name', name, name); + const encodedName = encodeQueryComponent(name); + for (const segment of splitQuery(url.search)) { + if (nameOf(segment) !== encodedName) continue; + const eq = segment.indexOf('='); + return eq === -1 ? '' : decodeQueryComponent(segment.slice(eq + 1)); + } + return undefined; +} + +/** + * Split a raw query into `&`-separated segments, dropping the leading `?` and any stray empty segments. + * + * Dropping empty segments (`?a=1&&b=2` → two segments) is not a byte-for-byte violation to apologize for — it + * is the same leniency `HTTP-31` already mandates for query *parsing*, "stray `&` is skipped." Doing something + * different here would put two disagreeing readings of the same query string in one codebase. + */ +function splitQuery(search: string): string[] { + const raw = search.startsWith('?') ? search.slice(1) : search; + return raw.length === 0 + ? [] + : raw.split('&').filter(segment => segment.length > 0); +} + +/** The raw (still-encoded) name of a segment. A value-less flag is all name. */ +function nameOf(segment: string): string { + const eq = segment.indexOf('='); + return eq === -1 ? segment : segment.slice(0, eq); +} diff --git a/packages/core/src/pagination/strategies.test.ts b/packages/core/src/pagination/strategies.test.ts new file mode 100644 index 0000000..8eebdf1 --- /dev/null +++ b/packages/core/src/pagination/strategies.test.ts @@ -0,0 +1,335 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/strategies.test.ts +// Exercises: PAGE-16 (cursor: single body read, null OR empty ends, configurable parameter), PAGE-17 +// (page-number: empty items ends, start-page fallback on absent/empty/garbage, configurable name and start), +// PAGE-18/19/20 (link header: rel=next, RFC 3986 reference resolution, query-only reference preserves the path, +// unresolvable target ends the stream without throwing, and the spec's own `<not a url>` conformance fixture +// resolving as a relative reference instead -- recorded as a deliberate reading in docs/deviations.md under +// "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69)), PAGE-22 (a server-supplied cursor with +// no UTF-8 form fails inside the error tree). +import {expect, test} from 'bun:test'; +import {DexpaceError, UrlConstructionError} from '../http/errors.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import { + cursorStrategy, + linkHeaderStrategy, + pageNumberStrategy, +} from './strategies.js'; + +const template = (href: string): Request => + ({ + url: new URL(href), + newBuilder() { + let target = new URL(href); + return { + url(next: URL) { + target = next; + return this; + }, + build: () => ({url: target}) as unknown as Request, + }; + }, + }) as unknown as Request; + +const response = (init: { + url?: string; + headers?: Record<string, readonly string[]>; +}): Response => + ({ + request: {url: new URL(init.url ?? 'https://api.test/repo/issues?page=1')}, + headers: { + get: (name: string) => init.headers?.[name.toLowerCase()]?.[0], + getAll: (name: string) => init.headers?.[name.toLowerCase()] ?? [], + }, + }) as unknown as Response; + +// ---- cursor (PAGE-16) ---- + +test('a cursor sets the configured query parameter on the next request (PAGE-16)', async () => { + let reads = 0; + const strategy = cursorStrategy<string>({ + extract: () => { + reads += 1; + return Promise.resolve({items: ['a'], cursor: 'c'}); + }, + }); + const info = await strategy.parse( + response({}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.search).toBe('?cursor=c'); + expect(reads).toBe(1); +}); + +test('the cursor parameter name is configurable (PAGE-16)', async () => { + const strategy = cursorStrategy<string>({ + extract: () => Promise.resolve({items: ['a'], cursor: 'c'}), + parameterName: 'after', + }); + const info = await strategy.parse( + response({}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.search).toBe('?after=c'); +}); + +test.each([null, '', undefined])( + 'a %p cursor ends the stream (PAGE-16)', + async cursor => { + const strategy = cursorStrategy<string>({ + extract: () => + Promise.resolve({items: ['a'], cursor: cursor as string | null}), + }); + const info = await strategy.parse( + response({}), + template('https://api.test/items'), + ); + expect(info.nextRequest).toBeUndefined(); + expect(info.items).toEqual(['a']); + }, +); + +// ---- page number (PAGE-17) ---- + +test('the first page with no parameter advances to start+1 (PAGE-17)', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({url: 'https://api.test/items'}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.search).toBe('?page=2'); +}); + +test('an empty items list ends the stream, defensively (PAGE-17)', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve([]), + }); + const info = await strategy.parse( + response({url: 'https://api.test/items?page=4'}), + template('https://api.test/items?page=4'), + ); + expect(info.nextRequest).toBeUndefined(); +}); + +test('the current page comes from the EXECUTED request, not the template (PAGE-17)', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({url: 'https://api.test/items?page=7'}), + template('https://api.test/items?page=1'), + ); + expect(info.nextRequest?.url.search).toBe('?page=8'); +}); + +test.each(['', 'garbage', '1.5', '-3'])( + 'a %p page value falls back to the start page (PAGE-17)', + async value => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({url: `https://api.test/items?page=${value}`}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.search).toBe('?page=2'); + }, +); + +test('the parameter name and start page are configurable, supporting 0-based servers (PAGE-17)', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + parameterName: 'offset', + startPage: 0, + }); + const info = await strategy.parse( + response({url: 'https://api.test/items'}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.search).toBe('?offset=1'); +}); + +// ---- link header (PAGE-18/19/20) ---- + +test('an absolute rel=next target is used as-is (PAGE-19)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({headers: {link: ['<https://other.test/x?page=2>; rel="next"']}}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.href).toBe('https://other.test/x?page=2'); +}); + +test('a query-only reference preserves the base path and replaces only the query (PAGE-19)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({ + url: 'https://api.test/repo/issues?page=1', + headers: {link: ['<?page=2>; rel="next"']}, + }), + template('https://api.test/repo/issues?page=1'), + ); + // RFC 2396 would drop the last path segment here; RFC 3986 (and WHATWG URL) does not. + expect(info.nextRequest?.url.pathname).toBe('/repo/issues'); + expect(info.nextRequest?.url.search).toBe('?page=2'); +}); + +test('a relative path reference resolves against the response URL (PAGE-19)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({ + url: 'https://api.test/repo/issues?page=1', + headers: {link: ['<../pulls>; rel="next"']}, + }), + template('https://api.test/repo/issues'), + ); + expect(info.nextRequest?.url.pathname).toBe('/pulls'); +}); + +test('an unresolvable target ends the stream rather than throwing (PAGE-19)', async () => { + // Picking this fixture takes care. With a base supplied, WHATWG `URL` resolves almost *anything* as a + // relative reference rather than failing — `ht!tp://%%%` has no valid scheme, so it parses happily as a path + // and yields a defined next request, which would make this test assert nothing. A genuinely unparseable + // target needs a valid scheme and a broken authority, so the absolute-URL path is taken and fails: `http://[` + // opens an IPv6 literal that never closes. + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({headers: {link: ['<http://[>; rel="next"']}}), + template('https://api.test/items'), + ); + expect(info.nextRequest).toBeUndefined(); + expect(info.items).toEqual(['a']); +}); + +test("the spec's `<not a url>` fixture is a RELATIVE reference, so it is followed (PAGE-19)", async () => { + // PAGE-19's conformance note (`docs/product-spec/12-pagination.md:52`) gives `<not a url>; rel=next` as an + // example of "stream ends, no exception". Under WHATWG URL — the resolver `strategies.ts` uses, and the + // only one available without a runtime dependency (SEAM-1) — a base makes that string a perfectly valid + // relative path reference: it resolves to `/repo/not%20a%20url`. The requirement's own normative sentence + // is "a target that CANNOT RESOLVE into a valid URL", and this one resolves, so the port follows it. Only + // the illustrative fixture disagrees; the test below keeps the end-of-stream half honest with a target that + // genuinely fails to resolve. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase" — + // rejected alternative: an ad-hoc "looks unparseable" heuristic in front of the resolver. + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({ + url: 'https://api.test/repo/issues?page=1', + headers: {link: ['<not a url>; rel="next"']}, + }), + template('https://api.test/repo/issues?page=1'), + ); + expect( + () => new URL('not a url', 'https://api.test/repo/issues'), + ).not.toThrow(); + expect(info.nextRequest?.url.href).toBe( + 'https://api.test/repo/not%20a%20url', + ); +}); + +test('the fixture above really is unparseable — the guard is not vacuous (PAGE-19)', () => { + expect(() => new URL('http://[', 'https://api.test/items')).toThrow(); + // And the near-miss that does NOT throw, pinned so nobody "simplifies" the fixture back to it later. + expect(() => new URL('ht!tp://%%%', 'https://api.test/items')).not.toThrow(); +}); + +test('no Link header ends the stream (PAGE-18)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({}), + template('https://api.test/items'), + ); + expect(info.nextRequest).toBeUndefined(); +}); + +test('the header name is configurable (PAGE-18)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + headerName: 'X-Links', + }); + const info = await strategy.parse( + response({headers: {'x-links': ['</next>; rel="next"']}}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.pathname).toBe('/next'); +}); + +test('two separate Link header instances are both considered (PAGE-20)', async () => { + const strategy = linkHeaderStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({headers: {link: ['</a>; rel="last"', '</b>; rel="next"']}}), + template('https://api.test/items'), + ); + expect(info.nextRequest?.url.pathname).toBe('/b'); +}); + +test('one strategy instance is safe across two concurrent walks (PAGE-5)', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + }); + const [first, second] = await Promise.all([ + strategy.parse( + response({url: 'https://api.test/items?page=1'}), + template('https://api.test/items'), + ), + strategy.parse( + response({url: 'https://api.test/items?page=9'}), + template('https://api.test/items'), + ), + ]); + expect(first.nextRequest?.url.search).toBe('?page=2'); + expect(second.nextRequest?.url.search).toBe('?page=10'); +}); + +// ---- a server-supplied component with no UTF-8 form (PAGE-22, audit #67 / #79) ---- + +test('a cursor carrying an unpaired surrogate fails as UrlConstructionError, not URIError', async () => { + // `{"next":"\ud800"}` is well-formed JSON, so `extract` can hand one back without the caller + // having done anything wrong. Before the fix this surfaced as a bare `URIError: URI malformed` + // from inside `encodeURIComponent`, outside the `DexpaceError` tree. + const strategy = cursorStrategy<string>({ + extract: () => Promise.resolve({items: ['a'], cursor: 'next\uD800'}), + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); + expect(caught).toBeInstanceOf(DexpaceError); +}); + +test('a page-number parameter name carrying an unpaired surrogate fails the same way', async () => { + const strategy = pageNumberStrategy<string>({ + extract: () => Promise.resolve(['a']), + parameterName: 'p\uD800', + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); +}); diff --git a/packages/core/src/pagination/strategies.ts b/packages/core/src/pagination/strategies.ts new file mode 100644 index 0000000..84bf409 --- /dev/null +++ b/packages/core/src/pagination/strategies.ts @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/strategies.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {findNextLink} from './link-header.js'; +import {pageInfo, type PageInfo} from './page.js'; +import {readQueryParam, spliceQueryParam} from './query-splice.js'; +import type {PaginationStrategy} from './strategy.js'; + +/** Build the next request by swapping only the URL, preserving method, headers, and body (PAGE-23). */ +function withUrl(template: Request, url: URL): Request { + return template.newBuilder().url(url).build(); +} + +/** + * Cursor/continuation-token pagination (PAGE-16). + * + * `extract` reads items and the next cursor from **one** read of the response body. It is caller-supplied + * rather than codec-driven because §12 requires the engine to be serde-agnostic: naming a `Serde` here would + * couple pagination to a wire format it has no business knowing about. A caller using `@dexpace/codec-json` + * simply closes over it inside `extract`. + * + * A `null` **or empty** cursor ends the stream — both, because a server returning `""` for "no more pages" is + * common enough that treating it as a real cursor produces an infinite walk. + * + * @throws UrlConstructionError from `parse` when the cursor the server sent, or `parameterName`, carries an + * unpaired surrogate: such a string has no UTF-8 form and so no percent-encoded form either (PAGE-22). The + * engine closes the response on that path like any other parse failure (PAGE-13). + * + * @public + */ +export function cursorStrategy<T>(init: { + extract: ( + response: Response, + ) => Promise<{items: readonly T[]; cursor?: string | null | undefined}>; + parameterName?: string | undefined; +}): PaginationStrategy<T> { + const parameterName = init.parameterName ?? 'cursor'; + return Object.freeze({ + async parse(response: Response, template: Request): Promise<PageInfo<T>> { + const {items, cursor} = await init.extract(response); + if (cursor === null || cursor === undefined || cursor.length === 0) + return pageInfo(items); + return pageInfo( + items, + withUrl( + template, + spliceQueryParam(template.url, parameterName, cursor), + ), + ); + }, + }); +} + +/** + * Page-number pagination (PAGE-17). + * + * An empty items list ends the stream **before** any arithmetic runs — defensive against servers that keep + * returning an empty page past the end instead of signalling termination, which would otherwise walk forever. + * + * The current page comes from the *executed* request's query (`response.request.url`), not the template's. + * The template is not a fixed page-1 request -- it advances with the walk, since this function returns the + * next one as `nextRequest` and the engine makes that the following hop's template + * (`paginator.ts:165,213`; the contract is on `PaginationStrategy.parse` in `strategy.ts:10-15`). It is the + * *pre-flight* request for this hop, so it is the response's own request that reflects a redirect or any + * rewrite a step applied on the way out, and that is the page number worth incrementing. An absent, empty, + * or non-numeric value falls back to `startPage`; `startPage: 0` supports 0-based servers. + * + * @throws UrlConstructionError from `parse` when `parameterName` carries an unpaired surrogate, which has no + * percent-encoded form (PAGE-22). + * + * @public + */ +export function pageNumberStrategy<T>(init: { + extract: (response: Response) => Promise<readonly T[]>; + parameterName?: string | undefined; + startPage?: number | undefined; +}): PaginationStrategy<T> { + const parameterName = init.parameterName ?? 'page'; + const startPage = init.startPage ?? 1; + return Object.freeze({ + async parse(response: Response, template: Request): Promise<PageInfo<T>> { + const items = await init.extract(response); + if (items.length === 0) return pageInfo(items); + + const raw = readQueryParam(response.request.url, parameterName); + const parsed = + raw === undefined || raw.length === 0 ? Number.NaN : Number(raw); + const current = + Number.isInteger(parsed) && parsed >= 0 ? parsed : startPage; + + const nextUrl = spliceQueryParam( + template.url, + parameterName, + String(current + 1), + ); + return pageInfo(items, withUrl(template, nextUrl)); + }, + }); +} + +/** + * `Link`-header pagination (PAGE-18, PAGE-19, PAGE-20). + * + * The target resolves as an RFC 3986 reference against the originating response's URL. WHATWG `URL` gets the + * query-only (`?page=2`) case right natively — it preserves the base path and replaces only the query, where + * RFC 2396's older rule would drop the last path segment. + * + * A target that cannot resolve into a valid URL is **end-of-stream, not an error** (PAGE-19). That is why the + * `URL` constructor's throw is caught and converted here — one of the few places in this codebase where + * swallowing an exception is the specified behavior rather than a smell. + * + * @public + */ +export function linkHeaderStrategy<T>(init: { + extract: (response: Response) => Promise<readonly T[]>; + headerName?: string | undefined; +}): PaginationStrategy<T> { + const headerName = init.headerName ?? 'Link'; + return Object.freeze({ + async parse(response: Response, template: Request): Promise<PageInfo<T>> { + const items = await init.extract(response); + const target = findNextLink(response.headers.getAll(headerName)); + if (target === undefined) return pageInfo(items); + + let resolved: URL; + try { + resolved = new URL(target, response.request.url); + } catch { + return pageInfo(items); // PAGE-19: unresolvable means end of stream, never an error. + } + return pageInfo(items, withUrl(template, resolved)); + }, + }); +} diff --git a/packages/core/src/pagination/strategy.test.ts b/packages/core/src/pagination/strategy.test.ts new file mode 100644 index 0000000..01834a1 --- /dev/null +++ b/packages/core/src/pagination/strategy.test.ts @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/strategy.test.ts +// Exercises: PAGE-5 (strategy contract), PAGE-29 (async parse boundary), PAGE-30 (synchronous item array). +// Pure type declarations, so the assertions are expect-type only (styleguide 11.6) and fire under `bun run typecheck`. +import {test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import type {PageInfo} from './page.js'; +import type {PaginationStrategy} from './strategy.js'; + +test('parse returns a promise — a synchronous body read does not exist in this runtime (PAGE-5)', () => { + expectTypeOf<PaginationStrategy<number>['parse']>().returns.toEqualTypeOf< + Promise<PageInfo<number>> + >(); +}); + +test('parse receives the response and the original request template (PAGE-5)', () => { + expectTypeOf<PaginationStrategy<number>['parse']>().parameters.toBeArray(); +}); + +test('a strategy is generic in its item type, not in a codec (PAGE-5, §12 serde-agnostic)', () => { + expectTypeOf<PaginationStrategy<{id: string}>>().not.toBeAny(); +}); diff --git a/packages/core/src/pagination/strategy.ts b/packages/core/src/pagination/strategy.ts new file mode 100644 index 0000000..abbf5bb --- /dev/null +++ b/packages/core/src/pagination/strategy.ts @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pagination/strategy.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import type {PageInfo} from './page.js'; + +/** + * A stateless parser turning one response into that page's items plus the next page's request (PAGE-5). + * + * **On `template`.** The glossary calls this "the original request template," but the engine passes the request + * that produced *this* response — i.e. it advances with the walk. That is deliberate and is what the built-in + * strategies need: `cursorStrategy` splices the new cursor onto the request actually just executed, so a walk + * accumulates one cursor parameter rather than re-deriving page N's URL from page 1's every time. Read the + * parameter as "the request to derive the next one from," and use `response.request` when you specifically want + * the executed request's own URL (`pageNumberStrategy` does). + * + * **Contract obligations on implementors** — none of these can be enforced by the type system, so they are + * stated here and covered by the engine's own tests: + * + * - **Read the body at most once.** The body is single-use. The engine hands you the response exactly once and + * never reads the body itself, so it is entirely yours — but only once. + * - **Do not retain the response or its body past the call.** The engine closes the response as soon as `parse` + * resolves, so a retained body is already dead; holding one produces an intermittent failure rather than a + * clean one. + * - **Do not close or mutate the response.** Lifecycle ownership belongs to the engine. + * - **Be immutable and safe to share.** One strategy instance may drive several concurrent walks. Keep no + * per-call state on `this`. + * - **Never signal termination by throwing.** Return `pageInfo(items)` with no next request. A throw means a + * genuine parse failure, and the engine treats it as one (PAGE-13). + * + * **Why `parse` is asynchronous.** `PAGE-5` says a strategy must read what it needs "synchronously inside + * parse." This runtime has no synchronous body read — the bytes may not have arrived — so the literal reading + * is unimplementable. Every enforceable part of the requirement's intent survives the promise, as listed above. + * Do not "fix" this back to a synchronous signature; it cannot work. + * + * @public + */ +export interface PaginationStrategy<T> { + /** + * Parses one HTTP response into this page's items and the request for the next page (PAGE-5). + * + * @param response - the executed response to extract items from. + * @param template - the request to derive the next page's request from. + * @returns a {@link PageInfo} containing items and the optional next request. + */ + parse(response: Response, template: Request): Promise<PageInfo<T>>; +} diff --git a/packages/core/src/pipeline/builder.test.ts b/packages/core/src/pipeline/builder.test.ts new file mode 100644 index 0000000..9e5b306 --- /dev/null +++ b/packages/core/src/pipeline/builder.test.ts @@ -0,0 +1,649 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/builder.test.ts +// Exercises: PIPE-4/5/6 (a pillar admits at most one step; a distinct collision throws; the same type is +// idempotent), PIPE-7 (non-pillar stages preserve insertion order through append/prepend), PIPE-8 (SEND +// rejects any insertion), PIPE-18/19 (insertAfter/insertBefore/replace act relative to the first anchor +// instance; cross-stage is rejected), PIPE-20 (remove deletes every instance, no-op when absent), PIPE-21 +// (a missing anchor fails), PIPE-22 (an edit sequence flattens the same as constructing the final set from +// scratch), PIPE-23 (a colliding reload leaves prior content untouched, and a same-type pillar repeat inside +// one batch seats only one step), PIPE-25 (flatten order), PIPE-38 (appendAll preserves batch order; +// prependAll reverses it), PIPE-1/PIPE-2 (a built pipeline, driven: entry in STAGE_ORDER, exit reversed), +// PIPE-35 (seedFrom's explicit, non-defaulted flatten-vs-nest modes), OBS-29 + CTX-16 (the public +// instrumentation options bag: the supplied bundle opens the operation span, the operation name reaches +// the request context, and flatten seeding carries both) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; +import type {Transport} from '../seams/transport.js'; +import {PipelineBuilder} from './builder.js'; +import { + AnchorNotFoundError, + CrossStageEditError, + PillarCollisionError, + ReservedStageError, +} from './errors.js'; +import type {Runtime} from './runtime.js'; +import {PILLAR_STAGES, STAGE_ORDER, type Stage} from './stage.js'; +import type {Step, StepDescriptor} from './step.js'; + +function aRequest(url: string): Request { + return Request.newBuilder().url(url).build(); +} + +function aResponse(status: number): Response { + return Response.newBuilder() + .request(aRequest('https://example.com')) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .build(); +} + +class StubTransport implements Transport { + #response: Response; + + constructor(response: Response) { + this.#response = response; + } + + send(): Promise<Response> { + return Promise.resolve(this.#response); + } + + close(): Promise<void> { + return Promise.resolve(); + } +} + +// A driven pipeline installs a context per call and evicts it again in `Runtime.send()`'s own `finally`, so +// this file leaves the process-wide `contextStore` exactly as it found it -- no `afterEach(clear)`, which +// would wipe entries a sibling test file installed (4a's plan Global Constraints; testing.md:50,52). + +const noopStep: Step = async (_request, ctx) => ctx.next(); + +function descriptor( + label: string, + stage: StepDescriptor['stage'], +): StepDescriptor { + return {type: Symbol(label), stage, fn: noopStep}; +} + +function labelsOf(runtime: Runtime): (string | undefined)[] { + return runtime.steps.map(d => d.type.description); +} + +function aBuilder(): PipelineBuilder { + return new PipelineBuilder(new StubTransport(aResponse(200))); +} + +describe('PipelineBuilder pillar rules (PIPE-4, PIPE-5, PIPE-6)', () => { + test('a pillar stage admits at most one step', () => { + const builder = aBuilder().append(descriptor('a', 'RETRY')); + + expect(builder.build().steps).toHaveLength(1); + }); + + test('installing a distinct second step onto an occupied pillar throws, naming both types', () => { + const builder = aBuilder(); + const a = descriptor('a', 'RETRY'); + const b = descriptor('b', 'RETRY'); + builder.append(a); + + try { + builder.append(b); + throw new Error( + 'unreachable -- append must throw for a distinct pillar collision', + ); + } catch (error) { + expect(error).toBeInstanceOf(PillarCollisionError); + expect((error as PillarCollisionError).existingType).toBe(a.type); + expect((error as PillarCollisionError).incomingType).toBe(b.type); + } + }); + + test('re-installing the identical descriptor type onto its own pillar is an idempotent no-op', () => { + const builder = aBuilder(); + const a = descriptor('a', 'RETRY'); + + builder.append(a).append(a); + + expect(builder.build().steps).toHaveLength(1); + }); +}); + +describe('PipelineBuilder remove then re-install (PIPE-20, PIPE-5)', () => { + test('a pillar emptied by remove accepts a step of a different type', () => { + const first = descriptor('first', 'RETRY'); + const builder = aBuilder().append(first); + + builder.remove(first.type); + builder.append(descriptor('second', 'RETRY')); + + // The emptied bucket must not read as still occupied: PIPE-5's collision is about an occupant, and + // remove left none. + expect(labelsOf(builder.build())).toEqual(['second']); + }); +}); + +describe('PipelineBuilder non-pillar ordering (PIPE-7)', () => { + test('append adds to the tail, prepend adds to the head, within one stage', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const b = descriptor('b', 'PRE_LOGGING'); + const c = descriptor('c', 'PRE_LOGGING'); + + const runtime = aBuilder().append(a).append(c).prepend(b).build(); + + expect(labelsOf(runtime)).toEqual(['b', 'a', 'c']); + }); +}); + +describe('PipelineBuilder batch edits (PIPE-38)', () => { + test('appendAll preserves the batch iteration order', () => { + const steps = ['a', 'b', 'c'].map(label => + descriptor(label, 'PRE_LOGGING'), + ); + + const runtime = aBuilder().appendAll(steps).build(); + + expect(labelsOf(runtime)).toEqual(['a', 'b', 'c']); + }); + + test('prependAll results in the reversed batch order', () => { + const steps = ['a', 'b', 'c'].map(label => + descriptor(label, 'PRE_LOGGING'), + ); + + const runtime = aBuilder().prependAll(steps).build(); + + expect(labelsOf(runtime)).toEqual(['c', 'b', 'a']); + }); +}); + +describe('PipelineBuilder anchor edits (PIPE-18, PIPE-19, PIPE-21)', () => { + test('insertAfter/insertBefore act relative to the FIRST existing instance of the anchor type', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const b = descriptor('b', 'PRE_LOGGING'); + const builder = aBuilder().append(a).append(b); + + builder.insertAfter(a.type, descriptor('c', 'PRE_LOGGING')); + builder.insertBefore(a.type, descriptor('d', 'PRE_LOGGING')); + + expect(labelsOf(builder.build())).toEqual(['d', 'a', 'c', 'b']); + }); + + test('insertAfter/insertBefore/replace reject a cross-stage edit', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const builder = aBuilder().append(a); + const wrongStage = descriptor('x', 'POST_LOGGING'); + + expect(() => builder.insertAfter(a.type, wrongStage)).toThrow( + CrossStageEditError, + ); + expect(() => builder.insertBefore(a.type, wrongStage)).toThrow( + CrossStageEditError, + ); + expect(() => builder.replace(a.type, wrongStage)).toThrow( + CrossStageEditError, + ); + }); + + test('an anchor edit against a missing type throws AnchorNotFoundError', () => { + const builder = aBuilder(); + const missing = Symbol('missing'); + + expect(() => + builder.insertAfter(missing, descriptor('x', 'PRE_LOGGING')), + ).toThrow(AnchorNotFoundError); + expect(() => + builder.replace(missing, descriptor('x', 'PRE_LOGGING')), + ).toThrow(AnchorNotFoundError); + }); + + test('replace swaps the anchor step in place, same stage, same position', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const b = descriptor('b', 'PRE_LOGGING'); + const builder = aBuilder().append(a).append(b); + + builder.replace(a.type, descriptor('a2', 'PRE_LOGGING')); + + expect(labelsOf(builder.build())).toEqual(['a2', 'b']); + }); +}); + +describe('PipelineBuilder remove (PIPE-20)', () => { + test('deletes every instance of a type, preserving relative order of the rest', () => { + const a1 = descriptor('a', 'PRE_LOGGING'); + const b = descriptor('b', 'PRE_LOGGING'); + const a2: StepDescriptor = { + type: a1.type, + stage: 'POST_LOGGING', + fn: noopStep, + }; + const builder = aBuilder().appendAll([a1, b]).append(a2); + + builder.remove(a1.type); + + expect(labelsOf(builder.build())).toEqual(['b']); + }); + + test('is a no-op when the type is absent', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const builder = aBuilder().append(a); + + expect(() => builder.remove(Symbol('absent'))).not.toThrow(); + expect(labelsOf(builder.build())).toEqual(['a']); + }); +}); + +describe('PipelineBuilder reload (PIPE-23)', () => { + test('a colliding batch leaves the existing collection completely unchanged', () => { + const builder = aBuilder().append(descriptor('original', 'PRE_LOGGING')); + + expect(() => + builder.reload([descriptor('x', 'RETRY'), descriptor('y', 'RETRY')]), + ).toThrow(PillarCollisionError); + expect(labelsOf(builder.build())).toEqual(['original']); + }); + + test('a valid batch fully replaces the prior collection', () => { + const builder = aBuilder().append(descriptor('stale', 'PRE_LOGGING')); + + builder.reload([descriptor('fresh', 'POST_LOGGING')]); + + expect(labelsOf(builder.build())).toEqual(['fresh']); + }); + + test('a batch rejected on a later element leaves the existing collection untouched', () => { + const builder = aBuilder().append(descriptor('original', 'PRE_LOGGING')); + + expect(() => + builder.reload([descriptor('ok', 'PRE_AUTH'), descriptor('bad', 'SEND')]), + ).toThrow(ReservedStageError); + + // PIPE-23: validation runs over the whole batch before `#buckets.clear()`, so a rejection that + // surfaces on the second element cannot leave the builder half-rebuilt. + expect(labelsOf(builder.build())).toEqual(['original']); + }); + + test('a batch repeating the SAME pillar type installs it once, not twice (PIPE-4, PIPE-6)', () => { + const retry = descriptor('retry', 'RETRY'); + const builder = aBuilder(); + + builder.reload([retry, retry]); + + // PIPE-4: a pillar admits at most one step. The incremental `append` path already treats a same-type + // re-install as an idempotent no-op (PIPE-6); a bulk reload must not be the back door that seats two. + expect(labelsOf(builder.build())).toEqual(['retry']); + }); +}); + +describe('PipelineBuilder reserved SEND stage (PIPE-8)', () => { + test('rejects any insertion targeting SEND', () => { + const sendShaped = descriptor('x', 'SEND'); + + expect(() => aBuilder().append(sendShaped)).toThrow(ReservedStageError); + expect(() => aBuilder().prepend(sendShaped)).toThrow(ReservedStageError); + expect(() => aBuilder().reload([sendShaped])).toThrow(ReservedStageError); + }); +}); + +describe('PipelineBuilder.build() flatten order (PIPE-1, PIPE-25)', () => { + test('flattens stages in declaration order regardless of append order', () => { + const preRedirect = descriptor('pre-redirect', 'PRE_REDIRECT'); + const postSerde = descriptor('post-serde', 'POST_SERDE'); + + const runtime = aBuilder().append(postSerde).append(preRedirect).build(); + + expect(labelsOf(runtime)).toEqual(['pre-redirect', 'post-serde']); + }); + + // PIPE-1/PIPE-2's conformance clause, in the one place that can express it: a built pipeline actually + // driven. Entry is the stage list top-down, exit is its exact reverse, with insertion order deliberately + // the reverse of declaration order so a flatten that leaked insertion order would fail loudly. + test('one probe step per stage enters in STAGE_ORDER and exits in its exact reverse', async () => { + const stages = STAGE_ORDER.filter(stage => stage !== 'SEND'); + const log: string[] = []; + const builder = aBuilder(); + for (const stage of [...stages].reverse()) { + builder.append({ + type: Symbol(stage), + stage, + fn: async (_request, ctx) => { + log.push(`enter:${stage}`); + const response = await ctx.next(); + log.push(`exit:${stage}`); + return response; + }, + }); + } + + await builder.build().send(aRequest('https://example.com')); + + expect(log).toEqual([ + ...stages.map(stage => `enter:${stage}`), + ...[...stages].reverse().map(stage => `exit:${stage}`), + ]); + }); +}); + +describe('PipelineBuilder edit-order independence (PIPE-22)', () => { + test('an edit sequence flattens the same as constructing the final set from scratch', () => { + const a = descriptor('a', 'PRE_LOGGING'); + const b = descriptor('b', 'PRE_LOGGING'); + const c = descriptor('c', 'POST_LOGGING'); + + const edited = new PipelineBuilder(new StubTransport(aResponse(200))) + .append(a) + .append(c) + .prepend(b) + .build(); + const fromScratch = new PipelineBuilder(new StubTransport(aResponse(200))) + .appendAll([b, a, c]) + .build(); + + expect(labelsOf(edited)).toEqual(labelsOf(fromScratch)); + expect(labelsOf(edited)).toEqual(['b', 'a', 'c']); + }); +}); + +// The two ordering laws the design calls for (PIPE-38's split across an append and a prepend test, one act +// each). `build()` is an invariant-bearing assembler, which +// docs/knowledge/harvested/testing.md:29 puts in property-test territory; the examples above pin concrete regressions, +// these prove the law over generated input. Generated over the non-pillar stages only: a generator that also +// emitted pillar stages would spend most of its cases hitting PIPE-5's collision instead of exercising order. +const editableStages = STAGE_ORDER.filter( + stage => stage !== 'SEND' && !PILLAR_STAGES.has(stage), +); + +describe('PipelineBuilder ordering properties (PIPE-22)', () => { + test('any append/prepend sequence flattens the same as building the final set from scratch (PIPE-22)', () => { + fc.assert( + fc.property( + fc.array( + fc.record({ + stage: fc.constantFrom(...editableStages), + where: fc.constantFrom('append' as const, 'prepend' as const), + }), + {maxLength: 24}, + ), + edits => { + const edited = aBuilder(); + const model = new Map<Stage, StepDescriptor[]>(); + for (const [index, edit] of edits.entries()) { + const step = descriptor(`s${String(index)}`, edit.stage); + const bucket = model.get(edit.stage) ?? []; + if (edit.where === 'append') { + bucket.push(step); + edited.append(step); + } else { + bucket.unshift(step); + edited.prepend(step); + } + model.set(edit.stage, bucket); + } + const finalSet = editableStages.flatMap( + stage => model.get(stage) ?? [], + ); + + const fromScratch = aBuilder().appendAll(finalSet).build(); + + expect(labelsOf(edited.build())).toEqual(labelsOf(fromScratch)); + }, + ), + ); + }); +}); + +describe('PipelineBuilder batch-order properties (PIPE-38)', () => { + test('appendAll preserves the batch order within a stage, for a batch of any size (PIPE-38)', () => { + fc.assert( + fc.property( + fc.integer({min: 1, max: 12}), + fc.constantFrom(...editableStages), + (size, stage) => { + const batch = Array.from({length: size}, (_unused, index) => + descriptor(`s${String(index)}`, stage), + ); + + const runtime = aBuilder().appendAll(batch).build(); + + expect(labelsOf(runtime)).toEqual( + batch.map(step => step.type.description), + ); + }, + ), + ); + }); + + test('prependAll reverses the batch order within a stage, for a batch of any size (PIPE-38)', () => { + fc.assert( + fc.property( + fc.integer({min: 1, max: 12}), + fc.constantFrom(...editableStages), + (size, stage) => { + const batch = Array.from({length: size}, (_unused, index) => + descriptor(`s${String(index)}`, stage), + ); + + const runtime = aBuilder().prependAll(batch).build(); + + expect(labelsOf(runtime)).toEqual( + batch.map(step => step.type.description).reverse(), + ); + }, + ), + ); + }); +}); + +// Module-scope, not describe-local: the seedFrom suite is split across sibling describes to stay +// inside `max-lines-per-function`, and both halves need these. +class RecordingTransport implements Transport { + readonly calls: Request[] = []; + + send(request: Request): Promise<Response> { + this.calls.push(request); + return Promise.resolve(aResponse(200)); + } + + close(): Promise<void> { + return Promise.resolve(); + } +} + +function probeStep( + label: string, + stage: StepDescriptor['stage'], + order: string[], +): StepDescriptor { + return { + type: Symbol(label), + stage, + // A plain pass-through probe never re-drives, so `next()` suffices -- no fork needed. + fn: async (request, ctx) => { + order.push(label); + return ctx.next(request); + }, + }; +} + +describe('PipelineBuilder.seedFrom (PIPE-35)', () => { + test('flatten: seeded steps run in the SAME pass as newly appended ones, reusing the original transport', async () => { + const transport = new RecordingTransport(); + const order: string[] = []; + const seeded = new PipelineBuilder(transport) + .append(probeStep('seeded', 'LOGGING', order)) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'flatten') + .append(probeStep('appended', 'SERDE', order)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(order).toEqual(['seeded', 'appended']); // one combined STAGE_ORDER pass + // The ORIGINAL transport is the terminal -- `seeded` itself is not in the chain. + expect(transport.calls).toHaveLength(1); + }); + + test('flatten: re-buckets each descriptor by its OWN stage, not by seeded array position', () => { + const transport = new RecordingTransport(); + const order: string[] = []; + const seeded = new PipelineBuilder(transport) + .append(probeStep('late', 'SERDE', order)) + .append(probeStep('early', 'PRE_REDIRECT', order)) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'flatten').build(); + + expect(labelsOf(runtime)).toEqual(['early', 'late']); + }); + + test('flatten: pillar-collision rules apply exactly as any other append sequence', () => { + const transport = new RecordingTransport(); + const seeded = new PipelineBuilder(transport) + .append(descriptor('retry-a', 'RETRY')) + .build(); + + expect(() => + PipelineBuilder.seedFrom(seeded, 'flatten').append( + descriptor('retry-b', 'RETRY'), + ), + ).toThrow(PillarCollisionError); + }); +}); + +describe('PipelineBuilder.seedFrom nest mode (PIPE-35)', () => { + test('nest: the seeded runtime is an opaque Transport -- its steps run in a separate, inner pass', async () => { + const transport = new RecordingTransport(); + const order: string[] = []; + const seeded = new PipelineBuilder(transport) + .append(probeStep('inner', 'LOGGING', order)) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'nest') + .append(probeStep('outer', 'LOGGING', order)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(order).toEqual(['outer', 'inner']); // the outer step runs BEFORE the nested runtime's + expect(transport.calls).toHaveLength(1); // still exactly one wire send at the bottom + }); + + test('nest: the same pillar may be occupied in BOTH layers -- they are separate builders', () => { + const transport = new RecordingTransport(); + const seeded = new PipelineBuilder(transport) + .append(descriptor('retry-inner', 'RETRY')) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'nest') + .append(descriptor('retry-outer', 'RETRY')) + .build(); + + expect(labelsOf(runtime)).toEqual(['retry-outer']); + expect(runtime.transport).toBe(seeded); + }); +}); + +/** Records the name of every span it is asked to open, so a test can count operations. */ +function countingTracer(): {tracer: Tracer; names: string[]} { + const names: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + return { + names, + tracer: { + startSpan(name: string): Span { + names.push(name); + return span; + }, + }, + }; +} + +/** Captures what the drive's `RequestContext` says, from inside the pipeline. */ +function contextProbe(seen: { + operationName?: string | undefined; +}): StepDescriptor { + return { + type: Symbol('context-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen.operationName = + 'operationName' in ctx.context ? ctx.context.operationName : undefined; + return ctx.next(request); + }, + }; +} + +describe('PipelineBuilder instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle is what opens the per-operation span', async () => { + const {tracer, names} = countingTracer(); + const runtime = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append(descriptor('probe', 'PRE_SERDE')) + .build(); + + await runtime.send(aRequest('https://example.com')); + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation', 'http.client.operation']); + }); + + test('operationName reaches the request context every step reads (CTX-16)', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport(), { + operationName: 'GetUser', + }) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBe('GetUser'); + }); + + test('no options bag means the no-op bundle and no operation name', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport()) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBeUndefined(); + }); + + test('flatten seeding carries the seed runtime’s options (PIPE-35)', async () => { + const {tracer, names} = countingTracer(); + const seen: {operationName?: string | undefined} = {}; + const seeded = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + operationName: 'GetUser', + }) + .append(descriptor('seeded', 'LOGGING')) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'flatten') + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation']); + expect(seen.operationName).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/pipeline/builder.ts b/packages/core/src/pipeline/builder.ts new file mode 100644 index 0000000..83b383e --- /dev/null +++ b/packages/core/src/pipeline/builder.ts @@ -0,0 +1,369 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/builder.ts +import type {InstrumentationBundle} from '../context/instrumentation.js'; +import {invariant} from '../invariant.js'; +import type {Transport} from '../seams/transport.js'; +import { + AnchorNotFoundError, + CrossStageEditError, + PillarCollisionError, + ReservedStageError, +} from './errors.js'; +import {createRuntime, pipelineOptionsOf, type Runtime} from './runtime.js'; +import {PILLAR_STAGES, STAGE_ORDER, type Stage} from './stage.js'; +import type {StepDescriptor} from './step.js'; + +interface AnchorLocation { + readonly stage: Stage; + readonly index: number; +} + +/** + * What a pipeline carries into every call it drives, as opposed to what a single call carries + * (`RequestOptions`) or what one step is configured with (a pillar's own settings). + * + * Both fields are per-pipeline by construction: `CTX-4` gives every `send()` its own context key, and + * the bundle is shared by reference across that call's three promotions (`CTX-2`/`CTX-3`), so a client + * that needs two operation names builds two pipelines — cheap, since `seedFrom(runtime, 'flatten')` + * derives the second from the first and carries these options with it. + * + * @public + */ +export interface PipelineOptions { + /** + * The correlation bundle every context of every call carries (`CTX-14`), and — through its + * `tracerFactory` — the source of the one `http.client.operation` span `Runtime.send()` opens per + * logical operation (`OBS-29`). Build one with `createInstrumentationBundle(tracerFactory)`. + * + * @defaultValue the disabled-tracing no-op bundle (`CTX-15`), which opens no span at all + */ + readonly instrumentation?: InstrumentationBundle | undefined; + /** + * The advisory operation label (`CTX-16`) — a schema-defined operation id such as `'GetUser'`. + * Carried unchanged from the request context through every promotion, exposed to the tracing seam + * (the LOGGING pillar step names its per-attempt span with it), and never an input to the request, + * the dispatch decision or the store key. + * + * @defaultValue `undefined` — a raw request that belongs to no named operation + */ + readonly operationName?: string | undefined; +} + +/** + * Assembles a stage-based pipeline via surgical edits (PIPE-7, PIPE-18..PIPE-24), flattening into an + * immutable Runtime at build() time (PIPE-25). Mutable while being built; the produced Runtime is frozen. + * + * @public + */ +export class PipelineBuilder { + readonly #buckets = new Map<Stage, StepDescriptor[]>(); + readonly #transport: Transport; + readonly #options: PipelineOptions; + + /** + * @param transport - the terminal transport the built pipeline dispatches to. Never closed by the + * pipeline (PIPE-27). + * @param options - what the built pipeline carries into every call: the instrumentation bundle and + * the advisory operation name. Optional, and optional in the second position deliberately — this + * is the only public way to reach `OBS-29`'s per-operation span and `CTX-16`'s operation name, + * and adding it must not break the one-argument construction every existing caller writes. + */ + constructor(transport: Transport, options: PipelineOptions = {}) { + this.#transport = transport; + this.#options = options; + } + + /** + * Seats `descriptor` at the tail of its own stage bucket (PIPE-7). + * + * @throws ReservedStageError when the descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws PillarCollisionError when its pillar stage already holds a step of a different type + * (PIPE-5); re-seating the same `type` symbol is an idempotent no-op instead (PIPE-6). + */ + append(descriptor: StepDescriptor): this { + this.#rejectReservedStage(descriptor.stage, 'append'); + if (this.#pillarSlot(descriptor) === 'occupied-same-type') return this; + this.#insertAt(descriptor.stage, descriptor, 'tail'); + return this; + } + + /** + * Seats `descriptor` at the head of its own stage bucket (PIPE-7). + * + * @throws ReservedStageError when the descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws PillarCollisionError when its pillar stage already holds a step of a different type + * (PIPE-5); re-seating the same `type` symbol is an idempotent no-op instead (PIPE-6). + */ + prepend(descriptor: StepDescriptor): this { + this.#rejectReservedStage(descriptor.stage, 'prepend'); + if (this.#pillarSlot(descriptor) === 'occupied-same-type') return this; + this.#insertAt(descriptor.stage, descriptor, 'head'); + return this; + } + + /** + * PIPE-38: batch iteration order preserved within a stage. + * + * @throws ReservedStageError as {@link PipelineBuilder.append}, on the first offending descriptor. + * @throws PillarCollisionError as {@link PipelineBuilder.append}. Not all-or-nothing: descriptors + * before the offending one are already seated — `reload` is the transactional bulk path (PIPE-23). + */ + appendAll(descriptors: readonly StepDescriptor[]): this { + for (const descriptor of descriptors) this.append(descriptor); + return this; + } + + /** + * PIPE-38: each element prepended individually -- the batch order comes out reversed, by + * construction. This asymmetry with {@link PipelineBuilder.appendAll} is the documented one PIPE-38 + * requires a port to state. + * + * @throws ReservedStageError as {@link PipelineBuilder.prepend}, on the first offending descriptor. + * @throws PillarCollisionError as {@link PipelineBuilder.prepend}. Not all-or-nothing, as above. + */ + prependAll(descriptors: readonly StepDescriptor[]): this { + for (const descriptor of descriptors) this.prepend(descriptor); + return this; + } + + /** + * Seats `descriptor` immediately after the first existing instance of `anchorType` (PIPE-18). + * + * @throws ReservedStageError when the descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws AnchorNotFoundError when no step of `anchorType` is present (PIPE-21). + * @throws CrossStageEditError when the descriptor's stage differs from the anchor's (PIPE-18). + * @throws PillarCollisionError when the anchor's pillar stage already holds a different type (PIPE-5). + */ + insertAfter(anchorType: symbol, descriptor: StepDescriptor): this { + this.#rejectReservedStage(descriptor.stage, 'insertAfter'); + const anchor = this.#requireAnchor(anchorType, 'insertAfter'); + this.#requireSameStage(anchor.stage, descriptor.stage); + if (this.#pillarSlot(descriptor) === 'occupied-same-type') return this; + const bucket = this.#requireAnchorBucket(anchor); + bucket.splice(anchor.index + 1, 0, descriptor); + return this; + } + + /** + * Seats `descriptor` immediately before the first existing instance of `anchorType` (PIPE-18). + * + * @throws ReservedStageError when the descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws AnchorNotFoundError when no step of `anchorType` is present (PIPE-21). + * @throws CrossStageEditError when the descriptor's stage differs from the anchor's (PIPE-18). + * @throws PillarCollisionError when the anchor's pillar stage already holds a different type (PIPE-5). + */ + insertBefore(anchorType: symbol, descriptor: StepDescriptor): this { + this.#rejectReservedStage(descriptor.stage, 'insertBefore'); + const anchor = this.#requireAnchor(anchorType, 'insertBefore'); + this.#requireSameStage(anchor.stage, descriptor.stage); + if (this.#pillarSlot(descriptor) === 'occupied-same-type') return this; + const bucket = this.#requireAnchorBucket(anchor); + bucket.splice(anchor.index, 0, descriptor); + return this; + } + + /** + * Swaps the first existing instance of `anchorType` for `descriptor`, in place (PIPE-19). The + * sanctioned way past a pillar collision: PIPE-5 exempts `replace` from the pillar check, since it + * swaps one occupant 1:1 within its own stage and the incoming type is distinct by definition. + * + * @throws ReservedStageError when the descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws AnchorNotFoundError when no step of `anchorType` is present (PIPE-21). + * @throws CrossStageEditError when the descriptor's stage differs from the anchor's (PIPE-19). + */ + replace(anchorType: symbol, descriptor: StepDescriptor): this { + this.#rejectReservedStage(descriptor.stage, 'replace'); + const anchor = this.#requireAnchor(anchorType, 'replace'); + this.#requireSameStage(anchor.stage, descriptor.stage); + const bucket = this.#requireAnchorBucket(anchor); + bucket.splice(anchor.index, 1, descriptor); + return this; + } + + /** + * PIPE-20: deletes every instance of `type`, preserving relative order; a no-op when absent. A stage + * left with no steps keeps an empty bucket, which flattening and the pillar check both read as absent. + */ + remove(type: symbol): this { + for (const [stage, bucket] of this.#buckets) { + const filtered = bucket.filter(entry => entry.type !== type); + if (filtered.length !== bucket.length) this.#buckets.set(stage, filtered); + } + return this; + } + + /** + * PIPE-23: all-or-nothing -- validated fully before any existing content is touched, so a rejected + * batch leaves the builder exactly as it was. + * + * @throws ReservedStageError when any descriptor declares the terminal `SEND` stage (PIPE-8). + * @throws PillarCollisionError when two descriptors of different types claim one pillar stage + * (PIPE-5); a repeat of the same `type` on one pillar is dropped instead, so the bulk path cannot + * seat two steps where `append` would seat one (PIPE-4/PIPE-6). + */ + reload(descriptors: readonly StepDescriptor[]): this { + const admitted: StepDescriptor[] = []; + const pillarTypes = new Map<Stage, symbol>(); + for (const descriptor of descriptors) { + this.#rejectReservedStage(descriptor.stage, 'reload'); + if (!PILLAR_STAGES.has(descriptor.stage)) { + admitted.push(descriptor); + continue; + } + const seenType = pillarTypes.get(descriptor.stage); + // PIPE-6: a repeat of the SAME type is idempotent, not a second step. + if (seenType === descriptor.type) continue; + if (seenType !== undefined) { + throw new PillarCollisionError( + descriptor.stage, + seenType, + descriptor.type, + ); // PIPE-5 + } + pillarTypes.set(descriptor.stage, descriptor.type); + admitted.push(descriptor); + } + // PIPE-4: `admitted` holds at most one entry per pillar stage by construction -- a same-type repeat was + // skipped above rather than pushed, so a batch cannot install two steps onto one pillar the way the + // incremental `append` path already refuses to. + this.#buckets.clear(); + for (const descriptor of admitted) { + const bucket = this.#buckets.get(descriptor.stage); + if (bucket === undefined) + this.#buckets.set(descriptor.stage, [descriptor]); + else bucket.push(descriptor); + } + return this; + } + + /** + * PIPE-35: derives a builder from an already-built `runtime`, under an explicit, non-defaulted + * `mode` — the requirement's own MUST is that the flatten-vs-nest choice be explicit, never + * accidental, so there is deliberately no default value. + * + * `flatten` re-buckets every seeded descriptor by its own stage and reuses `runtime`'s transport as + * the new builder's terminal, so seeded and newly-appended steps run in the SAME cursor pass. + * Pillar-collision rules apply exactly as they would to any other `append` sequence, because + * flatten IS an append sequence. + * + * Seeding re-seats the SAME descriptor objects, never copies: a `StepDescriptor` is a plain record + * around a closure, so any state that closure captured is now shared between `runtime` and the + * builder derived from it. `authStep`'s `BearerTokenCache` is the live example — a flattened + * builder shares one token cache, and therefore one single-flight slot, with the runtime it was + * seeded from. That is usually what a caller wants (AUTH-34's coalescing only works when concurrent + * calls meet at one instance), but it is sharing, not isolation; a caller who needs an independent + * cache constructs a fresh `authStep`. + * + * `nest` constructs a fresh builder whose transport IS `runtime`, treated as an opaque `Transport` + * — `Runtime implements Transport` (PIPE-26) makes this work with zero adapter code — so the new + * builder's own steps run once, outside `runtime`'s already-flattened loops. + * + * @param runtime - the built pipeline to seed from. + * @param mode - `'flatten'` to merge its steps into this builder's stages, `'nest'` to wrap it as + * this builder's transport. + * @returns a fresh builder seeded per `mode`. + * @throws PillarCollisionError in `'flatten'` mode when two seeded descriptors of different types + * claim one pillar stage (PIPE-5) — the same rule `append` enforces. + */ + static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder { + if (mode === 'flatten') { + // Flatten produces the pipeline that REPLACES `runtime`, so it inherits `runtime`'s + // `PipelineOptions` along with its steps: dropping them would silently un-trace a client that + // seeded from a traced preset, and there is no other way for the derived builder to recover + // them. `nest` needs no such carry -- `runtime` is still there, as the terminal transport, + // driving its own contexts with its own bundle. + return new PipelineBuilder( + runtime.transport, + pipelineOptionsOf(runtime), + ).appendAll(runtime.steps); + } + return new PipelineBuilder(runtime); + } + + /** PIPE-25: flattens stage buckets in declaration order, skipping SEND, into an immutable Runtime. */ + build(): Runtime { + const flattened: StepDescriptor[] = []; + for (const stage of STAGE_ORDER) { + if (stage === 'SEND') continue; // PIPE-8: terminal, reserved, flattening skips it. + const bucket = this.#buckets.get(stage); + if (bucket !== undefined) flattened.push(...bucket); + } + // Runtime copies and freezes -- PIPE-10/PIPE-25. `PipelineOptions` is structurally the public + // half of `ContextInit`; the `key` half stays in-package, because CTX-4 wants one key per call. + return createRuntime(flattened, this.#transport, this.#options); + } + + #rejectReservedStage(stage: Stage, operation: string): void { + if (stage === 'SEND') throw new ReservedStageError(operation); // PIPE-8 + } + + /** + * PIPE-4/5/6: `'ok'` when `descriptor` may be seated, `'occupied-same-type'` when its pillar already + * holds that exact `type` and the edit is an idempotent no-op. A bucket emptied by `remove` counts as + * unoccupied. + * + * @throws PillarCollisionError when the pillar holds a step of a different type (PIPE-5). + */ + #pillarSlot(descriptor: StepDescriptor): 'ok' | 'occupied-same-type' { + const {stage, type} = descriptor; + if (!PILLAR_STAGES.has(stage)) return 'ok'; + const bucket = this.#buckets.get(stage); + if (bucket === undefined || bucket.length === 0) return 'ok'; + const occupant = bucket[0]; + invariant( + occupant !== undefined, + 'pillar bucket has non-zero length but its first element is undefined', + ); + if (occupant.type === type) return 'occupied-same-type'; // PIPE-6: idempotent re-installation. + throw new PillarCollisionError(stage, occupant.type, type); // PIPE-5 + } + + #insertAt( + stage: Stage, + descriptor: StepDescriptor, + where: 'head' | 'tail', + ): void { + const bucket = this.#buckets.get(stage); + if (bucket === undefined) { + this.#buckets.set(stage, [descriptor]); + return; + } + if (where === 'tail') bucket.push(descriptor); + else bucket.unshift(descriptor); + } + + /** + * PIPE-18's "first existing instance", resolved in flattened order -- `STAGE_ORDER` first, then + * position within the stage bucket. A type installed in more than one stage therefore anchors on its + * earliest-staged instance, and an edit declaring one of the later stages is a cross-stage edit even + * though an instance does sit in that stage. + */ + /** + * The bucket `anchor` was found in. `#requireAnchor` has already located an entry there, so the + * absence of the bucket would be an internal inconsistency rather than a caller error — which is + * what the invariant says, once, instead of three times. + */ + #requireAnchorBucket(anchor: {stage: Stage}): StepDescriptor[] { + const bucket = this.#buckets.get(anchor.stage); + invariant( + bucket !== undefined, + 'anchor stage bucket must exist -- #requireAnchor just located an entry in it', + ); + return bucket; + } + + #requireAnchor(type: symbol, operation: string): AnchorLocation { + for (const stage of STAGE_ORDER) { + const bucket = this.#buckets.get(stage); + if (bucket === undefined) continue; + const index = bucket.findIndex(entry => entry.type === type); + if (index !== -1) return {stage, index}; + } + throw new AnchorNotFoundError(type, operation); // PIPE-21 + } + + #requireSameStage(anchorStage: Stage, incomingStage: Stage): void { + if (anchorStage !== incomingStage) + throw new CrossStageEditError(anchorStage, incomingStage); // PIPE-18/19 + } +} diff --git a/packages/core/src/pipeline/cursor.test.ts b/packages/core/src/pipeline/cursor.test.ts new file mode 100644 index 0000000..1f0433c --- /dev/null +++ b/packages/core/src/pipeline/cursor.test.ts @@ -0,0 +1,600 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/cursor.test.ts +// Exercises: PIPE-9 (Cursor-level: an exhausted position dispatches to the terminal transport), PIPE-11/15 +// (a reused next()/fork() continuation throws CursorAlreadyAdvancedError), PIPE-12 (ctx.context, ctx.fork +// gated by pillar stage, short-circuiting without invoking the chain, substituting the outbound response), +// PIPE-13 (terminal dispatch threads request/options/signal), PIPE-14 (a substituted request sticks +// downstream, across every later fork, and into the terminal dispatch), PIPE-15/16 (fork() returns +// independent, position-pinned one-shot continuations; a step that forks twice re-visits every downstream +// step both times), PIPE-17 (the caller's options are carried unchanged across every fork and into each +// dispatch) +import {describe, expect, test} from 'bun:test'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {RequestOptions} from '../http/request-options.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {invariant} from '../invariant.js'; +import type {Transport} from '../seams/transport.js'; +import {Cursor} from './cursor.js'; +import {CursorAlreadyAdvancedError} from './errors.js'; +import type {Next, Step, StepDescriptor} from './step.js'; + +function aRequest(url: string): Request { + return Request.newBuilder().url(url).build(); +} + +function aResponse(status: number): Response { + return Response.newBuilder() + .request(aRequest('https://example.com')) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .build(); +} + +class RecordingTransport implements Transport { + readonly calls: { + request: Request; + options: RequestOptions | undefined; + signal: AbortSignal | undefined; + }[] = []; + #response: Response; + + constructor(response: Response) { + this.#response = response; + } + + send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + this.calls.push({request, options, signal}); + return Promise.resolve(this.#response); + } + + close(): Promise<void> { + return Promise.resolve(); + } +} + +function passthroughStep(log: string[], label: string): Step { + return async (_request, ctx) => { + log.push(label); + return ctx.next(); + }; +} + +/** A non-pillar descriptor around `fn`, for tests that care about the walk rather than the stage. */ +function descriptorOf(fn: Step): StepDescriptor { + return {type: Symbol('probe'), stage: 'PRE_LOGGING', fn}; +} + +describe('Cursor terminal dispatch (PIPE-9, PIPE-13)', () => { + test('an exhausted cursor dispatches to the terminal transport, threading options and signal', async () => { + const canned = aResponse(200); + const transport = new RecordingTransport(canned); + const request = aRequest('https://example.com/a'); + const signal = new AbortController().signal; + const context = createRequestContext(request); + + const cursor = new Cursor({steps: [], transport, request, context, signal}); + const response = await cursor.advance(); + + expect(response).toBe(canned); + expect(transport.calls).toHaveLength(1); + expect(transport.calls[0]?.request).toBe(request); + expect(transport.calls[0]?.signal).toBe(signal); + }); +}); + +describe('Cursor honours an aborted signal before each step (T.F9/V15)', () => { + test('an already-aborted call runs NO step and never reaches the transport', async () => { + const {CancellationError} = await import('../seams/transport.js'); + const transport = new RecordingTransport(aResponse(200)); + const request = aRequest('https://example.com/a'); + const controller = new AbortController(); + const reason = new Error('caller went away'); + controller.abort(reason); + const log: string[] = []; + + const cursor = new Cursor({ + steps: [descriptorOf(passthroughStep(log, 'first'))], + transport, + request, + context: createRequestContext(request), + signal: controller.signal, + }); + + const surfaced = await cursor.advance().then( + () => undefined, + (error: unknown) => error, + ); + + // N1's mapper, not a bare DOMException: one cancellation type wherever it was observed. + expect(surfaced).toBeInstanceOf(CancellationError); + expect((surfaced as Error).cause).toBe(reason); + expect(log).toEqual([]); + expect(transport.calls).toHaveLength(0); + }); + + test('an abort raised BETWEEN steps stops the walk at the next step boundary', async () => { + const {CancellationError} = await import('../seams/transport.js'); + const transport = new RecordingTransport(aResponse(200)); + const request = aRequest('https://example.com/a'); + const controller = new AbortController(); + const log: string[] = []; + + const abortingStep: Step = async (_request, ctx) => { + log.push('first'); + controller.abort(new Error('gave up mid-walk')); + return ctx.next(); + }; + + const cursor = new Cursor({ + steps: [ + descriptorOf(abortingStep), + descriptorOf(passthroughStep(log, 'second')), + ], + transport, + request, + context: createRequestContext(request), + signal: controller.signal, + }); + + const surfaced = await cursor.advance().then( + () => undefined, + (error: unknown) => error, + ); + + expect(surfaced).toBeInstanceOf(CancellationError); + expect(log).toEqual(['first']); + expect(transport.calls).toHaveLength(0); + }); +}); + +describe('Cursor with a live (un-aborted) signal', () => { + test('an un-aborted signal changes nothing', async () => { + const transport = new RecordingTransport(aResponse(200)); + const request = aRequest('https://example.com/a'); + const log: string[] = []; + + const cursor = new Cursor({ + steps: [descriptorOf(passthroughStep(log, 'first'))], + transport, + request, + context: createRequestContext(request), + signal: new AbortController().signal, + }); + + expect((await cursor.advance()).status.code).toBe(200); + expect(log).toEqual(['first']); + expect(transport.calls).toHaveLength(1); + }); +}); + +describe('Cursor step invocation (PIPE-12)', () => { + test('ctx.context is the exact reference passed to the constructor, visible to every step', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + const seen: ExecutionContext[] = []; + const step: Step = async (_request, ctx) => { + seen.push(ctx.context); + return ctx.next(); + }; + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: step, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + expect(seen[0]).toBe(context); + }); +}); + +describe('Cursor fork availability (PIPE-12, PIPE-15)', () => { + test('ctx.fork is undefined for a non-pillar-stage step', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + const seenFork: ((() => Next) | undefined)[] = []; + const step: Step = async (_request, ctx) => { + seenFork.push(ctx.fork); + return ctx.next(); + }; + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: step, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + expect(seenFork[0]).toBeUndefined(); + }); + + test('ctx.fork is present for a pillar-stage step', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let sawFork: (() => Next) | undefined; + const step: Step = async (_request, ctx) => { + sawFork = ctx.fork; + invariant(sawFork !== undefined, 'pillar step must receive a fork'); + return sawFork()(); + }; + const descriptor: StepDescriptor = { + type: Symbol('retry'), + stage: 'RETRY', + fn: step, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + expect(sawFork).toBeDefined(); + }); +}); + +describe('Cursor bidirectionality (PIPE-12)', () => { + test('a step that short-circuits never reaches the terminal transport', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + const synthetic = aResponse(204); + const transport = new RecordingTransport(aResponse(200)); + const shortCircuit: Step = () => Promise.resolve(synthetic); + const descriptor: StepDescriptor = { + type: Symbol('short-circuit'), + stage: 'PRE_LOGGING', + fn: shortCircuit, + }; + + const response = await new Cursor({ + steps: [descriptor], + transport, + request, + context, + }).advance(); + + expect(response).toBe(synthetic); + expect(transport.calls).toHaveLength(0); + }); + + test('a step may substitute the outbound response on the way back out', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + const fromTransport = aResponse(200); + const substituted = aResponse(203); + const transport = new RecordingTransport(fromTransport); + let sawFromTransport: Response | undefined; + const substituteResponse: Step = async (_request, ctx) => { + sawFromTransport = await ctx.next(); + return substituted; + }; + const descriptor: StepDescriptor = { + type: Symbol('substitute-response'), + stage: 'PRE_LOGGING', + fn: substituteResponse, + }; + + const response = await new Cursor({ + steps: [descriptor], + transport, + request, + context, + }).advance(); + + expect(sawFromTransport).toBe(fromTransport); + expect(response).toBe(substituted); + }); +}); + +describe('Cursor continuation reuse (PIPE-11, PIPE-15)', () => { + test('a second call to the same next() throws CursorAlreadyAdvancedError', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let capturedNext: Next | undefined; + const step: Step = async (_request, ctx) => { + capturedNext = ctx.next; + return ctx.next(); + }; + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: step, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + invariant( + capturedNext !== undefined, + 'the step must have run and captured its next()', + ); + const rejection: unknown = await capturedNext().catch( + (error: unknown) => error, + ); + expect(rejection).toBeInstanceOf(CursorAlreadyAdvancedError); + }); + + test('a second call to the same fork()-returned continuation throws CursorAlreadyAdvancedError', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let capturedContinuation: Next | undefined; + const step: Step = async (_request, ctx) => { + invariant(ctx.fork !== undefined, 'pillar step must receive a fork'); + capturedContinuation = ctx.fork(); + return capturedContinuation(); + }; + const descriptor: StepDescriptor = { + type: Symbol('retry'), + stage: 'RETRY', + fn: step, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + invariant( + capturedContinuation !== undefined, + 'the step must have run and captured its fork() continuation', + ); + const rejection: unknown = await capturedContinuation().catch( + (error: unknown) => error, + ); + expect(rejection).toBeInstanceOf(CursorAlreadyAdvancedError); + }); +}); + +describe('Cursor request substitution (PIPE-14)', () => { + test('a substituted request propagates downstream and to the terminal dispatch', async () => { + const original = aRequest('https://example.com/a'); + const substituted = aRequest('https://example.com/b'); + const context = createRequestContext(original); + const seenByDownstream: Request[] = []; + const substituteStep: Step = async (_request, ctx) => ctx.next(substituted); + const downstreamStep: Step = async (request, ctx) => { + seenByDownstream.push(request); + return ctx.next(); + }; + const transport = new RecordingTransport(aResponse(200)); + const steps: StepDescriptor[] = [ + {type: Symbol('substitute'), stage: 'PRE_LOGGING', fn: substituteStep}, + {type: Symbol('downstream'), stage: 'POST_LOGGING', fn: downstreamStep}, + ]; + + await new Cursor({steps, transport, request: original, context}).advance(); + + expect(seenByDownstream[0]).toBe(substituted); + expect(transport.calls[0]?.request).toBe(substituted); + }); +}); + +describe('Cursor request substitution across forks (PIPE-14, PIPE-16)', () => { + test('a substitution made inside one fork is what the next fork dispatches', async () => { + const original = aRequest('https://example.com/original'); + const substituted = aRequest('https://example.com/substituted'); + const context = createRequestContext(original); + const transport = new RecordingTransport(aResponse(200)); + const reDriving: Step = async (_request, ctx) => { + invariant(ctx.fork !== undefined, 'a pillar step must receive a fork'); + await ctx.fork()(substituted); + return ctx.fork()(); + }; + const descriptor: StepDescriptor = { + type: Symbol('retry'), + stage: 'RETRY', + fn: reDriving, + }; + + await new Cursor({ + steps: [descriptor], + transport, + request: original, + context, + }).advance(); + + // PIPE-14's stickiness is global to the call, not scoped to the fork that substituted: PIPE-16's + // "forks advance independently" is about cursor position, not about request isolation. + expect(transport.calls.map(call => call.request)).toEqual([ + substituted, + substituted, + ]); + }); +}); + +describe('Cursor fork (PIPE-15, PIPE-16, PIPE-17)', () => { + test('a step forking twice re-visits every downstream step on both attempts', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + const options = RequestOptions.EMPTY; + const log: string[] = []; + const retryStep: Step = async (_request, ctx) => { + invariant(ctx.fork !== undefined, 'retryStep must occupy a pillar stage'); + log.push('retry:attempt-1'); + await ctx.fork()(); + log.push('retry:attempt-2'); + return ctx.fork()(); + }; + const steps: StepDescriptor[] = [ + {type: Symbol('retry'), stage: 'RETRY', fn: retryStep}, + { + type: Symbol('downstream'), + stage: 'POST_RETRY', + fn: passthroughStep(log, 'downstream'), + }, + ]; + const transport = new RecordingTransport(aResponse(200)); + + await new Cursor({steps, transport, request, context, options}).advance(); + + expect(log).toEqual([ + 'retry:attempt-1', + 'downstream', + 'retry:attempt-2', + 'downstream', + ]); + expect(transport.calls).toHaveLength(2); + // PIPE-17: the caller's per-call options are carried unchanged across every re-drive fork and threaded + // into each terminal dispatch -- shared by reference, never copied-and-diverged per fork. + expect(transport.calls.map(call => call.options)).toEqual([ + options, + options, + ]); + }); +}); + +describe('StepContext.signal', () => { + test('a step observes the signal the cursor was constructed with', async () => { + const controller = new AbortController(); + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let observed: AbortSignal | undefined; + const descriptor: StepDescriptor = { + type: Symbol('observer'), + stage: 'LOGGING', + fn: async (_request, ctx) => { + observed = ctx.signal; + return ctx.next(); + }, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + signal: controller.signal, + }).advance(); + + expect(observed).toBe(controller.signal); + }); + + test('signal is undefined when the cursor was constructed without one', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let observed: AbortSignal | undefined = new AbortController().signal; + const descriptor: StepDescriptor = { + type: Symbol('observer'), + stage: 'LOGGING', + fn: async (_request, ctx) => { + observed = ctx.signal; + return ctx.next(); + }, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + expect(observed).toBeUndefined(); + }); +}); + +describe('StepContext.signal on a pillar step', () => { + test('a pillar step observes the signal too', async () => { + const controller = new AbortController(); + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let observed: AbortSignal | undefined; + const descriptor: StepDescriptor = { + type: Symbol('observer'), + stage: 'RETRY', + fn: async (_request, ctx) => { + observed = ctx.signal; + return ctx.next(); + }, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + signal: controller.signal, + }).advance(); + + expect(observed).toBe(controller.signal); + }); +}); + +describe('StepContext.options (PIPE-17)', () => { + test('a step reads the per-call options the cursor was constructed with', async () => { + const options = RequestOptions.newBuilder().maxRetries(0).build(); + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let observed: RequestOptions | undefined; + const descriptor: StepDescriptor = { + type: Symbol('observer'), + stage: 'LOGGING', + fn: async (_request, ctx) => { + observed = ctx.options; + return ctx.next(); + }, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + options, + }).advance(); + + // PIPE-17: the same immutable instance, not a copy. + expect(observed).toBe(options); + }); + + test('options is undefined when the caller supplied none', async () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request); + let observed: RequestOptions | undefined = + RequestOptions.newBuilder().build(); + const descriptor: StepDescriptor = { + type: Symbol('observer'), + stage: 'LOGGING', + fn: async (_request, ctx) => { + observed = ctx.options; + return ctx.next(); + }, + }; + + await new Cursor({ + steps: [descriptor], + transport: new RecordingTransport(aResponse(200)), + request, + context, + }).advance(); + + expect(observed).toBeUndefined(); + }); +}); diff --git a/packages/core/src/pipeline/cursor.ts b/packages/core/src/pipeline/cursor.ts new file mode 100644 index 0000000..335a85b --- /dev/null +++ b/packages/core/src/pipeline/cursor.ts @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/cursor.ts +import {abortToSdkError} from '../cancellation.js'; +import type {ExecutionContext} from '../context/context.js'; +import type {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import {invariant} from '../invariant.js'; +import type {Transport} from '../seams/transport.js'; +import {CursorAlreadyAdvancedError} from './errors.js'; +import {PILLAR_STAGES, type Stage} from './stage.js'; +import type {Next, StepContext, StepDescriptor} from './step.js'; + +/** + * Everything a `Cursor` needs, bundled into one object. Six positional parameters would fail ESLint's + * `max-params: 3`, and Phase 1 reserves the `eslint-disable` escape hatch for private builder-internal + * constructors only -- the same trap 4a's `ContextInit` and 4b's `DispatchConfig` were built to dodge. + * + * @internal + */ +export interface CursorInit { + readonly steps: readonly StepDescriptor[]; + readonly transport: Transport; + readonly request: Request; + readonly context: ExecutionContext; + readonly options?: RequestOptions | undefined; + readonly signal?: AbortSignal | undefined; +} + +/** + * Drives one call through the flattened step array (PIPE-9..PIPE-17). One instance per `Runtime.send()` + * call (PIPE-10); `advance()` is its single public entry point. Internally a private recursive dispatcher + * indexed by array position -- `next` and every `fork()` call are one-shot closures built over the same + * dispatcher (PIPE-15/16), sharing a single mutable in-flight request so a substitution sticks globally for + * the rest of the call (PIPE-14). + * + * There is deliberately no settable start position: a fork produces a fresh one-shot closure over the + * existing dispatcher, never a second `Cursor`, so every instance starts at position 0. + * + * @internal + */ +export class Cursor { + readonly #steps: readonly StepDescriptor[]; + readonly #transport: Transport; + #request: Request; + readonly #options: RequestOptions | undefined; + readonly #signal: AbortSignal | undefined; + readonly #context: ExecutionContext; + + constructor(init: CursorInit) { + this.#steps = init.steps; + this.#transport = init.transport; + this.#request = init.request; + this.#options = init.options; + this.#signal = init.signal; + this.#context = init.context; + } + + /** + * The in-flight request as of now: the one passed in, or whatever a step last substituted (PIPE-14). + * `Runtime` reads this after the drive so the exchange context describes the request actually sent. + */ + get request(): Request { + return this.#request; + } + + /** + * Drives the call from position 0 through every step and on to the terminal transport dispatch. + * Called exactly once per cursor -- `Runtime.send()` allocates a fresh cursor per call (PIPE-10). + * + * Every step boundary is a cancellation checkpoint: an aborted `signal` stops the walk before the + * next step runs, so a pre-aborted call does no work at all. + * + * @returns the response the outermost step returned, which may be a synthetic one it short-circuited + * with, a substituted one, or the terminal transport's own (PIPE-12). + * @throws CursorAlreadyAdvancedError when a step reuses an already-invoked continuation (PIPE-15). + * @throws CancellationError when the caller's signal has aborted, carrying the caller's own abort + * reason as `cause` — or `TransportFailureError` when the abort was a timeout (XCUT-3). + */ + async advance(): Promise<Response> { + return this.#dispatch(0); + } + + async #dispatch(position: number): Promise<Response> { + // `concurrency-and-async.md:46`: check the signal at the top of each loop iteration or before + // each expensive step. The step walk is exactly that, and a pillar step's fork-driven re-drives + // are worse -- an already-aborted call used to walk every installed step and could do real work + // on the way (the auth step's bearer refresh is the concrete case) before the terminal transport + // hop finally rejected. Each pillar already guards its OWN loop (RETRY-32, and redirect's + // per-hop check); what was unguarded is the walk itself and any step without a loop of its own. + // + // Mapped through `abortToSdkError` rather than `throwIfAborted()`, whose `DOMException` is the + // very inconsistency N1 closed: one cancellation type wherever the abort was observed, with the + // caller's own reason kept as `cause`. + if (this.#signal?.aborted === true) { + throw abortToSdkError(this.#signal, this.#signal.reason); + } + if (position >= this.#steps.length) { + // PIPE-13: exhausted -- dispatch the current in-flight request to the terminal transport. + return this.#transport.send(this.#request, this.#options, this.#signal); + } + const descriptor = this.#steps[position]; + invariant( + descriptor !== undefined, + `pipeline cursor position ${String(position)} is within bounds but undefined`, + ); + const next = this.#continuationAt(position + 1, descriptor.stage); + // PIPE-17: `signal` and `options` are readable by every step, pillar or not, and are shared by + // reference across every fork -- never copied, so a step cannot diverge them per attempt. + const shared = { + next, + context: this.#context, + signal: this.#signal, + options: this.#options, + }; + const ctx: StepContext = PILLAR_STAGES.has(descriptor.stage) + ? { + ...shared, + fork: (): Next => + this.#continuationAt(position + 1, descriptor.stage), + } + : shared; + return descriptor.fn(this.#request, ctx); + } + + /** + * Builds a ONE-SHOT continuation targeting `targetPosition` (PIPE-11/15: a second call throws + * CursorAlreadyAdvancedError). `ctx.next` and every `ctx.fork()` call share this helper -- both always + * target `position + 1` of the requesting step; `fork()` may simply be called again to obtain a fresh + * one-shot continuation bound to that same target (PIPE-16). + */ + #continuationAt(targetPosition: number, ownerStage: Stage): Next { + let used = false; + return async (replacementRequest?: Request): Promise<Response> => { + if (used) throw new CursorAlreadyAdvancedError(ownerStage); + used = true; + if (replacementRequest !== undefined) { + this.#request = replacementRequest; // PIPE-14: sticks for every later step and the terminal dispatch. + } + return this.#dispatch(targetPosition); + }; + } +} diff --git a/packages/core/src/pipeline/errors.test.ts b/packages/core/src/pipeline/errors.test.ts new file mode 100644 index 0000000..2e5c52c --- /dev/null +++ b/packages/core/src/pipeline/errors.test.ts @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/errors.test.ts +// Exercises: PIPE-5 (PillarCollisionError), PIPE-21 (AnchorNotFoundError), PIPE-18/19 (CrossStageEditError), +// PIPE-11/15 (CursorAlreadyAdvancedError), PIPE-8 (ReservedStageError) +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import { + AnchorNotFoundError, + CrossStageEditError, + CursorAlreadyAdvancedError, + PillarCollisionError, + ReservedStageError, +} from './errors.js'; + +describe('PillarCollisionError (PIPE-5)', () => { + test('carries the stage and both colliding type symbols, extends DexpaceError', () => { + const existing = Symbol('existing'); + const incoming = Symbol('incoming'); + + const error = new PillarCollisionError('RETRY', existing, incoming); + + expect(error).toBeInstanceOf(DexpaceError); + expect(error.name).toBe('PillarCollisionError'); + expect(error.stage).toBe('RETRY'); + expect(error.existingType).toBe(existing); + expect(error.incomingType).toBe(incoming); + // PIPE-5: the message itself names both types, not just the instance fields. + expect(error.message).toContain('Symbol(existing)'); + expect(error.message).toContain('Symbol(incoming)'); + }); +}); + +describe('AnchorNotFoundError (PIPE-21)', () => { + test('carries the missing anchor type and the attempted operation', () => { + const anchorType = Symbol('missing'); + + const error = new AnchorNotFoundError(anchorType, 'insertAfter'); + + expect(error).toBeInstanceOf(DexpaceError); + expect(error.anchorType).toBe(anchorType); + expect(error.operation).toBe('insertAfter'); + expect(error.message).toContain('Symbol(missing)'); // PIPE-21: the message identifies the type + }); +}); + +describe('CrossStageEditError (PIPE-18, PIPE-19)', () => { + test('carries the anchor stage and the incoming stage', () => { + const error = new CrossStageEditError('RETRY', 'AUTH'); + + expect(error).toBeInstanceOf(DexpaceError); + expect(error.anchorStage).toBe('RETRY'); + expect(error.incomingStage).toBe('AUTH'); + }); +}); + +describe('CursorAlreadyAdvancedError (PIPE-11, PIPE-15)', () => { + test('carries the stage of the step that reused its continuation', () => { + const error = new CursorAlreadyAdvancedError('RETRY'); + + expect(error).toBeInstanceOf(DexpaceError); + expect(error.stage).toBe('RETRY'); + }); +}); + +describe('ReservedStageError (PIPE-8)', () => { + test('carries the attempted operation', () => { + const error = new ReservedStageError('append'); + + expect(error).toBeInstanceOf(DexpaceError); + expect(error.operation).toBe('append'); + }); +}); diff --git a/packages/core/src/pipeline/errors.ts b/packages/core/src/pipeline/errors.ts new file mode 100644 index 0000000..ab57491 --- /dev/null +++ b/packages/core/src/pipeline/errors.ts @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/errors.ts +import {DexpaceError} from '../http/errors.js'; +import type {Stage} from './stage.js'; + +/** + * PIPE-5: installing a distinct second step onto an occupied pillar; names both types and the stage. + * + * @public + */ +export class PillarCollisionError extends DexpaceError { + /** The pillar stage that was already occupied. */ + readonly stage: Stage; + /** The step type already installed there. */ + readonly existingType: symbol; + /** The step type the rejected install tried to add. */ + readonly incomingType: symbol; + + // eslint-disable-next-line max-params -- constructor parameters fixed by error model: PIPE-5 requires the stage and BOTH colliding types, plus the taxonomy's trailing `options?: ErrorOptions`; same exemption as HttpStatusError. Revisit only if the error model drops a field. + constructor( + stage: Stage, + existingType: symbol, + incomingType: symbol, + options?: ErrorOptions, + ) { + // PIPE-5: the error names BOTH step types and points at the replace path. Symbols are rendered with + // String() (`Symbol(retry)`) -- a bare symbol field is invisible in a stack trace or log line + // (docs/knowledge/harvested/error-handling.md:40), the same reason 4a's DuplicateContextKeyError renders its key. + super( + `pillar stage '${stage}' already holds ${String(existingType)}; cannot install ${String(incomingType)} (use replace() to swap it)`, + options, + ); + this.stage = stage; + this.existingType = existingType; + this.incomingType = incomingType; + } +} + +/** + * PIPE-21: an insertAfter/insertBefore/replace whose anchor type matches nothing in the pipeline. + * + * @public + */ +export class AnchorNotFoundError extends DexpaceError { + /** The step type named as the anchor, which no installed step carries. */ + readonly anchorType: symbol; + /** The builder operation that failed -- `insertAfter`, `insertBefore` or `replace`. */ + readonly operation: string; + + constructor(anchorType: symbol, operation: string, options?: ErrorOptions) { + // PIPE-21: "fail with an error identifying the missing type" -- in the message, not only as a field. + super( + `${operation}: no step of type ${String(anchorType)} is present in the pipeline`, + options, + ); + this.anchorType = anchorType; + this.operation = operation; + } +} + +/** + * PIPE-18/PIPE-19: a cross-stage insert/replace -- the incoming descriptor's stage differs from the + * anchor's. + * + * @public + */ +export class CrossStageEditError extends DexpaceError { + /** The stage the anchor step occupies. */ + readonly anchorStage: Stage; + /** The stage the incoming descriptor declares, which differs from the anchor's. */ + readonly incomingStage: Stage; + + constructor( + anchorStage: Stage, + incomingStage: Stage, + options?: ErrorOptions, + ) { + super( + `cannot insert/replace across stages: anchor is in '${anchorStage}', incoming step declares '${incomingStage}'`, + options, + ); + this.anchorStage = anchorStage; + this.incomingStage = incomingStage; + } +} + +/** + * PIPE-11/PIPE-15: a step reused an already-invoked next()/fork() continuation instead of forking again. + * + * @public + */ +export class CursorAlreadyAdvancedError extends DexpaceError { + /** The stage of the step that reused its continuation. */ + readonly stage: Stage; + + constructor(stage: Stage, options?: ErrorOptions) { + super( + `step at stage '${stage}' reused an already-invoked continuation; a re-driving step must call fork() again`, + options, + ); + this.stage = stage; + } +} + +/** + * PIPE-8: an attempt to install a user step onto the reserved, terminal SEND stage. + * + * @public + */ +export class ReservedStageError extends DexpaceError { + /** The builder operation that tried to write to the reserved SEND stage. */ + readonly operation: string; + + constructor(operation: string, options?: ErrorOptions) { + super( + `${operation}: the SEND stage is reserved for the terminal transport hop and cannot hold a user step`, + options, + ); + this.operation = operation; + } +} diff --git a/packages/core/src/pipeline/runtime.test.ts b/packages/core/src/pipeline/runtime.test.ts new file mode 100644 index 0000000..6844ab4 --- /dev/null +++ b/packages/core/src/pipeline/runtime.test.ts @@ -0,0 +1,592 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/runtime.test.ts +// Exercises: PIPE-9 (an empty pipeline dispatches directly, no cursor/context allocated), PIPE-10 (each +// send() allocates its own per-call state, interleaved calls share none of it, and the built step view is +// frozen and copied), PIPE-11 (per-call mutable state lives on the cursor, never on the runtime), PIPE-14 +// (a substituted request reaches the wire, and is what the exchange context is built from), PIPE-25 +// (get steps() exposes the flattened, immutable array), PIPE-26 (Runtime itself satisfies the Transport SPI +// with one send() method, and nests inside another pipeline with the caller's options intact), PIPE-27 +// (close() never touches the wrapped transport), CTX-17's positive half (the first store entry is installed +// by the first promotion), CTX-1/2/3/6 (exchangeSource pins the call key and instrumentation when it +// rebuilds), OBS-22/OBS-23 (the caller's active span and diagnostic fields are what they were once +// send() settles, either way), OBS-29 (one operation span per send, ended exactly once even when end() +// throws, and a second send gets its own), CTX-11 (a throwing tracerFactory leaks no store entry), +// CTX-16 (the pipeline's operation name reaches the request context) +import {describe, expect, test} from 'bun:test'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {contextStore} from '../context/store.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {RequestOptions} from '../http/request-options.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {invariant} from '../invariant.js'; +import { + getDiagnosticContext, + pushDiagnosticFields, +} from '../observability/diagnostic-context.js'; +import { + NOOP_SPAN, + createInstrumentationBundle, + getActiveSpan, + type Span, + type Tracer, +} from '../observability/tracing.js'; +import type {Transport} from '../seams/transport.js'; +import {createRuntime, exchangeSource} from './runtime.js'; +import type {Step, StepDescriptor} from './step.js'; + +function aRequest(url: string): Request { + return Request.newBuilder().url(url).build(); +} + +function aResponse(status: number): Response { + return Response.newBuilder() + .request(aRequest('https://example.com')) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .build(); +} + +class RecordingTransport implements Transport { + readonly calls: { + request: Request; + options: RequestOptions | undefined; + signal: AbortSignal | undefined; + }[] = []; + closeCalls = 0; + #response: Response; + + constructor(response: Response) { + this.#response = response; + } + + send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + this.calls.push({request, options, signal}); + return Promise.resolve(this.#response); + } + + close(): Promise<void> { + this.closeCalls += 1; + return Promise.resolve(); + } +} + +// No `afterEach(() => contextStore.clear())`: the singleton is shared by every test file in the run, so a +// blanket clear wipes entries a sibling installed (4a's plan forbids it by name; testing.md:50,52). Nothing +// here needs one -- `Runtime.send()` evicts its own entry in a `finally`, on the success and the throw path. + +describe('Runtime.send empty pipeline (PIPE-9)', () => { + test('dispatches directly to the terminal transport, threading options and signal, no context installed', async () => { + const canned = aResponse(200); + const transport = new RecordingTransport(canned); + const runtime = createRuntime([], transport); + const request = aRequest('https://example.com/a'); + const signal = new AbortController().signal; + const sizeBefore = contextStore.size; + + const response = await runtime.send(request, undefined, signal); + + expect(response).toBe(canned); + expect(transport.calls).toEqual([{request, options: undefined, signal}]); + // A delta, not an absolute size: `contextStore` is process-wide, so a sibling test file sharing the + // process must not be able to turn this assertion red (styleguide 11.7 -- tests survive any order). + expect(contextStore.size).toBe(sizeBefore); + }); +}); + +describe('Runtime.send context-store wiring (CTX-17, CTX-8)', () => { + test('installs a RequestContext before dispatch, then evicts it after the call resolves', async () => { + let observed: ExecutionContext | undefined; + const step: Step = async (_request, ctx) => { + observed = contextStore.get(ctx.context.key); + return ctx.next(); + }; + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: step, + }; + const runtime = createRuntime( + [descriptor], + new RecordingTransport(aResponse(200)), + ); + + const response = await runtime.send(aRequest('https://example.com')); + + invariant( + observed !== undefined, + 'the step must have observed an installed context', + ); + expect(observed.kind).toBe('request'); + expect(contextStore.get(observed.key)).toBeUndefined(); // evicted in send()'s finally + expect(response.status.code).toBe(200); + }); + + test('evicts the installed context even when a step throws', async () => { + let observedKey: symbol | undefined; + + // eslint-disable-next-line @typescript-eslint/require-await -- throwing before any await IS the case under test: PIPE-29/30 hold structurally because an `async` step body that throws synchronously still surfaces as a rejected promise. `Promise.reject` would exercise something else. Revisit if a step ever throws through a real await. + const step: Step = async (_request, ctx) => { + observedKey = ctx.context.key; + throw new Error('boom'); + }; + const descriptor: StepDescriptor = { + type: Symbol('throws'), + stage: 'PRE_LOGGING', + fn: step, + }; + const runtime = createRuntime( + [descriptor], + new RecordingTransport(aResponse(200)), + ); + + const rejection: unknown = await runtime + .send(aRequest('https://example.com')) + .catch((error: unknown) => error); + expect(rejection).toBeInstanceOf(Error); + expect((rejection as Error).message).toBe('boom'); + invariant( + observedKey !== undefined, + 'the step must have run and captured its call key', + ); + expect(contextStore.get(observedKey)).toBeUndefined(); + }); +}); + +describe('exchangeSource (PIPE-14, CTX-1, CTX-2, CTX-3, CTX-6)', () => { + // Tested directly rather than by spying on `contextStore.install`: the exchange context is evicted in + // `send()`'s own `finally`, so observing it end-to-end would mean patching a method on the process-wide + // singleton -- a mock of an owned interface (styleguide 11.3) that also leaks across test files sharing + // the process if a run is ever parallelised. `exchangeSource` is a pure function; the end-to-end half that + // remains observable (the substituted request is what actually reached the wire) is asserted below. + test('returns the SAME context object when no step substituted the request', () => { + const request = aRequest('https://example.com'); + const context = createRequestContext(request, {operationName: 'GetWidget'}); + + expect(exchangeSource(context, request)).toBe(context); + }); + + test('rebuilds around the substituted request, pinning the same key and instrumentation', () => { + const original = aRequest('https://example.com/original'); + const substituted = aRequest('https://example.com/substituted'); + const context = createRequestContext(original, { + operationName: 'GetWidget', + }); + + const rebuilt = exchangeSource(context, substituted); + + expect(rebuilt.request).toBe(substituted); + expect(rebuilt.key).toBe(context.key); // CTX-3: one call key for the whole chain + expect(rebuilt.instrumentation).toBe(context.instrumentation); // CTX-2: carried forward by reference + expect(rebuilt.operationName).toBe('GetWidget'); + }); +}); + +describe('Runtime.send request substitution reaches the wire (PIPE-14)', () => { + test('the transport receives the substituted request, not the original', async () => { + const original = aRequest('https://example.com/original'); + const substituted = aRequest('https://example.com/substituted'); + const substituteStep: Step = async (_request, ctx) => ctx.next(substituted); + const descriptor: StepDescriptor = { + type: Symbol('substitute'), + stage: 'PRE_LOGGING', + fn: substituteStep, + }; + const transport = new RecordingTransport(aResponse(200)); + + await createRuntime([descriptor], transport).send(original); + + expect(transport.calls[0]?.request).toBe(substituted); + }); +}); + +describe('Runtime concurrency (PIPE-10, PIPE-11)', () => { + test("two interleaved sends never observe each other's in-flight request", async () => { + const transport = new RecordingTransport(aResponse(200)); + const rewrite: Step = async (request, ctx) => { + await Promise.resolve(); // hand the event loop over, so both drives are mid-flight at once + return ctx.next(aRequest(`${request.url.href}rewritten`)); + }; + const descriptor: StepDescriptor = { + type: Symbol('rewrite'), + stage: 'PRE_LOGGING', + fn: rewrite, + }; + const runtime = createRuntime([descriptor], transport); + + await Promise.all([ + runtime.send(aRequest('https://example.com/a/')), + runtime.send(aRequest('https://example.com/b/')), + ]); + + // PIPE-11: per-call mutable state lives on the per-call cursor, so one call's substituted request + // (PIPE-14 makes it stick for the rest of *that* call) cannot leak into the other's dispatch. + expect(transport.calls.map(call => call.request.url.href).sort()).toEqual([ + 'https://example.com/a/rewritten', + 'https://example.com/b/rewritten', + ]); + }); +}); + +describe('Runtime.steps (PIPE-25)', () => { + test('exposes the exact flattened array it was constructed with', () => { + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: async (_r, ctx) => ctx.next(), + }; + const runtime = createRuntime( + [descriptor], + new RecordingTransport(aResponse(200)), + ); + + expect(runtime.steps).toEqual([descriptor]); + }); + + test('the exposed view is frozen', () => { + const runtime = createRuntime([], new RecordingTransport(aResponse(200))); + + expect(Object.isFrozen(runtime.steps)).toBe(true); + }); + + test("copies the caller's array, so a later mutation of it cannot reach the built runtime", () => { + const descriptor: StepDescriptor = { + type: Symbol('probe'), + stage: 'PRE_LOGGING', + fn: async (_r, ctx) => ctx.next(), + }; + const source: StepDescriptor[] = [descriptor]; + const runtime = createRuntime( + source, + new RecordingTransport(aResponse(200)), + ); + + source.push({...descriptor, type: Symbol('smuggled')}); + + // PIPE-10: immutable after construction, and not by caller discipline. + expect(runtime.steps).toEqual([descriptor]); + }); +}); + +describe('Runtime as a nested transport (PIPE-26)', () => { + test("a built pipeline stands in as another pipeline's transport, options and signal surviving both hops", async () => { + const transport = new RecordingTransport(aResponse(200)); + const log: string[] = []; + const probe = + (label: string): Step => + async (_request, ctx) => { + log.push(`enter:${label}`); + const response = await ctx.next(); + log.push(`exit:${label}`); + return response; + }; + const inner = createRuntime( + [{type: Symbol('inner'), stage: 'PRE_SERDE', fn: probe('inner')}], + transport, + ); + const outer = createRuntime( + [{type: Symbol('outer'), stage: 'PRE_REDIRECT', fn: probe('outer')}], + inner, + ); + const options = RequestOptions.EMPTY; + const signal = new AbortController().signal; + + await outer.send(aRequest('https://example.com'), options, signal); + + expect(log).toEqual([ + 'enter:outer', + 'enter:inner', + 'exit:inner', + 'exit:outer', + ]); + // PIPE-26: "options survive the indirection" -- through the outer cursor, the nested runtime's own + // send(), and its cursor, reaching the terminal transport as the same references the caller passed. + expect(transport.calls[0]?.options).toBe(options); + expect(transport.calls[0]?.signal).toBe(signal); + }); +}); + +describe('Runtime.close (PIPE-27)', () => { + test('never calls the underlying transport close', async () => { + const transport = new RecordingTransport(aResponse(200)); + const runtime = createRuntime([], transport); + + await runtime.close(); + + expect(transport.closeCalls).toBe(0); + }); +}); +/** A tracer recording the whole lifecycle of every span it opens. */ +function recordingTracer(): { + tracer: Tracer; + spans: {name: string; ended: number; exceptions: unknown[]}[]; +} { + const spans: {name: string; ended: number; exceptions: unknown[]}[] = []; + const tracer: Tracer = { + startSpan(name: string): Span { + const record = {name, ended: 0, exceptions: [] as unknown[]}; + spans.push(record); + const span: Span = { + isRecording: true, + setAttribute(): Span { + return span; + }, + recordException(error: unknown): Span { + record.exceptions.push(error); + return span; + }, + end(): void { + record.ended += 1; + }, + }; + return span; + }, + }; + return {tracer, spans}; +} + +/** A step that does nothing but advance, so the pipeline is non-empty (PIPE-9's other branch). */ +function passthroughStep(): StepDescriptor { + return { + type: Symbol('passthrough'), + stage: 'PRE_REDIRECT', + fn: (request, ctx) => ctx.next(request), + }; +} + +function runtimeWith( + tracer: Tracer, + transport: Transport, + steps: readonly StepDescriptor[] = [passthroughStep()], +): ReturnType<typeof createRuntime> { + return createRuntime(steps, transport, { + instrumentation: createInstrumentationBundle(() => tracer), + }); +} + +describe('the per-operation span: opened once, ended once (OBS-29)', () => { + test('one span is opened per send() and ended exactly once on success', async () => { + const {tracer, spans} = recordingTracer(); + const transport = new RecordingTransport(aResponse(200)); + + await runtimeWith(tracer, transport).send(aRequest('https://example.com')); + + expect(spans.length).toBe(1); + expect(spans[0]?.ended).toBe(1); + expect(spans[0]?.exceptions).toEqual([]); + }); + + test('a failing drive records the exception and still ends the span exactly once', async () => { + const {tracer, spans} = recordingTracer(); + const boom = new Error('boom'); + const failing: StepDescriptor = { + type: Symbol('failing'), + stage: 'PRE_REDIRECT', + fn: () => Promise.reject(boom), + }; + + const thrown = await runtimeWith( + tracer, + new RecordingTransport(aResponse(200)), + [failing], + ) + .send(aRequest('https://example.com')) + .then( + () => undefined, + (e: unknown) => e, + ); + expect(thrown).toBe(boom); + + expect(spans.length).toBe(1); + expect(spans[0]?.ended).toBe(1); + expect(spans[0]?.exceptions).toEqual([boom]); + }); +}); + +describe('the per-operation span: 1:1 with a logical operation (OBS-29)', () => { + test('a re-drive inside the pillars does NOT open a second operation span (PIPE-2)', async () => { + const {tracer, spans} = recordingTracer(); + const transport = new RecordingTransport(aResponse(200)); + // Forks twice, the way RETRY and REDIRECT do. OBS-29's 1:1 binding is exactly what this asserts: + // two transmissions, one logical operation, one span. + const forking: StepDescriptor = { + type: Symbol('forking'), + stage: 'RETRY', + fn: async (request, ctx) => { + invariant(ctx.fork !== undefined, 'pillar stage expected'); + await ctx.fork()(request); + return ctx.fork()(request); + }, + }; + + await runtimeWith(tracer, transport, [forking]).send( + aRequest('https://example.com'), + ); + + expect(transport.calls.length).toBe(2); + expect(spans.length).toBe(1); + expect(spans[0]?.ended).toBe(1); + }); + + test('a nested Runtime used as a transport opens no second span (PIPE-26)', async () => { + const {tracer, spans} = recordingTracer(); + const inner = runtimeWith(tracer, new RecordingTransport(aResponse(200))); + + await runtimeWith(tracer, inner).send(aRequest('https://example.com')); + + expect(spans.length).toBe(1); + expect(spans[0]?.ended).toBe(1); + }); + + test('an empty pipeline opens no span at all (PIPE-9)', async () => { + const {tracer, spans} = recordingTracer(); + await runtimeWith(tracer, new RecordingTransport(aResponse(200)), []).send( + aRequest('https://example.com'), + ); + expect(spans).toEqual([]); + }); + + test('no instrumentation override means no tracer and no throw', async () => { + const transport = new RecordingTransport(aResponse(200)); + const response = await createRuntime( + [ + { + type: Symbol('plain'), + stage: 'PRE_REDIRECT', + fn: (request, ctx) => ctx.next(request), + }, + ], + transport, + ).send(aRequest('https://example.com')); + expect(response.status.code).toBe(200); + }); +}); + +describe('async-context hygiene across send() (OBS-22, OBS-23, OBS-29)', () => { + test('the caller observes no active span after `await send()` resolves', async () => { + const {tracer} = recordingTracer(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + + await runtimeWith(tracer, new RecordingTransport(aResponse(200))).send( + aRequest('https://example.com'), + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('the caller observes no active span after `await send()` rejects', async () => { + const {tracer} = recordingTracer(); + const failing: StepDescriptor = { + type: Symbol('failing'), + stage: 'PRE_REDIRECT', + fn: () => Promise.reject(new Error('boom')), + }; + + await runtimeWith(tracer, new RecordingTransport(aResponse(200)), [failing]) + .send(aRequest('https://example.com')) + .then( + () => undefined, + () => undefined, + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('diagnostic fields a step pushed do not outlive the call (OBS-23)', async () => { + const pushing: StepDescriptor = { + type: Symbol('pushing'), + stage: 'PRE_REDIRECT', + fn: (request, ctx) => { + // What `activateSpanForCorrelation` does inside the LOGGING pillar step: a handle-based push + // whose restore runs in a later continuation and therefore never reaches this caller. + pushDiagnosticFields({'trace.id': 't-leaked', 'span.id': 's-leaked'}); + return ctx.next(request); + }, + }; + + await runtimeWith( + recordingTracer().tracer, + new RecordingTransport(aResponse(200)), + [pushing], + ).send(aRequest('https://example.com')); + + expect(getDiagnosticContext(null)).toEqual({}); + }); + + test('a second send() on the same runtime opens its own operation span (OBS-29)', async () => { + const {tracer, spans} = recordingTracer(); + const runtime = runtimeWith(tracer, new RecordingTransport(aResponse(200))); + + await runtime.send(aRequest('https://example.com/one')); + await runtime.send(aRequest('https://example.com/two')); + + expect(spans.length).toBe(2); + expect(spans.map(span => span.ended)).toEqual([1, 1]); + }); +}); + +describe('store and span hygiene on a failing tracer (CTX-11, OBS-29)', () => { + test('a throwing tracerFactory leaves the context store the size it found it', async () => { + const boom = new Error('tracer down'); + const runtime = createRuntime( + [passthroughStep()], + new RecordingTransport(aResponse(200)), + { + instrumentation: createInstrumentationBundle(() => { + throw boom; + }), + }, + ); + + const before = contextStore.size; + const thrown = await runtime.send(aRequest('https://example.com')).then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(boom); + expect(contextStore.size).toBe(before); + }); + + test('an end() that throws on the success path is not called a second time', async () => { + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span: Span = { + isRecording: true, + setAttribute(): Span { + return span; + }, + recordException(error: unknown): Span { + exceptions.push(error); + return span; + }, + end(): void { + ends += 1; + throw endFailed; + }, + }; + + const thrown = await runtimeWith( + {startSpan: () => span}, + new RecordingTransport(aResponse(200)), + ) + .send(aRequest('https://example.com')) + .then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([endFailed]); + }); +}); diff --git a/packages/core/src/pipeline/runtime.ts b/packages/core/src/pipeline/runtime.ts new file mode 100644 index 0000000..fb87486 --- /dev/null +++ b/packages/core/src/pipeline/runtime.ts @@ -0,0 +1,373 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/runtime.ts +import { + createDispatchContext, + createRequestContext, + promoteToExchange, + promoteToRequest, + type ContextInit, + type ExecutionContext, + type RequestContext, +} from '../context/context.js'; +import {contextStore} from '../context/store.js'; +import { + captureDiagnosticSnapshot, + runWithSnapshot, +} from '../observability/diagnostic-context.js'; +import { + getActiveSpan, + NOOP_TRACER, + runWithActiveSpan, + type Span, + type Tracer, +} from '../observability/tracing.js'; +import type {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import type {Transport} from '../seams/transport.js'; +import {Cursor} from './cursor.js'; +import type {StepDescriptor} from './step.js'; + +/** + * What a built pipeline carries into every drive. `createDispatchContext` takes the `instrumentation` + * and `key` halves -- `operationName` is not a dispatch-stage concept (CTX-16 introduces it at the + * request stage) -- and `send()` hands the name to `promoteToRequest` itself, one promotion later. + */ +type RuntimeContextInit = ContextInit; + +/** The advisory span name, matching what the LOGGING pillar step uses for its per-attempt spans. */ +const OPERATION_SPAN_NAME = 'http.client.operation'; + +/** + * Opens the one span that corresponds 1:1 to a logical operation (OBS-29), or returns `undefined` + * when there is nothing to open. + * + * **Why here and not in the LOGGING pillar step.** `PIPE-2` fixes that step *inside* the `RETRY` and + * `REDIRECT` pipelines, so its span is opened per transmission attempt and per redirect hop — the + * right scope for an attempt, the wrong one for an operation. `OBS-29` asks for "one tracer instance + * per logical operation", and `send()` is the only place in this package that runs exactly once per + * one. The two spans are complementary rather than duplicative: this is the parent, the LOGGING + * step's are the children. + * + * **Nesting is a real case, not a hypothetical.** `Runtime implements Transport` (PIPE-26), so a + * runtime can be another runtime's terminal transport, and a caller can also have activated a span + * of their own. Either way the outermost one is the logical operation, so an already-active + * recording span means this call is *inside* an operation rather than starting one. + */ +function startOperationSpan(context: RequestContext): Span | undefined { + if (getActiveSpan().isRecording) return undefined; + + const factory = context.instrumentation.tracerFactory as + ((operationName: string) => Tracer | undefined) | undefined; + if (typeof factory !== 'function') return undefined; + + const tracer = factory(OPERATION_SPAN_NAME) ?? NOOP_TRACER; + const span = tracer.startSpan(OPERATION_SPAN_NAME); + return span.isRecording ? span : undefined; +} + +/** + * Runs `drive` as the body of `span` and ends that span EXACTLY once, whichever way it finishes + * (OBS-29: `operationSucceeded` and `operationFailed` are mutually exclusive and happen once each). + * + * The `ended` latch is not belt-and-braces. `end()` is caller-supplied through `tracerFactory`, and + * OBS-20 deliberately does not wrap tracer calls -- so a throwing `end()` on the success path lands + * in the `catch` below, which is obliged to `recordException` the failure it now has to surface. + * Without the latch that path called `end()` a second time on a span the tracer already closed. + */ +async function driveWithSpan( + span: Span, + drive: () => Promise<Response>, +): Promise<Response> { + let ended = false; + const endOnce = (): void => { + if (ended) return; + ended = true; + span.end(); + }; + try { + const response = await drive(); + endOnce(); + return response; + } catch (error: unknown) { + span.recordException(error); + endOnce(); + throw error; + } +} + +/** + * The request context to promote from once the drive finishes: the original, unless a step substituted the + * outbound request (PIPE-14), in which case an off-chain rebuild around the request that was actually sent, + * pinned to the SAME call key (CTX-6's explicit-key path) and carrying the same instrumentation bundle by + * reference (CTX-2/CTX-3). Promoting straight off the original would pair the response with a request that + * never left the process, against CTX-1's "the exchange stage exposes the request and the response". Doing it + * here rather than widening `promoteToExchange` with a request-override keeps promotion strictly additive. + * + * Exported (still internal-only, still absent from the package barrel) so its two branches can be asserted as + * the pure function they are. The alternative -- observing the exchange context end-to-end -- would require + * patching `install` on the process-wide `contextStore` singleton, since `send()` evicts the entry in its own + * `finally`. + * + * @internal + */ +export function exchangeSource( + context: RequestContext, + finalRequest: Request, +): RequestContext { + if (finalRequest === context.request) return context; + return createRequestContext(finalRequest, { + key: context.key, + instrumentation: context.instrumentation, + operationName: context.operationName, + }); +} + +/** + * TypeScript has no friend classes, so `PipelineBuilder` -- a different module -- reaches `Runtime`'s + * private constructor through this module-scoped `let`, assigned exactly once inside the class's + * `static {}` block. Init-once wiring, not mutable state, the same shape every builder-based model in + * `src/http/` uses (`createHeaders`, `createRequest`, ...). It is surfaced as {@link createRuntime} + * rather than kept module-local because the sanctioned construction site lives in another file. + */ +let create: ( + steps: readonly StepDescriptor[], + transport: Transport, + contextInit: RuntimeContextInit, +) => Runtime; + +/** + * The read half of the same friend-class hook: `PipelineBuilder.seedFrom(runtime, 'flatten')` builds + * the pipeline that replaces `runtime`, so it has to recover the options `runtime` was built with, + * and `#contextInit` is private. Surfaced as {@link pipelineOptionsOf} rather than as a getter, + * because a getter on this `@public` class would publish `ContextInit`'s in-package `key` slot. + */ +let readContextInit: (runtime: Runtime) => RuntimeContextInit; + +/** + * The built, immutable pipeline (PIPE-10, PIPE-25). Implements `Transport` itself (PIPE-26) -- Phase 2's + * `Transport` SPI has one method (`send`), so there is no second `sendAsync` entry point to delegate through. + * `close()` deliberately never touches the wrapped transport (PIPE-27): the pipeline never owns it. + * + * The constructor is TS-`private`, so no field-wise constructor appears in the emitted `.d.ts` and a + * consumer cannot assemble a `Runtime` around `PipelineBuilder.build()`'s validation. That matters + * now that this class is public surface: a hand-built `new Runtime([authStep(a), authStep(b)], t)` + * would put two steps in the single AUTH pillar slot (PIPE-4/PIPE-5, AUTH-27) and a hand-ordered step + * array would invert PIPE-2's pillar precedence chain, both without any collision error, because + * `Cursor` runs whatever array it is handed. `PipelineBuilder` is the only path that enforces either. + * + * @public + */ +export class Runtime implements Transport { + readonly #steps: readonly StepDescriptor[]; + readonly #transport: Transport; + readonly #contextInit: RuntimeContextInit; + + private constructor( + steps: readonly StepDescriptor[], + transport: Transport, + contextInit: RuntimeContextInit, + ) { + // PIPE-10/PIPE-25: the built runtime is immutable, and `get steps()` hands out a read-only view. Copying + // and freezing here rather than trusting the caller makes both structural -- `createRuntime` is reachable + // from any in-package caller, so an unfrozen array passed in would leave the "immutable after + // construction" guarantee resting on caller discipline. + this.#steps = Object.freeze([...steps]); + this.#transport = transport; + this.#contextInit = contextInit; + } + + static { + create = (steps, transport, contextInit) => + new Runtime(steps, transport, contextInit); + readContextInit = runtime => runtime.#contextInit; + } + + /** + * Drives `request` through the flattened step array and on to the wrapped transport, installing this + * call's `ExecutionContext` in the store for the duration of the drive and evicting it again on both + * the resolve and the throw path (CTX-17, CTX-9). + * + * A pipeline with no steps skips both the cursor and the context entirely and dispatches straight to + * the wrapped transport (PIPE-9). + * + * @param request - the request to send. + * @param options - per-call operational overrides, carried unchanged across every re-drive fork and + * threaded into each terminal dispatch (PIPE-17). + * @param signal - the caller's abort signal, threaded to the terminal dispatch. Not observed between + * steps in this phase -- see the roadmap's Phase 4c open finding F9. + * @returns whatever the outermost step returned (PIPE-12). + * + * @remarks Opens **one** span for the whole call when the context's instrumentation supplies a + * tracer and no span is already active — `OBS-29`'s "one tracer instance per logical operation". + * It is the parent of whatever per-attempt spans the LOGGING pillar step opens inside the RETRY + * and REDIRECT pipelines, which `PIPE-2` fixes there and which are therefore per *transmission*. + * Supply the tracer through `PipelineOptions.instrumentation`. + * + * @remarks The caller's async context is restored when this settles, resolved or rejected: the + * active span and the diagnostic fields (`trace.id`, `span.id`) are what they were before the + * call, so an application log emitted after `await send()` carries nothing from it. Both stores + * are scoped with `AsyncLocalStorage.run`, which also unwinds a scope a step left open. + */ + async send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + if (this.#steps.length === 0) { + // PIPE-9: an empty pipeline dispatches directly to the terminal transport, no cursor allocated. + return this.#transport.send(request, options, signal); + } + // Every async-scoped store this call touches is RE-RUN around the drive rather than entered in + // place. `runWithSnapshot(captureDiagnosticSnapshot())` re-enters the caller's OWN diagnostic + // store under `AsyncLocalStorage.run`, which changes nothing a step can observe and everything + // about what survives the call: a `pushDiagnosticFields` below -- the LOGGING pillar's OBS-23 + // correlation scope is the shipped one -- now unwinds when `send()` returns. `#drive` does the + // same for the span slot with `runWithActiveSpan`. + // + // Until 2026-09-05 both slots were `enterWith` plus a restore closure called from a `finally`. + // `enterWith` installs on the async resource running it, and that resource is the CALLER's -- + // `send`'s synchronous prefix runs there -- while the `finally` runs on a resource created by + // the first `await` inside. So the restore reached nothing the caller could see: after + // `await send()` the ended operation span was still "active", suppressing the next call's span + // (OBS-29's 1:1 binding), and this call's `trace.id`/`span.id` rode into every subsequent + // application log through any core `Logger` (audit #67 / #80). + return runWithSnapshot(captureDiagnosticSnapshot(), () => + this.#drive(request, options, signal), + ); + } + + /** + * One drive, inside the re-entered stores `send()` established. Split out so `send()` is the + * scoping statement and nothing else: the whole body has to sit inside the `run` callback for the + * unwind to cover it, and a body that long inline reads as if the callback were optional. + */ + async #drive( + request: Request, + options: RequestOptions | undefined, + signal: AbortSignal | undefined, + ): Promise<Response> { + const dispatchContext = createDispatchContext(this.#contextInit); + // CTX-16: the operation name this pipeline was built with enters at the request stage and is + // carried unchanged by every promotion after it. + const requestContext = promoteToRequest( + dispatchContext, + request, + this.#contextInit.operationName, + ); + let currentContext: ExecutionContext = requestContext; // tracks the latest install for the finally below. + const drive = async (): Promise<Response> => { + const cursor = new Cursor({ + steps: this.#steps, + transport: this.#transport, + request, + context: requestContext, + options, + signal, + }); + const response = await cursor.advance(); + // PIPE-14: a step may have substituted the outbound request -- promote from whatever was actually sent. + const exchangeContext = promoteToExchange( + exchangeSource(requestContext, cursor.request), + response, + ); + contextStore.install(exchangeContext); // install-or-replace under the same key (CTX-8). + currentContext = exchangeContext; + return response; + }; + // CTX-11/CTX-17: the install and everything that can throw after it are inside ONE try, so the + // `finally` evicts on every path. `startOperationSpan` calls a caller-supplied `tracerFactory`, + // which OBS-30 says must not throw and nothing enforces; installed outside the try, one throwing + // factory left an entry in the process-wide store per failed send. + try { + contextStore.install(requestContext); // CTX-17's positive half: the first store entry, at the first promotion. + // OBS-29's 1:1 binding. Started before the drive and outside every pillar, so a retry's second + // attempt and a redirect's second hop are the same operation as the first. + const span = startOperationSpan(requestContext); + if (span === undefined) return await drive(); + return await runWithActiveSpan(span, () => driveWithSpan(span, drive)); + } finally { + contextStore.close(currentContext); // always the most recently installed context for this call. + } + } + + /** + * A no-op, deliberately (PIPE-27). The pipeline never OWNS its terminal transport, so closing the + * runtime must not close the transport a caller handed it and may still be using elsewhere. The + * method exists only to satisfy the `Transport` SPI, so a `Runtime` can be nested as another + * pipeline's transport (PIPE-26) without the outer one leaking a close through. + * + * @returns a promise that is already resolved. + */ + async close(): Promise<void> { + // PIPE-27: the pipeline never owns its transport and MUST NOT close it. + } + + /** + * The flattened step array, in the order the cursor drives it (PIPE-25). + * + * Frozen at construction, so the returned array is a read-only view and not a defensive copy — + * there is nothing a caller can mutate through it. + * + * @returns the ordered, immutable step array. + */ + get steps(): readonly StepDescriptor[] { + return this.#steps; // PIPE-25: "exposes a read-only, ordered view of its steps." + } + + /** + * The wrapped terminal transport. + * + * Exposed for `PipelineBuilder.seedFrom(runtime, 'flatten')` (PIPE-35), which must reuse this + * runtime's own transport as the seeded builder's terminal — flatten mode is not implementable + * without it. Read-only: the pipeline never owns its transport (PIPE-27), so there is nothing to + * copy defensively and nothing a caller can change by holding the reference. + * + * @returns the transport this pipeline dispatches to innermost. + */ + get transport(): Transport { + return this.#transport; + } +} + +/** + * The in-package construction hook for {@link Runtime}, whose own constructor is `private` so no + * consumer can build one around `PipelineBuilder.build()`'s pillar and ordering validation. + * + * Exported (still internal-only, still absent from the package barrel) for the same reason + * {@link exchangeSource} is: the sanctioned caller -- `PipelineBuilder.build()` -- lives in a + * different module, and TypeScript has no friend-class visibility to express that with. + * + * @param steps - the flattened, stage-ordered step array. Copied and frozen. + * @param transport - the terminal transport. Never closed by the pipeline (PIPE-27). + * @param contextInit - what each drive's context chain is built from: the `instrumentation` bundle + * whose `tracerFactory` supplies `OBS-29`'s per-operation span, the advisory `operationName` + * every promotion carries (CTX-16), and an optional `key` pinning two contexts to one store slot + * (CTX-5). Defaults to the no-op bundle, no operation name, and a fresh key. `PipelineBuilder`'s + * second constructor argument is the public way to supply the first two. + * @returns the built, immutable runtime. + * + * @internal + */ +export function createRuntime( + steps: readonly StepDescriptor[], + transport: Transport, + contextInit: RuntimeContextInit = {}, +): Runtime { + return create(steps, transport, contextInit); +} + +/** + * What `runtime` was built to carry into every call — the instrumentation bundle and the advisory + * operation name. The `key` slot of `ContextInit` rides along in the returned object when the + * in-package caller pinned one; `PipelineBuilder` never does. + * + * @param runtime - the built pipeline to read. + * @returns its context init, by reference. Not a copy: `createDispatchContext` and + * `promoteToRequest` only read it, and the object came from a caller that already owns it. + * + * @internal + */ +export function pipelineOptionsOf(runtime: Runtime): RuntimeContextInit { + return readContextInit(runtime); +} diff --git a/packages/core/src/pipeline/stage.test.ts b/packages/core/src/pipeline/stage.test.ts new file mode 100644 index 0000000..7a329b8 --- /dev/null +++ b/packages/core/src/pipeline/stage.test.ts @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/stage.test.ts +// Exercises: PIPE-2 (the mandatory chain, outermost pre-redirect slot through terminal SEND), PIPE-3 +// (pre/post extension slots around every pillar), PIPE-4 (exactly the 5 configurable pillars), PIPE-8 (SEND +// is the final, terminal stage) +import {describe, expect, test} from 'bun:test'; +import {PILLAR_STAGES, STAGE_ORDER} from './stage.js'; + +describe('STAGE_ORDER (PIPE-2, PIPE-3)', () => { + test('lists every stage exactly once, in declaration order', () => { + expect(STAGE_ORDER).toEqual([ + 'PRE_REDIRECT', + 'REDIRECT', + 'POST_REDIRECT', + 'PRE_RETRY', + 'RETRY', + 'POST_RETRY', + 'PRE_AUTH', + 'AUTH', + 'POST_AUTH', + 'PRE_LOGGING', + 'LOGGING', + 'POST_LOGGING', + 'PRE_SERDE', + 'SERDE', + 'POST_SERDE', + 'SEND', + ]); + expect(new Set(STAGE_ORDER).size).toBe(STAGE_ORDER.length); + }); + + test('PRE_REDIRECT is the outermost slot (PIPE-2)', () => { + expect(STAGE_ORDER.at(0)).toBe('PRE_REDIRECT'); + }); + + test('SEND is the terminal, final stage (PIPE-8)', () => { + expect(STAGE_ORDER.at(-1)).toBe('SEND'); + }); +}); + +describe('PILLAR_STAGES (PIPE-4)', () => { + test('is exactly REDIRECT, RETRY, AUTH, LOGGING, SERDE', () => { + expect([...PILLAR_STAGES].sort()).toEqual([ + 'AUTH', + 'LOGGING', + 'REDIRECT', + 'RETRY', + 'SERDE', + ]); + }); + + test('does not include SEND or any extension slot', () => { + expect(PILLAR_STAGES.has('SEND')).toBe(false); + expect(PILLAR_STAGES.has('PRE_REDIRECT')).toBe(false); + expect(PILLAR_STAGES.has('POST_LOGGING')).toBe(false); + }); +}); diff --git a/packages/core/src/pipeline/stage.ts b/packages/core/src/pipeline/stage.ts new file mode 100644 index 0000000..3b1599c --- /dev/null +++ b/packages/core/src/pipeline/stage.ts @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/stage.ts + +/** + * Fixed, totally-ordered pipeline stages (PIPE-1, PIPE-2). A string-literal union, not a TS `enum` -- + * `erasableSyntaxOnly` bars enums, and `Stage` has no behavior beyond ordering, which `STAGE_ORDER` alone + * provides. `PRE_REDIRECT` is the outermost slot PIPE-2 mandates; `POST_REDIRECT`..`POST_SERDE` are PIPE-3's + * SHOULD extension slots around every pillar. `SEND` is terminal and reserved -- PIPE-8, flattening skips it + * and `PipelineBuilder` rejects any attempt to install a step there. + * + * @public + */ +export type Stage = + | 'PRE_REDIRECT' + | 'REDIRECT' + | 'POST_REDIRECT' + | 'PRE_RETRY' + | 'RETRY' + | 'POST_RETRY' + | 'PRE_AUTH' + | 'AUTH' + | 'POST_AUTH' + | 'PRE_LOGGING' + | 'LOGGING' + | 'POST_LOGGING' + | 'PRE_SERDE' + | 'SERDE' + | 'POST_SERDE' + | 'SEND'; + +/** + * Declaration order (PIPE-1, PIPE-25): `PipelineBuilder.build()` flattens by walking this array. Inserting a + * further stage later is one splice here -- no existing `Stage` value needs to change, so there is no + * numeric-gap "renumbering" concern to design around. + * + * @public + */ +export const STAGE_ORDER: readonly Stage[] = [ + 'PRE_REDIRECT', + 'REDIRECT', + 'POST_REDIRECT', + 'PRE_RETRY', + 'RETRY', + 'POST_RETRY', + 'PRE_AUTH', + 'AUTH', + 'POST_AUTH', + 'PRE_LOGGING', + 'LOGGING', + 'POST_LOGGING', + 'PRE_SERDE', + 'SERDE', + 'POST_SERDE', + 'SEND', +]; + +/** A pillar stage admits at most one step (PIPE-4). @public */ +export const PILLAR_STAGES: ReadonlySet<Stage> = new Set([ + 'REDIRECT', + 'RETRY', + 'AUTH', + 'LOGGING', + 'SERDE', +]); diff --git a/packages/core/src/pipeline/step.ts b/packages/core/src/pipeline/step.ts new file mode 100644 index 0000000..f924572 --- /dev/null +++ b/packages/core/src/pipeline/step.ts @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/pipeline/step.ts +import type {ExecutionContext} from '../context/context.js'; +import type {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import type {Stage} from './stage.js'; + +/** + * Advances the pipeline once, optionally substituting a replacement request first (PIPE-14). `Request` + * values are immutable, so "substitute" means constructing a new one and passing it downstream -- the + * substitution sticks for every remaining step and the terminal dispatch for the rest of the current call. + * Calling with no argument carries the current request through unchanged. + * + * One-shot: each handle advances the chain exactly once (PIPE-15). A step that needs to re-drive the + * chain calls `ctx.fork()` again for a fresh handle rather than reusing this one. + * + * @throws CursorAlreadyAdvancedError -- as a rejected promise -- when an already-invoked handle is + * invoked a second time (PIPE-11/PIPE-15). + * + * @public + */ +export type Next = (request?: Request) => Promise<Response>; + +/** + * What a step receives on each invocation (PIPE-12). `fork` is present only when the invoking step occupies + * a pillar stage (PIPE-15/16); an ordinary step's `ctx.fork` is `undefined`. + * + * @public + */ +export interface StepContext { + /** + * Advances the chain exactly once (PIPE-14/PIPE-15). The ordinary way a step delegates downstream; + * a step that never calls it short-circuits the rest of the pipeline. + */ + readonly next: Next; + /** + * Mints a FRESH one-shot continuation, so a pillar step can drive the downstream chain more than + * once — retry's attempts, redirect's hops, auth's challenge replay (PIPE-15/PIPE-16). + * + * Present only when the invoking step occupies a pillar stage; `undefined` for an ordinary step. A + * step that forks more than once owns closing whatever response its own prior fork produced before + * forking again (PIPE-40). + */ + readonly fork?: (() => Next) | undefined; + /** + * This call's execution context, at whichever promotion stage the drive has reached (CTX-1). + * Branch on `context.kind` to tell which: `'dispatch'` before a request exists, `'request'` once + * one is assembled, `'exchange'` once a response has arrived. Shared by reference across every + * fork, so it is the same object on every attempt and every hop. + */ + readonly context: ExecutionContext; + /** + * The call's cancellation signal, threaded from the cursor (PIPE-13). Undefined when the caller + * supplied none. A pillar step that waits between drives (retry's backoff, auth's token fetch) + * MUST honor it (RETRY-26/RETRY-32). + */ + readonly signal?: AbortSignal | undefined; + /** + * The caller's per-call options, immutable and shared across every fork (PIPE-17: "readable by + * any step"). Undefined when the caller supplied none. The retry step reads `maxRetries` + * (RETRY-41/HTTP-35); the auth step reads the per-call auth descriptor (5c). + */ + readonly options?: RequestOptions | undefined; +} + +/** + * A pipeline step (PIPE-12): receives the inbound request, MAY invoke the rest of the chain via `ctx.next` + * (or `ctx.fork` to re-drive more than once), and MAY inspect or substitute the outbound response -- + * including short-circuiting by never calling `next` at all. + * + * A step that forks more than once owns closing whatever response its own prior fork produced before + * invoking `fork()` again (PIPE-40) -- that responsibility sits on the wrapping step, not on `Cursor`. + * + * @public + */ +export type Step = (request: Request, ctx: StepContext) => Promise<Response>; + +/** + * A registered step: its function plus the identity (`type`) PIPE-6's reference-identity pillar check and + * PIPE-18/19's anchor-type matching both key off, and the `stage` it occupies. + * + * @public + */ +export interface StepDescriptor { + /** + * This step's identity. PIPE-6's pillar-occupancy check and PIPE-18/PIPE-19's anchor matching both + * compare it by REFERENCE, so a factory must mint one module-level symbol and reuse it across every + * descriptor it produces — never a fresh `Symbol()` per call. + */ + readonly type: symbol; + /** The stage this step occupies. A pillar stage admits at most one step (PIPE-4/PIPE-5). */ + readonly stage: Stage; + /** The step itself (PIPE-12). */ + readonly fn: Step; +} diff --git a/packages/core/src/recovery/cancellation.test.ts b/packages/core/src/recovery/cancellation.test.ts new file mode 100644 index 0000000..c275b53 --- /dev/null +++ b/packages/core/src/recovery/cancellation.test.ts @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/cancellation.test.ts +// Exercises: RECOV-11 (wrapping a cancellation throwable into a Failure), reframed for Node — an +// AbortSignal is durable once aborted and the SDK never holds the caller's AbortController, so +// there is nothing to re-assert. What the requirement still buys is the guarantee that a +// cancellation surfaces through the SAME Failure channel as every other throwable, never through a +// side exit (RECOV-2). +import {describe, expect, test} from 'bun:test'; +import {CancellationError} from '../seams/transport.js'; +import {wrapCancellation} from './cancellation.js'; + +describe('wrapCancellation (RECOV-11)', () => { + test('wraps a CancellationError into a Failure carrying it unchanged', () => { + const error = new CancellationError('cancelled by caller'); + + const outcome = wrapCancellation(error); + + expect(outcome.kind).toBe('failure'); + expect(outcome.kind === 'failure' && outcome.error).toBe(error); + }); + + test('wraps an ordinary error into a Failure carrying it unchanged', () => { + const error = new Error('an ordinary failure'); + + const outcome = wrapCancellation(error); + + expect(outcome.kind).toBe('failure'); + expect(outcome.kind === 'failure' && outcome.error).toBe(error); + }); + + test('wraps a non-Error throw unchanged — a JS throw can raise any value', () => { + const outcome = wrapCancellation('a string throw'); + + expect(outcome.kind === 'failure' && outcome.error).toBe('a string throw'); + }); + + test('never throws, for any input', () => { + // RECOV-2 depends on this: dispatchWithRecovery calls it from inside its own catch, so a throw + // here would let a transport failure bypass the response and recovery chains entirely. + expect(() => wrapCancellation(new CancellationError('x'))).not.toThrow(); + expect(() => wrapCancellation(undefined)).not.toThrow(); + }); +}); diff --git a/packages/core/src/recovery/cancellation.ts b/packages/core/src/recovery/cancellation.ts new file mode 100644 index 0000000..e85bc7c --- /dev/null +++ b/packages/core/src/recovery/cancellation.ts @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/cancellation.ts +import {failure, type Outcome} from './outcome.js'; + +/** + * Wraps a cancellation or interruption throwable into a Failure (RECOV-11). + * + * The reference requires re-asserting the cancellation signal on the current context when wrapping, + * so code later blocked on the outcome still observes cancellation — a concern specific to a + * clearable `Thread.interrupt()` flag. Node has nothing to re-assert: an `AbortSignal` stays + * aborted once fired, and the SDK holds a signal, never the caller's `AbortController`, so it could + * not set one anyway. The helper therefore degenerates to `failure(error)`, and exists as the one + * named, findable site where RECOV-11's Node disposition lives. + * + * It deliberately does **not** crash on a `CancellationError` whose paired signal never aborted. + * `Transport` is a pluggable seam, so that mismatch is a misbehaving third-party implementation — + * an operational failure, not a violated precondition of this codebase, and crash-loud treatment is + * reserved for the latter (`docs/knowledge/harvested/error-handling.md`). It would also break RECOV-2: this + * runs inside `dispatchWithRecovery`'s own `catch`, so throwing here would let a transport failure + * skip the response and recovery chains entirely, which is precisely what RECOV-2 forbids. A + * transport that aborts its in-flight requests from `close()` — which SEAM-14 permits — produces + * exactly that shape while the caller passed no signal at all. + * + * This function never throws, for any input. + * + * @param error - whatever the request chain or the transport raised. + * @returns a failure outcome carrying `error` unchanged. + * + * @public + */ +export function wrapCancellation(error: unknown): Outcome<never> { + return failure(error); +} diff --git a/packages/core/src/recovery/idempotency-key.test.ts b/packages/core/src/recovery/idempotency-key.test.ts new file mode 100644 index 0000000..4f98213 --- /dev/null +++ b/packages/core/src/recovery/idempotency-key.test.ts @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/idempotency-key.test.ts +// Exercises: RECOV-32 (method gating, respect-existing default, strategy invoked at most once per +// applicable request, other methods untouched, defensive method-set copy). +import {describe, expect, test} from 'bun:test'; +import {Headers} from '../http/headers.js'; +import type {Method} from '../http/method.js'; +import {Request} from '../http/request.js'; +import {idempotencyKeyStep} from './idempotency-key.js'; + +function aRequest( + method: 'GET' | 'POST' | 'PUT' | 'PATCH', + existing?: string, +): Request { + const builder = Request.newBuilder() + .method(method) + .url('https://example.com'); + if (existing === undefined) return builder.build(); + return builder + .headers(Headers.newBuilder().add('Idempotency-Key', existing).build()) + .build(); +} + +describe('idempotencyKeyStep', () => { + test('stamps the default header on POST, PUT, and PATCH', async () => { + const step = idempotencyKeyStep({generate: () => 'generated'}); + for (const method of ['POST', 'PUT', 'PATCH'] as const) { + const stamped = await step(aRequest(method)); + expect(stamped.headers.get('Idempotency-Key')).toBe('generated'); + } + }); + + test('passes other methods through untouched', async () => { + const step = idempotencyKeyStep({generate: () => 'generated'}); + const request = aRequest('GET'); + expect(await step(request)).toBe(request); + }); + + test('respects an existing header by default and does NOT invoke the strategy', async () => { + let invocations = 0; + const step = idempotencyKeyStep({ + generate: () => { + invocations += 1; + return 'generated'; + }, + }); + const request = aRequest('POST', 'caller-supplied'); + + const result = await step(request); + + expect(result).toBe(request); + expect(invocations).toBe(0); + }); + + test('overwrites an existing header when respectExisting is false', async () => { + const step = idempotencyKeyStep({ + generate: () => 'generated', + respectExisting: false, + }); + const stamped = await step(aRequest('POST', 'caller-supplied')); + expect(stamped.headers.get('Idempotency-Key')).toBe('generated'); + }); + + test('invokes the strategy at most once per applicable request', async () => { + let invocations = 0; + const step = idempotencyKeyStep({ + generate: () => { + invocations += 1; + return `key-${String(invocations)}`; + }, + }); + + await step(aRequest('POST')); + + expect(invocations).toBe(1); + }); +}); + +describe('idempotencyKeyStep configuration (RECOV-32)', () => { + test('honors a configured header name and method set', async () => { + const step = idempotencyKeyStep({ + generate: () => 'generated', + headerName: 'X-Request-Id', + methods: new Set<Method>(['GET']), + }); + expect((await step(aRequest('GET'))).headers.get('X-Request-Id')).toBe( + 'generated', + ); + const post = aRequest('POST'); + expect(await step(post)).toBe(post); + }); + + test('defensively copies the method set', async () => { + const methods = new Set<Method>(['GET']); + const step = idempotencyKeyStep({generate: () => 'generated', methods}); + methods.add('POST'); + + const post = aRequest('POST'); + + expect(await step(post)).toBe(post); + }); + + test('never mutates the request it was given', async () => { + const step = idempotencyKeyStep({generate: () => 'generated'}); + const request = aRequest('POST'); + + await step(request); + + expect(request.headers.get('Idempotency-Key')).toBeUndefined(); + }); +}); diff --git a/packages/core/src/recovery/idempotency-key.ts b/packages/core/src/recovery/idempotency-key.ts new file mode 100644 index 0000000..e2b967f --- /dev/null +++ b/packages/core/src/recovery/idempotency-key.ts @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/idempotency-key.ts +import type {Method} from '../http/method.js'; +import type {Request} from '../http/request.js'; +import type {RequestStep} from './request-chain.js'; + +const DEFAULT_HEADER = 'Idempotency-Key'; +const DEFAULT_METHODS: readonly Method[] = ['POST', 'PUT', 'PATCH']; + +/** + * Everything {@link idempotencyKeyStep} accepts (RECOV-32). + * + * @public + */ +export interface IdempotencyKeyOptions { + /** The key strategy. Invoked at most once per applicable request (RECOV-32). */ + readonly generate: () => string; + /** + * The header to stamp. + * + * @defaultValue `'Idempotency-Key'` + */ + readonly headerName?: string | undefined; + /** Defaults to the non-idempotent write methods; defensively copied at construction. */ + readonly methods?: ReadonlySet<Method> | undefined; + /** When true (the default) a request already carrying the header is left entirely alone. */ + readonly respectExisting?: boolean | undefined; +} + +/** + * A `RequestStep` that stamps an idempotency key on write requests (RECOV-32). + * + * Runs ONCE per logical request, upstream of retry -- not per attempt. `retry/attempt-stamp.ts` is + * its sibling: that one writes the attempt ordinal on each per-attempt copy and preserves whatever + * this wrote (RETRY-38), so the server sees one stable key across every retry of the same logical + * request. + * + * **That is a property of the composition, and the SDK's own retry adapter is what supplies it**: + * `retry/retry-dispatch.ts` applies the `RequestRecoveryChain` once, above the retry loop, and each + * attempt re-sends a copy of the request it produced. On its own a step can only promise RECOV-32's + * letter -- `generate()` is invoked at most once per *application* to an applicable request -- so a + * caller who re-applies their own chain per attempt will get a fresh key per attempt. Install the + * chain once and let the retry layer sit below it. + * + * @param options - the key strategy plus the header name, method set, and existing-key policy. + * @returns the request step to install in a `RequestRecoveryChain`. + * + * @public + */ +export function idempotencyKeyStep( + options: IdempotencyKeyOptions, +): RequestStep { + const headerName = options.headerName ?? DEFAULT_HEADER; + const methods = new Set<Method>(options.methods ?? DEFAULT_METHODS); + const respectExisting = options.respectExisting ?? true; + + return (request: Request): Promise<Request> => { + if (!methods.has(request.method)) return Promise.resolve(request); + if (respectExisting && request.headers.get(headerName) !== undefined) { + return Promise.resolve(request); + } + return Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set(headerName, options.generate()) + .build(), + ) + .build(), + ); + }; +} diff --git a/packages/core/src/recovery/orchestrator.test.ts b/packages/core/src/recovery/orchestrator.test.ts new file mode 100644 index 0000000..aae4ec0 --- /dev/null +++ b/packages/core/src/recovery/orchestrator.test.ts @@ -0,0 +1,370 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/orchestrator.test.ts +// Exercises: RECOV-2 (one try/catch wraps the request chain AND the transport invocation; no +// throwable from either bypasses the recovery hooks), RECOV-10 (unwrap: a Success returns the +// response, a Failure rethrows the throwable unchanged, no wrapping or substitution), RECOV-11 (the +// catch routes every throwable through wrapCancellation, so the helper sits on the real dispatch +// path rather than being an unwired primitive) +import {describe, expect, test} from 'bun:test'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {RequestOptions} from '../http/request-options.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {CancellationError, type Transport} from '../seams/transport.js'; +import {dispatchWithRecovery} from './orchestrator.js'; +import {success} from './outcome.js'; +import {RequestRecoveryChain, type RequestStep} from './request-chain.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, + type ResponseStep, +} from './response-chain.js'; + +function aRequest(): Request { + return Request.newBuilder().url('https://example.com').build(); +} + +function aResponse( + request: Request, + body: ReadableStream<Uint8Array> | null = null, +): Response { + return Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .body(body) + .build(); +} + +/** Close is observed through the body stream's `cancel()` — `Response` is frozen. */ +function countingCloseBody(): { + body: ReadableStream<Uint8Array>; + closeCount: () => number; +} { + let cancels = 0; + return { + body: new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + cancels += 1; + }, + }), + closeCount: () => cancels, + }; +} + +/** Awaits `promise`, returning whatever it rejected with — `undefined` when it resolved. */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return undefined; + } catch (error) { + return error; + } +} + +type SendImpl = ( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, +) => Promise<Response>; + +/** A minimal, file-local Transport stub — no shared FakeTransport exists yet. */ +class StubTransport implements Transport { + readonly #impl: SendImpl; + + constructor(impl: SendImpl) { + this.#impl = impl; + } + + send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + return this.#impl(request, options, signal); + } + + close(): Promise<void> { + return Promise.resolve(); + } +} + +function emptyChains(): { + requestChain: RequestRecoveryChain; + responseChain: ResponseRecoveryChain; +} { + return { + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([], []), + }; +} + +describe('dispatchWithRecovery happy path', () => { + test('returns the transport response when everything succeeds', async () => { + const request = aRequest(); + const response = aResponse(request); + const transport = new StubTransport(() => Promise.resolve(response)); + + const result = await dispatchWithRecovery(request, { + transport, + ...emptyChains(), + }); + + expect(result).toBe(response); + }); + + test('sends the request the request chain produced, not the one the caller handed in', async () => { + const tagStep: RequestStep = request => + Promise.resolve( + request + .newBuilder() + .headers( + request.headers.newBuilder().set('X-Trace', 'tagged').build(), + ) + .build(), + ); + let sent: Request | undefined; + const transport = new StubTransport(request => { + sent = request; + return Promise.resolve(aResponse(request)); + }); + + await dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([tagStep]), + responseChain: new ResponseRecoveryChain([], []), + }); + + expect(sent?.headers.get('X-Trace')).toBe('tagged'); + }); + + test('threads per-call options and signal through to the transport unchanged', async () => { + const request = aRequest(); + const options = RequestOptions.EMPTY; + const controller = new AbortController(); + let receivedOptions: RequestOptions | undefined; + let receivedSignal: AbortSignal | undefined; + const transport = new StubTransport((req, opts, signal) => { + receivedOptions = opts; + receivedSignal = signal; + return Promise.resolve(aResponse(req)); + }); + + await dispatchWithRecovery(request, { + transport, + ...emptyChains(), + options, + signal: controller.signal, + }); + + expect(receivedOptions).toBe(options); + expect(receivedSignal).toBe(controller.signal); + }); +}); + +describe('RECOV-2: every throwable from the request chain or the transport is caught', () => { + test('a throwing request step surfaces as a Failure to a recovery hook, not an unhandled throw', async () => { + const thrownError = new Error('request step failed'); + const failingStep: RequestStep = () => { + throw thrownError; + }; + const seenByRecovery: unknown[] = []; + const recoveryStep: RecoveryStep = outcome => { + if (outcome.kind === 'failure') seenByRecovery.push(outcome.error); + return Promise.resolve(outcome); + }; + const transport = new StubTransport(() => { + throw new Error('must not run — the request chain already failed'); + }); + + const error = await rejection( + dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([failingStep]), + responseChain: new ResponseRecoveryChain([], [recoveryStep]), + }), + ); + + expect(error).toBe(thrownError); + expect(seenByRecovery).toEqual([thrownError]); + }); + + test('a throwing transport surfaces as a Failure to a recovery hook, not an unhandled throw', async () => { + const thrownError = new Error('transport failed'); + const seenByRecovery: unknown[] = []; + const recoveryStep: RecoveryStep = outcome => { + if (outcome.kind === 'failure') seenByRecovery.push(outcome.error); + return Promise.resolve(outcome); + }; + const transport = new StubTransport(() => { + throw thrownError; + }); + + const error = await rejection( + dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([], [recoveryStep]), + }), + ); + + expect(error).toBe(thrownError); + expect(seenByRecovery).toEqual([thrownError]); + }); + + test('a recovery step can turn a transport failure back into a Success', async () => { + const request = aRequest(); + const fallback = aResponse(request); + const recoverStep: RecoveryStep = outcome => + Promise.resolve(outcome.kind === 'failure' ? success(fallback) : outcome); + const transport = new StubTransport(() => { + throw new Error('transport failed'); + }); + + const result = await dispatchWithRecovery(request, { + transport, + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([], [recoverStep]), + }); + + expect(result).toBe(fallback); + }); +}); + +describe('RECOV-10: the final unwrap is unchanged, no wrapping or substitution', () => { + test('a response step throwing a typed error surfaces exactly that error, by identity', async () => { + class MyTypedError extends Error {} + const typedError = new MyTypedError('mapped'); + const mapToTypedError: ResponseStep = () => { + throw typedError; + }; + const transport = new StubTransport(request => + Promise.resolve(aResponse(request)), + ); + + const error = await rejection( + dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([mapToTypedError], []), + }), + ); + + expect(error).toBe(typedError); + }); + + test('a non-Error throwable is rethrown as-is, not coerced into an Error', async () => { + const transport = new StubTransport(() => { + // The point of the case: a JS throw can legally raise any value, and RECOV-10 requires the + // orchestrator to rethrow it by identity rather than coercing it into an Error. Re-enable if + // the transport seam ever narrows what an implementation may reject with. + // eslint-disable-next-line @typescript-eslint/only-throw-error -- see the comment above + throw 'a string throw'; + }); + + const error = await rejection( + dispatchWithRecovery(aRequest(), {transport, ...emptyChains()}), + ); + + expect(error).toBe('a string throw'); + }); +}); + +describe('RECOV-11: the catch routes every throwable through wrapCancellation', () => { + test('a transport CancellationError paired with an aborted signal surfaces unchanged', async () => { + const controller = new AbortController(); + const cancellation = new CancellationError('aborted by caller'); + const transport = new StubTransport(() => { + controller.abort(); + throw cancellation; + }); + + const error = await rejection( + dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([], []), + signal: controller.signal, + }), + ); + + expect(error).toBe(cancellation); + }); + + test('a CancellationError raised with no caller signal still reaches the recovery chain (RECOV-2)', async () => { + // A transport may abort its own in-flight requests for reasons the caller never signalled — + // SEAM-14 permits close() to cancel them. That must surface as an ordinary Failure through the + // recovery hooks, not as a side exit: RECOV-2 admits no throwable from the transport bypassing + // them. + const cancellation = new CancellationError( + 'aborted by the transport itself', + ); + const seenByRecovery: unknown[] = []; + const recoveryStep: RecoveryStep = outcome => { + if (outcome.kind === 'failure') seenByRecovery.push(outcome.error); + return Promise.resolve(outcome); + }; + const transport = new StubTransport(() => { + throw cancellation; + }); + + const error = await rejection( + dispatchWithRecovery(aRequest(), { + transport, + requestChain: new RequestRecoveryChain([]), + responseChain: new ResponseRecoveryChain([], [recoveryStep]), + }), + ); + + expect(error).toBe(cancellation); + expect(seenByRecovery).toEqual([cancellation]); + }); +}); + +describe('negative space: the orchestrator releases nothing of its own', () => { + test('the response it hands back is left open for the caller to close', async () => { + // RECOV-10 returns the contained response; ownership passes to the caller. A future + // "helpful" close here would hand back a response whose body is already cancelled. + const request = aRequest(); + const {body, closeCount} = countingCloseBody(); + const transport = new StubTransport(() => + Promise.resolve(aResponse(request, body)), + ); + + const result = await dispatchWithRecovery(request, { + transport, + ...emptyChains(), + }); + + expect(closeCount()).toBe(0); + expect(result.body).not.toBeNull(); + }); + + test('it never closes the transport it was handed', async () => { + // SEAM-14/PIPE-27's discipline, asserted early: the orchestrator borrows the transport, it + // does not own it. + let closeCalls = 0; + const request = aRequest(); + const transport = new StubTransport(() => + Promise.resolve(aResponse(request)), + ); + const countingTransport: Transport = { + send: (...args) => transport.send(...args), + close: () => { + closeCalls += 1; + return Promise.resolve(); + }, + }; + + await dispatchWithRecovery(request, { + transport: countingTransport, + ...emptyChains(), + }); + + expect(closeCalls).toBe(0); + }); +}); diff --git a/packages/core/src/recovery/orchestrator.ts b/packages/core/src/recovery/orchestrator.ts new file mode 100644 index 0000000..16e9645 --- /dev/null +++ b/packages/core/src/recovery/orchestrator.ts @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/orchestrator.ts +import type {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import type {Transport} from '../seams/transport.js'; +import {wrapCancellation} from './cancellation.js'; +import {fold, success, type Outcome} from './outcome.js'; +import type {RequestRecoveryChain} from './request-chain.js'; +import type {ResponseRecoveryChain} from './response-chain.js'; + +/** + * Everything {@link dispatchWithRecovery} needs beyond the request itself, bundled into one + * trailing object. Five positional parameters would fail ESLint's `max-params: 3`. + * + * @public + */ +export interface DispatchConfig { + /** The terminal transport hop. */ + readonly transport: Transport; + /** Run before the transport hop; its throwables become a Failure (RECOV-2). */ + readonly requestChain: RequestRecoveryChain; + /** Run on the outcome, whatever it is (RECOV-4 … RECOV-8). */ + readonly responseChain: ResponseRecoveryChain; + /** Per-call operational overrides, threaded to the transport unchanged. */ + readonly options?: RequestOptions | undefined; + /** The caller's abort signal, threaded to the transport unchanged. */ + readonly signal?: AbortSignal | undefined; +} + +/** + * {@link DispatchConfig} without the request chain: everything the phases BELOW that chain need. + * + * A retry loop drives {@link dispatchPrepared} through this, having already run the request chain + * once above itself. Naming it as a subset rather than duplicating the fields keeps the two shapes + * from drifting when `DispatchConfig` grows. + * + * @internal + */ +export type PreparedDispatchConfig = Omit<DispatchConfig, 'requestChain'>; + +/** + * The request chain, run ONCE per logical request, with RECOV-2's conversion already applied: a + * throwing step becomes a Failure here rather than propagating, so no caller has to catch it. That + * conversion goes through {@link wrapCancellation} (RECOV-11); RECOV-2's guarantee rests on that + * helper never throwing, since this `catch` is the last place a request-chain throwable could + * escape without meeting the recovery hooks. + * + * Split out of {@link dispatchWithRecovery} on 2026-09-05 so `retry/retry-dispatch.ts` can run this + * half once for a whole logical request and repeat only the half below it per attempt. Before that + * split every retry attempt re-ran the request chain, and `recovery/idempotency-key.ts` generated a + * fresh key on each one — three attempts of one logical request reached the server as three + * distinct idempotency keys, which is exactly what RECOV-32's key is bought to prevent. + * + * @param request - the request to prepare. + * @param requestChain - the ordered request steps (RECOV-3). + * @returns a Success carrying the prepared request, or a Failure carrying whatever a step threw. + * Never throws, for any input. + * + * @internal + */ +export async function prepareRequest( + request: Request, + requestChain: RequestRecoveryChain, +): Promise<Outcome<Request>> { + try { + return success(await requestChain.apply(request)); + } catch (error) { + return wrapCancellation(error); + } +} + +/** + * The transport hop, with RECOV-2's conversion applied to whatever it throws. + * + * A `prepared` that is already a Failure short-circuits it: the transport is not called, and the + * failure is handed on for the response chain to see. Both branches widen `Outcome<Request>` and + * `Outcome<never>` to `Outcome<Response>` without a cast, because the failure variant does not + * mention the type parameter — and neither branch throws, which is what keeps RECOV-2 absolute. + */ +async function sendPrepared( + prepared: Outcome<Request>, + config: PreparedDispatchConfig, +): Promise<Outcome<Response>> { + if (prepared.kind === 'failure') return prepared; + try { + return success( + await config.transport.send( + prepared.value, + config.options, + config.signal, + ), + ); + } catch (error) { + return wrapCancellation(error); + } +} + +/** + * Everything below the request chain — the transport hop, the response chain, and RECOV-10's + * terminal unwrap. This is the part a retry loop repeats, once per wire send (RETRY-44's + * "downstream chain"). + * + * It takes {@link prepareRequest}'s outcome rather than a bare `Request` because a request-chain + * failure still owes RECOV-2 a trip through the response and recovery chains before it surfaces. + * On that input the transport is not called at all, which is the whole difference between the two + * variants. + * + * @param prepared - {@link prepareRequest}'s result for this logical request. + * @param config - transport, response chain, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @internal + */ +export async function dispatchPrepared( + prepared: Outcome<Request>, + config: PreparedDispatchConfig, +): Promise<Response> { + const finalOutcome = await config.responseChain.apply( + await sendPrepared(prepared, config), + ); + return fold( + finalOutcome, + response => response, + error => { + throw error; + }, + ); +} + +/** + * The unified recovery-chain orchestrator (RECOV-2, RECOV-10, RECOV-11). + * + * The two halves it composes are named: `prepareRequest` runs the request chain, and + * `dispatchPrepared` runs the transport hop and the response chain. Neither is exported from the + * package, so both are backticked rather than `{@link}`ed — api-extractor cannot resolve a + * reference out of the published surface into one, and the unresolved link is an error, not a + * warning to live with. Every throwable from either half is caught and converted into a Failure + * before the response chain runs — a before-request throw cannot skip after-error handling. + * + * The final unwrap returns the response on a Success, or rethrows the Failure's throwable + * **unchanged** — no wrapping, no substitution (RECOV-10). Surfacing a typed exception is a + * recovery step's own responsibility, never this function's. + * + * **One dispatch is one wire send.** Nothing here retries; a caller that wants retries composes the + * two halves itself so that the request chain runs once and only the second half repeats + * (`retry/retry-dispatch.ts`). + * + * @param request - the request to prepare and send. + * @param config - transport, chains, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @public + */ +export async function dispatchWithRecovery( + request: Request, + config: DispatchConfig, +): Promise<Response> { + return dispatchPrepared( + await prepareRequest(request, config.requestChain), + config, + ); +} diff --git a/packages/core/src/recovery/outcome.test.ts b/packages/core/src/recovery/outcome.test.ts new file mode 100644 index 0000000..3b3e9a3 --- /dev/null +++ b/packages/core/src/recovery/outcome.test.ts @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/outcome.test.ts +// Exercises: RECOV-1 (closed two-variant sum type, mutually exclusive and jointly exhaustive, with a +// fold that applies exactly one of two branches at most once per call) +import {describe, expect, test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import fc from 'fast-check'; +import {failure, fold, success, type Outcome} from './outcome.js'; + +describe('success / failure (RECOV-1)', () => { + test('success carries its value under kind "success"', () => { + const outcome = success(42); + + expect(outcome.kind).toBe('success'); + expect(outcome.kind === 'success' && outcome.value).toBe(42); + }); + + test('failure carries its error under kind "failure", typed unknown', () => { + // A JS throw can legally raise any value, not only an Error (sdk-design-nodejs/05). + const outcome = failure('a string throw'); + + expect(outcome.kind).toBe('failure'); + expect(outcome.kind === 'failure' && outcome.error).toBe('a string throw'); + }); +}); + +describe('fold (RECOV-1)', () => { + test('applies onSuccess for a success outcome', () => { + const result = fold( + success(10), + v => v * 2, + () => -1, + ); + + expect(result).toBe(20); + }); + + test('applies onFailure for a failure outcome', () => { + const error = new Error('boom'); + + const result = fold( + failure<number>(error), + () => 'unreachable', + e => e, + ); + + expect(result).toBe(error); + }); + + test('invokes exactly one branch, never both, for either variant', () => { + let successCalls = 0; + let failureCalls = 0; + const onSuccess = (): string => { + successCalls += 1; + return 'ok'; + }; + const onFailure = (): string => { + failureCalls += 1; + return 'err'; + }; + + fold(success(1), onSuccess, onFailure); + fold(failure<number>(new Error('x')), onSuccess, onFailure); + + expect(successCalls).toBe(1); + expect(failureCalls).toBe(1); + }); +}); + +describe('fold identity law (RECOV-1)', () => { + // Canonical law for an invariant-bearing function (docs/knowledge/harvested/testing.md): folding a success + // through the identity success-handler, and a failure through the identity failure-handler, must + // each recover the original payload, for arbitrary values. + test('fold(success(x), id, _) === x for arbitrary x', () => { + fc.assert( + fc.property(fc.anything(), value => { + expect( + fold( + success(value), + v => v, + () => 'unreachable', + ), + ).toBe(value); + }), + ); + }); + + test('fold(failure(e), _, id) === e for arbitrary e', () => { + fc.assert( + fc.property(fc.anything(), error => { + expect( + fold( + failure<unknown>(error), + () => 'unreachable', + e => e, + ), + ).toBe(error); + }), + ); + }); +}); + +describe('Outcome<T> as a type (RECOV-1)', () => { + // An exported generic type ships with a type-level test (styleguide 11.6). These only fire under + // `bun run typecheck` — `bun test` executes this file but strips its types without checking them. + test('the two variants are closed and jointly exhaustive', () => { + expectTypeOf<Outcome<number>['kind']>().toEqualTypeOf< + 'success' | 'failure' + >(); + }); + + test('narrowing on kind reaches the variant payload, and only that payload', () => { + expectTypeOf< + Extract<Outcome<number>, {kind: 'success'}>['value'] + >().toEqualTypeOf<number>(); + expectTypeOf< + Extract<Outcome<number>, {kind: 'failure'}>['error'] + >().toEqualTypeOf<unknown>(); + }); + + test('a narrowed success has no error field (negative case)', () => { + const outcome: Outcome<number> = success(1); + if (outcome.kind !== 'success') throw new Error('unreachable seed'); + + // @ts-expect-error -- RECOV-1: the variants are mutually exclusive, so a narrowed success has + // no `error` to read. If this line ever compiles, the union has stopped being closed. + const read: unknown = outcome.error; + + expect(read).toBeUndefined(); + }); + + test('a narrowed failure has no value field (negative case)', () => { + const outcome: Outcome<number> = failure(new Error('x')); + if (outcome.kind !== 'failure') throw new Error('unreachable seed'); + + // @ts-expect-error -- the mirror of the case above. + const read: unknown = outcome.value; + + expect(read).toBeUndefined(); + }); + + test('fold collapses both branches to one result type', () => { + expectTypeOf( + fold( + success(1), + v => v, + () => 0, + ), + ).toEqualTypeOf<number>(); + }); + + test('failure() infers the caller-declared payload type, not the error type', () => { + expectTypeOf(failure<string>(new Error('x'))).toEqualTypeOf< + Outcome<string> + >(); + }); +}); diff --git a/packages/core/src/recovery/outcome.ts b/packages/core/src/recovery/outcome.ts new file mode 100644 index 0000000..1f50594 --- /dev/null +++ b/packages/core/src/recovery/outcome.ts @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/outcome.ts +import {assertNever} from '../invariant.js'; + +/** + * The recovery chain's closed two-variant outcome (RECOV-1): a success carrying a value, or a + * failure carrying whatever was thrown. + * + * `error` is `unknown`, not `Error` — a JavaScript `throw` can legally raise any value, and this + * type sits directly under a `catch`. The discriminated union is what RECOV-1's "derivable + * accessors" buys in TypeScript: narrowing on `kind` is compiler-checked, so no `isSuccess()` / + * `getOrThrow()` pair is shipped. + * + * @public + */ +export type Outcome<T> = + | {readonly kind: 'success'; readonly value: T} + | {readonly kind: 'failure'; readonly error: unknown}; + +/** + * Builds the success variant. + * + * @param value - the value carried by the outcome. + * @returns a success outcome holding `value`. + * + * @public + */ +export function success<T>(value: T): Outcome<T> { + return {kind: 'success', value}; +} + +/** + * Builds the failure variant. + * + * @param error - whatever was thrown; any value, not necessarily an `Error`. + * @returns a failure outcome holding `error`. + * + * @public + */ +export function failure<T>(error: unknown): Outcome<T> { + return {kind: 'failure', error}; +} + +/** + * Applies exactly one of `onSuccess` / `onFailure`, never both, satisfying RECOV-1's "a fold that + * applies exactly one of two branches at most once per call." + * + * Three positional parameters rather than an options object: `max-params` errors at four, and this + * matches Phase 2's already-shipped `Transport.send(request, options?, signal?)`. Recorded as a + * corpus deviation in the phase design's ledger. + * + * @param outcome - the outcome to fold. + * @param onSuccess - applied to the value of a success outcome. + * @param onFailure - applied to the error of a failure outcome. + * @returns whichever branch ran. + * + * @public + */ +export function fold<T, R>( + outcome: Outcome<T>, + onSuccess: (value: T) => R, + onFailure: (error: unknown) => R, +): R { + switch (outcome.kind) { + case 'success': + return onSuccess(outcome.value); + case 'failure': + return onFailure(outcome.error); + default: + return assertNever(outcome); + } +} diff --git a/packages/core/src/recovery/release.test.ts b/packages/core/src/recovery/release.test.ts new file mode 100644 index 0000000..1267788 --- /dev/null +++ b/packages/core/src/recovery/release.test.ts @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/release.test.ts +// Exercises: RECOV-12 (a teardown failure rides along as `suppressed` and never becomes primary), +// RETRY-22 and REDIR-22's shared consequence — the error that must propagate is the upstream/decision +// failure, not the release that ran on its way out. Extracted from `retry/engine.ts` in Phase 5b so the +// redirect step consumes it rather than shipping a second copy. +import {describe, expect, test} from 'bun:test'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {releaseQuietly, withReleaseFailure} from './release.js'; + +const REQUEST = Request.newBuilder().url('https://example.com').build(); + +/** `cancel` decides the release outcome: `undefined` releases cleanly, an `Error` is rethrown by close. */ +function responseWith(cancelFailure?: Error): Response { + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + if (cancelFailure !== undefined) throw cancelFailure; + }, + }); + return Response.newBuilder() + .request(REQUEST) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers(Headers.newBuilder().build()) + .body(body) + .build(); +} + +describe('releaseQuietly', () => { + test('reports a clean release with a token withReleaseFailure treats as "nothing happened"', async () => { + const primary = new Error('upstream'); + const token = await releaseQuietly(responseWith()); + expect(withReleaseFailure(primary, token)).toBe(primary); + }); + + test('an absent response releases cleanly', async () => { + const primary = new Error('upstream'); + const token = await releaseQuietly(undefined); + expect(withReleaseFailure(primary, token)).toBe(primary); + }); + + test('reports rather than raises whatever close() threw', async () => { + const boom = new Error('cancel exploded'); + expect(await releaseQuietly(responseWith(boom))).toBe(boom); + }); + + test('a locked-stream TypeError is swallowed by close() itself, so the release reads clean', async () => { + const response = responseWith(); + const body = response.body; + expect(body).not.toBeNull(); + body?.getReader(); // hold the lock: cancel() now rejects with TypeError + const primary = new Error('upstream'); + expect(withReleaseFailure(primary, await releaseQuietly(response))).toBe( + primary, + ); + }); +}); + +describe('withReleaseFailure', () => { + test('keeps the primary primary and carries the release failure as suppressed (RECOV-12)', async () => { + const primary = new Error('upstream'); + const boom = new Error('cancel exploded'); + + const result = withReleaseFailure( + primary, + await releaseQuietly(responseWith(boom)), + ); + + expect(result).not.toBe(primary); + const suppressed = result as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect(suppressed.error).toBe(primary); + expect(suppressed.suppressed).toBe(boom); + }); + + test('an identical instance is never suppressed under itself', () => { + // `Response.close()` memoizes its release promise, so a close that already failed hands the SAME + // rejection back to a second caller. Without the identity guard that instance wraps itself. + const shared = new Error('same instance twice'); + expect(withReleaseFailure(shared, shared)).toBe(shared); + }); + + test('a non-Error primary survives unchanged', async () => { + const boom = new Error('cancel exploded'); + const result = withReleaseFailure( + 'a bare string throw', + await releaseQuietly(responseWith(boom)), + ); + expect((result as SuppressedErrorLike).error).toBe('a bare string throw'); + }); +}); diff --git a/packages/core/src/recovery/release.ts b/packages/core/src/recovery/release.ts new file mode 100644 index 0000000..8848ebf --- /dev/null +++ b/packages/core/src/recovery/release.ts @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/release.ts +import type {Response} from '../http/response.js'; +import {suppress} from '../suppress.js'; + +/** Marks "the response was released without incident", distinct from any value `close()` could throw. */ +const RELEASED_CLEANLY = Symbol('dexpace.recovery.released'); + +/** + * Releases a discarded response, reporting rather than raising whatever release itself threw. + * + * `Response.close()` is documented to rethrow whatever cancelling the body raises (everything except + * the `TypeError` a locked stream reports), so it is not a call that can sit in a bare `finally`: + * there it would replace the value being returned, or replace an in-flight throwable with the + * teardown failure -- the exact inversion RECOV-12 forbids and `suppress()` exists to prevent. + * + * @param response - the response to release, or `undefined` when there is none. + * @returns an opaque release token for {@link withReleaseFailure}: whatever `close()` threw, or a + * sentinel meaning it released cleanly. + * + * @internal + */ +export async function releaseQuietly( + response: Response | undefined, +): Promise<unknown> { + if (response === undefined) return RELEASED_CLEANLY; + try { + await response.close(); + return RELEASED_CLEANLY; + } catch (error) { + return error; + } +} + +/** + * Whether {@link releaseQuietly}'s token means "released without incident". + * + * For a caller whose primary is a RETURNED value rather than a throwable, {@link withReleaseFailure} + * has nothing to pair the failure with -- `toHttpError` returns its `HttpStatusError` -- so it needs + * the plain question instead, and the answer must not be re-derived against this module's private + * sentinel from outside it. + * + * @param releaseToken - the token {@link releaseQuietly} returned. + * @returns `true` when the release was clean. + * + * @internal + */ +export function releasedCleanly(releaseToken: unknown): boolean { + return releaseToken === RELEASED_CLEANLY; +} + +/** + * Keeps `primary` primary, with a release failure riding along as suppressed (RECOV-12, RETRY-22's + * "a teardown failure can never mask the upstream failure"; REDIR-22's equivalent, where the error + * that must propagate is the decision failure, not the teardown that ran on its way out). + * + * The identity guard is not decorative. `Response.close()` memoizes its release promise, so a close + * that already failed inside `toHttpError`'s own `finally` hands the SAME rejection back to the + * second caller -- without this check that instance would be suppressed under itself. + * + * @param primary - the throwable the caller actually needs to see. + * @param releaseFailure - the token {@link releaseQuietly} returned. + * @returns `primary` unchanged when the release was clean, otherwise a `SuppressedError`-shaped + * pairing with `primary` primary. + * + * @internal + */ +export function withReleaseFailure( + primary: unknown, + releaseFailure: unknown, +): unknown { + if (releaseFailure === RELEASED_CLEANLY || releaseFailure === primary) { + return primary; + } + return suppress( + primary, + releaseFailure, + 'releasing the discarded response failed', + ); +} diff --git a/packages/core/src/recovery/request-chain.test.ts b/packages/core/src/recovery/request-chain.test.ts new file mode 100644 index 0000000..a63a69a --- /dev/null +++ b/packages/core/src/recovery/request-chain.test.ts @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/request-chain.test.ts +// Exercises: RECOV-3 (sequential left-to-right fold, empty chain is the identity, a throwing step +// aborts the remainder and propagates), RECOV-14 (defensive copy of the step list at construction) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {Request} from '../http/request.js'; +import {RequestRecoveryChain, type RequestStep} from './request-chain.js'; + +function aRequest(): Request { + return Request.newBuilder().url('https://example.com').build(); +} + +function tagAppendStep(char: string): RequestStep { + return request => { + const current = request.headers.get('X-Trace') ?? ''; + return Promise.resolve( + request + .newBuilder() + .headers( + request.headers + .newBuilder() + .set('X-Trace', current + char) + .build(), + ) + .build(), + ); + }; +} + +/** Awaits `promise`, returning whatever it rejected with — `undefined` when it resolved. */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return undefined; + } catch (error) { + return error; + } +} + +describe('RequestRecoveryChain.apply (RECOV-3)', () => { + test('an empty chain returns the input unchanged', async () => { + const chain = new RequestRecoveryChain([]); + const request = aRequest(); + + const result = await chain.apply(request); + + expect(result).toBe(request); + }); + + test('applies steps as a sequential left-to-right fold', async () => { + const chain = new RequestRecoveryChain([ + tagAppendStep('a'), + tagAppendStep('b'), + tagAppendStep('c'), + ]); + + const result = await chain.apply(aRequest()); + + expect(result.headers.get('X-Trace')).toBe('abc'); + }); + + test('a throwing step aborts the remainder and propagates', async () => { + const reached: string[] = []; + const thrownError = new Error('step failed'); + const failingStep: RequestStep = () => { + throw thrownError; + }; + const laterStep: RequestStep = request => { + reached.push('later'); + return Promise.resolve(request); + }; + const chain = new RequestRecoveryChain([ + tagAppendStep('a'), + failingStep, + laterStep, + ]); + + expect(await rejection(chain.apply(aRequest()))).toBe(thrownError); + expect(reached).toEqual([]); + }); +}); + +describe('RequestRecoveryChain construction (RECOV-14)', () => { + test('defensively copies its step list — mutating the source after construction has no effect', async () => { + const steps: RequestStep[] = [tagAppendStep('a')]; + const chain = new RequestRecoveryChain(steps); + steps.push(tagAppendStep('b')); + + const result = await chain.apply(aRequest()); + + expect(result.headers.get('X-Trace')).toBe('a'); + }); +}); + +describe('RequestRecoveryChain.apply fold law', () => { + // Canonical law for an invariant-bearing function: applying the chain equals manually reducing + // the same steps in order, for an arbitrary sequence of single-character append steps. + test('apply() equals a manual left-to-right reduce, for arbitrary step sequences', async () => { + await fc.assert( + // `fc.string({minLength: 1, maxLength: 1})` rather than `fc.char()`: the latter is deprecated + // in fast-check 3.22+ and would print a deprecation warning on every run. + fc.asyncProperty( + fc.array(fc.string({minLength: 1, maxLength: 1}), {maxLength: 10}), + async chars => { + const steps = chars.map(tagAppendStep); + const chain = new RequestRecoveryChain(steps); + + const chained = await chain.apply(aRequest()); + let manual = aRequest(); + for (const step of steps) manual = await step(manual); + + expect(chained.headers.get('X-Trace')).toBe( + manual.headers.get('X-Trace'), + ); + }, + ), + ); + }); +}); + +describe('RECOV-14: steps are safe for concurrent invocation', () => { + // RECOV-14's second normative clause binds both chains, not only the response one: per-request + // state lives in the value being transformed, never on the step or the chain instance. Guards + // the structural property that `apply()`'s only per-call state is its `current` local. + test('two interleaved apply() calls on ONE chain instance do not observe each other', async () => { + const gate: (() => void)[] = []; + const slowStep: RequestStep = async request => { + await new Promise<void>(resolve => gate.push(resolve)); + return request; + }; + const chain = new RequestRecoveryChain([slowStep, tagAppendStep('x')]); + const first = Request.newBuilder().url('https://example.com/first').build(); + const second = Request.newBuilder() + .url('https://example.com/second') + .build(); + + const firstCall = chain.apply(first); + const secondCall = chain.apply(second); + await Promise.resolve(); + for (const release of gate) release(); + const [firstResult, secondResult] = await Promise.all([ + firstCall, + secondCall, + ]); + + expect(firstResult.url.pathname).toBe('/first'); + expect(secondResult.url.pathname).toBe('/second'); + expect(firstResult.headers.get('X-Trace')).toBe('x'); + expect(secondResult.headers.get('X-Trace')).toBe('x'); + }); +}); diff --git a/packages/core/src/recovery/request-chain.ts b/packages/core/src/recovery/request-chain.ts new file mode 100644 index 0000000..3426126 --- /dev/null +++ b/packages/core/src/recovery/request-chain.ts @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/request-chain.ts +import type {Request} from '../http/request.js'; + +/** + * One link of the request-preparation chain. Async like every other step type in this layer — Node + * has a single execution model, so the phase does not mix sync and async step shapes. + * + * @public + */ +export type RequestStep = (request: Request) => Promise<Request>; + +/** + * A sequential left-to-right fold over request steps (RECOV-3): the output of step N is the input + * of step N+1, an empty chain returns its input unchanged, and a throwing step aborts the remainder + * and propagates — `prepareRequest` (`orchestrator.ts`) converts that propagation into a `Failure` + * per RECOV-2, which is the only reason propagating here is safe. Every entry point runs the chain + * through that one helper, the retry adapter included. + * + * **Apply the chain once per logical request, not once per wire send.** Steps here are the ones + * whose output must stay stable across a retry — `idempotencyKeyStep` above all — and the SDK's own + * retry adapter sits below this chain for that reason (`retry/retry-dispatch.ts`). + * + * Safe under concurrent `apply()` calls (RECOV-14): after construction the instance holds nothing + * but its step array, and every piece of per-call state lives in `apply()`'s locals. A later phase + * must not move per-call bookkeeping onto a field here. + * + * @public + */ +export class RequestRecoveryChain { + readonly #steps: readonly RequestStep[]; + + /** + * Defensively copies `steps` (RECOV-14). The reference implementation retains the caller's array + * by reference on this chain only — an asymmetry the requirement's own text recommends a port not + * reproduce. + * + * @param steps - the ordered request steps. + */ + constructor(steps: readonly RequestStep[]) { + this.#steps = [...steps]; + } + + /** + * Folds the request through every step in order. + * + * @param request - the request to prepare. + * @returns the request produced by the last step, or the input when the chain is empty. + * @throws Whatever a step throws, aborting the remaining steps (RECOV-3). + */ + async apply(request: Request): Promise<Request> { + let current = request; + for (const step of this.#steps) { + current = await step(current); + } + return current; + } +} diff --git a/packages/core/src/recovery/response-chain.test.ts b/packages/core/src/recovery/response-chain.test.ts new file mode 100644 index 0000000..a9e14ae --- /dev/null +++ b/packages/core/src/recovery/response-chain.test.ts @@ -0,0 +1,426 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/response-chain.test.ts +// Exercises: RECOV-4 (response steps run only on a Success), RECOV-5/RECOV-6 (recovery steps run on +// every outcome; fold order is all response steps then all recovery steps), RECOV-7 (a throwing +// response step becomes a Failure fed to recovery, never propagated), RECOV-8 (a throwing recovery +// step becomes a Failure fed to the NEXT recovery step; apply() never throws), RECOV-12 +// (close-on-throw while holding a Success, close failure attached as `suppressed` with the original +// throwable staying primary), RECOV-13 (a deliberately returned substitute outcome is never +// auto-closed), RECOV-14 (both step lists defensively copied, and steps safe for concurrent +// invocation — no per-call state on the chain instance) +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {failure, success, type Outcome} from './outcome.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, + type ResponseStep, +} from './response-chain.js'; + +function aResponse(body: ReadableStream<Uint8Array> | null = null): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .body(body) + .build(); +} + +/** + * Close is observed through the body stream's `cancel()`, exactly the way Phase 3b's own + * `response.test.ts` observes it — NOT by patching `response.close`. `Response` calls + * `Object.freeze(this)` at the end of its constructor, so `response.close = ...` throws + * `TypeError: Cannot add property close, object is not extensible` in an ES module's strict mode. + * `Response.close()` is memoized and cancels the body at most once, so the cancel count IS the + * effective-close count RECOV-12's "released exactly once" asks about. + */ +function countingCloseResponse(): { + response: Response; + closeCount: () => number; +} { + let cancels = 0; + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + cancels += 1; + }, + }); + return {response: aResponse(body), closeCount: () => cancels}; +} + +/** + * A response whose `close()` rejects. `Response.close()` awaits `body.cancel()` and swallows only + * `TypeError` (the locked-stream case), so a plain `Error` propagates out of `close()`. + */ +function failingCloseResponse(closeError: Error): Response { + return aResponse( + new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw closeError; + }, + }), + ); +} + +describe('response-step phase (RECOV-4, RECOV-6)', () => { + test('response steps run in order on a Success outcome', async () => { + const seen: string[] = []; + const stepA: ResponseStep = r => { + seen.push('a'); + return Promise.resolve(r); + }; + const stepB: ResponseStep = r => { + seen.push('b'); + return Promise.resolve(r); + }; + const chain = new ResponseRecoveryChain([stepA, stepB], []); + + await chain.apply(success(aResponse())); + + expect(seen).toEqual(['a', 'b']); + }); + + test('response steps do not run when the input outcome is already a Failure', async () => { + const original = new Error('original'); + const stepShouldNotRun: ResponseStep = () => { + throw new Error('must not run'); + }; + const chain = new ResponseRecoveryChain([stepShouldNotRun], []); + + const result = await chain.apply(failure(original)); + + expect(result.kind).toBe('failure'); + expect(result.kind === 'failure' && result.error).toBe(original); + }); +}); + +describe('recovery-step phase (RECOV-5, RECOV-6)', () => { + test('recovery steps run on every outcome, successes and failures, in order', async () => { + const seenKinds: string[] = []; + const record: RecoveryStep = outcome => { + seenKinds.push(outcome.kind); + return Promise.resolve(outcome); + }; + const chain = new ResponseRecoveryChain([], [record, record]); + + await chain.apply(success(aResponse())); + await chain.apply(failure(new Error('x'))); + + expect(seenKinds).toEqual(['success', 'success', 'failure', 'failure']); + }); + + test('fold order is all response steps first, then all recovery steps', async () => { + const order: string[] = []; + const responseStep: ResponseStep = r => { + order.push('response'); + return Promise.resolve(r); + }; + const recoveryStep: RecoveryStep = outcome => { + order.push('recovery'); + return Promise.resolve(outcome); + }; + const chain = new ResponseRecoveryChain([responseStep], [recoveryStep]); + + await chain.apply(success(aResponse())); + + expect(order).toEqual(['response', 'recovery']); + }); +}); + +describe('RECOV-7: a throwing response step converts to a Failure fed to recovery', () => { + test('the throwable never propagates out of apply(), and recovery observes the Failure', async () => { + const stepAfterThatMustNotRun: ResponseStep = () => { + throw new Error( + 'must not run — the response phase stops after the throw', + ); + }; + const thrownError = new Error('response step failed'); + const throwingStep: ResponseStep = () => { + throw thrownError; + }; + const seenByRecovery: Outcome<Response>[] = []; + const recoveryStep: RecoveryStep = outcome => { + seenByRecovery.push(outcome); + return Promise.resolve(outcome); + }; + const chain = new ResponseRecoveryChain( + [throwingStep, stepAfterThatMustNotRun], + [recoveryStep], + ); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('failure'); + expect(result.kind === 'failure' && result.error).toBe(thrownError); + expect(seenByRecovery).toHaveLength(1); + expect(seenByRecovery[0]?.kind).toBe('failure'); + }); +}); + +describe('RECOV-8: a throwing recovery step wraps into a Failure fed to the next step', () => { + test('apply() never throws, and the remaining recovery steps still run', async () => { + const secondStepSeen: Outcome<Response>[] = []; + const throwingRecoveryStep: RecoveryStep = () => { + throw new Error('recovery step failed'); + }; + const secondRecoveryStep: RecoveryStep = outcome => { + secondStepSeen.push(outcome); + return Promise.resolve(outcome); + }; + const chain = new ResponseRecoveryChain( + [], + [throwingRecoveryStep, secondRecoveryStep], + ); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('failure'); + expect(secondStepSeen).toHaveLength(1); + expect(secondStepSeen[0]?.kind).toBe('failure'); + }); +}); + +describe('RECOV-12: close-on-throw while holding a Success', () => { + test('closes the in-hand response exactly once before wrapping the throwable', async () => { + const {response, closeCount} = countingCloseResponse(); + const thrownError = new Error('step failed'); + const throwingStep: ResponseStep = () => { + throw thrownError; + }; + const chain = new ResponseRecoveryChain([throwingStep], []); + + const result = await chain.apply(success(response)); + + expect(closeCount()).toBe(1); + expect(result.kind === 'failure' && result.error).toBe(thrownError); + }); + + test('a close failure is attached as suppressed, with the original throwable staying primary', async () => { + const closeError = new Error('close failed'); + const response = failingCloseResponse(closeError); + const originalError = new Error('step failed'); + const throwingStep: ResponseStep = () => { + throw originalError; + }; + const chain = new ResponseRecoveryChain([throwingStep], []); + + const result = await chain.apply(success(response)); + + expect(result.kind).toBe('failure'); + const wrapped = result.kind === 'failure' ? result.error : undefined; + expect(wrapped).toBeInstanceOf(Error); + expect((wrapped as SuppressedErrorShape).name).toBe('SuppressedError'); + expect((wrapped as SuppressedErrorShape).error).toBe(originalError); + expect((wrapped as SuppressedErrorShape).suppressed).toBe(closeError); + }); + + test('a throwing recovery step holding a Success also closes it exactly once', async () => { + const {response, closeCount} = countingCloseResponse(); + const thrownError = new Error('recovery step failed'); + const throwingRecoveryStep: RecoveryStep = () => { + throw thrownError; + }; + const chain = new ResponseRecoveryChain([], [throwingRecoveryStep]); + + const result = await chain.apply(success(response)); + + expect(closeCount()).toBe(1); + expect(result.kind === 'failure' && result.error).toBe(thrownError); + }); + + test('a throwing step holding a Failure closes nothing — there is no response in hand', async () => { + const thrownError = new Error('recovery step failed'); + const throwingRecoveryStep: RecoveryStep = () => { + throw thrownError; + }; + const chain = new ResponseRecoveryChain([], [throwingRecoveryStep]); + + const result = await chain.apply(failure(new Error('seed'))); + + expect(result.kind === 'failure' && result.error).toBe(thrownError); + }); +}); + +interface SuppressedErrorShape extends Error { + readonly error: unknown; + readonly suppressed: unknown; +} + +describe('RECOV-13: a deliberate outcome substitution is never auto-closed', () => { + test('a recovery step returning a different Failure does not trigger a close', async () => { + const {response, closeCount} = countingCloseResponse(); + const substituteStep: RecoveryStep = () => + Promise.resolve(failure(new Error('substituted, not thrown'))); + const chain = new ResponseRecoveryChain([], [substituteStep]); + + await chain.apply(success(response)); + + expect(closeCount()).toBe(0); + }); + + test('a recovery step substituting a different Success does not trigger a close', async () => { + const {response: original, closeCount} = countingCloseResponse(); + const substitute = aResponse(); + const substituteStep: RecoveryStep = () => + Promise.resolve(success(substitute)); + const chain = new ResponseRecoveryChain([], [substituteStep]); + + const result = await chain.apply(success(original)); + + expect(closeCount()).toBe(0); + expect(result.kind === 'success' && result.value).toBe(substitute); + }); +}); + +describe('RECOV-14: both step lists are defensively copied', () => { + test('mutating the source arrays after construction has no effect on apply()', async () => { + const responseSteps: ResponseStep[] = []; + const recoverySteps: RecoveryStep[] = []; + const chain = new ResponseRecoveryChain(responseSteps, recoverySteps); + responseSteps.push(() => { + throw new Error('must not run — pushed after construction'); + }); + recoverySteps.push(outcome => Promise.resolve(outcome)); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('success'); + }); +}); + +describe('RECOV-8: apply() never throws, including on a step that lies about its type', () => { + // RECOV-8 is absolute — "MUST NOT throw under any input" — and `recovery/` is plumbing a later + // phase (and eventually a caller) installs steps into. TypeScript cannot enforce the return type + // across that seam, so the two shapes a mistyped step produces are pinned here: a step whose + // return value is not an outcome at all, and a step that then trips over it. Before this was + // guarded, the second case raised `TypeError: undefined is not an object` out of `apply()`. + test('a recovery step returning a non-outcome does not make apply() throw', async () => { + const returnsNothing = (() => + Promise.resolve(undefined)) as unknown as RecoveryStep; + const readsTheOutcome: RecoveryStep = outcome => + Promise.resolve(outcome.kind === 'failure' ? outcome : outcome); + const chain = new ResponseRecoveryChain( + [], + [returnsNothing, readsTheOutcome], + ); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('failure'); + }); + + test('a response step returning a non-response does not make apply() throw', async () => { + const returnsNothing = (() => + Promise.resolve(undefined)) as unknown as ResponseStep; + const readsTheResponse: ResponseStep = response => + Promise.resolve(response.status.isError ? response : response); + const chain = new ResponseRecoveryChain( + [returnsNothing, readsTheResponse], + [], + ); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('failure'); + }); + + test("the step's own throwable stays primary when the plumbing also fails", async () => { + const returnsNothing = (() => + Promise.resolve(undefined)) as unknown as ResponseStep; + const thrownError = new Error('step failed on a poisoned outcome'); + const throwingStep: ResponseStep = () => { + throw thrownError; + }; + const chain = new ResponseRecoveryChain([returnsNothing, throwingStep], []); + + const result = await chain.apply(success(aResponse())); + + expect(result.kind).toBe('failure'); + const error = result.kind === 'failure' ? result.error : undefined; + expect((error as SuppressedErrorShape).name).toBe('SuppressedError'); + expect((error as SuppressedErrorShape).error).toBe(thrownError); + }); +}); + +describe('apply() never throws (RECOV-8 property)', () => { + // Canonical law for an invariant-bearing function: for an arbitrary mix of throwing and + // non-throwing RESPONSE AND RECOVERY steps, over a seed outcome that is arbitrarily a Success or + // a Failure, apply() always settles and never re-raises a step's throw (RECOV-8) — and no + // response step runs on any generated case whose seed was already a Failure (RECOV-4). + // + // BOTH phases and BOTH seed variants must be generated: a generator emitting recovery steps only, + // or seeding Success only, proves the RECOV-8 law and silently leaves RECOV-4 to the example + // tests above. + test('apply() settles and skips the response phase on a Failure seed, for arbitrary step sequences', async () => { + await fc.assert( + fc.asyncProperty( + fc.array(fc.boolean(), {maxLength: 4}), + fc.array(fc.boolean(), {maxLength: 4}), + fc.boolean(), + async (responseFlags, recoveryFlags, seedIsSuccess) => { + let responseStepRuns = 0; + const responseSteps: ResponseStep[] = responseFlags.map( + (shouldThrow, index) => response => { + responseStepRuns += 1; + if (shouldThrow) + throw new Error(`response step ${String(index)} failed`); + return Promise.resolve(response); + }, + ); + const recoverySteps: RecoveryStep[] = recoveryFlags.map( + (shouldThrow, index) => outcome => { + if (shouldThrow) + throw new Error(`recovery step ${String(index)} failed`); + return Promise.resolve(outcome); + }, + ); + const chain = new ResponseRecoveryChain(responseSteps, recoverySteps); + const seed = seedIsSuccess + ? success(aResponse()) + : failure<Response>(new Error('seed failure')); + + const result = await chain.apply(seed); + + expect(['success', 'failure']).toContain(result.kind); + if (!seedIsSuccess) expect(responseStepRuns).toBe(0); // RECOV-4 + }, + ), + ); + }); +}); + +describe('RECOV-14: steps are safe for concurrent invocation', () => { + // RECOV-14's SECOND normative clause: per-request state lives in the passed value, never on the + // step or the chain. Guards the structural property that apply()'s only per-call state is its + // `current` local — a later phase adding per-call bookkeeping to a chain field would fail here. + test('two interleaved apply() calls on ONE chain instance do not observe each other', async () => { + const gate: (() => void)[] = []; + const slowStep: RecoveryStep = async outcome => { + await new Promise<void>(resolve => gate.push(resolve)); + return outcome; + }; + const chain = new ResponseRecoveryChain([], [slowStep]); + const successSeed = success(aResponse()); + const failureSeed = failure<Response>(new Error('second call')); + + const first = chain.apply(successSeed); + const second = chain.apply(failureSeed); + await Promise.resolve(); + for (const release of gate) release(); + const [firstResult, secondResult] = await Promise.all([first, second]); + + expect(firstResult.kind).toBe('success'); + expect( + secondResult.kind === 'failure' && (secondResult.error as Error).message, + ).toBe('second call'); + }); +}); diff --git a/packages/core/src/recovery/response-chain.ts b/packages/core/src/recovery/response-chain.ts new file mode 100644 index 0000000..8b60730 --- /dev/null +++ b/packages/core/src/recovery/response-chain.ts @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/response-chain.ts +import type {Response} from '../http/response.js'; +import {suppress} from '../suppress.js'; +import {failure, success, type Outcome} from './outcome.js'; + +/** + * One link of the response phase, run only while the outcome is a Success (RECOV-4). + * + * @public + */ +export type ResponseStep = (response: Response) => Promise<Response>; + +/** + * One link of the recovery phase, run on every outcome, success or failure (RECOV-5). + * + * A recovery step SHOULD return a `Failure` rather than throw (RECOV-9); both are handled + * identically, so this is a convention rather than something the chain enforces. + * + * @public + */ +export type RecoveryStep = ( + outcome: Outcome<Response>, +) => Promise<Outcome<Response>>; + +/** + * The response and recovery step folds (RECOV-4 … RECOV-9, RECOV-12, RECOV-13). + * + * Response steps run first and only while the outcome is a Success, in declared order; a throwing + * response step becomes a Failure fed to the recovery phase (RECOV-7) rather than propagating. + * Recovery steps then run on whatever the outcome is by then, always, in declared order; a throwing + * recovery step becomes a Failure fed to the NEXT recovery step (RECOV-8). `apply()` itself never + * throws, for any input. + * + * A step that *returns* a substitute outcome is never auto-closed (RECOV-13) — only a caught throw + * reaches this module's `toFailureClosingSuccess`. A transforming step owns releasing whatever it + * drops. + * + * Safe under concurrent `apply()` calls (RECOV-14): after construction the instance holds nothing + * but its two step arrays, and all per-call state lives in the phase methods' locals. + * + * @public + */ +export class ResponseRecoveryChain { + readonly #responseSteps: readonly ResponseStep[]; + readonly #recoverySteps: readonly RecoveryStep[]; + + /** + * Defensively copies both lists (RECOV-14). + * + * @param responseSteps - steps run on a Success, in order. + * @param recoverySteps - steps run on every outcome, in order. + */ + constructor( + responseSteps: readonly ResponseStep[], + recoverySteps: readonly RecoveryStep[], + ) { + this.#responseSteps = [...responseSteps]; + this.#recoverySteps = [...recoverySteps]; + } + + /** + * Folds `outcome` through the response phase and then the recovery phase (RECOV-6). + * + * @param outcome - the outcome produced by the transport, or by an earlier failure. + * @returns the terminal outcome. Never throws (RECOV-8). + */ + async apply(outcome: Outcome<Response>): Promise<Outcome<Response>> { + const afterResponsePhase = await this.#runResponsePhase(outcome); + return this.#runRecoveryPhase(afterResponsePhase); + } + + async #runResponsePhase( + outcome: Outcome<Response>, + ): Promise<Outcome<Response>> { + let current = outcome; + for (const step of this.#responseSteps) { + // RECOV-4: the whole response phase is skipped once the outcome is not a Success. + if (current.kind !== 'success') break; + try { + current = success(await step(current.value)); + } catch (thrownError) { + current = await toFailureClosingSuccess(thrownError, current); // RECOV-7, RECOV-12 + break; // the remaining response steps do not run once converted to a Failure + } + } + return current; + } + + async #runRecoveryPhase( + outcome: Outcome<Response>, + ): Promise<Outcome<Response>> { + let current = outcome; + for (const step of this.#recoverySteps) { + try { + // RECOV-13: a normal return substituting the outcome is never auto-closed. + current = await step(current); + } catch (thrownError) { + current = await toFailureClosingSuccess(thrownError, current); // RECOV-8, RECOV-12 + // RECOV-8: the remaining recovery steps still run — deliberately no `break` here. + } + } + return current; + } +} + +/** + * Shared close-on-throw handling for both phases (RECOV-12): when the outcome held at the moment of + * the throw was a Success, its response is released before the throwable is wrapped into a Failure, + * exactly once. + * + * A close failure rides along as `suppressed` on the ORIGINAL throwable — built by hand through + * {@link suppress}, original first — never via `using` / `await using`, whose auto-generated + * `SuppressedError` puts the *teardown* failure first and would silently invert which error the + * caller ends up seeing (`docs/knowledge/harvested/resource-management.md:72`). + * + * **This function is total: it never throws, for any argument.** RECOV-8 makes "`apply()` MUST NOT + * throw under any input" absolute, and this runs inside both phases' `catch` blocks — the last place + * a throwable could escape the chain. The discriminant read and the `close()` call are therefore + * inside the same `try`, not just the `close()`: a step that lies about its return type (a JS caller, + * or one returning `undefined`) can leave `current` holding something with no `kind` and no + * `close()`, and reading through it would otherwise raise a `TypeError` out of `apply()`. Handling it + * here rather than crashing is the same call `wrapCancellation` makes — a step is a pluggable seam, + * so a step that misbehaves is an operational failure, not a violated precondition of this codebase. + * The step's own throwable stays primary either way; the plumbing failure rides along as + * `suppressed`. + */ +async function toFailureClosingSuccess( + thrownError: unknown, + current: Outcome<Response>, +): Promise<Outcome<Response>> { + try { + if (current.kind === 'success') { + // Awaited, not fire-and-forget: `Response.close()` returns a promise, so an un-awaited call + // would settle outside this try with nothing to catch its rejection. + await current.value.close(); + } + } catch (closeError) { + return failure( + suppress( + thrownError, + closeError, + 'response close failed while handling a step error', + ), + ); + } + return failure(thrownError); +} diff --git a/packages/core/src/recovery/status-mapping.test.ts b/packages/core/src/recovery/status-mapping.test.ts new file mode 100644 index 0000000..0963257 --- /dev/null +++ b/packages/core/src/recovery/status-mapping.test.ts @@ -0,0 +1,154 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/status-mapping.test.ts +// Exercises: RECOV-15 (only 400..599 map to the matching typed exception; every other status passes +// through unchanged, and §8.2's conformance clause that an error status reaches a recovery hook as +// a Failure), RECOV-16 (the mapping reuses Phase 3b's already-bounded, replayable buffering — this +// file proves the wiring only; the 1 MiB cap and its truncation are 3b's own suite's job, at +// `body/http-status-error.test.ts`), RECOV-7 and RECOV-4 where the step meets the chain +import {describe, expect, test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import {HttpStatusError} from '../body/http-status-error.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {failure, success, type Outcome} from './outcome.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, + type ResponseStep, +} from './response-chain.js'; +import {statusMappingStep} from './status-mapping.js'; + +function aResponse( + status: number, + body: ReadableStream<Uint8Array> | null = null, +): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .body(body) + .build(); +} + +function bodyOf(text: string): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); +} + +/** Awaits `promise`, returning whatever it rejected with — `undefined` when it resolved. */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return undefined; + } catch (error) { + return error; + } +} + +describe('statusMappingStep (RECOV-15)', () => { + test('returns a 2xx response unchanged', async () => { + const response = aResponse(200); + + const result = await statusMappingStep(response); + + expect(result).toBe(response); + }); + + test('returns a 3xx response unchanged', async () => { + const response = aResponse(304); + + const result = await statusMappingStep(response); + + expect(result).toBe(response); + }); + + test('returns a non-standard 6xx response unchanged — only 400..599 map', async () => { + const response = aResponse(600); + + const result = await statusMappingStep(response); + + expect(result).toBe(response); + }); + + test('throws HttpStatusError naming the status for a 404', async () => { + const error = await rejection(statusMappingStep(aResponse(404))); + + expect(error).toBeInstanceOf(HttpStatusError); + expect((error as HttpStatusError).status).toBe(404); + }); + + test('throws HttpStatusError for a 500', async () => { + const error = await rejection(statusMappingStep(aResponse(500))); + + expect(error).toBeInstanceOf(HttpStatusError); + expect((error as HttpStatusError).status).toBe(500); + }); +}); + +describe('statusMappingStep buffering (RECOV-16)', () => { + test('the error body survives on the thrown exception, replayable after the response is closed', async () => { + const error = await rejection( + statusMappingStep(aResponse(422, bodyOf('{"detail":"nope"}'))), + ); + + expect(error).toBeInstanceOf(HttpStatusError); + expect((error as HttpStatusError).preview()).toBe('{"detail":"nope"}'); + }); +}); + +describe('statusMappingStep inside a recovery chain (RECOV-15, RECOV-7)', () => { + // §8.2's own conformance clause for RECOV-15 is about the outcome the chain produces, not about + // the step in isolation: a 400..599 must reach a recovery hook as a Failure carrying the typed + // exception, exactly the way a transport error does. The step throwing is the mechanism; this is + // the requirement. + test('a 404 surfaces to a recovery hook as a Failure carrying the typed exception', async () => { + const seen: Outcome<Response>[] = []; + const recorder: RecoveryStep = outcome => { + seen.push(outcome); + return Promise.resolve(outcome); + }; + const chain = new ResponseRecoveryChain([statusMappingStep], [recorder]); + + const result = await chain.apply(success(aResponse(404, bodyOf('nope')))); + + expect(result.kind).toBe('failure'); + const error = result.kind === 'failure' ? result.error : undefined; + expect(error).toBeInstanceOf(HttpStatusError); + expect((error as HttpStatusError).status).toBe(404); + expect((error as HttpStatusError).preview()).toBe('nope'); + expect(seen).toEqual([result]); + }); + + test('a 200 passes through the chain as a Success carrying the same response', async () => { + const response = aResponse(200); + const chain = new ResponseRecoveryChain([statusMappingStep], []); + + const result = await chain.apply(success(response)); + + expect(result.kind === 'success' && result.value).toBe(response); + }); + + test('the step never runs on a Failure input — RECOV-4 governs, not the status', async () => { + const seedError = new Error('transport failed'); + const chain = new ResponseRecoveryChain([statusMappingStep], []); + + const result = await chain.apply(failure(seedError)); + + expect(result.kind === 'failure' && result.error).toBe(seedError); + }); +}); + +describe('statusMappingStep conforms to ResponseStep', () => { + // The compile-time proof the discarded `: ResponseStep` annotation used to provide, kept out of + // the module so nothing dead reaches `dist/`. Only fires under `bun run typecheck` — `bun test` + // executes this file but strips its types without checking them (styleguide 11.6). + test('its signature is exactly the ResponseStep signature', () => { + expectTypeOf<typeof statusMappingStep>().toEqualTypeOf<ResponseStep>(); + }); +}); diff --git a/packages/core/src/recovery/status-mapping.ts b/packages/core/src/recovery/status-mapping.ts new file mode 100644 index 0000000..ae8026f --- /dev/null +++ b/packages/core/src/recovery/status-mapping.ts @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/recovery/status-mapping.ts +import {toHttpError} from '../body/http-status-error.js'; +import type {Response} from '../http/response.js'; + +/** + * The status → typed-exception mapping response step (RECOV-15, RECOV-16). + * + * Phase 3b's `toHttpError()` already satisfies both requirements in full: it treats only 400..599 + * as errors and hands every other status back unchanged (RECOV-15), and it buffers the error body + * into a bounded, replayable in-memory copy inside the response's own close-guaranteeing scope + * before mapping, sharing the same 1 MiB cap 3b's logging tees use (RECOV-16). `HttpStatusError` — + * flat, carrying `status` and the buffered body — IS the "matching typed exception"; no new + * buffering, no per-status class hierarchy. + * + * The `throw` is deliberate: it lets RECOV-7 in `response-chain.ts` convert an error status into a + * Failure exactly the way any other response-step throw is handled, rather than this step + * special-casing its own error path. + * + * @param response - the response to inspect. + * @returns the response unchanged when its status is not an error status. + * @throws HttpStatusError when the status is in 400..599. + * + * @public + */ +export async function statusMappingStep(response: Response): Promise<Response> { + const httpError = await toHttpError(response); + if (httpError === null) return response; + throw httpError; +} + +// A named declaration, not `const statusMappingStep: ResponseStep = async response => ...`: arrows +// are reserved for inline callbacks (docs/knowledge/harvested/function-design.md:18-21), and a named +// declaration survives in stack traces — which a function whose whole job is to throw actually +// depends on. `func-style`'s `allowArrowFunctions: true` would not have flagged the arrow form, so +// this is on the author, not the gate. +// +// The proof that the signature still conforms to `ResponseStep` lives in the test file, as an +// `expectTypeOf` assertion. A module-level `statusMappingStep satisfies ResponseStep;` would do the +// same job, but `satisfies` erases to its operand rather than to nothing, leaving a dead +// `statusMappingStep;` expression statement in the published `dist/`. diff --git a/packages/core/src/redirect/codes.test.ts b/packages/core/src/redirect/codes.test.ts new file mode 100644 index 0000000..5da7d55 --- /dev/null +++ b/packages/core/src/redirect/codes.test.ts @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/codes.test.ts +// Exercises: REDIR-1 (the recognized set is exactly {301,302,303,307,308}; any other status is returned +// verbatim without consulting redirect logic), REDIR-2 (300/304/305 are never auto-followed even with a +// Location), REDIR-3 (301/302 gated on method membership, default {GET,HEAD}), REDIR-4 (307/308 gated the +// same way), REDIR-5 (303 gated ONLY on the opt-in, independent of the original method). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import type {Method} from '../http/method.js'; +import { + DEFAULT_ALLOWED_METHODS, + REDIRECT_STATUSES, + isEligibleByCode, + isRecognizedRedirect, +} from './codes.js'; + +describe('isRecognizedRedirect', () => { + test('301, 302, 303, 307, 308 are recognized', () => { + for (const code of [301, 302, 303, 307, 308]) { + expect(isRecognizedRedirect(code)).toBe(true); + } + }); + + test('300, 304, 305 are never recognized (REDIR-2)', () => { + for (const code of [300, 304, 305]) { + expect(isRecognizedRedirect(code)).toBe(false); + } + }); + + test('non-3xx statuses are not recognized (REDIR-1)', () => { + for (const code of [200, 404, 500]) { + expect(isRecognizedRedirect(code)).toBe(false); + } + }); + + test('the exported set and the predicate are the same source', () => { + fc.assert( + fc.property(fc.integer({min: 100, max: 599}), code => { + expect(isRecognizedRedirect(code)).toBe(REDIRECT_STATUSES.has(code)); + }), + ); + }); +}); + +describe('isEligibleByCode', () => { + const eligibility = { + allowedMethods: DEFAULT_ALLOWED_METHODS, + allow303: false, + }; + + test('301/302/307/308 are eligible for GET/HEAD, the default allowed set', () => { + for (const status of [301, 302, 307, 308]) { + expect(isEligibleByCode(status, 'GET', eligibility)).toBe(true); + expect(isEligibleByCode(status, 'HEAD', eligibility)).toBe(true); + } + }); + + test('301/302/307/308 are NOT eligible outside the allowed set (REDIR-3/REDIR-4)', () => { + for (const status of [301, 302, 307, 308]) { + expect(isEligibleByCode(status, 'POST', eligibility)).toBe(false); + } + }); + + test('a caller-widened allowed set makes POST eligible', () => { + const widened = { + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + allow303: false, + }; + expect(isEligibleByCode(301, 'POST', widened)).toBe(true); + }); + + test('303 is never eligible by default, regardless of method (REDIR-5)', () => { + expect(isEligibleByCode(303, 'GET', eligibility)).toBe(false); + expect(isEligibleByCode(303, 'POST', eligibility)).toBe(false); + }); + + test('303 is eligible once opted in, regardless of method (REDIR-5)', () => { + const opted = {allowedMethods: DEFAULT_ALLOWED_METHODS, allow303: true}; + expect(isEligibleByCode(303, 'DELETE', opted)).toBe(true); + }); + + test('303 ignores the allowed-methods set entirely (REDIR-5)', () => { + const empty = {allowedMethods: new Set<Method>(), allow303: true}; + expect(isEligibleByCode(303, 'POST', empty)).toBe(true); + }); +}); diff --git a/packages/core/src/redirect/codes.ts b/packages/core/src/redirect/codes.ts new file mode 100644 index 0000000..460662b --- /dev/null +++ b/packages/core/src/redirect/codes.ts @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/codes.ts +import type {Method} from '../http/method.js'; + +/** + * REDIR-1/REDIR-2: the only statuses redirect logic is ever consulted for. 300, 304, and 305 are + * deliberately excluded even when they carry a `Location` -- 305 in particular must never redirect a + * request to a server-chosen proxy. + * + * @internal + */ +export const REDIRECT_STATUSES: ReadonlySet<number> = new Set([ + 301, 302, 303, 307, 308, +]); + +/** + * REDIR-3/REDIR-4's default allowed-method set. + * + * @internal + */ +export const DEFAULT_ALLOWED_METHODS: ReadonlySet<Method> = new Set([ + 'GET', + 'HEAD', +]); + +/** + * REDIR-1: any status outside {@link REDIRECT_STATUSES} -- 2xx, 4xx, 5xx, and non-redirect 3xx alike -- + * is returned verbatim without consulting redirect logic at all. + * + * @param status - the response status code. + * @returns `true` when the status is one redirect logic may act on. + * + * @internal + */ +export function isRecognizedRedirect(status: number): boolean { + return REDIRECT_STATUSES.has(status); +} + +/** + * The policy slice {@link isEligibleByCode} reads. A `RedirectSettings` value satisfies this + * structurally, so callers pass their settings directly rather than building an adapter object. + * + * @internal + */ +export interface CodeEligibility { + readonly allowedMethods: ReadonlySet<Method>; + readonly allow303: boolean; +} + +/** + * REDIR-3/REDIR-4/REDIR-5: 301/302/307/308 are eligible only when the ORIGINAL method is in + * `allowedMethods` -- when followed, method and body are preserved, deliberately with no automatic + * POST-to-GET rewrite. 303 is eligible only when opted in via `allow303`, independent of method; the + * GET rebuild and body drop that follow are `decide.ts`'s job, not this predicate's. + * + * @param status - the response status code; assumed recognized. + * @param method - the current hop's request method. + * @param eligibility - the allowed-method set and the 303 opt-in. + * @returns `true` when code and method alone permit following. + * + * @internal + */ +export function isEligibleByCode( + status: number, + method: Method, + eligibility: CodeEligibility, +): boolean { + if (status === 303) return eligibility.allow303; + return eligibility.allowedMethods.has(method); +} diff --git a/packages/core/src/redirect/cross-origin.test.ts b/packages/core/src/redirect/cross-origin.test.ts new file mode 100644 index 0000000..2f6f0bc --- /dev/null +++ b/packages/core/src/redirect/cross-origin.test.ts @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/cross-origin.test.ts +// Exercises: REDIR-8 (the RFC 6454 origin tuple -- scheme, case-insensitive host, effective port -- +// compared against a fixed SEED origin, never the previous hop; path/query/fragment never participate), +// REDIR-11 (the credential-suppression marker is cleared-then-conditionally-set, so a server-supplied +// Location can never forge an inbound copy into a surviving one). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {Headers} from '../http/headers.js'; +import { + CROSS_ORIGIN_MARKER_HEADER, + clearCrossOriginMarker, + hasCrossOriginMarker, + isCrossOrigin, + originOf, + withCrossOriginMarker, +} from './cross-origin.js'; + +describe('originOf / isCrossOrigin', () => { + const seed = originOf(new URL('https://example.com/a')); + + test('identical scheme/host/port is same-origin', () => { + expect(isCrossOrigin(seed, new URL('https://example.com/b?x=1#y'))).toBe( + false, + ); + }); + + test('a differing path/query/fragment alone is never cross-origin', () => { + fc.assert( + fc.property(fc.webPath(), fc.string(), (path, fragment) => { + const target = new URL(`https://example.com${path}`); + target.hash = fragment.replaceAll(/[^\w-]/gu, ''); + expect(isCrossOrigin(seed, target)).toBe(false); + }), + ); + }); + + test('host comparison is case-insensitive', () => { + expect(isCrossOrigin(seed, new URL('https://EXAMPLE.com/b'))).toBe(false); + }); + + test('a differing host is cross-origin', () => { + expect(isCrossOrigin(seed, new URL('https://evil.example/b'))).toBe(true); + }); + + test('a differing scheme is cross-origin even on the same host', () => { + expect(isCrossOrigin(seed, new URL('http://example.com/b'))).toBe(true); + }); + + test('an explicit default port equals an omitted one', () => { + expect(isCrossOrigin(seed, new URL('https://example.com:443/b'))).toBe( + false, + ); + }); + + test('a non-default port is cross-origin', () => { + expect(isCrossOrigin(seed, new URL('https://example.com:8443/b'))).toBe( + true, + ); + }); + + test('a bracketed IPv6 literal host round-trips unchanged (REDIR-13)', () => { + const v6 = originOf(new URL('https://[2001:db8::1]:8443/a')); + expect(v6.host).toBe('[2001:db8::1]'); + expect(v6.port).toBe(8443); + expect(isCrossOrigin(v6, new URL('https://[2001:db8::1]:8443/b'))).toBe( + false, + ); + expect(isCrossOrigin(v6, new URL('https://[2001:db8::2]:8443/b'))).toBe( + true, + ); + }); + + test('comparison is against the SEED, not a previous hop', () => { + // simulates: seed(example.com) -> hop1(other.example, cross-origin) -> hop2(example.com again). + // Anchored to the seed, hop2 is same-origin again -- which is exactly why the comparison must not + // walk hop to hop: a foreign host must not be able to hand the credential back to its own origin. + expect(isCrossOrigin(seed, new URL('https://example.com/final'))).toBe( + false, + ); + }); +}); + +describe('the cross-origin marker', () => { + test('withCrossOriginMarker sets the header to 1', () => { + const headers = withCrossOriginMarker(Headers.newBuilder().build()); + expect(hasCrossOriginMarker(headers)).toBe(true); + expect(headers.get(CROSS_ORIGIN_MARKER_HEADER)).toBe('1'); + }); + + test('withCrossOriginMarker clears a forged inbound copy before setting its own', () => { + const forged = Headers.newBuilder() + .add(CROSS_ORIGIN_MARKER_HEADER, 'anything') + .build(); + const marked = withCrossOriginMarker(forged); + expect(marked.getAll(CROSS_ORIGIN_MARKER_HEADER)).toEqual(['1']); + }); + + test('clearCrossOriginMarker removes it', () => { + const marked = withCrossOriginMarker(Headers.newBuilder().build()); + expect(hasCrossOriginMarker(clearCrossOriginMarker(marked))).toBe(false); + }); + + test('clearCrossOriginMarker is idempotent when already absent', () => { + const bare = Headers.newBuilder().build(); + expect(hasCrossOriginMarker(clearCrossOriginMarker(bare))).toBe(false); + }); + + test('hasCrossOriginMarker is false when never set', () => { + expect(hasCrossOriginMarker(Headers.newBuilder().build())).toBe(false); + }); +}); diff --git a/packages/core/src/redirect/cross-origin.ts b/packages/core/src/redirect/cross-origin.ts new file mode 100644 index 0000000..b472f6e --- /dev/null +++ b/packages/core/src/redirect/cross-origin.ts @@ -0,0 +1,119 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/cross-origin.ts +import type {Headers} from '../http/headers.js'; + +/** + * The RFC 6454 origin tuple REDIR-8 compares on. Held as a value rather than reusing `URL.origin`'s + * string form so the port is already normalized to the scheme default and the host already lower-cased + * -- `URL.origin` renders an omitted default port and an explicit one identically, but does nothing for + * a scheme this SDK does not follow. + * + * @internal + */ +export interface Origin { + readonly scheme: string; + readonly host: string; + readonly port: number; +} + +const DEFAULT_PORT_BY_SCHEME: ReadonlyMap<string, number> = new Map([ + ['http:', 80], + ['https:', 443], +]); + +/** REDIR-8: an omitted port normalizes to the scheme's default before comparison. */ +function effectivePort(url: URL): number { + if (url.port !== '') return Number(url.port); + return DEFAULT_PORT_BY_SCHEME.get(url.protocol.toLowerCase()) ?? 0; +} + +/** + * Extracts the comparable origin tuple. `URL.hostname` keeps a bracketed IPv6 literal bracketed and + * already lower-cases a registered name, so nothing here re-encodes the host (REDIR-13). + * + * @param url - the URL whose origin is wanted. + * @returns the normalized scheme/host/effective-port tuple. + * + * @internal + */ +export function originOf(url: URL): Origin { + return { + scheme: url.protocol.toLowerCase(), + host: url.hostname.toLowerCase(), + port: effectivePort(url), + }; +} + +/** + * REDIR-8: scheme/host(case-insensitive)/effective-port comparison against the SEED request's origin -- + * never the previous hop -- so a same-origin sub-redirect on a foreign host cannot re-expose the + * credential a cross-origin hop already stripped. `new URL(...)` never performs DNS resolution, so there + * is no `java.net.URL.equals()` hostname-resolution trap of the kind the JVM reference works around. + * + * @param seedOrigin - the origin of the ORIGINAL request, fixed for the whole chain. + * @param target - the resolved redirect target. + * @returns `true` when the target differs in scheme, host, or effective port. + * + * @internal + */ +export function isCrossOrigin(seedOrigin: Origin, target: URL): boolean { + const targetOrigin = originOf(target); + return ( + targetOrigin.scheme !== seedOrigin.scheme || + targetOrigin.host !== seedOrigin.host || + targetOrigin.port !== seedOrigin.port + ); +} + +/** + * REDIR-11's out-of-band signal, carried as a real header rather than an in-process marker. + * + * A `WeakSet<Request>` keyed by object identity was the alternative and is unforgeable, but stage order + * is REDIRECT -> RETRY -> AUTH and 5a's attempt-stamping builds a FRESH per-attempt `Request` copy when + * enabled -- an identity-keyed signal would silently stop matching the moment a retry sits between + * redirect and auth, which is exactly when cross-origin credential suppression must still hold. Stamping + * preserves headers, so a header survives that intermediate copy. `strip-marker-step.ts` is what keeps + * it off the wire. + * + * @internal + */ +export const CROSS_ORIGIN_MARKER_HEADER = + 'x-dexpace-internal-redirect-cross-origin'; + +/** + * REDIR-11(a): `HeadersBuilder.set` with a non-null value REPLACES the whole value list, so this is + * clear-then-set in one call -- a forged or stale inbound copy cannot survive alongside our own. + * + * @param headers - the next hop's headers so far. + * @returns headers carrying exactly one marker value. + * + * @internal + */ +export function withCrossOriginMarker(headers: Headers): Headers { + return headers.newBuilder().set(CROSS_ORIGIN_MARKER_HEADER, '1').build(); +} + +/** + * Idempotent -- clearing an already-absent header is a no-op. + * + * @param headers - the headers to strip the marker from. + * @returns headers with no marker. + * + * @internal + */ +export function clearCrossOriginMarker(headers: Headers): Headers { + return headers.newBuilder().set(CROSS_ORIGIN_MARKER_HEADER, null).build(); +} + +/** + * REDIR-11(b): the marker only ever SUPPRESSES credential stamping; nothing reads it to cause one. + * Phase 5c's auth step is its first real consumer. + * + * @param headers - the headers to inspect. + * @returns `true` when the marker is present. + * + * @internal + */ +export function hasCrossOriginMarker(headers: Headers): boolean { + return headers.has(CROSS_ORIGIN_MARKER_HEADER); +} diff --git a/packages/core/src/redirect/decide.test.ts b/packages/core/src/redirect/decide.test.ts new file mode 100644 index 0000000..1a90f60 --- /dev/null +++ b/packages/core/src/redirect/decide.test.ts @@ -0,0 +1,965 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/decide.test.ts +// Exercises: XCUT-17's two clauses a plaintext conformance fixture cannot reach -- (c) userinfo embedded +// in a Location is dropped before re-issue (REDIR-12), and (d) an HTTPS-to-HTTP downgrade is denied by +// default and permitted only by explicit opt-in (REDIR-14/15). The stripping clauses (a)/(b) are asserted +// end-to-end in tests/conformance/xcut/security-by-default.conformance.test.ts. +// Exercises every numbered step of decide()'s contract: REDIR-1/REDIR-2 (the non-redirect fast path and +// the never-followed 300/304/305), REDIR-21 (a recognized 3xx always allocates the snapshot and consults +// the predicate, even with no usable Location; a non-redirect status never does), REDIR-20 (the predicate +// fully overrides code/method eligibility, over a DEFENSIVELY COPIED snapshot), REDIR-14 (relative +// resolution against the CURRENT hop), REDIR-12 (userinfo dropped), REDIR-13 (no re-encoding of an +// already-percent-encoded path/query), REDIR-18/REDIR-19 (malformed, unsupported-scheme, and +// missing/empty Location all return-current without throwing), REDIR-16 (loop detection), REDIR-17 (the +// hop cap, including maxHops: 0), REDIR-15 (the per-hop HTTPS-to-HTTP guard), REDIR-6 (the body +// replayability gate; 303 exempt), REDIR-7 (Authorization always stripped), REDIR-9/REDIR-10 (Cookie and +// Proxy-Authorization stripped only cross-origin), REDIR-11 (the marker set only on a cross-origin hop), +// REDIR-5 (the 303 GET rebuild drops the body and every Content-* header), REDIR-3/REDIR-4 (a followed +// method-preserving redirect keeps the original method), and REDIR-3's eligibility reference point -- the +// CURRENT hop's method rather than the spec's literal "original request method", recorded as a deliberate +// reading in docs/deviations.md under "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import type {Body} from '../body/body.js'; +import {stringBody} from '../body/simple-bodies.js'; +import {streamBody} from '../body/stream-body.js'; +import {Headers} from '../http/headers.js'; +import type {Method} from '../http/method.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {CROSS_ORIGIN_MARKER_HEADER, originOf} from './cross-origin.js'; +import {decide, type RedirectContext} from './decide.js'; +import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; +import {redirectSettings} from './settings.js'; + +interface RequestOpts { + readonly method?: Method; + readonly url?: string; + readonly headers?: Headers; + readonly body?: Body; +} + +function aRequest(opts: RequestOpts = {}): Request { + const builder = Request.newBuilder() + .method(opts.method ?? 'GET') + .url(opts.url ?? 'https://example.com/a') + .headers(opts.headers ?? Headers.newBuilder().build()); + return opts.body === undefined + ? builder.build() + : builder.body(opts.body).build(); +} + +/** A single-use body -- `replayable: false` is the only property the gate reads (BODY-9). */ +function oneShotBody(): Body { + return streamBody( + new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); +} + +/** + * Drops exactly what the LENIENT inbound header validator rejects -- C0 controls except HTAB, plus DEL -- + * mirroring `hasForbiddenInboundValueByte`. obs-text (>= 0x80) is legal on an inbound value and must + * reach `decide()` unfiltered, so it is deliberately kept. A code-point filter rather than a regex: the + * equivalent character class is a literal control-character range, which `no-control-regex` rejects for + * exactly the reason that does not apply to a deliberate sanitizer. + */ +function inboundSafe(raw: string): string { + let out = ''; + for (const ch of raw) { + const code = ch.codePointAt(0) ?? 0; + if ((code <= 0x1f && code !== 0x09) || code === 0x7f) continue; + out += ch; + } + return out; +} + +// `setInbound`, not `set`: these are RESPONSE headers, and the outbound-strict `set` rejects every +// non-ASCII byte -- which would make the totality property test below throw inside its own fixture +// rather than reaching the code under test (HTTP-19). +function aResponse( + status: number, + location?: string, + extraHeaders?: Headers, +): Response { + let headers = extraHeaders ?? Headers.newBuilder().build(); + if (location !== undefined) { + headers = headers.newBuilder().setInbound('Location', location).build(); + } + return Response.newBuilder() + .request(aRequest()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(headers) + .body(null) + .build(); +} + +function contextFor( + request: Request, + overrides?: Partial<RedirectContext>, +): RedirectContext { + return { + currentRequest: request, + seedOrigin: originOf(request.url), + visited: new Set([request.url.href]), + redirectsFollowed: 0, + ...overrides, + }; +} + +describe('the shared return-current value', () => { + test('is frozen -- one instance is handed to every caller on every no-follow path', () => { + const decision = decide( + aResponse(200), + contextFor(aRequest()), + redirectSettings(), + ); + expect(Object.isFrozen(decision)).toBe(true); + }); +}); + +describe('fast path', () => { + test('a non-3xx status returns-current without consulting anything (REDIR-1/REDIR-21)', () => { + let consulted = false; + const settings = redirectSettings({ + predicate: () => { + consulted = true; + return true; + }, + }); + const decision = decide(aResponse(200), contextFor(aRequest()), settings); + expect(decision).toEqual({ + kind: 'return-current', + reason: 'not-a-redirect', + }); + expect(consulted).toBe(false); + }); + + test('300/304/305 are never followed even with a Location header (REDIR-2)', () => { + for (const status of [300, 304, 305]) { + const decision = decide( + aResponse(status, 'https://example.com/b'), + contextFor(aRequest()), + redirectSettings(), + ); + // 'not-a-redirect' rather than 'not-eligible': 300/304/305 never reach the eligibility gate, + // because `isRecognizedRedirect` excludes them and REDIR-21's fast path short-circuits first. + expect(decision).toEqual({ + kind: 'return-current', + reason: 'not-a-redirect', + }); + } + }); +}); + +describe('predicate override', () => { + test('a configured predicate REPLACES code/method eligibility (REDIR-20)', () => { + const settings = redirectSettings({predicate: () => true}); + const decision = decide( + aResponse(301, 'https://example.com/b'), + contextFor(aRequest({method: 'POST'})), + settings, + ); + expect(decision.kind).toBe('follow'); + }); + + test('a predicate is consulted even with no usable Location (REDIR-21)', () => { + let observed = false; + const settings = redirectSettings({ + predicate: condition => { + observed = true; + expect(condition.redirectsFollowed).toBe(0); + expect(condition.visited.has('https://example.com/a')).toBe(true); + return true; + }, + }); + const decision = decide(aResponse(301), contextFor(aRequest()), settings); + expect(observed).toBe(true); + expect(decision).toEqual({ + kind: 'return-current', + reason: 'malformed-location', + }); // still no Location to follow to + }); + + test('a predicate saying no wins over an otherwise-eligible code/method', () => { + const settings = redirectSettings({predicate: () => false}); + const decision = decide( + aResponse(301, 'https://example.com/b'), + contextFor(aRequest({method: 'GET'})), + settings, + ); + expect(decision).toEqual({ + kind: 'return-current', + reason: 'not-eligible', + }); + }); +}); + +describe('predicate override: the snapshot and the safety mechanics (REDIR-20)', () => { + test('the condition snapshot is a defensive COPY -- a predicate cannot poison loop detection', () => { + const live = new Set(['https://example.com/a']); + const settings = redirectSettings({ + predicate: condition => { + // A predicate that casts the readonly type away and tries to pre-seed the visited set. + (condition.visited as Set<string>).add('https://example.com/b'); + return true; + }, + }); + const context = contextFor(aRequest(), {visited: live}); + const decision = decide( + aResponse(302, 'https://example.com/b'), + context, + settings, + ); + + expect(decision.kind).toBe('follow'); // the injected entry never reached the live set, so /b is unvisited + expect(live.has('https://example.com/b')).toBe(false); + }); + + test('the predicate does NOT bypass the safety mechanics (see the Deviation Ledger)', () => { + // A predicate opting into a 307 re-send cannot make a single-use body replayable. + const request = Request.newBuilder() + .method('POST') + .url('https://example.com/a') + .body(oneShotBody()) + .build(); + const decision = decide( + aResponse(307, 'https://example.com/b'), + contextFor(request), + redirectSettings({predicate: () => true}), + ); + expect(decision.kind).toBe('fail'); + }); +}); + +describe('Location resolution', () => { + test('a relative Location resolves against the current request URL (REDIR-14)', () => { + const decision = decide( + aResponse(302, '/next'), + contextFor(aRequest({url: 'https://example.com/a/b'})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe('https://example.com/next'); + } + }); + + test('an absolute Location is used as-is (REDIR-14)', () => { + const decision = decide( + aResponse(302, 'https://other.example/x'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe('https://other.example/x'); + } + }); + + test('userinfo embedded in the Location is dropped unconditionally (REDIR-12)', () => { + const decision = decide( + aResponse(302, 'https://user:pass@other.example/x'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.username).toBe(''); + expect(decision.nextRequest.url.password).toBe(''); + expect(decision.nextRequest.url.href).toBe('https://other.example/x'); + } + }); + + test('an already-encoded path/query is never re-encoded (REDIR-13)', () => { + const decision = decide( + aResponse(302, 'https://example.com/a%2Fb?q=x%26y'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.pathname).toBe('/a%2Fb'); + expect(decision.nextRequest.url.search).toBe('?q=x%26y'); + } + }); + + test('a bracketed IPv6 host and explicit port survive resolution (REDIR-13)', () => { + const decision = decide( + aResponse(302, 'https://[2001:db8::1]:8443/x'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.hostname).toBe('[2001:db8::1]'); + expect(decision.nextRequest.url.port).toBe('8443'); + } + }); +}); + +describe('Location resolution -- the unfollowed paths', () => { + test('a missing Location returns-current (REDIR-19)', () => { + expect( + decide(aResponse(302), contextFor(aRequest()), redirectSettings()), + ).toEqual({ + kind: 'return-current', + reason: 'malformed-location', + }); + }); + + test('an empty Location returns-current (REDIR-19)', () => { + expect( + decide(aResponse(302, ''), contextFor(aRequest()), redirectSettings()), + ).toEqual({ + kind: 'return-current', + reason: 'malformed-location', + }); + }); + + test('an unparseable absolute Location returns-current rather than throwing (REDIR-18)', () => { + // A malformed ABSOLUTE form is the narrow case `new URL(raw, base)` actually throws on. + expect( + decide( + aResponse(302, 'http://['), + contextFor(aRequest()), + redirectSettings(), + ), + ).toEqual({ + kind: 'return-current', + reason: 'malformed-location', + }); + }); + + test('an unsupported scheme is returned unfollowed, never dispatched (REDIR-18)', () => { + for (const raw of [ + 'javascript:alert(1)', + 'data:text/html,x', + 'file:///etc/passwd', + 'mailto:a@b.c', + ]) { + expect( + decide(aResponse(302, raw), contextFor(aRequest()), redirectSettings()), + ).toEqual({ + kind: 'return-current', + reason: 'malformed-location', + }); + } + }); +}); + +describe('Location resolution -- totality and configuration', () => { + test('garbage that parses as a RELATIVE reference is followed, percent-encoded (REDIR-14)', () => { + // Documents WHATWG `URL` behavior deliberately: with a base supplied, a non-URL string is a + // relative reference, not a parse failure. The server said to go there, so we go there. + const decision = decide( + aResponse(302, ' not a url'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe( + 'https://example.com/not%20a%20url', + ); + } + }); + + test('the location header is configurable (REDIR-27)', () => { + const headers = Headers.newBuilder() + .setInbound('X-Redirect-To', 'https://example.com/b') + .build(); + const response = aResponse(302, undefined, headers); + const decision = decide( + response, + contextFor(aRequest()), + redirectSettings({locationHeader: 'X-Redirect-To'}), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe('https://example.com/b'); + } + }); + + test('property: decide() never throws for arbitrary garbage in Location (REDIR-18)', () => { + fc.assert( + fc.property(fc.string(), raw => { + expect(() => + decide( + aResponse(302, inboundSafe(raw)), + contextFor(aRequest()), + redirectSettings(), + ), + ).not.toThrow(); + }), + ); + }); +}); + +describe('loop detection', () => { + test('a Location matching an already-visited URI returns-current (REDIR-16)', () => { + const context = contextFor(aRequest({url: 'https://example.com/a'}), { + visited: new Set(['https://example.com/a', 'https://example.com/b']), + }); + expect( + decide( + aResponse(302, 'https://example.com/b'), + context, + redirectSettings(), + ), + ).toEqual({ + kind: 'return-current', + reason: 'loop-detected', + }); + }); + + test('a self-referencing Location returns-current (REDIR-16)', () => { + const context = contextFor(aRequest({url: 'https://example.com/a'})); + expect( + decide( + aResponse(302, 'https://example.com/a'), + context, + redirectSettings(), + ), + ).toEqual({ + kind: 'return-current', + reason: 'loop-detected', + }); + }); +}); + +describe('hop cap', () => { + test('following would exceed maxHops -> return-current (REDIR-17)', () => { + const context = contextFor(aRequest(), {redirectsFollowed: 3}); + const decision = decide( + aResponse(302, 'https://example.com/b'), + context, + redirectSettings({maxHops: 3}), + ); + expect(decision).toEqual({ + kind: 'return-current', + reason: 'hop-cap', + }); + }); + + test('maxHops: 0 fails on the very first follow attempt (REDIR-17)', () => { + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(aRequest()), + redirectSettings({maxHops: 0}), + ); + expect(decision).toEqual({ + kind: 'return-current', + reason: 'hop-cap', + }); + }); + + test('property: the hop cap bounds every synthetic chain regardless of length', () => { + fc.assert( + fc.property( + fc.integer({min: 0, max: 50}), + fc.integer({min: 1, max: 10}), + (followed, maxHops) => { + const context = contextFor(aRequest(), {redirectsFollowed: followed}); + const decision = decide( + aResponse(302, 'https://example.com/never-visited-before'), + context, + redirectSettings({maxHops}), + ); + if (followed + 1 > maxHops) { + expect(decision).toEqual({ + kind: 'return-current', + reason: 'hop-cap', + }); + } else { + expect(decision.kind).toBe('follow'); + } + }, + ), + ); + }); +}); + +describe('scheme-downgrade guard', () => { + test('HTTPS to HTTP is rejected by default (REDIR-15)', () => { + const decision = decide( + aResponse(302, 'http://example.com/b'), + contextFor(aRequest({url: 'https://example.com/a'})), + redirectSettings(), + ); + expect(decision.kind).toBe('fail'); + if (decision.kind === 'fail') { + expect(decision.error).toBeInstanceOf(SchemeDowngradeError); + } + }); + + test('HTTPS to HTTP is permitted when allowSchemeDowngrade is set (REDIR-15)', () => { + const decision = decide( + aResponse(302, 'http://example.com/b'), + contextFor(aRequest({url: 'https://example.com/a'})), + redirectSettings({allowSchemeDowngrade: true}), + ); + expect(decision.kind).toBe('follow'); + }); + + test('credential stripping still applies on a permitted downgrade (REDIR-15)', () => { + const headers = Headers.newBuilder() + .add('Authorization', 'Bearer x') + .build(); + const decision = decide( + aResponse(302, 'http://example.com/b'), + contextFor(aRequest({url: 'https://example.com/a', headers})), + redirectSettings({allowSchemeDowngrade: true}), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.get('Authorization')).toBeUndefined(); + } + }); + + test('HTTP to HTTPS is never a downgrade', () => { + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(aRequest({url: 'http://example.com/a'})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + }); + + test('the guard is keyed to the CURRENT hop scheme, not the seed (REDIR-15)', () => { + // Seed is http, the current hop is already https (a prior upgrade) -- a further downgrade off THIS + // hop must still be caught even though the seed itself was http. + const context: RedirectContext = { + currentRequest: aRequest({url: 'https://example.com/mid'}), + seedOrigin: originOf(new URL('http://example.com/a')), + visited: new Set(['http://example.com/a', 'https://example.com/mid']), + redirectsFollowed: 1, + }; + const decision = decide( + aResponse(302, 'http://example.com/b'), + context, + redirectSettings(), + ); + expect(decision.kind).toBe('fail'); + }); +}); + +describe('body replayability gate', () => { + test('a method-preserving redirect with a non-replayable body fails (REDIR-6)', () => { + const request = Request.newBuilder() + .method('POST') + .url('https://example.com/a') + .body(oneShotBody()) + .build(); + const decision = decide( + aResponse(307, 'https://example.com/b'), + contextFor(request), + redirectSettings({ + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + }), + ); + expect(decision.kind).toBe('fail'); + if (decision.kind === 'fail') { + expect(decision.error).toBeInstanceOf(NonReplayableBodyError); + } + }); + + test('a method-preserving redirect with a replayable body follows, body preserved (REDIR-6)', () => { + const request = Request.newBuilder() + .method('POST') + .url('https://example.com/a') + .body(stringBody('x')) + .build(); + const decision = decide( + aResponse(307, 'https://example.com/b'), + contextFor(request), + redirectSettings({ + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + }), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.body).toBeDefined(); + } + }); + + test('303 is exempt -- its body is dropped, not checked (REDIR-5/REDIR-6)', () => { + const request = Request.newBuilder() + .method('POST') + .url('https://example.com/a') + .body(oneShotBody()) + .build(); + const decision = decide( + aResponse(303, 'https://example.com/b'), + contextFor(request), + redirectSettings({allow303: true}), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.body).toBeUndefined(); + } + }); +}); + +describe('header construction', () => { + test('Authorization is stripped unconditionally, even same-origin (REDIR-7)', () => { + const headers = Headers.newBuilder() + .add('Authorization', 'Bearer x') + .build(); + const request = aRequest({url: 'https://example.com/a', headers}); + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(request), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.get('Authorization')).toBeUndefined(); + } + }); + + test('Cookie and Proxy-Authorization survive a same-origin hop (REDIR-10)', () => { + const headers = Headers.newBuilder() + .add('Cookie', 'a=b') + .add('Proxy-Authorization', 'y') + .build(); + const request = aRequest({url: 'https://example.com/a', headers}); + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(request), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.get('Cookie')).toBe('a=b'); + expect(decision.nextRequest.headers.get('Proxy-Authorization')).toBe('y'); + } + }); + + test('Cookie and Proxy-Authorization are stripped on a cross-origin hop (REDIR-9)', () => { + const headers = Headers.newBuilder() + .add('Cookie', 'a=b') + .add('Proxy-Authorization', 'y') + .build(); + const request = aRequest({url: 'https://example.com/a', headers}); + const decision = decide( + aResponse(302, 'https://evil.example/b'), + contextFor(request), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.get('Cookie')).toBeUndefined(); + expect( + decision.nextRequest.headers.get('Proxy-Authorization'), + ).toBeUndefined(); + } + }); +}); + +describe('header construction -- the cross-origin marker', () => { + test('the cross-origin marker is set only on a cross-origin follow (REDIR-11)', () => { + const sameOrigin = decide( + aResponse(302, 'https://example.com/b'), + contextFor(aRequest()), + redirectSettings(), + ); + const crossOrigin = decide( + aResponse(302, 'https://evil.example/b'), + contextFor(aRequest()), + redirectSettings(), + ); + expect(sameOrigin.kind === 'follow' && sameOrigin.crossOrigin).toBe(false); + expect(crossOrigin.kind === 'follow' && crossOrigin.crossOrigin).toBe(true); + }); + + test('a forged inbound marker never survives a same-origin hop (REDIR-11a)', () => { + const headers = Headers.newBuilder() + .add('x-dexpace-internal-redirect-cross-origin', '1') + .build(); + const request = aRequest({url: 'https://example.com/a', headers}); + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(request), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect( + decision.nextRequest.headers.has( + 'x-dexpace-internal-redirect-cross-origin', + ), + ).toBe(false); + } + }); +}); + +describe('header construction -- the 303 rebuild and method preservation', () => { + test('a 303 rebuild strips every Content-* header case-insensitively and forces GET (REDIR-5)', () => { + const headers = Headers.newBuilder() + .add('content-type', 'application/json') + .add('Content-Length', '3') + .add('CONTENT-ENCODING', 'gzip') + .add('X-Other', 'kept') + .build(); + const request = aRequest({ + method: 'POST', + url: 'https://example.com/a', + headers, + }); + const decision = decide( + aResponse(303, 'https://example.com/b'), + contextFor(request), + redirectSettings({allow303: true}), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.method).toBe('GET'); + expect(decision.nextRequest.headers.get('Content-Type')).toBeUndefined(); + expect( + decision.nextRequest.headers.get('Content-Length'), + ).toBeUndefined(); + expect( + decision.nextRequest.headers.get('Content-Encoding'), + ).toBeUndefined(); + expect(decision.nextRequest.headers.get('X-Other')).toBe('kept'); + } + }); + + test('a 301/302/307/308 follow preserves the original method (REDIR-3/REDIR-4)', () => { + const decision = decide( + aResponse(307, 'https://example.com/b'), + contextFor( + aRequest({ + method: 'POST', + url: 'https://example.com/a', + body: stringBody('x'), + }), + ), + redirectSettings({ + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + }), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.method).toBe('POST'); + } + }); +}); + +describe('REDIR-3 measures eligibility against the CURRENT hop, not the original request', () => { + // The spec says "the ORIGINAL request method" (`docs/product-spec/10-redirect-handling.md:8`); the port + // reads `currentRequest.method` (`decide.ts:241` into `codes.ts:69`). The two readings agree on every + // chain except one, and this is it: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 arriving on + // that rewritten hop is followed under the DEFAULT {GET, HEAD} set, where the literal reading would refuse + // it because the request that started the chain was a POST. Kept deliberately -- the rewritten GET is + // idempotent and carries no body, so the literal reading buys no safety, and refusing the hop would make + // `allow303` half-useful. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase". + test('a 303-rewritten GET makes a following 301 eligible under the default method set (REDIR-3/REDIR-5)', () => { + const original = aRequest({ + method: 'POST', + url: 'https://example.com/a', + body: stringBody('x'), + }); + const first = decide( + aResponse(303, 'https://example.com/b'), + contextFor(original), + redirectSettings({allow303: true}), + ); + expect(first.kind).toBe('follow'); + if (first.kind !== 'follow') return; + expect(first.nextRequest.method).toBe('GET'); + + // Nothing about the second hop is opted into: the default set is {GET, HEAD} and excludes the method + // the chain started with, which is what makes the two readings disagree here rather than coincide. + const settings = redirectSettings(); + expect(settings.allowedMethods.has('POST')).toBe(false); + expect(original.method).toBe('POST'); + + const second = decide( + aResponse(301, 'https://example.com/c'), + contextFor(first.nextRequest, { + visited: new Set([original.url.href, first.nextRequest.url.href]), + redirectsFollowed: 1, + }), + settings, + ); + expect(second.kind).toBe('follow'); + if (second.kind === 'follow') { + expect(second.nextRequest.url.href).toBe('https://example.com/c'); + expect(second.nextRequest.method).toBe('GET'); + } + }); +}); + +describe('loop detection survives URL normalization', () => { + // `visited` keys on `URL.href`, which WHATWG normalizes -- so a server cannot spin the loop past the + // cap by varying only the case of the scheme/host or by writing the scheme's default port out. Both + // resolve to a href already in the set. Worth pinning: if `visited` ever keyed on the raw Location + // string instead, both of these would silently become followable and the guard would be evadable. + test('an uppercase scheme and host still hit the visited set (REDIR-16)', () => { + const request = aRequest({url: 'https://example.com/a'}); + expect( + decide( + aResponse(302, 'HTTPS://EXAMPLE.COM/a'), + contextFor(request), + redirectSettings(), + ), + ).toEqual({ + kind: 'return-current', + reason: 'loop-detected', + }); + }); + + test("the scheme's default port written explicitly still hits the visited set (REDIR-16)", () => { + const request = aRequest({url: 'https://example.com/a'}); + expect( + decide( + aResponse(302, 'https://example.com:443/a'), + contextFor(request), + redirectSettings(), + ), + ).toEqual({ + kind: 'return-current', + reason: 'loop-detected', + }); + }); +}); + +describe('Location forms RFC 3986 resolution has to get right', () => { + test('a protocol-relative Location inherits the scheme and is judged cross-origin (REDIR-14)', () => { + const decision = decide( + aResponse(302, '//other.example/x'), + contextFor(aRequest({url: 'https://example.com/a'})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe('https://other.example/x'); + expect(decision.crossOrigin).toBe(true); + } + }); + + test('a query-only Location keeps the path and does not re-encode (REDIR-13/REDIR-14)', () => { + const decision = decide( + aResponse(302, '?q=a%26b'), + contextFor(aRequest({url: 'https://example.com/a/b'})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe( + 'https://example.com/a/b?q=a%26b', + ); + } + }); + + test('dot segments resolve against the current hop (REDIR-14)', () => { + const cases: readonly (readonly [string, string])[] = [ + ['.', 'https://example.com/a/b/'], + ['..', 'https://example.com/a/'], + ['../../x', 'https://example.com/x'], + ]; + for (const [location, expected] of cases) { + const decision = decide( + aResponse(302, location), + contextFor(aRequest({url: 'https://example.com/a/b/c'})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.url.href).toBe(expected); + } + } + }); +}); + +describe('credential and marker hygiene against multi-valued headers', () => { + test('every Authorization value is stripped, whatever its casing (REDIR-7)', () => { + const headers = Headers.newBuilder() + .add('authorization', 'Bearer x') + .add('AUTHORIZATION', 'Bearer y') + .build(); + const decision = decide( + aResponse(302, 'https://example.com/b'), + contextFor(aRequest({url: 'https://example.com/a', headers})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.getAll('Authorization')).toEqual([]); + } + }); + + test('a multi-valued forged marker collapses to exactly one own value (REDIR-11a)', () => { + // Clearing must precede the conditional set. If it did not, a server that got two marker values + // onto the request would leave the SDK appending a third rather than replacing both. + const headers = Headers.newBuilder() + .add(CROSS_ORIGIN_MARKER_HEADER, 'forged') + .add(CROSS_ORIGIN_MARKER_HEADER, 'twice') + .build(); + const decision = decide( + aResponse(302, 'https://evil.example/b'), + contextFor(aRequest({url: 'https://example.com/a', headers})), + redirectSettings(), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect( + decision.nextRequest.headers.getAll(CROSS_ORIGIN_MARKER_HEADER), + ).toEqual(['1']); + } + }); + + test('the 303 rebuild clears an inbound marker too (REDIR-11a)', () => { + const headers = Headers.newBuilder() + .add(CROSS_ORIGIN_MARKER_HEADER, 'forged') + .add('Content-Type', 'application/json') + .build(); + const decision = decide( + aResponse(303, 'https://example.com/b'), + contextFor( + aRequest({method: 'POST', url: 'https://example.com/a', headers}), + ), + redirectSettings({allow303: true}), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') { + expect(decision.nextRequest.headers.has(CROSS_ORIGIN_MARKER_HEADER)).toBe( + false, + ); + expect(decision.nextRequest.headers.has('Content-Type')).toBe(false); + } + }); +}); + +describe('the followed request carries the body instance itself', () => { + test('a replayable body is re-sent, not rebuilt (REDIR-3/REDIR-4/REDIR-6)', () => { + // The rewind is 3b's `writeTo` contract (BODY-9), not this step's -- so the step must hand the + // SAME body across, never a copy that would have its own materialize-once state. + const body = stringBody('x'); + const request = Request.newBuilder() + .method('POST') + .url('https://example.com/a') + .body(body) + .build(); + const decision = decide( + aResponse(307, 'https://example.com/b'), + contextFor(request), + redirectSettings({ + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + }), + ); + expect(decision.kind).toBe('follow'); + if (decision.kind === 'follow') + expect(decision.nextRequest.body).toBe(body); + }); +}); diff --git a/packages/core/src/redirect/decide.ts b/packages/core/src/redirect/decide.ts new file mode 100644 index 0000000..8928894 --- /dev/null +++ b/packages/core/src/redirect/decide.ts @@ -0,0 +1,284 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/decide.ts +import type {Headers} from '../http/headers.js'; +import type {Method} from '../http/method.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {isEligibleByCode, isRecognizedRedirect} from './codes.js'; +import { + clearCrossOriginMarker, + isCrossOrigin, + withCrossOriginMarker, + type Origin, +} from './cross-origin.js'; +import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; +import type {RedirectCondition, RedirectSettings} from './settings.js'; + +/** + * Everything one hop's decision reads, bundled: `decide()` would otherwise take five positional + * parameters against the codebase's `max-params: 3`. + * + * `seedOrigin` is the ORIGINAL request's origin and never advances with the chain (REDIR-8); `visited` + * is the step's live cycle-detection set, seeded with the seed request's URI (REDIR-16). + * + * @internal + */ +export interface RedirectContext { + readonly currentRequest: Request; + readonly seedOrigin: Origin; + readonly visited: ReadonlySet<string>; + readonly redirectsFollowed: number; +} + +/** + * WHY a hop stopped, on a `'return-current'` decision. + * + * `REDIR-28` names four structured events, and two of them -- loop-detected and malformed-Location + * -- are indistinguishable from ordinary termination once the decision is a bare `{kind}`. They were + * blocked on this discriminant and are emitted by `redirectStep` as of 2026-09-02. The other three + * reasons are carried for symmetry: a discriminant set on some paths and absent on others is a worse + * shape than either extreme. + * + * @internal + */ +export type RedirectStopReason = + /** + * The status is not a redirect code this SDK follows. Covers a non-3xx status AND the three 3xx + * codes REDIR-2 excludes by name (300/304/305), both of which take REDIR-21's fast path before + * the eligibility gate is reached. + */ + | 'not-a-redirect' + /** A caller predicate said no, or the code/method pair is not eligible (REDIR-2, REDIR-20). */ + | 'not-eligible' + /** Location was absent, empty, unparseable, or named an unsupported scheme (REDIR-18, REDIR-19). */ + | 'malformed-location' + /** The target is already in the visited set (REDIR-16). */ + | 'loop-detected' + /** Following would exceed `maxHops` (REDIR-17). */ + | 'hop-cap'; + +/** + * One hop's outcome. `'return-current'` hands the live response back to the caller unclosed (REDIR-16, + * REDIR-17, REDIR-18, REDIR-19, PIPE-40); `'fail'` is the caller's to close before rethrowing (REDIR-22b). + * + * @internal + */ +export type Decision = + | { + readonly kind: 'follow'; + readonly nextRequest: Request; + readonly crossOrigin: boolean; + } + | {readonly kind: 'return-current'; readonly reason: RedirectStopReason} + | {readonly kind: 'fail'; readonly error: Error}; + +// Frozen because each is SHARED: one instance per reason is handed to every caller taking that path, +// so an accidental write would corrupt every later decision in the process. `outcome.ts`'s +// `success`/`failure` build a fresh object per call and have no equivalent exposure. +const RETURN_CURRENT: Readonly<Record<RedirectStopReason, Decision>> = + Object.freeze({ + 'not-a-redirect': Object.freeze({ + kind: 'return-current', + reason: 'not-a-redirect', + }), + 'not-eligible': Object.freeze({ + kind: 'return-current', + reason: 'not-eligible', + }), + 'malformed-location': Object.freeze({ + kind: 'return-current', + reason: 'malformed-location', + }), + 'loop-detected': Object.freeze({ + kind: 'return-current', + reason: 'loop-detected', + }), + 'hop-cap': Object.freeze({kind: 'return-current', reason: 'hop-cap'}), + }); + +/** + * The only schemes this SDK will re-issue a request against. Anything else -- `javascript:`, `data:`, + * `file:`, `mailto:` -- is REDIR-18's "unsupported scheme", returned unfollowed rather than dispatched. + */ +const FOLLOWABLE_SCHEMES: ReadonlySet<string> = new Set(['http:', 'https:']); + +/** + * REDIR-14/REDIR-12/REDIR-13: resolves relative-or-absolute per RFC 3986 via WHATWG `URL`, drops + * userinfo, and never re-encodes an already-percent-encoded path/query/fragment. Total -- REDIR-18 says + * a malformed or unresolvable Location MUST NOT throw. + * + * Two things WHATWG `URL` does NOT do for us, both handled explicitly here: + * + * 1. **It almost never throws when a base is supplied.** `new URL(' not a url', 'https://example.com/a')` + * does not fail -- it resolves to `https://example.com/not%20a%20url`, because any string that is not + * a valid absolute URL is treated as a relative reference. So the `catch` below is a genuine but + * NARROW path (a malformed absolute form such as `http://[` still throws); it is not the general + * "garbage in the Location header" guard it might look like. Garbage that parses as a relative + * reference is followed, which is correct per RFC 3986 -- the server said so. + * 2. **It happily parses schemes we must never dispatch against.** `new URL('javascript:alert(1)', base)` + * succeeds, and the scheme-downgrade guard would wave it through (the target is not `http:`). The + * {@link FOLLOWABLE_SCHEMES} check is what makes REDIR-18's unsupported-scheme clause true rather + * than aspirational. + */ +function resolveLocation(raw: string | undefined, base: URL): URL | null { + if (raw === undefined || raw.trim() === '') return null; // REDIR-19: missing or empty. + try { + const resolved = new URL(raw, base); + if (!FOLLOWABLE_SCHEMES.has(resolved.protocol.toLowerCase())) return null; + // REDIR-12: assigning the empty string clears the component without touching path/query/fragment. + resolved.username = ''; + resolved.password = ''; + return resolved; + } catch { + return null; + } +} + +/** REDIR-5: the 303 GET rebuild drops every `Content-*` request header, matched case-insensitively. */ +function stripContentHeaders(headers: Headers): Headers { + let builder = headers.newBuilder(); + for (const name of headers.names()) { + if (name.toLowerCase().startsWith('content-')) + builder = builder.set(name, null); + } + return builder.build(); +} + +/** + * REDIR-7: `Authorization` is stripped on EVERY re-issue, same-origin and the 303 rebuild included. + * REDIR-9/REDIR-10: `Cookie` and `Proxy-Authorization` are origin-scoped, so they survive a same-origin + * hop and are stripped cross-origin. REDIR-11(a): the marker is cleared before it is conditionally set, + * so a forged or stale inbound copy can never survive a hop that should not carry it. + */ +function nextHopHeaders(headers: Headers, crossOrigin: boolean): Headers { + let builder = headers.newBuilder().set('Authorization', null); + if (crossOrigin) { + builder = builder.set('Cookie', null).set('Proxy-Authorization', null); + } + const cleared = clearCrossOriginMarker(builder.build()); + return crossOrigin ? withCrossOriginMarker(cleared) : cleared; +} + +interface FollowPlan { + readonly target: URL; + readonly status: number; + readonly crossOrigin: boolean; +} + +/** REDIR-3/REDIR-4 preserve method and body; REDIR-5 forces GET and drops the body. */ +function buildFollowRequest(current: Request, plan: FollowPlan): Request { + const {target, status, crossOrigin} = plan; + const is303 = status === 303; + const method: Method = is303 ? 'GET' : current.method; + let headers = nextHopHeaders(current.headers, crossOrigin); + if (is303) headers = stripContentHeaders(headers); + const builder = current + .newBuilder() + .url(target) + .method(method) + .headers(headers); + return is303 ? builder.body(undefined).build() : builder.build(); +} + +/** + * The per-hop redirect decision. Pure -- no I/O, no clock, no header-mutation side effects beyond the + * `nextRequest` value it returns -- mirroring 5a's split of `classify.ts`/`backoff.ts` away from the + * imperative loop. + * + * Step order: + * + * 1. **Fast path** (REDIR-1/REDIR-21): a status outside the recognized set short-circuits BEFORE + * allocating a condition snapshot and never consults a configured predicate. + * 2. **Snapshot and the follow/no-follow call** (REDIR-20/REDIR-21): any recognized 3xx allocates the + * snapshot and is offered to a configured predicate, EVEN with no usable Location. A configured + * predicate's boolean return IS the decision, replacing `isEligibleByCode`. + * 3. **Location resolution** (REDIR-12/13/14/18/19), including the followable-scheme gate. + * 4. **Loop detection** (REDIR-16). + * 5. **Hop cap** (REDIR-17) -- the one gate `maxHops: 0` always fails, which is what "disables redirect + * following entirely" reduces to; no separate branch. + * 6. **Scheme-downgrade guard** (REDIR-15), keyed to the CURRENT hop's scheme, not the seed's. This is a + * deliberately different reference point from step 8's seed-relative cross-origin check: downgrade + * catches a single transition wherever it happens, while cross-origin must stay anchored to the + * origin the credential was attached at, for the whole chain. + * 7. **Body-replayability gate** (REDIR-6); 303 is exempt because it drops the body. + * 8. **Cross-origin determination** (REDIR-8) and header construction for the next hop. + * + * **Scope of the predicate override.** REDIR-20's "MUST fully override the built-in decision" is read + * here as scoped to the code/method eligibility question only -- not as license to bypass steps 4-7's + * wire-safety invariants, which the same spec document states as unconditional MUSTs elsewhere. A + * caller predicate opting to follow a 307 with a non-replayable body still cannot make that body + * re-sendable. If this reading is wrong the fix is narrow and mechanical: gate step 3 onward behind the + * predicate's answer. Recorded in the design doc's Deviation Ledger. + * + * @param response - the hop's response. + * @param context - the current request, the seed origin, the live visited set, and the hop count. + * @param settings - the validated redirect policy. + * @returns whether to follow, return the current response, or fail. + * + * @internal + */ +export function decide( + response: Response, + context: RedirectContext, + settings: RedirectSettings, +): Decision { + if (!isRecognizedRedirect(response.status.code)) { + return RETURN_CURRENT['not-a-redirect']; + } + + const {currentRequest, seedOrigin, visited, redirectsFollowed} = context; + // REDIR-20: the snapshot is defensively COPIED, not merely typed `ReadonlySet`. `visited` is the + // step's LIVE cycle-detection set, and the type annotation is erased at runtime -- a predicate that + // casts it away could otherwise pre-seed or clear loop detection for the rest of the call. The spec's + // wording is about the object, not the type. + const condition: RedirectCondition = { + response, + redirectsFollowed, + visited: new Set(visited), + }; + const eligible = + settings.predicate === undefined + ? isEligibleByCode(response.status.code, currentRequest.method, settings) + : settings.predicate(condition); + if (!eligible) return RETURN_CURRENT['not-eligible']; + + // `Request.url` hands back a FRESH `URL` on every access (HTTP-5) -- read it once. + const currentUrl = currentRequest.url; + const target = resolveLocation( + response.headers.get(settings.locationHeader), + currentUrl, + ); + if (target === null) return RETURN_CURRENT['malformed-location']; + if (visited.has(target.href)) return RETURN_CURRENT['loop-detected']; + if (redirectsFollowed + 1 > settings.maxHops) { + return RETURN_CURRENT['hop-cap']; + } + + if ( + currentUrl.protocol.toLowerCase() === 'https:' && + target.protocol.toLowerCase() === 'http:' && + !settings.allowSchemeDowngrade + ) { + return { + kind: 'fail', + error: new SchemeDowngradeError(currentUrl.href, target.href), + }; + } + + const status = response.status.code; + const body = currentRequest.body; + if (status !== 303 && body !== undefined && !body.replayable) { + return {kind: 'fail', error: new NonReplayableBodyError(target.href)}; + } + + const crossOrigin = isCrossOrigin(seedOrigin, target); + return { + kind: 'follow', + nextRequest: buildFollowRequest(currentRequest, { + target, + status, + crossOrigin, + }), + crossOrigin, + }; +} diff --git a/packages/core/src/redirect/errors.test.ts b/packages/core/src/redirect/errors.test.ts new file mode 100644 index 0000000..67f6ec1 --- /dev/null +++ b/packages/core/src/redirect/errors.test.ts @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/errors.test.ts +// Exercises: REDIR-6 (a non-replayable body fails with a clear error NAMING replayability, rather than +// corrupting or truncating the re-send), REDIR-15 (an HTTPS->HTTP hop is rejected with a clear error by +// default). Both are operational failures a caller can legitimately hit mid-redirect, so both are typed +// error leaves rather than `invariant()` programmer-error assertions. +// Also: OBS-11 (userinfo is always redacted to `***:***@`), OBS-12 (query values are `***` unless +// allow-listed), OBS-15 (redaction is total -- an unparseable input yields the sentinel, never a throw) +// and XCUT-19(a)/(b) as they apply to the ERROR MESSAGE rather than to a log field. The message is what +// every logger, `cause` chain and consumer `console.error` renders, so it is redacted at construction; +// `targetUrl` / `fromUrl` / `toUrl` stay raw for program use. +import {describe, expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; + +describe('NonReplayableBodyError', () => { + test('names the target URL and mentions replayability', () => { + const error = new NonReplayableBodyError('https://example.com/next'); + expect(error).toBeInstanceOf(Error); + expect(error).toBeInstanceOf(DexpaceError); + expect(error.name).toBe('NonReplayableBodyError'); + expect(error.message).toContain('https://example.com/next'); + expect(error.message.toLowerCase()).toContain('replayable'); + }); + + test('carries the target as a readonly field, not only in the message', () => { + // docs/knowledge/harvested/error-handling.md: identifying inputs are fields so they survive serialization + // and reach a structured log without anyone parsing the message back apart. + const error = new NonReplayableBodyError('https://example.com/next'); + expect(error.targetUrl).toBe('https://example.com/next'); + }); + + test('accepts a cause', () => { + const cause = new Error('underlying'); + expect( + new NonReplayableBodyError('https://example.com/next', {cause}).cause, + ).toBe(cause); + }); + + test('redacts userinfo and non-allow-listed query values in the message', () => { + const error = new NonReplayableBodyError( + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET&api-version=2', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + // OBS-12's default allow-list is exactly {api-version}, and the message inherits it whole. + expect(error.message).toContain('api-version=2'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + }); + + test('keeps the RAW target on the field even when the message is redacted', () => { + const raw = + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET'; + expect(new NonReplayableBodyError(raw).targetUrl).toBe(raw); + }); + + test('degrades an unparseable target to the sentinel rather than throwing (OBS-15)', () => { + // A target that never parsed cannot be redacted, and OBS-15 makes that total: the sentinel, not + // the raw string, because "unparseable" is not the same as "carries no secret". + const error = new NonReplayableBodyError('not a url?token=SUPERSECRET'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.targetUrl).toBe('not a url?token=SUPERSECRET'); + }); +}); + +describe('SchemeDowngradeError', () => { + test('names both the current and target URLs', () => { + const error = new SchemeDowngradeError( + 'https://example.com/a', + 'http://example.com/b', + ); + expect(error).toBeInstanceOf(DexpaceError); + expect(error.name).toBe('SchemeDowngradeError'); + expect(error.message).toContain('https://example.com/a'); + expect(error.message).toContain('http://example.com/b'); + }); + + test('carries both URLs as readonly fields, not only in the message', () => { + const error = new SchemeDowngradeError( + 'https://example.com/a', + 'http://example.com/b', + ); + expect(error.fromUrl).toBe('https://example.com/a'); + expect(error.toUrl).toBe('http://example.com/b'); + }); + + test('accepts a cause', () => { + const cause = new Error('underlying'); + const error = new SchemeDowngradeError('https://a', 'http://b', {cause}); + expect(error.cause).toBe(cause); + }); + + test('redacts userinfo and non-allow-listed query values on BOTH sides of the message', () => { + const error = new SchemeDowngradeError( + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET', + 'http://example.com/next?code=ALSOSECRET', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + expect(error.message).toContain('code=***'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.message).not.toContain('ALSOSECRET'); + }); + + test('keeps BOTH raw URLs on the fields even when the message is redacted', () => { + const from = 'https://alice:hunter2@example.com/start?access_token=SECRET'; + const to = 'http://example.com/next?code=ALSOSECRET'; + const error = new SchemeDowngradeError(from, to); + + expect(error.fromUrl).toBe(from); + expect(error.toUrl).toBe(to); + }); + + test('degrades an unparseable side to the sentinel rather than throwing (OBS-15)', () => { + const error = new SchemeDowngradeError('::::', 'http://example.com/next'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).toContain('http://example.com/next'); + expect(error.fromUrl).toBe('::::'); + }); +}); diff --git a/packages/core/src/redirect/errors.ts b/packages/core/src/redirect/errors.ts new file mode 100644 index 0000000..b893f3a --- /dev/null +++ b/packages/core/src/redirect/errors.ts @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/errors.ts +import {DexpaceError} from '../http/errors.js'; +import {redactUrl} from '../observability/redaction.js'; + +// Both messages below interpolate `redactUrl(...)`, never the raw URL -- XCUT-19(a)/(b) and +// OBS-11/OBS-12 applied to the MESSAGE rather than to a log field. An error message is a public API: +// it travels into every logger, every `cause` chain, and every consumer's own `console.error`, and +// this SDK owns none of those. `http.redirect.rejected` in particular hands the decision error to +// `LogEvent.cause()`, which renders it as `name: message` (`observability/logger.ts`), so a raw +// `from`/`to` URL here put userinfo and query-string tokens into the log record in clear text however +// carefully the surrounding fields were redacted. Redacting at construction is the only placement that +// also covers the paths this SDK does not own. OBS-15 makes the call safe from a constructor: an input +// that will not parse yields `[malformed url]` rather than throwing. The raw value stays on the +// error's own property, which is what program code reads. + +/** + * REDIR-6: a method-preserving redirect (301/302/307/308) re-sends the original body, so the body must + * be replayable. Distinct from 3b's `ConsumedBodyError`, which fires on a SECOND write against an + * already-consumed single-use body -- this one is a fail-fast gate evaluated BEFORE any write is + * attempted, and names replayability specifically as the requirement demands. + * + * @remarks + * The `message` names the target in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link NonReplayableBodyError.targetUrl} for the raw URL. + * + * @public + */ +export class NonReplayableBodyError extends DexpaceError { + /** + * The redirect target that would have received the re-send, **raw and unredacted**. + * + * Carried as a field, not only interpolated into the message, per + * `docs/knowledge/harvested/error-handling.md` -- so it survives serialization and reaches a structured log + * without anyone parsing the message back apart. Phase 7b's rejection event reads it directly. + * + * This is the raw URL; the `message` carries the redacted form. Program code that needs the real + * target reads this property; anything that renders to a human or to a log backend reads the + * message. + */ + readonly targetUrl: string; + + /** + * @param targetUrl - the redirect target that would have received the re-send. + * @param options - standard error options; pass `{cause}` when wrapping a caught error. + */ + constructor(targetUrl: string, options?: ErrorOptions) { + super( + `cannot follow redirect to '${redactUrl(targetUrl)}': request body is not replayable`, + options, + ); + this.targetUrl = targetUrl; + } +} + +/** + * REDIR-15: an HTTPS-to-HTTP hop, rejected unless `RedirectSettings.allowSchemeDowngrade` is set. + * Evaluated per hop transition, so an HTTPS-to-HTTP-to-HTTPS chain flags only the hop that downgraded. + * + * @remarks + * The `message` names both URLs in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link SchemeDowngradeError.fromUrl} and + * {@link SchemeDowngradeError.toUrl} for the raw URLs. + * + * @public + */ +export class SchemeDowngradeError extends DexpaceError { + /** + * The current hop's request URL -- the HTTPS side of the rejected transition, **raw and + * unredacted**. The `message` carries the redacted form. + */ + readonly fromUrl: string; + /** + * The resolved redirect target -- the HTTP side of the rejected transition, **raw and unredacted**. + * The `message` carries the redacted form. + */ + readonly toUrl: string; + + /** + * @param fromUrl - the current hop's request URL. + * @param toUrl - the resolved redirect target. + * @param options - standard error options; pass `{cause}` when wrapping a caught error. + */ + constructor(fromUrl: string, toUrl: string, options?: ErrorOptions) { + super( + `redirect from '${redactUrl(fromUrl)}' to '${redactUrl(toUrl)}' would downgrade HTTPS to HTTP`, + options, + ); + this.fromUrl = fromUrl; + this.toUrl = toUrl; + } +} diff --git a/packages/core/src/redirect/redirect-step.test.ts b/packages/core/src/redirect/redirect-step.test.ts new file mode 100644 index 0000000..9fb52cd --- /dev/null +++ b/packages/core/src/redirect/redirect-step.test.ts @@ -0,0 +1,732 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/redirect-step.test.ts +// Exercises: PIPE-36 (the stage is baked into the descriptor, not subclassable), PIPE-15 (every dispatch +// takes a FRESH ctx.fork() continuation -- ctx.next()'s single-invocation guard would trip on hop two), +// PIPE-40/REDIR-22 (the 2-hop conformance clause: wire-send count, per-hop close of each superseded +// response, the final response left OPEN for the caller), REDIR-22(b) (a throw out of the decision -- +// including from caller predicate code -- closes the current response before propagating), REDIR-16 +// (a detected loop returns the loop response open, without throwing), REDIR-15 (a rejected downgrade +// closes the current response and propagates SchemeDowngradeError), and the cancellation check (an +// an abort DURING a hop returns the current response open rather than issuing a further hop, while a +// signal already aborted at entry is refused by the cursor before the step runs -- see V15). +// +// Also: REDIR-28 / XCUT-19 / OBS-11 / OBS-12 for the `http.redirect.rejected` record specifically. The +// other three redirect events already route their URL fields through `redactUrl()`; the rejection +// event carried the decision error's MESSAGE, which interpolated the raw `from`/`to` URLs, so a +// userinfo password or a query-string token reached the log record in clear text. +import {describe, expect, test} from 'bun:test'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {streamBody} from '../body/stream-body.js'; +import {Headers} from '../http/headers.js'; +import type {Method} from '../http/method.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {Cursor} from '../pipeline/cursor.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import type {Transport} from '../seams/transport.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; +import {REDIRECT_STEP_TYPE, redirectStep} from './redirect-step.js'; +import type {RedirectSettings} from './settings.js'; + +const SEED = Request.newBuilder().url('https://example.com/start').build(); +const CANCEL_FAILURE = new Error('cancel exploded'); + +// Constructed inline rather than imported: 4c keeps `aRequestContext()` file-local to `cursor.test.ts`, +// and importing across `*.test.ts` files is not acceptable -- the same call 5a's `retry-step.test.ts` made. +function aRequestContext(request: Request = SEED): ExecutionContext { + return createRequestContext(request); +} + +function runThrough( + descriptor: StepDescriptor, + transport: FakeTransport, + signal?: AbortSignal, +): Promise<Response> { + return new Cursor({ + steps: [descriptor], + transport, + request: SEED, + context: aRequestContext(), + signal, + }).advance(); +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this runner's + * type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper keeps the + * assertion honest without a lint suppression. Same shape 5a's `retry-step.test.ts` settled on. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +// `FakeTransport` does not itself set a Location -- `decide()` reads it off `Response.headers`, so a +// scripted 3xx entry must carry one explicitly. `ResponseBuilder` carries the SAME body instance through +// `response.newBuilder()`, so the rebuilt response still reports through `countingResponse`'s counter. +// `setInbound`, not `set`: a Location is an inbound (response) header (HTTP-19). +function withLocation(response: Response, location: string): Response { + return response + .newBuilder() + .headers( + response.headers.newBuilder().setInbound('Location', location).build(), + ) + .build(); +} + +/** + * A response whose body `cancel()` REJECTS with a non-`TypeError` -- the one thing `Response.close()` + * is documented to rethrow. Models a transport releasing over an already-broken socket. + */ +function hostileResponse(status: number, location: string): Response { + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw CANCEL_FAILURE; + }, + }); + return Response.newBuilder() + .request(SEED) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(Headers.newBuilder().setInbound('Location', location).build()) + .body(body) + .build(); +} + +describe('redirectStep', () => { + test('is pinned to the REDIRECT pillar stage (PIPE-36)', () => { + const descriptor = redirectStep(); + expect(descriptor.stage).toBe('REDIRECT'); + expect(descriptor.type).toBe(REDIRECT_STEP_TYPE); + }); + + test('closes PIPE-40: two chained 301s then a 200', async () => { + const first = countingResponse(301); + const second = countingResponse(301); + const third = countingResponse(200); + const hop1 = withLocation(first.response, 'https://example.com/mid'); + const hop2 = withLocation(second.response, '/final'); // relative, resolved against /mid + const transport = new FakeTransport([hop1, hop2, third.response]); + + const response = await runThrough(redirectStep(), transport); + + expect(transport.sendCount).toBe(3); + expect(first.cancelCount()).toBe(1); + expect(second.cancelCount()).toBe(1); + expect(third.cancelCount()).toBe(0); // left open for the caller + expect(response).toBe(third.response); + }); + + test('each hop is dispatched against the rewritten request (REDIR-7/REDIR-14)', async () => { + const first = countingResponse(301); + const final = countingResponse(200); + const seedWithAuth = Request.newBuilder() + .url('https://example.com/start') + .headers(Headers.newBuilder().add('Authorization', 'Bearer x').build()) + .build(); + const transport = new FakeTransport([ + withLocation(first.response, '/next'), + final.response, + ]); + + await new Cursor({ + steps: [redirectStep()], + transport, + request: seedWithAuth, + context: aRequestContext(seedWithAuth), + }).advance(); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[0]?.request.headers.get('Authorization')).toBe( + 'Bearer x', + ); + expect(transport.calls[1]?.request.url.href).toBe( + 'https://example.com/next', + ); + expect( + transport.calls[1]?.request.headers.get('Authorization'), + ).toBeUndefined(); + }); + + test('a non-redirect response is returned open, untouched, on the very first hop', async () => { + const only = countingResponse(200); + const transport = new FakeTransport([only.response]); + + const response = await runThrough(redirectStep(), transport); + + expect(transport.sendCount).toBe(1); + expect(only.cancelCount()).toBe(0); + expect(response).toBe(only.response); + }); +}); + +describe('redirectStep -- termination without a throw', () => { + test('a loop is detected and the loop response returned open, not thrown (REDIR-16)', async () => { + const loopHop = countingResponse(301); + const located = withLocation(loopHop.response, 'https://example.com/start'); + const transport = new FakeTransport([located]); + + const response = await runThrough(redirectStep(), transport); + + expect(response).toBe(located); // Location === seed URI -> visited hit -> return-current, unclosed + expect(loopHop.cancelCount()).toBe(0); + expect(transport.sendCount).toBe(1); + }); + + test('the hop cap returns the last response as-is, even a 3xx, without throwing (REDIR-17)', async () => { + const hopA = countingResponse(301); + const hopB = countingResponse(301); + const hopC = countingResponse(301); + const capped = countingResponse(301); + const transport = new FakeTransport([ + withLocation(hopA.response, 'https://example.com/1'), + withLocation(hopB.response, 'https://example.com/2'), + withLocation(hopC.response, 'https://example.com/3'), + withLocation(capped.response, 'https://example.com/4'), + ]); + + const response = await runThrough(redirectStep(), transport); + + expect(transport.sendCount).toBe(4); // seed + 3 followed hops, the default cap + expect(response.status.code).toBe(301); + expect(capped.cancelCount()).toBe(0); // returned open even though it is itself a redirect + }); + + test('maxHops: 0 disables following entirely (REDIR-17)', async () => { + const only = countingResponse(301); + const located = withLocation(only.response, 'https://example.com/next'); + const transport = new FakeTransport([located]); + + const response = await runThrough(redirectStep({maxHops: 0}), transport); + + expect(transport.sendCount).toBe(1); + expect(response).toBe(located); + expect(only.cancelCount()).toBe(0); + }); +}); + +describe('redirectStep -- cancellation and the failure paths', () => { + test('an abort DURING the first hop returns that response open, never dispatching a second', async () => { + // The redirect step's own per-hop `signal?.aborted` check, which is what discharges PIPE-40's + // "the in-flight response MUST be returned unclosed" on the abandon path. It runs BEFORE the + // step forks again, so the cursor's own step-boundary check (V15) never sees this abort and + // cannot pre-empt the open hand-back. Aborting mid-flight rather than up front is what keeps + // this test on the step's guard instead of the cursor's. + const controller = new AbortController(); + const hop = countingResponse(301); + const located = withLocation(hop.response, 'https://example.com/next'); + const never = countingResponse(200); + const transport = new FakeTransport([located, never.response]); + const aborting: Transport = { + send: async (request, options, signal) => { + const response = await transport.send(request, options, signal); + controller.abort(); + return response; + }, + close: () => Promise.resolve(), + }; + + const response = await new Cursor({ + steps: [redirectStep()], + transport: aborting, + request: SEED, + context: aRequestContext(), + signal: controller.signal, + }).advance(); + + expect(transport.sendCount).toBe(1); // the first hop dispatched; the second never does + expect(response).toBe(located); // returned open -- the caller owns it + expect(hop.cancelCount()).toBe(0); + }); +}); + +describe('redirectStep -- cancellation at entry (V15)', () => { + test('a signal already aborted at entry never dispatches at all', async () => { + // Distinct from the case above: the cursor now refuses the walk before the step runs, so there + // is no in-flight response to hand back and nothing to leak. Before 2026-09-02 this dispatched + // the first hop and returned it open. + const {CancellationError} = await import('../seams/transport.js'); + const controller = new AbortController(); + controller.abort(); + const never = countingResponse(200); + const transport = new FakeTransport([never.response]); + + const error = await rejectionOf( + runThrough(redirectStep(), transport, controller.signal), + ); + + expect(error).toBeInstanceOf(CancellationError); + expect(transport.sendCount).toBe(0); + expect(never.cancelCount()).toBe(0); + }); + + test('a rejected scheme downgrade closes the current response first (REDIR-15/REDIR-22b)', async () => { + const hop = countingResponse(301); + const located = withLocation(hop.response, 'http://example.com/next'); + const transport = new FakeTransport([located]); + + const error = await rejectionOf(runThrough(redirectStep(), transport)); + + expect(error).toBeInstanceOf(SchemeDowngradeError); + expect(hop.cancelCount()).toBe(1); // the hop's body is not leaked + }); + + test('a throwing predicate closes the current response before the error propagates (REDIR-22b)', async () => { + const hop = countingResponse(301); + const located = withLocation(hop.response, 'https://example.com/next'); + const transport = new FakeTransport([located]); + const boom = new Error('predicate exploded'); + const step = redirectStep({ + predicate: () => { + throw boom; + }, + }); + + const error = await rejectionOf(runThrough(step, transport)); + + expect(error).toBe(boom); // the caller's own error, not remapped to a redirect error type + expect(hop.cancelCount()).toBe(1); // decideOrClose closed it -- the hop's body is not leaked + }); + + test('a cross-origin hop carries the suppression marker to the next dispatch (REDIR-11)', async () => { + const hop = countingResponse(302); + const final = countingResponse(200); + const transport = new FakeTransport([ + withLocation(hop.response, 'https://other.example/next'), + final.response, + ]); + + await runThrough(redirectStep(), transport); + + expect( + transport.calls[1]?.request.headers.get( + 'x-dexpace-internal-redirect-cross-origin', + ), + ).toBe('1'); + }); +}); + +describe('redirectStep -- a failing release never masks the primary error', () => { + test('a rejecting close() keeps SchemeDowngradeError primary (REDIR-22b, RECOV-12)', async () => { + const transport = new FakeTransport([ + hostileResponse(301, 'http://example.com/next'), + ]); + + const error = await rejectionOf(runThrough(redirectStep(), transport)); + + // Without withReleaseFailure this was `Error: cancel exploded` -- the typed, caller-catchable, + // security-relevant error silently replaced by the teardown failure. + const suppressed = error as SuppressedErrorLike; + expect(suppressed.error).toBeInstanceOf(SchemeDowngradeError); + expect(suppressed.suppressed).toBe(CANCEL_FAILURE); + }); + + test("a rejecting close() keeps the caller's predicate error primary (REDIR-22b)", async () => { + const boom = new Error('predicate exploded'); + const transport = new FakeTransport([ + hostileResponse(301, 'https://example.com/next'), + ]); + const step = redirectStep({ + predicate: () => { + throw boom; + }, + }); + + const error = await rejectionOf(runThrough(step, transport)); + + const suppressed = error as SuppressedErrorLike; + expect(suppressed.error).toBe(boom); + expect(suppressed.suppressed).toBe(CANCEL_FAILURE); + }); + + test('a clean release leaves the primary error untouched, unwrapped', async () => { + const hop = countingResponse(301); + const transport = new FakeTransport([ + withLocation(hop.response, 'http://example.com/next'), + ]); + + const error = await rejectionOf(runThrough(redirectStep(), transport)); + + expect(error).toBeInstanceOf(SchemeDowngradeError); // NOT wrapped when nothing was suppressed + expect(hop.cancelCount()).toBe(1); + }); +}); + +describe("redirectStep -- REDIR-22(b)'s other named trigger, and concurrency", () => { + test('a non-replayable body closes the current response before the error propagates', async () => { + // REDIR-22(b) names exactly two triggers -- "non-replayable body, downgrade rejection". The + // downgrade one is covered above; this is the other. Note the deliberate reading of a conflict: + // PIPE-40's parenthetical lists "non-replayable body" among the paths whose in-flight response is + // "returned unclosed", which is the opposite disposition. REDIR-6 ("MUST fail with a clear error") + // and REDIR-22(b) agree that this path THROWS, and a response that is never returned cannot be + // "returned unclosed" -- so the redirect chapter governs. Recorded in the design's Deviation Ledger. + const oneShot = streamBody( + new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); + const seed = Request.newBuilder() + .method('POST') + .url('https://example.com/start') + .body(oneShot) + .build(); + const hop = countingResponse(307); + const transport = new FakeTransport([ + withLocation(hop.response, 'https://example.com/next'), + ]); + const step = redirectStep({ + allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST']), + }); + + const error = await rejectionOf( + new Cursor({ + steps: [step], + transport, + request: seed, + context: aRequestContext(seed), + }).advance(), + ); + + expect(error).toBeInstanceOf(NonReplayableBodyError); + expect(hop.cancelCount()).toBe(1); // closed, not leaked + expect(transport.sendCount).toBe(1); // the redirect was not attempted + }); + + test('one descriptor drives concurrent calls without sharing loop state', async () => { + // Every piece of per-call state -- `visited`, `redirectsFollowed`, `request`, `seedOrigin` -- is a + // local inside `fn`, so a single installed descriptor is safe under concurrency. The same property + // 5a asserts for its own engine (RETRY-42/RECOV-28). + const step = redirectStep(); + const drive = async (host: string): Promise<string | undefined> => { + const hop = countingResponse(301); + const final = countingResponse(200); + const transport = new FakeTransport([ + withLocation(hop.response, `https://${host}/final`), + final.response, + ]); + const seed = Request.newBuilder().url(`https://${host}/start`).build(); + const response = await new Cursor({ + steps: [step], + transport, + request: seed, + context: aRequestContext(seed), + }).advance(); + expect(response).toBe(final.response); + return transport.calls[1]?.request.url.href; + }; + + const [a, b] = await Promise.all([drive('a.example'), drive('b.example')]); + + expect(a).toBe('https://a.example/final'); + expect(b).toBe('https://b.example/final'); + }); +}); + +describe('Phase 7b retrofit: redirect hop and downgrade logging', () => { + test('emits http.redirect.hop for followed hops', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + const testLogger = createLogger((_level, fields) => { + events.push(new Map(fields)); + }); + setGlobalLogger(testLogger); + + try { + const hop = countingResponse(302); + const final = countingResponse(200); + const transport = new FakeTransport([ + withLocation(hop.response, 'https://example.com/dest'), + final.response, + ]); + + await runThrough(redirectStep(), transport); + + const hops = events.filter(e => e.get('event') === 'http.redirect.hop'); + expect(hops).toHaveLength(1); + expect(hops[0]?.get('hop')).toBe(1); + expect(hops[0]?.get('status')).toBe(302); + expect(hops[0]?.get('url.full')).toBe('https://example.com/dest'); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); + +describe('REDIR-28: the loop-detected and malformed-Location events (G3)', () => { + test('emits http.redirect.loopDetected when the target is already visited', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + + try { + const hop = countingResponse(302); + const loop = countingResponse(302); + const transport = new FakeTransport([ + withLocation(hop.response, 'https://example.com/dest'), + withLocation(loop.response, 'https://example.com/dest'), + ]); + + await runThrough(redirectStep(), transport); + + const detected = events.filter( + e => e.get('event') === 'http.redirect.loopDetected', + ); + expect(detected).toHaveLength(1); + expect(detected[0]?.get('location')).toBe('https://example.com/dest'); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); + + test('emits http.redirect.malformedLocation with the RAW header', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + + try { + const bad = countingResponse(302); + const transport = new FakeTransport([ + withLocation(bad.response, 'javascript:alert(1)'), + ]); + + await runThrough(redirectStep(), transport); + + const malformed = events.filter( + e => e.get('event') === 'http.redirect.malformedLocation', + ); + expect(malformed).toHaveLength(1); + // REDIR-28's carve-out: unredacted, because it never parsed into a URL. + expect(malformed[0]?.get('location.raw')).toBe('javascript:alert(1)'); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); + +describe('Phase 7b retrofit: the permitted-downgrade event', () => { + test('emits http.redirect.downgradePermitted when downgrade policy permits http redirect', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + const testLogger = createLogger((_level, fields) => { + events.push(new Map(fields)); + }); + setGlobalLogger(testLogger); + + try { + const hop = countingResponse(302); + const final = countingResponse(200); + const transport = new FakeTransport([ + withLocation(hop.response, 'http://example.com/downgraded'), + final.response, + ]); + + await runThrough(redirectStep({allowSchemeDowngrade: true}), transport); + + const downgrades = events.filter( + e => e.get('event') === 'http.redirect.downgradePermitted', + ); + expect(downgrades).toHaveLength(1); + expect(downgrades[0]?.get('from_url')).toBe('https://example.com/start'); + expect(downgrades[0]?.get('to_url')).toBe( + 'http://example.com/downgraded', + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); + +describe('Phase 7b retrofit: redirect rejection logging', () => { + test('emits http.redirect.rejected on rejected redirect or loop', async () => { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + const testLogger = createLogger((_level, fields) => { + events.push(new Map(fields)); + }); + setGlobalLogger(testLogger); + + try { + const hop = countingResponse(302); + const transport = new FakeTransport([ + withLocation(hop.response, 'https://example.com/start'), // loop to self + ]); + + await runThrough(redirectStep(), transport); + + const rejections = events.filter( + e => e.get('event') === 'http.redirect.rejected', + ); + expect(rejections).toHaveLength(1); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); +}); + +// --- REDIR-28 / XCUT-19: the rejection record --------------------------------------------------- + +/** A seed carrying both things OBS-11 and OBS-12 name: userinfo, and a non-allow-listed query value. */ +const SECRET_SEED = + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET'; +/** What `redactUrl()` makes of {@link SECRET_SEED}; the shape every URL field on this path must have. */ +const REDACTED_SEED = 'https://***:***@example.com/start?access_token=***'; + +/** + * Drives one redirect step to settlement over `transport`, capturing every record it emits, and + * restores the shipped no-op logger before returning. Rejections are swallowed: each caller here is + * asserting on the LOG, and the thrown error's own redaction is `errors.test.ts`'s row. + */ +async function recordsFromRun( + seed: Request, + transport: FakeTransport, + overrides?: Partial<RedirectSettings>, +): Promise<Map<string, unknown>[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await rejectionOf( + new Cursor({ + steps: [redirectStep(overrides)], + transport, + request: seed, + context: aRequestContext(seed), + }).advance(), + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + +/** The one `http.redirect.rejected` record, rendered field-by-field as a logger backend sees it. */ +function rejection( + events: Map<string, unknown>[], +): Map<string, unknown> | undefined { + const rejected = events.filter( + e => e.get('event') === 'http.redirect.rejected', + ); + expect(rejected).toHaveLength(1); + return rejected[0]; +} + +/** Asserts no field of `record` carries any of `secrets` in clear text -- XCUT-19's actual claim. */ +function expectNoSecret( + record: Map<string, unknown> | undefined, + secrets: readonly string[], +): void { + for (const field of record?.values() ?? []) { + for (const secret of secrets) { + expect(String(field)).not.toContain(secret); + } + } +} + +const SECRETS = ['alice', 'hunter2', 'SUPERSECRET', 'ALSOSECRET'] as const; + +describe('REDIR-28/XCUT-19: the rejection record never carries a raw URL', () => { + test('redacts the downgrade rejection cause and its url.full field', async () => { + const hop = countingResponse(301); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([ + withLocation(hop.response, 'http://example.com/next?code=ALSOSECRET'), + ]), + ); + + const record = rejection(records); + // The logger renders a cause as `name: message` (observability/logger.ts), so the MESSAGE is the + // log field. The 8192-byte field cap truncates nothing here, and would not have helped if it did. + expect(String(record?.get('cause'))).toStartWith('SchemeDowngradeError: '); + expect(String(record?.get('cause'))).toContain('***:***@'); + expect(String(record?.get('cause'))).toContain('access_token=***'); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('redacts the non-replayable-body rejection cause and its url.full field', async () => { + const oneShot = streamBody( + new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); + const hop = countingResponse(307); + const records = await recordsFromRun( + Request.newBuilder() + .method('POST') + .url(SECRET_SEED) + .body(oneShot) + .build(), + new FakeTransport([ + withLocation(hop.response, 'https://example.com/next?code=ALSOSECRET'), + ]), + {allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST'])}, + ); + + const record = rejection(records); + expect(String(record?.get('cause'))).toStartWith( + 'NonReplayableBodyError: ', + ); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('carries url.full on a rejection with no error, matching the other events', async () => { + // The `return-current` rejections -- loop detected, hop cap, malformed Location -- emit the same + // event with no cause. They still name the hop, and it is still redacted. The malformed-Location + // one is the sharpest case: REDIR-28 lets the sibling `malformedLocation` event log the header + // RAW, so this record is the only redacted URL anywhere on that path. + const hop = countingResponse(302); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([withLocation(hop.response, 'javascript:alert(1)')]), + ); + + const record = rejection(records); + expect(record?.has('cause')).toBe(false); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); +}); diff --git a/packages/core/src/redirect/redirect-step.ts b/packages/core/src/redirect/redirect-step.ts new file mode 100644 index 0000000..c324d3f --- /dev/null +++ b/packages/core/src/redirect/redirect-step.ts @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/redirect-step.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {invariant} from '../invariant.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; +import {originOf} from './cross-origin.js'; +import { + decide, + type Decision, + type RedirectContext, + type RedirectStopReason, +} from './decide.js'; +import {redirectSettings, type RedirectSettings} from './settings.js'; +import {getGlobalLogger} from '../observability/logger.js'; +import {redactHeaderValue, redactUrl} from '../observability/redaction.js'; + +/** Stable identity for pillar-slot occupancy and anchor matching (PIPE-6/PIPE-18). @internal */ +export const REDIRECT_STEP_TYPE: unique symbol = Symbol('dexpace.redirect'); + +/** + * REDIR-22(b): if deciding or building the follow-up throws, the current response MUST be closed before + * the error propagates. `decide()` is pure EXCEPT that it invokes `settings.predicate`, which is caller + * code and may throw for reasons this step cannot enumerate; `Request`/`Headers` builder validation is a + * second, thinner vector. A raw `decide()` call in the loop would let either escape with the hop's + * response still open, leaking the body. + * + * The decision's error stays PRIMARY. `Response.close()` rethrows whatever cancelling the body raised + * (everything but the `TypeError` a locked stream reports), so a bare `await response.close()` here + * would replace the caller's own error with the teardown failure -- the inversion RECOV-12 forbids. + * `releaseQuietly`/`withReleaseFailure` (4b's helpers, shared with the retry engine) keep the primary + * primary and hang the release failure off it as `suppressed`. + * + * Beyond that the error is rethrown unchanged. Note the deliberate asymmetry with retry, where + * RETRY-40 converts a throwing predicate into a typed illegal-state error: redirect's spec states no + * such conversion, so a caller's own error passes through as its own. + */ +async function decideOrClose( + response: Response, + context: RedirectContext, + settings: RedirectSettings, +): Promise<Decision> { + try { + return decide(response, context, settings); + } catch (error) { + throw withReleaseFailure(error, await releaseQuietly(response)); + } +} + +/** + * REDIR-28's loop-detected and malformed-Location events, the two that were blocked on `decide()` + * carrying a reason. Both fire alongside `http.redirect.rejected`, which says only THAT the hop + * stopped. + * + * The malformed-Location event logs the header **raw**, unredacted -- REDIR-28's own carve-out: + * the value failed to parse into a URL, so `redactUrl` has nothing to key off, and a port receiving + * credential-bearing malformed Location values inherits that exposure knowingly. + */ +function emitStopReason(stop: { + readonly reason: RedirectStopReason; + readonly request: Request; + readonly rawLocation: string | undefined; +}): void { + try { + const {reason, request, rawLocation} = stop; + if (reason === 'loop-detected') { + getGlobalLogger() + .atLevel('warning') + .event('http.redirect.loopDetected') + .field('url.full', redactUrl(request.url)) + // The header, not the resolved target: the target is `decide`'s own and never leaves it, + // and REDIR-27 lets the header be renamed, so 'location' names the POLICY to apply here. + .field('location', redactHeaderValue('location', rawLocation ?? '')) + .emit(); + return; + } + if (reason === 'malformed-location') { + getGlobalLogger() + .atLevel('warning') + .event('http.redirect.malformedLocation') + .field('url.full', redactUrl(request.url)) + .field('location.raw', rawLocation ?? '') + .emit(); + } + } catch { + // OBS-20: logger failure must never fail the request + } +} + +/** + * REDIR-28's rejection event. Carries `url.full` -- the hop the rejection is about -- through + * `redactUrl()`, the same field and the same policy `http.redirect.hop`, `loopDetected` and + * `malformedLocation` already use; before that this record named no URL at all, and the only URL a + * reader could recover was the raw one interpolated into the cause's message. + * + * The cause itself is safe to attach because `redirect/errors.ts` now builds both message texts from + * `redactUrl()` output. That is deliberately fixed at the ERROR rather than dropped here: `cause` is + * also what a caller's own `console.error` renders, and this step cannot redact that one. + */ +function emitRejected(request: Request, error?: unknown): void { + try { + const event = getGlobalLogger() + .atLevel('warning') + .event('http.redirect.rejected') + .field('url.full', redactUrl(request.url)); + if (error !== undefined) { + event.cause(error); + } + event.emit(); + } catch { + // OBS-20: logger failure must never fail the request + } +} + +function emitFollowEvents( + context: {readonly request: Request; readonly nextRequest: Request}, + response: Response, + hop: number, +): void { + try { + const {request, nextRequest} = context; + if ( + request.url.protocol === 'https:' && + nextRequest.url.protocol === 'http:' + ) { + getGlobalLogger() + .atLevel('warning') + .event('http.redirect.downgradePermitted') + .field('from_url', redactUrl(request.url)) + .field('to_url', redactUrl(nextRequest.url)) + .emit(); + } + + getGlobalLogger() + .atLevel('info') + .event('http.redirect.hop') + .field('hop', hop) + .field('status', response.status.code) + .field('url.full', redactUrl(nextRequest.url)) + .emit(); + } catch { + // OBS-20: log emission must never fail the request + } +} + +/** + * The REDIRECT pillar step (REDIR-1..REDIR-27, PIPE-40). + * + * `stage: 'REDIRECT'` is baked into the descriptor this factory returns, which is how PIPE-36 ("a shipped + * pillar family must not be relocatable out of its pillar") is satisfied structurally: steps are + * functions carrying a descriptor, not classes with a subclassable stage assignment. `ctx.fork` is + * asserted rather than checked -- REDIRECT is in `PILLAR_STAGES`, so its absence means the descriptor was + * installed somewhere it cannot be, which is a programmer error. + * + * Every dispatch, INCLUDING the first, goes through a fresh `ctx.fork()` -- never `ctx.next()` -- since + * the step may re-drive the downstream chain an unknown number of times and `next()`'s single-invocation + * guard would trip on the second hop (PIPE-15). + * + * **Response lifecycle** (PIPE-40/REDIR-22): a superseded intermediate response is closed before the next + * hop's dispatch; on `'fail'` the current response is closed before the error propagates; on every + * `'return-current'` outcome -- not-a-redirect, opted-out, malformed or missing Location, loop detected, + * hop cap reached -- the response is returned OPEN, the caller's to close. Close-responsibility passes + * outward. + * + * **What a caller catches.** Normally the decision's own error -- `SchemeDowngradeError`, + * `NonReplayableBodyError`, or whatever a caller predicate threw -- so `instanceof` works directly. In + * the one case where releasing that hop ALSO fails, the throw is a `SuppressedError`-shaped pairing + * (`suppress.ts`) carrying the decision error as `.error` and the release failure as `.suppressed`, per + * RECOV-12's "keep the primary primary". Code that must handle both reads `.error` when the caught value + * has one. The same shape 5a's retry engine already surfaces on its equivalent path. + * + * Iterative, not recursive, so it is stack-safe regardless of `maxHops` (REDIR-23): each `await` + * releases its iteration's frame before the next begins. + * + * `ctx.signal` is checked once per iteration, in the `follow` branch, BEFORE closing that hop's response + * and re-driving -- the only placement under which "return the current response, open" is meaningful, + * since `return-current` and `fail` already have their own disposition by the time it would run. No + * cancellable wait is needed (unlike retry, there is nothing to sleep between hops), so this is one cheap + * check rather than a timer race. + * + * **Caller obligation.** A caller installing this descriptor directly, rather than through + * `withRedirect()`, must also install `stripCrossOriginMarkerStep()` -- otherwise REDIR-11's internal + * marker reaches the transport whenever no auth step is present to strip it. + * + * **Exceeding `maxHops` does NOT throw** (REDIR-17). The hop cap returns the current 3xx response to + * the caller unfollowed, which is also what `maxHops: 0` reduces to -- there is no separate "disable + * redirects" branch. `decide.ts:205` is the gate. Stated here rather than after the tags below + * because TSDoc folds trailing prose into the preceding block tag, where it would render as part of + * a `@throws` description in the emitted `.d.ts`. + * + * @param overrides - redirect policy overrides; a zero-argument call yields the spec defaults. + * @returns the descriptor to install in a pipeline's REDIRECT slot. + * @throws SchemeDowngradeError - when an HTTPS to HTTP redirect is rejected by downgrade policy (REDIR-14, REDIR-15). + * @throws NonReplayableBodyError - when a redirect requiring body resend encounters a single-use body (REDIR-6, REDIR-22). + * + * @public + */ +export function redirectStep( + overrides?: Partial<RedirectSettings>, +): StepDescriptor { + const settings = redirectSettings(overrides); + return { + type: REDIRECT_STEP_TYPE, + stage: 'REDIRECT', + fn: async (seedRequest, ctx) => { + const {fork, signal} = ctx; + invariant( + fork !== undefined, + 'redirectStep must occupy the REDIRECT pillar stage', + ); + const seedUrl = seedRequest.url; + const seedOrigin = originOf(seedUrl); + const visited = new Set<string>([seedUrl.href]); + let request: Request = seedRequest; + let redirectsFollowed = 0; + + for (;;) { + const response = await fork()(request); + const context: RedirectContext = { + currentRequest: request, + seedOrigin, + visited, + redirectsFollowed, + }; + const decision = await decideOrClose(response, context, settings); + + if (decision.kind === 'return-current') { + if (response.status.isRedirect) { + emitRejected(request); + emitStopReason({ + reason: decision.reason, + request, + rawLocation: response.headers.get(settings.locationHeader), + }); + } + return response; + } + if (decision.kind === 'fail') { + const releaseError = await releaseQuietly(response); + emitRejected(request, decision.error); + throw withReleaseFailure(decision.error, releaseError); + } + if (signal?.aborted === true) return response; + + emitFollowEvents( + {request, nextRequest: decision.nextRequest}, + response, + redirectsFollowed + 1, + ); + + await response.close(); + visited.add(decision.nextRequest.url.href); + redirectsFollowed += 1; + request = decision.nextRequest; + } + }, + }; +} diff --git a/packages/core/src/redirect/settings.test.ts b/packages/core/src/redirect/settings.test.ts new file mode 100644 index 0000000..52f5170 --- /dev/null +++ b/packages/core/src/redirect/settings.test.ts @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/settings.test.ts +// Exercises: REDIR-17 (maxHops default 3; 0 is an ordinary value, not a special-cased branch -- decide()'s +// hop-cap gate is what makes it "disable following"), REDIR-26 (the allowed-method set is stored as an +// immutable defensive COPY, so mutating the caller's collection afterwards cannot change policy), +// REDIR-27 (the location header is configurable, defaulting to 'Location'), REDIR-20 (the predicate slot), +// REDIR-3/4/5 (the default allowed-method set and the 303 opt-in default). +import {describe, expect, test} from 'bun:test'; +import type {Method} from '../http/method.js'; +import {DEFAULT_ALLOWED_METHODS} from './codes.js'; +import {DEFAULT_REDIRECT_SETTINGS, redirectSettings} from './settings.js'; + +describe('defaults', () => { + test('ship the spec defaults', () => { + expect(DEFAULT_REDIRECT_SETTINGS.maxHops).toBe(3); + expect(DEFAULT_REDIRECT_SETTINGS.allow303).toBe(false); + expect(DEFAULT_REDIRECT_SETTINGS.allowSchemeDowngrade).toBe(false); + expect(DEFAULT_REDIRECT_SETTINGS.locationHeader).toBe('Location'); + expect([...DEFAULT_REDIRECT_SETTINGS.allowedMethods].sort()).toEqual( + [...DEFAULT_ALLOWED_METHODS].sort(), + ); + }); + + test('no predicate by default', () => { + expect(DEFAULT_REDIRECT_SETTINGS.predicate).toBeUndefined(); + }); + + test('a zero-config call yields the defaults', () => { + expect(redirectSettings().maxHops).toBe(3); + expect(redirectSettings().locationHeader).toBe('Location'); + }); + + test('one field can be overridden without restating the rest', () => { + const settings = redirectSettings({allow303: true}); + expect(settings.allow303).toBe(true); + expect(settings.maxHops).toBe(3); + }); +}); + +describe('validation', () => { + test('rejects a negative maxHops', () => { + expect(() => redirectSettings({maxHops: -1})).toThrow(); + }); + + test('accepts maxHops of 0 as an ordinary value, not a special case', () => { + expect(redirectSettings({maxHops: 0}).maxHops).toBe(0); + }); + + test('rejects a non-finite maxHops', () => { + expect(() => redirectSettings({maxHops: Number.NaN})).toThrow(); + expect(() => + redirectSettings({maxHops: Number.POSITIVE_INFINITY}), + ).toThrow(); + }); + + test('rejects a fractional maxHops rather than silently truncating it', () => { + // `2.5` would otherwise pass the cap gate for two hops and fail on the third -- a budget the + // caller never wrote. Same `Number.isInteger` guard `retryStep`'s per-call override applies. + expect(() => redirectSettings({maxHops: 2.5})).toThrow(); + }); + + test('rejects a blank locationHeader', () => { + expect(() => redirectSettings({locationHeader: ''})).toThrow(); + expect(() => redirectSettings({locationHeader: ' '})).toThrow(); + }); + + test('rejects a locationHeader carrying a byte HTTP-17 forbids in a header name', () => { + // `Headers.get()` neither trims nor validates -- it lower-cases and looks up. An unvalidated name + // would therefore never throw and never match: redirects silently unfollowed, no error anywhere. + // The predicate is the codebase's own header-name rule (control bytes, DEL, non-ASCII), the same + // one `HeadersBuilder` applies -- printable ASCII such as a space is legal here and stays legal. + expect(() => + redirectSettings({locationHeader: 'Loc\u0000ation'}), + ).toThrow(); + expect(() => redirectSettings({locationHeader: 'Loc\u00e1tion'})).toThrow(); + }); + + test('stores locationHeader trimmed, so a padded name still matches', () => { + expect( + redirectSettings({locationHeader: ' Location '}).locationHeader, + ).toBe('Location'); + }); + + test('accepts a caller-supplied predicate', () => { + const predicate = (): boolean => true; + expect(redirectSettings({predicate}).predicate).toBe(predicate); + }); +}); + +describe('immutability', () => { + test('the allowed-methods set is defensively copied (REDIR-26)', () => { + const caller = new Set<Method>(['GET']); + const settings = redirectSettings({allowedMethods: caller}); + caller.add('POST'); + expect(settings.allowedMethods.has('POST')).toBe(false); + }); + + test('the returned settings object is frozen', () => { + expect(Object.isFrozen(redirectSettings())).toBe(true); + }); + + test('DEFAULT_REDIRECT_SETTINGS is frozen', () => { + expect(Object.isFrozen(DEFAULT_REDIRECT_SETTINGS)).toBe(true); + }); +}); diff --git a/packages/core/src/redirect/settings.ts b/packages/core/src/redirect/settings.ts new file mode 100644 index 0000000..2e217fc --- /dev/null +++ b/packages/core/src/redirect/settings.ts @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/settings.ts +import {hasForbiddenNameByte} from '../http/ascii-validation.js'; +import type {Method} from '../http/method.js'; +import type {Response} from '../http/response.js'; +import {invariant} from '../invariant.js'; +import {DEFAULT_ALLOWED_METHODS} from './codes.js'; + +/** + * REDIR-20's read-only condition snapshot. Allocated for EVERY recognized 3xx -- including one carrying + * no usable `Location` -- and never on the non-redirect fast path (REDIR-21). + * + * `visited` is insertion-ordered and includes the current request's URI. + * + * @public + */ +export interface RedirectCondition { + /** The 3xx response being judged. Open; the predicate MUST NOT consume or close its body. */ + readonly response: Response; + /** How many hops this call has already followed, before the one under consideration. */ + readonly redirectsFollowed: number; + /** Every URI seen on this call, insertion-ordered, including the current request's (REDIR-19). */ + readonly visited: ReadonlySet<string>; +} + +/** + * REDIR-20: fully overrides the built-in code/method eligibility decision. It does NOT override the + * wire-safety mechanics that follow it -- credential stripping, the downgrade guard, body replayability, + * loop and hop-cap detection -- see `decide.ts`'s note on the scope of that override. + * + * @public + */ +export type RedirectPredicate = ( + condition: Readonly<RedirectCondition>, +) => boolean; + +/** + * Redirect policy. Every field is optional at the construction surface -- the internal + * `redirectSettings()` factory takes a `Partial` -- so a zero-config call yields the spec defaults and a + * caller can override one field without restating the rest. `redirectStep()` and `standardResilience()` + * both accept that same `Partial<RedirectSettings>`. + * + * The factory is named in plain prose rather than as a TSDoc link: it is internal and absent from the + * package barrel, so a link to it cannot resolve from a published declaration. + * + * @public + */ +export interface RedirectSettings { + /** REDIR-17: a non-negative integer, default 3. `0` disables following, with no special branch anywhere downstream. */ + readonly maxHops: number; + /** REDIR-3/REDIR-4: default `{GET, HEAD}`; stored as a defensive copy (REDIR-26). */ + readonly allowedMethods: ReadonlySet<Method>; + /** REDIR-5: 303 is not followed unless this is opted in. */ + readonly allow303: boolean; + /** REDIR-15: permits an HTTPS-to-HTTP hop, which is then surfaced observably by the step. */ + readonly allowSchemeDowngrade: boolean; + /** REDIR-27: the response header the target is read from, default `Location`. Stored trimmed. */ + readonly locationHeader: string; + /** REDIR-20: replaces the built-in code/method eligibility decision when present. */ + readonly predicate?: RedirectPredicate | undefined; +} + +/** + * The spec defaults, frozen. + * + * @internal + */ +export const DEFAULT_REDIRECT_SETTINGS: RedirectSettings = Object.freeze({ + maxHops: 3, + allowedMethods: DEFAULT_ALLOWED_METHODS, + allow303: false, + allowSchemeDowngrade: false, + locationHeader: 'Location', +}); + +/** + * Builds validated, frozen redirect settings. + * + * An invalid value is a PROGRAMMER error, the same split 5a's `retrySettings()` applied -- `invariant()`, + * not a new error leaf. `NonReplayableBodyError` and `SchemeDowngradeError` are the two OPERATIONAL + * failures a caller can legitimately hit mid-redirect; a bad `maxHops` is neither. + * + * `maxHops: 0` needs no special branch here or downstream: `decide()`'s hop-cap gate applies uniformly to + * every value, and a 0-hop budget simply fails it on the first follow attempt. + * + * `Object.freeze` is SHALLOW and does not disarm `Set.prototype.add` at all, so what REDIR-26 actually + * asks for is the defensive COPY below -- mutating the caller's collection afterwards cannot change + * policy. The `ReadonlySet` type is what keeps SDK-internal code from writing to it. A "frozen `Set`" + * would be a promise the runtime cannot keep; do not "fix" this with one. + * + * @param overrides - the fields to change; everything else takes the spec default. + * @returns frozen, validated settings. + * @throws InvariantViolation for a `maxHops` that is not a non-negative integer, or a `locationHeader` + * that is blank or carries a byte the header-name grammar forbids. + * + * @internal + */ +export function redirectSettings( + overrides?: Partial<RedirectSettings>, +): RedirectSettings { + const merged = {...DEFAULT_REDIRECT_SETTINGS, ...overrides}; + invariant( + Number.isInteger(merged.maxHops) && merged.maxHops >= 0, + `redirect maxHops must be a non-negative integer, got ${String(merged.maxHops)}`, + ); + // Trimmed and STORED trimmed, then validated as a header name. `HeadersBuilder` trims and validates + // on the way in, but `Headers.get()` does neither -- it lower-cases the string and looks it up. So an + // untrimmed or malformed `locationHeader` would not throw anywhere: it would simply never match, and + // every redirect would come back unfollowed with no error at any layer. Same class of mistake 5a's + // `attemptHeaderName` check exists for, with a quieter failure mode. + const locationHeader = merged.locationHeader.trim(); + invariant( + locationHeader.length > 0 && !hasForbiddenNameByte(locationHeader), + `redirect locationHeader must be a valid header name, got '${merged.locationHeader}'`, + ); + return Object.freeze({ + ...merged, + locationHeader, + allowedMethods: new Set(merged.allowedMethods), + }); +} diff --git a/packages/core/src/redirect/strip-marker-step.test.ts b/packages/core/src/redirect/strip-marker-step.test.ts new file mode 100644 index 0000000..5faedfb --- /dev/null +++ b/packages/core/src/redirect/strip-marker-step.test.ts @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/strip-marker-step.test.ts +// Exercises: REDIR-11(c) (the internal cross-origin marker is removed before dispatch, INDEPENDENTLY of +// whether a credential-attaching layer runs -- the porter caveat the spec names, and a live leak today +// since no auth step exists until Phase 5c). The guard is an ordinary single-invocation step: it calls +// ctx.next() and never forks. Also: withRedirect() installs the pillar step and the guard together, so a +// caller reaching for redirect support gets the safety net without knowing the marker exists. +import {describe, expect, test} from 'bun:test'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {PipelineBuilder} from '../pipeline/builder.js'; +import {Cursor} from '../pipeline/cursor.js'; +import {FakeTransport} from '../testing/fake-transport.js'; +import { + CROSS_ORIGIN_MARKER_HEADER, + withCrossOriginMarker, +} from './cross-origin.js'; +import {REDIRECT_STEP_TYPE} from './redirect-step.js'; +import { + STRIP_MARKER_STEP_TYPE, + stripCrossOriginMarkerStep, + withRedirect, +} from './strip-marker-step.js'; + +function aResponse(): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers(Headers.newBuilder().build()) + .body(null) + .build(); +} + +function aRequestContext(request: Request): ExecutionContext { + return createRequestContext(request); +} + +describe('stripCrossOriginMarkerStep', () => { + test('occupies POST_AUTH and is not a pillar step', () => { + const descriptor = stripCrossOriginMarkerStep(); + expect(descriptor.stage).toBe('POST_AUTH'); + expect(descriptor.type).toBe(STRIP_MARKER_STEP_TYPE); + }); + + test('clears a marker present on the request, then calls next (REDIR-11c)', async () => { + const marked = Request.newBuilder() + .url('https://example.com') + .headers(withCrossOriginMarker(Headers.newBuilder().build())) + .build(); + const transport = new FakeTransport([aResponse()]); + const cursor = new Cursor({ + steps: [stripCrossOriginMarkerStep()], + transport, + request: marked, + context: aRequestContext(marked), + }); + + await cursor.advance(); + + expect(transport.sendCount).toBe(1); + expect( + transport.calls[0]?.request.headers.get(CROSS_ORIGIN_MARKER_HEADER), + ).toBeUndefined(); + }); + + test('is a no-op when the marker is already absent', async () => { + const bare = Request.newBuilder() + .url('https://example.com') + .headers(Headers.newBuilder().add('X-Other', 'kept').build()) + .build(); + const transport = new FakeTransport([aResponse()]); + const cursor = new Cursor({ + steps: [stripCrossOriginMarkerStep()], + transport, + request: bare, + context: aRequestContext(bare), + }); + + await cursor.advance(); + + expect( + transport.calls[0]?.request.headers.has(CROSS_ORIGIN_MARKER_HEADER), + ).toBe(false); + expect(transport.calls[0]?.request.headers.get('X-Other')).toBe('kept'); // nothing else disturbed + // The guard runs on every request, so the common (unmarked) case must not rebuild anything: the + // request reaches the transport as the SAME instance it was handed. + expect(transport.calls[0]?.request).toBe(bare); + }); +}); + +describe('withRedirect', () => { + test('installs both the pillar step and the guard onto the builder', () => { + const runtime = withRedirect( + new PipelineBuilder(new FakeTransport([aResponse()])), + ).build(); + const types = runtime.steps.map(step => step.type); + expect(types).toContain(REDIRECT_STEP_TYPE); + expect(types).toContain(STRIP_MARKER_STEP_TYPE); + }); + + test('is idempotent -- a second call does not seat a second guard', () => { + // `PipelineBuilder.append` dedupes by `type` only for PILLAR stages (PIPE-6). POST_AUTH is not + // one, so without withRedirect()'s own `remove` the pillar half would be idempotent while the + // guard half silently duplicated. + const builder = new PipelineBuilder(new FakeTransport([aResponse()])); + const runtime = withRedirect(withRedirect(builder)).build(); + const types = runtime.steps.map(step => step.type); + expect(types.filter(type => type === STRIP_MARKER_STEP_TYPE)).toHaveLength( + 1, + ); + expect(types.filter(type => type === REDIRECT_STEP_TYPE)).toHaveLength(1); + }); + + test('the guard sits after the pillar step in flattened order', () => { + const runtime = withRedirect( + new PipelineBuilder(new FakeTransport([aResponse()])), + ).build(); + const types = runtime.steps.map(step => step.type); + expect(types.indexOf(REDIRECT_STEP_TYPE)).toBeLessThan( + types.indexOf(STRIP_MARKER_STEP_TYPE), + ); + }); + + test('a cross-origin redirect never reaches the wire carrying the marker (REDIR-11c)', async () => { + const hop = Response.newBuilder() + .request(Request.newBuilder().url('https://example.com/start').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(302)) + .headers( + Headers.newBuilder() + .setInbound('Location', 'https://other.example/next') + .build(), + ) + .body(null) + .build(); + const transport = new FakeTransport([hop, aResponse()]); + const runtime = withRedirect(new PipelineBuilder(transport)).build(); + const seed = Request.newBuilder().url('https://example.com/start').build(); + + await runtime.send(seed); + + // The redirect step set the marker for the (not-yet-existing) auth layer; the guard took it off + // again before the terminal dispatch. Without the guard this second send would carry it to the wire. + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.url.href).toBe( + 'https://other.example/next', + ); + expect( + transport.calls[1]?.request.headers.has(CROSS_ORIGIN_MARKER_HEADER), + ).toBe(false); + }); +}); diff --git a/packages/core/src/redirect/strip-marker-step.ts b/packages/core/src/redirect/strip-marker-step.ts new file mode 100644 index 0000000..d8a2699 --- /dev/null +++ b/packages/core/src/redirect/strip-marker-step.ts @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/redirect/strip-marker-step.ts +import type {PipelineBuilder} from '../pipeline/builder.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {clearCrossOriginMarker, hasCrossOriginMarker} from './cross-origin.js'; +import {redirectStep} from './redirect-step.js'; +import type {RedirectSettings} from './settings.js'; + +/** Stable identity for anchor matching (PIPE-18). @internal */ +export const STRIP_MARKER_STEP_TYPE: unique symbol = Symbol( + 'dexpace.redirect.strip-marker', +); + +/** + * REDIR-11(c)'s independent safety net. + * + * The requirement says the signal MUST be removed by the credential-attaching layer before dispatch -- + * and names the porter caveat that in the reference only the auth step strips it, so "a pipeline with no + * auth step, including the sync standard-resilience preset, forwards the internal marker to the + * transport", recommending a robust port strip it independently of whether a credential layer runs. + * + * That is not a future concern here: 5b ships before 5c, so today there IS no auth step, and without this + * guard the marker would reach the wire on every cross-origin hop. `POST_AUTH` is 4c's inert + * user-installable extension slot (PIPE-3) -- inside AUTH, outside SEND -- so the guard needs no change to + * 4c's `Cursor` and no coordination with 5c. When 5c ships, its auth step becomes the marker's real + * CONSUMER and first stripper; this stays installed as a redundant, idempotent backstop, since stripping + * an already-absent header costs nothing. + * + * An ordinary single-invocation step: it calls `ctx.next()` once and never re-drives, so it needs no fork. + * + * @returns the descriptor to install in a pipeline's POST_AUTH slot. + * + * @public + */ +export function stripCrossOriginMarkerStep(): StepDescriptor { + return { + type: STRIP_MARKER_STEP_TYPE, + stage: 'POST_AUTH', + // The guard runs on EVERY request through a redirect-enabled pipeline, while the marker is present + // only on a cross-origin hop -- so the common case must not pay for the rare one. Rebuilding is not + // cheap: `HeadersBuilder.build()` deep-copies every value list plus both name maps, and + // `Request.newBuilder()` re-parses the URL. The guard below is a single `Map.has`, and the + // "no-op when the marker is already absent" test pins the branch it introduces. + fn: (request, ctx) => { + if (!hasCrossOriginMarker(request.headers)) return ctx.next(); + return ctx.next( + request + .newBuilder() + .headers(clearCrossOriginMarker(request.headers)) + .build(), + ); + }, + }; +} + +/** + * Installs {@link redirectStep} and its bundled guard together, so a caller reaching for redirect support + * gets REDIR-11(c)'s safety net without needing to know the marker exists. A caller who installs + * `redirectStep()` directly against the builder's lower-level API is responsible for installing the guard + * too. + * + * Idempotent: calling it twice leaves one pillar step and one guard. A guard the caller had already + * installed is relocated to the tail of `POST_AUTH` rather than duplicated -- which is where it + * belongs anyway, since a step seated after it runs closer to `SEND` and could otherwise put the + * marker back. + * + * @param builder - the pipeline being assembled. + * @param overrides - redirect policy overrides; omitted yields the spec defaults. + * @returns the same builder, for chaining. + * + * @public + */ +export function withRedirect( + builder: PipelineBuilder, + overrides?: Partial<RedirectSettings>, +): PipelineBuilder { + // `remove` first so a second `withRedirect()` call does not seat a second guard. `append` dedupes by + // `type` only for PILLAR stages (PIPE-6), and `POST_AUTH` is not one -- so without this the pillar + // half of this call would be idempotent while the guard half silently duplicated. `remove` is a no-op + // when absent, and the type symbol is this module's own, so it can only ever match this guard. + return builder + .remove(STRIP_MARKER_STEP_TYPE) + .append(redirectStep(overrides)) + .append(stripCrossOriginMarkerStep()); +} diff --git a/packages/core/src/retry/attempt-stamp.test.ts b/packages/core/src/retry/attempt-stamp.test.ts new file mode 100644 index 0000000..9e27fca --- /dev/null +++ b/packages/core/src/retry/attempt-stamp.test.ts @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-stamp.test.ts +// Exercises: RETRY-38/RECOV-31 (1-based ordinal on a FRESH copy, never mutating the template, +// preserving the idempotency key and every other header, zero-allocation no-op when disabled). +import {describe, expect, test} from 'bun:test'; +import {Headers} from '../http/headers.js'; +import {Request} from '../http/request.js'; +import {stampAttempt} from './attempt-stamp.js'; + +function aRequest(): Request { + return Request.newBuilder() + .method('POST') + .url('https://example.com') + .headers( + Headers.newBuilder() + .add('Idempotency-Key', 'abc-123') + .add('X-Trace', 't1') + .build(), + ) + .build(); +} + +describe('stampAttempt', () => { + test('returns the ORIGINAL instance when no header name is configured (RETRY-38)', () => { + const request = aRequest(); + expect(stampAttempt(request, 2, undefined)).toBe(request); + }); + + test('writes the 1-based ordinal under the configured header', () => { + const stamped = stampAttempt(aRequest(), 3, 'X-Attempt'); + expect(stamped.headers.get('X-Attempt')).toBe('3'); + }); + + test('never mutates the captured template', () => { + const request = aRequest(); + stampAttempt(request, 3, 'X-Attempt'); + expect(request.headers.get('X-Attempt')).toBeUndefined(); + }); + + test('preserves the idempotency key and every other header', () => { + const stamped = stampAttempt(aRequest(), 2, 'X-Attempt'); + expect(stamped.headers.get('Idempotency-Key')).toBe('abc-123'); + expect(stamped.headers.get('X-Trace')).toBe('t1'); + }); + + test('preserves method, url, and body', () => { + const request = aRequest(); + const stamped = stampAttempt(request, 2, 'X-Attempt'); + expect(stamped.method).toBe(request.method); + expect(stamped.url.href).toBe(request.url.href); + expect(stamped.body).toBe(request.body); + }); + + test('re-stamping replaces rather than appends', () => { + const once = stampAttempt(aRequest(), 2, 'X-Attempt'); + const twice = stampAttempt(once, 3, 'X-Attempt'); + expect(twice.headers.get('X-Attempt')).toBe('3'); + }); +}); diff --git a/packages/core/src/retry/attempt-stamp.ts b/packages/core/src/retry/attempt-stamp.ts new file mode 100644 index 0000000..f5e2005 --- /dev/null +++ b/packages/core/src/retry/attempt-stamp.ts @@ -0,0 +1,34 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-stamp.ts +import type {Request} from '../http/request.js'; + +/** + * Stamps the 1-based attempt ordinal onto a FRESH copy of the request (RETRY-38/RECOV-31). + * + * The captured template is never mutated -- `Request` is immutable and frozen, so "stamping" means + * building a new value. `set()` replaces only the named header, so an idempotency key written + * upstream by `recovery/idempotency-key.ts` (RECOV-32) and every other header survive untouched. + * + * Disabled by default: when `headerName` is undefined this returns the ORIGINAL instance and + * allocates nothing, which is the zero-allocation no-op path RETRY-38 requires. + * + * @param request - the captured template, never mutated. + * @param attempt - the 1-based attempt ordinal. + * @param headerName - the header to stamp under, or undefined to disable stamping. + * @returns the stamped copy, or the original instance when stamping is disabled. + * + * @internal + */ +export function stampAttempt( + request: Request, + attempt: number, + headerName: string | undefined, +): Request { + if (headerName === undefined) return request; + return request + .newBuilder() + .headers( + request.headers.newBuilder().set(headerName, String(attempt)).build(), + ) + .build(); +} diff --git a/packages/core/src/retry/attempt-trail.test.ts b/packages/core/src/retry/attempt-trail.test.ts new file mode 100644 index 0000000..b5835be --- /dev/null +++ b/packages/core/src/retry/attempt-trail.test.ts @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.test.ts +// Exercises: RETRY-34 (the prior-attempt trail rides alongside the surfaced error rather than +// replacing it; the surfaced instance is skipped; a run with no priors leaves no trail behind), +// XCUT-1 (recording a trail never changes the surfaced value's class). +import {describe, expect, test} from 'bun:test'; +import {IoError} from '../io/errors.js'; +import {CancellationError} from '../seams/transport.js'; +import {recordAttempts, retryAttempts} from './attempt-trail.js'; + +describe('retryAttempts -- the read side', () => { + test('returns an empty list for an error that never went through the engine', () => { + expect(retryAttempts(new IoError('never retried'))).toEqual([]); + }); + + test('returns an empty list for a primitive, which cannot carry a trail at all', () => { + // RETRY-34's trail is keyed by identity, so a string, a number or a symbol throw passes + // through the engine unchanged and unannotated rather than being wrapped to make room. + expect(retryAttempts('a bare string throw')).toEqual([]); + expect(retryAttempts(42)).toEqual([]); + expect(retryAttempts(Symbol('thrown'))).toEqual([]); + expect(retryAttempts(null)).toEqual([]); + expect(retryAttempts(undefined)).toEqual([]); + }); + + test('returns the recorded attempts oldest first', () => { + const first = new IoError('first'); + const second = new IoError('second'); + const surfaced = new IoError('third'); + + recordAttempts(surfaced, [first, second]); + + expect(retryAttempts(surfaced)).toEqual([first, second]); + }); + + test('hands back a frozen list, so one caller cannot edit a trail another caller holds', () => { + const surfaced = new IoError('surfaced'); + recordAttempts(surfaced, [new IoError('prior')]); + + const attempts = retryAttempts(surfaced); + + expect(Object.isFrozen(attempts)).toBe(true); + }); + + test('does not read a trail off the cause chain -- only off the instance itself', () => { + const inner = new IoError('inner'); + recordAttempts(inner, [new IoError('prior')]); + const outer = new IoError('outer', {cause: inner}); + + expect(retryAttempts(outer)).toEqual([]); + }); +}); + +describe('recordAttempts -- the write side', () => { + test('leaves the class and identity of the error untouched (XCUT-1)', () => { + const surfaced = new CancellationError('operation cancelled'); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(surfaced).toBeInstanceOf(CancellationError); + expect(surfaced.name).toBe('CancellationError'); + }); + + test('adds no own property, so a JSON or structured-clone round trip is unchanged', () => { + const surfaced = new IoError('surfaced'); + const before = Object.getOwnPropertyNames(surfaced).sort(); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(Object.getOwnPropertyNames(surfaced).sort()).toEqual(before); + }); + + test('copies the trail, so a later push by the engine cannot mutate a published list', () => { + const surfaced = new IoError('surfaced'); + const trail: unknown[] = [new IoError('prior')]; + + recordAttempts(surfaced, trail); + trail.push(new IoError('added afterwards')); + + expect(retryAttempts(surfaced)).toHaveLength(1); + }); +}); + +describe('recordAttempts -- the values it can and cannot key on', () => { + test('is a no-op on a frozen error rather than throwing', () => { + // The reason the trail is a side table and not an own property: a foreign error may be frozen + // or non-extensible, and `defineProperty` inside the failure path of the engine would then replace + // the failure the caller cares about with a TypeError. + const surfaced = Object.freeze(new IoError('frozen by its author')); + const prior = new IoError('prior'); + + expect(() => { + recordAttempts(surfaced, [prior]); + }).not.toThrow(); + expect(retryAttempts(surfaced)).toEqual([prior]); + }); + + test('is a no-op for a primitive surfaced value', () => { + expect(() => { + recordAttempts('a bare string throw', [new IoError('prior')]); + }).not.toThrow(); + expect(retryAttempts('a bare string throw')).toEqual([]); + }); + + test('records against a thrown function, which is an object for these purposes', () => { + const surfaced = (): void => undefined; + const prior = new IoError('prior'); + + recordAttempts(surfaced, [prior]); + + expect(retryAttempts(surfaced)).toEqual([prior]); + }); +}); + +describe('recordAttempts -- a reused error instance', () => { + test('an empty trail clears any entry a previous run left on a reused instance', () => { + // Error singletons are ordinary in fakes and in transports that reuse one instance. The RETRY-34 + // clause "on eventual success the prior trail MUST be discarded" is worth nothing if the NEXT run + // to surface that same instance still reports the old one. + const reused = new IoError('reused across runs'); + recordAttempts(reused, [new IoError('from the first run')]); + + recordAttempts(reused, []); + + expect(retryAttempts(reused)).toEqual([]); + }); + + test('the latest recording wins for a reused instance', () => { + const reused = new IoError('reused across runs'); + const older = new IoError('from the first run'); + const newer = new IoError('from the second run'); + + recordAttempts(reused, [older]); + recordAttempts(reused, [newer]); + + expect(retryAttempts(reused)).toEqual([newer]); + }); +}); diff --git a/packages/core/src/retry/attempt-trail.ts b/packages/core/src/retry/attempt-trail.ts new file mode 100644 index 0000000..b526913 --- /dev/null +++ b/packages/core/src/retry/attempt-trail.ts @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.ts + +/** + * RETRY-34's prior-attempt trail, held in a side table keyed by the surfaced throwable rather than + * written onto it. + * + * A `WeakMap` because the key is the error a caller is about to catch: the entry has to disappear + * when that error does, and a `Map` here would pin every failed request's error graph -- including + * whatever the buffered `HttpStatusError` bodies hold -- for the life of the process. + */ +const attemptTrails = new WeakMap<object, readonly unknown[]>(); + +/** One shared frozen empty list, so the common "no trail" answer allocates nothing. */ +const NO_ATTEMPTS: readonly unknown[] = Object.freeze([]); + +/** + * The subset of throwables a `WeakMap` can key on. Objects and functions qualify; primitives do + * not, and a registered symbol throws when used as a weak key, so symbols are excluded outright + * rather than probed. + */ +function trailKey(value: unknown): object | undefined { + if (typeof value === 'function') return value; + return typeof value === 'object' && value !== null ? value : undefined; +} + +/** + * Records the errors of the earlier attempts against the error the retry engine is about to + * surface (`RETRY-34`). + * + * Written into a side table instead of onto the error for three reasons the engine cannot rule out + * about a throwable it did not construct: it may be frozen or otherwise non-extensible, so a + * `defineProperty` in the failure path would itself throw and replace the failure the caller cares + * about; it may be a primitive, which can carry nothing at all; and `suppressed` already means + * "the one secondary" on `SuppressedErrorLike`, so reusing the name would collide with + * `RECOV-12`'s pairing. + * + * `attempts` MUST already have the surfaced instance filtered out -- `RETRY-34`'s skip-self clause, + * applied by the engine's `attachTrail` (`engine.ts`), which is the one caller. An empty + * `attempts` DELETES any entry a previous run left, so a transport that reuses a single error + * instance across calls reports the trail of the run that just surfaced it rather than a stale one. + * + * The list is copied and frozen, so the engine's own mutable `trail` array cannot be observed + * growing after the fact. + * + * @param error - the throwable the engine is surfacing; a primitive is silently ignored. + * @param attempts - the earlier attempts' errors, oldest first, surfaced instance excluded. + * + * @internal + */ +export function recordAttempts( + error: unknown, + attempts: readonly unknown[], +): void { + const key = trailKey(error); + if (key === undefined) return; + if (attempts.length === 0) { + attemptTrails.delete(key); + return; + } + attemptTrails.set(key, Object.freeze([...attempts])); +} + +/** + * The errors of the attempts that came before the one you caught. + * + * The retry pillar surfaces the **final** attempt's own error, unwrapped: `instanceof` against it + * answers the same for one attempt as for ten, and a cancellation that ended a backoff wait arrives + * as `CancellationError` rather than as something carrying one. The earlier attempts are not + * discarded — they are recorded here, one entry per attempt that failed BEFORE the error you caught. + * A worked example is in `docs/sdk-documentation/pipelines.md`. + * + * **That is not an attempt count, and `length + 1` is not one either.** The arithmetic holds only + * when the surfaced error is itself an attempt's, and on three reachable paths it is not: a + * cancellation or timeout the engine observes at its `RETRY-32` gate is synthesized there rather + * than raised by a send; a failure from stamping the attempt header is raised before the request + * goes out; and a `Clock.sleep` that rejects for something other than an abort fails after the + * attempt it followed is already in the trail. On each of those the trail already accounts for every + * send, so adding one overstates it. Narrowing the catch does not rescue the sum — `abortToSdkError` + * yields `TransportFailureError` for a timeout signal, so even that class can reach you without a + * send behind it. + * + * Oldest first, and the error you passed in is never a member of its own trail (`RETRY-34`'s + * skip-self clause, which matters because a transport may reuse one error instance across + * attempts). A run that succeeded, a failure that was never retried, and any error this SDK did not + * surface from a retry loop all answer with an empty list — this never throws and never returns + * `undefined`. + * + * The result is frozen, and it is read by identity: an error reached through another error's + * `cause` has its own trail or none, never its wrapper's. + * + * @param error - the throwable a retrying pipeline surfaced; any value, not necessarily an `Error`. + * @returns the earlier attempts' errors, oldest first, or an empty list when there are none. + * + * @public + */ +export function retryAttempts(error: unknown): readonly unknown[] { + const key = trailKey(error); + if (key === undefined) return NO_ATTEMPTS; + return attemptTrails.get(key) ?? NO_ATTEMPTS; +} diff --git a/packages/core/src/retry/backoff.test.ts b/packages/core/src/retry/backoff.test.ts new file mode 100644 index 0000000..ed872eb --- /dev/null +++ b/packages/core/src/retry/backoff.test.ts @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/backoff.test.ts +// Exercises: RETRY-9 (initialDelay * multiplier^(attempt-1), 1-indexed, capped), RETRY-10 (symmetric +// jitter bounds, midpoint, j=0 identity, negative floors to zero), RETRY-11 (attempt < 1 rejected, +// overflow saturates -- INCLUDING at a zero initial delay, where `0 * Infinity` used to give NaN; +// audit #67 / #78), RETRY-43 (fixed delay disables backoff AND jitter). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {computeDelay, type BackoffSettings} from './backoff.js'; + +const SETTINGS: BackoffSettings = { + initialDelayMs: 200, + multiplier: 2, + maxDelayMs: 8000, + jitter: 0, +}; +const never = (): number => 0.5; + +describe('exponential schedule', () => { + test('attempt 1 is the initial delay, 1-indexed (RETRY-9)', () => { + expect(computeDelay(1, SETTINGS, never)).toBe(200); + }); + + test('each attempt multiplies the previous (RETRY-9)', () => { + expect(computeDelay(2, SETTINGS, never)).toBe(400); + expect(computeDelay(3, SETTINGS, never)).toBe(800); + expect(computeDelay(4, SETTINGS, never)).toBe(1600); + }); + + test('growth is clamped to maxDelayMs (RETRY-9)', () => { + expect(computeDelay(20, SETTINGS, never)).toBe(8000); + }); + + test('an overflowing attempt saturates to the cap instead of throwing (RETRY-11)', () => { + expect(computeDelay(5000, SETTINGS, never)).toBe(8000); + expect(Number.isFinite(computeDelay(5000, SETTINGS, never))).toBe(true); + }); + + test('attempt < 1 is a programmer error (RETRY-11)', () => { + expect(() => computeDelay(0, SETTINGS, never)).toThrow(); + expect(() => computeDelay(-1, SETTINGS, never)).toThrow(); + }); +}); + +/** + * RETRY-11's "saturating rather than throwing" has one hole, and it is the zero base. Every other + * accepted setting overflows into `Math.min`'s cap; `0 * Infinity` overflows into `NaN`, which + * `Math.min` propagates. Audit #67 / #78. + */ +describe('a zero initial delay (RETRY-11)', () => { + test('stays zero where the power overflows', () => { + // Downstream, NaN is worse than a large number: `overshootsBudget` reads false for it, the + // engine's `delayMs <= 0` guard reads false for it, and it lands in `Clock.sleep` as a + // RangeError that replaces the failure being retried. `retrySettings()` accepts both settings + // below (`initialDelayMs >= 0`, finite `multiplier >= 1`), so RETRY-11 covers them. + const hugeMultiplier: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + }; + expect(computeDelay(3, hugeMultiplier, never)).toBe(0); + + const manyAttempts: BackoffSettings = {...SETTINGS, initialDelayMs: 0}; + // 2 ** 1099 is Infinity: the first attempt at which the doubling schedule overflows a double. + expect(computeDelay(1100, manyAttempts, never)).toBe(0); + }); + + test('stays zero under jitter too (RETRY-10)', () => { + const jitteredZero: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + jitter: 1, + }; + expect(computeDelay(4, jitteredZero, () => 0)).toBe(0); + expect(computeDelay(4, jitteredZero, () => 1)).toBe(0); + }); + + test('property: every accepted schedule is finite and non-negative (RETRY-11)', () => { + // The ranges are exactly what `retrySettings()` admits, so a passing property means no + // configuration a caller can build reaches the engine as a non-finite delay. `initialDelayMs` + // is drawn through an explicit `constant(0)` arm: the failing region needs a zero base AND an + // overflowing power together, and 100 runs of an unbiased double never produced the pair. + const accepted = fc.record({ + initialDelayMs: fc.oneof( + fc.constant(0), + fc.double({min: 0, max: 1e9, noNaN: true}), + ), + multiplier: fc.double({min: 1, max: 1e300, noNaN: true}), + maxDelayMs: fc.double({min: 0, max: 1e9, noNaN: true}), + jitter: fc.double({min: 0, max: 1, noNaN: true}), + }); + + fc.assert( + fc.property( + fc.integer({min: 1, max: 5000}), + accepted, + (attempt, settings) => { + const delay = computeDelay(attempt, settings, never); + expect(Number.isFinite(delay)).toBe(true); + expect(delay).toBeGreaterThanOrEqual(0); + }, + ), + ); + }); +}); + +describe('symmetric jitter', () => { + const jittered: BackoffSettings = {...SETTINGS, jitter: 0.2}; + + test('jitter 0 returns the base delay unperturbed (RETRY-10)', () => { + expect(computeDelay(3, SETTINGS, () => 0)).toBe(800); + expect(computeDelay(3, SETTINGS, () => 1)).toBe(800); + }); + + test('the midpoint sample returns the base delay (RETRY-10)', () => { + expect(computeDelay(3, jittered, () => 0.5)).toBeCloseTo(800, 6); + }); + + test('the sample spans exactly [d(1-j/2), d(1+j/2)] (RETRY-10)', () => { + expect(computeDelay(3, jittered, () => 0)).toBeCloseTo(720, 6); + expect(computeDelay(3, jittered, () => 1)).toBeCloseTo(880, 6); + }); + + test('a negative sample floors to zero (RETRY-10)', () => { + const wide: BackoffSettings = { + initialDelayMs: 10, + multiplier: 1, + maxDelayMs: 10, + jitter: 1, + }; + expect(computeDelay(1, wide, () => -100)).toBe(0); + }); + + test('property: every sample lies inside the symmetric window (RETRY-10)', () => { + fc.assert( + fc.property( + fc.integer({min: 1, max: 12}), + fc.double({min: 0, max: 1, noNaN: true}), + fc.double({min: 0, max: 1, noNaN: true}), + (attempt, jitter, sample) => { + const settings: BackoffSettings = {...SETTINGS, jitter}; + const base = Math.min(200 * 2 ** (attempt - 1), 8000); + const delay = computeDelay(attempt, settings, () => sample); + expect(delay).toBeGreaterThanOrEqual(base * (1 - jitter / 2) - 1e-9); + expect(delay).toBeLessThanOrEqual(base * (1 + jitter / 2) + 1e-9); + }, + ), + ); + }); + + test('property: the unjittered delay never exceeds the cap and never decreases (RETRY-9)', () => { + fc.assert( + fc.property(fc.integer({min: 1, max: 200}), attempt => { + const delay = computeDelay(attempt, SETTINGS, never); + expect(delay).toBeLessThanOrEqual(SETTINGS.maxDelayMs); + expect(delay).toBeGreaterThanOrEqual( + computeDelay(Math.max(1, attempt - 1), SETTINGS, never), + ); + }), + ); + }); +}); + +describe('fixed delay (RETRY-43)', () => { + test('a fixed delay disables both backoff growth and jitter', () => { + const fixed: BackoffSettings = { + ...SETTINGS, + jitter: 0.5, + fixedDelayMs: 1234, + }; + expect(computeDelay(1, fixed, () => 0)).toBe(1234); + expect(computeDelay(9, fixed, () => 1)).toBe(1234); + }); + + test('a fixed delay of zero is honored, not treated as absent', () => { + expect(computeDelay(4, {...SETTINGS, fixedDelayMs: 0}, never)).toBe(0); + }); +}); diff --git a/packages/core/src/retry/backoff.ts b/packages/core/src/retry/backoff.ts new file mode 100644 index 0000000..b505258 --- /dev/null +++ b/packages/core/src/retry/backoff.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/backoff.ts +import {invariant} from '../invariant.js'; + +/** + * The pure-math half of the retry schedule (RETRY-9..RETRY-11, RETRY-43). Carried inside + * `RetrySettings`, never constructed standalone by a caller. + * + * @public + */ +export interface BackoffSettings { + /** The first attempt's delay in milliseconds, before any multiplier or jitter (RETRY-9). */ + readonly initialDelayMs: number; + /** The exponential growth factor applied per attempt: delay(n) = initialDelayMs * multiplier^n (RETRY-9). */ + readonly multiplier: number; + /** The ceiling the exponential schedule saturates at, in milliseconds (RETRY-11). */ + readonly maxDelayMs: number; + /** Symmetric jitter fraction in [0,1]; 0 disables perturbation (RETRY-10). */ + readonly jitter: number; + /** + * When set, forces a flat delay and makes the exponential path unreachable (RETRY-43). + * + * Deliberately NOT clamped to `maxDelayMs`: RETRY-43 describes the mode as "zeroing the base and + * cap so only the fixed delay applies", so the cap is part of the schedule this mode replaces + * rather than a bound that outlives it. A fixed delay longer than `maxDelayMs` is honored. + */ + readonly fixedDelayMs?: number | undefined; +} + +/** + * Draws uniformly from [delayMs*(1-jitter/2), delayMs*(1+jitter/2)], midpoint delayMs (RETRY-10). + * A negative sample from a hostile random source floors to zero rather than producing a negative + * delay. + */ +function applyJitter( + delayMs: number, + jitter: number, + random: () => number, +): number { + if (jitter === 0) return delayMs; + const width = delayMs * jitter; + return Math.max(0, delayMs - width / 2 + random() * width); +} + +/** + * The single backoff calculator (RETRY-13): `initialDelay * multiplier^(attempt-1)`, clamped to the + * cap, then jittered. `attempt` is 1-indexed, where 1 is the wait BEFORE the first retry (RETRY-9). + * + * Overflow-safe by construction (RETRY-11): a large attempt makes `**` return `Infinity`, which + * `Math.min` absorbs into the cap. It saturates; it never throws. + * + * Except at a zero base, where the saturation does not hold and the guard below is what supplies it. + * `0 * Infinity` is `NaN`, and `Math.min` propagates `NaN` rather than clamping it -- so + * `initialDelayMs: 0` with any multiplier above 1 produced a `NaN` delay at the attempt where the + * power overflows (`multiplier: 2` reaches it at attempt 1100; `multiplier: 1e200` at attempt 3). + * `retrySettings()` accepts both configurations. Downstream, `NaN` is worse than a large number: it + * fails every comparison, so the engine's budget check, its overshoot check and its `delayMs <= 0` + * short-circuit all read false and it arrives at `Clock.sleep`, which rejects with a `RangeError` + * that replaces the failure being retried. Short-circuiting the zero base before the power is taken + * is exact rather than a repair: the schedule's value there is `0` at every attempt, and jitter + * around `0` is `0` for any sample (audit #67 / #78). + * + * `random` is injected so jitter is assertable rather than statistical -- the same determinism seam + * CFG-15 wants for the clock. + * + * @param attempt - the 1-indexed retry ordinal; 1 is the wait before the first retry. + * @param settings - the schedule's shape. + * @param random - the uniform [0,1) source jitter draws from. + * @returns the delay in milliseconds. + * @throws InvariantViolation when `attempt` is below 1 -- a programmer error, not an operational one + * (RETRY-11). + * + * @internal + */ +export function computeDelay( + attempt: number, + settings: BackoffSettings, + random: () => number, +): number { + invariant( + attempt >= 1, + `retry attempt must be 1-indexed and >= 1, got ${String(attempt)}`, + ); + if (settings.fixedDelayMs !== undefined) return settings.fixedDelayMs; + // Before the power, not after: `0 * Infinity` is the one product `Math.min` cannot absorb. + if (settings.initialDelayMs === 0) return 0; + const growth = settings.initialDelayMs * settings.multiplier ** (attempt - 1); + return applyJitter( + Math.min(growth, settings.maxDelayMs), + settings.jitter, + random, + ); +} diff --git a/packages/core/src/retry/classify.test.ts b/packages/core/src/retry/classify.test.ts new file mode 100644 index 0000000..03ee970 --- /dev/null +++ b/packages/core/src/retry/classify.test.ts @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/classify.test.ts +// Exercises: RETRY-1 (single-sourced status set, 501/505 excluded), RETRY-2 (iterative +// identity-tracking cause walk, cycle-safe; and the I/O boundary the walk tests -- one case per +// error class in `io/errors.ts`, see the block below), RETRY-3 (retryability derived from status, +// not a stored flag), RETRY-4 (transport failures always retryable), RETRY-5/6/7 (re-sendability), +// RETRY-8 (both axes required), RETRY-23/24 (cancellation vs timeout), RETRY-25 (allow-list makes +// the fatal exclusion vacuous), RETRY-37 (configured set is authoritative -- widens AND narrows), +// TRANSPORT-20 (a no-response send surfaces as a retryable I/O subtype), +// XCUT-5 (the baked retryability flag comes from ONE shared status classifier covering 408/429/all +// 5xx except 501 and 505 -- asserted below. This port has no separately-cached boolean field: the +// classifier is a pure function of HttpStatusError.status, which never changes post-construction +// (XCUT-15), so querying it at any later time is equivalent to reading a flag baked at construction). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {HttpStatusError} from '../body/http-status-error.js'; +import {stringBody} from '../body/simple-bodies.js'; +import {streamBody} from '../body/stream-body.js'; +import type {Body} from '../body/body.js'; +import {Request} from '../http/request.js'; +import { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, + TransportFailureError, +} from '../io/errors.js'; +import {CancellationError} from '../seams/transport.js'; +import { + RETRYABLE_STATUSES, + isResendable, + isRetryableFailure, + isRetryableStatus, +} from './classify.js'; + +function aRequest(method: 'GET' | 'POST' | 'PUT', body?: Body): Request { + const builder = Request.newBuilder() + .method(method) + .url('https://example.com'); + return body === undefined ? builder.build() : builder.body(body).build(); +} + +describe('isRetryableStatus', () => { + test('408 and 429 are retryable', () => { + expect(isRetryableStatus(408)).toBe(true); + expect(isRetryableStatus(429)).toBe(true); + }); + + test('500-599 are retryable except 501 and 505', () => { + expect(isRetryableStatus(500)).toBe(true); + expect(isRetryableStatus(503)).toBe(true); + expect(isRetryableStatus(599)).toBe(true); + expect(isRetryableStatus(501)).toBe(false); + expect(isRetryableStatus(505)).toBe(false); + }); + + test('other statuses are not retryable', () => { + for (const code of [200, 201, 301, 400, 401, 404, 409, 418, 499, 600]) { + expect(isRetryableStatus(code)).toBe(false); + } + }); + + test('the exported set and the predicate are the same source', () => { + fc.assert( + fc.property(fc.integer({min: 100, max: 700}), code => { + expect(isRetryableStatus(code)).toBe(RETRYABLE_STATUSES.has(code)); + }), + ); + }); +}); + +describe('isRetryableFailure', () => { + test('an IoError is retryable', () => { + expect( + isRetryableFailure(new IoError('connection refused'), RETRYABLE_STATUSES), + ).toBe(true); + }); + + test('an IoError buried in the cause chain is retryable (RETRY-2)', () => { + const buried = new Error('wrapper', { + cause: new Error('middle', {cause: new IoError('reset')}), + }); + expect(isRetryableFailure(buried, RETRYABLE_STATUSES)).toBe(true); + }); + + test('a cyclic cause chain terminates instead of hanging (RETRY-2)', () => { + const first = new Error('first'); + const second = new Error('second', {cause: first}); + Object.defineProperty(first, 'cause', {value: second, configurable: true}); + + expect(isRetryableFailure(first, RETRYABLE_STATUSES)).toBe(false); + }); + + test('an HttpStatusError derives retryability from its status (RETRY-3)', () => { + expect( + isRetryableFailure( + new HttpStatusError(503, undefined, undefined), + RETRYABLE_STATUSES, + ), + ).toBe(true); + expect( + isRetryableFailure( + new HttpStatusError(501, undefined, undefined), + RETRYABLE_STATUSES, + ), + ).toBe(false); + }); + + test('the configured set is authoritative and can widen (RETRY-37)', () => { + const widened = new Set([...RETRYABLE_STATUSES, 404]); + expect( + isRetryableFailure( + new HttpStatusError(404, undefined, undefined), + widened, + ), + ).toBe(true); + }); + + test('the configured set is authoritative and can narrow (RETRY-37)', () => { + const narrowed = new Set([500]); + expect( + isRetryableFailure( + new HttpStatusError(503, undefined, undefined), + narrowed, + ), + ).toBe(false); + }); +}); + +/** + * One case per class in `io/errors.ts`, pinning the boundary RETRY-2's "an I/O error" is read as. + * + * `isIoError` accepts all six classes the file declares; `classify.ts`'s walk tests + * `instanceof IoError`, which two of them satisfy. That gap was undecided until audit #67 / #78 + * decided it (`docs/deviations.md` item 17): the branch means "the wire failed", so `IoError` and + * `TransportFailureError` retry and the four flat leaves do not. Each leaf case asserts BOTH halves + * -- that `isIoError` accepts the value, and what the classifier answers for it -- because the two + * disagreeing is the decision, and a test that only asserted the classifier would read as an + * oversight rather than a choice. + * + * These are the guard on re-parenting: moving any leaf back under `IoError`, or switching the branch + * to `isIoError`, turns four of them red instead of quietly making a deterministic failure retryable. + * Measured 2026-09-05 by making that one-line change: exactly these four fail. + */ +describe('the I/O boundary the cause-walk tests (RETRY-2/RETRY-4, TRANSPORT-20)', () => { + test('IoError itself is retryable', () => { + const error = new IoError('connection refused'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('TransportFailureError is retryable (TRANSPORT-20)', () => { + // The one class TRANSPORT-20 requires to BE an IoError. A send that produced no response is the + // canonical retryable condition (RETRY-4), and the `extends` is what carries it here. + const error = new TransportFailureError('ECONNREFUSED'); + expect(error).toBeInstanceOf(IoError); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('EndOfStreamError is NOT retryable, buried in a cause chain either', () => { + // The exact-length-copy contract inside io/, not a wire truncation: a short copy repeats on the + // next attempt. A truncated response is the transport's to report, as TransportFailureError. + const error = new EndOfStreamError(3, 8); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + // Asserted through a wrapper too: the walk is what would rescue it if the branch widened, so the + // shallow case alone would not catch a change made one hop up. + expect( + isRetryableFailure( + new Error('read failed', {cause: error}), + RETRYABLE_STATUSES, + ), + ).toBe(false); + }); + + test('SourceContractViolationError is NOT retryable', () => { + // A foreign source that returned zero bytes for a positive read (IO-17) is a programming error + // in the source, deterministic on re-send. + const error = new SourceContractViolationError('source returned 0 bytes'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('ClosedResourceError is NOT retryable', () => { + // Using a closed resource (IO-42) is a caller lifecycle error; the resource stays closed. + const error = new ClosedResourceError('response body'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('AllocationLimitError is NOT retryable', () => { + // A cap the same request hits again (IO-9); retrying spends the budget to fail identically. + const error = new AllocationLimitError(2 ** 32, 2 ** 31 - 1); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); +}); + +describe('isRetryableFailure -- cancellation, timeouts, and the allow-list', () => { + test('a user abort is never retryable (RETRY-23)', () => { + const controller = new AbortController(); + controller.abort(); + expect( + isRetryableFailure(controller.signal.reason, RETRYABLE_STATUSES), + ).toBe(false); + }); + + test('a CancellationError is never retryable, even nested (RETRY-23, XCUT-1)', () => { + // Phase 2 declares `CancellationError extends DexpaceError`, NOT the IoError family, so the + // allow-list already excludes it. Asserted rather than assumed: were it ever re-parented under + // IoError, cancellation would silently become a retryable condition and XCUT-1 would break. + const cancelled = new CancellationError('caller aborted'); + expect(isRetryableFailure(cancelled, RETRYABLE_STATUSES)).toBe(false); + expect( + isRetryableFailure( + new Error('send failed', {cause: cancelled}), + RETRYABLE_STATUSES, + ), + ).toBe(false); + }); + + test('a throwing cause accessor ends the walk instead of masking the failure', () => { + // `cause` is an ordinary property, so a lazily-built one can raise from the read. Classifying a + // failure must never replace it with a classification error. + const hostile = new IoError('connection refused'); + Object.defineProperty(hostile, 'cause', { + get() { + throw new Error('hostile accessor'); + }, + }); + const wrapper = new Error('wrapper'); + Object.defineProperty(wrapper, 'cause', { + get() { + throw new Error('hostile accessor'); + }, + }); + + expect(isRetryableFailure(hostile, RETRYABLE_STATUSES)).toBe(true); + expect(isRetryableFailure(wrapper, RETRYABLE_STATUSES)).toBe(false); + }); + + test('a timeout abort is retryable (RETRY-24)', () => { + const reason = new DOMException('The operation timed out.', 'TimeoutError'); + expect(isRetryableFailure(reason, RETRYABLE_STATUSES)).toBe(true); + }); + + test('a timeout abort wrapped as a cause is retryable (RETRY-24)', () => { + const reason = new DOMException('The operation timed out.', 'TimeoutError'); + expect( + isRetryableFailure( + new Error('send failed', {cause: reason}), + RETRYABLE_STATUSES, + ), + ).toBe(true); + }); + + test('an unlisted throwable is not retryable, no deny-list needed (RETRY-25)', () => { + expect( + isRetryableFailure( + new RangeError('Maximum call stack size exceeded'), + RETRYABLE_STATUSES, + ), + ).toBe(false); + expect(isRetryableFailure(new TypeError('bad'), RETRYABLE_STATUSES)).toBe( + false, + ); + expect(isRetryableFailure('a bare string throw', RETRYABLE_STATUSES)).toBe( + false, + ); + expect(isRetryableFailure(undefined, RETRYABLE_STATUSES)).toBe(false); + }); +}); + +describe('isResendable', () => { + test('a body-less idempotent request is re-sendable (RETRY-5/6)', () => { + expect(isResendable(aRequest('GET'))).toBe(true); + expect(isResendable(aRequest('PUT'))).toBe(true); + }); + + test('a bare POST is NOT re-sendable even with nothing to resend (RETRY-7)', () => { + expect(isResendable(aRequest('POST'))).toBe(false); + }); + + test('a POST with a replayable body is re-sendable (RETRY-5)', () => { + expect(isResendable(aRequest('POST', stringBody('payload')))).toBe(true); + }); + + test('a request with a non-replayable body is NOT re-sendable (RETRY-5)', () => { + const oneShot = streamBody( + new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); + const request = Request.newBuilder() + .method('POST') + .url('https://example.com') + .body(oneShot) + .build(); + expect(isResendable(request)).toBe(false); + }); +}); diff --git a/packages/core/src/retry/classify.ts b/packages/core/src/retry/classify.ts new file mode 100644 index 0000000..f911e70 --- /dev/null +++ b/packages/core/src/retry/classify.ts @@ -0,0 +1,114 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/classify.ts +import {HttpStatusError} from '../body/http-status-error.js'; +import {isIdempotent} from '../http/method.js'; +import type {Request} from '../http/request.js'; +import {IoError} from '../io/errors.js'; + +// Phase 7a retrofit: RETRY-1's status set and predicate previously lived here as a private +// `buildRetryableStatuses()`/`RETRYABLE_STATUSES`/`isRetryableStatus`. Phase 7a's CFG-35 promotes the +// exact same set to a utility at `config/retryable.js` (for callers with no retry-engine +// dependency); this module re-exports that single source instead of keeping a second definition +// (RETRY-13's single-sourcing mandate, structural under ES modules). +export {RETRYABLE_STATUSES, isRetryableStatus} from '../config/retryable.js'; + +/** True for the abort reason `AbortSignal.timeout()` produces, false for a caller abort (RETRY-23/24). */ +function isTimeoutAbort(value: unknown): boolean { + return ( + typeof value === 'object' && + value !== null && + 'name' in value && + (value as {readonly name: unknown}).name === 'TimeoutError' + ); +} + +/** + * Reads `.cause` without trusting it. `cause` is an ordinary property, so a throwable built with a + * lazy or hostile accessor can raise from the read itself -- and this walk runs while classifying a + * failure that already happened, where a throw would replace the transport error with a + * classification error and turn a retryable condition into a terminal one. RETRY-22's rule that a + * secondary failure can never mask the upstream one applies here for the same reason it applies to + * the pacing parser: ending the walk is always a safe answer, raising never is. + */ +function causeOf(value: unknown): unknown { + if (typeof value !== 'object' || value === null || !('cause' in value)) { + return undefined; + } + try { + return (value as {readonly cause: unknown}).cause; + } catch { + return undefined; + } +} + +/** + * Retryability as an ALLOW-list (RETRY-2): a throwable qualifies only if it, or something in its + * cause chain, is an I/O error, a timeout, or a status the caller configured as retryable. The walk + * is iterative and identity-tracking, so a cyclic `cause` chain terminates instead of spinning. + * + * The allow-list shape is why RETRY-25 needs no code: a stack-overflow `RangeError` is non-retryable + * because it was never opted in, not because it was screened out. A caller's `AbortError` is + * likewise non-retryable for free (RETRY-23), while a `TimeoutError` is explicitly listed + * (RETRY-24). A transport-level failure -- connection refused, TLS or DNS failure, peer reset -- + * surfaces as an `IoError` subclass and is therefore retryable unconditionally at this level + * (RETRY-4). + * + * **The I/O branch tests `instanceof IoError`, not `isIoError`, and the difference is the rule.** + * `io/errors.ts` groups six classes under `isIoError`, but only two of them descend from `IoError`: + * `IoError` itself, and `TransportFailureError` -- the class TRANSPORT-20 requires a send that + * produced no response to surface, and the reason that `extends` is a requirement rather than a + * modelling choice (`docs/deviations.md` item 17). Those two mean "the wire failed", and RETRY-2's + * "an I/O error" is read as exactly that boundary. The other four -- `EndOfStreamError`, + * `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` -- extend + * `DexpaceError` directly and are deliberately outside it: they are this package's own contract and + * lifecycle failures and are deterministic on re-send. A closed resource or a violated source + * contract is a caller programming error, an allocation cap is a limit the same request hits again, + * and `EndOfStreamError` is the exact-length-copy contract inside `io/` -- a *wire* truncation is the + * transport's to report, as a `TransportFailureError`. Widening this branch to `isIoError` would + * retry all four. Decided by audit #67 / #78; one case per class in `classify.test.ts` pins the + * answer, so a later re-parenting of any leaf under `IoError` changes a test rather than passing + * silently. + * + * @param error - whatever was thrown; any value, not necessarily an `Error`. + * @param statuses - the CONFIGURED set, authoritative on its own -- it both widens and narrows + * relative to `RETRYABLE_STATUSES`, and the built-in classifier is not AND-ed in (RETRY-37). + * @returns true when the failure is a retryable condition. + * + * @internal + */ +export function isRetryableFailure( + error: unknown, + statuses: ReadonlySet<number>, +): boolean { + const seen = new Set<unknown>(); + let current = error; + while (current !== undefined && current !== null && !seen.has(current)) { + seen.add(current); + // RETRY-3: derived from the carried status at classification time, never a stored per-subclass flag. + if (current instanceof HttpStatusError) return statuses.has(current.status); + // Deliberately `instanceof IoError`, never `isIoError` -- see the boundary paragraph above. + if (current instanceof IoError) return true; + if (isTimeoutAbort(current)) return true; + current = causeOf(current); + } + return false; +} + +/** + * The second, orthogonal axis (RETRY-5/RETRY-8): a body-less request is re-sendable iff its method + * is idempotent; a body-bearing one iff its body is replayable. A bare non-idempotent POST is + * therefore not re-sendable even though it has nothing to physically re-send -- the case RETRY-7 + * calls out explicitly. + * + * RETRY-6's `{GET, HEAD, OPTIONS, PUT, DELETE}` set is Phase 1's `http/method.ts` (HTTP-9), imported + * rather than restated. + * + * @param request - the request a retry would re-send. + * @returns true when the request may be sent again. + * + * @internal + */ +export function isResendable(request: Request): boolean { + const {body} = request; + return body === undefined ? isIdempotent(request.method) : body.replayable; +} diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts new file mode 100644 index 0000000..b1fdd20 --- /dev/null +++ b/packages/core/src/retry/engine.test.ts @@ -0,0 +1,1206 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/engine.test.ts +// Exercises: RETRY-7/8 (both axes gate), RETRY-20 (a hint replaces the schedule, unjittered), RETRY-22 +// (a pacing failure never masks the upstream failure), RETRY-26/31 (cancellable wait, zero delay +// inline), RETRY-27/RECOV-20 (total-timeout budget with per-attempt shrinking), RETRY-32 (no attempts +// after cancellation), RETRY-34 (the prior-attempt trail rides BESIDE the surfaced error, discarded +// on success, skip-self), XCUT-1 (the surfaced type does not depend on how many attempts ran -- the +// final attempt's own error is what the engine hands back, cancellation included), +// RETRY-35/RECOV-16 (body released before the wait, bounded buffering), RETRY-36/RECOV-19 (503,503,200 +// terminates on the 200; a surviving response is returned LIVE), RETRY-39/40 (delay precedence; a +// throwing override is non-fatal -- and a non-finite RETURN from one is the same case, falling back +// to the schedule and logging through the same event; audit #67 / #78), RETRY-42/RECOV-28 (per-call +// state). +import {describe, expect, test} from 'bun:test'; +import {HttpStatusError} from '../body/http-status-error.js'; +import type {Clock} from '../config/clock.js'; +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {IoError, TransportFailureError} from '../io/errors.js'; +import {failure, success, type Outcome} from '../recovery/outcome.js'; +import {CancellationError} from '../seams/transport.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {countingResponse} from '../testing/fake-transport.js'; +import {retryAttempts} from './attempt-trail.js'; +import {runWithRetry, type RetryConfig, type RetryDispatch} from './engine.js'; +import {retrySettings, type RetrySettings} from './settings.js'; + +const GET = Request.newBuilder().url('https://example.com').build(); +const BARE_POST = Request.newBuilder() + .method('POST') + .url('https://example.com') + .build(); + +/** + * The one remaining pairing this engine can build is RECOV-12's -- a release failure riding along + * with the primary it must not mask. The retry TRAIL is no longer a `SuppressedError` chain, so the + * only assertions below that use this predicate are the release ones. + * + * Asserted on SHAPE, never `instanceof SuppressedError`: the native class is absent on this + * package's Node floor (>=20.3), where `suppress()` returns a structural stand-in and an + * `instanceof` assertion would silently assert nothing. + */ +function isSuppressedShape(value: unknown): value is SuppressedErrorLike { + return ( + value instanceof Error && + value.name === 'SuppressedError' && + 'error' in value && + 'suppressed' in value + ); +} + +/** + * A fake Clock whose `now`/`monotonic` both advance only when a test advances `clockState.ms`, and + * whose `sleep` returns instantly while still honoring CFG-17's cancellation contract -- rejecting + * with the abort reason for an already-aborted signal. Modelling that half matters: the engine + * delegates its inter-attempt wait to `clock.sleep`, so a fake that always resolved would make + * RETRY-26's cancellation path untestable without a real timer. + */ +function fakeClock(clockState: {ms: number}): Clock { + return { + now: () => clockState.ms, + monotonic: () => clockState.ms, + sleep: (_ms, signal) => + signal?.aborted === true + ? Promise.reject(signal.reason as Error) + : Promise.resolve(), + }; +} + +/** A config whose clock advances only when a test advances it, jitter pinned to the midpoint. */ +function configOf( + overrides?: Partial<RetrySettings>, + clockState = {ms: 0}, +): RetryConfig { + return { + settings: retrySettings(overrides), + clock: fakeClock(clockState), + random: () => 0.5, + }; +} + +/** Serves outcomes in order; the last repeats. Records the requests it saw. */ +function scriptedDispatch( + script: readonly Outcome<Response>[], +): RetryDispatch & {calls: Request[]} { + const calls: Request[] = []; + const dispatch = (request: Request): Promise<Outcome<Response>> => { + calls.push(request); + return Promise.resolve( + script[Math.min(calls.length - 1, script.length - 1)] ?? + failure(new Error('empty script')), + ); + }; + return Object.assign(dispatch, {calls}); +} + +/** A clock that records every duration it is asked to sleep for, and returns immediately. */ +function recordingClock(slept: number[]): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => { + slept.push(durationMs); + return Promise.resolve(); + }, + }; +} + +/** + * `defaultClock`'s own precondition, modelled: `Clock.sleep` rejects a non-finite duration with a + * `RangeError` (`config/clock.ts:148-157`). A fake that slept for any duration at all would hide the + * bug; this guard is what surfaces it. + */ +function guardingClock(): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => + Number.isFinite(durationMs) + ? Promise.resolve() + : Promise.reject( + new RangeError( + `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, + ), + ), + }; +} + +/** One retryable failure, then a 200: the shortest script that drives exactly one delay decision. */ +function oneFailureThenSuccess(): RetryDispatch { + return scriptedDispatch([ + failure(new IoError('first')), + success(countingResponse(200).response), + ]); +} + +/** Installs a capturing global logger for the duration of `body` and returns what it emitted. */ +async function captureLogEvents( + body: () => Promise<void>, +): Promise<Map<string, unknown>[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await body(); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + +describe('eligibility (RETRY-7/8)', () => { + test('a non-retryable failure is surfaced after exactly one attempt', async () => { + const dispatch = scriptedDispatch([failure(new TypeError('bad'))]); + const outcome = await runWithRetry(GET, dispatch, configOf()); + + expect(dispatch.calls).toHaveLength(1); + expect(outcome.kind).toBe('failure'); + }); + + test('a bare POST is not retried even on a retryable failure (RETRY-7)', async () => { + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + await runWithRetry(BARE_POST, dispatch, configOf()); + + expect(dispatch.calls).toHaveLength(1); + }); + + test('a retryable failure on an idempotent request exhausts the budget', async () => { + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(dispatch.calls).toHaveLength(3); + }); + + test('maxAttempts of 1 disables retries entirely', async () => { + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + await runWithRetry(GET, dispatch, configOf({maxAttempts: 1})); + + expect(dispatch.calls).toHaveLength(1); + }); +}); + +describe('status-driven retry (RETRY-36)', () => { + test('503, 503, 200 terminates on the 200', async () => { + const first = countingResponse(503); + const second = countingResponse(503); + const third = countingResponse(200); + const dispatch = scriptedDispatch([ + success(first.response), + success(second.response), + success(third.response), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 0}), + ); + + expect(dispatch.calls).toHaveLength(3); + expect(outcome).toEqual(success(third.response)); + }); + + test('each discarded response is released before the next attempt (RETRY-35)', async () => { + const first = countingResponse(503); + const second = countingResponse(200); + const dispatch = scriptedDispatch([ + success(first.response), + success(second.response), + ]); + + await runWithRetry(GET, dispatch, configOf({fixedDelayMs: 0})); + + expect(first.cancelCount()).toBe(1); + expect(second.cancelCount()).toBe(0); + }); + + test('a response that SURVIVES the gates is returned live and unread', async () => { + const only = countingResponse(503); + const dispatch = scriptedDispatch([success(only.response)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 1}), + ); + + expect(outcome).toEqual(success(only.response)); + expect(only.cancelCount()).toBe(0); + }); + + test('a non-retryable error status is returned as a live response, never remapped', async () => { + const only = countingResponse(404); + const dispatch = scriptedDispatch([success(only.response)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 0}), + ); + + expect(outcome).toEqual(success(only.response)); + expect(only.cancelCount()).toBe(0); + }); +}); + +describe('delay resolution (RETRY-39/40)', () => { + test('a caller override wins over every other source', async () => { + const clock = {ms: 0}; + const config: RetryConfig = { + ...configOf({fixedDelayMs: 5000}, clock), + delayOverride: () => 0, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('reset')), + success(countingResponse(200).response), + ]); + + await runWithRetry(GET, dispatch, config); + + expect(dispatch.calls).toHaveLength(2); + }); + + test('a throwing override is non-fatal and falls back to the schedule (RETRY-40)', async () => { + const config: RetryConfig = { + ...configOf({fixedDelayMs: 0}), + delayOverride: () => { + throw new Error('override exploded'); + }, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('reset')), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(outcome.kind).toBe('success'); + expect(dispatch.calls).toHaveLength(2); + }); + + test('a finite override is honored unchanged, fractional and huge alike (RETRY-39)', async () => { + // The finiteness guard below screens `NaN` and the two infinities and nothing else. A fractional + // or very large delay is still a delay, and RETRY-39 gives the caller precedence over the + // schedule -- 5000 ms of `fixedDelayMs` here, which neither run waits. + const slept: number[] = []; + const clock = recordingClock(slept); + + for (const override of [0.5, Number.MAX_SAFE_INTEGER]) { + await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('reset'))]), + { + settings: retrySettings({maxAttempts: 2, fixedDelayMs: 5000}), + clock, + random: () => 0.5, + delayOverride: () => override, + }, + ); + } + + expect(slept).toEqual([0.5, Number.MAX_SAFE_INTEGER]); + }); +}); + +/** + * RETRY-40 makes a bad override non-fatal. A throw was handled; a non-finite RETURN was not, and it + * is the worse of the two, because `NaN` fails every comparison downstream instead of failing loudly + * at the override. Audit #67 / #78 reads the two as one case: drop the value, use the computed + * schedule, keep going. + */ +describe('a non-finite delayOverride is the throwing case (RETRY-40)', () => { + test('every non-finite value falls back to the computed schedule', async () => { + // Asserted on the delays the clock was ASKED for. Three sends alone would also pass against a + // fake that quietly slept for `NaN`, which is most of them. + for (const bad of [ + Number.NaN, + Number.POSITIVE_INFINITY, + Number.NEGATIVE_INFINITY, + ]) { + const slept: number[] = []; + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('connection refused')), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({ + maxAttempts: 3, + initialDelayMs: 200, + multiplier: 2, + jitter: 0, + }), + clock: recordingClock(slept), + random: () => 0.5, + delayOverride: () => bad, + }); + + expect(dispatch.calls).toHaveLength(3); + expect(slept).toEqual([200, 400]); + expect(outcome.kind).toBe('failure'); + } + }); + + test('it never reaches Clock.sleep as a duration, so the real failure survives', async () => { + // The reported symptom, against a clock that guards its input the way `defaultClock` does: the + // rejection was folded into the terminal failure by RETRY-33's catch-all, so `() => NaN` with + // `maxAttempts: 3` gave ONE send and surfaced a `RangeError` about `durationMs` in place of the + // transport failure being retried -- with the real error demoted to the trail. + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('first')), + failure(new TransportFailureError('second')), + failure(new TransportFailureError('third')), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + clock: guardingClock(), + random: () => 0.5, + delayOverride: () => Number.NaN, + }); + + expect(dispatch.calls).toHaveLength(3); + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect((outcome.error as Error).message).toBe('third'); + expect(retryAttempts(outcome.error)).toHaveLength(2); + }); +}); + +describe('server pacing hints (RETRY-20/22)', () => { + test('a malformed pacing header never masks the upstream failure (RETRY-22)', async () => { + const response = countingResponse(503) + .response.newBuilder() + .headers(Headers.newBuilder().add('Retry-After', 'garbage').build()) + .build(); + const dispatch = scriptedDispatch([ + success(response), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('success'); + }); + + test('a server pacing hint replaces the schedule for that decision (RETRY-20)', async () => { + const clock = {ms: 0}; + const response = countingResponse(503) + .response.newBuilder() + .headers(Headers.newBuilder().add('Retry-After', '0').build()) + .build(); + // fixedDelayMs would be 60s; the hint of 0 replaces it, so the test does not hang. + const dispatch = scriptedDispatch([ + success(response), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 60_000}, clock), + ); + + expect(outcome.kind).toBe('success'); + expect(dispatch.calls).toHaveLength(2); + }); +}); + +describe('total-timeout budget (RETRY-27)', () => { + test('an exhausted budget stops the loop', async () => { + const clock = {ms: 0}; + const config = configOf({totalTimeoutMs: 50, fixedDelayMs: 0}, clock); + const calls: number[] = []; + const counting: RetryDispatch = (_request, attempt) => { + calls.push(attempt); + clock.ms += 40; + return Promise.resolve(failure(new IoError('reset'))); + }; + + await runWithRetry(GET, counting, config); + + expect(calls).toEqual([1, 2]); + }); + + test('a delay that would overshoot the budget is suppressed, not merely clamped', async () => { + const clock = {ms: 0}; + const config = configOf( + {totalTimeoutMs: 100, fixedDelayMs: 500, maxAttempts: 5}, + clock, + ); + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + + await runWithRetry(GET, dispatch, config); + + // elapsed(0) + 500 > 100, so the loop surfaces after the first send rather than sleeping out the + // remaining 100ms and dispatching a second attempt with no budget left (RETRY-27, RECOV-20). + expect(dispatch.calls).toHaveLength(1); + }); + + test('a zero budget means unbounded, not immediately exhausted', async () => { + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + await runWithRetry( + GET, + dispatch, + configOf({totalTimeoutMs: 0, maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(dispatch.calls).toHaveLength(3); + }); +}); + +describe('cancellation (RETRY-26/32)', () => { + test('an already-aborted signal launches no attempt at all', async () => { + const controller = new AbortController(); + controller.abort(); + const dispatch = scriptedDispatch([ + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf(), + signal: controller.signal, + }); + + expect(dispatch.calls).toHaveLength(0); + expect(outcome.kind).toBe('failure'); + }); + + test('surfaces the SDK CancellationError, not the raw abort reason (N1/XCUT-1)', async () => { + const {CancellationError} = await import('../seams/transport.js'); + const controller = new AbortController(); + const reason = new Error('caller went away'); + controller.abort(reason); + const dispatch = scriptedDispatch([ + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf(), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + const error = outcome.kind === 'failure' ? outcome.error : undefined; + expect(error).toBeInstanceOf(CancellationError); + expect((error as Error).cause).toBe(reason); + }); + + test('aborting during the backoff wait stops the loop promptly', async () => { + const controller = new AbortController(); + const config: RetryConfig = { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }; + const dispatch: RetryDispatch = () => { + queueMicrotask(() => { + controller.abort(); + }); + return Promise.resolve(failure(new IoError('reset'))); + }; + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(outcome.kind).toBe('failure'); + }); +}); + +describe('the prior-attempt trail (RETRY-34)', () => { + test('the FINAL attempt error is surfaced, with the priors reachable beside it', async () => { + const first = new IoError('first'); + const second = new IoError('second'); + const dispatch = scriptedDispatch([failure(first), failure(second)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 2, fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // Not a wrapper: the surfaced value IS attempt 2's own error, so a caller's `instanceof` reads + // the same here as it does after a single attempt. + expect(outcome.error).toBe(second); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); + + test('the surfaced TYPE does not depend on how many attempts ran (XCUT-1)', async () => { + const one = await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('refused'))]), + configOf({maxAttempts: 1, fixedDelayMs: 0}), + ); + // Distinct instances, because `scriptedDispatch` repeats its last entry and RETRY-34's + // skip-self guard would otherwise empty the trail this row needs to be non-empty. + const three = await runWithRetry( + GET, + scriptedDispatch([ + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(one.kind).toBe('failure'); + expect(three.kind).toBe('failure'); + if (one.kind !== 'failure' || three.kind !== 'failure') return; + // Until 2026-09-05 the second of these was a `SuppressedError` and this row was false: the + // surfaced CLASS was a function of the attempt budget, which is what XCUT-1's conformance + // clause ("assert the surfaced error is the cancellation type") catches on the abort path. + expect(one.error).toBeInstanceOf(TransportFailureError); + expect(three.error).toBeInstanceOf(TransportFailureError); + expect(retryAttempts(three.error)).toHaveLength(2); + }); + + test('the trail is discarded entirely on eventual success', async () => { + const first = new IoError('first'); + const dispatch = scriptedDispatch([ + failure(first), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('success'); + // Nothing was recorded anywhere: the discarded failure carries no trail of its own either. + expect(retryAttempts(first)).toEqual([]); + }); +}); + +describe('the trail -- skip-self and single-attempt shapes (RETRY-34)', () => { + test('a reused instance never suppresses itself (RETRY-34 skip-self)', async () => { + const reused = new IoError('same instance every time'); + const dispatch = scriptedDispatch([failure(reused)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(outcome).toEqual(failure(reused)); + // Skip-self leaves nothing at all for a transport that reuses one instance across attempts. + expect(retryAttempts(reused)).toEqual([]); + }); + + test('a single failed attempt surfaces its error unwrapped', async () => { + const only = new TypeError('not retryable'); + const dispatch = scriptedDispatch([failure(only)]); + + expect(await runWithRetry(GET, dispatch, configOf())).toEqual( + failure(only), + ); + expect(retryAttempts(only)).toEqual([]); + }); + + test('a discarded 503 becomes a buffered HttpStatusError in the trail (RECOV-16)', async () => { + // The 503 is DISCARDED (attempt 1 retries), so it is remapped and buffered into the trail; the + // second attempt's IoError is what the loop surfaces. A 503 that instead SURVIVES the gates is + // never remapped -- covered by 'a response that SURVIVES the gates is returned live and unread'. + const dispatch = scriptedDispatch([ + success(countingResponse(503).response), + failure(new IoError('final')), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 2, fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(IoError); + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); + }); +}); + +describe('a discarded response OUTSIDE the 400-599 band (V14/N2, XCUT-8)', () => { + test('a widened retryable status yields RetryDiscardedResponseError, not a "successful exception"', async () => { + const {RetryDiscardedResponseError} = await import('./errors.js'); + // A caller may widen `retryableStatuses` to include a sub-400 code. `toHttpError` correctly + // returns null for it (BODY-31), and the engine used to fabricate + // `new HttpStatusError(200, ...)` -- exactly the "successful exception" XCUT-8 forbids, built by + // core itself. The trail now carries a leaf that says what actually happened. + const dispatch = scriptedDispatch([ + success(countingResponse(200).response), + failure(new IoError('final')), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({ + maxAttempts: 2, + fixedDelayMs: 0, + retryableStatuses: new Set([200]), + }), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + const [discarded] = retryAttempts(outcome.error); + expect(discarded).toBeInstanceOf(RetryDiscardedResponseError); + expect(discarded).not.toBeInstanceOf(HttpStatusError); + expect((discarded as {status: number}).status).toBe(200); + }); + + test('a discarded 404 still becomes an HttpStatusError -- the band is unchanged', async () => { + const dispatch = scriptedDispatch([ + success(countingResponse(404).response), + failure(new IoError('final')), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({ + maxAttempts: 2, + fixedDelayMs: 0, + retryableStatuses: new Set([404]), + }), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); + }); +}); + +describe('the inter-attempt wait (RETRY-26/31)', () => { + test('a positive delay is awaited through the injected clock, and the loop then continues', async () => { + const slept: number[] = []; + const config: RetryConfig = { + settings: retrySettings({fixedDelayMs: 250, maxAttempts: 2}), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: ms => { + slept.push(ms); + return Promise.resolve(); + }, + }, + random: () => 0.5, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('reset')), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(outcome.kind).toBe('success'); + // RETRY-13: the wait goes through the single-sourced Clock seam, never a private timer. + expect(slept).toEqual([250]); + }); + + test('a zero delay short-circuits the clock entirely (RETRY-31)', async () => { + let sleeps = 0; + const config: RetryConfig = { + settings: retrySettings({fixedDelayMs: 0, maxAttempts: 2}), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: () => { + sleeps += 1; + return Promise.resolve(); + }, + }, + random: () => 0.5, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('reset')), + success(countingResponse(200).response), + ]); + + await runWithRetry(GET, dispatch, config); + + expect(sleeps).toBe(0); + }); +}); + +describe('the inter-attempt wait -- degenerate and hostile delays', () => { + test('a negative delay from a caller override never reaches the clock (RETRY-40)', async () => { + let sleeps = 0; + const config: RetryConfig = { + settings: retrySettings({maxAttempts: 2}), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: () => { + sleeps += 1; + return Promise.reject(new RangeError('negative')); + }, + }, + random: () => 0.5, + delayOverride: () => -5, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('reset')), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(outcome.kind).toBe('success'); + expect(sleeps).toBe(0); + }); + + test('a clock whose sleep fails for a reason other than abort is not swallowed', async () => { + const config: RetryConfig = { + settings: retrySettings({fixedDelayMs: 10, maxAttempts: 3}), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: () => Promise.reject(new RangeError('misbehaving clock')), + }, + random: () => 0.5, + }; + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + + const outcome = await runWithRetry(GET, dispatch, config); + + // Folded into the outcome rather than escaping as a bare rejection, and it stops the loop + // instead of silently becoming an extra attempt. + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(RangeError); + expect(dispatch.calls).toHaveLength(1); + }); +}); + +describe('cancellation while an attempt is in flight (RETRY-32)', () => { + test('a retryable response arriving after the abort is released, not leaked (RETRY-32)', async () => { + const controller = new AbortController(); + const inFlight = countingResponse(503); + const dispatch: RetryDispatch = () => { + // Aborts while this very attempt is in flight, so its response arrives to a cancelled call. + controller.abort(); + return Promise.resolve(success(inFlight.response)); + }; + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 0, maxAttempts: 3}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + expect(inFlight.cancelCount()).toBe(1); + }); + + test('a response that ENDS the loop is handed to the caller live, even after an abort (RETRY-32)', async () => { + const controller = new AbortController(); + const arriving = countingResponse(200); + const dispatch: RetryDispatch = () => { + controller.abort(); + return Promise.resolve(success(arriving.response)); + }; + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 0}), + signal: controller.signal, + }); + + // Not a leak: ownership transfers to the caller, which is the only reader that could close it. + // RETRY-32's "closed rather than leaked" bites on responses the ENGINE discards, above. + expect(outcome).toEqual(success(arriving.response)); + expect(arriving.cancelCount()).toBe(0); + }); + + test('an abort raised WHILE the wait is pending settles it promptly (RETRY-26)', async () => { + const controller = new AbortController(); + const config: RetryConfig = { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }; + const dispatch: RetryDispatch = () => { + controller.abort(); + return Promise.resolve(failure(new IoError('reset'))); + }; + + const outcome = await runWithRetry(GET, dispatch, config); + + // The fake clock rejects with the abort reason; the engine absorbs it and the next iteration's + // RETRY-32 check is what actually stops the loop. + expect(outcome.kind).toBe('failure'); + }); +}); + +describe('a cancelled backoff surfaces the cancellation TYPE (XCUT-1)', () => { + /** Aborts from inside attempt 1, so the loop reaches its RETRY-32 check with a one-entry trail. */ + function abortAfterOneAttempt( + controller: AbortController, + first: unknown, + ): RetryDispatch & {sends: number} { + const dispatch = (): Promise<Outcome<Response>> => { + dispatch.sends += 1; + controller.abort(); + return Promise.resolve(failure(first)); + }; + dispatch.sends = 0; + return dispatch; + } + + test('the surfaced error is CancellationError, with the prior attempt beside it', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // `abortToSdkError` maps the abort to this type one line before the trail is attached; until + // 2026-09-05 the trail wrapper undid the mapping immediately, and a cancelled backoff ALWAYS + // has a non-empty trail -- so `instanceof CancellationError` was false for every one of them. + expect(outcome.error).toBeInstanceOf(CancellationError); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); + + test('the trail already covers every send, so length is NOT one less than the count', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // The surfaced error is SYNTHESIZED at the RETRY-32 gate, not raised by a send, so it is not an + // attempt's error and the trail already accounts for all of them. `length + 1` would say two + // sends where one happened -- which is why no TSDoc here offers that arithmetic. + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); +}); + +describe('the RETRY-32 gate can synthesize a TransportFailureError too', () => { + test('a TIMEOUT signal takes the same synthesized path, as TransportFailureError', async () => { + // `abortToSdkError` branches on `isTimeoutSignal`, so the engine's own RETRY-32 gate can + // synthesize a `TransportFailureError` too. Narrowing a catch to that class therefore does NOT + // guarantee the caught error came from a send. + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch: RetryDispatch & {sends: number} = Object.assign( + (): Promise<Outcome<Response>> => { + dispatch.sends += 1; + controller.abort(new DOMException('timed out', 'TimeoutError')); + return Promise.resolve(failure(first)); + }, + {sends: 0}, + ); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect(outcome.error).not.toBeInstanceOf(CancellationError); + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); +}); + +describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { + test('a throw from inside the attempt is folded into a failure outcome, trail intact', async () => { + const calls: number[] = []; + const dispatch: RetryDispatch = (_request, attempt) => { + calls.push(attempt); + if (attempt === 1) return Promise.resolve(failure(new IoError('first'))); + throw new RangeError('decision blew up'); + }; + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(calls).toEqual([1, 2]); + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(RangeError); + // RETRY-34: attempt 1's failure would have been lost had the throw escaped as a rejection. + expect((retryAttempts(outcome.error)[0] as Error).message).toBe('first'); + }); +}); + +describe('the trail with more than two entries (RETRY-34)', () => { + test('three distinct attempt failures list flat, oldest first', async () => { + const dispatch = scriptedDispatch([ + failure(new IoError('first')), + failure(new IoError('second')), + failure(new IoError('third')), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect((outcome.error as Error).message).toBe('third'); + // A flat list in wire order. The `SuppressedError` pair is binary, so N priors used to fold + // into a nested chain a caller had to walk; nothing about RETRY-34 asked for that shape. + expect( + retryAttempts(outcome.error).map(entry => (entry as Error).message), + ).toEqual(['first', 'second']); + }); +}); + +describe('a failing release never becomes primary (RECOV-12, RETRY-35)', () => { + /** + * A response the engine must close ITSELF. A caller-widened sub-400 status makes `toHttpError` + * return null without consuming or closing (BODY-31 hands it back intact), so the engine's own + * release is the first and only close -- and unlike the 4xx/5xx path, where the body is already + * drained and `cancel()` is a no-op, here the source's cancel hook really runs and can fail. + */ + function uncancellableResponse(): Response { + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw new IoError('cancel blew up'); + }, + }); + return Response.newBuilder() + .request(GET) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(204)) + .body(body) + .build(); + } + + test('a release that throws does not discard the retry decision it was released for', async () => { + let sends = 0; + const dispatch: RetryDispatch = () => { + sends += 1; + return Promise.resolve(success(uncancellableResponse())); + }; + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({ + maxAttempts: 2, + fixedDelayMs: 0, + retryableStatuses: new Set([204]), + }), + }); + + // The whole budget is spent and the surviving response is returned, exactly as if the release + // had succeeded. A bare `finally { await close() }` would instead have thrown the teardown + // failure out of the decision it was returning -- one send, and a cancel error where a retry + // decision belonged. + expect(sends).toBe(2); + expect(outcome.kind).toBe('success'); + }); +}); + +describe('a failing release -- masking and self-suppression', () => { + test('the drain failure stays primary when the release fails too', async () => { + const body = new ReadableStream<Uint8Array>({ + pull() { + throw new IoError('socket died mid-drain'); + }, + cancel() { + throw new IoError('cancel failed too'); + }, + }); + const hostile = Response.newBuilder() + .request(GET) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(503)) + .body(body) + .build(); + const dispatch = scriptedDispatch([success(hostile)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + const primary = isSuppressedShape(outcome.error) + ? outcome.error.error + : outcome.error; + expect((primary as Error).message).toBe('socket died mid-drain'); + }); + + test('a release failure is never suppressed under itself', async () => { + // `Response.close()` memoizes its release promise, and cancelling an ERRORED stream rejects with + // the stream's stored error rather than calling the cancel hook -- so the release hands back the + // very instance already propagating. Without an identity guard that value would suppress itself. + const body = new ReadableStream<Uint8Array>({ + pull() { + throw new IoError('socket died mid-drain'); + }, + cancel() { + throw new IoError('cancel failed too'); + }, + }); + const hostile = Response.newBuilder() + .request(GET) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(503)) + .body(body) + .build(); + const dispatch = scriptedDispatch([success(hostile)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(isSuppressedShape(outcome.error)).toBe(false); + }); +}); + +describe('budget precondition', () => { + test('a non-finite maxAttempts is rejected at the engine, not left to loop forever', async () => { + // Both adapters reach the engine with settings a caller supplied. `retryStep` guards the + // per-call override route; this is the guard for every other route, including + // `dispatchWithRetry`, which takes a RetryConfig straight from its caller. + const rogue = { + ...retrySettings({fixedDelayMs: 0}), + maxAttempts: Number.POSITIVE_INFINITY, + }; + const dispatch = scriptedDispatch([failure(new IoError('reset'))]); + + const reason = await runWithRetry(GET, dispatch, { + ...configOf(), + settings: rogue, + }).then( + () => undefined, + (error: unknown) => error, + ); + + expect((reason as Error).message).toContain('finite count >= 1'); + expect(dispatch.calls).toHaveLength(0); + }); +}); + +describe('per-call state (RETRY-42, RECOV-28)', () => { + test('concurrent invocations do not clobber each other’s budget', async () => { + const settings = retrySettings({maxAttempts: 3, fixedDelayMs: 0}); + const config: RetryConfig = { + settings, + clock: fakeClock({ms: 0}), + random: () => 0.5, + }; + const left = scriptedDispatch([failure(new IoError('left'))]); + const right = scriptedDispatch([failure(new IoError('right'))]); + + await Promise.all([ + runWithRetry(GET, left, config), + runWithRetry(GET, right, config), + ]); + + expect(left.calls).toHaveLength(3); + expect(right.calls).toHaveLength(3); + }); +}); + +describe('Phase 7b retrofit: structured retry logging', () => { + test('emits attemptFailed per retry and exhausted when attempts run out', async () => { + const events = await captureLogEvents(async () => { + await runWithRetry( + GET, + scriptedDispatch([ + failure(new IoError('first')), + failure(new IoError('second')), + failure(new IoError('third')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + }); + + const failedEvents = events.filter( + e => e.get('event') === 'http.retry.attemptFailed', + ); + expect(failedEvents).toHaveLength(2); + expect(failedEvents[0]?.get('attempt')).toBe(1); + expect(failedEvents[1]?.get('attempt')).toBe(2); + + const exhaustedEvents = events.filter( + e => e.get('event') === 'http.retry.exhausted', + ); + expect(exhaustedEvents).toHaveLength(1); + expect(exhaustedEvents[0]?.get('attempts')).toBe(3); + }); + + test('emits delayOverrideFailed when delayOverride throws', async () => { + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { + ...configOf({maxAttempts: 2, fixedDelayMs: 0}), + delayOverride: () => { + throw new Error('bad override'); + }, + }); + }); + + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe('Error: bad override'); + }); + + test('emits delayOverrideFailed when delayOverride returns a non-finite delay', async () => { + // "Treated exactly like one that throws" (RETRY-40) is a claim about the diagnostic too: an + // override dropped in silence is a schedule the operator cannot explain from the configuration. + // Same event, same level, same emit path -- only the cause differs, and it names the value. + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { + ...configOf({maxAttempts: 2, fixedDelayMs: 0}), + delayOverride: () => Number.NaN, + }); + }); + + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe( + 'delayOverride returned a non-finite delay: NaN', + ); + }); +}); diff --git a/packages/core/src/retry/engine.ts b/packages/core/src/retry/engine.ts new file mode 100644 index 0000000..056a6f3 --- /dev/null +++ b/packages/core/src/retry/engine.ts @@ -0,0 +1,474 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/engine.ts +import {toHttpError} from '../body/http-status-error.js'; +import {abortToSdkError} from '../cancellation.js'; +import {invariant} from '../invariant.js'; +import type {Clock} from '../config/clock.js'; +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {failure, type Outcome} from '../recovery/outcome.js'; +import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; +import {recordAttempts} from './attempt-trail.js'; +import {stampAttempt} from './attempt-stamp.js'; +import {computeDelay} from './backoff.js'; +import {RetryDiscardedResponseError} from './errors.js'; +import {isResendable, isRetryableFailure} from './classify.js'; +import {parsePacingHint} from './pacing.js'; +import type {RetrySettings} from './settings.js'; +import {getGlobalLogger} from '../observability/logger.js'; + +// Structured logging events (http.retry.attemptFailed, http.retry.exhausted, http.retry.delayOverrideFailed) +// are emitted through the global logger facade (OBS-39, RETRY-40). + +/** + * One attempt: dispatch the (possibly stamped) request and report the outcome without throwing. + * + * @internal + */ +export type RetryDispatch = ( + request: Request, + attempt: number, +) => Promise<Outcome<Response>>; + +/** + * Everything {@link runWithRetry} needs beyond the request and the dispatch callback, bundled into + * one trailing object so the function stays at ESLint's three-parameter ceiling. + * + * @internal + */ +export interface RetryConfig { + readonly settings: RetrySettings; + readonly signal?: AbortSignal | undefined; + /** + * Phase 7a's `Clock` seam (CFG-15). `clock.monotonic()` measures the total-timeout budget (CFG-16: + * elapsed-time math never uses wall-clock, which MAY move backwards); `clock.now()` supplies + * `parsePacingHint`'s wall-clock instant, since a `Retry-After` HTTP-date is an absolute instant, + * not an elapsed duration. Never `Date.now()` directly. + */ + readonly clock: Clock; + /** Injectable randomness -- jitter and the X-RateLimit-Reset spread both draw from it. */ + readonly random: () => number; + /** Highest-precedence delay source (RETRY-39). A throw, or a non-finite result, is non-fatal (RETRY-40). */ + readonly delayOverride?: + ((attempt: number) => number | undefined) | undefined; +} + +interface LoopState { + readonly config: RetryConfig; + readonly request: Request; + readonly attempt: number; + readonly startedAt: number; +} + +type Decision = + | {readonly kind: 'stop'; readonly outcome: Outcome<Response>} + | {readonly kind: 'retry'; readonly error: unknown; readonly delayMs: number}; + +function elapsed(state: LoopState): number { + return state.config.clock.monotonic() - state.startedAt; +} + +/** A budget of `undefined` or `0` disables the deadline (RETRY-27, RECOV-20). */ +function budgetExhausted(state: LoopState): boolean { + const budget = state.config.settings.totalTimeoutMs; + if (budget === undefined || budget === 0) return false; + return elapsed(state) >= budget; +} + +/** + * RETRY-27's separate belt-and-braces clause ("the computed delay is additionally clamped so it + * cannot overshoot the budget"). Deliberately defensive: {@link overshootsBudget} runs first on the + * same delay and stops the loop unless `delay <= budget - elapsed`, so this `Math.min` narrows + * nothing except across the clock drift between the two `elapsed()` reads. It ships because the + * requirement lists it separately from the abort, not because a test can drive it. + */ +function clampToBudget(delayMs: number, state: LoopState): number { + const budget = state.config.settings.totalTimeoutMs; + if (budget === undefined || budget === 0) return delayMs; + return Math.max(0, Math.min(delayMs, budget - elapsed(state))); +} + +/** + * RETRY-27/RECOV-20's third abort condition: a delay that would push cumulative elapsed time PAST + * the budget is SUPPRESSED and the last failure surfaced, not merely shortened. The clamp above is + * the requirement's separate belt-and-braces clause, not a substitute for this check -- without it + * the loop would sleep out the remainder of the budget and then dispatch one more attempt with + * nothing left. + */ +function overshootsBudget(delayMs: number, state: LoopState): boolean { + const budget = state.config.settings.totalTimeoutMs; + if (budget === undefined || budget === 0) return false; + return elapsed(state) + delayMs > budget; +} + +/** + * RETRY-40's diagnostic half, shared by both ways an override can fail. An ignored override is a + * schedule the operator cannot explain from the configuration alone, so neither way is silent. + */ +function reportOverrideFailure(cause: unknown): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('http.retry.delayOverrideFailed') + .cause(cause) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request or retry loop + } +} + +/** + * RETRY-40: a misbehaving user override is ignored, never fatal. Emits + * http.retry.delayOverrideFailed at warning level. + * + * TWO ways to misbehave, one answer. A throw was always handled here. A non-finite RETURN was not, + * and it was the more damaging of the two: `NaN` and the infinities pass every guard downstream -- + * `overshootsBudget` and `budgetExhausted` compare false, {@link waitFor}'s `delayMs <= 0` + * short-circuit compares false -- and arrive at `Clock.sleep`, which rejects a non-finite duration + * with a `RangeError`. RETRY-33's catch-all then folds that rejection into the terminal failure, so + * a `delayOverride` returning `NaN` under `maxAttempts: 3` produced ONE send and surfaced a + * `RangeError` about `durationMs`, with the transport failure it was retrying demoted to the trail. + * Audit #67 / #78 reads the two as one case: drop the value, use the computed schedule, keep going. + * + * The screen is finiteness alone. A finite negative keeps its existing behaviour -- {@link waitFor} + * continues inline without a timer (RETRY-31), which is the same answer the budget clamp already + * produces -- and a fractional or very large delay is a delay RETRY-39 gives the caller precedence + * for. + * + * The check sits OUTSIDE the `try` on purpose: a logger that throws while reporting a non-finite + * result must not be re-reported as an override that threw. + */ +function callerOverride(state: LoopState): number | undefined { + const {delayOverride} = state.config; + if (delayOverride === undefined) return undefined; + let delayMs: number | undefined; + try { + delayMs = delayOverride(state.attempt); + } catch (error) { + reportOverrideFailure(error); + return undefined; + } + if (delayMs !== undefined && !Number.isFinite(delayMs)) { + // A string cause, not a synthesized Error: nothing threw, and the value that was rejected is + // the whole diagnostic. + reportOverrideFailure( + `delayOverride returned a non-finite delay: ${String(delayMs)}`, + ); + return undefined; + } + return delayMs; +} + +/** RETRY-39: caller override -> server pacing hint -> fixed delay -> exponential backoff. */ +function resolveDelay(hint: number | null, state: LoopState): number { + const override = callerOverride(state); + if (override !== undefined) return override; + // RETRY-20/RECOV-22: a hint REPLACES the schedule for this one decision and receives no additional + // symmetric jitter. + if (hint !== null) return hint; + return computeDelay( + state.attempt, + state.config.settings, + state.config.random, + ); +} + +/** + * Turns a response the loop is DISCARDING into the throwable its trail entry carries, buffering a + * bounded copy of the body (RETRY-35/RECOV-16). Only ever called on a response that already failed + * the gates -- a surviving response is returned live and untouched. + */ +async function retire(response: Response): Promise<unknown> { + // `toHttpError` returns null for a status outside 400-599, reachable only when a caller widens the + // retryable set to include one. 5a fabricated `new HttpStatusError(<that status>, ...)` here, which + // carried a status outside BODY-31's band -- the "successful exception" XCUT-8 forbids, built by + // core itself, and the reason N2's "nothing in packages/core constructs one this way" was false. + // The discarded response still owes RETRY-34 a trail entry, so it gets a leaf that says what + // actually happened rather than one that claims an HTTP failure that did not occur. The response + // is NOT consumed on this path (BODY-31 hands it back intact), so the caller's `finally` closes + // it. + return ( + (await toHttpError(response)) ?? + new RetryDiscardedResponseError(response.status.code) + ); +} + +/** What the schedule step decided, before the release outcome is folded in. */ +interface Schedule { + readonly error: unknown; + readonly delayMs: number; + readonly overshootsBudget: boolean; +} + +/** + * Reads the pacing hint off the STILL-OPEN response and resolves the delay. + * + * Ordering is load-bearing: `toHttpError` drains the body and drops the headers, so the hint must be + * read first. + */ +async function scheduleFrom( + outcome: Outcome<Response>, + state: LoopState, +): Promise<Schedule> { + // The exception path skips the header step, having no headers (RETRY-39). + const hint = + outcome.kind === 'success' + ? parsePacingHint( + outcome.value.headers, + state.config.clock.now(), + state.config.random, + ) + : null; + const error = + outcome.kind === 'success' ? await retire(outcome.value) : outcome.error; + const delayMs = resolveDelay(hint, state); + // Tested BEFORE the clamp: the clamp would hide the overshoot it exists to report. + return { + error, + overshootsBudget: overshootsBudget(delayMs, state), + delayMs: clampToBudget(delayMs, state), + }; +} + +/** + * Retires the response the loop is discarding and schedules the wait, releasing the response on + * every exit (RETRY-35's second clause) without ever letting the release outcome become primary. + * + * The budget-overshoot abort lands HERE rather than in `decideRetry`'s gate block because the delay + * it tests is not known until the pacing hint has been read off the live response. By that point the + * response is already retired, so RETRY-27's "surface the last failure unchanged" surfaces the + * retired `HttpStatusError` as a Failure -- never a live response, which is what the gates above + * return. + */ +async function retireAndSchedule( + outcome: Outcome<Response>, + state: LoopState, +): Promise<Decision> { + const response = outcome.kind === 'success' ? outcome.value : undefined; + let schedule: Schedule; + try { + schedule = await scheduleFrom(outcome, state); + } catch (error) { + throw withReleaseFailure(error, await releaseQuietly(response)); + } + const error = withReleaseFailure( + schedule.error, + await releaseQuietly(response), + ); + return schedule.overshootsBudget + ? {kind: 'stop', outcome: failure(error)} + : {kind: 'retry', error, delayMs: schedule.delayMs}; +} + +function isRetryableOutcome( + outcome: Outcome<Response>, + settings: RetrySettings, +): boolean { + return outcome.kind === 'success' + ? settings.retryableStatuses.has(outcome.value.status.code) + : isRetryableFailure(outcome.error, settings.retryableStatuses); +} + +async function decideRetry( + outcome: Outcome<Response>, + state: LoopState, +): Promise<Decision> { + const {settings} = state.config; + // RETRY-8: BOTH axes must hold. Gates run BEFORE any remap so a surviving response stays live. + if (!isRetryableOutcome(outcome, settings)) return {kind: 'stop', outcome}; + if (!isResendable(state.request)) return {kind: 'stop', outcome}; + if (state.attempt >= settings.maxAttempts) return {kind: 'stop', outcome}; + if (budgetExhausted(state)) return {kind: 'stop', outcome}; + return retireAndSchedule(outcome, state); +} + +/** + * RETRY-34: prior failures ride ALONGSIDE the surfaced error, recorded in `attempt-trail.ts`'s side + * table and read back through the public `retryAttempts()`. The surfaced instance itself is skipped, + * so a reused throwable never appears in its own trail. On success the trail is discarded whole -- + * nothing is written, and the outcome is returned untouched. + * + * **The outcome's error is returned unchanged, class and identity intact.** Until 2026-09-05 this + * function wrapped it in a `SuppressedError` pair instead, which made the surfaced TYPE a function of + * how many attempts ran: one attempt surfaced `TransportFailureError`, three surfaced a wrapper with + * the `TransportFailureError` at `.error`. XCUT-1's conformance clause -- "assert the surfaced error + * is the cancellation type" -- is the row that catches it, because a cancellation during backoff + * ALWAYS has a non-empty trail: `abortToSdkError` maps the abort to `CancellationError` below, and + * the wrapper undid that mapping on the very next line. RETRY-34 asks for the prior failures to be + * "attached to the surfaced exception", which is the JVM's `addSuppressed` -- the exception stays + * what it is and grows a list -- not for the exception to be replaced by a container. + * + * `suppress()` keeps its RECOV-12 job elsewhere in this file: `withReleaseFailure` pairs a release + * failure with the primary it must not mask. That is a genuine two-value pairing; an N-entry attempt + * history folded into a binary shape was never one. + */ +function attachTrail( + outcome: Outcome<Response>, + trail: readonly unknown[], +): Outcome<Response> { + if (outcome.kind === 'success') return outcome; + recordAttempts( + outcome.error, + trail.filter(entry => entry !== outcome.error), + ); + return outcome; +} + +/** + * RETRY-26/31: the cancellable inter-attempt wait. + * + * Delegates to Phase 7a's `Clock.sleep` (CFG-17) rather than hand-rolling a second + * `setTimeout`-plus-abort-listener: `sleep` already races the timer against the signal, clears the + * timer on both exits (RETRY-45's scheduler hygiene, which has no scheduler object to own in this + * port), and rejects promptly for a signal that aborted earlier. Duplicating it here would be the + * same second-implementation the Phase 7a retrofit removed for the RFC 1123 parser and the + * retryable-status set, and it would put the wait outside the injected seam -- forcing real timers + * into a unit suite `docs/knowledge/harvested/testing.md` requires to be deterministic. + * + * A non-positive delay short-circuits before `sleep` is reached: it continues inline with no timer + * (RETRY-31), which is reachable after RETRY-17's past-instant hint and after the budget clamp, and + * it is also what keeps a caller `delayOverride` returning a negative number out of `sleep`'s + * negative-duration rejection (RETRY-40 makes a bad override non-fatal). It does NOT catch a + * non-finite one -- `NaN <= 0` is false -- which is why {@link callerOverride} screens those at the + * source rather than here. + * + * Cancellation RESOLVES here rather than propagating: RETRY-26 wants the loop's next iteration to + * observe the signal and stop through its own RETRY-32 path, so the abort rejection is the one + * expected failure and is deliberately absorbed. Any other rejection is re-thrown. + */ +async function waitFor(delayMs: number, config: RetryConfig): Promise<void> { + if (delayMs <= 0) return; + try { + await config.clock.sleep(delayMs, config.signal); + } catch (error) { + // The only tolerable rejection is the abort reason CFG-17 rejects with; anything else (a + // misbehaving injected clock) must not be swallowed into a silent extra attempt. + if (config.signal?.aborted !== true) throw error; + } +} + +/** One attempt: stamp, dispatch, and decide. Extracted so the loop can wrap it in a single catch. */ +async function runAttempt( + dispatch: RetryDispatch, + state: LoopState, +): Promise<Decision> { + const stamped = stampAttempt( + state.request, + state.attempt, + state.config.settings.attemptHeaderName, + ); + return decideRetry(await dispatch(stamped, state.attempt), state); +} + +function maybeEmitExhausted( + outcome: Outcome<Response>, + trailLength: number, + state: LoopState, +): void { + if (outcome.kind === 'failure' && trailLength > 0) { + try { + const elapsedMs = state.config.clock.monotonic() - state.startedAt; + getGlobalLogger() + .atLevel('info') + .event('http.retry.exhausted') + .field('attempts', state.attempt) + .field('elapsed_ms', elapsedMs) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request + } + } +} + +/** + * The one retry loop (RETRY-13/RETRY-14, RECOV-30). Both entry points -- the RETRY pillar step and + * the recovery-chain wrapper -- call this, so the schedule, the classifier, and the budget cannot + * drift. + * + * Every piece of per-call state is a local (RETRY-42/RECOV-28): concurrent invocations sharing one + * `RetryConfig` cannot clobber each other's attempt count or start instant. + * + * RETRY-30's trampoline requirement is satisfied by the language: an `await` loop is already + * iterative, so N retries build no continuation chain and no stack growth. RETRY-33's "every + * terminal path returns an Outcome" is honored literally -- an attempt that throws is folded into a + * failure outcome carrying the trail, rather than left to surface as a bare rejected promise that + * would drop RETRY-34's prior attempts on the floor. + * + * @param request - the captured template every attempt re-sends. Whatever the caller captured is + * already final: the pillar adapter passes the request arriving at the RETRY stage, and the + * recovery adapter passes the output of a request chain it applied ONCE, above this loop + * (RECOV-32 -- one idempotency key per logical request). This loop only ever copies it. + * @param dispatch - performs one attempt and reports its outcome without throwing. + * @param config - settings, clock, randomness, signal, and the optional delay override. + * @returns the terminal outcome. On failure the error is the FINAL attempt's own, unwrapped, with + * RETRY-34's prior-attempt trail recorded beside it for `retryAttempts()`. + * + * @internal + */ +export async function runWithRetry( + request: Request, + dispatch: RetryDispatch, + config: RetryConfig, +): Promise<Outcome<Response>> { + // The one precondition both adapters share. `retrySettings()` already enforces it on the + // configured route and `retryStep` re-enforces it on the per-call override route, but this is the + // single choke point every caller passes through -- and a non-finite budget does not fail loudly + // on its own: it makes the `attempt >= maxAttempts` gate permanently false, so the loop simply + // never stops. Asserted once per call, never per attempt. + invariant( + Number.isFinite(config.settings.maxAttempts) && + config.settings.maxAttempts >= 1, + `retry maxAttempts must be a finite count >= 1, got ${String(config.settings.maxAttempts)}`, + ); + const startedAt = config.clock.monotonic(); + const trail: unknown[] = []; + + for (let attempt = 1; ; attempt += 1) { + // RETRY-32: once the caller has cancelled, launch no further attempt. + // + // Mapped, not surfaced verbatim (N1/XCUT-1). The engine used to hand back `signal.reason` -- + // a bare `DOMException` named `AbortError` for an ordinary `AbortController` -- while the + // transport layer mapped the identical abort to `CancellationError`. A caller writing + // `catch (e) { if (e instanceof CancellationError) ... }` therefore handled a cancelled dispatch + // and silently missed a cancelled backoff. The raw reason is kept as `.cause`. + if (config.signal?.aborted === true) { + const cancellation = abortToSdkError(config.signal, config.signal.reason); + return attachTrail(failure(cancellation), trail); + } + + try { + const state: LoopState = { + config, + request, + attempt, + startedAt, + }; + const decision = await runAttempt(dispatch, state); + if (decision.kind === 'stop') { + maybeEmitExhausted(decision.outcome, trail.length, state); + return attachTrail(decision.outcome, trail); + } + + trail.push(decision.error); + + try { + getGlobalLogger() + .atLevel('info') + .event('http.retry.attemptFailed') + .field('attempt', attempt) + .field('delay_ms', decision.delayMs) + .cause(decision.error) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request or abort retry loop + } + await waitFor(decision.delayMs, config); + } catch (error) { + // RETRY-33 literally, not merely as a rejected promise. Three things under here can throw -- + // `stampAttempt`'s header build, `toHttpError`'s body drain, and a misbehaving injected + // clock's `sleep` -- and letting any of them escape would discard the whole suppressed trail + // RETRY-34 requires the surfaced failure to carry. + return attachTrail(failure(error), trail); + } + } +} diff --git a/packages/core/src/retry/errors.ts b/packages/core/src/retry/errors.ts new file mode 100644 index 0000000..bfaeccc --- /dev/null +++ b/packages/core/src/retry/errors.ts @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * A response the retry engine DISCARDED whose status is outside HTTP-11's 400-599 error band. + * + * Reachable only when a caller widens `RetrySettings.retryableStatuses` to include a non-error code: + * the engine then retries a 2xx or 3xx, and every response it discards still owes `RETRY-34` an + * entry in the prior-attempt trail `retryAttempts()` reads. `toHttpError` correctly returns `null` + * for such a status (`BODY-31` hands a non-error response back intact), so there is nothing for it + * to build. + * + * Until 2026-09-02 the engine fabricated `new HttpStatusError(200, …)` here — precisely the + * "successful exception" `XCUT-8` forbids, constructed by core itself, and contradicting + * `HttpStatusError`'s own documented invariant. This leaf exists so the trail can say what actually + * happened: a response was discarded by a caller-widened retry policy, which is not an HTTP failure. + * + * A two-level leaf under {@link DexpaceError}, deliberately: checkpoint §5.2 caps the taxonomy at two + * levels. `DomainModelError`, the one middle tier that existed purely as a grouping device, was deleted + * rather than grown, with the exported `isDomainModelError` guard taking over the narrowing it provided. + * `IoError` remains a middle tier under `TransportFailureError` only because `TRANSPORT-20` requires that + * subtyping and this module's own `classify.ts` cause-walk is load-bearing on it. + * + * @public + */ +export class RetryDiscardedResponseError extends DexpaceError { + /** + * The discarded response's status code, outside 400-599 by construction. + * + * Carried as a field rather than only interpolated into the message, per + * `docs/knowledge/harvested/error-handling.md` — so it survives serialization and reaches a + * structured log without anyone parsing the message back apart. + */ + readonly status: number; + + /** + * @param status - the discarded response's status code. + * @param options - standard error options; pass `{cause}` when wrapping a caught error. + */ + constructor(status: number, options?: ErrorOptions) { + super( + `retry discarded a response with status ${String(status)}, which is outside the 400-599 error band; a caller-widened retryableStatuses is the only way to reach this`, + options, + ); + this.status = status; + } +} diff --git a/packages/core/src/retry/pacing.test.ts b/packages/core/src/retry/pacing.test.ts new file mode 100644 index 0000000..85619d6 --- /dev/null +++ b/packages/core/src/retry/pacing.test.ts @@ -0,0 +1,246 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/pacing.test.ts +// Exercises: RETRY-15 (all recognized forms), RETRY-16/RECOV-23 (total, malformed -> null not 0), +// RETRY-17 (past instant -> 0), RETRY-18/RECOV-26 (365-day ceiling), RETRY-19 (strict decimal grammar +// before any float parse), RETRY-21/RECOV-24 (fixed precedence, first parseable wins), RECOV-25 +// (X-RateLimit-Reset positive jitter). +import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; +import {Headers} from '../http/headers.js'; +import {parsePacingHint} from './pacing.js'; + +const NOW = Date.UTC(2026, 0, 1, 0, 0, 0); +const noJitter = (): number => 0; + +function headersOf(entries: Record<string, string>): Headers { + let builder = Headers.newBuilder(); + for (const [name, value] of Object.entries(entries)) { + builder = builder.add(name, value); + } + return builder.build(); +} + +describe('Retry-After as delta-seconds (RETRY-15)', () => { + test('an integer is honored', () => { + expect( + parsePacingHint(headersOf({'Retry-After': '30'}), NOW, noJitter), + ).toBe(30_000); + }); + + test('a fractional value is honored to sub-second resolution', () => { + expect( + parsePacingHint(headersOf({'Retry-After': '1.5'}), NOW, noJitter), + ).toBe(1500); + }); + + test('zero is honored as an immediate retry', () => { + expect( + parsePacingHint(headersOf({'Retry-After': '0'}), NOW, noJitter), + ).toBe(0); + }); +}); + +describe('Retry-After as an HTTP-date (RETRY-15)', () => { + test('a full RFC 1123 date resolves to the delta', () => { + const value = 'Thu, 01 Jan 2026 00:00:10 GMT'; + expect( + parsePacingHint(headersOf({'Retry-After': value}), NOW, noJitter), + ).toBe(10_000); + }); + + test('a single-digit day is tolerated', () => { + const value = 'Thu, 1 Jan 2026 00:00:10 GMT'; + expect( + parsePacingHint(headersOf({'Retry-After': value}), NOW, noJitter), + ).toBe(10_000); + }); + + test('the informational weekday is ignored, even when wrong', () => { + const value = 'Mon, 01 Jan 2026 00:00:10 GMT'; + expect( + parsePacingHint(headersOf({'Retry-After': value}), NOW, noJitter), + ).toBe(10_000); + }); + + test('a date already in the past yields zero, not null (RETRY-17)', () => { + const value = 'Thu, 01 Jan 2026 00:00:00 GMT'; + expect( + parsePacingHint(headersOf({'Retry-After': value}), NOW + 5000, noJitter), + ).toBe(0); + }); + + test('a year below 100 is a valid past instant, so it yields zero (RETRY-17)', () => { + // Until Phase 7a's shared parser landed, this module's private one REJECTED a year in [0,99], + // and this case asserted null. `config/http-date.ts` reads the year literally instead (never + // `Date.UTC`, whose legacy mapping would turn 0026 into 1926), which makes this a well-formed + // HTTP-date already in the past -- and RETRY-17 governs that case: a valid past instant MUST + // yield a zero delay, distinct from RETRY-16's unparseable-value-yields-no-hint. Recorded at + // docs/work/mvp/2026-09-04-open-items-dissolution.md K20. + expect( + parsePacingHint( + headersOf({'Retry-After': 'Thu, 01 Jan 0026 00:00:00 GMT'}), + NOW, + noJitter, + ), + ).toBe(0); + }); + + test('an out-of-range field is rejected rather than rolled over (RETRY-16)', () => { + expect( + parsePacingHint( + headersOf({'Retry-After': 'Thu, 32 Jan 2026 00:00:10 GMT'}), + NOW, + noJitter, + ), + ).toBeNull(); + expect( + parsePacingHint( + headersOf({'Retry-After': 'Thu, 01 Jan 2026 24:00:10 GMT'}), + NOW, + noJitter, + ), + ).toBeNull(); + expect( + parsePacingHint( + headersOf({'Retry-After': 'Thu, 01 Foo 2026 00:00:10 GMT'}), + NOW, + noJitter, + ), + ).toBeNull(); + }); +}); + +describe('strict decimal screening (RETRY-19)', () => { + test('type-suffixed, hex-float, NaN, and Infinity forms are rejected', () => { + for (const value of [ + '30d', + '30f', + '0x1p3', + 'NaN', + 'Infinity', + '-Infinity', + '1e3', + '+30', + ' 30 ', + ]) { + expect( + parsePacingHint(headersOf({'Retry-After': value}), NOW, noJitter), + ).toBeNull(); + } + }); + + test('a negative delta maps to no hint, never a zero delay (RETRY-16)', () => { + expect( + parsePacingHint(headersOf({'Retry-After': '-5'}), NOW, noJitter), + ).toBeNull(); + }); +}); + +describe('millisecond variants (RETRY-15)', () => { + test('retry-after-ms is honored', () => { + expect( + parsePacingHint(headersOf({'retry-after-ms': '250'}), NOW, noJitter), + ).toBe(250); + }); + + test('x-ms-retry-after-ms is honored', () => { + expect( + parsePacingHint(headersOf({'x-ms-retry-after-ms': '250'}), NOW, noJitter), + ).toBe(250); + }); + + test('a malformed millisecond value falls through to no hint', () => { + expect( + parsePacingHint(headersOf({'retry-after-ms': '25.5'}), NOW, noJitter), + ).toBeNull(); + }); +}); + +describe('X-RateLimit-Reset (RETRY-15, RECOV-25)', () => { + test('an epoch-seconds reset resolves to the delta', () => { + const reset = String(Math.floor(NOW / 1000) + 10); + expect( + parsePacingHint(headersOf({'X-RateLimit-Reset': reset}), NOW, noJitter), + ).toBe(10_000); + }); + + test('positive jitter tops out at 120% of the delta (RECOV-25)', () => { + const reset = String(Math.floor(NOW / 1000) + 10); + expect( + parsePacingHint(headersOf({'X-RateLimit-Reset': reset}), NOW, () => 1), + ).toBeCloseTo(12_000, 6); + }); + + test('a past reset yields zero (RETRY-17)', () => { + const reset = String(Math.floor(NOW / 1000) - 10); + expect( + parsePacingHint(headersOf({'X-RateLimit-Reset': reset}), NOW, () => 1), + ).toBe(0); + }); +}); + +describe('precedence (RETRY-21)', () => { + test('numeric Retry-After beats every other form', () => { + const headers = headersOf({ + 'Retry-After': '30', + 'retry-after-ms': '1', + 'x-ms-retry-after-ms': '2', + 'X-RateLimit-Reset': String(Math.floor(NOW / 1000) + 99), + }); + expect(parsePacingHint(headers, NOW, noJitter)).toBe(30_000); + }); + + test('an unparseable Retry-After falls through to retry-after-ms, not to null', () => { + const headers = headersOf({ + 'Retry-After': 'garbage', + 'retry-after-ms': '250', + }); + expect(parsePacingHint(headers, NOW, noJitter)).toBe(250); + }); + + test('retry-after-ms beats x-ms-retry-after-ms', () => { + const headers = headersOf({ + 'retry-after-ms': '250', + 'x-ms-retry-after-ms': '999', + }); + expect(parsePacingHint(headers, NOW, noJitter)).toBe(250); + }); +}); + +describe('bounds and totality', () => { + test('a huge delta is clamped to the 365-day ceiling (RETRY-18)', () => { + const yearMs = 365 * 24 * 60 * 60 * 1000; + expect( + parsePacingHint(headersOf({'Retry-After': '99999999999'}), NOW, noJitter), + ).toBe(yearMs); + }); + + test('no pacing header at all yields no hint', () => { + expect(parsePacingHint(headersOf({}), NOW, noJitter)).toBeNull(); + }); + + test('property: the parser never throws for any header value (RETRY-16)', () => { + fc.assert( + fc.property(fc.string(), value => { + const headers = Headers.newBuilder() + .add('Retry-After', value.replaceAll(/[\r\n\0]/gu, '')) + .build(); + expect(() => parsePacingHint(headers, NOW, noJitter)).not.toThrow(); + }), + ); + }); + + test('property: the result is null or a finite non-negative number, never NaN (RETRY-16)', () => { + fc.assert( + fc.property(fc.string(), value => { + const headers = Headers.newBuilder() + .add('Retry-After', value.replaceAll(/[\r\n\0]/gu, '')) + .build(); + const hint = parsePacingHint(headers, NOW, noJitter); + if (hint === null) return; + expect(Number.isFinite(hint)).toBe(true); + expect(hint).toBeGreaterThanOrEqual(0); + }), + ); + }); +}); diff --git a/packages/core/src/retry/pacing.ts b/packages/core/src/retry/pacing.ts new file mode 100644 index 0000000..42703aa --- /dev/null +++ b/packages/core/src/retry/pacing.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/pacing.ts +// Phase 7a retrofit: this module previously hand-rolled its own private RFC 1123 parser here (a +// HTTP_DATE regex, a MONTHS table, and a local `parseHttpDate` function, tolerant of an +// informational weekday and a single-digit day -- never `Date.parse`, since JS date-string parsing +// is permissive and non-standardized across engines, the opposite of RETRY-16's totality mandate). +// Phase 7a's `config/http-date.ts` is a superset (it adds the formatter this module never needed) +// built to the identical grammar, so that private copy is deleted and this line imports the shared +// one instead -- one RFC 1123 parser in the codebase, not two. +import {parseHttpDate} from '../config/http-date.js'; +import type {Headers} from '../http/headers.js'; + +/** RETRY-18/RECOV-26: every computed delta is clamped to this ceiling before use. */ +const MAX_PACING_MS = 365 * 24 * 60 * 60 * 1000; + +/** + * RETRY-19: the strict decimal grammar that screens a value BEFORE any float parse. Deliberately + * rejects a leading sign, exponent notation, whitespace, and every type-suffixed or hex-float form -- + * `Number()` would happily accept several of them and produce a wildly wrong instant. + */ +const DECIMAL_SECONDS = /^\d+(?:\.\d+)?$/u; +const DECIMAL_INTEGER = /^\d+$/u; + +function clampPacing(deltaMs: number): number { + return Math.min(Math.max(0, deltaMs), MAX_PACING_MS); +} + +function parseDeltaSeconds(raw: string): number | null { + if (!DECIMAL_SECONDS.test(raw)) return null; + const seconds = Number(raw); + return Number.isFinite(seconds) ? seconds * 1000 : null; +} + +function parseIntegerValue(raw: string | undefined): number | null { + if (raw === undefined || !DECIMAL_INTEGER.test(raw)) return null; + const value = Number(raw); + return Number.isFinite(value) ? value : null; +} + +function parseRetryAfter(raw: string, nowMs: number): number | null { + const seconds = parseDeltaSeconds(raw); + if (seconds !== null) return clampPacing(seconds); + const instant = parseHttpDate(raw); + return instant === null ? null : clampPacing(instant - nowMs); +} + +function parseRateLimitReset( + headers: Headers, + nowMs: number, + random: () => number, +): number | null { + const epochSeconds = parseIntegerValue(headers.get('X-RateLimit-Reset')); + if (epochSeconds === null) return null; + const delta = clampPacing(epochSeconds * 1000 - nowMs); + // RECOV-25: positive jitter to [100%,120%] so many clients released at one reset instant do not + // stampede. A literal Retry-After receives no such perturbation (RETRY-20). + return delta === 0 ? 0 : clampPacing(delta * (1 + random() * 0.2)); +} + +/** + * Resolves a server pacing hint from a response's headers, honoring the fixed precedence of + * RETRY-21/RECOV-24: `Retry-After` numeric, then `Retry-After` as an HTTP-date, then + * `retry-after-ms`, then `x-ms-retry-after-ms`, then `X-RateLimit-Reset`. First parseable value + * wins. + * + * TOTAL by contract (RETRY-16/RECOV-23): it never throws for any input. Malformed, negative, or + * out-of-range values map to `null` -- "no hint" -- so the caller falls back to exponential backoff. + * They MUST NOT map to `0`, which would hammer a server that just asked for room. `0` is reserved + * for a validly-parsed instant already in the past (RETRY-17). + * + * @param headers - the discarded response's headers, read while it is still live. + * @param nowMs - the wall-clock instant the date forms are measured against. + * @param random - the uniform [0,1) source RECOV-25's reset jitter draws from. + * @returns milliseconds to wait, or `null` when no usable hint is present. + * + * @internal + */ +export function parsePacingHint( + headers: Headers, + nowMs: number, + random: () => number, +): number | null { + const retryAfter = headers.get('Retry-After'); + if (retryAfter !== undefined) { + const parsed = parseRetryAfter(retryAfter, nowMs); + if (parsed !== null) return parsed; + } + const deltaMs = + parseIntegerValue(headers.get('retry-after-ms')) ?? + parseIntegerValue(headers.get('x-ms-retry-after-ms')); + if (deltaMs !== null) return clampPacing(deltaMs); + return parseRateLimitReset(headers, nowMs, random); +} diff --git a/packages/core/src/retry/retry-dispatch.test.ts b/packages/core/src/retry/retry-dispatch.test.ts new file mode 100644 index 0000000..de83837 --- /dev/null +++ b/packages/core/src/retry/retry-dispatch.test.ts @@ -0,0 +1,260 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/retry-dispatch.test.ts +// Exercises: RECOV-17..20 (the recovery stack's retry lands here), RECOV-32 (one idempotency key per +// LOGICAL request -- the strategy runs once and every wire send carries its result), RETRY-38 (the +// per-attempt ordinal is stamped on a fresh copy and preserves that key), RETRY-44 (fresh per-attempt +// state below the retry point; upstream steps do not run between attempts), RECOV-2 (a request-chain +// throw still meets the response and recovery hooks), RETRY-13/14/RECOV-30 (both entry points share +// one engine, so the schedule cannot drift). +import {describe, expect, test} from 'bun:test'; +import {stringBody} from '../body/simple-bodies.js'; +import type {Clock} from '../config/clock.js'; +import {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import {IoError} from '../io/errors.js'; +import {idempotencyKeyStep} from '../recovery/idempotency-key.js'; +import {failure, success, type Outcome} from '../recovery/outcome.js'; +import {RequestRecoveryChain} from '../recovery/request-chain.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, +} from '../recovery/response-chain.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {dispatchWithRetry, type RetryDispatchConfig} from './retry-dispatch.js'; +import {retrySettings, type RetrySettings} from './settings.js'; + +const GET = Request.newBuilder().url('https://example.com').build(); + +/** A POST the re-send gate lets through: RETRY-5 wants a body, and a replayable one. */ +function replayablePost(): Request { + return Request.newBuilder() + .method('POST') + .url('https://example.com') + .body(stringBody('payload')) + .build(); +} + +const zeroClock: Clock = { + now: () => 0, + monotonic: () => 0, + sleep: () => Promise.resolve(), +}; + +/** The third parameter is an object because `max-params` errors at four. */ +interface ConfigExtras { + readonly settings?: Partial<RetrySettings>; + readonly recoverySteps?: readonly RecoveryStep[]; +} + +function configOf( + transport: FakeTransport, + requestSteps = new RequestRecoveryChain([]), + extras: ConfigExtras = {}, +): RetryDispatchConfig { + return { + transport, + requestChain: requestSteps, + responseChain: new ResponseRecoveryChain([], extras.recoverySteps ?? []), + retry: { + settings: retrySettings({ + maxAttempts: 3, + fixedDelayMs: 0, + ...extras.settings, + }), + clock: zeroClock, + random: () => 0.5, + }, + }; +} + +/** + * A one-step request chain whose key strategy is COUNTED and whose keys are DISTINCT. Both matter: + * a strategy returning one constant value would pass every assertion below even if it were called + * once per attempt, which is the bug these cases exist to catch. + */ +function countingKeyChain(): { + chain: RequestRecoveryChain; + generated: () => number; +} { + let generated = 0; + const chain = new RequestRecoveryChain([ + idempotencyKeyStep({ + generate: () => { + generated += 1; + return `key-${String(generated)}`; + }, + }), + ]); + return {chain, generated: () => generated}; +} + +/** The named header seen by each wire send, in order. */ +function headerSent( + transport: FakeTransport, + name = 'Idempotency-Key', +): (string | undefined)[] { + return transport.calls.map(call => call.request.headers.get(name)); +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this + * runner's type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper + * keeps the assertion honest without a lint suppression. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('dispatchWithRetry', () => { + test('retries a transport failure and returns the eventual success', async () => { + const transport = new FakeTransport([ + new IoError('reset'), + countingResponse(200).response, + ]); + + const response = await dispatchWithRetry(GET, configOf(transport)); + + expect(response.status.code).toBe(200); + expect(transport.sendCount).toBe(2); + }); + + test('applies the request recovery chain ONCE per logical request, not per attempt (RETRY-44)', async () => { + let applications = 0; + const chain = new RequestRecoveryChain([ + request => { + applications += 1; + return Promise.resolve(request); + }, + ]); + const transport = new FakeTransport([ + new IoError('reset'), + countingResponse(200).response, + ]); + + await dispatchWithRetry(GET, configOf(transport, chain)); + + expect(transport.sendCount).toBe(2); + expect(applications).toBe(1); + }); + + test('rethrows the terminal failure unchanged in shape', async () => { + const transport = new FakeTransport([new IoError('reset')]); + + expect( + await rejectionOf(dispatchWithRetry(GET, configOf(transport))), + ).toBeDefined(); + expect(transport.sendCount).toBe(3); + }); + + test('a bare POST is dispatched exactly once (RETRY-7 holds on this entry point too)', async () => { + const post = Request.newBuilder() + .method('POST') + .url('https://example.com') + .build(); + const transport = new FakeTransport([new IoError('reset')]); + + expect( + await rejectionOf(dispatchWithRetry(post, configOf(transport))), + ).toBeDefined(); + expect(transport.sendCount).toBe(1); + }); +}); + +describe('dispatchWithRetry and the idempotency key (RECOV-32)', () => { + test('generates ONE key for three attempts and sends it on all three', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + const response = await dispatchWithRetry( + replayablePost(), + configOf(transport, chain), + ); + + expect(response.status.code).toBe(200); + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the run that exhausts the budget sends the same key on every attempt too', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([new IoError('reset')]); + + await rejectionOf( + dispatchWithRetry(replayablePost(), configOf(transport, chain)), + ); + + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the attempt ordinal varies per send while the key does not (RETRY-38)', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + await dispatchWithRetry( + replayablePost(), + configOf(transport, chain, {settings: {attemptHeaderName: 'X-Attempt'}}), + ); + + // The ordinal is the ENGINE's, written per attempt on `stampAttempt`'s fresh copy; the key is + // the request chain's, written once above the loop. Both survive on every send. + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + expect(headerSent(transport, 'X-Attempt')).toEqual(['1', '2', '3']); + }); +}); + +describe('dispatchWithRetry and a failing request chain (RECOV-2)', () => { + const boom = new IoError('request step failed'); + const throwingChain = (): RequestRecoveryChain => + new RequestRecoveryChain([() => Promise.reject(boom)]); + + test('does not retry it, never reaches the transport, and runs the recovery phase once', async () => { + const seen: Outcome<Response>[] = []; + const recovery: RecoveryStep = outcome => { + seen.push(outcome); + return Promise.resolve(outcome); + }; + const transport = new FakeTransport([countingResponse(200).response]); + + const thrown = await rejectionOf( + dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ), + ); + + expect(thrown).toBe(boom); + expect(transport.sendCount).toBe(0); + expect(seen).toEqual([failure(boom)]); + }); + + test('a recovery step may still convert that failure into a success', async () => { + const substitute = countingResponse(204).response; + const recovery: RecoveryStep = () => Promise.resolve(success(substitute)); + const transport = new FakeTransport([countingResponse(200).response]); + + const response = await dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ); + + expect(response).toBe(substitute); + expect(transport.sendCount).toBe(0); + }); +}); diff --git a/packages/core/src/retry/retry-dispatch.ts b/packages/core/src/retry/retry-dispatch.ts new file mode 100644 index 0000000..f641b44 --- /dev/null +++ b/packages/core/src/retry/retry-dispatch.ts @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/retry-dispatch.ts +import type {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; +import { + dispatchPrepared, + prepareRequest, + type DispatchConfig, +} from '../recovery/orchestrator.js'; +import {failure, fold, success} from '../recovery/outcome.js'; +import {runWithRetry, type RetryConfig, type RetryDispatch} from './engine.js'; + +/** + * 4b's `DispatchConfig` plus the retry policy the wrapper drives it with. + * + * @internal + */ +export interface RetryDispatchConfig extends DispatchConfig { + readonly retry: RetryConfig; +} + +/** + * One attempt: the transport hop and the response chain over the ALREADY-prepared request the + * engine handed back from `stampAttempt`. The request chain is deliberately not in here — see + * {@link dispatchWithRetry}. + */ +function attemptVia(config: RetryDispatchConfig): RetryDispatch { + return async request => { + try { + return success(await dispatchPrepared(success(request), config)); + } catch (error) { + return failure(error); + } + }; +} + +/** + * The recovery-chain entry point for retry (RECOV-17..RECOV-20). + * + * NOT a `RecoveryStep` -- a recovery step receives an outcome and has no way to re-dispatch. This + * composes 4b's orchestrator halves instead, mirroring its `(request, config)` shape. + * + * **The request chain runs ONCE, above the loop; each attempt repeats only what is below it** -- + * the transport hop and the response chain, over `stampAttempt`'s fresh copy of the one prepared + * request. That is the layering `recovery/idempotency-key.ts` documents and RECOV-32 needs: its + * `generate()` is invoked once per logical request, so all N attempts reach the server under one + * key, and RETRY-38's per-attempt ordinal is written on the copy without disturbing it. Until + * 2026-09-05 this function re-ran the whole recovery chain per attempt and three attempts produced + * three different keys, defeating the header's entire purpose (audit #67, issue #73). + * + * RETRY-44 is satisfied, not traded away. Its "downstream chain" is whatever sits below the retry + * point, which here is transport plus response chain, and that is re-executed with fresh + * per-attempt state every time. Its second clause -- upstream steps MUST NOT mutate the shared + * in-flight request between attempts -- holds by construction now, because upstream steps no longer + * run between attempts at all. + * + * A request-chain failure is NOT retried: it never reached the wire, so RETRY-5's re-send gate has + * nothing to judge and re-running the step that just threw would only throw again. It still passes + * through the response and recovery chains exactly once, so RECOV-2's "no throwable bypasses the + * recovery hooks" and RECOV-10's unwrap are unchanged. + * + * One consequence worth naming: the engine's re-send gate (RETRY-5/RECOV-18, `isResendable`) now + * reads the PREPARED request rather than the caller's, so a request step that swaps in a + * non-replayable body makes the call non-retryable -- which is the honest answer, since the + * prepared request is what a retry would have to re-send. + * + * Shares `runWithRetry` with the pillar adapter, which is what makes RETRY-13/RETRY-14/RECOV-30's + * "the two stacks must not drift" structural rather than a discipline. + * + * @param request - the request to prepare, send, and possibly re-send. + * @param config - the recovery chains, transport, and retry policy. + * @returns the response of the terminal successful attempt. + * @throws Whatever the FINAL attempt failed with, unwrapped -- the same class a single-attempt run + * would have thrown. RETRY-34's earlier attempts are recorded beside it and read back through + * `retryAttempts()`. A request-chain throwable surfaces the same way, with no trail. + * + * @internal + */ +export async function dispatchWithRetry( + request: Request, + config: RetryDispatchConfig, +): Promise<Response> { + const prepared = await prepareRequest(request, config.requestChain); + if (prepared.kind === 'failure') return dispatchPrepared(prepared, config); + const outcome = await runWithRetry( + prepared.value, + attemptVia(config), + config.retry, + ); + return fold( + outcome, + response => response, + error => { + throw error; + }, + ); +} diff --git a/packages/core/src/retry/retry-step.test.ts b/packages/core/src/retry/retry-step.test.ts new file mode 100644 index 0000000..5b8cca5 --- /dev/null +++ b/packages/core/src/retry/retry-step.test.ts @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/retry-step.test.ts +// Exercises: PIPE-36 (stage assignment is baked into the descriptor, not subclassable), RETRY-44 (a +// FRESH continuation per attempt via ctx.fork), RETRY-8 (both axes still gate inside the pipeline), +// RETRY-32 (the step honors the call's signal, which only exists thanks to Task 1), RETRY-41/HTTP-35 +// (the per-call RequestOptions.maxRetries override, read via ctx.options from Task 1's amendment). +import {describe, expect, test} from 'bun:test'; +import { + createRequestContext, + type ExecutionContext, +} from '../context/context.js'; +import {Request} from '../http/request.js'; +import {RequestOptions} from '../http/request-options.js'; +import type {Response} from '../http/response.js'; +import {IoError} from '../io/errors.js'; +import {Cursor} from '../pipeline/cursor.js'; +import type {StepDescriptor} from '../pipeline/step.js'; +import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; +import {RETRY_STEP_TYPE, retryStep} from './retry-step.js'; + +const GET = Request.newBuilder().url('https://example.com').build(); + +// Constructed inline rather than imported: 4c keeps `aRequestContext()` file-local to +// `cursor.test.ts`, and importing across `*.test.ts` files is not acceptable. +function aRequestContext(): ExecutionContext { + return createRequestContext(GET); +} + +function runThrough( + descriptor: StepDescriptor, + transport: FakeTransport, + signal?: AbortSignal, +): Promise<Response> { + return new Cursor({ + steps: [descriptor], + transport, + request: GET, + context: aRequestContext(), + signal, + }).advance(); +} + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this + * runner's type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper + * keeps the assertion honest without a lint suppression. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('retryStep', () => { + test('is pinned to the RETRY pillar stage (PIPE-36)', () => { + const descriptor = retryStep(); + expect(descriptor.stage).toBe('RETRY'); + expect(descriptor.type).toBe(RETRY_STEP_TYPE); + }); + + test('re-drives the chain on a retryable status and returns the eventual success (RETRY-44)', async () => { + const succeeded = countingResponse(200).response; + const transport = new FakeTransport([ + countingResponse(503).response, + succeeded, + ]); + const descriptor = retryStep({settings: {maxAttempts: 3, fixedDelayMs: 0}}); + + const response = await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(response).toBe(succeeded); + }); + + test('each attempt gets a fresh continuation, so no cursor is reused (RETRY-44)', async () => { + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + const descriptor = retryStep({settings: {maxAttempts: 3, fixedDelayMs: 0}}); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(3); + }); + + test('rethrows the terminal failure rather than returning a failed outcome', async () => { + const boom = new IoError('reset'); + const transport = new FakeTransport([boom]); + const descriptor = retryStep({settings: {maxAttempts: 2, fixedDelayMs: 0}}); + + expect(await rejectionOf(runThrough(descriptor, transport))).toBeDefined(); + }); + + test('honors the call signal from StepContext (RETRY-32)', async () => { + const controller = new AbortController(); + controller.abort(); + const transport = new FakeTransport([countingResponse(200).response]); + const descriptor = retryStep(); + + expect( + await rejectionOf(runThrough(descriptor, transport, controller.signal)), + ).toBeDefined(); + expect(transport.sendCount).toBe(0); + }); +}); + +describe('retryStep per-call budget override (RETRY-41, HTTP-35)', () => { + test('per-call maxRetries: 0 disables retries for this call only (RETRY-41, HTTP-35)', async () => { + const the503 = countingResponse(503); + const transport = new FakeTransport([ + the503.response, + countingResponse(200).response, + ]); + const descriptor = retryStep({settings: {maxAttempts: 3, fixedDelayMs: 0}}); + const options = RequestOptions.newBuilder().maxRetries(0).build(); + + const response = await new Cursor({ + steps: [descriptor], + transport, + request: GET, + context: aRequestContext(), + options, + }).advance(); + + // The configured budget of 3 was overridden per call. + expect(transport.sendCount).toBe(1); + expect(response.status.code).toBe(503); + // A surviving response is returned LIVE and unread (RETRY-36's discarding-only remap). + expect(the503.cancelCount()).toBe(0); + }); + + test('a non-finite per-call maxRetries cannot reach the step at all', () => { + // First line of defence, and the one a caller actually meets: HTTP-35 rejects at the setter. + for (const value of [Number.POSITIVE_INFINITY, Number.NaN, 1.5]) { + expect(() => RequestOptions.newBuilder().maxRetries(value)).toThrow(); + } + }); + + test('the step re-checks it anyway, so a builder regression cannot make the loop unbounded', async () => { + // Backstop, exercised through a hand-shaped options object the public builder would refuse to + // produce. Worth asserting rather than trusting: the value lands directly in `maxAttempts`, and + // a non-finite budget does not fail loudly -- it makes `attempt >= maxAttempts` permanently + // false and the retry loop endless. + const transport = new FakeTransport([new IoError('reset')]); + const descriptor = retryStep({settings: {fixedDelayMs: 0}}); + const forged = { + maxRetries: Number.POSITIVE_INFINITY, + } as unknown as RequestOptions; + + const reason = await rejectionOf( + new Cursor({ + steps: [descriptor], + transport, + request: GET, + context: aRequestContext(), + options: forged, + }).advance(), + ); + + expect((reason as Error).message).toContain( + 'maxRetries must be a non-negative integer', + ); + expect(transport.sendCount).toBe(0); + }); +}); + +describe('retryStep per-call budget widening (RETRY-41)', () => { + test('per-call maxRetries widens the configured budget too (RETRY-41 is present-override-wins)', async () => { + const transport = new FakeTransport([ + countingResponse(503).response, + countingResponse(503).response, + countingResponse(200).response, + ]); + const descriptor = retryStep({settings: {maxAttempts: 1, fixedDelayMs: 0}}); + const options = RequestOptions.newBuilder().maxRetries(2).build(); + + await new Cursor({ + steps: [descriptor], + transport, + request: GET, + context: aRequestContext(), + options, + }).advance(); + + // 2 retries + the initial send. + expect(transport.sendCount).toBe(3); + }); +}); diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts new file mode 100644 index 0000000..2680959 --- /dev/null +++ b/packages/core/src/retry/retry-step.ts @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/retry-step.ts +import {defaultClock, type Clock} from '../config/clock.js'; +import {invariant} from '../invariant.js'; +import type {Next, StepContext, StepDescriptor} from '../pipeline/step.js'; +import {failure, fold, success} from '../recovery/outcome.js'; +import {runWithRetry, type RetryConfig, type RetryDispatch} from './engine.js'; +import {retrySettings, type RetrySettings} from './settings.js'; + +/** Stable identity for pillar-slot occupancy and anchor matching (PIPE-6/PIPE-18). */ +export const RETRY_STEP_TYPE: unique symbol = Symbol('dexpace.retry'); + +/** + * Everything {@link retryStep} accepts. An options object rather than a bare `RetrySettings`: the + * engine's two injected seams (`clock`, `random`) and RETRY-39's caller delay-override all have to + * reach `RetryConfig`, and a no-argument `retryStep()` must stay the default-tuned pillar step + * (RETRY-12). + * + * @public + */ +export interface RetryStepOptions { + /** + * Policy overrides. Any omitted field takes its spec default (RETRY-12). + * + * @defaultValue the spec defaults `retrySettings()` supplies + */ + readonly settings?: Partial<RetrySettings> | undefined; + /** + * The wall-clock and sleep seam, injected so backoff is testable without real time. The same + * instance also satisfies `AuthStepSettings.clock`, so one `Clock` drives every pillar in a + * pipeline (CFG-15..CFG-18). + * + * @defaultValue a `Clock` over `Date.now`, `performance.now`, and a cancellable `setTimeout` + */ + readonly clock?: Clock | undefined; + /** + * The randomness seam jitter draws from, in `[0, 1)`. Injected so a jittered schedule is + * assertable (RETRY-10). + * + * @defaultValue `Math.random` + */ + readonly random?: (() => number) | undefined; + /** + * RETRY-39's caller override: returns the delay in milliseconds to use for `attempt`, or + * `undefined` to fall through to the configured schedule for that attempt. + * + * A throw, or a non-finite result, is ignored: the configured schedule is used for that attempt + * and `http.retry.delayOverrideFailed` is logged at warning level (RETRY-40). Neither aborts the + * retry loop. A finite negative is honored as a delay and continues inline without a timer + * (RETRY-31). + * + * @defaultValue absent, so every attempt uses the configured schedule + */ + readonly delayOverride?: + ((attempt: number) => number | undefined) | undefined; +} + +/** Each attempt drives a FRESH one-shot continuation -- RETRY-44's per-attempt state, PIPE-15's fork. */ +function attemptVia(fork: () => Next): RetryDispatch { + return async request => { + try { + return success(await fork()(request)); + } catch (error) { + return failure(error); + } + }; +} + +/** + * RETRY-41/HTTP-35: the per-call `RequestOptions.maxRetries` override wins over the configured budget + * when present. The option counts retries; `maxAttempts` counts total sends, hence the `+ 1`. + * + * The value IS revalidated here, and the two guards now agree. `RequestOptionsBuilder.maxRetries` + * rejects a negative, fractional or non-finite value at the call site that supplied it + * (`../http/request-options.ts:212-219`, pinned by `request-options.test.ts`'s + * `maxRetries validation (HTTP-35)` block), so this `invariant` is the engine asserting its own + * precondition rather than the only thing enforcing it -- it should be unreachable, and tripping it + * means the builder's guard was weakened. That was not true when this comment was first written: the + * builder then rejected only a negative value, which was strictly weaker than the + * `Number.isFinite(...) && >= 1` guard `retrySettings()` applies to the configured budget. The + * assertion stays either way, because `Infinity` or `NaN` reaching `maxAttempts` makes the engine's + * `attempt >= maxAttempts` gate permanently false and the retry loop unbounded, and the per-call + * route must not be the one path into the engine that skips the check the configured route enforces. + * + * The derived object is frozen: a spread of a frozen source is NOT itself frozen, and RETRY-42 + * requires every policy component to be immutable after construction, not merely typed `readonly`. + */ +function effectiveSettings( + base: RetrySettings, + perCallMaxRetries: number | undefined, +): RetrySettings { + if (perCallMaxRetries === undefined) return base; + invariant( + Number.isInteger(perCallMaxRetries) && perCallMaxRetries >= 0, + `RequestOptions.maxRetries must be a non-negative integer, got ${String(perCallMaxRetries)}`, + ); + return Object.freeze({...base, maxAttempts: perCallMaxRetries + 1}); +} + +function configFrom( + base: RetryConfig, + ctx: Pick<StepContext, 'signal' | 'options'>, +): RetryConfig { + return { + ...base, + settings: effectiveSettings(base.settings, ctx.options?.maxRetries), + signal: ctx.signal, + }; +} + +/** + * The RETRY pillar step. + * + * `stage: 'RETRY'` is baked into the descriptor this factory returns, which is how PIPE-36 ("a shipped + * pillar family must not be relocatable out of its pillar") is satisfied structurally: steps are + * functions carrying a descriptor, not classes with a subclassable stage assignment. + * + * `ctx.fork` is asserted rather than checked -- RETRY is in `PILLAR_STAGES`, so its absence means the + * descriptor was installed somewhere it cannot be, which is a programmer error. + * + * **What it throws when it gives up is the FINAL attempt's own error, unwrapped.** The class you + * catch does not depend on how many attempts ran: a transport failure surfaces as + * `TransportFailureError` whether `maxAttempts` was 1 or 3, and an abort that ended a backoff wait + * surfaces as `CancellationError` (`XCUT-1`). The earlier attempts' errors are not lost -- read them + * with `retryAttempts(caught)`, oldest first: one entry per attempt that failed BEFORE the error you + * caught, which is not the same as an attempt count (`RETRY-34`, and see `retryAttempts` for why the + * difference bites). A response the loop discards is always closed first; the response that ENDS the + * loop is returned live and unread, and closing it is yours. + * + * @param options - settings overrides and the injected clock, randomness, and delay override. + * @returns the descriptor to install in a pipeline's RETRY slot. + * + * @public + */ +export function retryStep(options: RetryStepOptions = {}): StepDescriptor { + // Built ONCE per installed step, not per request: `retrySettings()` validates every field and + // takes a defensive copy of the retryable-status set, which is ~110 entries at the default. Only + // the per-call `maxRetries` override and the call's signal are genuinely per-request, and + // `configFrom` derives just those (RETRY-42: the policy is immutable and stateless after + // construction, so one instance is safe to share across concurrent calls). + const base: RetryConfig = { + settings: retrySettings(options.settings), + clock: options.clock ?? defaultClock, + random: options.random ?? ((): number => Math.random()), + delayOverride: options.delayOverride, + }; + return { + type: RETRY_STEP_TYPE, + stage: 'RETRY', + fn: async (request, ctx) => { + const {fork} = ctx; + invariant( + fork !== undefined, + 'retryStep must occupy the RETRY pillar stage', + ); + const outcome = await runWithRetry( + request, + attemptVia(fork), + configFrom(base, ctx), + ); + return fold( + outcome, + response => response, + error => { + throw error; + }, + ); + }, + }; +} diff --git a/packages/core/src/retry/settings.test.ts b/packages/core/src/retry/settings.test.ts new file mode 100644 index 0000000..63365f6 --- /dev/null +++ b/packages/core/src/retry/settings.test.ts @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/settings.test.ts +// Exercises: RETRY-12 (defaults), RETRY-14 (one budget, so nothing to reconcile), RETRY-27/28 (opt-in +// total timeout, 0 disables), RETRY-41 (a negative retry count is REJECTED, not clamped -- HTTP-35 +// wins the MUST-vs-MUST collision), RETRY-42 (immutable after construction), RECOV-34 (construction +// validation, defensive collection copies). +import {describe, expect, test} from 'bun:test'; +import {RETRYABLE_STATUSES} from './classify.js'; +import {DEFAULT_RETRY_SETTINGS, retrySettings} from './settings.js'; + +describe('defaults (RETRY-12)', () => { + test('ship the spec defaults', () => { + expect(DEFAULT_RETRY_SETTINGS.initialDelayMs).toBe(200); + expect(DEFAULT_RETRY_SETTINGS.multiplier).toBe(2); + expect(DEFAULT_RETRY_SETTINGS.maxDelayMs).toBe(8000); + expect(DEFAULT_RETRY_SETTINGS.jitter).toBe(0.2); + expect(DEFAULT_RETRY_SETTINGS.maxAttempts).toBe(3); + }); + + test('the total timeout is opt-in, undefined by default (RETRY-28)', () => { + expect(DEFAULT_RETRY_SETTINGS.totalTimeoutMs).toBeUndefined(); + }); + + test('the default retryable statuses are the single-sourced set', () => { + expect([...DEFAULT_RETRY_SETTINGS.retryableStatuses].sort()).toEqual( + [...RETRYABLE_STATUSES].sort(), + ); + }); +}); + +describe('validation (RECOV-34)', () => { + test('rejects a multiplier below 1.0', () => { + expect(() => retrySettings({multiplier: 0.5})).toThrow(); + }); + + test('rejects a non-finite multiplier (P2 sweep)', () => { + expect(() => + retrySettings({multiplier: Number.POSITIVE_INFINITY}), + ).toThrow(); + expect(() => retrySettings({multiplier: Number.NaN})).toThrow(); + }); + + test('rejects maxAttempts below 1, never clamping to the default (RETRY-41/HTTP-35)', () => { + expect(() => retrySettings({maxAttempts: 0})).toThrow(); + expect(() => retrySettings({maxAttempts: -3})).toThrow(); + }); + + test('rejects a fractional maxAttempts -- an attempt count is integral (P2 sweep)', () => { + expect(() => retrySettings({maxAttempts: 2.5})).toThrow(); + }); + + test('accepts maxAttempts of 1, which disables retries', () => { + expect(retrySettings({maxAttempts: 1}).maxAttempts).toBe(1); + }); +}); + +describe('validation: duration bounds (RECOV-34, V4/V13)', () => { + test('accepts a delay past what ONE timer can carry (V13)', () => { + // V4 briefly bounded these at `Clock`'s MAX_SLEEP_MS, because `Clock.sleep` rejected anything + // larger. V13 made the clock chain timers instead, so the bound became a restriction on a + // duration the platform CAN wait -- and it would have made RETRY-18's 365-day pacing clamp + // (~14x one timer's reach) unconfigurable. + const pastOneTimer = 2 ** 31; + expect(retrySettings({initialDelayMs: pastOneTimer}).initialDelayMs).toBe( + pastOneTimer, + ); + expect(retrySettings({maxDelayMs: pastOneTimer}).maxDelayMs).toBe( + pastOneTimer, + ); + expect(retrySettings({fixedDelayMs: pastOneTimer}).fixedDelayMs).toBe( + pastOneTimer, + ); + }); + + test("accepts RETRY-18's 365-day pacing ceiling as a configured delay (V13)", () => { + const oneYearMs = 365 * 24 * 60 * 60 * 1000; + expect(retrySettings({maxDelayMs: oneYearMs}).maxDelayMs).toBe(oneYearMs); + }); + + test('still rejects a non-finite or negative duration', () => { + expect(() => + retrySettings({initialDelayMs: Number.POSITIVE_INFINITY}), + ).toThrow(); + expect(() => retrySettings({maxDelayMs: Number.NaN})).toThrow(); + expect(() => retrySettings({fixedDelayMs: -1})).toThrow(); + }); + + test('leaves totalTimeoutMs on the same rule -- it is a budget, never a sleep', () => { + expect(retrySettings({totalTimeoutMs: 2 ** 32}).totalTimeoutMs).toBe( + 2 ** 32, + ); + }); +}); + +describe('validation: the remaining fields (RECOV-34)', () => { + test('rejects a jitter outside [0,1]', () => { + expect(() => retrySettings({jitter: -0.1})).toThrow(); + expect(() => retrySettings({jitter: 1.1})).toThrow(); + }); + + test('rejects negative durations', () => { + expect(() => retrySettings({initialDelayMs: -1})).toThrow(); + expect(() => retrySettings({maxDelayMs: -1})).toThrow(); + expect(() => retrySettings({totalTimeoutMs: -1})).toThrow(); + expect(() => retrySettings({fixedDelayMs: -1})).toThrow(); + }); + + test('rejects non-finite durations', () => { + expect(() => retrySettings({initialDelayMs: Number.NaN})).toThrow(); + expect(() => + retrySettings({maxDelayMs: Number.POSITIVE_INFINITY}), + ).toThrow(); + }); + + test('rejects a malformed attempt header name at construction, not at the first retry', () => { + expect(() => + retrySettings({attemptHeaderName: 'X-Bad\r\nInjected'}), + ).toThrow(); + expect(() => retrySettings({attemptHeaderName: ''})).toThrow(); + }); + + test('accepts a valid attempt header name', () => { + expect( + retrySettings({attemptHeaderName: 'X-Attempt'}).attemptHeaderName, + ).toBe('X-Attempt'); + }); + + test('a total timeout of zero is legal and means unbounded (RETRY-27)', () => { + expect(retrySettings({totalTimeoutMs: 0}).totalTimeoutMs).toBe(0); + }); +}); + +describe('immutability (RETRY-42, RECOV-34)', () => { + test('the status set is defensively copied, so later caller mutation cannot change policy', () => { + const caller = new Set([500]); + const settings = retrySettings({retryableStatuses: caller}); + caller.add(404); + expect(settings.retryableStatuses.has(404)).toBe(false); + }); + + test('the returned settings object is frozen', () => { + const settings = retrySettings(); + expect(Object.isFrozen(settings)).toBe(true); + }); + + test('DEFAULT_RETRY_SETTINGS is frozen', () => { + expect(Object.isFrozen(DEFAULT_RETRY_SETTINGS)).toBe(true); + }); +}); diff --git a/packages/core/src/retry/settings.ts b/packages/core/src/retry/settings.ts new file mode 100644 index 0000000..87dab68 --- /dev/null +++ b/packages/core/src/retry/settings.ts @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/settings.ts +import {hasForbiddenNameByte} from '../http/ascii-validation.js'; +import {invariant} from '../invariant.js'; +import type {BackoffSettings} from './backoff.js'; +import {RETRYABLE_STATUSES} from './classify.js'; + +/** + * The complete retry policy: the backoff schedule plus the budget, the authoritative status set, and + * the two opt-in knobs (RETRY-12, RETRY-27/28, RETRY-38, RECOV-34). + * + * Immutable and stateless after construction, so one instance is safe for concurrent invocation + * (RETRY-42/RECOV-28). + * + * @public + */ +export interface RetrySettings extends BackoffSettings { + /** Total wire sends including the initial one; 1 disables retries (RETRY-14, RECOV-34). */ + readonly maxAttempts: number; + /** Authoritative on its own -- it both widens and narrows the built-in classifier (RETRY-37). */ + readonly retryableStatuses: ReadonlySet<number>; + /** + * OPT-IN total-timeout budget spanning attempts and inter-attempt delays (RETRY-27). Undefined by + * default and `0` also disabling it -- RETRY-28 instructs a port that unifies the two reference + * retry stacks to make this explicitly opt-in rather than always-on. + */ + readonly totalTimeoutMs?: number | undefined; + /** When set, each attempt is stamped with its 1-based ordinal under this header (RETRY-38). */ + readonly attemptHeaderName?: string | undefined; +} + +/** + * RETRY-12's defaults: 200 ms initial delay, doubling, an 8 s cap, 20% symmetric jitter, and three + * total wire sends. + * + * @internal + */ +export const DEFAULT_RETRY_SETTINGS: RetrySettings = Object.freeze({ + initialDelayMs: 200, + multiplier: 2, + maxDelayMs: 8000, + jitter: 0.2, + maxAttempts: 3, + retryableStatuses: RETRYABLE_STATUSES, +}); + +/** + * A duration this module will hand to `Clock.sleep`, or compare against elapsed time. + * + * Bounded below only. A ceiling of `Clock`'s `MAX_SLEEP_MS` sat here between 2026-09-02 and later + * the same day: it was the right guard while `Clock.sleep` REFUSED a longer wait, and it became + * unnecessary the moment the clock started chaining timers to honor any finite duration. Re-adding + * it would now reject a duration the platform can wait -- and would make `RETRY-18`'s 365-day + * pacing clamp unconfigurable, which is the very collision V13 closed. + */ +function validateDuration(label: string, value: number | undefined): void { + if (value === undefined) return; + invariant( + Number.isFinite(value) && value >= 0, + `${label} must be a finite, non-negative duration, got ${String(value)}`, + ); +} + +/** + * Builds validated, frozen retry settings (RECOV-34). Invalid values are PROGRAMMER errors -- a + * caller passing `multiplier: 0.5` has a bug, not an operational failure -- so they trip + * `invariant()` rather than a typed error class. + * + * A negative `maxAttempts` is REJECTED, never clamped to the default: RETRY-41 says clamp, HTTP-35 + * (also MUST) says reject precisely so a negative value cannot be silently reinterpreted as "use + * default". The port takes HTTP-35's line on both surfaces. + * + * The status set is defensively copied at build time so later mutation of the caller's collection + * cannot alter policy (RECOV-34). + * + * @param overrides - the fields to change; everything else takes RETRY-12's default. + * @returns frozen, validated settings. + * @throws InvariantViolation for a negative or non-finite duration; a multiplier below 1.0 or + * non-finite; a `maxAttempts` that is not an integer >= 1; or a jitter outside [0,1]. + * + * @internal + */ +export function retrySettings( + overrides?: Partial<RetrySettings>, +): RetrySettings { + const merged = {...DEFAULT_RETRY_SETTINGS, ...overrides}; + validateDuration('initialDelayMs', merged.initialDelayMs); + validateDuration('maxDelayMs', merged.maxDelayMs); + validateDuration('totalTimeoutMs', merged.totalTimeoutMs); + validateDuration('fixedDelayMs', merged.fixedDelayMs); + // `Number.isFinite` on both, and integrality on the one that counts wire sends. The lower bound + // alone let `Infinity` through on the multiplier -- which makes the second delay `Infinity` and + // fails inside the retry loop at `Clock.sleep`'s ceiling instead of at the call that configured it + // -- and let a fractional `maxAttempts` through, which is not a count. Same reasoning as HTTP-35's + // on `RequestOptionsBuilder.maxRetries`: these two are among the four holes the 2026-09-02 sweep + // of every public numeric setter closed to the full range. + invariant( + Number.isFinite(merged.multiplier) && merged.multiplier >= 1, + `retry multiplier must be a finite number >= 1.0, got ${String(merged.multiplier)}`, + ); + invariant( + Number.isInteger(merged.maxAttempts) && merged.maxAttempts >= 1, + `retry maxAttempts must be an integer >= 1 (1 disables retries), got ${String(merged.maxAttempts)}`, + ); + invariant( + merged.jitter >= 0 && merged.jitter <= 1, + `retry jitter must lie in [0,1], got ${String(merged.jitter)}`, + ); + // Validated HERE rather than left to the first stamped attempt (RETRY-38). `HeadersBuilder` + // rejects a malformed name (HTTP-26), so an unchecked value would surface as a throw from inside + // the retry loop on some later request -- a configuration mistake reported as a request failure, + // far from the call that made it, and only on the code path that actually retries. + invariant( + merged.attemptHeaderName === undefined || + (merged.attemptHeaderName.length > 0 && + !hasForbiddenNameByte(merged.attemptHeaderName)), + `retry attemptHeaderName must be a valid header name, got ${String(merged.attemptHeaderName)}`, + ); + return Object.freeze({ + ...merged, + retryableStatuses: new Set(merged.retryableStatuses), + }); +} diff --git a/packages/core/src/seams/index.ts b/packages/core/src/seams/index.ts deleted file mode 100644 index 0b07a1c..0000000 --- a/packages/core/src/seams/index.ts +++ /dev/null @@ -1,13 +0,0 @@ -// SPDX-License-Identifier: MIT -// packages/core/src/seams/index.ts -// Internal-facing seams barrel — includes Serde<T>, unlike the package's public entry point -// (packages/core/src/index.ts), which deliberately omits it (SEAM-21 is deferred to Phase 6). -export type {Transport} from './transport.js'; -export { - composeSignal, - isTimeoutSignal, - CancellationError, -} from './transport.js'; -export type {Serde} from './serde.js'; -export type {OperationDescriptor} from './operation.js'; -export {buildRequest, OperationAssemblyError} from './operation.js'; diff --git a/packages/core/src/seams/operation.test.ts b/packages/core/src/seams/operation.test.ts index d166bc1..5e588a9 100644 --- a/packages/core/src/seams/operation.test.ts +++ b/packages/core/src/seams/operation.test.ts @@ -1,7 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/operation.test.ts // Exercises: SEAM-26 (the four projections default to empty), SEAM-27 (buildRequest's encoding and base-URL -// composition rules), reusing HTTP-29's encodeRfc3986Component for path-segment encoding. +// composition rules, and that a placeholder is satisfied only by an OWN property of pathParams), +// HTTP-7 (a body projected onto a body-forbidding method fails assembly), reusing +// HTTP-29's encodeRfc3986Component for path-segment encoding — including the unpaired-surrogate input it +// cannot encode, which is rejected here rather than allowed to escape as a bare URIError. import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -9,7 +12,10 @@ import { OperationAssemblyError, type OperationDescriptor, } from './operation.js'; -import {UrlConstructionError} from '../http/errors.js'; +import { + RequestBodyNotAllowedError, + UrlConstructionError, +} from '../http/errors.js'; import {QueryParams} from '../http/query-params.js'; import {Headers} from '../http/headers.js'; @@ -98,17 +104,34 @@ describe('SEAM-27: base-URL composition rules', () => { }); }); +import {stringBody} from '../body/simple-bodies.js'; + describe('operation headers and body projections are threaded through', () => { test('supplied headers and body appear on the built request', () => { const headers = Headers.newBuilder().add('X-Trace', 'abc').build(); + const body = stringBody('Fido'); const request = buildRequest('https://host', { method: 'POST', pathTemplate: '/pets', headers, - body: {name: 'Fido'}, + body, }); expect(request.headers.get('x-trace')).toBe('abc'); - expect(request.body).toEqual({name: 'Fido'}); + expect(request.body).toBe(body); + }); + + // HTTP-7: assembly ends at `Request.Builder.build()`, so the builder's method/body legality check is + // buildRequest's. Pinned because the throw reaches a caller through `buildRequest` and was absent from + // its `@throws` list until audit #67 / #68 added it -- an undocumented, unpinned throw path is exactly + // the kind that a later refactor swallows. + test('a body on a body-forbidding method throws RequestBodyNotAllowedError', () => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/pets', + body: stringBody('Fido'), + }), + ).toThrow(RequestBodyNotAllowedError); }); }); @@ -137,6 +160,136 @@ describe('SEAM-27: dot-segment path-param values are rejected, not silently norm }); }); +describe('SEAM-27: a placeholder is satisfied only by an OWN property of pathParams', () => { + // `pathParams?.[name]` reached the whole prototype chain, so `{constructor}` against `{}` resolved to + // `Object`'s own constructor, stringified, and shipped + // `/users/function%20Object%28%29%20%7B%20%5Bnative%20code%5D%20%7D` instead of failing assembly. Every + // placeholder MUST have a *supplied* value (SEAM-27); a name the caller never supplied is a missing value + // whatever `Object.prototype` happens to carry. Measured on the pre-fix tree, audit #67 / #76. + test.each([ + 'constructor', + 'toString', + 'hasOwnProperty', + 'valueOf', + '__proto__', + ])( + 'a {%s} placeholder against empty pathParams throws OperationAssemblyError', + name => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: `/users/{${name}}`, + pathParams: {}, + }), + ).toThrow(OperationAssemblyError); + }, + ); + + test('the error names the placeholder, not the inherited member it resolved to', () => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {}, + }), + ).toThrow(/missing value for path parameter "constructor"/); + }); + + test('an own property named like a prototype member is still honored', () => { + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {constructor: 'me'}, + }); + expect(request.url.pathname).toBe('/users/me'); + }); + + test('a null-prototype pathParams object still resolves its own keys', () => { + const pathParams = Object.assign(Object.create(null) as object, { + id: 'x', + }) as Record<string, string>; + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{id}', + pathParams, + }); + expect(request.url.pathname).toBe('/users/x'); + }); +}); + +describe('SEAM-27: an unpaired surrogate in a path-param value is rejected', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string with no UTF-8 form. Before audit #67 / #76 that escaped `buildRequest` outside the + // `DexpaceError` tree and outside its `@throws` list. It is now `OperationAssemblyError`, the + // class this call site already throws for a path-param value it cannot use. + test.each([ + ['a lone high surrogate', '\uD800'], + ['a lone low surrogate', '\uDFFF'], + ['a lone surrogate inside a longer value', 'ok\uD800ok'], + ])('%s throws OperationAssemblyError', (_label, value) => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }), + ).toThrow(OperationAssemblyError); + }); + + test('a well-formed surrogate pair is ordinary text and is encoded', () => { + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: '\u{1F600}'}, + }); + expect(request.url.pathname).toBe('/things/%F0%9F%98%80'); + }); + + test('the query projection cannot carry one either, from a builder or from parse()', () => { + // `composeQuery` calls `operationQuery.encode()`, the second `encodeRfc3986Component` path into + // `buildRequest`. Both ways of obtaining a `QueryParams` are now closed: the builder rejects an + // unpaired surrogate, `parse` replaces it. + expect(() => QueryParams.newBuilder().add('c', '\uD800')).toThrow( + /unpaired surrogate/, + ); + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things', + query: QueryParams.parse('c=\uD800'), + }); + expect(request.url.search).toBe('?c=%EF%BF%BD'); + }); + + test('no URIError escapes buildRequest, whatever the descriptor carries (property)', () => { + fc.assert( + fc.property( + fc.string({ + unit: fc.oneof( + fc.constantFrom('a', '/', '.', '%', ' ', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + minLength: 1, + maxLength: 6, + }), + value => { + try { + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }); + } catch (e: unknown) { + expect(e).toBeInstanceOf(OperationAssemblyError); + } + }, + ), + {numRuns: 500}, + ); + }); +}); + describe('a path-param value containing / is encoded, not split (property)', () => { test('holds for arbitrary generated path-param values', () => { fc.assert( diff --git a/packages/core/src/seams/operation.ts b/packages/core/src/seams/operation.ts index cb05c5a..0683691 100644 --- a/packages/core/src/seams/operation.ts +++ b/packages/core/src/seams/operation.ts @@ -1,17 +1,19 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/operation.ts +import type {Body} from '../body/body.js'; import {Request} from '../http/request.js'; import type {Headers} from '../http/headers.js'; import type {QueryParams} from '../http/query-params.js'; import type {Method} from '../http/method.js'; import {UrlConstructionError, DexpaceError} from '../http/errors.js'; -import {encodeRfc3986Component} from '../http/rfc3986.js'; +import {encodeRfc3986Component, hasLoneSurrogate} from '../http/rfc3986.js'; /** * Thrown when `buildRequest()` cannot assemble a request from its descriptor: a `{name}` - * placeholder in `pathTemplate` has no value in `pathParams`, or a supplied value is a dot segment - * (`.`/`..`) that the WHATWG URL parser would normalize into a path rewrite instead of keeping as - * one literal segment. + * placeholder in `pathTemplate` has no OWN value in `pathParams` — an inherited member such as + * `constructor` does not satisfy one — or a supplied value is a dot segment (`.`/`..`) that the + * WHATWG URL parser would normalize into a path rewrite instead of keeping as one literal segment, + * or a supplied value carries an unpaired surrogate and so has no percent-encoded form. * * @public */ @@ -49,9 +51,10 @@ export interface OperationDescriptor { readonly pathTemplate: string; /** - * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have a value here; - * each value is percent-encoded as a single path segment, so a value containing `/` cannot inject - * an extra segment (SEAM-27). Defaults to empty. + * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have an OWN property + * here — a name reachable only through the prototype chain, such as `constructor`, is treated as + * absent — and each value is percent-encoded as a single path segment, so a value containing `/` + * cannot inject an extra segment (SEAM-27). Defaults to empty. */ readonly pathParams?: Readonly<Record<string, string>> | undefined; @@ -68,7 +71,7 @@ export interface OperationDescriptor { * The operation's body. Carried, not encoded — serialization is a separate seam's concern * (SEAM-26). Defaults to absent. */ - readonly body?: unknown; + readonly body?: Body | undefined; } const PATH_PARAM_RE = /\{([^{}]+)\}/g; @@ -102,7 +105,16 @@ function substitutePathParams( pathParams: Readonly<Record<string, string>> | undefined, ): string { return template.replace(PATH_PARAM_RE, (_match, name: string) => { - const value = pathParams?.[name]; + // `Object.hasOwn`, not `pathParams?.[name]`: the indexed read walks the prototype chain, so + // `{constructor}` against `{}` resolved to `Object.prototype.constructor`, stringified through + // `encodeRfc3986Component`, and shipped a native-code source text as a path segment instead of + // failing assembly. SEAM-27 requires every placeholder to have a *supplied* value, and a name + // the caller never supplied is missing whatever `Object.prototype` happens to carry + // (audit #67 / #76). + const value = + pathParams !== undefined && Object.hasOwn(pathParams, name) + ? pathParams[name] + : undefined; if (value === undefined) { throw new OperationAssemblyError( `missing value for path parameter "${name}"`, @@ -119,6 +131,17 @@ function substitutePathParams( name, ); } + // A string carrying an unpaired surrogate has no UTF-8 form, so `encodeRfc3986Component` — i.e. + // `encodeURIComponent` — throws a bare `URIError: URI malformed`. That escaped `buildRequest` + // outside the `DexpaceError` tree and outside its documented `@throws`. Rejected here with the + // class this call site already throws, rather than guarded inside the encoder, so the failure + // names the parameter (audit #67 / #76). + if (hasLoneSurrogate(value)) { + throw new OperationAssemblyError( + `path parameter "${name}" contains an unpaired surrogate and cannot be percent-encoded`, + name, + ); + } return encodeRfc3986Component(value); }); } @@ -157,10 +180,15 @@ function composeQuery( * @param baseUrl - the absolute base URL to project the operation onto. * @param operation - the operation to assemble into a request. * @returns the assembled request. - * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no value in `pathParams`, - * or a supplied value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. + * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no own value in + * `pathParams` (a name inherited from `Object.prototype` does not count as supplied), or a supplied + * value is a dot segment (`.`/`..`), or a supplied value carries an unpaired surrogate — fix the + * descriptor; no request was assembled. * @throws {@link UrlConstructionError} when `baseUrl` is malformed, non-absolute, or carries a * fragment — supply a clean absolute base URL. + * @throws {@link RequestBodyNotAllowedError} when the descriptor pairs a body with GET, HEAD, TRACE + * or CONNECT (HTTP-7). Assembly ends at `Request.Builder.build()`, so that builder's validation is + * this function's validation. * * @public */ diff --git a/packages/core/src/seams/serde.test.ts b/packages/core/src/seams/serde.test.ts index ccd911b..ca378fb 100644 --- a/packages/core/src/seams/serde.test.ts +++ b/packages/core/src/seams/serde.test.ts @@ -1,33 +1,71 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/serde.test.ts -// Exercises: SEAM-19 (mediaType required, never defaulted) — a compile-time check only (styleguide 11.6); -// `bun test` executes this file but does not typecheck it (its transpiler strips types without checking them). -// The assertions only actually fire under `bun run typecheck` — see the plan's Task 5 Step 3. +// Exercises: SERDE-1 (one bundle, one encoder, one decoder), SERDE-2 (mediaType required, never optional), +// SERDE-5 (decode takes an explicit schema witness — SEAM-21), SERDE-6 (parametric targets via combinators), +// SEAM-19/SEAM-20 (the bundle's shape and its four allocation profiles). +// `bun test` executes this file but does not typecheck it; the assertions only fire under `bun run typecheck`. import {test} from 'bun:test'; import {expectTypeOf} from 'expect-type'; -import type {Serde} from './serde.js'; +import type {Deserializer, Schema, Serde, Serializer} from './serde.js'; -test('mediaType is a required, non-optional string', () => { - expectTypeOf<Serde<string>>() - .toHaveProperty('mediaType') - .toEqualTypeOf<string>(); -}); - -test("deserialize's return type is bound to the instance's T", () => { - expectTypeOf<Serde<number>['deserialize']>().returns.toEqualTypeOf<number>(); -}); - -test("serialize's parameter type is bound to the instance's T", () => { - expectTypeOf<Serde<boolean>['serialize']>() - .parameter(0) - .toEqualTypeOf<boolean>(); +test('Serde bundles exactly one serializer and one deserializer for one media type', () => { + expectTypeOf<Serde>().toHaveProperty('mediaType').toEqualTypeOf<string>(); + expectTypeOf<Serde>() + .toHaveProperty('serializer') + .toEqualTypeOf<Serializer>(); + expectTypeOf<Serde>() + .toHaveProperty('deserializer') + .toEqualTypeOf<Deserializer>(); }); test('an implementation without mediaType is rejected (negative case, styleguide 11.6)', () => { // @ts-expect-error -- SEAM-19: mediaType is required and never defaulted; omitting it must not compile - const missingMediaType: Serde<string> = { - serialize: (value: string): unknown => value, - deserialize: (data: unknown): string => String(data), + const missingMediaType: Serde = { + serializer: {} as Serializer, + deserializer: {} as Deserializer, }; void missingMediaType; }); + +test("decode's return type is driven by the schema argument, not by the bundle", () => { + type Decoded = ReturnType<Deserializer['deserialize']>; + // Unconstrained call site infers `unknown`; the constrained one below is the real assertion. + expectTypeOf<Decoded>().toBeUnknown(); + + const decode = (d: Deserializer, s: Schema<{id: number}>): {id: number} => + d.deserialize(new Uint8Array(), {schema: s}); + expectTypeOf(decode).returns.toEqualTypeOf<{id: number}>(); +}); + +test('a parametric target needs no special carrier — the schema is a combinator over element schemas', () => { + const decodeMany = ( + d: Deserializer, + s: Schema<readonly {id: number}[]>, + ): readonly {id: number}[] => d.deserialize(new Uint8Array(), {schema: s}); + expectTypeOf(decodeMany).returns.toEqualTypeOf<readonly {id: number}[]>(); +}); + +test('serializeInto returns a byte count and accepts an optional offset', () => { + expectTypeOf<Serializer['serializeInto']>().returns.toEqualTypeOf<number>(); + expectTypeOf<Serializer['serializeInto']>() + .parameter(2) + .toEqualTypeOf<number | undefined>(); +}); + +test('all four SEAM-20 allocation profiles are present, including the fresh-string one', () => { + expectTypeOf< + Serializer['serializeToString'] + >().returns.toEqualTypeOf<string>(); + expectTypeOf<Serializer['serialize']>().returns.toEqualTypeOf<Uint8Array>(); + expectTypeOf<Serializer>().toHaveProperty('serializeTo'); + expectTypeOf<Serializer>().toHaveProperty('serializeInto'); +}); + +test('the stream profiles take platform stream types, never a core-internal io type', () => { + expectTypeOf<Serializer['serializeTo']>() + .parameter(1) + .toEqualTypeOf<WritableStream<Uint8Array>>(); + expectTypeOf<Deserializer['deserializeFrom']>() + .parameter(0) + .toEqualTypeOf<ReadableStream<Uint8Array>>(); +}); diff --git a/packages/core/src/seams/serde.ts b/packages/core/src/seams/serde.ts index ecc5c79..adced9b 100644 --- a/packages/core/src/seams/serde.ts +++ b/packages/core/src/seams/serde.ts @@ -2,14 +2,252 @@ // packages/core/src/seams/serde.ts /** - * @internal - * Provisional. `deserialize(data: unknown): T` with `T` inferred from the instance is exactly the - * erased/inferred generic SEAM-21 forbids ("deserialization MUST require an explicit runtime type - * token"). Phase 6's type-witness mechanism will change this interface's shape — do not export this - * from `packages/core/src/index.ts`. + * The runtime type witness a decode operation requires (SERDE-5, closing SEAM-21). + * + * TypeScript erases types completely — there is no runtime class token to reflect over, so an + * erased generic cannot be recovered the way a JVM port recovers one. Instead the caller supplies a + * *value* that already carries the same information: a schema. This interface is deliberately + * structural and minimal so that Zod, Valibot, ArkType, effect/schema, and anything following the + * community "Standard Schema" convention satisfy it without an adapter. `@dexpace/core` defines this + * shape and depends on none of them (SEAM-1). + * + * Because TypeScript infers `T` from the schema's own generic parameter, the compile-time type and + * the runtime witness are one artifact, not two things kept in sync by convention. + * + * @public */ -export interface Serde<T> { +export interface Schema<T> { + /** + * Validate an already-parsed wire value and return it as `T`. + * + * @param input - the decoded-but-unvalidated wire value. + * @returns the value as `T`. + * @throws Whatever the schema library raises; a {@link Deserializer} wraps it as a + * `DeserializationError` with the original chained. + */ + parse(input: unknown): T; +} + +/** + * The encode half of a {@link Serde} (SERDE-3, SERDE-4). + * + * No method takes a {@link Schema} — encoding has the value in hand and needs no witness. + * + * @public + */ +export interface Serializer { + /** + * Encode to a freshly allocated string. + * + * One of `SEAM-20`'s four allocation profiles. A codec whose wire form is not textual (CBOR, + * protobuf) throws a `SerializationError` from this method rather than inventing a lossy + * rendering. + * + * @param value - the value to encode. + * @returns the encoded payload as a fresh string. + * @throws SerializationError when the value cannot be encoded, or when this codec has no textual + * wire form. + */ + serializeToString(value: unknown): string; + + /** + * Encode to a freshly allocated buffer. + * + * @param value - the value to encode. + * @returns the encoded payload as fresh bytes. + * @throws SerializationError when the value cannot be encoded. + */ + serialize(value: unknown): Uint8Array; + + /** + * Encode into a caller-owned buffer at `offset` (default 0), returning the number of bytes + * written. + * + * Bytes before `offset` are left untouched, and the buffer is never resized, reallocated, or + * otherwise taken ownership of. + * + * @param value - the value to encode. + * @param target - the caller-owned buffer to write into. + * @param offset - where to start writing; defaults to 0. + * @returns the number of bytes written. + * @throws RangeError — a plain one, **not** a serde error and with no chained cause — when + * `offset` is out of range or the payload does not fit (SERDE-4). + * @throws SerializationError when the value cannot be encoded. + */ + serializeInto(value: unknown, target: Uint8Array, offset?: number): number; + + /** + * Encode into a caller-owned sink, writing the payload fully. + * + * Does **not** close, abort, or otherwise take ownership of `sink` — the caller opened it and the + * caller closes it (SERDE-3). + * + * @param value - the value to encode. + * @param sink - the caller-owned destination; never closed or aborted by this call. + * @returns a promise resolving once the whole payload has been written. + * @throws SerializationError when the value cannot be encoded. + * @throws TypeError when `sink` is already locked by another writer — the plain platform error, + * not re-typed, because a contended sink is a programmer error rather than an encoding failure. + * @throws Whatever writing to `sink` raised, propagated unwrapped: SERDE-12's rule is + * directional-agnostic, so a genuine write failure is never re-wrapped as a serde exception. + * The writer lock is released on that path too; the sink itself is left errored and unclosed, + * because the caller owns it (SERDE-3). + * + * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named + * `'AbortError'` when none was given. Checked before the writer lock is taken, and then raced + * against each pending write, so an aborted call never leaves the caller's sink locked and never + * closes it (SERDE-3). A write parked against a slow sink is the case the pre-check cannot cover; + * the write itself is left outstanding, because aborting it would be taking ownership. + * + * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the + * project-wide test for whether an API owes one. Buffered-bytes APIs — `serialize`, + * `serializeToString`, `toHttpError`, `Response.bytes()` — correctly take none. + */ + serializeTo( + value: unknown, + sink: WritableStream<Uint8Array>, + options?: {readonly signal?: AbortSignal | undefined}, + ): Promise<void>; +} + +/** + * What to decode into: the runtime witness, the optional label that names it in an error message, + * and whether the target admits a top-level wire `null`. + * + * These describe one thing and travel together, at every layer that decodes — the SPI's + * `deserialize`/`deserializeFrom` and the `decodeResponse`/`decodeSuccessResponse` handlers above + * them. Bundling is also what keeps both inside `max-params: 3`: + * `(response, deserializer, schema, typeName?)` is four parameters, and the optional one counts. + * + * @public + */ +export interface DecodeTarget<T> { + /** The runtime type witness; also the source of the decode's static return type (SERDE-5). */ + readonly schema: Schema<T>; + /** An optional label naming the target in error messages; falls back to `'the target type'`. */ + readonly typeName?: string | undefined; + /** + * Whether a top-level wire `null` is a legal value for this target. Defaults to `false`. + * + * `SERDE-13` requires that a wire `null` decoded into a **non-null** target fail, and an + * implementation cannot tell from a schema *value* whether the target is nullable — a schema + * carries no nullability a codec could read. So the rejection is unconditional by default, and + * runs *before* the schema: moving it after would let a permissive schema such as + * `{parse: (i) => i}` launder a wire `null` into a non-null `T`, which is the heap pollution + * `SERDE-5` and `SERDE-13` exist to prevent. + * + * Setting this to `true` is the caller stating what the schema cannot: that `T` includes `null`. + * The check is then skipped and the `null` reaches the schema, which is free to reject it. Use it + * for an operation whose success body is legitimately the literal `null`, and for the one place + * `tristate(inner)` can serve as a top-level target rather than a field combinator. + * + * A codec MUST honor this flag. Every implementor faces the same limitation, which is why the + * opt-in lives on the target rather than in any one codec's options. + */ + readonly admitsNull?: boolean | undefined; +} + +/** + * The decode half of a {@link Serde} (SERDE-5, SERDE-6, SERDE-13). + * + * `typeName` is an optional diagnostic label, never a witness: a structural schema value carries no + * reliable name, so when a wire `null` is decoded into a non-null target the implementation names + * the target from this label, falling back to `'the target type'`. That literal is part of the + * contract and each codec repeats it — SEAM-1 leaves core with no exported constant to share, so the + * duplication between `response-handlers.ts` and `@dexpace/codec-json` is deliberate, not drift. + * + * **Contract obligation on implementors (SERDE-13).** A wire `null` decoded into a non-null target + * MUST throw `DeserializationError` naming that target, on *every* entry point, and MUST NOT return + * a `null` that flows through the non-null result and detonates at some later field access. Enforce + * it before delegating to the schema — a schema library may or may not reject a bare `null`, and may + * or may not name the target when it does. Core cannot enforce this for you: `decodeResponse` + * streams bytes straight into {@link Deserializer.deserializeFrom} and never holds a parsed value to + * inspect, and core owning a parser would violate SEAM-1. + * + * **A decode target is treated as non-null unless the caller says otherwise.** An implementation + * sees a schema *value*, which carries no nullability it could read, so the check above cannot be + * derived from the schema — it rejects a top-level wire `null` unconditionally *by default*. The one + * way to admit one is {@link DecodeTarget.admitsNull}, the caller stating what the schema value + * cannot: that `T` includes `null`. With it set the check is skipped, the `null` reaches the schema, + * which is free to reject it, and `tristate(inner)` can serve as a top-level target rather than only + * a *field* combinator. Off by default, and deliberately so — the alternative lets a permissive + * schema such as `{parse: (i) => i}` launder a wire `null` into a non-null `T`, which is the heap + * pollution SERDE-5 and SERDE-13 exist to prevent. + * + * **One spelling, both layers.** Every decode entry point takes the schema and its diagnostic label + * bundled as a {@link DecodeTarget}: the SPI here, and `decodeResponse` / `decodeSuccessResponse` + * above it. The two used to differ — positional on the SPI, bundled at the handler layer — so a codec + * author implemented one shape while a caller used the other. Unified 2026-09-04, before the first + * published version, on the object form (`docs/knowledge/harvested/api-design.md:14`). + * + * @public + */ +export interface Deserializer { + /** + * Decode from a complete in-memory payload. + * + * @param data - the encoded bytes. + * @param target - the schema witness, its optional diagnostic label, and whether it admits a + * top-level wire `null`. + * @returns the decoded value. + * @throws DeserializationError on malformed input, a schema rejection, or a wire `null` decoded + * into a target that does not admit one. + */ + deserialize<T>(data: Uint8Array, target: DecodeTarget<T>): T; + + /** + * Decode from a caller-owned source, reading to EOF. + * + * Does **not** cancel or otherwise take ownership of `source` — the caller closes it (SERDE-3). + * + * @param source - the caller-owned byte stream; read to EOF, never cancelled. + * @param target - the schema witness, its optional diagnostic label, and whether it admits a + * top-level wire `null`. + * @param options - `{signal}` to abort the drain. + * @returns a promise of the decoded value. + * @throws DeserializationError on malformed input, a schema rejection, or a wire `null` decoded + * into a non-null target. A genuine stream failure propagates unwrapped (SERDE-12), with the + * reader lock released on that path too and `source` left uncancelled (SERDE-3). + * @throws TypeError when `source` is already locked by another reader — the plain platform error, + * not re-typed, because a contended source is a programmer error rather than a decode failure. + * + * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named + * `'AbortError'` when none was given. Checked before the reader lock is taken, and then raced + * against each pending read, so an aborted call never leaves the caller's source locked and never + * cancels it (SERDE-3). Racing is the load-bearing half: a source that stalls mid-body parks the + * drain inside a read that a between-reads check can never reach again, and an implementation + * that only checks between reads leaves that call unsettled and that source locked forever. + * + * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the + * project-wide test for whether an API owes one. The abort reaches the drain loop; the CPU-bound + * parse that follows is not interruptible by any signal, and `JSON.parse` has no incremental form + * on which a streaming parser could be built. + */ + deserializeFrom<T>( + source: ReadableStream<Uint8Array>, + target: DecodeTarget<T>, + options?: {readonly signal?: AbortSignal | undefined}, + ): Promise<T>; +} + +/** + * The SDK's format-agnostic serialization seam: one encoder, one decoder, and one declared wire + * media type, acquired through a single reference (SERDE-1, SEAM-19). + * + * Not generic in a payload type. A bundle is per-*format*, not per-*type* — the payload type arrives + * as a {@link Schema} parameter of each decode call, so one `jsonSerde()` instance serves every DTO + * in an application. + * + * `mediaType` is required and non-optional so a body built from a value plus a serde can never fall + * back to a format-agnostic default `Content-Type` (SERDE-2). + * + * @public + */ +export interface Serde { + /** The wire media type this bundle's serializer produces; never defaulted at the seam (SERDE-2). */ readonly mediaType: string; - serialize(value: T): unknown; - deserialize(data: unknown): T; + /** The encode half. */ + readonly serializer: Serializer; + /** The decode half. */ + readonly deserializer: Deserializer; } diff --git a/packages/core/src/seams/transport.test.ts b/packages/core/src/seams/transport.test.ts index 81a78ea..d43d497 100644 --- a/packages/core/src/seams/transport.test.ts +++ b/packages/core/src/seams/transport.test.ts @@ -3,12 +3,16 @@ // Exercises: SEAM-18's residual (composeSignal is the per-call-options-threading helper's cancellation half), // XCUT-2 (timeout vs. caller-cancellation told apart by signal.reason.name, not a message string). // No stub Transport is constructed — neither composeSignal nor isTimeoutSignal takes or returns one. +// Also HTTP-35 (composeSignal's documented RangeError is AbortSignal.timeout()'s own, and no value +// RequestOptionsBuilder accepts can produce it). import {describe, expect, test} from 'bun:test'; import { composeSignal, isTimeoutSignal, CancellationError, } from './transport.js'; +import {RequestOptions} from '../http/request-options.js'; +import {RequestOptionsValidationError} from '../http/errors.js'; describe('composeSignal', () => { test('returns undefined when neither input is supplied', () => { @@ -55,3 +59,29 @@ describe('isTimeoutSignal', () => { expect(isTimeoutSignal(new AbortController().signal)).toBe(false); }); }); + +describe('composeSignal timeout range (HTTP-35)', () => { + // `composeSignal` hands `timeoutMs` straight to `AbortSignal.timeout()`, and what that does with + // an out-of-range delay is a RUNTIME decision, measured 2026-09-05: Node raises + // `RangeError: The value of "delay" is out of range` for `1.5`, for `2**32` and for `-1`; Bun + // accepts `1.5` and `2**32` and raises a `TypeError` for `-1`. So the only claim assertable on + // both is the one below — that no value `RequestOptionsBuilder` accepts can reach that fork at + // all. `tests/node-conformance/seams.test.mjs` asserts the Node half, where the throw is real. + // This is why audit #67 / #76 put the range check in the model rather than clamping here. + test('every timeout RequestOptionsBuilder accepts composes without throwing', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + expect(() => composeSignal(undefined, accepted)).not.toThrow(); + } + }); + + test('a timeout the builder rejects never reaches composeSignal', () => { + for (const value of [1.5, 2 ** 32, -1, 0]) { + expect(() => RequestOptions.newBuilder().timeoutMs(value)).toThrow( + RequestOptionsValidationError, + ); + } + }); +}); diff --git a/packages/core/src/seams/transport.ts b/packages/core/src/seams/transport.ts index 8cd62fc..8388603 100644 --- a/packages/core/src/seams/transport.ts +++ b/packages/core/src/seams/transport.ts @@ -76,8 +76,20 @@ export interface Transport { * logic. * * @param userSignal - an optional caller-supplied abort signal. - * @param timeoutMs - an optional timeout, in milliseconds. + * @param timeoutMs - an optional timeout, in milliseconds. Must be an integer in + * `1 .. 2**32 - 1` — the range Node's `AbortSignal.timeout()` accepts. A value taken from + * {@link RequestOptions.timeoutMs} always is, because {@link RequestOptionsBuilder.timeoutMs} + * rejects everything else at the call site (HTTP-35). A transport's own `defaultTimeoutMs` + * construction option is NOT validated by this package and is the one remaining way an + * out-of-range value reaches here. * @returns the composed signal, the sole supplied signal, or `undefined` when neither is supplied. + * @throws Whatever the host runtime's `AbortSignal.timeout()` raises for an out-of-range delay, + * unwrapped. The runtimes disagree, measured 2026-09-05: Node raises `RangeError` for a fractional + * value, for anything above `4294967295`, and for a negative one; Bun accepts the first two and + * raises `TypeError` for the third. Not wrapped in a `DexpaceError` and not clamped here — it is a + * programming error in whatever supplied the value, and the divergence is exactly why the range + * lives on {@link RequestOptionsBuilder.timeoutMs}, which rejects every such value identically on + * both runtimes (HTTP-35, audit #67 / #76). * * @public */ diff --git a/packages/core/src/serde/errors.test.ts b/packages/core/src/serde/errors.test.ts new file mode 100644 index 0000000..b1c919e --- /dev/null +++ b/packages/core/src/serde/errors.test.ts @@ -0,0 +1,77 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/errors.test.ts +// Exercises: SERDE-9 (stable SDK type, cause chained), SERDE-10 (directional subtypes off one root), +// SERDE-11 (unchecked — nothing to assert in JS, documented), SERDE-28 (status/etag/location as fields), +// SEAM-23 (a stable SDK-owned failure hierarchy — here two flat leaves plus the isSerdeError guard). +import {expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import { + DeserializationError, + SerializationError, + isSerdeError, +} from './errors.js'; + +test('both leaves extend DexpaceError directly (two-level tree)', () => { + expect(new SerializationError('x')).toBeInstanceOf(DexpaceError); + expect(new DeserializationError('x')).toBeInstanceOf(DexpaceError); + // The tree is flat: neither is an instance of the other. + expect(new SerializationError('x')).not.toBeInstanceOf(DeserializationError); +}); + +test('cause is chained, not swallowed', () => { + const backing = new Error('JSON.parse blew up'); + const error = new DeserializationError('decode failed', {cause: backing}); + expect(error.cause).toBe(backing); +}); + +test('isSerdeError groups both directions and rejects everything else', () => { + expect(isSerdeError(new SerializationError('x'))).toBe(true); + expect(isSerdeError(new DeserializationError('x'))).toBe(true); + expect(isSerdeError(new DexpaceError('x'))).toBe(false); + expect(isSerdeError(new Error('x'))).toBe(false); + expect(isSerdeError(null)).toBe(false); +}); + +test('DeserializationError carries status/etag/location as readable fields, not only in the message', () => { + const error = new DeserializationError('304 Not Modified: body not decoded', { + status: 304, + etag: 'W/"abc"', + location: null, + }); + expect(error.status).toBe(304); + expect(error.etag).toBe('W/"abc"'); + expect(error.location).toBeNull(); +}); + +test('the optional fields default to null/undefined rather than throwing', () => { + const error = new DeserializationError('plain'); + expect(error.status).toBeUndefined(); + expect(error.etag).toBeNull(); + expect(error.location).toBeNull(); +}); + +test('the write leaf carries no response context — there is no response behind an encode', () => { + // The read path owns status/etag/location. Declaring them on the write leaf too would put three + // permanently-empty fields on a published class; a caller narrows direction first, and + // `instanceof DeserializationError` is what reaches the response context. + const error: SerializationError = new SerializationError('x'); + expect('status' in error).toBe(false); + expect('etag' in error).toBe(false); + expect('location' in error).toBe(false); +}); + +test('isSerdeError narrows to the union; direction is narrowed before response context', () => { + const caught: unknown = new DeserializationError('x', {status: 502}); + expect(isSerdeError(caught)).toBe(true); + // `isSerdeError` alone does not reach `.status` — that is the read leaf's, by design. + expect( + isSerdeError(caught) && caught instanceof DeserializationError + ? caught.status + : undefined, + ).toBe(502); +}); + +test('name is set so a stack trace identifies the leaf', () => { + expect(new SerializationError('x').name).toBe('SerializationError'); + expect(new DeserializationError('x').name).toBe('DeserializationError'); +}); diff --git a/packages/core/src/serde/errors.ts b/packages/core/src/serde/errors.ts new file mode 100644 index 0000000..92f6128 --- /dev/null +++ b/packages/core/src/serde/errors.ts @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * Options common to both serde error leaves. + * + * Carries only the chained cause, which both directions have. Response context belongs to the read + * path alone — see {@link DeserializationErrorOptions}. + * + * @public + */ +export interface SerdeErrorOptions { + /** The backing failure, always chained rather than swallowed (SERDE-9). */ + readonly cause?: unknown; +} + +/** + * Options for the read-path leaf, adding the response context a status-aware handler preserves. + * + * Separate from {@link SerdeErrorOptions} because only a decode can have a response behind it. A + * write-path failure has no status, no `ETag`, and no `Location` to describe, so the fields do not + * exist there rather than existing and being permanently empty. + * + * @public + */ +export interface DeserializationErrorOptions extends SerdeErrorOptions { + /** HTTP status, present only when the error was raised by a status-aware response handler (SERDE-28). */ + readonly status?: number | undefined; + /** `ETag` of the originating response, preserved so conditional-request context survives (SERDE-28). */ + readonly etag?: string | null | undefined; + /** `Location` of the originating response, preserved so redirect context survives (SERDE-28). */ + readonly location?: string | null | undefined; +} + +/** + * A write-path serde failure: an unencodable value, or a codec failure while encoding (SERDE-10). + * + * Sits directly under {@link DexpaceError} — the error tree is deliberately two levels deep, so + * there is no `SerdeError` base class. Use {@link isSerdeError} to catch both directions at once, + * then narrow with `instanceof` when the direction matters. + * + * @public + */ +export class SerializationError extends DexpaceError { + /** + * @param message - the human-readable failure description. + * @param options - the chained cause. + */ + constructor(message: string, options?: SerdeErrorOptions) { + super(message, {cause: options?.cause}); + // No `this.name = ...` here: DexpaceError's constructor already does `this.name = new.target.name`. + } +} + +/** + * A read-path serde failure: malformed input, a shape mismatch, a wire `null` into a non-null target + * (SERDE-13), a missing response body, or a non-decodable status (SERDE-10, SERDE-27, SERDE-28). + * + * A genuine stream failure is **not** this type — it propagates unwrapped (SERDE-12), so a caught + * value for which {@link isSerdeError} is `false` came off the stream rather than out of the codec. + * + * @public + */ +export class DeserializationError extends DexpaceError { + // Declared `T | undefined` rather than `status?: number`: `exactOptionalPropertyTypes` is on, and + // the constructor assigns a possibly-undefined value. The key must exist either way — a reader + // checking `'status' in error` should get a straight answer. + /** The originating HTTP status when a status-aware handler raised this, else `undefined` (SERDE-28). */ + readonly status: number | undefined; + /** The originating response's `ETag`, so conditional context survives the closed response (SERDE-28). */ + readonly etag: string | null; + /** The originating response's `Location`, so redirect context survives the closed response (SERDE-28). */ + readonly location: string | null; + + /** + * @param message - the human-readable failure description; status-led when a handler raised it. + * @param options - the chained cause, plus any response context worth surviving the close. + */ + constructor(message: string, options?: DeserializationErrorOptions) { + super(message, {cause: options?.cause}); + this.status = options?.status; + this.etag = options?.etag ?? null; + this.location = options?.location ?? null; + } +} + +/** + * Type guard grouping both serde directions, so a caller can catch one category without a base class + * (SERDE-9/SERDE-10). Same mechanism as Phase 3b's `isIoError`/`isBodyError`. + * + * Narrows to the union of the two leaves. Direction is the first thing to branch on: a further + * `e instanceof DeserializationError` reaches the read path's response context. + * + * @param e - the caught value. + * @returns whether `e` is either serde leaf. + * @public + */ +export function isSerdeError( + e: unknown, +): e is SerializationError | DeserializationError { + return e instanceof SerializationError || e instanceof DeserializationError; +} diff --git a/packages/core/src/serde/response-handlers.test.ts b/packages/core/src/serde/response-handlers.test.ts new file mode 100644 index 0000000..b921b2b --- /dev/null +++ b/packages/core/src/serde/response-handlers.test.ts @@ -0,0 +1,475 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/response-handlers.test.ts +// Exercises: SERDE-27 (stream through the deserializer without materializing, close on every path, missing +// body names the target, codec failure wrapped with cause, genuine I/O error propagates unwrapped), +// SERDE-12 (a stream failure is never re-wrapped — every leaf of this SDK's typed tree, not just +// `IoError`, whose tree is FLAT), SERDE-28 (status-aware routing, preserved ETag/Location). +import {expect, test} from 'bun:test'; +import {HttpStatusError} from '../body/http-status-error.js'; +import {Status} from '../http/status.js'; +import { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + SourceContractViolationError, +} from '../io/errors.js'; +import type {DecodeTarget, Deserializer, Schema} from '../seams/serde.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {DeserializationError} from './errors.js'; +import {decodeResponse, decodeSuccessResponse} from './response-handlers.js'; + +interface Dto { + readonly id: number; +} + +const dtoSchema: Schema<Dto> = { + parse(input: unknown): Dto { + // `as Dto`: probing one field on an `unknown` already proven a non-null object. + if ( + typeof input !== 'object' || + input === null || + typeof (input as Dto).id !== 'number' + ) { + throw new Error('not a Dto'); + } + return input as Dto; + }, +}; + +/** A deserializer that reads the source to EOF and JSON-parses it. Never cancels the source. */ +const jsonish: Deserializer = { + deserialize<T>(data: Uint8Array, target: DecodeTarget<T>): T { + // `as unknown`: `JSON.parse` is typed `any`; the cast narrows away from it at the boundary. + return target.schema.parse( + JSON.parse(new TextDecoder().decode(data)) as unknown, + ); + }, + async deserializeFrom<T>( + source: ReadableStream<Uint8Array>, + target: DecodeTarget<T>, + options?: {readonly signal?: AbortSignal | undefined}, + ): Promise<T> { + options?.signal?.throwIfAborted(); + const reader = source.getReader(); + const chunks: Uint8Array[] = []; + for (;;) { + const {done, value} = await reader.read(); + if (done) break; + chunks.push(value); + } + const text = chunks.map(c => new TextDecoder().decode(c)).join(''); + return target.schema.parse(JSON.parse(text) as unknown); + }, +}; + +function bodyOf(text: string): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); +} + +function failingBody(error: Error): ReadableStream<Uint8Array> { + return new ReadableStream<Uint8Array>({ + start(controller) { + controller.error(error); + }, + }); +} + +type FakeResponse = Parameters<typeof decodeResponse>[0]; + +/** Distinguishes "the promise resolved" from a rejection value that happens to be falsy. */ +const RESOLVED = Symbol('resolved'); + +/** + * Settles `promise` and hands back whatever it rejected with. + * + * `expect(p).rejects.toX()` is typed `void` under `bun:test`, so awaiting it trips `await-thenable`; + * capturing the rejection as a value is the idiom the rest of this package's async tests use. + */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return RESOLVED; + } catch (e: unknown) { + return e; + } +} + +/** Minimal close-counting stand-in for `Response`. Only the fields the handler touches. */ +function fakeResponse(body: ReadableStream<Uint8Array> | null): { + response: FakeResponse; + closes: () => number; +} { + let closeCount = 0; + // `as unknown as FakeResponse`: `Response` carries `#private` fields, so no object literal is + // assignable to it; `decodeResponse` reads only `body` and calls `close()`. + const response = { + body, + close(): Promise<void> { + closeCount += 1; + return Promise.resolve(); + }, + } as unknown as FakeResponse; + return {response, closes: () => closeCount}; +} + +/** A response whose `close()` always rejects, for the suppression cases. */ +function failingCloseResponse( + body: ReadableStream<Uint8Array> | null, + closeFailure: Error, +): FakeResponse { + // `as unknown as FakeResponse`: see `fakeResponse` above. + return { + body, + close(): Promise<void> { + return Promise.reject(closeFailure); + }, + } as unknown as FakeResponse; +} + +const dtoTarget = {schema: dtoSchema, typeName: 'Dto'} as const; + +test('a valid body decodes to the typed value and the response closes exactly once', async () => { + const {response, closes} = fakeResponse(bodyOf('{"id":7}')); + + expect(await decodeResponse(response, jsonish, dtoTarget)).toEqual({id: 7}); + expect(closes()).toBe(1); +}); + +test('a missing body throws DeserializationError naming the target, and still closes', async () => { + const {response, closes} = fakeResponse(null); + + const caught = await rejection(decodeResponse(response, jsonish, dtoTarget)); + + expect(caught).toBeInstanceOf(DeserializationError); + expect(caught).toHaveProperty('message', expect.stringContaining('Dto')); + expect(closes()).toBe(1); +}); + +test('a missing body with no typeName falls back to a documented label', async () => { + const {response} = fakeResponse(null); + + const caught = await rejection( + decodeResponse(response, jsonish, {schema: dtoSchema}), + ); + + expect(caught).toHaveProperty( + 'message', + expect.stringContaining('the target type'), + ); +}); + +test('a codec/shape failure is wrapped as DeserializationError with the original chained', async () => { + const {response, closes} = fakeResponse(bodyOf('{"id":"not-a-number"}')); + + const caught = await rejection(decodeResponse(response, jsonish, dtoTarget)); + + expect(caught).toBeInstanceOf(DeserializationError); + expect(caught).toHaveProperty('cause', expect.any(Error)); + expect(closes()).toBe(1); +}); + +test('a DeserializationError the codec already raised is not double-wrapped', async () => { + const original = new DeserializationError('codec said no'); + const rejecting: Deserializer = { + deserialize: () => { + throw original; + }, + deserializeFrom: () => Promise.reject(original), + }; + const {response} = fakeResponse(bodyOf('{}')); + + expect(await rejection(decodeResponse(response, rejecting, dtoTarget))).toBe( + original, + ); +}); + +test('a close failure does NOT mask the decode failure — decode primary, close suppressed', async () => { + // A bare `finally { await response.close() }` would replace the DeserializationError with the close + // error, telling the caller their socket died when in fact their payload was malformed. + // + // Asserted on SHAPE, never `instanceof SuppressedError`: that class is absent on the declared + // `engines.node` floor, so the instanceof form would silently assert nothing there (see suppress.ts). + const closeFailure = new IoError('close failed'); + + const caught = await rejection( + decodeResponse( + failingCloseResponse(bodyOf('{"id":"not-a-number"}'), closeFailure), + jsonish, + dtoTarget, + ), + ); + + // `as SuppressedErrorLike`: narrowed by the name assertion below, which the compiler cannot follow. + const paired = caught as SuppressedErrorLike; + expect(paired.name).toBe('SuppressedError'); + expect(paired.error).toBeInstanceOf(DeserializationError); + expect(paired.suppressed).toBe(closeFailure); +}); + +test('a close failure on the SUCCESS path surfaces plainly — it is the only failure there is', async () => { + const closeFailure = new IoError('close failed'); + + const caught = await rejection( + decodeResponse( + failingCloseResponse(bodyOf('{"id":7}'), closeFailure), + jsonish, + dtoTarget, + ), + ); + + expect(caught).toBe(closeFailure); +}); + +/** + * Adds the status/header surface `decodeSuccessResponse` reads, on top of the stand-in above. + * + * `text()`/`bytes()` are present because the 4xx/5xx branch delegates to 3b's real `toHttpError()`, + * which buffers a bounded copy of the error body — a stand-in carrying only + * `status`/`headers`/`body`/`close` would fail inside `toHttpError`, not inside the code under test, + * and the resulting error would be misleading. Both read the same `body` stream once, matching the + * real `Response`'s single-use discipline. + */ +function fakeStatusResponse( + code: number, + body: ReadableStream<Uint8Array> | null, + headers: Readonly<Record<string, string>> = {}, +): {response: FakeResponse; closes: () => number} { + let closeCount = 0; + const drain = async (): Promise<Uint8Array> => { + if (body === null) return new Uint8Array(); + const reader = body.getReader(); + const chunks: Uint8Array[] = []; + for (;;) { + const {done, value} = await reader.read(); + if (done) break; + chunks.push(value); + } + reader.releaseLock(); + const total = chunks.reduce((n, c) => n + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; + }; + // `as unknown as FakeResponse`: see `fakeResponse` above. + const response = { + status: Status.of(code), + headers: {get: (name: string) => headers[name.toLowerCase()]}, + body, + bytes: drain, + text: async () => new TextDecoder().decode(await drain()), + close(): Promise<void> { + closeCount += 1; + return Promise.resolve(); + }, + } as unknown as FakeResponse; + return {response, closes: () => closeCount}; +} + +test('2xx decodes the body', async () => { + const {response, closes} = fakeStatusResponse(200, bodyOf('{"id":1}')); + + expect(await decodeSuccessResponse(response, jsonish, dtoTarget)).toEqual({ + id: 1, + }); + expect(closes()).toBe(1); +}); + +test('500 throws the mapped HTTP error, not a decode of the error payload as the success type', async () => { + const {response, closes} = fakeStatusResponse( + 500, + bodyOf('{"error":"boom"}'), + ); + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, dtoTarget), + ); + + expect(caught).toBeInstanceOf(HttpStatusError); + // SERDE-27's close-on-every-path covers this branch too, even though the close happens inside + // `toHttpError`. Asserting it here keeps that delegation honest if 3b's implementation changes. + expect(closes()).toBe(1); +}); + +test('a non-canonical 599 is treated as a server error, not as an "other" status', async () => { + const {response, closes} = fakeStatusResponse(599, bodyOf('nope')); + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, dtoTarget), + ); + + expect(caught).toBeInstanceOf(HttpStatusError); + expect(closes()).toBe(1); +}); + +test('304 closes and raises a status-leading DeserializationError preserving ETag/Location', async () => { + const {response, closes} = fakeStatusResponse(304, null, { + etag: 'W/"v1"', + location: '/next', + }); + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, dtoTarget), + ); + + expect(caught).toBeInstanceOf(DeserializationError); + // `as DeserializationError`: narrowed by the assertion above, which the compiler cannot follow. + const error = caught as DeserializationError; + expect(error.message.startsWith('304')).toBe(true); + expect(error.status).toBe(304); + expect(error.etag).toBe('W/"v1"'); + expect(error.location).toBe('/next'); + expect(closes()).toBe(1); +}); + +test('a 1xx is also an "other" status, closed and reported, never decoded', async () => { + const {response, closes} = fakeStatusResponse(102, bodyOf('{"id":1}')); + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, dtoTarget), + ); + + expect(caught).toBeInstanceOf(DeserializationError); + expect(closes()).toBe(1); +}); + +test('an "other" status whose close also fails keeps the status error primary', async () => { + const closeFailure = new IoError('close failed'); + // `as unknown as FakeResponse`: see `fakeResponse` above. + const response = { + status: Status.of(304), + headers: {get: () => undefined}, + body: null, + close: () => Promise.reject(closeFailure), + } as unknown as FakeResponse; + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, dtoTarget), + ); + + // `as SuppressedErrorLike`: narrowed by the name assertion below. + const paired = caught as SuppressedErrorLike; + expect(paired.name).toBe('SuppressedError'); + expect(paired.error).toBeInstanceOf(DeserializationError); + expect(paired.suppressed).toBe(closeFailure); +}); + +test('the "other" status message falls back to the documented label with no typeName', async () => { + const {response} = fakeStatusResponse(304, null); + + const caught = await rejection( + decodeSuccessResponse(response, jsonish, {schema: dtoSchema}), + ); + + expect(caught).toHaveProperty( + 'message', + expect.stringContaining('the target type'), + ); +}); + +// --- SERDE-12: the typed-tree pass-through, leaf by leaf --------------------------------------- +// +// `io/errors.ts` is a FLAT tree: `EndOfStreamError`, `ClosedResourceError`, `AllocationLimitError` +// and `SourceContractViolationError` all extend `DexpaceError` DIRECTLY, not `IoError`. Any guard +// narrower than `DexpaceError` — `e instanceof IoError` being the obvious one — whitelists exactly +// one of the five and re-stamps the other four as `DeserializationError`, telling a caller their +// payload was malformed when their stream had ended early. One case per leaf, so a future reshuffle +// of that tree cannot regress them silently. + +const streamLeaves: readonly (readonly [string, Error])[] = [ + ['IoError', new IoError('socket reset')], + ['EndOfStreamError', new EndOfStreamError(3, 10)], + ['ClosedResourceError', new ClosedResourceError('source closed')], + ['AllocationLimitError', new AllocationLimitError(1_048_577, 1_048_576)], + [ + 'SourceContractViolationError', + new SourceContractViolationError('zero bytes for a positive read'), + ], +]; + +for (const [name, failure] of streamLeaves) { + test(`a stream failure of type ${name} propagates unwrapped (SERDE-12)`, async () => { + const {response, closes} = fakeResponse(failingBody(failure)); + + const caught = await rejection( + decodeResponse(response, jsonish, dtoTarget), + ); + + expect(caught).toBe(failure); + expect(caught).not.toBeInstanceOf(DeserializationError); + expect(closes()).toBe(1); + }); +} + +test('an HttpStatusError raised mid-decode is passed through, never re-typed', async () => { + // Not an I/O leaf and not a serde leaf, but still the SDK's own typed tree: nothing already + // carrying an SDK type may be re-stamped by this handler. + const statusFailure = new HttpStatusError(503, undefined, undefined); + const {response, closes} = fakeResponse(failingBody(statusFailure)); + + const caught = await rejection(decodeResponse(response, jsonish, dtoTarget)); + + expect(caught).toBe(statusFailure); + expect(closes()).toBe(1); +}); + +test('a FOREIGN stream error is wrapped — the documented, irreducible limit of the discriminator', async () => { + // Pins the limitation rather than the ideal, so it stays visible instead of latent. Core hands + // the live stream to the codec and never reads it, so at the catch a transport's raw `Error` and + // a non-conforming codec leaking one are the same shape — and SERDE-27 requires the codec case be + // surfaced as a serde exception. Fixing this needs the transport to tag its stream errors; when + // that lands, THIS test is the one that should change. + const foreign = new Error('ECONNRESET'); + const {response, closes} = fakeResponse(failingBody(foreign)); + + const caught = await rejection(decodeResponse(response, jsonish, dtoTarget)); + + expect(caught).toBeInstanceOf(DeserializationError); + expect((caught as DeserializationError).cause).toBe(foreign); + expect(closes()).toBe(1); +}); + +// --- a locked body is a programmer error, not a payload failure -------------------------------- + +test('a body already locked by another consumer raises a plain TypeError, not a decode failure', async () => { + const body = bodyOf('{"id":1}'); + const stolen = body.getReader(); // an external consumer got there first + const {response, closes} = fakeResponse(body); + + const caught = await rejection(decodeResponse(response, jsonish, dtoTarget)); + + expect(caught).toBeInstanceOf(TypeError); + expect(caught).not.toBeInstanceOf(DeserializationError); + expect((caught as TypeError).message).toContain('Dto'); + // Still closed: a programmer error must not also strand the connection. + expect(closes()).toBe(1); + stolen.releaseLock(); +}); + +test('two concurrent decodes of one response: the loser reports contention, not a bad payload', async () => { + const {response, closes} = fakeResponse(bodyOf('{"id":1}')); + + const [first, second] = await Promise.allSettled([ + decodeResponse(response, jsonish, dtoTarget), + decodeResponse(response, jsonish, dtoTarget), + ]); + + // One wins outright; the other is told it raced, in the platform's own vocabulary. + const outcomes = [first, second]; + const fulfilled = outcomes.filter(o => o.status === 'fulfilled'); + const rejected = outcomes.filter(o => o.status === 'rejected'); + expect(fulfilled).toHaveLength(1); + expect(rejected).toHaveLength(1); + expect(rejected[0]?.reason).toBeInstanceOf(TypeError); + expect(rejected[0]?.reason).not.toBeInstanceOf(DeserializationError); + expect(closes()).toBe(2); +}); diff --git a/packages/core/src/serde/response-handlers.ts b/packages/core/src/serde/response-handlers.ts new file mode 100644 index 0000000..9af5f29 --- /dev/null +++ b/packages/core/src/serde/response-handlers.ts @@ -0,0 +1,217 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/response-handlers.ts +import {toHttpError} from '../body/http-status-error.js'; +import {DexpaceError} from '../http/errors.js'; +import type {Response} from '../http/response.js'; +import {invariant} from '../invariant.js'; +import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; +import type {DecodeTarget, Deserializer} from '../seams/serde.js'; + +// Re-exported so the handler layer and the seam name one type, not two. It is DECLARED on the seam +// because the seam is what a third-party codec implements against. +export type {DecodeTarget}; +import {DeserializationError} from './errors.js'; + +const UNNAMED_TARGET = 'the target type'; + +/** + * Run `work`, then close `response` — on every path, but **without letting a close failure eat the + * real one**. + * + * A bare `finally { await response.close() }` looks equivalent and is not: when `close()` rejects + * while an error is already in flight, the `finally`'s rejection *replaces* it, and the caller is + * told their connection dropped when in fact their payload was malformed. So: + * + * - work threw, close succeeded → the work error propagates + * - work threw, close also threw → the work error stays **primary**, the close error attaches as + * `suppressed` + * - work succeeded, close threw → the close error propagates; it is the only failure there is + * + * Built on 4b's `releaseQuietly`/`withReleaseFailure` rather than a second suppression mechanism, so + * the ordering — and the identity guard those helpers carry for `Response.close()`'s memoized + * rejection — cannot drift between the retry, redirect, auth, and serde subsystems. + */ +async function closingAfter<T>( + response: Response, + work: () => Promise<T>, +): Promise<T> { + let result: T; + try { + result = await work(); + } catch (primary: unknown) { + throw withReleaseFailure(primary, await releaseQuietly(response)); + } + await response.close(); + return result; +} + +/** + * Decode a response body directly through a {@link Deserializer} into the schema's type (SERDE-27). + * + * The live body stream is handed to the deserializer — **this function never buffers it**. Whether + * the codec on the other side buffers is the codec's business: `@dexpace/codec-json` must, because + * `JSON.parse` has no incremental form, and that limitation is recorded in the phase's Deviation + * Ledger. + * + * The response is closed on every path — success, missing body, codec failure, and stream failure + * alike — so no path can strand the connection, and a close failure never displaces the failure that + * actually matters. + * + * Failure routing follows SERDE-12: only malformed-input and shape-mismatch failures are wrapped as + * {@link DeserializationError} with the original chained. A genuine stream failure propagates + * untouched, because re-wrapping it would tell a caller their payload was malformed when their + * socket dropped. + * + * **Telling the two apart.** `isSerdeError(e)` is the discriminator: `true` means the payload was + * the problem — malformed bytes, a shape mismatch, a missing body — and the value is a + * {@link DeserializationError}. `false` means the failure came off the stream (or out of `close()`) + * and was deliberately not re-typed. The stream-failure class itself is not part of this package's + * public surface, so the structural check is the supported test rather than an `instanceof`. + * + * **The limit of that discriminator, stated plainly.** Every error already in the SDK's own typed + * tree passes through untouched, so a stream failure raised by this SDK's I/O layer is always + * recognizable. A **foreign** stream error is not: core hands the live stream to the codec and + * never reads it, so at the point of the catch a transport's raw error is indistinguishable from a + * non-conforming codec leaking one. Since SERDE-27 requires a codec/parse failure be surfaced as a + * serde exception, the untyped case is wrapped — and a foreign transport's stream error is + * therefore reported as a {@link DeserializationError}. Affected in practice: any transport whose + * response body is not built by this SDK — a `fetch`/undici body (`TypeError('terminated')`), a + * hand-built `ReadableStream` errored with a bare `Error`, or an aborted body + * (`DOMException` named `'AbortError'`). Distinguishing them needs the transport to tag its stream + * errors; until it does, treat `isSerdeError(e) === true` as "payload **or** foreign stream", not + * as proof of a payload failure. + * + * @param response - the response to decode and close. + * @param deserializer - the decode half of a `Serde`; explicit because core owns no codec (SEAM-1). + * @param target - the runtime witness plus its optional diagnostic label. + * @returns a promise of the decoded value. + * @throws DeserializationError when the response carried no body, when the payload is malformed or + * does not match the schema, or — see the limit above — when a foreign stream error could not be + * told apart from a codec failure. + * @throws TypeError when the response body is already locked by another consumer. A programmer + * error (two consumers racing one response), reported as the same plain `TypeError` + * `Response.bytes()` raises for it rather than being demoted to a payload failure. + * @throws Whatever reading the body raised — every error in this SDK's typed tree propagates + * unwrapped (SERDE-12) — plus whatever `close()` raised when the decode itself succeeded. + * @throws An error carrying a suppressed secondary, when the decode failed **and** releasing the + * response then failed too. Its `name` is `'SuppressedError'`, `.error` is the primary failure (the + * one worth acting on) and `.suppressed` is the release failure. `instanceof SuppressedError` is + * **not** a valid test: the class is absent on this package's declared Node floor and a + * structurally identical stand-in is built there instead. Test the shape, or read `.error` + * unconditionally. + * @public + */ +export async function decodeResponse<T>( + response: Response, + deserializer: Deserializer, + target: DecodeTarget<T>, +): Promise<T> { + const label = target.typeName ?? UNNAMED_TARGET; + return closingAfter(response, async () => { + const body = response.body; + if (body === null) { + throw new DeserializationError( + `response carried no body to decode into ${label}`, + ); + } + if (body.locked) { + // A locked body means two consumers are racing one response — a programmer error, not a + // malformed payload. Raised as the plain `TypeError` `Response.bytes()` already surfaces for + // the same mistake, and raised HERE so the catch below cannot demote it to a + // `DeserializationError` that blames the server's payload for the caller's bug. + throw new TypeError( + `the body of the response being decoded into ${label} is already locked by another consumer`, + ); + } + try { + return await deserializer.deserializeFrom(body, target); + } catch (e: unknown) { + // SERDE-12: anything already in the SDK's typed error tree passes through untouched. That + // covers every I/O leaf (`IoError`, `EndOfStreamError`, `ClosedResourceError`, + // `AllocationLimitError`, `SourceContractViolationError` — a FLAT tree, so an + // `instanceof IoError` check caught only one of the five), `DeserializationError` from a + // conforming codec, and `HttpStatusError`. Re-typing any of them would tell a caller their + // payload was malformed when their socket dropped. + // + // The wrap that remains exists for SERDE-27's "surface a codec failure as a serde exception" + // clause, against a NON-CONFORMING codec that leaks a raw `SyntaxError` instead of the + // `DeserializationError` the `Deserializer` contract obliges it to throw. Its cost is stated + // in this function's `@throws` block and cannot be removed here: core hands the live stream + // to the codec and never reads it, so at this point a foreign transport's stream error and a + // foreign codec's leaked error are the same shape. + if (e instanceof DexpaceError) throw e; + throw new DeserializationError( + `failed to decode the response body into ${label}`, + {cause: e}, + ); + } + }); +} + +/** + * Decode only on success; map failure statuses instead of decoding them (SERDE-28). + * + * - **2xx** — delegates to {@link decodeResponse}. + * - **4xx/5xx** — delegates to Phase 3b's `toHttpError()`, which buffers a bounded in-memory copy of + * the error body inside the response's own close-guaranteeing scope, at the shared 1 MiB cap + * (`BODY-30`/`HTTP-52`). There is deliberately no second cap here: §14 points at that one + * explicitly, and a second would drift. + * - **anything else** (1xx, an unfollowed 3xx such as 304) — closes the response and raises a + * {@link DeserializationError} whose message leads with the status code, carrying `ETag` and + * `Location` as readable fields so conditional and redirect context survives the closed response. + * + * Decoding an error payload as the success type is the failure mode this function exists to prevent: + * it produces a shape mismatch that blames the caller's schema for the server's 500. + * + * @param response - the response to inspect, decode or map, and close. + * @param deserializer - the decode half of a `Serde`; explicit because core owns no codec (SEAM-1). + * @param target - the runtime witness plus its optional diagnostic label. + * @returns a promise of the decoded value, for a 2xx only. + * @throws HttpStatusError on 4xx/5xx, carrying a bounded copy of the error body. + * @throws DeserializationError on any other non-2xx status, and on a 2xx whose body is missing, + * malformed, or does not match the schema. + * @throws TypeError when a 2xx response's body is already locked by another consumer, exactly as + * {@link decodeResponse} documents. + * @throws Whatever reading a 2xx body raised — every error in this SDK's typed tree propagates + * unwrapped (SERDE-12). The same discriminator {@link decodeResponse} documents applies here, with + * the same stated limit for a foreign transport's stream errors. + * @throws An error carrying a suppressed secondary, when the failure that should propagate and the + * release that ran on its way out **both** failed. `name` is `'SuppressedError'`, `.error` is + * primary and `.suppressed` rides along; `instanceof` is not a valid test on the declared floor. + * See {@link decodeResponse}. + * @public + */ +export async function decodeSuccessResponse<T>( + response: Response, + deserializer: Deserializer, + target: DecodeTarget<T>, +): Promise<T> { + const status = response.status; + + if (status.isSuccess) { + return decodeResponse(response, deserializer, target); + } + + if (status.isClientError || status.isServerError) { + const httpError = await toHttpError(response); + // `toHttpError` returns null only for a non-4xx/5xx response, which this branch has already excluded. + invariant( + httpError !== null, + 'toHttpError returned null for a 4xx/5xx response', + ); + throw httpError; + } + + const etag = response.headers.get('ETag') ?? null; + const location = response.headers.get('Location') ?? null; + // Routed through the same helper as `decodeResponse` rather than a `try { throw } finally { close }`: if the + // close fails here too, the status error must stay primary, not be replaced by it. + return closingAfter(response, () => + Promise.reject( + new DeserializationError( + `${String(status.code)}: response status is not decodable into ${target.typeName ?? UNNAMED_TARGET}`, + {status: status.code, etag, location}, + ), + ), + ); +} diff --git a/packages/core/src/serde/tristate.test.ts b/packages/core/src/serde/tristate.test.ts new file mode 100644 index 0000000..ec5119b --- /dev/null +++ b/packages/core/src/serde/tristate.test.ts @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/tristate.test.ts +// Exercises: SERDE-14 (three states, Present-of-null unrepresentable), SERDE-18 (helpers, ofNullable never +// yields Absent), SERDE-30 (stable identity-free string form). +import {expect, test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import type {Tristate} from './tristate.js'; +import { + absent, + foldTristate, + isAbsent, + isNull, + isPresent, + isTristate, + nullValue, + ofNullable, + present, + tristateToString, + valueOrNull, +} from './tristate.js'; + +test('exactly three states, discriminated by kind', () => { + expect(absent().kind).toBe('absent'); + expect(nullValue().kind).toBe('null'); + expect(present(42).kind).toBe('present'); +}); + +test('Present carries its value', () => { + const t = present({id: 7}); + expect(isPresent(t) ? t.value : undefined).toEqual({id: 7}); +}); + +test('predicates are mutually exclusive', () => { + expect([isAbsent(absent()), isNull(absent()), isPresent(absent())]).toEqual([ + true, + false, + false, + ]); + expect([ + isAbsent(nullValue()), + isNull(nullValue()), + isPresent(nullValue()), + ]).toEqual([false, true, false]); + expect([ + isAbsent(present(1)), + isNull(present(1)), + isPresent(present(1)), + ]).toEqual([false, false, true]); +}); + +test('ofNullable maps null and undefined to Null, never to Absent', () => { + expect(ofNullable(null).kind).toBe('null'); + expect(ofNullable(undefined).kind).toBe('null'); + expect(ofNullable('x').kind).toBe('present'); +}); + +test('foldTristate dispatches all three branches', () => { + const label = <T>(t: Tristate<T>): string => + foldTristate(t, { + onAbsent: () => 'A', + onNull: () => 'N', + onPresent: v => `P:${String(v)}`, + }); + expect(label(absent())).toBe('A'); + expect(label(nullValue())).toBe('N'); + expect(label(present('hi'))).toBe('P:hi'); +}); + +test('valueOrNull collapses both empty branches to null', () => { + expect(valueOrNull(absent())).toBeNull(); + expect(valueOrNull(nullValue())).toBeNull(); + expect(valueOrNull(present(5))).toBe(5); +}); + +test('sentinels have a stable, identity-free string form (SERDE-30)', () => { + expect(tristateToString(absent())).toBe('Absent'); + expect(tristateToString(nullValue())).toBe('Null'); + expect(tristateToString(present(3))).toBe('Present(3)'); + // Two separately constructed sentinels render identically — no identity hash leaks. + expect(tristateToString(absent())).toBe(tristateToString(absent())); +}); + +test('values are frozen — a Tristate cannot be mutated after construction', () => { + // `as unknown as {kind: string}`: deliberately widening away `readonly` and the literal type to + // prove the *runtime* freeze, which the type system alone cannot demonstrate. + const t = present(1) as unknown as {kind: string}; + expect(() => { + t.kind = 'absent'; + }).toThrow(); +}); + +test('isTristate accepts only branded values — truth table', () => { + // A custom type guard needs the full table, not just the happy case (docs/knowledge/harvested/testing.md:34). + expect([ + isTristate(absent()), + isTristate(nullValue()), + isTristate(present(1)), + ]).toEqual([true, true, true]); + expect([ + isTristate(null), + isTristate(undefined), + isTristate({}), + isTristate({kind: 'absent'}), + isTristate('absent'), + isTristate(0), + isTristate([]), + ]).toEqual([false, false, false, false, false, false, false]); +}); + +test('Present of null does not type-check — the illegal fourth state is unrepresentable (SERDE-14)', () => { + expectTypeOf<Parameters<typeof present<string>>[0]>().toEqualTypeOf<string>(); + // `NonNullable<string | null>` is `string`, so `present<string | null>(null)` is rejected by the compiler. + expectTypeOf< + Parameters<typeof present<string | null>>[0] + >().toEqualTypeOf<string>(); + // @ts-expect-error — SERDE-14: Present-of-null must not compile + present<string | null>(null); +}); + +test('Absent and Null are assignable to any parameterization (SERDE-14 covariance)', () => { + expectTypeOf(absent()).toExtend<Tristate<number>>(); + expectTypeOf(nullValue()).toExtend<Tristate<{deep: string}>>(); +}); diff --git a/packages/core/src/serde/tristate.ts b/packages/core/src/serde/tristate.ts new file mode 100644 index 0000000..915047a --- /dev/null +++ b/packages/core/src/serde/tristate.ts @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/serde/tristate.ts + +/** + * Branding symbol. A wire codec recognizes a {@link Tristate} by this key rather than by structural + * shape, so a caller DTO that happens to carry a `kind` field is never mistaken for one. + * + * Exported because `@dexpace/codec-json` — a *separate package* — needs it. That is also why + * `@dexpace/codec-json` declares `@dexpace/core` as a `peerDependency`: two copies of core in one + * dependency tree would mean two distinct symbols, and the codec would silently stop recognizing a + * caller's Tristate values — emitting a key the caller asked to omit. `Symbol.for` resolves through + * the cross-realm registry, so even two non-identical copies of core agree on this key. + * + * @public + */ +export const TRISTATE_BRAND: unique symbol = Symbol.for( + '@dexpace/core.Tristate', +); + +/** + * The PATCH three-state type: a key missing from the wire, a key present with an explicit `null`, or + * a key present with a value (SERDE-14). + * + * A discriminated union over frozen object literals, never a class hierarchy + * (`styleguide/typescript/06` §6.4/§6.5) — the same pattern as `Body`'s `kind` union and + * `Outcome<T>`. + * + * The illegal fourth state (Present of `null`) is unrepresentable *at the type level*, because + * {@link present} takes `NonNullable<T>`. That is strictly earlier than a construction-time runtime + * rejection. + * + * @public + */ +export type Tristate<T> = + | {readonly [TRISTATE_BRAND]: true; readonly kind: 'absent'} + | {readonly [TRISTATE_BRAND]: true; readonly kind: 'null'} + | { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'present'; + readonly value: T; + }; + +const ABSENT = Object.freeze({[TRISTATE_BRAND]: true, kind: 'absent'} as const); +const NULL = Object.freeze({[TRISTATE_BRAND]: true, kind: 'null'} as const); + +/** + * The key was absent from the wire — a PATCH server reads this as "leave unchanged". + * + * @returns the shared Absent sentinel. + * @public + */ +export function absent(): Tristate<never> { + return ABSENT; +} + +/** + * The key was present with an explicit wire `null` — a PATCH server reads this as "clear". + * + * Named `nullValue`, not `null`, because `null` is a reserved word. + * + * @returns the shared Null sentinel. + * @public + */ +export function nullValue(): Tristate<never> { + return NULL; +} + +/** + * The key was present with a value. `value` cannot be `null` or `undefined` (SERDE-14). + * + * @param value - the non-nullish inner value. + * @returns a frozen Present carrying `value`. + * @public + */ +export function present<T>(value: NonNullable<T>): Tristate<T> { + return Object.freeze({ + [TRISTATE_BRAND]: true, + kind: 'present', + value, + } as const); +} + +/** + * Map a nullable value into a Tristate. Never yields Absent (SERDE-18) — a caller holding a + * `T | null` has by definition observed the field, so "missing" is not one of the outcomes available + * to it. + * + * @param value - the nullable value to lift. + * @returns Null for `null`/`undefined`, otherwise Present. + * @public + */ +export function ofNullable<T>(value: T | null | undefined): Tristate<T> { + return value === null || value === undefined ? NULL : present<T>(value); +} + +/** + * The three branches {@link foldTristate} dispatches to. + * + * @public + */ +export interface TristateBranches<T, R> { + /** Called when the key was missing from the wire. */ + readonly onAbsent: () => R; + /** Called when the key carried an explicit wire `null`. */ + readonly onNull: () => R; + /** Called with the inner value when the key carried one. */ + readonly onPresent: (value: T) => R; +} + +/** + * Exhaustive three-way dispatch (SERDE-18). + * + * Named `foldTristate`, not `fold`, because `Outcome<T>` (Phase 4b) already owns a `fold` in this + * codebase. Both land in the same public barrel eventually; two different `fold`s exported from one + * entry point would be an ambiguity a caller has to resolve at every import site. + * + * The branches travel in one object rather than as three trailing parameters: positionally this is a + * four-parameter function, and ESLint's `max-params: 3` counts them all. It also reads better — + * three bare arrow arguments in a row are indistinguishable at the call site. + * + * @param tristate - the value to dispatch on. + * @param branches - the three handlers. + * @returns whatever the matching branch returned. + * @public + */ +export function foldTristate<T, R>( + tristate: Tristate<T>, + branches: TristateBranches<T, R>, +): R { + switch (tristate.kind) { + case 'absent': + return branches.onAbsent(); + case 'null': + return branches.onNull(); + case 'present': + return branches.onPresent(tristate.value); + } +} + +/** + * Collapse both empty branches to `null` (SERDE-18). Lossy by design — use {@link foldTristate} to + * distinguish them. + * + * @param tristate - the value to unwrap. + * @returns the inner value, or `null` for Absent and Null alike. + * @public + */ +export function valueOrNull<T>(tristate: Tristate<T>): T | null { + return tristate.kind === 'present' ? tristate.value : null; +} + +/** + * True when the key was missing from the wire — "leave unchanged" (SERDE-18). + * + * @param tristate - the value to test. + * @returns whether it is Absent, narrowing on true. + * @public + */ +export function isAbsent<T>( + tristate: Tristate<T>, +): tristate is {readonly [TRISTATE_BRAND]: true; readonly kind: 'absent'} { + return tristate.kind === 'absent'; +} + +/** + * True when the key carried an explicit wire `null` — "clear" (SERDE-18). + * + * @param tristate - the value to test. + * @returns whether it is Null, narrowing on true. + * @public + */ +export function isNull<T>( + tristate: Tristate<T>, +): tristate is {readonly [TRISTATE_BRAND]: true; readonly kind: 'null'} { + return tristate.kind === 'null'; +} + +/** + * True when the key carried a value, narrowing so `.value` is reachable without a second check + * (SERDE-18). + * + * All three predicates narrow, so a caller can branch on any of them; they are not a mix of + * narrowing and plain-boolean forms. + * + * @param tristate - the value to test. + * @returns whether it is Present, narrowing on true. + * @public + */ +export function isPresent<T>(tristate: Tristate<T>): tristate is { + readonly [TRISTATE_BRAND]: true; + readonly kind: 'present'; + readonly value: T; +} { + return tristate.kind === 'present'; +} + +/** + * True when `value` was produced by this module — the codec's recognition test (SERDE-15/SERDE-19). + * + * @param value - any candidate value, typically a key encountered mid-serialization. + * @returns whether it carries this module's brand. + * @public + */ +export function isTristate(value: unknown): value is Tristate<unknown> { + return typeof value === 'object' && value !== null && TRISTATE_BRAND in value; +} + +/** + * Stable, identity-free rendering for logs and assertions (SERDE-30). + * + * @param tristate - the value to render. + * @returns `'Absent'`, `'Null'`, or `Present(` followed by the rendered value and `)`. + * @public + */ +export function tristateToString<T>(tristate: Tristate<T>): string { + return foldTristate(tristate, { + onAbsent: () => 'Absent', + onNull: () => 'Null', + onPresent: value => `Present(${String(value)})`, + }); +} diff --git a/packages/core/src/sse/errors.test.ts b/packages/core/src/sse/errors.test.ts new file mode 100644 index 0000000..48a2565 --- /dev/null +++ b/packages/core/src/sse/errors.test.ts @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/errors.test.ts +// Exercises: SSE-26/SSE-27 (loud failure on re-iteration or post-close iteration), SSE-32 (bodyless response). +import {expect, test} from 'bun:test'; +import {DexpaceError} from '../http/errors.js'; +import {SseStreamError} from './errors.js'; + +test('sits directly under DexpaceError (two-level tree)', () => { + expect(new SseStreamError('x')).toBeInstanceOf(DexpaceError); +}); + +test('name identifies the leaf in a stack trace', () => { + expect(new SseStreamError('x').name).toBe('SseStreamError'); +}); + +test('chains a cause when given one', () => { + const backing = new Error('root'); + expect(new SseStreamError('x', {cause: backing}).cause).toBe(backing); +}); diff --git a/packages/core/src/sse/errors.ts b/packages/core/src/sse/errors.ts new file mode 100644 index 0000000..4bce953 --- /dev/null +++ b/packages/core/src/sse/errors.ts @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/errors.ts +import {DexpaceError} from '../http/errors.js'; + +/** + * A misuse or precondition failure of the SSE stream facade: a second iterator on a single-pass stream + * (SSE-26), an iterator requested after close (SSE-27), or a stream opened over a response with no body + * (SSE-32). + * + * Distinct from `IoError`, which is a genuine read failure. This type always means the *caller* did something + * the contract forbids, or the *server* sent a response the contract cannot work with. + * + * @public + */ +export class SseStreamError extends DexpaceError { + // eslint-disable-next-line @typescript-eslint/no-useless-constructor -- explicit constructor required for Bun test function coverage instrumentation + constructor(message: string, options?: ErrorOptions) { + super(message, options); + } +} diff --git a/packages/core/src/sse/event.test.ts b/packages/core/src/sse/event.test.ts new file mode 100644 index 0000000..7219bd2 --- /dev/null +++ b/packages/core/src/sse/event.test.ts @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/event.test.ts +// Exercises: SSE-20 (immutable, defensively-copied data list), SSE-21 (structural equality, stable string form), +// SSE-22 (is-empty true only when all five fields are unset; a comment counts as content). +import {expect, test} from 'bun:test'; +import {InvariantViolation} from '../invariant.js'; +import { + isSseEventEmpty, + makeSseEvent, + sseEventToString, + sseEventsEqual, +} from './event.js'; + +test('the data list is defensively copied at construction (SSE-20)', () => { + const supplied = ['a', 'b']; + const event = makeSseEvent({data: supplied}); + supplied.push('c'); + expect(event.data).toEqual(['a', 'b']); +}); + +test('the event and its data list are frozen (SSE-20)', () => { + const event = makeSseEvent({data: ['a']}); + expect(Object.isFrozen(event)).toBe(true); + expect(Object.isFrozen(event.data)).toBe(true); +}); + +test('unset fields are undefined and data defaults to an empty list', () => { + const event = makeSseEvent({}); + expect(event.id).toBeUndefined(); + expect(event.event).toBeUndefined(); + expect(event.comment).toBeUndefined(); + expect(event.retryMs).toBeUndefined(); + expect(event.data).toEqual([]); +}); + +test('equality is structural over all five fields (SSE-21)', () => { + const a = makeSseEvent({ + id: '1', + event: 'ping', + data: ['x'], + comment: 'c', + retryMs: 5, + }); + const b = makeSseEvent({ + id: '1', + event: 'ping', + data: ['x'], + comment: 'c', + retryMs: 5, + }); + expect(sseEventsEqual(a, b)).toBe(true); +}); + +test('equality distinguishes present-but-empty from absent (SSE-4 seen through SSE-21)', () => { + expect(sseEventsEqual(makeSseEvent({event: ''}), makeSseEvent({}))).toBe( + false, + ); +}); + +test('equality is order-sensitive across the data list', () => { + expect( + sseEventsEqual( + makeSseEvent({data: ['a', 'b']}), + makeSseEvent({data: ['b', 'a']}), + ), + ).toBe(false); +}); + +test('the string form is stable and leaks no identity (SSE-21)', () => { + const rendered = sseEventToString(makeSseEvent({id: '1', data: ['x']})); + expect(rendered).toBe(sseEventToString(makeSseEvent({id: '1', data: ['x']}))); + expect(rendered).not.toMatch(/\[object|0x[0-9a-f]+/); +}); + +test('is-empty is true only when every field is unset (SSE-22)', () => { + expect(isSseEventEmpty(makeSseEvent({}))).toBe(true); + expect(isSseEventEmpty(makeSseEvent({data: ['']}))).toBe(false); + expect(isSseEventEmpty(makeSseEvent({event: ''}))).toBe(false); +}); + +test('a comment-only event is NOT empty — a comment counts as content (SSE-22)', () => { + expect(isSseEventEmpty(makeSseEvent({comment: 'keep-alive'}))).toBe(false); +}); + +test('a NUL-bearing id cannot be built into an event — SSE-9 drops it at the parser', () => { + expect(() => makeSseEvent({id: 'a\u0000b'})).toThrow(InvariantViolation); +}); + +test('a negative or non-integer retryMs cannot be built into an event (SSE-11)', () => { + expect(() => makeSseEvent({retryMs: -1})).toThrow(InvariantViolation); + expect(() => makeSseEvent({retryMs: 1.5})).toThrow(InvariantViolation); +}); diff --git a/packages/core/src/sse/event.ts b/packages/core/src/sse/event.ts new file mode 100644 index 0000000..0080e03 --- /dev/null +++ b/packages/core/src/sse/event.ts @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/event.ts +import {invariant} from '../invariant.js'; + +/** + * One parsed Server-Sent Event (SSE-20). + * + * A frozen plain object, not a class: it has no lifecycle, no invariant to maintain past construction, and no + * behavior — `styleguide/typescript/06` §6.3's test for a data structure rather than an object. Equality and + * rendering are therefore free functions in this module, not methods. + * + * `undefined` means the field was **absent** from the block. A field present with an empty value is `''`, and + * that distinction is load-bearing (SSE-4): an empty `event:` is a present empty event name, not a missing one. + * + * @public + */ +export interface SseEvent { + readonly id: string | undefined; + readonly event: string | undefined; + /** Raw per-line `data` values in wire order, never joined at this layer (SSE-8). */ + readonly data: readonly string[]; + readonly comment: string | undefined; + readonly retryMs: number | undefined; +} + +/** + * Fields used to construct an {@link SseEvent}. + * + * @public + */ +export interface SseEventFields { + readonly id?: string | undefined; + readonly event?: string | undefined; + readonly data?: readonly string[] | undefined; + readonly comment?: string | undefined; + readonly retryMs?: number | undefined; +} + +/** + * Construct a frozen event, defensively copying the data list so later mutation cannot reach inside (SSE-20). + * + * @throws an assertion failure (a caller bug, not a catchable condition) when a field carries a value the grammar can never produce — a NUL-bearing `id` + * (SSE-9 drops those at the parser) or a `retryMs` that is not a non-negative safe integer (SSE-11). Both are + * programmer errors, not stream conditions: the parser is the only production caller and it filters both. + * + * @public + */ +export function makeSseEvent(fields: SseEventFields): SseEvent { + // Positive and negative space on the two fields the grammar constrains: what must hold, and the impossible + // value that must be absent. + invariant( + fields.retryMs === undefined || + (Number.isSafeInteger(fields.retryMs) && fields.retryMs >= 0), + `retryMs must be a non-negative safe integer when set, got ${String(fields.retryMs)}`, + ); + invariant( + !fields.id?.includes('\u0000'), + 'an SSE id containing U+0000 must be dropped by the parser, never carried into an event (SSE-9)', + ); + + return Object.freeze({ + id: fields.id, + event: fields.event, + data: Object.freeze([...(fields.data ?? [])]), + comment: fields.comment, + retryMs: fields.retryMs, + }); +} + +/** + * Structural equality over all five fields, order-sensitive across `data` (SSE-21). + * + * @public + */ +export function sseEventsEqual(a: SseEvent, b: SseEvent): boolean { + return ( + a.id === b.id && + a.event === b.event && + a.comment === b.comment && + a.retryMs === b.retryMs && + a.data.length === b.data.length && + a.data.every((line, index) => line === b.data[index]) + ); +} + +/** + * True only when every field is unset or empty (SSE-22). + * + * A comment counts as content, so a comment-only event reports non-empty — that is the deliberate deviation from + * strict WHATWG this subsystem replicates, not an oversight. + * + * @public + */ +export function isSseEventEmpty(event: SseEvent): boolean { + return ( + event.id === undefined && + event.event === undefined && + event.comment === undefined && + event.retryMs === undefined && + event.data.length === 0 + ); +} + +/** + * Stable, identity-free rendering for logs and assertion messages (SSE-21). + * + * @public + */ +export function sseEventToString(event: SseEvent): string { + const parts = [ + `id=${event.id ?? '<absent>'}`, + `event=${event.event ?? '<absent>'}`, + `data=[${event.data.join('|')}]`, + `comment=${event.comment ?? '<absent>'}`, + `retryMs=${event.retryMs === undefined ? '<absent>' : String(event.retryMs)}`, + ]; + return `SseEvent(${parts.join(', ')})`; +} diff --git a/packages/core/src/sse/lifecycle.test.ts b/packages/core/src/sse/lifecycle.test.ts new file mode 100644 index 0000000..c352c4e --- /dev/null +++ b/packages/core/src/sse/lifecycle.test.ts @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/lifecycle.test.ts +// SSE-23: exactly one release across the stream's whole life, regardless of how it terminated. +import {expect, test} from 'bun:test'; +import {BufferedSource} from '../io/buffered-source.js'; +import {SseParser} from './parser.js'; +import {SseStream} from './stream.js'; +import {MAPPER_DONE, mapperValue, typedSseStream} from './typed.js'; + +function counted(text: string): {stream: SseStream; closes: () => number} { + let closeCount = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); + const stream = new SseStream(new SseParser(BufferedSource.overStream(web)), { + close(): Promise<void> { + closeCount += 1; + return Promise.resolve(); + }, + }); + return {stream, closes: () => closeCount}; +} + +const THREE_EVENTS = 'data: a\n\ndata: b\n\ndata: STOP\n\n'; + +test.each([ + [ + 'clean end of stream', + async (stream: SseStream) => { + for await (const event of stream) { + void event; + } + }, + ], + [ + 'explicit close with no iteration', + async (stream: SseStream) => { + await stream.close(); + }, + ], + [ + 'partial consume then explicit close', + async (stream: SseStream) => { + for await (const event of stream) { + void event; + break; + } + await stream.close(); + }, + ], + [ + 'early break alone', + async (stream: SseStream) => { + for await (const event of stream) { + void event; + break; + } + }, + ], + [ + 'consumer throws mid-iteration', + async (stream: SseStream) => { + try { + for await (const event of stream) { + void event; + throw new Error('consumer blew up'); + } + } catch { + /* expected */ + } + }, + ], + [ + 'typed mapper returns Done', + async (stream: SseStream) => { + for await (const value of typedSseStream(stream, (_n, d) => + d === 'STOP' ? MAPPER_DONE : mapperValue(d), + )) { + void value; + } + }, + ], +])('exactly one release: %s (SSE-23)', async (_name, terminate) => { + const {stream, closes} = counted(THREE_EVENTS); + await terminate(stream); + expect(closes()).toBe(1); +}); diff --git a/packages/core/src/sse/line-reader.property.test.ts b/packages/core/src/sse/line-reader.property.test.ts new file mode 100644 index 0000000..365a401 --- /dev/null +++ b/packages/core/src/sse/line-reader.property.test.ts @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/line-reader.property.test.ts +// The guarantee the carry buffer exists to provide: how bytes arrive must not change how lines come out. +import {test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSource} from '../io/buffered-source.js'; +import {SSE_END, SseLineReader} from './line-reader.js'; + +const FIXTURE = 'data: a\r\ndata: b\rdata: c\n\nid: 7\ndata: tail'; + +async function linesFromChunks( + chunks: readonly Uint8Array[], +): Promise<string[]> { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + for (const c of chunks) controller.enqueue(c); + controller.close(); + }, + }); + const reader = new SseLineReader(BufferedSource.overStream(stream)); + const lines: string[] = []; + for (;;) { + const line = await reader.nextLine(); + if (line === SSE_END) return lines; + lines.push(line); + } +} + +test('the line sequence is identical for every chunk split of the same bytes', async () => { + const all = new TextEncoder().encode(FIXTURE); + const expected = await linesFromChunks([all]); + + await fc.assert( + fc.asyncProperty( + fc.uniqueArray(fc.integer({min: 1, max: all.length - 1}), {maxLength: 4}), + async rawCuts => { + const cuts = [...rawCuts].sort((a, b) => a - b); + const chunks: Uint8Array[] = []; + let prev = 0; + for (const cut of cuts) { + chunks.push(all.slice(prev, cut)); + prev = cut; + } + chunks.push(all.slice(prev)); + const actual = await linesFromChunks(chunks); + return JSON.stringify(actual) === JSON.stringify(expected); + }, + ), + ); +}); diff --git a/packages/core/src/sse/line-reader.test.ts b/packages/core/src/sse/line-reader.test.ts new file mode 100644 index 0000000..2aae29c --- /dev/null +++ b/packages/core/src/sse/line-reader.test.ts @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/line-reader.test.ts +// Exercises: SSE-2 (LF, CR, CRLF; CRLF is one terminator; a lone CR terminates by itself), SSE-12 (one leading +// BOM consumed via lookahead, a later BOM preserved as data), SSE-14 (a final unterminated line is content), +// SSE-19 (optional line cap, off by default). +import {expect, test} from 'bun:test'; +import {BufferedSource} from '../io/buffered-source.js'; +import {SSE_END, SseLineReader, SseLineTooLongError} from './line-reader.js'; + +/** Build a BufferedSource over a byte stream delivered in the given chunks. */ +function sourceOf(chunks: readonly (string | Uint8Array)[]): BufferedSource { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + for (const chunk of chunks) { + controller.enqueue( + typeof chunk === 'string' ? new TextEncoder().encode(chunk) : chunk, + ); + } + controller.close(); + }, + }); + // 3a exposes no public constructor — `overStream` takes the ReadableStream itself and acquires the reader. + return BufferedSource.overStream(stream); +} + +async function drain(reader: SseLineReader): Promise<string[]> { + const lines: string[] = []; + for (;;) { + const line = await reader.nextLine(); + if (line === SSE_END) return lines; + lines.push(line); + } +} + +test('LF terminates a line (SSE-2)', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\nb\n'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('CRLF is a single terminator, not two (SSE-2)', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\r\nb\r\n'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('a lone CR terminates a line by itself (SSE-2)', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\rb\r'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('mixed terminators in one stream all work', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\nb\r\nc\rd\n'])))).toEqual([ + 'a', + 'b', + 'c', + 'd', + ]); +}); + +test('a CR ending one chunk and an LF starting the next is ONE terminator', async () => { + // The framing bug this reader exists to avoid: a naive splitter emits a spurious empty line here, which in + // SSE means a spurious event dispatch. + expect(await drain(new SseLineReader(sourceOf(['a\r', '\nb\n'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('a CR at the very end of the stream still terminates its line', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\r'])))).toEqual(['a']); +}); + +test('a final line with no terminator is returned as content (SSE-14)', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\nb'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('an empty stream yields no lines', async () => { + expect(await drain(new SseLineReader(sourceOf([])))).toEqual([]); +}); + +test('blank lines are preserved — they are the dispatch boundary', async () => { + expect(await drain(new SseLineReader(sourceOf(['a\n\nb\n'])))).toEqual([ + 'a', + '', + 'b', + ]); +}); + +test('one leading BOM is consumed exactly once (SSE-12)', async () => { + expect(await drain(new SseLineReader(sourceOf(['data: x\n'])))).toEqual([ + 'data: x', + ]); +}); + +test('a non-BOM prefix survives the lookahead intact (SSE-12)', async () => { + expect(await drain(new SseLineReader(sourceOf(['data: x\n'])))).toEqual([ + 'data: x', + ]); +}); + +test('a BOM later in the stream is preserved as ordinary data (SSE-12)', async () => { + expect(await drain(new SseLineReader(sourceOf(['data: ab\n'])))).toEqual([ + 'data: ab', + ]); +}); + +test('a multi-byte character split across chunks decodes correctly', async () => { + const bytes = new TextEncoder().encode('data: ü\n'); + const split = bytes.indexOf(0xc3); + expect( + await drain( + new SseLineReader( + sourceOf([bytes.slice(0, split + 1), bytes.slice(split + 1)]), + ), + ), + ).toEqual(['data: ü']); +}); + +test('no line cap applies by default (SSE-19)', async () => { + const long = 'x'.repeat(100_000); + expect(await drain(new SseLineReader(sourceOf([`${long}\n`])))).toEqual([ + long, + ]); +}); + +test('an explicit cap rejects an oversized line (SSE-19)', () => { + const reader = new SseLineReader(sourceOf(['x'.repeat(50)]), 10); + expect(reader.nextLine()).rejects.toBeInstanceOf(SseLineTooLongError); +}); + +test('the end sentinel is stable — repeated pulls past EOF keep reporting the end', async () => { + // Not a duplicate of the parser's SSE-15 test. The parser has its own `#ended` guard that would mask a reader + // which kept answering; this asserts the reader itself terminates, because a reader that returns `''` forever + // is an infinite supply of SSE dispatch boundaries, and `drain()` above would never return. + const reader = new SseLineReader(sourceOf(['a\n'])); + expect(await reader.nextLine()).toBe('a'); + expect(await reader.nextLine()).toBe(SSE_END); + expect(await reader.nextLine()).toBe(SSE_END); + expect(await reader.nextLine()).toBe(SSE_END); +}); + +test('a CRLF-terminated stream emits no trailing empty line', async () => { + // The LF of a final `\r\n` is swallowed as the second half of one terminator, leaving nothing buffered. If + // EOF-with-an-empty-buffer were treated as content rather than as the end, this would yield a phantom `''` — + // and a phantom `''` is a phantom event dispatch. + expect(await drain(new SseLineReader(sourceOf(['a\r\n'])))).toEqual(['a']); + expect(await drain(new SseLineReader(sourceOf(['a\r\nb\r\n'])))).toEqual([ + 'a', + 'b', + ]); +}); + +test('a BOM on subsequent lines is preserved as line content (SSE-12)', async () => { + expect( + await drain(new SseLineReader(sourceOf(['data: x\n\uFEFFdata: y\n']))), + ).toEqual(['data: x', '\uFEFFdata: y']); + expect( + await drain(new SseLineReader(sourceOf(['\uFEFF\uFEFFdata: x\n']))), + ).toEqual(['\uFEFFdata: x']); +}); + +test('oversized line error permanently marks the reader ended (SSE-19)', async () => { + const reader = new SseLineReader(sourceOf(['toolongline\n']), 5); + let caught: unknown; + try { + await reader.nextLine(); + } catch (e: unknown) { + caught = e; + } + expect(caught).toBeInstanceOf(SseLineTooLongError); + expect(await reader.nextLine()).toBe(SSE_END); +}); diff --git a/packages/core/src/sse/line-reader.ts b/packages/core/src/sse/line-reader.ts new file mode 100644 index 0000000..8020ee6 --- /dev/null +++ b/packages/core/src/sse/line-reader.ts @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/line-reader.ts +import {DexpaceError} from '../http/errors.js'; +import type {BufferedSource} from '../io/buffered-source.js'; +import {invariant} from '../invariant.js'; + +/** End-of-stream sentinel. A symbol, not `undefined`, so an empty line (`''`) is never mistaken for the end. @internal */ +export const SSE_END: unique symbol = Symbol('sse-end-of-stream'); + +/** + * Raised only when a caller opted into `maxLineBytes` and a line exceeded it (SSE-19). + * + * `DexpaceError`'s constructor already sets `name` from `new.target`, so no subclass restates it. + * + * @public + */ +export class SseLineTooLongError extends DexpaceError { + /** The configured cap, as a field so a log aggregator indexes it without parsing the message. */ + readonly limitBytes: number; + + constructor(limitBytes: number, options?: ErrorOptions) { + super( + `SSE line exceeded the configured maximum of ${String(limitBytes)} bytes`, + options, + ); + this.limitBytes = limitBytes; + } +} + +const LF = 0x0a; +const CR = 0x0d; + +/** + * Splits a byte stream into SSE lines (SSE-2). + * + * **Why this is not `BufferedSource.readUtf8Line()`.** Phase 3a's primitive treats `\n` and `\r\n` as + * terminators but keeps a lone `\r` as line *content* (`IO-14`). SSE-2 requires the opposite: a lone CR + * terminates a line by itself. Both contracts are normative for their own subsystem, so SSE frames its own + * lines rather than reshaping a frozen Phase 3a surface for one consumer. + * + * The awkward case is CR at a chunk boundary: a `\r` ending one read whose `\n` begins the next must resolve to + * a single terminator. That is why a pending CR is held in `#pendingCr` until the following byte — or EOF — is + * known, rather than being decided as soon as it is seen. + * + * Does **not** own or close `source` (SSE-17). Lifecycle belongs to the facade. + * + * @internal + */ +export class SseLineReader { + readonly #source: BufferedSource; + readonly #maxLineBytes: number | undefined; + readonly #decoder = new TextDecoder('utf-8', {ignoreBOM: true}); + #bomChecked = false; + #pendingCr = false; + #ended = false; + + constructor(source: BufferedSource, maxLineBytes?: number) { + invariant( + maxLineBytes === undefined || + (Number.isSafeInteger(maxLineBytes) && maxLineBytes > 0), + `maxLineBytes must be a positive safe integer when set, got ${String(maxLineBytes)}`, + ); + this.#source = source; + this.#maxLineBytes = maxLineBytes; + } + + async nextLine(): Promise<string | typeof SSE_END> { + // The end sentinel is stable at this layer too, and the guard has to be here rather than only in the + // parser: without it the EOF branch below falls through to `decode([])` on every later call and returns + // `''` forever, which is an infinite supply of blank lines — and a blank line is SSE's dispatch boundary. + if (this.#ended) return SSE_END; + + if (!this.#bomChecked) { + await this.#consumeLeadingBom(); + this.#bomChecked = true; + } + + const bytes: number[] = []; + + for (;;) { + // End of stream is detected BEFORE the read, never from its result: 3a's `readByte()` returns + // `Promise<number>` and *rejects* with `EndOfStreamError` when nothing remains (`IO-11`) — it has no + // `undefined` result to test. `exhausted()` is the sanctioned probe, and it is allowed to block waiting + // on the upstream source, which is exactly SSE-39's backpressure point. + if (await this.#source.exhausted()) { + // A held CR already terminated its own line on the previous call, so it contributes nothing here — + // in particular a stream ending `\r\n` must not emit a trailing empty line for the swallowed LF. + this.#pendingCr = false; + this.#ended = true; + // SSE-14: a final line with no terminator is returned as content; an empty tail is simply the end. + return bytes.length === 0 ? SSE_END : this.#decode(bytes); + } + + const byte = await this.#source.readByte(); + + if (this.#pendingCr) { + this.#pendingCr = false; + // The CR already terminated the previous line. An LF immediately after it is the second half of a + // CRLF and is swallowed; anything else begins this line. + if (byte === LF) continue; + } + + if (byte === LF) return this.#decode(bytes); + + if (byte === CR) { + this.#pendingCr = true; + return this.#decode(bytes); + } + + bytes.push(byte); + if ( + this.#maxLineBytes !== undefined && + bytes.length > this.#maxLineBytes + ) { + this.#ended = true; + throw new SseLineTooLongError(this.#maxLineBytes); + } + } + } + + /** + * Consume one leading UTF-8 BOM if present, leaving a non-BOM prefix untouched (SSE-12). + * + * Uses `peek()` — a non-consuming view over the same source (`IO-19`) — so the three bytes are only actually + * consumed once they are confirmed to be `EF BB BF`. + * + * Two 3a contracts shape this: `readByte()` *rejects* at end of stream rather than returning a sentinel, so a + * short stream must be probed with `exhausted()` first; and `readBytes()` takes no count (it drains + * everything), so the fixed three-byte consume is `readExactly(3)`. The view is closed on the way out — + * closing a derived view neither closes the parent nor advances its cursor (`IO-22`). + */ + async #consumeLeadingBom(): Promise<void> { + const view = this.#source.peek(); + try { + if (await view.exhausted()) return; + if ((await view.readByte()) !== 0xef) return; + if (await view.exhausted()) return; + if ((await view.readByte()) !== 0xbb) return; + if (await view.exhausted()) return; + if ((await view.readByte()) !== 0xbf) return; + } finally { + await view.close(); + } + await this.#source.readExactly(3); + } + + #decode(bytes: readonly number[]): string { + return this.#decoder.decode(new Uint8Array(bytes)); + } +} diff --git a/packages/core/src/sse/parser.property.test.ts b/packages/core/src/sse/parser.property.test.ts new file mode 100644 index 0000000..729bd18 --- /dev/null +++ b/packages/core/src/sse/parser.property.test.ts @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/parser.property.test.ts +import {test} from 'bun:test'; +import fc from 'fast-check'; +import {BufferedSource} from '../io/buffered-source.js'; +import {SSE_END} from './line-reader.js'; +import {SseParser} from './parser.js'; + +/** Text with no CR, LF, or NUL — the characters that would change framing or trigger SSE-9. */ +const safeText = fc + .stringMatching(/^[ -~]{0,20}$/) + .filter(s => !s.includes('\u0000')); + +test('serialize → parse round-trips any event with an id, event name, and data lines', async () => { + await fc.assert( + fc.asyncProperty( + safeText, + safeText, + fc.array(safeText, {maxLength: 4}), + async (id, name, data) => { + const wire = + `id: ${id}\n` + + `event: ${name}\n` + + data.map(d => `data: ${d}\n`).join('') + + '\n'; + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(wire)); + controller.close(); + }, + }); + const parser = new SseParser(BufferedSource.overStream(stream)); + const event = await parser.next(); + if (event === SSE_END) return false; + return ( + event.id === id && + event.event === name && + JSON.stringify(event.data) === JSON.stringify(data) + ); + }, + ), + ); +}); diff --git a/packages/core/src/sse/parser.test.ts b/packages/core/src/sse/parser.test.ts new file mode 100644 index 0000000..f7fab9e --- /dev/null +++ b/packages/core/src/sse/parser.test.ts @@ -0,0 +1,213 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/parser.test.ts +// Exercises: SSE-1 (blank-line dispatch, fresh accumulators), SSE-3 (first-colon split), SSE-4 (present-but-empty +// distinct from absent), SSE-5 (one leading space stripped), SSE-6 (comments), SSE-7 (unknown fields discarded), +// SSE-8 (data accumulation), SSE-9 (NUL id ignored entirely), SSE-10 (event never defaulted), SSE-11 (retry), +// SSE-13 (permissive dispatch), SSE-14 (EOF dispatch), SSE-15 (stable end sentinel), SSE-16 (no last-event-id). +import {expect, test} from 'bun:test'; +import {BufferedSource} from '../io/buffered-source.js'; +import {type SseEvent, makeSseEvent} from './event.js'; +import {SSE_END} from './line-reader.js'; +import {SseParser} from './parser.js'; + +function parserOf(text: string): SseParser { + const stream = new ReadableStream<Uint8Array>({ + start(controller) { + if (text.length > 0) { + controller.enqueue(new TextEncoder().encode(text)); + } + controller.close(); + }, + }); + return new SseParser(BufferedSource.overStream(stream)); +} + +async function eventsOf(text: string): Promise<SseEvent[]> { + const parser = parserOf(text); + const events: SseEvent[] = []; + for (;;) { + const event = await parser.next(); + if (event === SSE_END) return events; + events.push(event); + } +} + +test('a blank line dispatches exactly one event, with fresh accumulators after (SSE-1)', async () => { + const events = await eventsOf('data: 1\n\ndata: 2\n\n'); + expect(events.map(e => e.data)).toEqual([['1'], ['2']]); +}); + +test('per-event id does not carry forward (SSE-1, SSE-16)', async () => { + const events = await eventsOf('id: 1\ndata: a\n\ndata: b\n\n'); + expect(events[0]?.id).toBe('1'); + expect(events[1]?.id).toBeUndefined(); +}); + +test('a colon-less line is the whole field name with an empty value (SSE-3, SSE-4)', async () => { + expect((await eventsOf('data\n\n'))[0]?.data).toEqual(['']); +}); + +test('a trailing colon yields an empty value (SSE-3, SSE-4)', async () => { + expect((await eventsOf('data:\n\n'))[0]?.data).toEqual(['']); +}); + +test('an empty event field is present-but-empty, not absent (SSE-4)', async () => { + expect((await eventsOf('event:\ndata:x\n\n'))[0]?.event).toBe(''); +}); + +test('exactly one leading space is stripped; further spaces survive (SSE-5)', async () => { + expect((await eventsOf('data: hello\n\n'))[0]?.data).toEqual(['hello']); + expect((await eventsOf('data: hello\n\n'))[0]?.data).toEqual([' hello']); +}); + +test('a leading colon is a comment, and a comment-only block dispatches (SSE-6, SSE-13)', async () => { + const events = await eventsOf(':keep-alive\n\n'); + expect(events).toHaveLength(1); + expect(events[0]?.comment).toBe('keep-alive'); + expect(events[0]?.data).toEqual([]); +}); + +test('an unknown field sets no state and causes no dispatch (SSE-7)', async () => { + const events = await eventsOf('garbage: zzz\nevent: kept\ndata: p\n\n'); + expect(events).toHaveLength(1); + expect(events[0]?.event).toBe('kept'); + expect(events[0]?.data).toEqual(['p']); + expect(events[0]?.id).toBeUndefined(); +}); + +test('a colon-less unknown field alone dispatches nothing (SSE-7)', async () => { + expect(await eventsOf('garbage\n\n')).toEqual([]); +}); + +test('consecutive data fields accumulate in wire order, unjoined (SSE-8)', async () => { + expect((await eventsOf('data: line1\ndata: line2\n\n'))[0]?.data).toEqual([ + 'line1', + 'line2', + ]); +}); + +test('an id containing NUL is ignored entirely (SSE-9)', async () => { + expect((await eventsOf('id: a\u0000b\ndata:x\n\n'))[0]?.id).toBeUndefined(); +}); + +test('a NUL id does not overwrite a valid id from the same block (SSE-9)', async () => { + expect((await eventsOf('id: good\nid: a\u0000b\ndata:x\n\n'))[0]?.id).toBe( + 'good', + ); +}); + +test('a NUL-only block does not count as a field seen, so it dispatches nothing (SSE-9, SSE-13)', async () => { + expect(await eventsOf('id: a\u0000b\n\n')).toEqual([]); +}); + +test('an absent event field is undefined, never defaulted to "message" (SSE-10)', async () => { + expect((await eventsOf('data:x\n\n'))[0]?.event).toBeUndefined(); +}); + +test('event and id are latest-wins within a block (SSE-9, SSE-10)', async () => { + const event = ( + await eventsOf('event: a\nevent: b\nid: 1\nid: 2\ndata:x\n\n') + )[0]; + expect([event?.event, event?.id]).toEqual(['b', '2']); +}); + +test.each([ + ['retry: 5000', 5000], + ['retry: 0', 0], +])('an all-digit retry is accepted (SSE-11): %s', async (line, expected) => { + expect((await eventsOf(`${line}\ndata:x\n\n`))[0]?.retryMs).toBe(expected); +}); + +test.each([ + 'retry: bad', + 'retry: -100', + 'retry:', + 'retry: 12x', + 'retry: 1 2', + 'retry: 99999999999999999999', +])('a malformed or oversized retry is ignored (SSE-11): %s', async line => { + expect((await eventsOf(`${line}\ndata:x\n\n`))[0]?.retryMs).toBeUndefined(); +}); + +test('an id-only block dispatches (SSE-13)', async () => { + expect(await eventsOf('id: 42\n\n')).toHaveLength(1); +}); + +test('a block with no field set is skipped (SSE-13)', async () => { + expect(await eventsOf('\n\n\n')).toEqual([]); +}); + +test('EOF dispatches a pending unterminated block (SSE-14)', async () => { + const events = await eventsOf('data: hello'); + expect(events).toHaveLength(1); + expect(events[0]?.data).toEqual(['hello']); +}); + +test('an empty stream ends immediately (SSE-14)', async () => { + expect(await eventsOf('')).toEqual([]); +}); + +test('the parser never closes its source — ownership starts at the facade (SSE-17)', async () => { + let cancelled = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.close(); + }, + cancel() { + cancelled += 1; + }, + }); + const source = BufferedSource.overStream(web); + let sourceClosed = 0; + const originalClose = source.close.bind(source); + source.close = async () => { + sourceClosed += 1; + await originalClose(); + }; + + const parser = new SseParser(source); + await parser.next(); + await parser.next(); // drive to end of stream + + expect(sourceClosed).toBe(0); + expect(cancelled).toBe(0); +}); + +test('the end sentinel is stable across repeated pulls (SSE-15)', async () => { + const parser = parserOf('data: x\n\n'); + await parser.next(); + expect(await parser.next()).toBe(SSE_END); + expect(await parser.next()).toBe(SSE_END); + expect(await parser.next()).toBe(SSE_END); +}); + +test('a BOM on subsequent lines causes the line to be treated as an unknown field and discarded (SSE-7, SSE-12)', async () => { + const parser = parserOf('data: a\n\uFEFFdata: b\n\n'); + const event = await parser.next(); + expect(event).toEqual(makeSseEvent({data: ['a']})); +}); + +test('multiple comments in a single block resolve to latest-wins (SSE-6)', async () => { + const parser = parserOf(': first\n: second\n\n'); + const event = await parser.next(); + expect(event).toEqual(makeSseEvent({comment: 'second'})); +}); + +test('single leading space after colon on comment line is stripped (SSE-5, SSE-6)', async () => { + const parser = parserOf(': two spaces\n\n'); + const event = await parser.next(); + expect(event).toEqual(makeSseEvent({comment: ' two spaces'})); +}); + +test('invalid retry value does not overwrite a prior valid retry in the same block (SSE-11)', async () => { + const parser = parserOf('retry: 1000\nretry: bad\n\n'); + const event = await parser.next(); + expect(event).toEqual(makeSseEvent({retryMs: 1000})); +}); + +test('a block containing only a retry field dispatches an event (SSE-13)', async () => { + const parser = parserOf('retry: 2500\n\n'); + const event = await parser.next(); + expect(event).toEqual(makeSseEvent({retryMs: 2500})); +}); diff --git a/packages/core/src/sse/parser.ts b/packages/core/src/sse/parser.ts new file mode 100644 index 0000000..b976a4d --- /dev/null +++ b/packages/core/src/sse/parser.ts @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/parser.ts +import type {BufferedSource} from '../io/buffered-source.js'; +import {makeSseEvent, type SseEvent} from './event.js'; +import {SSE_END, SseLineReader} from './line-reader.js'; + +/** + * The documented cap for `retry` (SSE-11). + * + * `SSE-11` requires a port to pick a cap and reject beyond it "rather than wrap." In JavaScript the wrapping + * hazard is silent rounding, not overflow: past 2^53−1 an integer literal no longer round-trips, so a larger + * value would parse to a *different* number than the server sent. Reject instead. + */ +const MAX_RETRY_MS = Number.MAX_SAFE_INTEGER; + +interface BlockState { + id: string | undefined; + event: string | undefined; + data: string[]; + comment: string | undefined; + retryMs: number | undefined; + sawAnyField: boolean; +} + +function emptyBlock(): BlockState { + return { + id: undefined, + event: undefined, + data: [], + comment: undefined, + retryMs: undefined, + sawAnyField: false, + }; +} + +/** + * The WHATWG SSE line/field grammar as a state machine (SSE-1, SSE-3–SSE-16). + * + * A class rather than a generator, for two reasons the spec forces: SSE-15 requires the end sentinel to stay + * stable across repeated pulls, and SSE-16 requires exactly one piece of state (BOM-consumed) to persist while + * the last-event-id explicitly does **not**. A generator would also make SSE-17's "must not own or close the + * source" the harder thing to guarantee, since a `finally` is the natural place to clean up. Ownership is + * introduced one layer up, by the stream facade. + * + * Deliberately deviates from strict WHATWG in three ways the spec mandates replicating: comments are exposed, + * dispatch is permissive (any field set emits), and a pending block dispatches at EOF without a blank line. + * + * @internal + */ +export class SseParser { + readonly #lines: SseLineReader; + #block = emptyBlock(); + #ended = false; + + constructor( + source: BufferedSource, + options?: {maxLineBytes?: number | undefined}, + ) { + this.#lines = new SseLineReader(source, options?.maxLineBytes); + } + + async next(): Promise<SseEvent | typeof SSE_END> { + if (this.#ended) return SSE_END; + + for (;;) { + const line = await this.#lines.nextLine(); + + if (line === SSE_END) { + this.#ended = true; + // SSE-14: a pending block dispatches at EOF even with no terminating blank line. + return this.#block.sawAnyField ? this.#dispatch() : SSE_END; + } + + if (line === '') { + // SSE-1: the dispatch boundary. SSE-13: a block with no field set is skipped, not emitted. + if (this.#block.sawAnyField) return this.#dispatch(); + this.#block = emptyBlock(); + continue; + } + + this.#consumeLine(line); + } + } + + #dispatch(): SseEvent { + const block = this.#block; + this.#block = emptyBlock(); + return makeSseEvent({ + id: block.id, + event: block.event, + data: block.data, + comment: block.comment, + retryMs: block.retryMs, + }); + } + + #consumeLine(line: string): void { + if (line.startsWith(':')) { + // SSE-6: a comment. Latest-wins, and it counts as a field seen, so a comment-only block dispatches. + this.#block.comment = stripOneLeadingSpace(line.slice(1)); + this.#block.sawAnyField = true; + return; + } + + // SSE-3: split at the FIRST colon. No colon → the whole line is the name with an empty value. + const colon = line.indexOf(':'); + const name = colon === -1 ? line : line.slice(0, colon); + const rawValue = colon === -1 ? '' : line.slice(colon + 1); + const value = stripOneLeadingSpace(rawValue); + + switch (name) { + case 'data': + this.#block.data.push(value); + this.#block.sawAnyField = true; + return; + case 'event': + this.#block.event = value; + this.#block.sawAnyField = true; + return; + case 'id': + // SSE-9: an id containing NUL is ignored ENTIRELY — it does not set the id, does not count as a field + // seen, and does not overwrite a valid id already seen in this block. + if (!value.includes('\u0000')) { + this.#block.id = value; + this.#block.sawAnyField = true; + } + return; + case 'retry': { + const parsed = parseRetry(value); + if (parsed !== undefined) { + this.#block.retryMs = parsed; + this.#block.sawAnyField = true; + } + return; + } + default: + // SSE-7: any other field name is silently discarded — no state, no dispatch. + return; + } + } +} + +/** SSE-5: strip exactly one leading U+0020 if present; further leading spaces are content. */ +function stripOneLeadingSpace(value: string): string { + return value.startsWith(' ') ? value.slice(1) : value; +} + +/** SSE-11: accept only all-ASCII-digit values within the documented cap; anything else is ignored. */ +function parseRetry(value: string): number | undefined { + if (value.length === 0 || !/^[0-9]+$/.test(value)) return undefined; + const parsed = Number(value); + return Number.isSafeInteger(parsed) && parsed <= MAX_RETRY_MS + ? parsed + : undefined; +} diff --git a/packages/core/src/sse/stream.test.ts b/packages/core/src/sse/stream.test.ts new file mode 100644 index 0000000..1c361a4 --- /dev/null +++ b/packages/core/src/sse/stream.test.ts @@ -0,0 +1,594 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/stream.test.ts +// Exercises: SSE-23 (exactly one close across every termination path), SSE-24 (clean end releases), +// SSE-25 (partial consume releases), SSE-26 (single-pass), SSE-27 (post-close and mid-flight close), +// SSE-28 (idempotent close), SSE-29 (mid-stream failure releases first, close error suppressed), +// SSE-30 (auto-terminal release failure swallowed vs explicit close propagating), SSE-31 (close during a +// pending read surfaces as a read failure), SSE-32 (bodyless response), SSE-39 (no read-ahead). +import {expect, test} from 'bun:test'; +import {BufferedSource} from '../io/buffered-source.js'; +import {IoError} from '../io/errors.js'; +import {suppress, type SuppressedErrorLike} from '../suppress.js'; +import {SseStreamError} from './errors.js'; +import type {SseEvent} from './event.js'; +import {SseParser} from './parser.js'; +import {SseStream, sseStreamFrom, type SseResource} from './stream.js'; + +function streamOver( + text: string, + closeImpl?: () => Promise<void>, +): {stream: SseStream; closes: () => number} { + let closeCount = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); + const parser = new SseParser(BufferedSource.overStream(web)); + const resource = { + async close(): Promise<void> { + closeCount += 1; + if (closeImpl !== undefined) await closeImpl(); + }, + }; + return {stream: new SseStream(parser, resource), closes: () => closeCount}; +} + +test('a fully consumed stream releases without an explicit close (SSE-24)', async () => { + const {stream, closes} = streamOver('data: a\n\ndata: b\n\n'); + const seen = []; + for await (const event of stream) seen.push(event.data[0]); + expect(seen).toEqual(['a', 'b']); + expect(closes()).toBe(1); +}); + +test('a partial consume followed by close releases exactly once (SSE-25, SSE-23)', async () => { + const {stream, closes} = streamOver('data: a\n\ndata: b\n\ndata: c\n\n'); + for await (const event of stream) { + void event; + break; + } + await stream.close(); + expect(closes()).toBe(1); +}); + +test('an early break alone releases, via the iterator protocol (SSE-25)', async () => { + const {stream, closes} = streamOver('data: a\n\ndata: b\n\n'); + for await (const event of stream) { + void event; + break; + } + expect(closes()).toBe(1); +}); + +test('close is idempotent — three calls release once (SSE-28)', async () => { + const {stream, closes} = streamOver('data: a\n\n'); + await stream.close(); + await stream.close(); + await stream.close(); + expect(closes()).toBe(1); +}); + +test('close after an automatic release keeps the count at one (SSE-28)', async () => { + const {stream, closes} = streamOver('data: a\n\n'); + for await (const event of stream) { + void event; + } + await stream.close(); + expect(closes()).toBe(1); +}); + +test('the stream is single-pass — a second iterator throws (SSE-26)', () => { + const {stream} = streamOver('data: a\n\n'); + stream[Symbol.asyncIterator](); + expect(() => stream[Symbol.asyncIterator]()).toThrow(SseStreamError); +}); + +test('requesting an iterator after close throws (SSE-27)', async () => { + const {stream} = streamOver('data: a\n\n'); + await stream.close(); + expect(() => stream[Symbol.asyncIterator]()).toThrow(SseStreamError); +}); + +test('a close observed between pulls ends iteration cleanly (SSE-27, SSE-31)', async () => { + const {stream, closes} = streamOver('data: a\n\ndata: b\n\ndata: c\n\n'); + const iterator = stream[Symbol.asyncIterator](); + await iterator.next(); + await stream.close(); + expect((await iterator.next()).done).toBe(true); + expect(closes()).toBe(1); +}); + +test('an explicit close whose release fails propagates (SSE-30)', () => { + const {stream} = streamOver('data: a\n\n', () => + Promise.reject(new IoError('close failed')), + ); + expect(stream.close()).rejects.toBeInstanceOf(IoError); +}); + +test('a release failure on the clean-terminal path is swallowed, not thrown (SSE-30)', async () => { + const reported: unknown[] = []; + let closeCount = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\ndata: b\n\n')); + controller.close(); + }, + }); + const stream = new SseStream( + new SseParser(BufferedSource.overStream(web)), + { + close(): Promise<void> { + closeCount += 1; + return Promise.reject(new IoError('close failed')); + }, + }, + {onReleaseFailure: e => reported.push(e)}, + ); + + const seen = []; + for await (const event of stream) seen.push(event.data[0]); + + // Every delivered event survives; the failure is reported out-of-band instead of discarding them. + expect(seen).toEqual(['a', 'b']); + expect(reported).toHaveLength(1); + expect(closeCount).toBe(1); +}); + +test('a mid-stream read failure releases before propagating, with the close error suppressed (SSE-29)', async () => { + const readFailure = new IoError('socket reset'); + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.error(readFailure); + }, + }); + let closeCount = 0; + const closeFailure = new IoError('close failed too'); + const stream = new SseStream(new SseParser(BufferedSource.overStream(web)), { + close(): Promise<void> { + closeCount += 1; + return Promise.reject(closeFailure); + }, + }); + + let caught: unknown; + try { + for await (const event of stream) { + void event; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect(suppressed.error).toBe(readFailure); + expect(suppressed.suppressed).toBe(closeFailure); + expect(closeCount).toBe(1); +}); + +test('a release failure during an in-flight error is reported exactly once (SSE-29, SSE-30)', async () => { + const readFailure = new IoError('socket reset'); + const closeFailure = new IoError('close failed too'); + const reported: unknown[] = []; + let closeCount = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.error(readFailure); + }, + }); + const stream = new SseStream( + new SseParser(BufferedSource.overStream(web)), + { + close(): Promise<void> { + closeCount += 1; + return Promise.reject(closeFailure); + }, + }, + {onReleaseFailure: e => reported.push(e)}, + ); + + let caught: unknown; + try { + for await (const event of stream) { + void event; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.error).toBe(readFailure); + expect(suppressed.suppressed).toBe(closeFailure); + // SSE-30 scopes the hook to the automatic CLEAN terminal path. With an error already in flight the + // release failure is on the thrown error, and calling the hook as well makes one failure arrive + // twice — once in whatever logs `onReleaseFailure`, once in whatever logs the caught error. + expect(reported).toEqual([]); + expect(closeCount).toBe(1); +}); + +test('sseStreamFrom binds lifecycle to the response body (SSE-32)', async () => { + let responseClosed = 0; + const response = { + body: new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.close(); + }, + }), + close(): Promise<void> { + responseClosed += 1; + return Promise.resolve(); + }, + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + for await (const event of sseStreamFrom(response)) { + void event; + } + expect(responseClosed).toBe(1); +}); + +test('the disposal member releases exactly once where the runtime has it (styleguide 13.1/13.2)', async () => { + const {stream, closes} = streamOver('data: a\n\ndata: b\n\n'); + const asyncDispose = (Symbol as {asyncDispose?: symbol}).asyncDispose; + + if (typeof asyncDispose !== 'symbol') { + // The pinned runtime floor (Node 20.3) predates Symbol.asyncDispose (Node 20.4). + await stream.close(); + expect(closes()).toBe(1); + return; + } + + const dispose = ( + stream as unknown as Record<symbol, (() => Promise<void>) | undefined> + )[asyncDispose]; + expect(dispose).toBeDefined(); + + for await (const event of stream) { + void event; + break; + } + await dispose?.call(stream); + expect(closes()).toBe(1); + + // Dispose delegates to close, so it inherits close's idempotence rather than adding a second guard. + await stream.close(); + expect(closes()).toBe(1); +}); + +test('aborting the signal closes the stream, ending an idle iterator cleanly (SSE-25, SSE-27)', async () => { + let responseClosed = 0; + // The abort listener deliberately discards its close promise, so the test needs its own completion signal + // rather than a timer — `new Promise` with a synchronous executor adapting a callback is the sanctioned form. + let markClosed = (): void => undefined; + const closed = new Promise<void>(resolve => { + markClosed = resolve; + }); + + const controller = new AbortController(); + const response = { + body: new ReadableStream<Uint8Array>({ + start(streamController) { + streamController.enqueue( + new TextEncoder().encode('data: a\n\ndata: b\n\n'), + ); + streamController.close(); + }, + }), + close(): Promise<void> { + responseClosed += 1; + markClosed(); + return Promise.resolve(); + }, + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + const stream = sseStreamFrom(response, {signal: controller.signal}); + const iterator = stream[Symbol.asyncIterator](); + const first = await iterator.next(); + expect((first.value as SseEvent | undefined)?.data).toEqual(['a']); + + controller.abort(); + await closed; + + expect(responseClosed).toBe(1); + expect((await iterator.next()).done).toBe(true); +}); + +test('sseStreamFrom releases the byte source as well as the response (SSE-23, SSE-32)', async () => { + // docs/knowledge/harvested/sse-streaming.md:84 — the facade's release must reach `response.body.cancel()` exactly once. + // The BufferedSource holds the reader lock on that body, so unless the facade closes the *source*, a real + // Response.close() would be cancelling a locked stream. A close-counting double cannot catch this; asserting + // the body's own cancel hook fired is what does. + let bodyCancelled = 0; + let responseClosed = 0; + const response = { + body: new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + }, + cancel() { + bodyCancelled += 1; + }, + }), + close(): Promise<void> { + responseClosed += 1; + return Promise.resolve(); + }, + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + const stream = sseStreamFrom(response); + for await (const event of stream) { + void event; + break; + } + await stream.close(); + + expect(bodyCancelled).toBe(1); + expect(responseClosed).toBe(1); +}); + +test('sseStreamFrom fails loudly on a bodyless response (SSE-32)', () => { + const response = { + body: null, + close: () => Promise.resolve(), + } as unknown as Parameters<typeof sseStreamFrom>[0]; + expect(() => sseStreamFrom(response)).toThrow(SseStreamError); +}); + +test('delivery is pull-based: no event is parsed before the consumer asks (SSE-39)', async () => { + let delivered = 0; + const web = new ReadableStream<Uint8Array>( + { + pull(controller) { + delivered += 1; + if (delivered > 3) { + controller.close(); + return; + } + controller.enqueue( + new TextEncoder().encode(`data: ${String(delivered)}\n\n`), + ); + }, + }, + {highWaterMark: 0}, + ); + const stream = new SseStream(new SseParser(BufferedSource.overStream(web)), { + close: () => Promise.resolve(), + }); + + const iterator = stream[Symbol.asyncIterator](); + const before = delivered; + await iterator.next(); + // One consumer pull draws at most one source chunk beyond whatever the stream had already buffered. + expect(delivered - before).toBeLessThanOrEqual(1); + await stream.close(); +}); + +test('close during a pending read surfaces as an IoError, releasing exactly once (SSE-31)', async () => { + let closeCount = 0; + let controllerRef: ReadableStreamDefaultController<Uint8Array> | undefined; + + // A source that delivers one event and then never resolves again — so the second pull is genuinely pending + // when the close lands, rather than racing a queued chunk. + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controllerRef = controller; + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + }, + }); + const source = BufferedSource.overStream(web); + const stream = new SseStream(new SseParser(source), { + async close(): Promise<void> { + closeCount += 1; + await source.close(); + }, + }); + + const iterator = stream[Symbol.asyncIterator](); + const first = await iterator.next(); + expect((first.value as SseEvent | undefined)?.data).toEqual(['a']); + + const pending = iterator.next(); + await stream.close(); + + expect(pending).rejects.toBeInstanceOf(IoError); + expect(closeCount).toBe(1); + expect(controllerRef).toBeDefined(); +}); + +test('normal stream close removes the abort event listener from the signal', async () => { + const controller = new AbortController(); + let addCount = 0; + let removeCount = 0; + const originalAdd = controller.signal.addEventListener.bind( + controller.signal, + ); + const originalRemove = controller.signal.removeEventListener.bind( + controller.signal, + ); + controller.signal.addEventListener = ( + type: string, + listener: EventListenerOrEventListenerObject, + options?: boolean | AddEventListenerOptions, + ): void => { + if (type === 'abort') addCount++; + originalAdd(type, listener, options); + }; + controller.signal.removeEventListener = ( + type: string, + listener: EventListenerOrEventListenerObject, + options?: boolean | EventListenerOptions, + ): void => { + if (type === 'abort') removeCount++; + originalRemove(type, listener, options); + }; + + const response = { + body: new ReadableStream<Uint8Array>({ + start(streamController) { + streamController.enqueue(new TextEncoder().encode('data: hello\n\n')); + streamController.close(); + }, + }), + close: () => Promise.resolve(), + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + const stream = sseStreamFrom(response, {signal: controller.signal}); + expect(addCount).toBe(1); + expect(removeCount).toBe(0); + + for await (const event of stream) { + void event; + } + expect(removeCount).toBe(1); +}); + +test('close() awaits in-flight quiet release and propagates any release error (SSE-30)', async () => { + let releaseStarted = false; + let releaseFinished = false; + let finishRelease: (err?: Error) => void = (): void => undefined; + const releasePromise = new Promise<void>((resolve, reject) => { + finishRelease = (err?: Error): void => { + releaseFinished = true; + if (err) reject(err); + else resolve(); + }; + }); + + const resource: SseResource = { + close(): Promise<void> { + releaseStarted = true; + return releasePromise; + }, + }; + + const stream = new SseStream( + new SseParser( + BufferedSource.overStream( + new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: 1\n\n')); + controller.close(); + }, + }), + ), + ), + resource, + ); + + const iterator = stream[Symbol.asyncIterator](); + const first = await iterator.next(); + expect((first.value as SseEvent | undefined)?.data).toEqual(['1']); + + // Next pull drives past EOF into #releaseQuietly, which awaits releasePromise + const pendingNext = iterator.next(); + // Microtask tick to ensure #releaseQuietly is entered + await new Promise(r => setTimeout(r, 5)); + expect(releaseStarted).toBe(true); + expect(releaseFinished).toBe(false); + + // Calling close() while release is in flight awaits that same promise and propagates its error + const closePromise = stream.close(); + const testError = new Error('teardown failed'); + finishRelease(testError); + + expect(closePromise).rejects.toThrow(testError); + await pendingNext; + expect(releaseFinished).toBe(true); +}); + +test('closingBoth attaches response close failure as suppressed when source close also fails', async () => { + const sourceError = new Error('source failed'); + const responseError = new Error('response failed'); + + const failingSource = { + close() { + return Promise.reject(sourceError); + }, + } as unknown as BufferedSource; + + const failingResponse = { + body: new ReadableStream<Uint8Array>({ + start(c) { + c.close(); + }, + }), + close() { + return Promise.reject(responseError); + }, + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + const stream = new SseStream( + new SseParser( + BufferedSource.overStream( + new ReadableStream<Uint8Array>({ + start(c) { + c.close(); + }, + }), + ), + ), + { + async close(): Promise<void> { + let sourceFailure: unknown; + let sourceFailed = false; + try { + await failingSource.close(); + } catch (e: unknown) { + sourceFailure = e; + sourceFailed = true; + } + try { + await failingResponse.close(); + } catch (responseFailure: unknown) { + if (sourceFailed) { + throw suppress(sourceFailure, responseFailure, 'both failed'); + } + throw responseFailure; + } + if (sourceFailed) throw sourceFailure; + }, + }, + ); + + let caught: unknown; + try { + await stream.close(); + } catch (e: unknown) { + caught = e; + } + expect((caught as SuppressedErrorLike).error).toBe(sourceError); + expect((caught as SuppressedErrorLike).suppressed).toBe(responseError); +}); + +test('bindAbort routes release failure to onReleaseFailure', async () => { + let releaseFailure: unknown; + const controller = new AbortController(); + const closeError = new Error('abort close failed'); + + const response = { + body: new ReadableStream<Uint8Array>({ + start(c) { + c.enqueue(new TextEncoder().encode('data: 1\n\n')); + }, + }), + close: () => Promise.reject(closeError), + } as unknown as Parameters<typeof sseStreamFrom>[0]; + + const stream = sseStreamFrom(response, { + signal: controller.signal, + onReleaseFailure: err => { + releaseFailure = err; + }, + }); + void stream; + + controller.abort(); + // Allow microtasks to settle + await new Promise(r => setTimeout(r, 10)); + + expect(releaseFailure).toBe(closeError); +}); diff --git a/packages/core/src/sse/stream.ts b/packages/core/src/sse/stream.ts new file mode 100644 index 0000000..919827e --- /dev/null +++ b/packages/core/src/sse/stream.ts @@ -0,0 +1,342 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/stream.ts +import type {Response} from '../http/response.js'; +import {BufferedSource} from '../io/buffered-source.js'; +import {IoError} from '../io/errors.js'; +import {suppress} from '../suppress.js'; +import type {SseEvent} from './event.js'; +import {SseStreamError} from './errors.js'; +import {SSE_END} from './line-reader.js'; +import {SseParser} from './parser.js'; + +/** + * Anything the facade can own and release exactly once. + * + * @internal + */ +export interface SseResource { + close(): Promise<void>; +} + +/** + * Options for configuring an {@link (SseStream:class)}. + * + * @public + */ +export interface SseStreamOptions { + /** + * Called when a release fails on a clean automatic terminal path, where SSE-30 requires the failure to be + * reported out-of-band and swallowed rather than thrown (throwing would discard events already delivered). + * + * Called on **that** path only, which is SSE-30's own scope: "no error in flight". A release that fails + * while an error is already propagating is attached to that error as `suppressed` instead, and a release + * that fails during an explicit `close()` rejects that call — reporting either one here as well would + * deliver a single failure twice. + * + * Defaults to a no-op. Phase 7 wires a real `Logger` in here without reshaping this class — the same + * "mechanism now, wiring later" split Phase 3b used for its logging tees. + */ + readonly onReleaseFailure?: ((error: unknown) => void) | undefined; +} + +/** + * Options for opening an SSE stream from a Response. + * + * @public + */ +export interface SseStreamFromOptions extends SseStreamOptions { + /** Opt-in line cap (SSE-19). Off by default, matching the reference's own absence of a cap. */ + readonly maxLineBytes?: number | undefined; + + /** + * Cancellation for this long-running operation + * (`docs/knowledge/harvested/concurrency-and-async.md:18`, `docs/knowledge/harvested/api-design.md:34`). + * + * Aborting closes the stream, which is the only cancellation a pull-based reader needs: an iterator sitting + * *between* pulls then ends cleanly (SSE-27) and one blocked *in* a read surfaces an `IoError` (SSE-31). + * Both paths release the owned resource exactly once, so this adds a trigger rather than a code path. + */ + readonly signal?: AbortSignal | undefined; +} + +/** + * A single-pass, resource-owning view over a parsed SSE byte stream (SSE-23–SSE-32). + * + * Owns exactly one closeable resource and releases it exactly once across every termination path: clean + * end-of-stream, explicit `close()`, early `break`, a mid-stream read failure, or a typed mapper's Done. + * + * @public + */ +export class SseStream implements AsyncIterable<SseEvent> { + readonly #parser: SseParser; + readonly #resource: SseResource; + readonly #onReleaseFailure: (error: unknown) => void; + #iteratorTaken = false; + #closed = false; + #closing: Promise<void> | undefined; + + /** @internal */ + constructor( + parser: SseParser, + resource: SseResource, + options?: SseStreamOptions, + ) { + this.#parser = parser; + this.#resource = resource; + this.#onReleaseFailure = options?.onReleaseFailure ?? (() => undefined); + } + + /** + * The stream's one iterator (SSE-26). + * + * @throws SseStreamError when an iterator was already taken, or when the stream is already closed — both are + * caller-contract violations, not stream conditions, so neither is recoverable by retrying. + * @throws IoError from a pull, when the source fails mid-stream or is torn down under an in-flight read + * (SSE-29 / SSE-31). The resource is released before either reaches the consumer. + */ + [Symbol.asyncIterator](): AsyncIterator<SseEvent> { + if (this.#closed) { + throw new SseStreamError( + 'cannot iterate an SSE stream that has already been closed', + ); + } + if (this.#iteratorTaken) { + throw new SseStreamError( + 'an SSE stream is single-pass; its iterator may be obtained at most once', + ); + } + this.#iteratorTaken = true; + return this.#iterate(); + } + + /** + * Release the owned resource. Idempotent (SSE-28): only the first call reaches the resource, and that holds + * even after an automatic release on a terminal or failure path. + * + * A release failure here **propagates** — the caller asked for the close, so the caller hears about it. That + * is the opposite of the automatic path, and the split is SSE-30's actual portable contract. + * + * @throws IoError when releasing the owned resource fails. Nothing is left to retry: the release is marked + * done either way, so a second `close()` is a no-op rather than a second attempt. + */ + async close(): Promise<void> { + this.#closed = true; + this.#closing ??= this.#resource.close(); + return this.#closing; + } + + async *#iterate(): AsyncGenerator<SseEvent> { + // Whether the catch below already released and already accounted for a release failure. A local, + // not a field: it is read exactly once, by the `finally` of this one generator activation. + let releasedWithError = false; + try { + for (;;) { + // A close observed between pulls ends iteration cleanly, without reading from a torn-down resource. + if (this.#closed) return; + const event = await this.#pullNext(); + if (event === SSE_END) return; + yield event; + } + } catch (e: unknown) { + // SSE-29: release BEFORE the error propagates, and attach a release failure as suppressed rather than + // letting it mask the real cause. + releasedWithError = true; + await this.#releaseWithInFlightError(e); + } finally { + // Covers clean end-of-stream and early `break` (the runtime calls `.return()`, which runs this block). + // + // Skipped after the catch, which has already released. Running it there awaited the same rejected + // `#closing` promise and handed the close failure to `onReleaseFailure` as well — so one failure was + // reported twice, once out-of-band and once as `suppressed` on the error the consumer catches. SSE-30 + // scopes the hook to the automatic CLEAN terminal path, where there is nothing to throw to; with an + // error in flight there is (audit #67 / #79). + if (!releasedWithError) await this.#releaseQuietly(); + } + } + + async #pullNext(): Promise<SseEvent | typeof SSE_END> { + try { + return await this.#parser.next(); + } catch (e: unknown) { + // SSE-31: a close that tears the source down while this read was in flight surfaces here. Web + // Streams rejects a pending read with a bare TypeError when its reader's lock is released; map it so + // callers see one failure shape rather than a platform-specific type. + if (this.#closed && !(e instanceof IoError)) { + throw new IoError( + 'the SSE source was closed while a read was in flight', + {cause: e}, + ); + } + throw e; + } + } + + /** SSE-30's automatic clean-terminal path: a failing release is reported out-of-band and swallowed. */ + async #releaseQuietly(): Promise<void> { + this.#closed = true; + if (this.#closing !== undefined) { + try { + await this.#closing; + } catch (e: unknown) { + this.#onReleaseFailure(e); + } + return; + } + const releasePromise = this.#resource.close(); + this.#closing = releasePromise; + try { + await releasePromise; + } catch (e: unknown) { + this.#onReleaseFailure(e); + } + } + + /** + * SSE-29 / SSE-36: an error is already in flight, so it stays primary and the close error is suppressed — + * and NOT also handed to `onReleaseFailure`, which is the clean-terminal path's channel. + */ + async #releaseWithInFlightError(primary: unknown): Promise<never> { + this.#closed = true; + const releasePromise = (this.#closing ??= this.#resource.close()); + try { + await releasePromise; + } catch (closeError: unknown) { + throw suppress( + primary, + closeError, + 'the SSE stream failed and its release also failed', + ); + } + throw primary; + } +} + +// Scoped teardown for `await using` (styleguide 13.1/13.2), installed at run time only when the symbol +// exists. This is the original of the shape `Page` and both transport adapters now repeat. +// +// DO NOT restore this as a plain `[Symbol.asyncDispose]()` class member. Node 20.3 is this package's +// declared floor (`engines.node`, checked by verify:runtime-floor) and predates the symbol, which +// arrived in 20.4. On the floor the computed key evaluates to `undefined` and binds the method to the +// string key `"undefined"` — a junk prototype entry, and no working disposal. TypeScript does not +// polyfill the well-known symbol either, so declaring it on the class would emit it into the `.d.ts` +// unconditionally and break consumers compiling on ES2023 without esnext.disposable. +// +// `Response` (HTTP-38) goes one step further and ships no disposal member at all — `close()` is its +// whole teardown surface, and `http/response.test.ts` pins the junk key's absence there. +if (typeof Symbol.asyncDispose === 'symbol') { + Object.defineProperty(SseStream.prototype, Symbol.asyncDispose, { + value: function asyncDispose(this: SseStream): Promise<void> { + return this.close(); + }, + writable: true, + configurable: true, + }); +} + +/** + * Open an SSE stream over an HTTP response, binding the stream's lifecycle to the response (SSE-32). + * + * Closing the stream closes the response. A response with no body fails loudly rather than yielding an empty + * stream: a bodyless SSE response is a server or caller mistake, and silently producing zero events would hide + * it behind a successful-looking loop that does nothing. + * + * @throws SseStreamError when the response has no body (SSE-32). + * + * @public + */ +export function sseStreamFrom( + response: Response, + options?: SseStreamFromOptions, +): SseStream { + const body = response.body; + if (body === null) { + throw new SseStreamError( + 'cannot open an SSE stream over a response with no body', + ); + } + const source = BufferedSource.overStream(body); + const parser = new SseParser(source, {maxLineBytes: options?.maxLineBytes}); + const unbind = {fn: (): void => undefined}; + const resource: SseResource = { + async close(): Promise<void> { + unbind.fn(); + await closingBoth(source, response).close(); + }, + }; + const stream = new SseStream(parser, resource, options); + unbind.fn = bindAbort(stream, options?.signal, options?.onReleaseFailure); + return stream; +} + +/** + * Make an abort close the stream. + * + * This lives here rather than in `SseStream`'s constructor because a constructor may only assign its arguments + * to fields — no branching, no listener registration (`docs/knowledge/harvested/data-modeling.md:24`). The listener is + * registered with `{once: true}` and removed upon close, and the close promise is explicitly + * discarded with `void` plus a `.catch`, because an unhandled rejection on this path would take the process + * down under Node's default `unhandledRejection` policy + * (`docs/knowledge/harvested/cancellation-and-timeouts.md:26`). + */ +function bindAbort( + stream: SseStream, + signal: AbortSignal | undefined, + onReleaseFailure?: (error: unknown) => void, +): () => void { + if (signal === undefined) return () => undefined; + + const release = (): void => { + void stream.close().catch((error: unknown) => { + onReleaseFailure?.(error); + }); + }; + + if (signal.aborted) { + release(); + return () => undefined; + } + signal.addEventListener('abort', release, {once: true}); + return () => { + signal.removeEventListener('abort', release); + }; +} + +/** + * Bundle the two things this function acquired into the **one** resource SSE-23 says the facade owns. + * + * Passing the bare `response` here would leak the `BufferedSource` — and worse than leak it: the source holds a + * reader lock on `response.body`, and cancelling a `ReadableStream` that still has a locked reader throws + * `TypeError`, so `response.close()` would fail on a real `Response` while passing happily against a + * close-counting test double. Release order is reverse acquisition (source first, then response), per + * `styleguide/typescript/13` §13.5. + * + * Both closes always run — one failing must not skip the other — and if both fail the first stays primary with + * the second attached as suppressed, matching every other release path in Phase 6. + */ +function closingBoth(source: BufferedSource, response: Response): SseResource { + return { + async close(): Promise<void> { + let sourceFailure: unknown; + let sourceFailed = false; + try { + await source.close(); + } catch (e: unknown) { + sourceFailure = e; + sourceFailed = true; + } + try { + await response.close(); + } catch (responseFailure: unknown) { + if (sourceFailed) { + throw suppress( + sourceFailure, + responseFailure, + 'releasing the SSE source failed and releasing the response also failed', + ); + } + throw responseFailure; + } + if (sourceFailed) throw sourceFailure; + }, + }; +} diff --git a/packages/core/src/sse/typed.test.ts b/packages/core/src/sse/typed.test.ts new file mode 100644 index 0000000..ad303a0 --- /dev/null +++ b/packages/core/src/sse/typed.test.ts @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/typed.test.ts +// Exercises: SSE-33 (mapper receives event name + newline-joined data), SSE-34 (Value/Skip/Done honored), +// SSE-35 (lazy per-element decoding), SSE-36 (a throwing mapper releases the resource before propagating). +import {expect, test} from 'bun:test'; +import {BufferedSource} from '../io/buffered-source.js'; +import {IoError} from '../io/errors.js'; +import type {SuppressedErrorLike} from '../suppress.js'; +import {SseParser} from './parser.js'; +import {SseStream} from './stream.js'; +import { + MAPPER_DONE, + MAPPER_SKIP, + mapperValue, + typedSseStream, +} from './typed.js'; + +function streamOver(text: string): {stream: SseStream; closes: () => number} { + let closeCount = 0; + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); + const stream = new SseStream(new SseParser(BufferedSource.overStream(web)), { + close(): Promise<void> { + closeCount += 1; + return Promise.resolve(); + }, + }); + return {stream, closes: () => closeCount}; +} + +test('the mapper receives the raw event name and newline-joined data (SSE-33)', async () => { + const seen: [string | undefined, string][] = []; + const {stream} = streamOver('event: ping\ndata: l1\ndata: l2\n\n'); + for await (const value of typedSseStream(stream, (name, data) => { + seen.push([name, data]); + return mapperValue(1); + })) { + void value; + } + expect(seen).toEqual([['ping', 'l1\nl2']]); +}); + +test('a no-data event joins to the empty string, and an absent name stays undefined (SSE-33)', async () => { + const seen: [string | undefined, string][] = []; + const {stream} = streamOver('id: 1\n\n'); + for await (const value of typedSseStream(stream, (name, data) => { + seen.push([name, data]); + return MAPPER_SKIP; + })) { + void value; + } + expect(seen).toEqual([[undefined, '']]); +}); + +test('Value is yielded, Skip is dropped silently (SSE-34)', async () => { + const {stream} = streamOver('data: keep\n\ndata: drop\n\ndata: keep2\n\n'); + const out: string[] = []; + for await (const value of typedSseStream(stream, (_name, data) => + data === 'drop' ? MAPPER_SKIP : mapperValue(data), + )) { + out.push(value); + } + expect(out).toEqual(['keep', 'keep2']); +}); + +test('Done ends iteration cleanly, closes, and yields nothing for the sentinel (SSE-34)', async () => { + const {stream, closes} = streamOver( + 'data: a\n\ndata: STOP\n\ndata: never\n\n', + ); + const out: string[] = []; + for await (const value of typedSseStream(stream, (_name, data) => + data === 'STOP' ? MAPPER_DONE : mapperValue(data), + )) { + out.push(value); + } + expect(out).toEqual(['a']); + expect(closes()).toBe(1); +}); + +test('post-sentinel events are never decoded (SSE-34)', async () => { + let calls = 0; + const {stream} = streamOver( + 'data: a\n\ndata: STOP\n\ndata: never\n\ndata: also-never\n\n', + ); + for await (const value of typedSseStream(stream, (_name, data) => { + calls += 1; + return data === 'STOP' ? MAPPER_DONE : mapperValue(data); + })) { + void value; + } + expect(calls).toBe(2); +}); + +test('decoding is lazy and per-element (SSE-35)', async () => { + let decodes = 0; + const {stream} = streamOver('data: a\n\ndata: b\n\ndata: c\n\n'); + const iterator = typedSseStream(stream, (_name, data) => { + decodes += 1; + return mapperValue(data); + })[Symbol.asyncIterator](); + + await iterator.next(); + expect(decodes).toBe(1); + await iterator.next(); + expect(decodes).toBe(2); +}); + +test('a throwing mapper releases the resource before the error reaches the consumer (SSE-36)', async () => { + const boom = new Error('bad payload'); + const {stream, closes} = streamOver('data: a\n\ndata: b\n\n'); + let caught: unknown; + try { + for await (const value of typedSseStream(stream, (_name, data) => { + if (data === 'b') throw boom; + return mapperValue(data); + })) { + void value; + } + } catch (e: unknown) { + caught = e; + } + expect(caught).toBe(boom); + expect(closes()).toBe(1); +}); + +test('a release failure while a mapper error is in flight is attached as suppressed (SSE-36)', async () => { + const boom = new Error('bad payload'); + const closeFailure = new IoError('close failed'); + const web = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.close(); + }, + }); + const stream = new SseStream(new SseParser(BufferedSource.overStream(web)), { + close: () => Promise.reject(closeFailure), + }); + + let caught: unknown; + try { + for await (const value of typedSseStream(stream, () => { + throw boom; + })) { + void value; + } + } catch (e: unknown) { + caught = e; + } + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect(suppressed.error).toBe(boom); + expect(suppressed.suppressed).toBe(closeFailure); +}); diff --git a/packages/core/src/sse/typed.ts b/packages/core/src/sse/typed.ts new file mode 100644 index 0000000..fb3fda1 --- /dev/null +++ b/packages/core/src/sse/typed.ts @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/sse/typed.ts +import {assertNever} from '../invariant.js'; +import {suppress} from '../suppress.js'; +import type {SseEvent} from './event.js'; +import type {SseStream} from './stream.js'; + +/** + * A mapper's three outcomes (SSE-34): yield a decoded value, silently drop the event, or end the stream. + * + * **A sibling of Phase 4b's `Outcome<T>`, not a third variant on it.** `Outcome<T>` is a two-branch + * success/failure union threaded through the recovery chain; widening it with `skip`/`done` would force every + * existing `fold` call site in `src/recovery/` to handle variants that can never occur there. What + * `sdk-design-nodejs/07` §7.2 argues for reusing is the *idiom* — a `kind`-discriminated union over frozen + * literals — and that is exactly what this is. + * + * @public + */ +export type MapperOutcome<T> = + | {readonly kind: 'value'; readonly value: T} + | {readonly kind: 'skip'} + | {readonly kind: 'done'}; + +/** + * Yield this event's decoded value to the consumer. + * + * @public + */ +export function mapperValue<T>(value: T): MapperOutcome<T> { + return Object.freeze({kind: 'value', value} as const); +} + +/** + * Drop this event and advance. It never surfaces to the consumer — keep-alives and comments live here. + * + * @public + */ +export const MAPPER_SKIP: MapperOutcome<never> = Object.freeze({ + kind: 'skip', +} as const); + +/** + * End iteration cleanly and close the stream, yielding no model for the sentinel event itself. + * + * @public + */ +export const MAPPER_DONE: MapperOutcome<never> = Object.freeze({ + kind: 'done', +} as const); + +/** + * Decodes a raw event into a caller model (SSE-33). + * + * `eventName` is the raw `event` field, `undefined` when the server omitted it — never defaulted to + * `'message'`. `joinedData` is the event's data lines joined with a single `\n`, or `''` when the event carried + * no data. The parser deliberately does not join (SSE-8); joining is this layer's job. + * + * @public + */ +export type SseMapper<T> = ( + eventName: string | undefined, + joinedData: string, +) => MapperOutcome<T>; + +/** + * Lazily decode an {@link (SseStream:class)} into caller models (SSE-33–SSE-36). + * + * Decoding is per-element: the mapper runs inside the loop body, so a consumer taking one element decodes + * exactly one event. Skips drain inside the same pull, which is the one exception SSE-39 sanctions to its 1:1 + * polling rule — "only as many as needed to produce one element." + * + * A throwing mapper propagates to the consumer's pull, but only after the underlying resource is released, with + * a release failure attached as suppressed — see `runMapper`, which owns that path. + * + * @throws SseStreamError when `stream` has already been iterated or closed — the underlying facade is + * single-pass (SSE-26/SSE-27), and this adapter takes its one iterator. + * + * @public + */ +export function typedSseStream<T>( + stream: SseStream, + mapper: SseMapper<T>, +): AsyncIterable<T> { + return { + async *[Symbol.asyncIterator](): AsyncGenerator<T> { + for await (const event of stream) { + const outcome = await runMapper(stream, mapper, event); + switch (outcome.kind) { + case 'value': + yield outcome.value; + break; + case 'skip': + break; + case 'done': + return; + default: + return assertNever(outcome); + } + } + }, + }; +} + +/** + * Run the mapper for one event, honoring SSE-36 when it throws: release first, then propagate, with a release + * failure attached to the mapper's error as suppressed. + * + * **This cannot be left to the facade,** which is what an earlier draft assumed. The facade's `catch` only sees + * failures raised by *its own* pull of the parser. A throw from this loop's body is not that: it unwinds by + * calling the facade iterator's `return()`, which runs the facade's *quiet* release path — the one SSE-30 + * requires to swallow a close failure. So on that route the close error would be swallowed instead of attached, + * which is precisely what SSE-36 forbids. Releasing here, through the facade's public `close()`, gets the + * explicit-close semantics this case needs; the facade's later `return()` then finds the resource already + * released and does nothing (SSE-28). + */ +async function runMapper<T>( + stream: SseStream, + mapper: SseMapper<T>, + event: SseEvent, +): Promise<MapperOutcome<T>> { + try { + return mapper(event.event, event.data.join('\n')); + } catch (mapperError: unknown) { + try { + await stream.close(); + } catch (closeError: unknown) { + throw suppress( + mapperError, + closeError, + 'an SSE mapper failed and releasing the stream also failed', + ); + } + throw mapperError; + } +} diff --git a/packages/core/src/suppress.test.ts b/packages/core/src/suppress.test.ts new file mode 100644 index 0000000..c697c5f --- /dev/null +++ b/packages/core/src/suppress.test.ts @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/suppress.test.ts +// Exercises: the runtime-guarded stand-in for `SuppressedError` that RECOV-12 (and, later, Phases +// 5a/6a/6b/6c) need to attach a teardown failure to a primary throwable without inverting their +// priority. +// +// Neither branch of the guard is forced here by mutating `globalThis` — a test that deletes a +// global does not survive parallel execution, which docs/knowledge/harvested/testing.md:50 requires. The +// branch selection is covered where it is real instead: `suppress()` is asserted on its shape, +// which holds on either runtime, `FallbackSuppressedError` is constructed directly, and the +// `test:node` matrix runs both legs — `lts/*` has the native class, the pinned `20.3.0` floor does +// not. +import {describe, expect, test} from 'bun:test'; +import {expectTypeOf} from 'expect-type'; +import { + FallbackSuppressedError, + suppress, + type SuppressedErrorLike, +} from './suppress.js'; + +describe('suppress', () => { + test('keeps the primary error primary and the secondary suppressed', () => { + const primary = new Error('primary'); + const secondary = new Error('secondary'); + + const result = suppress(primary, secondary, 'teardown failed'); + + expect(result.error).toBe(primary); + expect(result.suppressed).toBe(secondary); + expect(result.message).toBe('teardown failed'); + }); + + test('reports the same identity on either branch of the guard', () => { + const result = suppress( + new Error('primary'), + new Error('secondary'), + 'msg', + ); + + expect(result).toBeInstanceOf(Error); + expect(result.name).toBe('SuppressedError'); + }); + + test('carries non-Error throwables unchanged — a JS throw can raise any value', () => { + const result = suppress('a string throw', undefined, 'teardown failed'); + + expect(result.error).toBe('a string throw'); + expect(result.suppressed).toBeUndefined(); + }); + + test('uses the native SuppressedError when the runtime provides one', () => { + const native = (globalThis as {SuppressedError?: unknown}).SuppressedError; + if (typeof native !== 'function') return; // the floor runtime has no native class to use + + const result = suppress(new Error('a'), new Error('b'), 'msg'); + + expect(result).toBeInstanceOf(native); + }); +}); + +describe('FallbackSuppressedError — the branch the declared floor takes', () => { + test('mirrors the native shape rather than reporting its own class name', () => { + const primary = new Error('primary'); + const secondary = new Error('secondary'); + + const result = new FallbackSuppressedError( + primary, + secondary, + 'teardown failed', + ); + + expect(result).toBeInstanceOf(Error); + expect(result.name).toBe('SuppressedError'); + expect(result.error).toBe(primary); + expect(result.suppressed).toBe(secondary); + expect(result.message).toBe('teardown failed'); + }); + + test('satisfies the SuppressedErrorLike shape suppress() promises', () => { + expectTypeOf<FallbackSuppressedError>().toExtend<SuppressedErrorLike>(); + expectTypeOf< + ReturnType<typeof suppress> + >().toEqualTypeOf<SuppressedErrorLike>(); + }); +}); diff --git a/packages/core/src/suppress.ts b/packages/core/src/suppress.ts new file mode 100644 index 0000000..9e15b5e --- /dev/null +++ b/packages/core/src/suppress.ts @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/suppress.ts + +/** + * The shape of an error carrying a suppressed secondary throwable — structurally identical to the + * ECMAScript `SuppressedError` this module produces when the runtime has one. + * + * @public + */ +export interface SuppressedErrorLike extends Error { + /** The primary throwable — the one the caller actually cares about. */ + readonly error: unknown; + /** The secondary throwable raised while unwinding, riding along rather than masking. */ + readonly suppressed: unknown; +} + +type SuppressedErrorConstructor = new ( + error: unknown, + suppressed: unknown, + message: string, +) => SuppressedErrorLike; + +/** + * Pairs a primary throwable with a secondary one raised while unwinding, keeping the **primary** + * primary (RECOV-12). + * + * `SuppressedError` is a V8 global from the full Explicit Resource Management proposal, absent on + * this package's declared floor (`engines.node >=20.3`, set by `AbortSignal.any()`) and absent from + * the `lib` this package compiles against. Raising the floor to reach it would drop every Node 20 + * and 22 consumer for one error class, so the class is used when the runtime happens to provide it + * and {@link FallbackSuppressedError} is built when it does not — the same guarded shape the + * roadmap already sanctioned for `Symbol.asyncDispose`. The global is read per call, not captured at + * module load, so the choice tracks the runtime rather than the import order. + * + * Both branches return the same observable shape, so no caller branches on which one it got, and no + * caller may test `instanceof SuppressedError` — that would silently assert nothing on the floor. + * CI covers both: the `lts/*` leg of the `test:node` matrix takes the native branch, the pinned + * `20.3.0` leg takes the fallback. + * + * Never built via `using`/`await using`: native disposal constructs + * `new SuppressedError(disposalError, originalError)`, making the *teardown* failure primary + * (`docs/knowledge/harvested/resource-management.md:72`) — the inverse of what RECOV-12 requires. + * + * @param error - the primary throwable; stays primary. + * @param suppressed - the secondary throwable raised while unwinding. + * @param message - describes the unwinding that produced `suppressed`. + * @returns an error carrying both, with `error` primary. + * + * @internal + */ +export function suppress( + error: unknown, + suppressed: unknown, + message: string, +): SuppressedErrorLike { + const {SuppressedError: native} = globalThis as typeof globalThis & { + SuppressedError?: SuppressedErrorConstructor; + }; + return typeof native === 'function' + ? new native(error, suppressed, message) + : new FallbackSuppressedError(error, suppressed, message); +} + +/** + * The stand-in {@link suppress} builds on runtimes without the native class. Mirrors its observable + * shape — `name`, `error`, `suppressed` — so a caller never has to branch on which one it received. + * + * `name` is pinned to `'SuppressedError'` rather than following `docs/knowledge/harvested/error-handling.md`'s + * `this.name = new.target.name`: the point of this class is to be indistinguishable from the native + * one, and reporting `FallbackSuppressedError` in a stack trace would make the runtime the reader is + * on part of the error's identity. + * + * Exported so its shape is unit-testable directly. The alternative — deleting + * `globalThis.SuppressedError` inside a test to force the fallback branch — would not survive + * parallel execution, which `docs/knowledge/harvested/testing.md:50` requires of every test. + * + * @internal + */ +export class FallbackSuppressedError + extends Error + implements SuppressedErrorLike +{ + override readonly name = 'SuppressedError'; + readonly error: unknown; + readonly suppressed: unknown; + + constructor(error: unknown, suppressed: unknown, message: string) { + super(message); + this.error = error; + this.suppressed = suppressed; + } +} diff --git a/packages/core/src/testing/fake-transport.test.ts b/packages/core/src/testing/fake-transport.test.ts new file mode 100644 index 0000000..84d05ae --- /dev/null +++ b/packages/core/src/testing/fake-transport.test.ts @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/testing/fake-transport.test.ts +// Exercises the double's own contract: scripted ordering, last-entry repetition, call recording, and +// the close-observation mechanism every later retry test depends on (RETRY-35/RETRY-36). +import {describe, expect, test} from 'bun:test'; +import {Request} from '../http/request.js'; +import {Status} from '../http/status.js'; +import {FakeTransport, countingResponse} from './fake-transport.js'; + +const request = Request.newBuilder().url('https://example.com').build(); + +/** + * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this + * runner's type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper + * keeps the assertion honest without a lint suppression. + */ +async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + } catch (error) { + return error; + } + return undefined; +} + +describe('FakeTransport', () => { + test('serves scripted responses in order', async () => { + const first = countingResponse(503).response; + const second = countingResponse(200).response; + const transport = new FakeTransport([first, second]); + + expect(await transport.send(request)).toBe(first); + expect(await transport.send(request)).toBe(second); + }); + + test('repeats the last scripted entry once exhausted', async () => { + const only = countingResponse(200).response; + const transport = new FakeTransport([only]); + + await transport.send(request); + expect(await transport.send(request)).toBe(only); + expect(transport.sendCount).toBe(2); + }); + + test('a scripted Error is thrown, not returned', async () => { + const boom = new Error('connection refused'); + const transport = new FakeTransport([boom]); + + expect(await rejectionOf(transport.send(request))).toBe(boom); + }); + + test('records the request, options, and signal of every send', async () => { + const controller = new AbortController(); + const transport = new FakeTransport([countingResponse(200).response]); + + await transport.send(request, undefined, controller.signal); + + expect(transport.calls).toHaveLength(1); + expect(transport.calls[0]?.request).toBe(request); + expect(transport.calls[0]?.signal).toBe(controller.signal); + }); + + test('an empty script is a programmer error', () => { + expect(() => new FakeTransport([])).toThrow(); + }); + + test('close releases nothing and resolves', async () => { + const transport = new FakeTransport([countingResponse(200).response]); + + await transport.close(); + + expect(transport.sendCount).toBe(0); + }); +}); + +describe('countingResponse', () => { + test('reports the requested status', () => { + expect(countingResponse(503).response.status).toEqual(Status.of(503)); + }); + + test('cancelCount observes close without patching the frozen Response', async () => { + const {response, cancelCount} = countingResponse(503); + expect(cancelCount()).toBe(0); + + await response.close(); + + expect(cancelCount()).toBe(1); + }); + + test('close is idempotent, so the body is cancelled at most once', async () => { + const {response, cancelCount} = countingResponse(503); + + await response.close(); + await response.close(); + + expect(cancelCount()).toBe(1); + }); + + test('a fully drained body is observed as released too, via pull rather than cancel', async () => { + const {response, cancelCount} = countingResponse(503); + const reader = response.body?.getReader(); + for (;;) { + const chunk = await reader?.read(); + if (chunk === undefined || chunk.done) break; + } + + expect(cancelCount()).toBe(1); + }); +}); diff --git a/packages/core/src/testing/fake-transport.ts b/packages/core/src/testing/fake-transport.ts new file mode 100644 index 0000000..567483a --- /dev/null +++ b/packages/core/src/testing/fake-transport.ts @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/testing/fake-transport.ts +import {Headers} from '../http/headers.js'; +import {Protocol} from '../http/protocol.js'; +import {Request} from '../http/request.js'; +import type {RequestOptions} from '../http/request-options.js'; +import {Response} from '../http/response.js'; +import {Status} from '../http/status.js'; +import {invariant} from '../invariant.js'; +import type {Transport} from '../seams/transport.js'; + +/** + * One recorded wire send. + * + * @internal + */ +export interface FakeCall { + readonly request: Request; + readonly options: RequestOptions | undefined; + readonly signal: AbortSignal | undefined; +} + +/** + * A scripted `Transport` for multi-attempt tests (`@internal`, never exported from the package + * barrel). + * + * Entries are served in order; once exhausted the LAST entry repeats, so a script of + * `[error, response]` models "fails once, then succeeds forever" without counting attempts by hand. + * A `Response` entry is returned; an `Error` entry is thrown. + * + * **The repeat serves the same instance, not a fresh one.** An `Error` repeats harmlessly, but a + * trailing `Response` is a single object whose body a consumer may already have drained or closed -- + * so a script ending in a retryable-status response models "the same, already-retired response + * arrives again", which is not what a multi-attempt test usually means. Script one entry per + * expected wire send whenever the repeated entry is a `Response` the code under test consumes. + * + * @internal + */ +export class FakeTransport implements Transport { + readonly #script: readonly (Response | Error)[]; + readonly #calls: FakeCall[] = []; + + constructor(script: readonly (Response | Error)[]) { + invariant( + script.length > 0, + 'FakeTransport needs at least one scripted entry', + ); + this.#script = [...script]; + } + + /** Every send this double has served, in order. */ + get calls(): readonly FakeCall[] { + return this.#calls; + } + + /** Wire-send count -- what RETRY-27's budget and RETRY-32's no-further-attempts rule assert on. */ + get sendCount(): number { + return this.#calls.length; + } + + /** Options passed to each send, in order (PAGE-36). */ + get sentOptions(): readonly (RequestOptions | undefined)[] { + return this.#calls.map(c => c.options); + } + + /** Abort signals passed to each send, in order (PAGE-25). */ + get sentSignals(): readonly (AbortSignal | undefined)[] { + return this.#calls.map(c => c.signal); + } + + /** + * Records the send and serves the scripted entry at this position. + * + * @param request - the request being sent. + * @param options - the per-call options, recorded verbatim. + * @param signal - the call's abort signal, recorded verbatim. + * @returns the scripted `Response`. + * @throws the scripted `Error` when this position holds one. + */ + send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + const index = Math.min(this.#calls.length, this.#script.length - 1); + this.#calls.push({request, options, signal}); + const entry = this.#script[index]; + invariant(entry !== undefined, 'FakeTransport script index out of range'); + if (entry instanceof Error) return Promise.reject(entry); + return Promise.resolve(entry); + } + + /** + * No-op: the double owns no resources (SEAM-14's ownership rule). + */ + close(): Promise<void> { + return Promise.resolve(); + } +} + +export interface CountingResponseInit { + readonly status?: number; + readonly headers?: Record<string, string>; + readonly body?: string; + readonly request?: Request; + readonly onCancel?: () => void; +} + +/** + * Builds a `Response` whose close can be OBSERVED. + * + * `Response` instances are `Object.freeze`d, so assigning a spy over `response.close` throws + * `TypeError: Cannot add property close, object is not extensible` under ESM strict mode. The only + * sanctioned observation point is the body stream itself. Every retry, redirect, and auth test that + * asserts a body was released uses this helper. + * + * `cancelCount()` counts RELEASE, by either of the two routes the engine can take, because the + * retire path and the abandon path release the same body differently: + * + * - abandoned unread -- `Response.close()` cancels the stream, firing `cancel()`; + * - retired -- `toHttpError()` DRAINS the body into its bounded buffer (HTTP-52), so the stream + * reaches EOF and the later `close()` finds nothing to cancel; `pull()` is the only hook that + * observes it. + * + * The stream MUST close (here, on the first `pull` after its single chunk is read). A + * `ReadableStream` that enqueues and never closes leaves `toHttpError()`'s drain awaiting a chunk + * that never arrives, and every engine test that discards a 503 hangs until the runner's timeout. + * + * @internal + */ +export function countingResponse( + status: number, + request?: Request, +): {response: Response; cancelCount: () => number}; +export function countingResponse(init: CountingResponseInit): Response; +export function countingResponse( + statusOrInit: number | CountingResponseInit, + requestParam?: Request, +): Response | {response: Response; cancelCount: () => number} { + if (typeof statusOrInit === 'number') { + let releases = 0; + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + pull(controller) { + // Reached only once the single chunk has been read (default highWaterMark 1), i.e. a full drain. + releases += 1; + controller.close(); + }, + cancel() { + releases += 1; + }, + }); + const response = Response.newBuilder() + .request( + requestParam ?? Request.newBuilder().url('https://example.com').build(), + ) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(statusOrInit)) + .body(body) + .build(); + return {response, cancelCount: () => releases}; + } + + const { + status = 200, + headers: headersRecord, + body: bodyString = '{}', + request = Request.newBuilder().url('https://example.com').build(), + onCancel, + } = statusOrInit; + + const headerBuilder = Headers.newBuilder(); + if (headersRecord) { + for (const [key, value] of Object.entries(headersRecord)) { + headerBuilder.set(key, value); + } + } + + const encoded = new TextEncoder().encode(bodyString); + let served = false; + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(encoded); + }, + pull(controller) { + if (!served) { + served = true; + } else { + controller.close(); + } + }, + cancel() { + onCancel?.(); + }, + }); + + return Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(headerBuilder.build()) + .body(body) + .build(); +} diff --git a/packages/core/tsconfig.build.json b/packages/core/tsconfig.build.json index d39dc7e..a8c27f3 100644 --- a/packages/core/tsconfig.build.json +++ b/packages/core/tsconfig.build.json @@ -6,6 +6,8 @@ "sourceMap": true }, "exclude": [ - "src/**/*.test.ts" + "src/**/*.test.ts", + "src/**/*.bench.ts", + "src/io/test-support/**" ] } diff --git a/packages/logging-debug/README.md b/packages/logging-debug/README.md new file mode 100644 index 0000000..291361e --- /dev/null +++ b/packages/logging-debug/README.md @@ -0,0 +1,70 @@ +# @dexpace/logging-debug + +Routes the dexpace SDK's structured log events into [`debug`](https://www.npmjs.com/package/debug). +Zero runtime dependencies: `@dexpace/core` and `debug` are both peers, and `debug` is an **optional** +one — this package only ever calls a duck-typed subset of it. + +```sh +bun add @dexpace/logging-debug @dexpace/core debug +``` + +```typescript +import debug from 'debug'; +import {setGlobalLogger} from '@dexpace/core'; +import {createDebugLogger} from '@dexpace/logging-debug'; + +setGlobalLogger(createDebugLogger(debug, 'dexpace')); +``` + +```sh +DEBUG='dexpace:*' node app.js # everything +DEBUG='dexpace:error,dexpace:warning' node app.js # just the loud levels +``` + +## One namespace per level + +Pass `debug` itself — the **factory** — and this adapter calls it once per level, lazily, and caches +the result: `dexpace:error`, `dexpace:warning`, `dexpace:info`, `dexpace:verbose`. That is the whole +design, and it is what makes `DEBUG` a level filter without `debug` having levels. Change the base +namespace with the second argument; it defaults to `dexpace`. + +Pass a single **debugger** instead — `createDebugLogger(debug('myapp'))` — and every level goes to +that one namespace. The adapter tells the two apart structurally, by whether the argument has a +boolean `enabled` property, so there is no mode flag to get wrong. + +## What a record looks like + +The SDK's `Logger` is fluent and field-oriented (`atLevel(level).event(name).field(k, v).emit()`); +`debug` takes a format string. Each event's field map is flattened to `key=value` pairs joined by +spaces and emitted through `%s`: + +``` +dexpace:warning event=http.transport.headerDropped name=content-length +0ms +``` + +Values go through `String(v)`, so this is a human-readable channel, not a machine-parseable one. If +you need to query your logs, use `@dexpace/logging-pino`, which passes the field map to pino as an +object. + +**Suppressed events cost nothing.** `isLevelEnabled` is wired to that level's `debugger.enabled`, +which `debug` computes from `DEBUG` at construction — so an event at a disabled level never builds +its field map. + +## Options + +`createDebugLogger(debugOrFactory, namespace, options)` forwards `CreateLoggerOptions` minus +`isLevelEnabled`, which this adapter owns: + +- `globalFields` — merged into every record. +- `diagnosticAllowList` — the query parameters that survive URL redaction. Everything else is + redacted before it reaches `debug`, so a URL in a log line cannot leak a token. `null` means + "redact every parameter". + +A `null`, or anything that is neither a function nor an object, is a construction-time `TypeError` — +loud, at wiring time, rather than a swallowed no-op at the first log line. + +## The alternative + +`@dexpace/logging-pino` does the same job over pino, with structured records and a runtime-adjustable +level. Neither is required: the SDK's default is `NOOP_LOGGER`, and `createLogger(sink)` in +`@dexpace/core` adapts anything else in a few lines. diff --git a/packages/logging-debug/api-extractor.json b/packages/logging-debug/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/logging-debug/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/logging-debug/etc/logging-debug.api.md b/packages/logging-debug/etc/logging-debug.api.md new file mode 100644 index 0000000..7a4f339 --- /dev/null +++ b/packages/logging-debug/etc/logging-debug.api.md @@ -0,0 +1,26 @@ +## API Report File for "@dexpace/logging-debug" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import { CreateLoggerOptions } from '@dexpace/core'; +import { Logger } from '@dexpace/core'; + +// @public +export function createDebugLogger(debugOrFactory: DebugLike | DebugFactory, namespace?: string, options?: Omit<CreateLoggerOptions, 'isLevelEnabled'>): Logger; + +// @public +export type DebugFactory = (namespace: string) => DebugLike; + +// @public +export interface DebugLike { + // (undocumented) + (formatter: string, ...args: unknown[]): void; + // (undocumented) + enabled: boolean; +} + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/logging-debug/package.json b/packages/logging-debug/package.json new file mode 100644 index 0000000..8728660 --- /dev/null +++ b/packages/logging-debug/package.json @@ -0,0 +1,55 @@ +{ + "name": "@dexpace/logging-debug", + "version": "0.0.0", + "description": "Debug logging adapter for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/logging-debug" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "debug": ">=4.0.0" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + }, + "debug": { + "optional": true + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/logging-debug/src/debug-logger.test.ts b/packages/logging-debug/src/debug-logger.test.ts new file mode 100644 index 0000000..0841c0e --- /dev/null +++ b/packages/logging-debug/src/debug-logger.test.ts @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: MIT +// packages/logging-debug/src/debug-logger.test.ts +// Exercises: OBS-1..9, OBS-25, OBS-40 +import {describe, expect, test} from 'bun:test'; +import {createDebugLogger, type DebugLike} from './debug-logger.js'; + +describe('createDebugLogger', () => { + test('single debug instance formatted output', () => { + const formatted: string[] = []; + const dbg: DebugLike = Object.assign( + (formatter: string, ...args: unknown[]) => { + formatted.push(`${formatter} -> ${args.join(' ')}`); + }, + {enabled: true}, + ); + + const logger = createDebugLogger(dbg); + logger.atLevel('info').event('my.event').field('k', 'v').emit(); + + expect(formatted).toHaveLength(1); + expect(formatted[0]).toContain('event=my.event'); + expect(formatted[0]).toContain('k=v'); + }); + + test('debug factory with enabled function routes per level namespace', () => { + const map = new Map<string, string[]>(); + const factory = Object.assign( + (namespace: string): DebugLike => { + const logs: string[] = []; + map.set(namespace, logs); + return Object.assign( + (_formatter: string, ...args: unknown[]) => { + logs.push(args.join(' ')); + }, + {enabled: namespace !== 'dexpace:verbose'}, + ); + }, + { + enabled: () => true, + }, + ); + + const logger = createDebugLogger(factory, 'dexpace'); + + logger.atLevel('info').event('info.event').emit(); + expect(map.get('dexpace:info')).toHaveLength(1); + expect(map.get('dexpace:info')?.[0]).toContain('event=info.event'); + + // verbose is disabled + logger.atLevel('verbose').event('verbose.event').emit(); + expect(map.get('dexpace:verbose')).toHaveLength(0); + }); + + test('rejects non-function/non-object input', () => { + expect(() => createDebugLogger(null as unknown as DebugLike)).toThrow(); + }); +}); diff --git a/packages/logging-debug/src/debug-logger.ts b/packages/logging-debug/src/debug-logger.ts new file mode 100644 index 0000000..36547de --- /dev/null +++ b/packages/logging-debug/src/debug-logger.ts @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: MIT +// packages/logging-debug/src/debug-logger.ts +// Exercises: OBS-1..9, OBS-25, OBS-40 +import { + createLogger, + type CreateLoggerOptions, + type Logger, + type LogLevel, +} from '@dexpace/core'; + +/** + * Structural subset of debug's Debugger so this package adds zero runtime dependencies beyond the core package. + * A real debug instance duck-types directly into this shape. + * + * @public + */ +export interface DebugLike { + enabled: boolean; + (formatter: string, ...args: unknown[]): void; +} + +/** + * Factory creating DebugLike instances per namespace. + * + * @public + */ +export type DebugFactory = (namespace: string) => DebugLike; + +/** + * Creates a Logger adapter wrapping a debug instance or factory. + * + * @param debugOrFactory - debug function or factory. + * @param namespace - base namespace (default: 'dexpace'). + * @param options - optional creation options such as diagnostic allow list. + * @returns a Logger routing to debug. + * + * @public + */ +export function createDebugLogger( + debugOrFactory: DebugLike | DebugFactory, + namespace = 'dexpace', + options?: Omit<CreateLoggerOptions, 'isLevelEnabled'>, +): Logger { + if ( + (debugOrFactory as unknown) === null || + (typeof debugOrFactory !== 'function' && typeof debugOrFactory !== 'object') + ) { + throw new TypeError( + 'createDebugLogger: debug instance or factory is required', + ); + } + + const isSingleDebugger = + typeof (debugOrFactory as {enabled?: unknown}).enabled === 'boolean'; + + const debuggers = new Map<LogLevel, DebugLike>(); + const getDebugger = (level: LogLevel): DebugLike => { + if (isSingleDebugger) { + return debugOrFactory as DebugLike; + } + let dbg = debuggers.get(level); + if (dbg === undefined) { + dbg = (debugOrFactory as DebugFactory)(`${namespace}:${level}`); + debuggers.set(level, dbg); + } + return dbg; + }; + + return createLogger( + (level, fields) => { + const dbg = getDebugger(level); + const parts: string[] = []; + for (const [k, v] of fields) parts.push(`${k}=${String(v)}`); + dbg('%s', parts.join(' ')); + }, + { + ...options, + isLevelEnabled: (level: LogLevel): boolean => getDebugger(level).enabled, + }, + ); +} diff --git a/packages/logging-debug/src/index.ts b/packages/logging-debug/src/index.ts new file mode 100644 index 0000000..65c9c9b --- /dev/null +++ b/packages/logging-debug/src/index.ts @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT +// packages/logging-debug/src/index.ts +export { + createDebugLogger, + type DebugLike, + type DebugFactory, +} from './debug-logger.js'; diff --git a/packages/logging-debug/tsconfig.build.json b/packages/logging-debug/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/logging-debug/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/logging-debug/tsconfig.json b/packages/logging-debug/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/logging-debug/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/logging-pino/README.md b/packages/logging-pino/README.md new file mode 100644 index 0000000..1f7f3c5 --- /dev/null +++ b/packages/logging-pino/README.md @@ -0,0 +1,68 @@ +# @dexpace/logging-pino + +Routes the dexpace SDK's structured log events into [pino](https://getpino.io). Zero runtime +dependencies: `@dexpace/core` and `pino` are both peers, and `pino` is an **optional** one — this +package only ever calls a duck-typed subset of it. + +```sh +bun add @dexpace/logging-pino @dexpace/core pino +``` + +```typescript +import pino from 'pino'; +import {setGlobalLogger} from '@dexpace/core'; +import {createPinoLogger} from '@dexpace/logging-pino'; + +setGlobalLogger(createPinoLogger(pino({level: 'info'}))); +``` + +That is the whole wiring. Every SDK event — retry attempts, redirect hops, dropped outbound headers, +auth refresh failures — now arrives as a pino record with its fields as top-level keys. + +## What it actually does + +The SDK's `Logger` is a fluent, four-level facade (`atLevel(level).event(name).field(k, v).emit()`). +pino's is a five-method object taking `(obj, msg?)`. This package is the mapping between them, and +it is three decisions wide: + +| SDK level | pino method | +|---|---| +| `error` | `error` | +| `warning` | `warn` | +| `info` | `info` | +| `verbose` | `debug` | + +- **Fields become the record, not the message.** Each event's field map is passed as pino's `obj` + argument, so `{event: 'http.retry.attemptFailed', attempt: 2}` lands as queryable keys rather than + an interpolated string. No `msg` is set. +- **Level checks are delegated, per call.** `isLevelEnabled` is wired straight to + `pino.isLevelEnabled`, so a suppressed event costs one predicate call and never builds its field + map. Changing pino's level at runtime takes effect immediately; nothing is cached. +- **`pino.trace` is never called.** The SDK has four levels; `verbose` is the floor and maps to + `debug`. + +## Options + +`createPinoLogger(instance, options)` forwards `CreateLoggerOptions` minus `isLevelEnabled`, which +this adapter owns: + +- `globalFields` — merged into every record (a service name, a build id). +- `diagnosticAllowList` — the query parameters that survive URL redaction. Everything else is + redacted before it reaches pino, so a URL in a log line cannot leak a token. `null` means "redact + every parameter". + +## Anything pino-shaped works + +The parameter type is `PinoLike`, a five-method structural interface — `isLevelEnabled`, `error`, +`warn`, `info`, `debug` — not pino's own type. A real pino instance duck-types into it, and so does a +child logger (`pino().child({req: id})`), a test double, or a wrapper of your own. That is why this +package can declare `pino` optional and still carry zero dependencies. + +A non-object, or an object without a callable `isLevelEnabled`, is a construction-time `TypeError` — +loud, at wiring time, rather than a swallowed no-op at the first log line. + +## The alternative + +`@dexpace/logging-debug` does the same job over [`debug`](https://www.npmjs.com/package/debug), with +namespace-per-level filtering instead of a level threshold. Neither is required: the SDK's default +is `NOOP_LOGGER`, and `createLogger(sink)` in `@dexpace/core` adapts anything else in a few lines. diff --git a/packages/logging-pino/api-extractor.json b/packages/logging-pino/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/logging-pino/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/logging-pino/etc/logging-pino.api.md b/packages/logging-pino/etc/logging-pino.api.md new file mode 100644 index 0000000..fb9d8fd --- /dev/null +++ b/packages/logging-pino/etc/logging-pino.api.md @@ -0,0 +1,31 @@ +## API Report File for "@dexpace/logging-pino" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import { CreateLoggerOptions } from '@dexpace/core'; +import { Logger } from '@dexpace/core'; + +// @public +export function createPinoLogger(pino: PinoLike, options?: Omit<CreateLoggerOptions, 'isLevelEnabled'>): Logger; + +// @public +export interface PinoLike { + // (undocumented) + debug(obj: object, msg?: string): void; + // (undocumented) + error(obj: object, msg?: string): void; + // (undocumented) + info(obj: object, msg?: string): void; + // (undocumented) + isLevelEnabled(level: string): boolean; + // (undocumented) + trace(obj: object, msg?: string): void; + // (undocumented) + warn(obj: object, msg?: string): void; +} + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/logging-pino/package.json b/packages/logging-pino/package.json new file mode 100644 index 0000000..03a77d3 --- /dev/null +++ b/packages/logging-pino/package.json @@ -0,0 +1,55 @@ +{ + "name": "@dexpace/logging-pino", + "version": "0.0.0", + "description": "Pino logging adapter for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/logging-pino" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "pino": ">=8.0.0" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + }, + "pino": { + "optional": true + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/logging-pino/src/index.ts b/packages/logging-pino/src/index.ts new file mode 100644 index 0000000..f16bf47 --- /dev/null +++ b/packages/logging-pino/src/index.ts @@ -0,0 +1,3 @@ +// SPDX-License-Identifier: MIT +// packages/logging-pino/src/index.ts +export {createPinoLogger, type PinoLike} from './pino-logger.js'; diff --git a/packages/logging-pino/src/pino-logger.test.ts b/packages/logging-pino/src/pino-logger.test.ts new file mode 100644 index 0000000..1e767ad --- /dev/null +++ b/packages/logging-pino/src/pino-logger.test.ts @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: MIT +// packages/logging-pino/src/pino-logger.test.ts +import {describe, expect, test} from 'bun:test'; +import {createPinoLogger, type PinoLike} from './pino-logger.js'; + +describe('createPinoLogger', () => { + test('maps SDK log levels to pino methods and checks isLevelEnabled', () => { + const calls: {level: string; obj: Record<string, unknown>}[] = []; + const pino: PinoLike = { + isLevelEnabled: (level: string) => level !== 'debug', + error: (obj: object) => + calls.push({level: 'error', obj: obj as Record<string, unknown>}), + warn: (obj: object) => + calls.push({level: 'warn', obj: obj as Record<string, unknown>}), + info: (obj: object) => + calls.push({level: 'info', obj: obj as Record<string, unknown>}), + debug: (obj: object) => + calls.push({level: 'debug', obj: obj as Record<string, unknown>}), + trace: (obj: object) => + calls.push({level: 'trace', obj: obj as Record<string, unknown>}), + }; + + const logger = createPinoLogger(pino); + + logger.atLevel('info').event('test.event').field('k', 'v').emit(); + expect(calls).toHaveLength(1); + expect(calls[0]?.level).toBe('info'); + expect(calls[0]?.obj.event).toBe('test.event'); + expect(calls[0]?.obj.k).toBe('v'); + + // verbose is mapped to debug, which is disabled in isLevelEnabled + logger.atLevel('verbose').event('debug.event').emit(); + expect(calls).toHaveLength(1); + }); + + test('rejects null or non-pino input', () => { + expect(() => createPinoLogger(null as unknown as PinoLike)).toThrow(); + expect(() => createPinoLogger({} as unknown as PinoLike)).toThrow(); + }); +}); diff --git a/packages/logging-pino/src/pino-logger.ts b/packages/logging-pino/src/pino-logger.ts new file mode 100644 index 0000000..b9919cc --- /dev/null +++ b/packages/logging-pino/src/pino-logger.ts @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT +// packages/logging-pino/src/pino-logger.ts +// Exercises: OBS-1..9, OBS-25, OBS-40 +import { + createLogger, + type CreateLoggerOptions, + type Logger, + type LogLevel, +} from '@dexpace/core'; + +/** + * Structural subset of pino's Logger so this package adds zero runtime dependencies beyond the core package. + * A real pino instance duck-types directly into this shape. + * + * @public + */ +export interface PinoLike { + isLevelEnabled(level: string): boolean; + error(obj: object, msg?: string): void; + warn(obj: object, msg?: string): void; + info(obj: object, msg?: string): void; + debug(obj: object, msg?: string): void; + trace(obj: object, msg?: string): void; +} + +const LEVEL_MAP: Record<LogLevel, 'error' | 'warn' | 'info' | 'debug'> = { + error: 'error', + warning: 'warn', + info: 'info', + verbose: 'debug', +}; + +/** + * Creates a Logger adapter wrapping a pino instance. + * + * @param pino - the pino logger instance or compatible object. + * @param options - optional creation options such as diagnostic allow list. + * @returns a Logger routing to pino. + * + * @public + */ +export function createPinoLogger( + pino: PinoLike, + options?: Omit<CreateLoggerOptions, 'isLevelEnabled'>, +): Logger { + if ((pino as unknown) === null || typeof pino !== 'object') { + throw new TypeError('createPinoLogger: pino instance is required'); + } + if (typeof pino.isLevelEnabled !== 'function') { + throw new TypeError( + 'createPinoLogger: pino.isLevelEnabled must be a function', + ); + } + return createLogger( + (level, fields) => { + const pinoLevel = LEVEL_MAP[level]; + const obj = Object.fromEntries(fields); + pino[pinoLevel](obj); + }, + { + ...options, + isLevelEnabled: (level: LogLevel): boolean => + pino.isLevelEnabled(LEVEL_MAP[level]), + }, + ); +} diff --git a/packages/logging-pino/tsconfig.build.json b/packages/logging-pino/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/logging-pino/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/logging-pino/tsconfig.json b/packages/logging-pino/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/logging-pino/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/rx/README.md b/packages/rx/README.md new file mode 100644 index 0000000..fd35cb4 --- /dev/null +++ b/packages/rx/README.md @@ -0,0 +1,92 @@ +# @dexpace/rx + +RxJS async-runtime bridge for the dexpace Node.js SDK. + +## Installation + +```bash +npm install @dexpace/rx rxjs +``` + +## Usage + +```typescript +import {sseEvents$, typedSse$, pageItems$, pages$} from '@dexpace/rx'; +import {sseStreamFrom, Paginator, type Response} from '@dexpace/core'; + +declare const response: Response; +declare const paginator: Paginator<unknown>; + +// Server-Sent Events +sseEvents$(sseStreamFrom(response)).subscribe({ + next: event => console.log(event.data), +}); + +// ...or decoded into your own models +typedSse$(sseStreamFrom(response), (eventName, data) => + eventName === 'done' + ? {kind: 'done'} + : {kind: 'value', value: JSON.parse(data)}, +).subscribe({next: model => console.log(model)}); + +// Pagination, item by item or page by page +pageItems$(paginator).subscribe({next: item => console.log(item)}); +pages$(paginator).subscribe({next: page => console.log(page.items)}); +``` + +## Two subscription models, on purpose + +`sseEvents$`/`typedSse$` are **single-subscription**. An `SseStream` wraps one already-open HTTP response body, +which is single-use (`BODY-14`) and single-pass (`SSE-26`); there is no honest way to make re-subscription +meaningful without a second HTTP call this package does not make. A second `subscribe()` reaches `SseStream`'s +own guard and surfaces its error through the `Observable`'s error channel. + +`pageItems$`/`pages$` are **cold and repeatable**. `Paginator.items()`/`.pages()` build a fresh generator per +call (`PAGE-8`), so each subscription drives an independent fetch sequence. + +Both release their source on `unsubscribe()`, including while idle — an SSE stream waiting on the next event is +closed immediately rather than when the server next sends something. Which of the two *owns* that source +differs, and that is the next section. + +## Who owns the stream + +`sseEvents$`/`typedSse$` **take ownership of the `SseStream` you hand them.** Subscribing takes the stream's one +iterator (`SSE-26`), and the adapter releases the stream on every termination — `unsubscribe()`, end-of-source +and a source error alike. Do not call `close()` on it yourself, and do not iterate it afterwards; `unsubscribe()` +is how you stop early. + +That covers stopping while the stream is **idle**, which is where a live event stream spends almost all of its +time. The adapter runs the release *ahead of* the iterator's `return()` on purpose: an async generator's +`return()` queues behind a suspended `next()`, so on its own it cannot settle a read that is waiting on a server +which will never send again. + +It is also what a plain `for await` over the same stream already does — `SseStream` releases its resource when +its iterator returns, so `break`ing out of the loop closes the response body too. The reactive form transfers +ownership for the same reason the loop does, not as an extra: + +```typescript +import {sseStreamFrom, type Response} from '@dexpace/core'; + +declare const response: Response; + +// No `close()` here either: leaving the loop releases the response body. +for await (const event of sseStreamFrom(response)) { + if (event.event === 'done') break; +} +``` + +This is a deliberate departure from `ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" +clause, and it is recorded as one — see the `ASYNC-21` row of [`docs/deviations.md`](../../docs/deviations.md). + +`pageItems$`/`pages$` transfer nothing, and attach no release callback. The `Paginator`'s own walk already owns +each page's response — `items()` closes a page before yielding any of its items (`PAGE-11`), and `pages()` +closes the held page from the generator's `finally` (`PAGE-12`), which the iterator's `return()` drives. That is +enough there because a paginator's pulls are bounded HTTP exchanges; an SSE pull is a wait on a server that may +never answer, which is the difference the callback exists for. + +## Notes + +- `rxjs` and `@dexpace/core` are **peer** dependencies. A duplicate copy of either would break the identity + checks (`Observable`/`Subscription`, and core's branded symbols) that a bundled copy silently defeats. +- This package installs no RxJS scheduler. Diagnostic context therefore propagates on its own through Node's + `AsyncLocalStorage`; a caller who adds `observeOn`/`subscribeOn` downstream owns reinstating it. diff --git a/packages/rx/api-extractor.json b/packages/rx/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/rx/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/rx/etc/rx.api.md b/packages/rx/etc/rx.api.md new file mode 100644 index 0000000..f54d12b --- /dev/null +++ b/packages/rx/etc/rx.api.md @@ -0,0 +1,26 @@ +## API Report File for "@dexpace/rx" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import type { Observable } from 'rxjs'; +import type { Page } from '@dexpace/core'; +import type { Paginator } from '@dexpace/core'; +import { SseEvent } from '@dexpace/core'; +import { SseMapper } from '@dexpace/core'; +import { SseStream } from '@dexpace/core'; + +// @public +export function pageItems$<T>(paginator: Paginator<T>): Observable<T>; + +// @public +export function pages$<T>(paginator: Paginator<T>): Observable<Page<T>>; + +// @public +export function sseEvents$(stream: SseStream): Observable<SseEvent>; + +// @public +export function typedSse$<T>(stream: SseStream, mapper: SseMapper<T>): Observable<T>; + +``` diff --git a/packages/rx/package.json b/packages/rx/package.json new file mode 100644 index 0000000..6cb6c1a --- /dev/null +++ b/packages/rx/package.json @@ -0,0 +1,56 @@ +{ + "name": "@dexpace/rx", + "version": "0.0.0", + "description": "RxJS async-runtime bridge for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/rx" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*", + "rxjs": "^7.8.0" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + }, + "rxjs": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "rxjs": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/rx/src/from-async-iterable.conformance.test.ts b/packages/rx/src/from-async-iterable.conformance.test.ts new file mode 100644 index 0000000..f012ba8 --- /dev/null +++ b/packages/rx/src/from-async-iterable.conformance.test.ts @@ -0,0 +1,331 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/from-async-iterable.conformance.test.ts +// +// Exercises: ASYNC-21 (poll-once-per-demand under a synchronous subscriber, complete on end-of-source, +// propagate a source error as an error signal), ASYNC-13 (no wrapper exception around a thrown value), +// ASYNC-6 (unsubscribe reaches the source's .return() exactly once across synchronous and asynchronous paths). +// +// This suite tests fromAsyncIterable against a hand-built async generator / iterator test double, deliberately not +// SseStream/Paginator, to isolate "does the async-iterable bridge satisfy the contract" from "does 6b/6c's own close discipline +// work" (already proven in their own test suites). +// +// The final describe block is the one that runs against rxjs's OWN from(), not ours: it pins the single +// ASYNC-6 clause the native operator fails, which is the entire justification for this package shipping a +// hand-written bridge instead of the one-liner its plan called for. If that case ever fails, RxJS has closed +// the gap and `from-async-iterable.ts` should be deleted in favor of `from()`. +import {describe, expect, test} from 'bun:test'; +import {firstValueFrom, from, take, toArray} from 'rxjs'; +import {fromAsyncIterable} from './from-async-iterable.js'; + +async function* countTo( + n: number, + onReturn?: () => void, +): AsyncGenerator<number> { + try { + for (let i = 1; i <= n; i++) { + await Promise.resolve(); + yield i; + } + } finally { + onReturn?.(); + } +} + +function makePendingIterableDouble( + onReturn?: () => void, +): AsyncIterable<number> { + let returned = false; + return { + [Symbol.asyncIterator]() { + return { + next(): Promise<IteratorResult<number, void>> { + if (returned) { + return Promise.resolve({done: true, value: undefined}); + } + return new Promise<IteratorResult<number, void>>(() => { + // never settles + }); + }, + return(): Promise<IteratorResult<number, void>> { + returned = true; + onReturn?.(); + return Promise.resolve({done: true, value: undefined}); + }, + }; + }, + }; +} + +describe('fromAsyncIterable — ASYNC-21', () => { + test('polls the source once per emission, never ahead of demand', async () => { + let pulls = 0; + async function* spy(): AsyncGenerator<number> { + for (let i = 1; i <= 10; i++) { + await Promise.resolve(); + pulls++; + yield i; + } + } + + // The source deliberately outlives demand. A generator yielding exactly as many values as the + // subscriber consumes cannot tell one-pull-per-emission apart from a bridge that prefetches -- it has + // nothing left to prefetch, so the assertion passes either way. Ten available, two taken, two pulled. + const values = await firstValueFrom( + fromAsyncIterable(spy()).pipe(take(2), toArray()), + ); + expect(values).toEqual([1, 2]); + expect(pulls).toBe(2); + }); + + test('completes the Observable when the source generator returns', async () => { + const values = await firstValueFrom( + fromAsyncIterable(countTo(2)).pipe(toArray()), + ); + expect(values).toEqual([1, 2]); + }); + + test('a source throw surfaces via the error channel with the original value, unwrapped', async () => { + async function* throwing(): AsyncGenerator<number> { + await Promise.resolve(); + yield 1; + throw new RangeError('boom'); + } + const errors: unknown[] = []; + await new Promise<void>(resolve => { + fromAsyncIterable(throwing()).subscribe({ + next() { + // ignore + }, + error(err: unknown) { + errors.push(err); + resolve(); + }, + }); + }); + expect(errors).toHaveLength(1); + expect(errors[0]).toBeInstanceOf(RangeError); + expect((errors[0] as RangeError).message).toBe('boom'); // ASYNC-13: not wrapped in an RxJS-internal type + }); +}); + +describe('fromAsyncIterable — ASYNC-6 (synchronous & asynchronous cancellation)', () => { + test("unsubscribing mid-stream synchronously calls the generator's .return() exactly once", async () => { + let returns = 0; + const generator = countTo(5, () => { + returns++; + }); + await new Promise<void>(resolve => { + const subscription = fromAsyncIterable(generator).subscribe({ + next(value) { + if (value === 2) { + subscription.unsubscribe(); + // allow the microtask queue to settle the generator's finally block + setTimeout(resolve, 10); + } + }, + }); + }); + expect(returns).toBe(1); + }); + + test('unsubscribing asynchronously while idle awaiting .next() calls .return() immediately', async () => { + let returns = 0; + let pushNextValue!: (val: number) => void; + + const stream: AsyncIterable<number> = { + [Symbol.asyncIterator]() { + let isDone = false; + let pendingResolve: + ((res: IteratorResult<number, void>) => void) | undefined; + pushNextValue = (val: number) => { + if (pendingResolve) { + const r = pendingResolve; + pendingResolve = undefined; + r({done: false, value: val}); + } + }; + return { + next() { + if (isDone) { + return Promise.resolve({done: true, value: undefined}); + } + return new Promise<IteratorResult<number, void>>(resolve => { + pendingResolve = resolve; + }); + }, + return() { + isDone = true; + returns++; + return Promise.resolve({done: true, value: undefined}); + }, + }; + }, + }; + + const received: number[] = []; + const subscription = fromAsyncIterable(stream).subscribe({ + next(value) { + received.push(value); + }, + }); + + // Push first value + pushNextValue(1); + await new Promise(r => setTimeout(r, 10)); + expect(received).toEqual([1]); + expect(returns).toBe(0); + + // Unsubscribe while waiting for second value + subscription.unsubscribe(); + expect(returns).toBe(1); + }); +}); + +describe('fromAsyncIterable — ASYNC-6 (edge release handling)', () => { + test('unsubscribing before first emission calls .return() immediately', () => { + let returns = 0; + const iterable = makePendingIterableDouble(() => { + returns++; + }); + + const subscription = fromAsyncIterable(iterable).subscribe({ + next() { + // ignore + }, + }); + + subscription.unsubscribe(); + expect(returns).toBe(1); + }); + + test('unsubscribing swallows release failures from close() or return() per ASYNC-21 / SSE-30', async () => { + const iterable = { + close(): Promise<void> { + return Promise.reject(new Error('close failed')); + }, + [Symbol.asyncIterator]() { + return { + next(): Promise<IteratorResult<number, void>> { + return new Promise<IteratorResult<number, void>>(() => { + // never settles + }); + }, + return(): Promise<IteratorResult<number, void>> { + return Promise.reject(new Error('return failed')); + }, + }; + }, + }; + + const subscription = fromAsyncIterable(iterable).subscribe({ + next() { + // ignore + }, + }); + + // Should not throw unhandled rejection + subscription.unsubscribe(); + await new Promise(r => setTimeout(r, 20)); + }); +}); + +describe('fromAsyncIterable — ASYNC-6 (caller-supplied source release)', () => { + test('releases the source before returning the iterator, so a suspended pull can settle', async () => { + const order: string[] = []; + let settlePendingPull: (() => void) | undefined; + + const iterable: AsyncIterable<number> = { + [Symbol.asyncIterator]() { + return { + next(): Promise<IteratorResult<number, void>> { + return new Promise<IteratorResult<number, void>>(resolve => { + settlePendingPull = () => { + resolve({done: true, value: undefined}); + }; + }); + }, + return(): Promise<IteratorResult<number, void>> { + order.push('return'); + return Promise.resolve({done: true, value: undefined}); + }, + }; + }, + }; + + const subscription = fromAsyncIterable(iterable, () => { + order.push('release'); + settlePendingPull?.(); + return Promise.resolve(); + }).subscribe({ + next() { + // ignore + }, + }); + + subscription.unsubscribe(); + await new Promise(r => setTimeout(r, 10)); + expect(order).toEqual(['release', 'return']); + }); + + test('a rejected release does not surface from unsubscribe()', () => { + const iterable: AsyncIterable<number> = { + [Symbol.asyncIterator]() { + return { + next(): Promise<IteratorResult<number, void>> { + return new Promise<IteratorResult<number, void>>(() => { + // never settles + }); + }, + }; + }, + }; + + const subscription = fromAsyncIterable(iterable, () => + Promise.reject(new Error('release failed')), + ).subscribe({ + next() { + // ignore + }, + }); + + expect(() => { + subscription.unsubscribe(); + }).not.toThrow(); + }); +}); + +describe("rxjs's own from() — the ASYNC-6 gap this module exists to close", () => { + test('does NOT reach the source when unsubscribed while a pull is suspended', async () => { + let returns = 0; + const iterable = makePendingIterableDouble(() => { + returns++; + }); + + const subscription = from(iterable).subscribe({ + next() { + // ignore + }, + }); + subscription.unsubscribe(); + await new Promise(r => setTimeout(r, 20)); + + // Deliberately asserting the DEFECT, not the fix. rxjs 7's async-iterable path only tests + // subscriber.closed after a pull resolves, so an idle SSE stream is never released. When this + // expectation starts failing, `fromAsyncIterable` has become redundant -- see this file's header. + expect(returns).toBe(0); + + // The same source through this module's bridge is released immediately. + let bridgedReturns = 0; + const bridged = fromAsyncIterable( + makePendingIterableDouble(() => { + bridgedReturns++; + }), + ).subscribe({ + next() { + // ignore + }, + }); + bridged.unsubscribe(); + expect(bridgedReturns).toBe(1); + }); +}); diff --git a/packages/rx/src/from-async-iterable.ts b/packages/rx/src/from-async-iterable.ts new file mode 100644 index 0000000..516c9fb --- /dev/null +++ b/packages/rx/src/from-async-iterable.ts @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/from-async-iterable.ts +import {Observable} from 'rxjs'; + +/** + * Drive an iterator's `return()` on a termination path, swallowing a release failure. + * + * Release is *quiet* here (`SSE-30`): the terminal signal the subscriber sees has already been decided by the + * time this runs, so a failure to release cannot preempt it. `SseStream` still reports the failure through its + * own `onReleaseFailure` hook — swallowing here drops nothing that layer records. + */ +async function returnQuietly(iterator: AsyncIterator<unknown>): Promise<void> { + if (typeof iterator.return !== 'function') { + return; + } + try { + await iterator.return(); + } catch { + // Quiet release per ASYNC-21 / SSE-30 -- see this function's own doc comment. + } +} + +/** As {@link returnQuietly}, for the caller-supplied source release. */ +async function releaseQuietly(release: () => Promise<void>): Promise<void> { + try { + await release(); + } catch { + // Quiet release per ASYNC-21 / SSE-30 -- see returnQuietly's doc comment. + } +} + +/** + * Converts an {@link AsyncIterable} into an RxJS {@link Observable}, attaching a finalizer that reaches the + * source on every termination path (`ASYNC-6`, `ASYNC-21`). + * + * **Why this is not `rxjs`'s own `from(asyncIterable)`.** RxJS 7's async-iterable path is a bare + * `for await` loop that tests `subscriber.closed` only *after* a pull resolves. Unsubscribing while a pull is + * suspended — the normal state of an idle SSE stream waiting for the next event — therefore reaches the source + * only if and when the server sends something, so the response body is never released and the connection is + * held open indefinitely. Verified against `rxjs@7.8.2`; pinned by + * `from-async-iterable.conformance.test.ts`'s "rxjs's own from()" case, which fails if a future RxJS closes the + * gap and makes this module redundant. + * + * On a termination path this runs `release` first and *then* returns the iterator: closing the source is what + * settles an in-flight pull, and an async generator's `return()` is queued behind that pull rather than + * preempting it. Cancellation is the case that ordering exists for — it is the only one where the pull may + * stay suspended indefinitely. `return()` runs exactly once across every path — early cancellation, + * end-of-source, and a source error alike. + * + * @param iterable - The source to bridge. Its iterator is taken once per subscription. + * @param release - Optional source-level release, run ahead of `iterator.return()`. RxJS runs a subscriber's + * finalizer on *every* termination, so this fires exactly once per subscription — on unsubscription, on + * end-of-source, and on a source error alike, not on cancellation alone. Pass only a release that tolerates + * being called after the source has already drained; `SseStream.close()` is idempotent (`SSE-28`), which is + * what makes the end-of-source call a no-op rather than a second release. + * + * @internal + */ +export function fromAsyncIterable<T>( + iterable: AsyncIterable<T>, + release?: () => Promise<void>, +): Observable<T> { + return new Observable<T>(subscriber => { + const iterator = iterable[Symbol.asyncIterator](); + + // A function, not a bare `subscriber.closed` read: cancellation lands during an `await`, and TypeScript's + // narrowing would otherwise treat every re-check inside the loop as dead code. It is the opposite -- those + // re-checks are the whole point. + const cancelled = (): boolean => subscriber.closed; + + // The single owner of `ASYNC-6`'s exactly-once obligation. Cancellation and end-of-source both reach it, + // and whichever arrives first is the one that releases. + let returned = false; + const returnOnce = async (): Promise<void> => { + if (returned) { + return; + } + returned = true; + await returnQuietly(iterator); + }; + + void (async (): Promise<void> => { + try { + while (!cancelled()) { + const result = await iterator.next(); + if (result.done === true || cancelled()) { + break; + } + subscriber.next(result.value); + } + if (!cancelled()) { + subscriber.complete(); + } + } catch (err: unknown) { + if (!cancelled()) { + subscriber.error(err); + } + } finally { + await returnOnce(); + } + })(); + + return () => { + if (release !== undefined) { + void releaseQuietly(release); + } + void returnOnce(); + }; + }); +} diff --git a/packages/rx/src/index.ts b/packages/rx/src/index.ts new file mode 100644 index 0000000..3ef3c7c --- /dev/null +++ b/packages/rx/src/index.ts @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/index.ts + +/** + * RxJS async-runtime bridge for the dexpace Node.js SDK. + * + * @packageDocumentation + */ + +export {sseEvents$, typedSse$} from './sse.js'; +export {pageItems$, pages$} from './pagination.js'; diff --git a/packages/rx/src/pagination.test.ts b/packages/rx/src/pagination.test.ts new file mode 100644 index 0000000..93102c7 --- /dev/null +++ b/packages/rx/src/pagination.test.ts @@ -0,0 +1,278 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/pagination.test.ts +// +// Exercises: PAGE-8 (cold and repeatable: multiple subscriptions drive independent fetch sequences), +// PAGE-1 (emits every item across all pages in server order, pages$ yields whole pages), +// ASYNC-6 (unsubscribing mid-walk cancels the generator cleanly), +// ASYNC-13 (a walk failure reaches the error channel unwrapped, after the items already delivered). +import {describe, expect, test} from 'bun:test'; +import {firstValueFrom, toArray} from 'rxjs'; +import { + Paginator, + Protocol, + Request, + Response, + Status, + type PageInfo, + type PaginationStrategy, + type Transport, +} from '@dexpace/core'; +import {pageItems$, pages$} from './pagination.js'; + +function createMockTransport(): { + transport: Transport; + getSendCount: () => number; +} { + let sendCount = 0; + const transport: Transport = { + send(req: Request): Promise<Response> { + sendCount++; + return Promise.resolve( + Response.newBuilder() + .request(req) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .build(), + ); + }, + close(): Promise<void> { + return Promise.resolve(); + }, + }; + return {transport, getSendCount: () => sendCount}; +} + +function createTwoPageStrategy(): PaginationStrategy<string> { + return { + parse(_response: Response, template: Request) { + const page = Number(template.url.searchParams.get('page') ?? '1'); + const items = [`item_${String(page)}_1`, `item_${String(page)}_2`]; + if (page >= 2) { + return Promise.resolve({items, nextRequest: undefined}); + } + const nextUrl = new URL(template.url); + nextUrl.searchParams.set('page', String(page + 1)); + const nextRequest = Request.newBuilder() + .method(template.method) + .url(nextUrl) + .build(); + return Promise.resolve({items, nextRequest}); + }, + }; +} + +function createInitialRequest(): Request { + return Request.newBuilder() + .method('GET') + .url('https://api.example.com/items?page=1') + .build(); +} + +/** Serves page 1 normally, then fails the page-2 exchange, so a walk breaks mid-stream rather than at the head. */ +function createFailingTransport(failure: Error): Transport { + return { + send(req: Request): Promise<Response> { + if (Number(req.url.searchParams.get('page') ?? '1') >= 2) { + return Promise.reject(failure); + } + return Promise.resolve( + Response.newBuilder() + .request(req) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .build(), + ); + }, + close(): Promise<void> { + return Promise.resolve(); + }, + }; +} + +describe('pageItems$', () => { + test('emits every item across all pages in order', async () => { + const {transport} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const items = await firstValueFrom(pageItems$(paginator).pipe(toArray())); + expect(items).toEqual(['item_1_1', 'item_1_2', 'item_2_1', 'item_2_2']); + }); + + test('is cold and repeatable: two subscriptions each drive a fresh fetch sequence', async () => { + const {transport, getSendCount} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const observable = pageItems$(paginator); + const firstWalk = await firstValueFrom(observable.pipe(toArray())); + expect(firstWalk).toEqual(['item_1_1', 'item_1_2', 'item_2_1', 'item_2_2']); + expect(getSendCount()).toBe(2); + + const secondWalk = await firstValueFrom(observable.pipe(toArray())); + expect(secondWalk).toEqual([ + 'item_1_1', + 'item_1_2', + 'item_2_1', + 'item_2_2', + ]); + expect(getSendCount()).toBe(4); // Re-fetched both pages on second subscription + }); + + test('unsubscribing mid-walk cancels the page iteration cleanly (ASYNC-6)', async () => { + const {transport, getSendCount} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const items: string[] = []; + await new Promise<void>(resolve => { + const subscription = pageItems$(paginator).subscribe({ + next(item) { + items.push(item); + if (items.length === 2) { + subscription.unsubscribe(); + resolve(); + } + }, + }); + }); + + expect(items).toEqual(['item_1_1', 'item_1_2']); + expect(getSendCount()).toBe(1); // Did not fetch page 2 + }); +}); + +describe('pages$', () => { + test('emits whole Page objects across all pages in order', async () => { + const {transport} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const pages = await firstValueFrom(pages$(paginator).pipe(toArray())); + expect(pages).toHaveLength(2); + const p0 = pages[0]; + const p1 = pages[1]; + expect(p0).toBeDefined(); + expect(p1).toBeDefined(); + if (p0 === undefined || p1 === undefined) { + throw new Error('expected 2 pages'); + } + expect(p0.items).toEqual(['item_1_1', 'item_1_2']); + expect(p0.status.code).toBe(200); + expect(p1.items).toEqual(['item_2_1', 'item_2_2']); + expect(p1.status.code).toBe(200); + }); + + test('is cold and repeatable for whole pages', async () => { + const {transport, getSendCount} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const observable = pages$(paginator); + await firstValueFrom(observable.pipe(toArray())); + expect(getSendCount()).toBe(2); + + await firstValueFrom(observable.pipe(toArray())); + expect(getSendCount()).toBe(4); + }); + + test('unsubscribing after page 1 cancels further fetches (ASYNC-6)', async () => { + const {transport, getSendCount} = createMockTransport(); + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const received: unknown[] = []; + await new Promise<void>(resolve => { + const subscription = pages$(paginator).subscribe({ + next(page) { + received.push(page); + subscription.unsubscribe(); + resolve(); + }, + }); + }); + + expect(received).toHaveLength(1); + expect(getSendCount()).toBe(1); + }); +}); + +describe('pagination error propagation (ASYNC-13)', () => { + test('a transport failure mid-walk reaches the error channel unwrapped, after the items already delivered', async () => { + const failure = new TypeError('the page-2 exchange failed'); + const paginator = new Paginator({ + transport: createFailingTransport(failure), + initialRequest: createInitialRequest(), + strategy: createTwoPageStrategy(), + }); + + const items: string[] = []; + const errors: unknown[] = []; + await new Promise<void>(resolve => { + pageItems$(paginator).subscribe({ + next(item) { + items.push(item); + }, + error(err: unknown) { + errors.push(err); + resolve(); + }, + }); + }); + + // Page 1's items are not rolled back by page 2's failure -- the error is a terminal signal, not an undo. + expect(items).toEqual(['item_1_1', 'item_1_2']); + expect(errors).toHaveLength(1); + // The exact instance, not an RxJS-internal or PaginationError wrapper. + expect(errors[0]).toBe(failure); + }); + + test('a strategy failure reaches pages$ error channel unwrapped', async () => { + const failure = new RangeError('cannot parse this page'); + const {transport} = createMockTransport(); + const strategy: PaginationStrategy<string> = { + parse(): Promise<PageInfo<string>> { + return Promise.reject(failure); + }, + }; + const paginator = new Paginator({ + transport, + initialRequest: createInitialRequest(), + strategy, + }); + + const errors: unknown[] = []; + await new Promise<void>(resolve => { + pages$(paginator).subscribe({ + next() { + // ignore + }, + error(err: unknown) { + errors.push(err); + resolve(); + }, + }); + }); + + expect(errors).toHaveLength(1); + expect(errors[0]).toBe(failure); + }); +}); diff --git a/packages/rx/src/pagination.ts b/packages/rx/src/pagination.ts new file mode 100644 index 0000000..11bd645 --- /dev/null +++ b/packages/rx/src/pagination.ts @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/pagination.ts +import type {Observable} from 'rxjs'; +import type {Page, Paginator} from '@dexpace/core'; +import {fromAsyncIterable} from './from-async-iterable.js'; + +/** + * Bridges a {@link @dexpace/core#Paginator}'s item stream to a cold, repeatable RxJS `Observable` (PAGE-8). + * + * Each subscription obtains a fresh iterator from `Paginator.items()`, driving an independent pagination sequence + * across all pages. + * + * @param paginator - The `Paginator` instance whose items to observe. + * @returns An `Observable` emitting items of type `T` in server order. + * + * @public + */ +export function pageItems$<T>(paginator: Paginator<T>): Observable<T> { + return fromAsyncIterable({ + [Symbol.asyncIterator]: () => paginator.items()[Symbol.asyncIterator](), + }); +} + +/** + * Bridges a {@link @dexpace/core#Paginator}'s page stream to a cold, repeatable RxJS `Observable` (PAGE-8). + * + * Each subscription obtains a fresh iterator from `Paginator.pages()`, driving an independent pagination sequence + * yielding whole {@link @dexpace/core#Page} objects. + * + * @param paginator - The `Paginator` instance whose pages to observe. + * @returns An `Observable` emitting {@link @dexpace/core#Page} objects. + * + * @public + */ +export function pages$<T>(paginator: Paginator<T>): Observable<Page<T>> { + return fromAsyncIterable({ + [Symbol.asyncIterator]: () => paginator.pages()[Symbol.asyncIterator](), + }); +} diff --git a/packages/rx/src/sse.test.ts b/packages/rx/src/sse.test.ts new file mode 100644 index 0000000..08425b4 --- /dev/null +++ b/packages/rx/src/sse.test.ts @@ -0,0 +1,481 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/sse.test.ts +// +// Exercises: SSE-41 (reactive adapter), SSE-26 (single-pass: second subscription fails loudly), +// SSE-33-36 (typed adapter mapping over reactive stream), ASYNC-21, ASYNC-6, SSE-28 (idempotent release), +// SSE-30 (quiet automatic release). +// +// The `resource ownership` blocks pin the deliberate departure from ASYNC-21's "MUST NOT close the +// caller-owned source on any termination" clause -- see the ASYNC-21 row of `docs/deviations.md`. They count +// the release the OWNED RESOURCE sees, never `SseStream.close()` calls: `close()` memoizes its release +// promise (SSE-28), so a facade-level count reads "once" no matter how many paths call it. +import {describe, expect, test} from 'bun:test'; +import {firstValueFrom, toArray} from 'rxjs'; +import { + Protocol, + Request, + Response, + SseLineTooLongError, + SseStream, + SseStreamError, + sseStreamFrom, + Status, +} from '@dexpace/core'; +import {sseEvents$, typedSse$} from './sse.js'; + +/** Distinguishes "the promise resolved" from a rejection value that happens to be falsy. */ +const RESOLVED = Symbol('resolved'); + +/** + * Settles `promise` and hands back whatever it rejected with. + * + * `expect(p).rejects.toX()` is typed `void` under `bun:test`, so awaiting it trips `await-thenable` -- the same + * idiom `@dexpace/codec-json`'s `json-serde.test.ts` settled on. + */ +async function rejection(promise: Promise<unknown>): Promise<unknown> { + try { + await promise; + return RESOLVED; + } catch (e: unknown) { + return e; + } +} + +function makeSseStreamFixture(text: string): SseStream { + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(body) + .build(); + return sseStreamFrom(response); +} + +function makeUnclosedSseStream(text: string, onCancel?: () => void): SseStream { + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const body = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + }, + cancel() { + onCancel?.(); + }, + }); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(body) + .build(); + return sseStreamFrom(response); +} + +/** + * The two halves of the ONE resource `sseStreamFrom` hands the facade -- `closingBoth(source, response)` in + * core's `sse/stream.ts`. `socket` is the byte stream's own teardown, which the platform invokes at most once + * per stream and not at all once a producer has already ended it, so it corroborates rather than carries the + * count. + */ +interface ReleaseCounts { + /** `BufferedSource.close()` -> `RetentionWindow.close()` -> `reader.cancel()`. */ + source: number; + /** `Response.close()` -> `body.cancel()`. */ + response: number; + /** The `ReadableStream`'s own `cancel` hook. */ + socket: number; +} + +interface CountingSseStream { + readonly stream: SseStream; + readonly releases: ReleaseCounts; + /** Settles the first time the byte stream itself is torn down. */ + readonly socketTornDown: Promise<void>; +} + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both levels + * `sseStreamFrom` uses. + * + * A structural double rather than a subclass, because `ResponseBuilder.body()` stores what it is handed and + * runs no `instanceof` check. A platform `ReadableStream` cannot do this job on its own: its underlying + * `cancel` hook is invoked at most once by specification and never at all after the producer closed the + * controller, so a second release would collapse into the first and read as clean. + */ +function countingBody( + bytes: ReadableStream<Uint8Array>, + releases: ReleaseCounts, +): ReadableStream<Uint8Array> { + const reader = (): ReadableStreamDefaultReader<Uint8Array> => { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => { + real.releaseLock(); + }, + cancel: async (reason?: unknown) => { + releases.source += 1; + return real.cancel(reason); + }, + } as unknown as ReadableStreamDefaultReader<Uint8Array>; + }; + return { + get locked(): boolean { + return bytes.locked; + }, + getReader: reader, + cancel: async (reason?: unknown) => { + releases.response += 1; + return bytes.cancel(reason); + }, + } as unknown as ReadableStream<Uint8Array>; +} + +/** + * An `SseStream` over a body that reports every release it is asked for. + * + * `ended: false` leaves the producer's controller open, which is the idle state a live event stream sits in + * between events -- the reader stays suspended in a pull and only a cancel can settle it. + */ +function makeCountingSseStream( + text: string, + options: {readonly ended: boolean; readonly maxLineBytes?: number}, +): CountingSseStream { + const releases: ReleaseCounts = {source: 0, response: 0, socket: 0}; + let tornDown = (): void => undefined; + const socketTornDown = new Promise<void>(resolve => { + tornDown = resolve; + }); + const bytes = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (options.ended) controller.close(); + }, + cancel() { + releases.socket += 1; + tornDown(); + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, releases)) + .build(); + return { + stream: sseStreamFrom(response, {maxLineBytes: options.maxLineBytes}), + releases, + socketTornDown, + }; +} + +/** Lets the adapter's teardown, which is driven off the microtask queue, run to completion. */ +const settle = async (): Promise<void> => { + await new Promise(resolve => setTimeout(resolve, 20)); +}; + +/** + * Awaits `promise`, failing with a line number rather than hanging when it never settles. + * + * The suspended-pull case regresses as a *hang*, and a bare `await` would surface that as a bare runner + * timeout naming no assertion. + */ +async function within(ms: number, promise: Promise<void>): Promise<void> { + let timer: ReturnType<typeof setTimeout> | undefined; + const deadline = new Promise<never>((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`the teardown did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +describe('sseEvents$', () => { + test('emits every parsed SseEvent in order and completes at end-of-stream', async () => { + const stream = makeSseStreamFixture('data: one\n\ndata: two\n\n'); + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + expect(events.map(e => e.data)).toEqual([['one'], ['two']]); + }); + + test('a second subscription fails loudly (SSE-26, inherited)', async () => { + const stream = makeSseStreamFixture('data: one\n\n'); + const observable = sseEvents$(stream); + await firstValueFrom(observable.pipe(toArray())); + + // SseStream's own single-pass guard, surfaced through the error channel rather than reimplemented. + expect( + await rejection(firstValueFrom(observable.pipe(toArray()))), + ).toBeInstanceOf(SseStreamError); + }); + + test('unsubscribing mid-stream synchronously releases the underlying stream resource', async () => { + let cancelCalled = false; + const stream = makeUnclosedSseStream( + 'data: one\n\ndata: two\n\ndata: three\n\n', + () => { + cancelCalled = true; + }, + ); + const subscription = sseEvents$(stream).subscribe({ + next(event) { + if (event.data[0] === 'one') { + subscription.unsubscribe(); + } + }, + }); + await new Promise(r => setTimeout(r, 20)); + expect(cancelCalled).toBe(true); + }); + + test('unsubscribing asynchronously while idle releases the underlying stream resource (ASYNC-6)', async () => { + let cancelCalled = false; + const stream = makeUnclosedSseStream('data: one\n\n', () => { + cancelCalled = true; + }); + const received: string[] = []; + const subscription = sseEvents$(stream).subscribe({ + next(event) { + const item = event.data[0]; + if (item !== undefined) { + received.push(item); + } + }, + }); + + await new Promise(r => setTimeout(r, 10)); + expect(received).toEqual(['one']); + expect(cancelCalled).toBe(false); + + subscription.unsubscribe(); + await new Promise(r => setTimeout(r, 20)); + expect(cancelCalled).toBe(true); + }); +}); + +describe('typedSse$', () => { + test('decodes events and honors Value, Skip, and Done outcomes', async () => { + const stream = makeSseStreamFixture( + ':ping\n\nevent: delta\ndata: {"num":1}\n\nevent: delta\ndata: {"num":2}\n\nevent: done\ndata: end\n\ndata: ignored\n\n', + ); + const observable = typedSse$(stream, (event, data) => { + if (event === undefined) return {kind: 'skip'}; + if (event === 'done') return {kind: 'done'}; + const parsed = JSON.parse(data) as {num: number}; + return {kind: 'value', value: parsed.num}; + }); + + const values = await firstValueFrom(observable.pipe(toArray())); + expect(values).toEqual([1, 2]); + }); + + test('a throwing mapper propagates error through the Observable error channel', async () => { + const stream = makeSseStreamFixture('data: invalid-json\n\n'); + const observable = typedSse$(stream, (_event, data) => { + if (data === 'invalid-json') { + throw new TypeError('invalid json payload'); + } + return {kind: 'value', value: data}; + }); + + const errors: unknown[] = []; + await new Promise<void>(resolve => { + observable.subscribe({ + next() { + // ignore + }, + error(err: unknown) { + errors.push(err); + resolve(); + }, + }); + }); + + expect(errors).toHaveLength(1); + expect(errors[0]).toBeInstanceOf(TypeError); + expect((errors[0] as TypeError).message).toBe('invalid json payload'); + }); + + test('unsubscribing asynchronously from typedSse$ releases the underlying stream (ASYNC-6)', async () => { + let cancelCalled = false; + const stream = makeUnclosedSseStream('data: 100\n\n', () => { + cancelCalled = true; + }); + const subscription = typedSse$(stream, (_e, d) => ({ + kind: 'value', + value: Number(d), + })).subscribe({ + next() { + // ignore + }, + }); + + await new Promise(r => setTimeout(r, 10)); + expect(cancelCalled).toBe(false); + + subscription.unsubscribe(); + await new Promise(r => setTimeout(r, 20)); + expect(cancelCalled).toBe(true); + }); +}); + +describe('sseEvents$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\n', + {ended: true}, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + expect(events).toHaveLength(2); + await settle(); + // `socket: 0` is not a miss: the producer ended the byte stream, so there is nothing left for the + // platform to tear down. The two halves the facade owns are still released, once each. + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a source error releases each half exactly once and surfaces the error unwrapped', async () => { + const {stream, releases} = makeCountingSseStream( + `data: ${'x'.repeat(64)}\n\n`, + {ended: false, maxLineBytes: 16}, + ); + + const failure = await rejection(firstValueFrom(sseEvents$(stream))); + + // SSE-29 releases before the error propagates; the adapter's own release then finds it already done. + expect(failure).toBeInstanceOf(SseLineTooLongError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('early unsubscribe releases each half exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\ndata: three\n\n', + {ended: false}, + ); + + const subscription = sseEvents$(stream).subscribe({ + next() { + subscription.unsubscribe(); + }, + }); + + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: one\n\n', + {ended: false}, + ); + const received: string[] = []; + const subscription = sseEvents$(stream).subscribe({ + next(event) { + const item = event.data[0]; + if (item !== undefined) received.push(item); + }, + }); + + await settle(); + expect(received).toEqual(['one']); + expect(releases.socket).toBe(0); + + // The server will never send another byte, so the reader is parked in a pull. Only the release running + // AHEAD of `iterator.return()` settles it -- a `return()` on an async generator queues behind the + // in-flight `next()`. Drop the release from `sseEvents$` and this never resolves. + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); + +describe('typedSse$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream('data: 1\n\ndata: 2\n\n', { + ended: true, + }); + + const values = await firstValueFrom( + typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).pipe(toArray()), + ); + + expect(values).toEqual([1, 2]); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a throwing mapper releases each half exactly once (SSE-36)', async () => { + const {stream, releases} = makeCountingSseStream('data: one\n\n', { + ended: false, + }); + + // Three release paths converge here: `runMapper`'s explicit `close()`, the adapter's `release`, and the + // mapping generator's `return()` unwinding into the facade's own quiet release. + const failure = await rejection( + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + expect(failure).toBeInstanceOf(TypeError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown through the mapping generator', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: 100\n\n', + {ended: false}, + ); + const subscription = typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).subscribe({ + next() { + // ignore + }, + }); + + await settle(); + expect(releases.socket).toBe(0); + + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); diff --git a/packages/rx/src/sse.ts b/packages/rx/src/sse.ts new file mode 100644 index 0000000..c7686e0 --- /dev/null +++ b/packages/rx/src/sse.ts @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: MIT +// packages/rx/src/sse.ts +import type {Observable} from 'rxjs'; +import { + typedSseStream, + type SseEvent, + type SseMapper, + type SseStream, +} from '@dexpace/core'; +import {fromAsyncIterable} from './from-async-iterable.js'; + +/** + * Bridges an {@link @dexpace/core#SseStream} to an RxJS `Observable` (SSE-41, ASYNC-21). + * + * **Subscribing transfers ownership of `stream` to the returned `Observable`.** The adapter releases it on + * every termination path -- unsubscription, end-of-source and a source error alike (ASYNC-6). Do not call + * `stream.close()` yourself, and do not iterate `stream` after passing it here: its iterator may be taken at + * most once (SSE-26) and this function takes it, so the subscription's lifetime is the stream's lifetime. + * Closing it out from under a live subscription corrupts nothing -- `close()` is idempotent (SSE-28) -- but it + * ends the stream from the wrong end; `unsubscribe()` is the supported way. + * + * That transfer is a deliberate departure from ASYNC-21's "MUST NOT close the caller-owned source on any + * termination" clause, recorded in this project's deviation register. It is what releases the response body + * when a subscriber unsubscribes from an *idle* stream -- the state a long-lived event stream sits in almost + * all of the time -- because an async generator's `return()` queues behind the suspended pull it would + * otherwise have to interrupt, and so cannot settle one on its own. + * + * Single-subscription: `SseStream` wraps an already-open, single-use HTTP response body (BODY-14) and is itself + * single-pass (SSE-26) -- obtaining an iterator succeeds at most once and a second attempt fails loudly. + * Subscribing to the returned `Observable` a second time reaches `SseStream`'s own guard and surfaces an error + * through the `Observable`'s error channel. + * + * Diagnostic context propagates on its own (ASYNC-8–ASYNC-11): every pull runs inside the continuation chain + * that called `subscribe()`, which is exactly what Node's `AsyncLocalStorage` tracks. That holds only for the + * `Observable` returned here -- a caller who pipes it through an RxJS scheduler operator (`observeOn`, + * `subscribeOn`) hands each emission to a task outside that chain, and owns reinstating the context on the far + * side. This package installs no scheduler of its own, precisely so that boundary is never introduced behind + * the caller's back. + * + * @param stream - The `SseStream` instance to observe. + * @returns An `Observable` emitting parsed {@link @dexpace/core#SseEvent}s. + * + * @public + */ +export function sseEvents$(stream: SseStream): Observable<SseEvent> { + return fromAsyncIterable(stream, () => stream.close()); +} + +/** + * Bridges an {@link @dexpace/core#SseStream} to a typed RxJS `Observable` via an {@link @dexpace/core#SseMapper} (SSE-41, ASYNC-21, SSE-33–SSE-36). + * + * **Subscribing transfers ownership of `stream` to the returned `Observable`,** exactly as in + * {@link sseEvents$} and for the same reasons: the adapter releases the stream on every termination path, so + * do not call `stream.close()` yourself and do not iterate `stream` after passing it here. The mapper sits + * between the stream and the subscriber; it does not change who owns the stream. + * + * Shares every other note on {@link sseEvents$} too: single-subscription, and automatic diagnostic-context + * propagation through the unscheduled path. + * + * A throwing mapper reaches the `Observable`'s error channel unwrapped (ASYNC-13), after `typedSseStream` has + * released the stream (SSE-36). + * + * @param stream - The `SseStream` instance to observe. + * @param mapper - The mapper decoding raw SSE events into domain items, skips, or done sentinels. + * @returns An `Observable` emitting decoded items of type `T`. + * + * @public + */ +export function typedSse$<T>( + stream: SseStream, + mapper: SseMapper<T>, +): Observable<T> { + return fromAsyncIterable(typedSseStream(stream, mapper), () => + stream.close(), + ); +} diff --git a/packages/rx/tsconfig.build.json b/packages/rx/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/rx/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/rx/tsconfig.json b/packages/rx/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/rx/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/shrink-test/package.json b/packages/shrink-test/package.json new file mode 100644 index 0000000..ef09166 --- /dev/null +++ b/packages/shrink-test/package.json @@ -0,0 +1,15 @@ +{ + "name": "@dexpace/shrink-test", + "version": "0.0.0", + "private": true, + "type": "module", + "devDependencies": { + "@dexpace/codec-json": "workspace:*", + "@dexpace/core": "workspace:*", + "@dexpace/transport-fetch": "workspace:*", + "esbuild": "^0.28.2" + }, + "scripts": { + "test": "bun test" + } +} diff --git a/packages/shrink-test/shrink-test.config.ts b/packages/shrink-test/shrink-test.config.ts new file mode 100644 index 0000000..be3b72b --- /dev/null +++ b/packages/shrink-test/shrink-test.config.ts @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/shrink-test.config.ts + +/** The knobs {@link SHRINK_TEST_CONFIG} fixes for the `NFR-9` guard. */ +export interface ShrinkTestConfig { + /** + * The hard ceiling `run-shrink-guard.ts` fails the build on, in bytes of minified, tree-shaken + * output. Not a footprint target: the number exists to catch a *regression in shape* -- a barrel + * that stops tree-shaking, a side-effectful module pulled in wholesale -- not to police normal + * growth. Raise it only with the measured before/after in the commit message. + */ + readonly budgetBytes: number; + /** + * The packages `fixture-app.ts` imports, and therefore the ones this guard proves survive a + * bundle-and-tree-shake round trip. Recorded here so the set is reviewable in one place rather + * than inferred from the fixture's import list. + */ + readonly participatingPackages: readonly string[]; +} + +/** + * Measured at 16,671 bytes on 2026-08-29, and 17,689 bytes on 2026-08-30 once the fixture also + * constructed a `Page` to probe the disposal-symbol install (esbuild 0.28.2; `@dexpace/core` + + * `@dexpace/transport-fetch` + `@dexpace/codec-json`, all three reached through their published entry + * points). The budget is 24 KiB -- ~39% headroom, which absorbs ordinary growth while still catching + * the failure this guard exists for: a tree-shaking regression pulls in core's barrel wholesale and + * shows up as a multiple of this figure, not a few percent over it. A loose budget would catch + * nothing. + */ +export const SHRINK_TEST_CONFIG: ShrinkTestConfig = Object.freeze({ + budgetBytes: 24_576, + participatingPackages: Object.freeze([ + '@dexpace/core', + '@dexpace/transport-fetch', + '@dexpace/codec-json', + ]), +}); diff --git a/packages/shrink-test/src/bundle.test.ts b/packages/shrink-test/src/bundle.test.ts new file mode 100644 index 0000000..3e0192d --- /dev/null +++ b/packages/shrink-test/src/bundle.test.ts @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/src/bundle.test.ts +// Exercises: NFR-9 (the shrink-and-run guard's bundle half -- a real minify + tree-shake pass over +// the shipped packages, which is what the round-trip check in run-shrink-guard.test.ts then runs). +import {describe, expect, test} from 'bun:test'; +import {SHRINK_TEST_CONFIG} from '../shrink-test.config.js'; +import {buildShrinkBundle} from './bundle.js'; + +describe('buildShrinkBundle', () => { + test('produces a single bundle within the configured budget', async () => { + const {code, bytes} = await buildShrinkBundle(); + + expect(code.length).toBeGreaterThan(0); + expect(bytes).toBeLessThanOrEqual(SHRINK_TEST_CONFIG.budgetBytes); + }); + + test('minifies, rather than emitting the readable source verbatim', async () => { + const {code} = await buildShrinkBundle(); + + // Source indentation would survive verbatim if `minify` silently stopped applying. + expect(code).not.toContain('\n runFixtureApp'); + }); + + test('tree-shakes, rather than inlining every package the workspace publishes', async () => { + const {bytes} = await buildShrinkBundle(); + + // The fixture touches a narrow slice of core. Pulling the barrel in wholesale -- the regression + // this guard exists to catch -- lands as a multiple of the budget, not a few bytes over it. + expect(bytes).toBeLessThan(SHRINK_TEST_CONFIG.budgetBytes * 2); + }); +}); diff --git a/packages/shrink-test/src/bundle.ts b/packages/shrink-test/src/bundle.ts new file mode 100644 index 0000000..48ac81f --- /dev/null +++ b/packages/shrink-test/src/bundle.ts @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/src/bundle.ts +import {fileURLToPath} from 'node:url'; +import {build} from 'esbuild'; + +/** The in-memory result of one bundle-and-minify pass; nothing is written to disk here. */ +export interface ShrinkBundle { + /** The bundled, minified, tree-shaken ESM source. */ + readonly code: string; + /** Its size in bytes, as the budget in `shrink-test.config.ts` measures it. */ + readonly bytes: number; +} + +/** + * Bundles `fixture-app.ts` and everything it imports into one minified, tree-shaken ESM module, the + * way a downstream consumer's bundler would. + * + * `write: false` keeps the pass in memory -- `run-shrink-guard.ts` is the only caller that needs the + * bytes on disk, and it writes them to a temp dir it owns. `platform: 'node'` matches the runtime the + * guard then executes the output on, so `node:` builtins stay external instead of being inlined or + * shimmed. + * + * @returns the bundled code and its byte length. + * @throws Error - when esbuild reports success but produces no output file, which would otherwise + * surface later as an unreadable `undefined` and be mistaken for a size regression. + */ +export async function buildShrinkBundle(): Promise<ShrinkBundle> { + const entryPoint = fileURLToPath( + new URL('./fixture-app.ts', import.meta.url), + ); + const result = await build({ + entryPoints: [entryPoint], + bundle: true, + minify: true, + treeShaking: true, + platform: 'node', + format: 'esm', + write: false, + }); + + const output = result.outputFiles[0]; + if (output === undefined) { + throw new Error('esbuild reported success but produced no output file'); + } + return {code: output.text, bytes: output.contents.byteLength}; +} diff --git a/packages/shrink-test/src/fixture-app.ts b/packages/shrink-test/src/fixture-app.ts new file mode 100644 index 0000000..3163b71 --- /dev/null +++ b/packages/shrink-test/src/fixture-app.ts @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/src/fixture-app.ts +import {jsonSerde} from '@dexpace/codec-json'; +import { + IoError, + Page, + Request, + type Response, + type Schema, + type Transport, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +/** + * What {@link runFixtureApp} reports back to the guard from the child process. + * + * Every field is a boolean the guard requires to be `true`; the generated runner fails on any `false` + * one without needing to be edited, so a new probe only has to be added here. + */ +export interface FixtureResult { + /** True when the error thrown by `transport-fetch` matched `IoError` imported from `core`. */ + readonly caughtViaCoreImport: boolean; + /** True when a serialize/deserialize round trip through `codec-json` returned the input. */ + readonly serdeRoundTripOk: boolean; + /** True when the module-scope `[Symbol.asyncDispose]` installs survived the tree-shaking pass. */ + readonly disposalSymbolSurvived: boolean; +} + +/** The one shape the round trip carries; a hand-written `Schema` keeps the fixture codec-agnostic. */ +interface ShrinkProbe { + readonly shrinkTest: boolean; +} + +/** + * `Deserializer.deserialize` takes a `Schema<T>` witness rather than a reflected type token + * (`docs/sdk-design-nodejs/10` item 7's schema-as-witness substitution), so the fixture supplies a + * minimal one instead of reaching for a validation library it would then have to bundle. + */ +const shrinkProbeSchema: Schema<ShrinkProbe> = { + parse(input: unknown): ShrinkProbe { + if ( + typeof input !== 'object' || + input === null || + typeof (input as {shrinkTest?: unknown}).shrinkTest !== 'boolean' + ) { + throw new TypeError('not a ShrinkProbe'); + } + return {shrinkTest: (input as ShrinkProbe).shrinkTest}; + }, +}; + +/** + * The narrowest stand-in for the three fields `Page`'s constructor reads. + * + * A real `Response` needs a live transport exchange to produce, and {@link probeDisposalSymbol} only + * needs an instance whose prototype came out of the bundle -- the response is never read again. + */ +function stubResponse(): Response { + return { + status: {code: 204}, + headers: {get: (): undefined => undefined}, + request: {method: 'GET'}, + close: (): Promise<void> => Promise.resolve(), + } as unknown as Response; +} + +/** True when `value` carries a callable `[Symbol.asyncDispose]`, however it was installed. */ +function hasAsyncDispose(value: object, disposeSymbol: symbol): boolean { + return ( + typeof (value as Record<symbol, unknown>)[disposeSymbol] === 'function' + ); +} + +/** + * Proves that the module-scope `[Symbol.asyncDispose]` installs survive a bundle round trip. + * + * `Page` and `FetchTransport` (and `SseStream`, and `UndiciTransport`) do not declare disposal as a + * class member: Node 20.3 is the workspace floor and predates the symbol, so declaring it would emit + * a `.d.ts` promise the floor cannot keep (NFR-10). The method is instead installed by a guarded + * `Object.defineProperty` **statement that runs when the module is evaluated** -- a module-level side + * effect, in packages that all declare `"sideEffects": false`. + * + * That manifest field entitles a bundler to drop a module whose exports go unused, and nothing stops + * a future one from also dropping a top-level statement it judges inert. Here the classes *are* used, + * so the modules are kept and the install runs; this asserts that outcome rather than assuming it, + * inside the same real `bundle + minify + treeShaking` pass the rest of the guard uses. + * + * Read through a cast rather than a bare `Symbol.asyncDispose` index, matching the guarded install: + * on the declared floor the symbol is `undefined` and the index would read the string key + * `"undefined"`. Absent symbol means the install is *supposed* to leave nothing behind, so the probe + * is vacuously true there. + */ +function probeDisposalSymbol(transport: Transport): boolean { + const disposeSymbol = (Symbol as {asyncDispose?: symbol}).asyncDispose; + if (typeof disposeSymbol !== 'symbol') return true; + return ( + hasAsyncDispose(new Page(stubResponse(), []), disposeSymbol) && + hasAsyncDispose(transport, disposeSymbol) + ); +} + +/** + * Runs inside the bundled, tree-shaken artifact -- never against `src/` directly, which is the whole + * point (see `run-shrink-guard.ts`). + * + * Proves the three properties a bundler round trip can silently break. First, cross-package + * `instanceof`: `TransportFailureError` is thrown by `@dexpace/transport-fetch` and its base class + * `IoError` is imported here from `@dexpace/core`, so the check passes only if the bundle contains + * exactly ONE copy of core's class identity. Two copies -- the dual-package hazard + * `docs/knowledge/harvested/tooling-and-quality-gates.md` names, and the risk this port substitutes for the + * reference's reflective keep-rules (`NFR-8`, deviation-ledger item 10) -- make it silently false + * while every type still checks. Second, that a real serde round trip still works once the codec has + * been through the same minifier. Third, that the module-scope disposal installs are still there -- + * see {@link probeDisposalSymbol}. + * + * Port 1 is chosen because nothing listens there: the connection is refused immediately, so the + * guard needs no fixture server and cannot hang on a slow socket. + * + * @returns every check, for the parent process to assert on. + */ +export async function runFixtureApp(): Promise<FixtureResult> { + const transport = fetchTransport(); + let caughtViaCoreImport = false; + try { + await transport.send( + Request.newBuilder().url('http://127.0.0.1:1/').method('GET').build(), + ); + } catch (error) { + caughtViaCoreImport = error instanceof IoError; + } finally { + await transport.close(); + } + + const serde = jsonSerde(); + const bytes = serde.serializer.serialize({shrinkTest: true}); + const decoded = serde.deserializer.deserialize(bytes, { + schema: shrinkProbeSchema, + }); + + return { + caughtViaCoreImport, + serdeRoundTripOk: decoded.shrinkTest, + disposalSymbolSurvived: probeDisposalSymbol(transport), + }; +} diff --git a/packages/shrink-test/src/run-shrink-guard.test.ts b/packages/shrink-test/src/run-shrink-guard.test.ts new file mode 100644 index 0000000..030e79b --- /dev/null +++ b/packages/shrink-test/src/run-shrink-guard.test.ts @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/src/run-shrink-guard.test.ts +// Exercises: NFR-9 (shrink-and-run regression guard, wired into the default build via the root +// `shrink-test` script), NFR-17 (that gate is blocking, not advisory), PAGE-12 and NFR-10 (the +// guarded, module-scope `[Symbol.asyncDispose]` installs that keep the emitted artifact on the +// declared Node floor are still present and callable after tree-shaking -- see +// `fixture-app.ts`'s `probeDisposalSymbol`). +// Substitutes for NFR-8's keep-configuration, which this port ships nothing for by design -- see the +// Phase 9 deviation ledger and item 11 of +// docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md, the normative ledger +// (docs/deviations.md is its as-built audit). The knowledge corpus deliberately holds no copy: a +// register goes stale on the next append -- see docs/knowledge/notes/deliberate-deviations.md. +import {describe, expect, test} from 'bun:test'; +import {runShrinkGuard} from './run-shrink-guard.js'; + +describe('runShrinkGuard', () => { + test('the shrunk bundle stays within budget and still runs standalone', async () => { + const result = await runShrinkGuard(); + + expect(result.bundleBytes).toBeLessThanOrEqual(result.budgetBytes); + expect(result.roundTripSucceeded).toBe(true); + }); +}); diff --git a/packages/shrink-test/src/run-shrink-guard.ts b/packages/shrink-test/src/run-shrink-guard.ts new file mode 100644 index 0000000..49d66aa --- /dev/null +++ b/packages/shrink-test/src/run-shrink-guard.ts @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: MIT +// packages/shrink-test/src/run-shrink-guard.ts +import {spawn} from 'node:child_process'; +import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {SHRINK_TEST_CONFIG} from '../shrink-test.config.js'; +import {buildShrinkBundle} from './bundle.js'; + +/** The `NFR-9` guard's verdict: the size half and the still-runs half, reported together. */ +export interface ShrinkGuardResult { + /** Size of the minified, tree-shaken bundle. */ + readonly bundleBytes: number; + /** The ceiling from `shrink-test.config.ts` it must not exceed. */ + readonly budgetBytes: number; + /** True when the bundled artifact ran standalone and both of its checks passed. */ + readonly roundTripSucceeded: boolean; +} + +/** Runs `runnerPath` under this process's own Node and resolves true on a clean exit. */ +function runInChild(runnerPath: string): Promise<boolean> { + return new Promise<boolean>(resolve => { + const child = spawn(process.execPath, [runnerPath], {stdio: 'ignore'}); + child.on('error', () => { + resolve(false); + }); + child.on('exit', code => { + resolve(code === 0); + }); + }); +} + +/** + * The `NFR-9` regression guard: bundle, shrink, then actually run the result. + * + * The bundled code executes in a **child process**, not through `eval` or a dynamic import of this + * one. That is the entire point of the guard rather than an implementation detail -- importing it + * here would resolve `@dexpace/core` through this process's already-warm module graph and prove + * nothing about the artifact standing on its own. A separate `node` sees only the bytes esbuild + * emitted, which is what a downstream consumer ships. + * + * The child is spawned with `stdio: 'ignore'` and reports through its exit code alone; the runner it + * executes exits non-zero when any fixture check comes back false, so a stripped `instanceof` or a + * dropped `[Symbol.asyncDispose]` install surfaces as a failed guard rather than as parsed output + * this function would have to trust. Which check failed is not carried back — read `fixture-app.ts`, + * whose `FixtureResult` names them all. + * + * @returns the measured size, the configured budget, and whether the artifact still worked. The + * caller decides what fails the build -- see `run-shrink-guard.test.ts`. + */ +export async function runShrinkGuard(): Promise<ShrinkGuardResult> { + const {code, bytes} = await buildShrinkBundle(); + const dir = await mkdtemp(join(tmpdir(), 'dexpace-shrink-test-')); + try { + const entryPath = join(dir, 'bundle.mjs'); + const runnerPath = join(dir, 'runner.mjs'); + await writeFile(entryPath, code, 'utf8'); + await writeFile( + runnerPath, + [ + `import {runFixtureApp} from ${JSON.stringify(entryPath)};`, + 'const result = await runFixtureApp();', + // Every FixtureResult field is a check that must come back true, so this stays correct when + // the fixture grows a new probe -- there is no list here to forget to update. + 'const failed = Object.values(result).filter(ok => ok !== true);', + 'process.exit(failed.length === 0 ? 0 : 1);', + '', + ].join('\n'), + 'utf8', + ); + + const roundTripSucceeded = await runInChild(runnerPath); + return { + bundleBytes: bytes, + budgetBytes: SHRINK_TEST_CONFIG.budgetBytes, + roundTripSucceeded, + }; + } finally { + await rm(dir, {recursive: true, force: true}); + } +} diff --git a/packages/shrink-test/tsconfig.json b/packages/shrink-test/tsconfig.json new file mode 100644 index 0000000..258bbb9 --- /dev/null +++ b/packages/shrink-test/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": ".", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts", + "shrink-test.config.ts" + ] +} diff --git a/packages/transport-conformance/package.json b/packages/transport-conformance/package.json new file mode 100644 index 0000000..c7b4f85 --- /dev/null +++ b/packages/transport-conformance/package.json @@ -0,0 +1,16 @@ +{ + "name": "@dexpace/transport-conformance", + "version": "0.0.0", + "private": true, + "type": "module", + "exports": { + ".": "./src/index.ts" + }, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "devDependencies": { + "@dexpace/core": "workspace:*" + } +} diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts new file mode 100644 index 0000000..c4a1bf7 --- /dev/null +++ b/packages/transport-conformance/src/fixtures.ts @@ -0,0 +1,300 @@ +// SPDX-License-Identifier: MIT +// packages/transport-conformance/src/fixtures.ts +import {createReadStream} from 'node:fs'; +import { + createServer, + type IncomingMessage, + type Server, + type ServerResponse, +} from 'node:http'; +import type {FileBodyDescriptor} from '@dexpace/core'; + +/** A running fixture server, addressable by URL and shut down through {@link TestServer.close}. */ +export interface TestServer { + /** The origin every fixture path is resolved against, e.g. `http://127.0.0.1:38211`. */ + readonly url: string; + /** Stops listening and resolves once the server has released its port. */ + close(): Promise<void>; +} + +/** How long `/slow` stalls before answering -- long enough that no timeout under test wins the race by luck. */ +const SLOW_RESPONSE_MS = 5_000; +/** `/drip`'s inter-chunk gap: long enough that a close-without-read happens mid-body, short enough not to pace the suite. */ +const DRIP_INTERVAL_MS = 50; +const DRIP_CHUNKS = 20; + +/** + * `/repeated-challenge`'s two `WWW-Authenticate` lines, in wire order. Exported so the suite asserts + * against what the server actually sent rather than against a second copy that can drift from it. + * + * The first algorithm is deliberately one no SDK handler supports, so a transport that keeps only the + * first value produces a challenge nothing can answer — the shape audit #67 / #74 found. + */ +export const REPEATED_CHALLENGES: readonly string[] = [ + 'Digest realm="conformance", nonce="n1", algorithm=SHA-512-256', + 'Digest realm="conformance", nonce="n1", algorithm=SHA-256, qop="auth"', +]; + +/** + * `/fixed-length`'s payload. Its length is what a HEAD response advertises and does not deliver, so + * it is exported: the row asserts the header survived the body-less decision rather than asserting + * a number written twice. + */ +export const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + +/** `/not-modified`'s validator, the one header a 304 exists to carry. */ +export const NOT_MODIFIED_ETAG = '"conformance-v1"'; + +/** + * The three fixtures whose responses can carry no body at all, in their own function because the + * main switch is at the 70-line lint cap -- and because they are one topic (TRANSPORT-24/25). + * `req` is not needed: `node:http` suppresses the body of a HEAD response by itself. + * + * @param pathname - the requested path. + * @param res - the response to write. + * @returns `true` when this function answered, `false` to fall through to {@link route}. + */ +function routeBodyless(pathname: string, res: ServerResponse): boolean { + switch (pathname) { + case '/no-content': + // TRANSPORT-24 with the WHATWG null-body rule: a 204 has no body and no framing to describe + // one. Node's `node:http` sends no `Content-Length` here at all; Bun 1.3.14's sends `0`. The + // row therefore asserts the header is absent-or-zero, never a positive length -- what a + // transport is answerable for is the body SHAPE, which is the same on both. + res.writeHead(204); + res.end(); + return true; + case '/not-modified': + // Deliberately WITHOUT a `Content-Length`, though RFC 9110 15.4.5 permits a 304 to carry the + // one a 200 would have had. undici 6.28.0 believes it: a 304 declaring 17 bytes leaves the + // dispatcher waiting for a body that cannot come, and the exchange dies with + // `UND_ERR_SOCKET: other side closed` (measured 2026-09-05, Node and Bun alike). That is + // undici's bug to have, not this suite's to provoke -- the row is about the ETag surviving. + res.writeHead(304, {etag: NOT_MODIFIED_ETAG}); + res.end(); + return true; + case '/fixed-length': + // The HEAD row's target. The declared length describes the body a GET would return, so the + // header promises bytes the HEAD response will not deliver: a transport that framed a stream + // from it hands the caller a read that never completes. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return true; + default: + return false; + } +} + +function route( + pathname: string, + req: IncomingMessage, + res: ServerResponse, +): void { + if (routeBodyless(pathname, res)) return; + switch (pathname) { + case '/echo-headers': + res.writeHead(200, {'content-type': 'application/json'}); + res.end(JSON.stringify(req.headers)); + return; + case '/echo-body': { + const chunks: Buffer[] = []; + req.on('data', (chunk: Buffer) => chunks.push(chunk)); + req.on('end', () => { + res.writeHead(200, {'content-type': 'application/octet-stream'}); + res.end(Buffer.concat(chunks)); + }); + return; + } + case '/early-response': + // Answers without ever draining the request body, so a streaming producer is still running + // when the response is delivered -- the window TRANSPORT-19's post-delivery clause lives in. + // + // `connection: close` because RFC 7230 6.3 requires it of a server that answers before + // draining the request body: the unread remainder would otherwise sit in a reusable socket and + // be parsed as the start line of whatever request came next. It is hygiene, not a fix -- the + // client is free to ignore it, and Bun 1.3.14 did, serving the resulting 400 to a later row + // from its own pool no matter what this server did (a socket destroy here changed nothing). + // The row that provokes this therefore runs against its own origin -- see `isolatedUrl` in + // run-suite.ts, which is what actually contains it. + res.writeHead(413, {'content-type': 'text/plain', connection: 'close'}); + res.end('too large'); + return; + case '/vendor-status': + res.writeHead(520, {'content-type': 'text/plain'}); + res.end('vendor status body'); + return; + case '/malformed-content-type': + // TRANSPORT-27: a syntactically invalid media type and a chunked (length-less) body. + // + // The chunked framing is *derived*, never declared: writing the body before `end()` with no + // declared length leaves the server no way to precompute one, so it must fall back to chunked. + // Setting `transfer-encoding: chunked` by hand looks more direct and is a trap -- Bun 1.3.14 + // (`.bun-version`, so exactly what CI runs) honours the header in the status line but still + // appends `Content-Length: 4` and writes the body UNCHUNKED. That response is malformed twice + // over, and the two transports disagree about how: undici rejects it with "Response body length + // does not match content-length header", while Bun's own `fetch` blocks for the chunk framing + // that never arrives until the test times out. Bun 1.4.0 emits it correctly, which is why this + // reproduced only on CI. Verified byte-for-byte on Bun 1.3.14, Bun 1.4.0, and Node 20.3.0. + res.writeHead(200, {'content-type': 'not-a-media-type'}); + res.write('body'); + res.end(); + return; + case '/drip': { + // Headers land immediately, the body trickles: the shape a lazily-streamed response body and an + // orphaned-response cleanup both need (TRANSPORT-9, TRANSPORT-25, SEAM-30). + res.writeHead(200, {'content-type': 'application/octet-stream'}); + let sent = 0; + const timer = setInterval(() => { + sent += 1; + if (sent >= DRIP_CHUNKS) { + clearInterval(timer); + res.end('end'); + return; + } + res.write(`chunk-${String(sent)};`); + }, DRIP_INTERVAL_MS); + res.on('close', () => { + clearInterval(timer); + }); + return; + } + case '/slow': + // Nothing is written at all, so a request against it is still awaiting response headers when + // the timeout or abort under test fires. + setTimeout(() => { + res.writeHead(200); + res.end('done'); + }, SLOW_RESPONSE_MS).unref(); + return; + case '/repeated-challenge': + // AUTH-12/AUTH-25: the same challenge header sent TWICE, which RFC 9110 5.3 permits for any + // list-valued field and RFC 7616 3.3 recommends for Digest algorithm discovery -- one challenge + // per algorithm, strongest first. The two transports legitimately surface it differently: + // WHATWG `Headers` comma-joins every name but `Set-Cookie`, so `@dexpace/transport-fetch` + // delivers one entry, while undici arrays any repeated header and `@dexpace/transport-undici` + // keeps two. Neither may LOSE one, which is what the row asserts. + // + // An array value in `writeHead`, not two `setHeader` calls: `setHeader` on the same name + // replaces, which would make the fixture single-valued and the row vacuous. Spread into a + // MUTABLE copy -- `OutgoingHttpHeader` is `string | string[]`, so a `readonly string[]` does + // not satisfy it, and handing the exported constant itself to `node:http` would alias it. + res.writeHead(401, {'www-authenticate': [...REPEATED_CHALLENGES]}); + res.end(); + return; + case '/redirect': + res.writeHead(302, {location: '/echo-headers'}); + res.end(); + return; + default: + res.writeHead(200, {'content-length': '0'}); + res.end(); + } +} + +/** + * Starts a local `node:http` server exposing the fixed set of endpoints every `TRANSPORT-N` assertion + * needs, on an ephemeral port so parallel test files never collide. + * + * @returns the listening server; the caller closes it in its own `afterAll`. + */ +export function startFixtureServer(): Promise<TestServer> { + return new Promise(resolve => { + const server: Server = createServer((req, res) => { + route(new URL(req.url ?? '/', 'http://localhost').pathname, req, res); + }); + server.listen(0, '127.0.0.1', () => { + const address = server.address(); + const port = + typeof address === 'object' && address !== null ? address.port : 0; + resolve({ + url: `http://127.0.0.1:${String(port)}`, + close: () => + new Promise<void>(done => { + // closeAllConnections, not close alone: a keep-alive socket a transport still holds open + // would otherwise stall this for the server's whole idle timeout. + server.closeAllConnections(); + server.close(() => { + done(); + }); + }), + }); + }); + }); +} + +/** What {@link fileBodyFixture} needs beyond the path; mirrors `fileBody()`'s own option bag. */ +export interface FileBodyFixtureOptions { + /** The byte offset the descriptor declares; defaults to 0. */ + readonly start?: number; + /** The byte count the descriptor declares, captured as `fileBody()` captures it from `stat`. */ + readonly count: number; + /** Incremented on every `writeTo` call, so a row can assert the transport used it. */ + readonly writes?: {count: number}; +} + +/** + * A `kind: 'file'` request body over a real path, carrying BODY-13's `transferred === count` check + * itself — the shape `@dexpace/body-file`'s `fileBody()` produces, minus the construction-time + * validation no row here needs. + * + * **Deliberately a stand-in, not the real factory.** `@dexpace/transport-conformance` is `private`, + * resolves unbuilt, and depends on `@dexpace/core` alone; taking `@dexpace/body-file` would put a + * ninth entry in the root `build:deps` chain for one row. A real `fileBody()` crossing a real + * transport already has a home — `tests/node-conformance/transport.test.mjs`, which is the only + * layer that can host it. What a *transport* is answerable for is narrower, and is exactly what this + * exercises: TRANSPORT-28's structural recognition on `kind` alone, and that the declared length is + * honoured by calling the descriptor's own `writeTo` rather than by reading `path` behind its back. + * + * @param path - the file to stream; read fresh on every `writeTo`, as BODY-11 requires. + * @param options - the declared range, and an optional write counter. + * @returns a frozen descriptor a transport must recognise structurally. + */ +export function fileBodyFixture( + path: string, + options: FileBodyFixtureOptions, +): FileBodyDescriptor { + const start = options.start ?? 0; + const {count} = options; + return Object.freeze({ + kind: 'file' as const, + mediaType: 'application/octet-stream', + contentLength: count, + replayable: true, + path, + start, + count, + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + if (options.writes !== undefined) options.writes.count += 1; + const writer = sink.getWriter(); + if (count === 0) { + writer.releaseLock(); + return; + } + const stream = createReadStream(path, {start, end: start + count - 1}); + let transferred = 0; + try { + for await (const chunk of stream) { + const bytes = chunk as Buffer; + await writer.write(new Uint8Array(bytes)); + transferred += bytes.byteLength; + } + if (transferred !== count) { + // BODY-13's exact sentence, and its exact wording in `@dexpace/body-file`: the error names + // transferred-of-total, so a row can assert the numbers rather than only the class. + throw new Error( + `short write: transferred ${String(transferred)} of ${String(count)} bytes`, + ); + } + } catch (error) { + await writer.abort(error); + throw error; + } finally { + stream.destroy(); + writer.releaseLock(); + } + }, + }); +} diff --git a/packages/transport-conformance/src/index.ts b/packages/transport-conformance/src/index.ts new file mode 100644 index 0000000..c5841b7 --- /dev/null +++ b/packages/transport-conformance/src/index.ts @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT +// packages/transport-conformance/src/index.ts +export { + runTransportConformanceSuite, + type TransportCapabilities, +} from './run-suite.js'; +export {startFixtureServer, type TestServer} from './fixtures.js'; diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts new file mode 100644 index 0000000..1e28538 --- /dev/null +++ b/packages/transport-conformance/src/run-suite.ts @@ -0,0 +1,1146 @@ +// SPDX-License-Identifier: MIT +// packages/transport-conformance/src/run-suite.ts +// The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..29, BODY-13, +// RETRY-2 (a permanent misconfiguration is outside the retryable IoError tree), +// HTTP-35 (a transport-wide default timeout outside AbortSignal.timeout()'s range is refused at the +// factory), +// SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is +// answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one +// outbound header pass both adapters call -- is asserted at its source in +// @dexpace/transport-shared; the rows here cover what each adapter decides for itself, which since +// audit #67 / #81 includes TRANSPORT-11/12's per-header degrade, because the two adapters had four +// different answers for the same model-valid header and only a shared row could say so. +// TRANSPORT-18's collapse is a Deviation Ledger row, as is TRANSPORT-28's zero-copy SHOULD, whose two +// MUSTs the file-body rows do assert; TRANSPORT-30's +// full proxy-challenge flow is transport-undici's challenge-handler.test.ts, and only its +// unsupported-type refusal is a row here. TRANSPORT-22 is NOT driven from here -- +// forcing an adaptation throw needs a per-transport hook into the native response, so each adapter +// asserts it against its own (transport-fetch's fetch-transport.test.ts:123, transport-undici's +// undici-transport.test.ts:615). TRANSPORT-9's producer-failure race is not driven from here +// either, and for the same reason: proving that a native call still pending when the producer fails +// is CANCELLED needs the signal the adapter handed it, which only an instrumented native client can +// show -- a `FetchLike` in transport-fetch's suite and a bring-your-own `Dispatcher` in +// transport-undici's, each with a call that resolves after the producer has already lost the race. +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + getBuildInfo, + getGlobalLogger, + Headers, + isIoError, + Request, + RequestOptions, + setGlobalLogger, + type Body, + type Logger, + type Transport, +} from '@dexpace/core'; +import { + fileBodyFixture, + FIXED_LENGTH_BODY, + NOT_MODIFIED_ETAG, + REPEATED_CHALLENGES, + startFixtureServer, + type TestServer, +} from './fixtures.js'; + +/** + * A proxy configuration the transport under test cannot honour, so TRANSPORT-30's + * "discoverable rather than silently misbehaving" clause has something to be asserted against. + * + * Supplied by the adapter because only the adapter knows which of `ProxyType`'s values its native + * client refuses — `@dexpace/core` resolves `socks4`/`socks5` from `ALL_PROXY` (CFG-22) and neither + * shipped transport can carry either. + */ +export interface UnsupportedProxy { + /** The `ProxyOptions.type` the native client cannot honour; the refusal must name it. */ + readonly type: string; + /** Builds a transport configured with that type. Expected to throw rather than return one. */ + build(): Transport; +} + +/** + * The clauses `docs/product-spec/17-transport-adapter-conformance-contract.md` scopes to only one + * reference transport, plus the one drop-set entry that legitimately differs between the two. + */ +export interface TransportCapabilities { + /** TRANSPORT-8: the transport has an internal-cancel path distinct from a caller abort. */ + readonly supportsInternalCancel: boolean; + /** + * TRANSPORT-30: the transport can be configured with a proxy at all. The proxy behaviour itself is + * asserted in `transport-undici`'s own tests, because only that package can construct one. + */ + readonly supportsProxy: boolean; + /** TRANSPORT-11: whether `Connection` is in this transport's outbound drop set. */ + readonly dropsConnectionHeader: boolean; + /** + * TRANSPORT-30: a proxy type this transport's configuration can express and its native client + * cannot honour. Omit it when the transport takes no proxy at all, or honours every type it can + * be handed — the row then asserts that omission is the truth rather than skipping. + */ + readonly unsupportedProxy?: UnsupportedProxy; + /** + * HTTP-35: builds a transport whose transport-wide default timeout is `value`. + * + * Required, not a capability flag, because §17 assumes every transport has one — TRANSPORT-5 is + * written as "a per-call override … overriding the configured default for that call only". The + * rows hand it values `AbortSignal.timeout()` cannot take and expect the factory to refuse them, + * because a default nobody checked is the last path by which such a value reaches a deadline + * (`RequestOptions.timeoutMs` has been checked at its setter since audit #67 / #76). + * + * Typed `number` on purpose: `0`, `-1`, `1.5`, `2**32` and `NaN` are all legitimately `number`, + * so the row needs no cast to express what it is testing. + */ + buildWithDefaultTimeoutMs(value: number): Transport; +} + +/** What every row below needs: a transport factory, the live fixture origin, and the capability flags. */ +interface SuiteContext { + readonly makeTransport: () => Transport; + readonly capabilities: TransportCapabilities; + /** Resolves a fixture path against the server started in `beforeAll`; read lazily, at run time. */ + url(path: string): string; + /** + * The same fixture, on a second origin, for rows that deliberately leave a connection unusable. + * + * A row that makes the server answer before draining the request body strands the remainder of + * that body in the socket. Whether the client then reuses it is the client's business, and a + * client that gets it wrong does not fail *here* -- it fails in whichever later row is handed the + * poisoned connection, which is a debugging problem of a different order. Bun 1.3.14 gets it + * wrong: it serves the resulting `400` from its pool, so `a per-call timeout is retryable` saw a + * 1ms resolve some thirty rows downstream. Neither `connection: close` nor destroying the socket + * server-side prevents it -- verified -- because the decision is entirely the client's. + * + * A separate origin is therefore the only thing this suite controls that contains the blast + * radius. Pathological rows get their own pool; every other row keeps the shared one. + */ + isolatedUrl(path: string): string; +} + +/** + * Awaits `pending` and hands back its rejection reason. + * + * Deliberately not `expect(pending).rejects.…`: that form is typed `void` here, so a row that has to + * assert something *after* the rejection (a `close()` that must not stall, say) would race its own + * assertion. This settles first, then asserts. + */ +/** How long the post-delivery producer stalls before failing; long enough to outlive `send`. */ +const POST_DELIVERY_MS = 150; + +async function rejection(pending: Promise<unknown>): Promise<unknown> { + try { + await pending; + } catch (error) { + return error; + } + throw new Error('expected the send to reject, but it resolved'); +} + +/** Creates a transport, runs `body` against it, and closes it on every exit path. */ +async function withTransport<T>( + make: () => Transport, + body: (transport: Transport) => Promise<T>, +): Promise<T> { + const transport = make(); + try { + return await body(transport); + } finally { + await transport.close(); + } +} + +/** + * Runs `body` with a capturing global logger installed and returns every `header` field the + * drop log emitted, lower-cased. Restores the previous logger on every exit path. + */ +async function captureDroppedHeaders( + body: () => Promise<void>, +): Promise<string[]> { + const dropped: string[] = []; + const previous: Logger = getGlobalLogger(); + const capturing: Logger = { + atLevel: () => { + let name: string | undefined; + const entry = { + field: (key: string, value: unknown) => { + if (key === 'header') name = String(value); + return entry; + }, + event: () => entry, + cause: () => entry, + emit: () => { + if (name !== undefined) dropped.push(name.toLowerCase()); + }, + }; + return entry; + }, + withContext: () => capturing, + }; + setGlobalLogger(capturing); + try { + await body(); + } finally { + setGlobalLogger(previous); + } + return dropped; +} + +async function readEchoedHeaders( + transport: Transport, + request: Request, +): Promise<Record<string, string>> { + const response = await transport.send(request); + return JSON.parse(await response.text()) as Record<string, string>; +} + +function registerDispatchRows(ctx: SuiteContext): void { + describe('TRANSPORT-1/2/21/23: dispatch, pipeline authority, null-safety', () => { + test('a 302 is returned raw, never followed by the native client', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/redirect')).build(); + const response = await transport.send(request); + expect(response.status.code).toBe(302); + expect(response.headers.get('location')).toBe('/echo-headers'); + await response.close(); + }); + }); + + test('a failure is delivered through the promise, never a synchronous throw', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url('http://127.0.0.1:1').build(); + // Reaching the next line at all is the assertion: a synchronous throw would abort the test + // here rather than surface through the promise (TRANSPORT-21). + const pending = transport.send(request); + expect(pending).toBeInstanceOf(Promise); + expect(await rejection(pending)).toBeDefined(); + }); + }); + + test('a success never resolves to a null or undefined response', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .build(); + const response = await transport.send(request); + expect(response).toBeDefined(); + expect(response.request.url.href).toBe(ctx.url('/echo-headers')); + await response.close(); + }); + }); + }); +} + +function registerStatusRows(ctx: SuiteContext): void { + describe('TRANSPORT-24/26/27: status fidelity and inbound downgrades', () => { + test('a vendor 520 is surfaced faithfully with a readable body', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/vendor-status')) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(520); + expect(await response.text()).toBe('vendor status body'); + }); + }); + + test('a body-less POST dispatches with a zero-length body, not a throw', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-headers')) + .build(); + const echoed = await readEchoedHeaders(transport, request); + // TRANSPORT-26: the zero-length substitution is observable as the framing the client + // computed, not as a rejected send. + expect(echoed['content-length']).toBe('0'); + }); + }); + + test('an unparseable Content-Type downgrades the response rather than failing it', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/malformed-content-type')) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(200); + expect(response.headers.get('content-type')).toBe('not-a-media-type'); + expect(await response.text()).toBe('body'); + }); + }); + }); +} + +function registerBodyRows(ctx: SuiteContext): void { + describe('TRANSPORT-17/19/25: request bodies written once, response bodies streamed lazily', () => { + test('a single-use body is written exactly once and its bytes reach the wire', async () => { + await withTransport(ctx.makeTransport, async transport => { + let writeCount = 0; + const payload = new TextEncoder().encode('payload'); + // Built from scratch rather than monkey-patching stringBody: every core model is frozen + // (HTTP-1), and a replayable body would not exercise the single-use path at all. + const body: Body = { + kind: 'stream', + mediaType: 'text/plain', + contentLength: payload.byteLength, + replayable: false, + async writeTo(sink) { + writeCount += 1; + const writer = sink.getWriter(); + await writer.write(payload); + await writer.close(); + }, + }; + const request = Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-body')) + .body(body) + .build(); + const response = await transport.send(request); + expect(await response.text()).toBe('payload'); + expect(writeCount).toBe(1); + }); + }); + + test('the response body streams on demand rather than arriving pre-buffered', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/drip')).build(); + const response = await transport.send(request); + const stream = response.body; + if (stream === null) throw new Error('the response carried no body'); + expect(stream).toBeInstanceOf(ReadableStream); + const reader = stream.getReader(); + const first = await reader.read(); + // The fixture drips for ~1s; a first chunk in hand while the stream is still open is the + // observable form of "not pre-buffered" (SEAM-11, TRANSPORT-25). + expect(first.done).toBe(false); + reader.releaseLock(); + await response.close(); + }); + }); + + test('closing without reading releases the connection, idempotently', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/drip')).build(); + const response = await transport.send(request); + await response.close(); + // Reaching the next line proves close() is idempotent: a second close that threw or hung + // would fail or time out the row (BODY-15). + await response.close(); + }); + }); + }); +} + +function registerProducerRows(ctx: SuiteContext): void { + describe('TRANSPORT-19: an abandoned or failed request-body producer', () => { + test('a producer that fails after delivery does not escape as an unhandled rejection', async () => { + await withTransport(ctx.makeTransport, async transport => { + // The fixture answers 413 without draining, so `send` resolves while `writeTo` is still + // parked. The producer then fails with nobody left awaiting it -- and a transport that does + // not keep a handler on the producer's settlement lets that rejection reach the runtime's + // default `unhandledRejection` policy, which terminates the process (TRANSPORT-19, SEAM-30). + // Both `bun test` and `node --test` fail a test that leaks one, so this row needs no + // process-level listener of its own to be the assertion. + const body: Body = { + kind: 'stream', + mediaType: 'application/octet-stream', + contentLength: -1, + replayable: false, + async writeTo(sink) { + const writer = sink.getWriter(); + await writer.write(new Uint8Array(1024)); + await new Promise(resolve => setTimeout(resolve, POST_DELIVERY_MS)); + throw new Error('producer failed after the response was delivered'); + }, + }; + const request = Request.newBuilder() + .method('POST') + // Quarantined: this row is the one that strands a request body mid-socket. + .url(ctx.isolatedUrl('/early-response')) + .body(body) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(413); + await response.close(); + // Outlives the producer, so the rejection has actually happened by the time the row ends. + await new Promise(resolve => setTimeout(resolve, POST_DELIVERY_MS * 3)); + }); + }); + }); +} + +/** + * A scheme every runtime under test refuses and `@dexpace/core` accepts. + * + * `Request` validates a URL by handing it to WHATWG `URL`, which parses `ftp:` perfectly well and + * gives it a real origin — so an `ftp://` request reaches the native client, which refuses it. + * `foo://` would be refused too, but its `origin` is the string `"null"`, which changes what undici + * is even asked; `ftp:` keeps the two adapters answering the same question. + */ +const UNSUPPORTED_SCHEME_URL = 'ftp://example.com/anything'; + +function registerPermanentFailureRows(ctx: SuiteContext): void { + describe('TRANSPORT-20, RETRY-2: a permanent misconfiguration is not a retryable failure', () => { + test('an unsupported URL scheme fails outside the IoError tree', async () => { + // The two adapters answered this oppositely until audit #67 / #82. undici's dispatcher + // rejects `ftp://` with `UND_ERR_INVALID_ARG`, which `transport-undici` already mapped to a + // TypeError; `fetch` rejects with a TypeError whose shape depends on the runtime -- Node's + // undici-backed one says `fetch failed` with an `unknown scheme` cause, Bun 1.3.14 says + // `protocol must be http:, https: or s3:` with `code: ERR_INVALID_ARG_VALUE` -- and + // `transport-fetch` classified all of it as the RETRYABLE TransportFailureError. + // + // `classify.ts` returns true for every IoError, so that verdict spends the caller's entire + // retry budget re-proving a URL no retry can fix. `isIoError(e) === false` is the assertion + // because it is exactly what the retry engine asks. + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(UNSUPPORTED_SCHEME_URL) + .build(); + const error = await rejection(transport.send(request)); + expect(isIoError(error)).toBe(false); + expect(error).toBeInstanceOf(TypeError); + }); + }); + }); +} + +function registerFailureRows(ctx: SuiteContext): void { + describe('TRANSPORT-4/5/6/20: failure classification and per-call timeouts', () => { + test('a dead port surfaces the retryable TransportFailureError', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url('http://127.0.0.1:1').build(); + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + }); + }); + }); + + test('a per-call timeout is retryable, not a cancellation', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const options = RequestOptions.newBuilder().timeoutMs(50).build(); + expect(await rejection(transport.send(request, options))).toMatchObject( + {name: 'TransportFailureError'}, + ); + }); + }); + + test('two concurrent calls are each bounded by their own timeout', async () => { + await withTransport(ctx.makeTransport, async transport => { + const slow = (): Request => + Request.newBuilder().url(ctx.url('/slow')).build(); + const started = Date.now(); + // TRANSPORT-5: the per-call override applies to that call only, and neither call waits on + // the other. Both are awaited, so the transport closes with nothing still in flight. + const brief = rejection( + transport.send( + slow(), + RequestOptions.newBuilder().timeoutMs(60).build(), + ), + ); + const patient = rejection( + transport.send( + slow(), + RequestOptions.newBuilder().timeoutMs(1_200).build(), + ), + ); + expect(await brief).toMatchObject({name: 'TransportFailureError'}); + // The short call cannot have been extended to the long call's deadline. + expect(Date.now() - started).toBeLessThan(1_000); + expect(await patient).toMatchObject({name: 'TransportFailureError'}); + }); + }); + + test('a sub-resolution 1ms timeout still times out rather than hanging', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const options = RequestOptions.newBuilder().timeoutMs(1).build(); + expect(await rejection(transport.send(request, options))).toMatchObject( + {name: 'TransportFailureError'}, + ); + }); + }); + }); +} + +function registerCancellationRows(ctx: SuiteContext): void { + describe('TRANSPORT-3/7/9: cancellation is terminal, and orphans are released', () => { + test('aborting mid-request yields a terminal CancellationError', async () => { + await withTransport(ctx.makeTransport, async transport => { + const controller = new AbortController(); + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const pending = transport.send(request, undefined, controller.signal); + setTimeout(() => { + controller.abort(); + }, 20); + expect(await rejection(pending)).toMatchObject({ + name: 'CancellationError', + }); + }); + }); + + test('a cancelled exchange leaves no handle that stalls close()', async () => { + const transport = ctx.makeTransport(); + const controller = new AbortController(); + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const pending = transport.send(request, undefined, controller.signal); + setTimeout(() => { + controller.abort(); + }, 20); + expect(await rejection(pending)).toBeDefined(); + // A dangling handle would stall this close() until the row times out. + await transport.close(); + }); + + test('an abort after the response was delivered does not close its body (SEAM-16)', async () => { + await withTransport(ctx.makeTransport, async transport => { + const controller = new AbortController(); + const request = Request.newBuilder() + .url(ctx.url('/vendor-status')) + .build(); + const response = await transport.send( + request, + undefined, + controller.signal, + ); + controller.abort(); + // The caller owns the delivered body even when the signal fires afterwards; a transport that + // wired an unconditional abort listener would truncate this read. + expect(await response.text()).toBe('vendor status body'); + }); + }); + + test('a timeout while headers are still pending releases the connection (SEAM-30)', async () => { + const transport = ctx.makeTransport(); + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const options = RequestOptions.newBuilder().timeoutMs(50).build(); + expect(await rejection(transport.send(request, options))).toMatchObject({ + name: 'TransportFailureError', + }); + await transport.close(); + }); + }); +} + +function registerLifecycleRows(ctx: SuiteContext): void { + describe('TRANSPORT-15/16/29, SEAM-12: lifecycle and concurrency', () => { + test('close is idempotent', async () => { + const transport = ctx.makeTransport(); + await transport.close(); + // A second close that threw or hung would fail or time out the row (TRANSPORT-16). + await transport.close(); + }); + + test('many concurrent sends each resolve to their own response', async () => { + await withTransport(ctx.makeTransport, async transport => { + const responses = await Promise.all( + Array.from({length: 20}, (_unused, index) => + transport.send( + Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers( + Headers.newBuilder().set('X-Call', String(index)).build(), + ) + .build(), + ), + ), + ); + const seen = await Promise.all( + responses.map(async response => { + const echoed = JSON.parse(await response.text()) as Record< + string, + string + >; + return echoed['x-call']; + }), + ); + // Per-request state confined to the promise graph: 20 distinct values, no interleaving. + expect(new Set(seen).size).toBe(20); + }); + }); + }); +} + +function registerHeaderRows(ctx: SuiteContext): void { + describe('TRANSPORT-10/11, NFR-15: the outbound header pass', () => { + test('a caller-supplied Content-Length never reaches the wire (framing is the client’s)', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-headers')) + .headers(Headers.newBuilder().set('Content-Length', '999').build()) + .body({ + kind: 'byte-array', + mediaType: 'text/plain', + contentLength: 5, + replayable: true, + async writeTo(sink) { + const writer = sink.getWriter(); + await writer.write(new TextEncoder().encode('hello')); + await writer.close(); + }, + }) + .build(); + const echoed = await readEchoedHeaders(transport, request); + expect(echoed['content-length']).not.toBe('999'); + }); + }); + + test('a body-derived Content-Type is stamped when the caller set none', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-headers')) + .body({ + kind: 'byte-array', + mediaType: 'application/x-conformance', + contentLength: 2, + replayable: true, + async writeTo(sink) { + const writer = sink.getWriter(); + await writer.write(new Uint8Array([1, 2])); + await writer.close(); + }, + }) + .build(); + const echoed = await readEchoedHeaders(transport, request); + expect(echoed['content-type']).toBe('application/x-conformance'); + }); + }); + + test('a stamped User-Agent survives the drop pass unmangled', async () => { + await withTransport(ctx.makeTransport, async transport => { + const identity = getBuildInfo().identityTokens.join(' '); + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers(Headers.newBuilder().set('User-Agent', identity).build()) + .build(); + const echoed = await readEchoedHeaders(transport, request); + expect(echoed['user-agent']).toBe(identity); + }); + }); + }); +} + +/** + * Headers every model layer in this SDK accepts and at least one shipped native client refuses + * outright, paired with the value that provokes the refusal. + * + * `expect`, `keep-alive` and `upgrade` are `undici`'s three unconditional rejections + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`), and Node's global `fetch` is undici-backed, so both adapters + * meet them. `X Custom` is the non-token name: `@dexpace/core` admits any printable ASCII byte in + * a header name (`http/ascii-validation.ts:29-33`), while both native layers require RFC 9110 + * `token`. + * + * TRANSPORT-12 is what makes these one table rather than four transport-specific quirks: whatever + * the native client refuses, the transport drops *that header only* and still dispatches. + */ +const NATIVE_REJECTED_HEADERS: readonly (readonly [string, string])[] = [ + ['Expect', '100-continue'], + ['Keep-Alive', 'timeout=5'], + ['Upgrade', 'websocket'], + ['X Custom', 'model-valid, non-token'], +]; + +function registerNativeRejectionRows(ctx: SuiteContext): void { + describe('TRANSPORT-11/12/13: a header the native client refuses degrades to a logged drop', () => { + for (const [name, value] of NATIVE_REJECTED_HEADERS) { + test(`${name} is dropped and logged, and the rest of the request still dispatches`, async () => { + let echoed: Record<string, string> = {}; + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers( + Headers.newBuilder() + .set(name, value) + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + echoed = await readEchoedHeaders(transport, request); + }); + }); + // The send resolved at all, which is TRANSPORT-12's "the resulting native exception MUST NOT + // escape the send contract"; the two header assertions are its "the bad header is absent, + // the normal header present". + expect(echoed[name.toLowerCase()]).toBeUndefined(); + expect(echoed['x-pass-through']).toBe('survives'); + // TRANSPORT-13: the drop is discoverable by name, never silent. + expect(dropped).toContain(name.toLowerCase()); + }); + } + + test('a Connection value the native client cannot carry is dropped on both transports', async () => { + // `Connection` is the one name whose drop set legitimately differs (TRANSPORT-11's own note, + // and `registerDropSetRows` below), so this row asserts the intersection: `upgrade` is neither + // `close` nor `keep-alive`, undici rejects it outright (`request.js:400-404`), and the WHATWG + // layer forbids the name entirely. Both must drop it, whichever reason applies. + // + // Asserted through the log rather than the echo for the same reason `registerDropSetRows` + // does: both clients set a `Connection` header of their own, so the wire cannot tell a + // forwarded caller header from the client's. + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers(Headers.newBuilder().set('Connection', 'upgrade').build()) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(200); + await response.close(); + }); + }); + expect(dropped).toContain('connection'); + }); + }); +} + +/** + * The declared length of the truncate-after-stat file body, deliberately **below** both shipped + * adapters' 1,000,000-byte materialize bound, so this row drives the buffered path. + * + * The streamed path is asserted in `tests/node-conformance/transport.test.mjs` instead, and that is + * not a preference. Bun 1.3.14's `Readable.fromWeb` leaks the abort reason as two or three unhandled + * rejections when the web readable behind it is aborted mid-pull, which is exactly what a producer + * failure on the streamed path does; `bun:test` then fails whichever row happens to be running. + * Isolated to sixteen lines with no SDK code in them, and clean under `node --test` on both + * transports (measured 2026-09-05, audit #67 / #81). Raising this constant past 1,000,000 will make + * the row red for that reason and no other. + */ +const TRUNCATED_FILE_BYTES = 64; + +/** How far a truncate-after-stat cuts the file back; small enough that no read can be a full one. */ +const TRUNCATED_TO_BYTES = 10; + +/** The intact file-body fixture's size, and the byte range the ranged row asks for inside it. */ +const INTACT_FILE_BYTES = 64; +const INTACT_RANGE = {start: 10, count: 20} as const; + +/** Distinguishable bytes, so a misaligned send fails on content and not merely on length. */ +function fileFixtureBytes(size: number): Uint8Array { + const bytes = new Uint8Array(size); + for (let index = 0; index < size; index += 1) { + bytes[index] = 33 + ((index * 7) % 94); + } + return bytes; +} + +/** Writes `size` fixture bytes to a fresh temporary file, runs `body`, and removes the directory. */ +async function withFixtureFile<T>( + size: number, + body: (path: string) => Promise<T>, +): Promise<T> { + const dir = await mkdtemp(join(tmpdir(), 'dexpace-conformance-file-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, fileFixtureBytes(size)); + return await body(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + +/** + * Every message on `error` and its `cause` chain, joined. + * + * BODY-13's transferred-of-total text surfaces at a different depth per path: the streamed path + * rethrows the producer's own failure as the message, while the buffered path wraps it as a cause + * under a fixed "request body could not be written". Both satisfy the requirement; asserting on the + * chain is what lets one row cover both without pinning either transport's wrapper wording. + */ +function messageChain(error: unknown): string { + const parts: string[] = []; + let current: unknown = error; + for (let depth = 0; depth < 8 && current instanceof Error; depth += 1) { + parts.push(current.message); + current = current.cause; + } + return parts.join(' <- '); +} + +function registerFileBodyRows(ctx: SuiteContext): void { + describe('TRANSPORT-28, BODY-13: a file body is dispatched through its own writeTo', () => { + test('an intact ranged file body puts exactly its declared bytes on the wire', async () => { + await withFixtureFile(INTACT_FILE_BYTES, async path => { + const writes = {count: 0}; + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-body')) + .body(fileBodyFixture(path, {...INTACT_RANGE, writes})) + .build(), + ); + // TRANSPORT-28's "assert exactly that byte range reaches the wire". + expect([...(await response.bytes())]).toEqual([ + ...fileFixtureBytes(INTACT_FILE_BYTES).slice( + INTACT_RANGE.start, + INTACT_RANGE.start + INTACT_RANGE.count, + ), + ]); + }); + // TRANSPORT-17's counterpart for a replayable body: a transport that reads `path` itself + // rather than calling `writeTo` would put the same bytes on the wire and leave this at 0. + expect(writes.count).toBe(1); + }); + }); + + test('a file body truncated after its length was captured fails the send', async () => { + await withFixtureFile(TRUNCATED_FILE_BYTES, async path => { + // The descriptor is built first, exactly as `fileBody()` captures `count` from `stat`, and + // the file shrinks underneath it afterwards. `content-length` is dropped outbound, so the + // wire cannot detect this either — BODY-13's check inside `writeTo` is the only thing that + // can, and a transport that hands the path to its native client never runs it + // (audit #67 / #81, where undici POSTed the ten surviving bytes and resolved 200). + const body = fileBodyFixture(path, {count: TRUNCATED_FILE_BYTES}); + await truncate(path, TRUNCATED_TO_BYTES); + const error = await withTransport(ctx.makeTransport, transport => + rejection( + transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.isolatedUrl('/echo-body')) + .body(body) + .build(), + ), + ), + ); + expect(error).toMatchObject({name: 'TransportFailureError'}); + expect(messageChain(error)).toContain( + `transferred ${String(TRUNCATED_TO_BYTES)} of ${String(TRUNCATED_FILE_BYTES)} bytes`, + ); + }); + }); + }); +} + +/** + * The fixture's challenge list, recovered from however this transport chose to split it. + * + * Scoped to `/repeated-challenge` on purpose, and deliberately NOT a general RFC 7235 parser: the two + * fixture challenges are `Digest`-schemed and carry no comma inside a quoted value, so a split at each + * `, Digest ` boundary recovers exactly what the server wrote. `@dexpace/core`'s real parser is + * `@internal` and unreachable from here — the core-side rows for it live in + * `packages/core/src/auth/auth-step.test.ts`. + */ +function challengeList(values: readonly string[]): readonly string[] { + return values.join(', ').split(/,\s+(?=Digest )/u); +} + +function registerBodylessRows(ctx: SuiteContext): void { + describe('TRANSPORT-24/25/27: a response that can carry no body reports none', () => { + // `body === null` is the WHATWG shape and the one `@dexpace/core` already types + // (`http/response.ts:18`); it is also the only shape a consumer can branch on without reading. + // Three of the four native combinations disagreed until audit #67 / #82 -- undici's dispatcher + // always hands back a `BodyReadable`, Node's `fetch` returns `null`, Bun 1.3.14's `fetch` + // returns a live `ReadableStream` -- so each adapter decides for itself now and these rows are + // what say so. The alternative, an empty stream on both, makes a consumer read to learn there + // is nothing there. + // + // `reasonPhrase` is `undefined`-or-string on purpose: `fetch` surfaces `statusText` and undici's + // `ResponseData` has no such field, a divergence recorded beside §10 item 13. Asserting the + // union is what keeps this row about the body shape. + test('a 204 carries a null body and no positive length', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/no-content')).build(), + ); + expect(response.status.code).toBe(204); + expect(response.body).toBeNull(); + // Absent on Node's `node:http`, `'0'` on Bun 1.3.14's -- what a transport is answerable for + // is that it never invented a length for a body that does not exist. + expect([undefined, '0']).toContain( + response.headers.get('content-length'), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + // Idempotent and non-blocking over a response the transport already released. + await response.close(); + await response.close(); + }); + }); + + test('a 304 carries a null body and still delivers its validator', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/not-modified')).build(), + ); + expect(response.status.code).toBe(304); + expect(response.body).toBeNull(); + // A 304 exists to carry validators; dropping the body must not drop them. + expect(response.headers.get('etag')).toBe(NOT_MODIFIED_ETAG); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('a HEAD carries a null body and keeps the length it advertises', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(ctx.url('/fixed-length')) + .build(), + ); + expect(response.status.code).toBe(200); + expect(response.body).toBeNull(); + // The header describes the body a GET would have returned and must survive verbatim -- + // this is the one body-less case where a length is meaningful (TRANSPORT-27). + expect(response.headers.get('content-length')).toBe( + String(FIXED_LENGTH_BODY.length), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('the same resource over GET does carry its body', async () => { + // The twin of the three rows above: nulling a body-less response must not null an ordinary + // one, and `/fixed-length` is the same route the HEAD row just read nothing from. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/fixed-length')).build(), + ); + expect(response.body).toBeInstanceOf(ReadableStream); + expect(await response.text()).toBe(FIXED_LENGTH_BODY); + }); + }); + }); +} + +function registerInboundHeaderRows(ctx: SuiteContext): void { + describe('TRANSPORT-14, AUTH-12/AUTH-25: a repeated inbound header keeps every value', () => { + test('two WWW-Authenticate lines reach the pipeline as the same challenge list', async () => { + // The two transports split this differently and both are right: WHATWG `Headers` comma-joins + // every name but `Set-Cookie`, so `@dexpace/transport-fetch` yields ONE `getAll` entry, while + // undici arrays any repeated header and `@dexpace/transport-undici` yields TWO. RFC 9110 5.3 + // makes the two wire shapes equivalent, so the entry count is not what either adapter is + // answerable for — the challenge list after the split is, and it must be identical. + // + // What this row guards is the loss: before audit #67 / #74 the auth step read + // `headers.get(...)`, saw only the first line, and left a 401 offering an answerable SHA-256 + // challenge unanswered through undici while the identical offer authenticated through fetch. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/repeated-challenge')).build(), + ); + expect(response.status.code).toBe(401); + expect( + challengeList(response.headers.getAll('WWW-Authenticate')), + ).toEqual([...REPEATED_CHALLENGES]); + await response.close(); + }); + }); + }); +} + +function registerDropSetRows(ctx: SuiteContext): void { + describe('TRANSPORT-11/13: the transport-specific drop set', () => { + test('the Connection header follows this transport’s documented drop set', async () => { + // Asserted through the drop log, not the echoed request: both clients set a `Connection` + // header of their own for connection management, so the wire cannot tell a forwarded + // caller header from the client's own. The log is where the decision is observable + // (TRANSPORT-11 with TRANSPORT-13). + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers( + Headers.newBuilder().set('Connection', 'keep-alive').build(), + ) + .build(); + const response = await transport.send(request); + await response.close(); + }); + }); + expect(dropped.includes('connection')).toBe( + ctx.capabilities.dropsConnectionHeader, + ); + }); + }); +} + +function registerProxyRefusalRows(ctx: SuiteContext): void { + describe('TRANSPORT-30: a proxy the native client cannot honour is refused, not attempted', () => { + test('an unsupported proxy type fails at construction with a typed, naming error', () => { + const unsupported = ctx.capabilities.unsupportedProxy; + if (unsupported === undefined) { + // The row still runs here rather than being skipped, and asserts the only thing left to + // assert: that there is genuinely no proxy surface to mis-set. `@dexpace/transport-fetch` + // is this leg — it ships no `proxy` option at all, deliberately (design doc §6), so + // TRANSPORT-30 has no subject on it and a silent skip would look the same as a gap. + expect(ctx.capabilities.supportsProxy).toBe(false); + return; + } + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first send, + // where it arrives as whatever the native client raises — the shape TRANSPORT-30 rules out. + void unsupported.build(); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // Not an `IoError`: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can + // fix is non-retryable for free (RETRY-2). An undici `InvalidArgumentError` escaping raw + // would fail this too, being neither a `TypeError` nor documented. + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the type that was refused, not merely that something was. + expect((thrown as Error).message).toContain(unsupported.type); + }); + }); +} + +/** + * Defaults `AbortSignal.timeout()` refuses. `1.5` and `2**32` are the two Bun 1.3.14 accepts and + * Node rejects with a `RangeError`, which is what made an unvalidated default a per-runtime + * behaviour rather than a per-caller error. + */ +const UNHONOURABLE_TIMEOUTS: readonly number[] = [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, +]; + +function registerDefaultTimeoutRows(ctx: SuiteContext): void { + describe('HTTP-35, TRANSPORT-5: an unhonourable default timeout is refused at construction', () => { + for (const value of UNHONOURABLE_TIMEOUTS) { + test(`a default of ${String(value)} fails the factory, not the first send`, () => { + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first + // send, where it arrives as a raw `RangeError` out of `AbortSignal.timeout()` on Node -- + // or, on Bun, as no failure at all and a deadline nobody asked for. + void ctx.capabilities.buildWithDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // The same shape every other construction-time refusal in these transports has, and + // outside the IoError tree for the same reason (RETRY-2). + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the value that was refused. + expect((thrown as Error).message).toContain(String(value)); + }); + } + + test('a default inside the range builds a transport that still sends', async () => { + // The twin: narrowing the accepted range must not reject a legitimate default. 30s is the + // shape a caller actually configures, and the send proves the value reached `composeSignal` + // without tripping it. + await withTransport( + () => ctx.capabilities.buildWithDefaultTimeoutMs(30_000), + async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/echo-headers')).build(), + ); + expect(response.status.code).toBe(200); + await response.close(); + }, + ); + }); + }); +} + +function registerScopedRows(ctx: SuiteContext): void { + if (ctx.capabilities.supportsInternalCancel) { + describe('TRANSPORT-8: an internal cancel is told apart from a timeout', () => { + test('the same slow endpoint yields a terminal cancel and a retryable timeout', async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder().url(ctx.url('/slow')).build(); + const controller = new AbortController(); + const cancelled = transport.send( + request, + undefined, + controller.signal, + ); + controller.abort(); + expect(await rejection(cancelled)).toMatchObject({ + name: 'CancellationError', + }); + const timedOut = transport.send( + Request.newBuilder().url(ctx.url('/slow')).build(), + RequestOptions.newBuilder().timeoutMs(30).build(), + ); + expect(await rejection(timedOut)).toMatchObject({ + name: 'TransportFailureError', + }); + }); + }); + }); + } + + if (ctx.capabilities.supportsProxy) { + describe('TRANSPORT-30: proxy-capable, but only when asked', () => { + test('an unconfigured proxy-capable transport still routes normally', async () => { + // §17's own conformance line for TRANSPORT-30 ("assert normal requests still route"). + // The regression it guards is a transport that installs a proxy dispatcher unconditionally + // and tunnels every request through nothing. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/echo-headers')).build(), + ); + expect(response.status.code).toBe(200); + await response.close(); + }); + }); + }); + } +} + +/** + * Registers the whole `TRANSPORT-N` conformance suite against one transport factory. + * + * @param name - the transport's name, used as the outer `describe` label. + * @param makeTransport - builds a fresh transport; called once per row and closed by the suite. + * @param capabilities - the clauses §17 scopes to a subset of transports. + */ +export function runTransportConformanceSuite( + name: string, + makeTransport: () => Transport, + capabilities: TransportCapabilities, +): void { + describe(`${name} conformance (TRANSPORT-1..30, SEAM-12/16/30, NFR-15)`, () => { + let server: TestServer; + let isolated: TestServer; + beforeAll(async () => { + server = await startFixtureServer(); + isolated = await startFixtureServer(); + }); + afterAll(async () => { + await server.close(); + await isolated.close(); + }); + + const ctx: SuiteContext = { + makeTransport, + capabilities, + url: path => `${server.url}${path}`, + isolatedUrl: path => `${isolated.url}${path}`, + }; + registerDispatchRows(ctx); + registerStatusRows(ctx); + registerBodyRows(ctx); + registerProducerRows(ctx); + registerFailureRows(ctx); + registerPermanentFailureRows(ctx); + registerCancellationRows(ctx); + registerLifecycleRows(ctx); + registerHeaderRows(ctx); + registerNativeRejectionRows(ctx); + registerFileBodyRows(ctx); + registerBodylessRows(ctx); + registerInboundHeaderRows(ctx); + registerDropSetRows(ctx); + registerProxyRefusalRows(ctx); + registerDefaultTimeoutRows(ctx); + registerScopedRows(ctx); + }); +} diff --git a/packages/transport-conformance/tsconfig.json b/packages/transport-conformance/tsconfig.json new file mode 100644 index 0000000..42c3719 --- /dev/null +++ b/packages/transport-conformance/tsconfig.json @@ -0,0 +1,15 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/transport-fetch/README.md b/packages/transport-fetch/README.md new file mode 100644 index 0000000..60891f4 --- /dev/null +++ b/packages/transport-fetch/README.md @@ -0,0 +1,88 @@ +# @dexpace/transport-fetch + +The zero-dependency `Transport` for the dexpace SDK, built on the runtime's own global `fetch`. +Nothing beyond a `@dexpace/core` peer is installed. + +```sh +bun add @dexpace/transport-fetch @dexpace/core +``` + +```typescript +import {Request} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +const transport = fetchTransport({headerDropLogging: 'first-per-name'}); + +const response = await transport.send( + Request.newBuilder().url('https://example.com/v1/users').build(), +); +try { + console.log(response.status.code, await response.text()); +} finally { + await response.close(); // the caller owns the body, always (BODY-15) +} +``` + +`close()` is the teardown, not `await using`. The factory returns a plain `Transport`: the disposal +member is installed only when `Symbol.asyncDispose` exists, which it does not on this package's +declared `engines.node` floor of `>=20.3` (the symbol arrived in 20.4). Declaring `AsyncDisposable` +in the `.d.ts` regardless would be a type that lies on the supported runtime — `NFR-10` forbids it, +and the [`await using` support row](https://github.com/dexpace/nodejs-sdk/blob/main/docs/open-items.md#d-nfr-10-await-using) in `docs/open-items.md` +records the decision and the four reasons the floor does not move instead. + +## What this transport deliberately does not do + +- **No proxy support, at all (`TRANSPORT-30`, scoped out).** There is no `proxy` option on + `FetchTransportOptions` — an absent option, not a silently ignored one — so a caller reaching for + proxying is type-directed to `@dexpace/transport-undici` rather than discovering the gap at + runtime. Node's bare global `fetch` exposes no proxy hook that does not route through `undici` + internals, and depending on `undici` would undo this package's entire reason to exist. +- **No native-internal cancel path (`TRANSPORT-8`, scoped out).** `fetch` has no teardown distinct + from an `AbortSignal` abort, so there is no second failure mode to tell apart from a timeout. +- **No connection pool to release.** `close()` is a sanctioned no-op over a runtime global this + package does not own, and `send()` keeps working after it — this transport's documented `SEAM-15` + post-close mode. `@dexpace/transport-undici` is the one with real close semantics. +- **`Response.protocol` is always `HTTP_1_1`.** A documented best-effort default: the WHATWG + `Response` object exposes no negotiated-HTTP-version field to read. Recorded in the Deviation + Ledger, not silently papered over. + +## Behavior worth knowing + +- Redirects are **never** followed (`redirect: 'manual'`). The SDK pipeline is the redirect + authority (`TRANSPORT-1`/`TRANSPORT-2`). +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped outbound because the client computes + its own framing; `Connection`, `Expect`, `Keep-Alive` and `Upgrade` because the layer underneath + refuses them. WHATWG names all four forbidden request headers, but the implementations do not + enforce that list and disagree about what happens instead: on Node the global `fetch` is + undici-backed, so an undropped `Expect`/`Keep-Alive`/`Upgrade` reaches undici's own validation and + fails the send with the **retryable** `TransportFailureError` — a permanent misconfiguration + spending the caller's whole retry budget — while Bun 1.3.14 forwards the first two to the wire and + hangs on the third until something else times the call out. Dropping the name is the one behaviour + `TRANSPORT-12` asks for, + and it matches `@dexpace/transport-undici` (measured 2026-09-05; audit #67 / #81). +- A header name the WHATWG `Headers` layer rejects — `@dexpace/core` admits every printable ASCII + byte in a name, so `X Custom` is model-valid and unsendable — degrades to the same drop, never a + failed send (`TRANSPORT-12`). +- Every drop is logged by name (never by value) through the global logger, deduped per name by + default (`TRANSPORT-11`/`TRANSPORT-13`). +- An abort that fires **after** `send()` resolved does not close the delivered body: the caller owns + it (`SEAM-16`). Cancellation stays live for the whole in-flight window. +- A timeout surfaces as the retryable `TransportFailureError`; a caller abort as the terminal + `CancellationError` (`TRANSPORT-3`/`TRANSPORT-4`). A raw `DOMException` is never surfaced. +- A request `fetch` refused to make — an unsupported scheme such as `ftp://`, a forbidden method, an + argument its own validation rejects — is a bare `TypeError` outside the `IoError` tree, so + `retry/classify.ts`'s allow-list makes it non-retryable (`RETRY-2`). A failed *exchange* stays the + retryable `TransportFailureError` (`TRANSPORT-20`). The table that tells them apart is + `@dexpace/transport-shared`'s, shared with `@dexpace/transport-undici`, because the runtimes report + the same refusal in three different shapes (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. Node's `fetch` says so itself; Bun + 1.3.14's returns a live `ReadableStream` for all three, which this transport cancels and replaces + with `null` so the shape is the SDK's rather than the runtime's. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `fetchTransport()`, not a + failure on the first send (`HTTP-35`). + +## Conformance + +Proven against the shared `TRANSPORT-N` suite in `@dexpace/transport-conformance`, the same one +`@dexpace/transport-undici` runs, so the two adapters cannot drift. diff --git a/packages/transport-fetch/api-extractor.json b/packages/transport-fetch/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/transport-fetch/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/transport-fetch/etc/transport-fetch.api.md b/packages/transport-fetch/etc/transport-fetch.api.md new file mode 100644 index 0000000..61fd06d --- /dev/null +++ b/packages/transport-fetch/etc/transport-fetch.api.md @@ -0,0 +1,27 @@ +## API Report File for "@dexpace/transport-fetch" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import { HeaderDropLogging } from '@dexpace/transport-shared'; +import { Transport } from '@dexpace/core'; + +// @public +export type FetchLike = (input: string, init: RequestInit & { + duplex?: 'half'; +}) => Promise<globalThis.Response>; + +// @public +export function fetchTransport(options?: FetchTransportOptions): Transport; + +// @public +export interface FetchTransportOptions { + readonly defaultTimeoutMs?: number; + readonly fetch?: FetchLike; + readonly headerDropLogging?: HeaderDropLogging; +} + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/transport-fetch/package.json b/packages/transport-fetch/package.json new file mode 100644 index 0000000..8c78f89 --- /dev/null +++ b/packages/transport-fetch/package.json @@ -0,0 +1,54 @@ +{ + "name": "@dexpace/transport-fetch", + "version": "0.0.0", + "description": "Fetch-based transport adapter for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/transport-fetch" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": { + "@dexpace/transport-shared": "workspace:*" + }, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/transport-fetch/src/fetch-transport.conformance.test.ts b/packages/transport-fetch/src/fetch-transport.conformance.test.ts new file mode 100644 index 0000000..260693b --- /dev/null +++ b/packages/transport-fetch/src/fetch-transport.conformance.test.ts @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT +// packages/transport-fetch/src/fetch-transport.conformance.test.ts +// Runs the shared TRANSPORT-N suite (@dexpace/transport-conformance) against fetchTransport(). +import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; +import {fetchTransport} from './fetch-transport.js'; + +runTransportConformanceSuite('fetchTransport', () => fetchTransport(), { + // TRANSPORT-8 scoped out: the global fetch has no internal-cancel path distinct from an abort. + supportsInternalCancel: false, + // TRANSPORT-30 scoped out: proxying would mean depending on undici internals (design doc s6). + supportsProxy: false, + // TRANSPORT-11: `Connection` is a WHATWG forbidden request header, so fetch drops it either way. + dropsConnectionHeader: true, + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => fetchTransport({defaultTimeoutMs: value}), +}); diff --git a/packages/transport-fetch/src/fetch-transport.test.ts b/packages/transport-fetch/src/fetch-transport.test.ts new file mode 100644 index 0000000..6b3c036 --- /dev/null +++ b/packages/transport-fetch/src/fetch-transport.test.ts @@ -0,0 +1,431 @@ +// SPDX-License-Identifier: MIT +// packages/transport-fetch/src/fetch-transport.test.ts +// Exercises: XCUT-13 (close is idempotent -- a repeat call is a latched no-op that neither throws nor +// blocks), XCUT-22 (the SDK closes only what it created; this transport creates no pooled resource, so +// its close owns nothing to release), +// TRANSPORT-2 (no retrying/redirecting dispatcher is ever composed), TRANSPORT-15/16 +// (close is a documented no-op), TRANSPORT-17/19 (single-use body written once, abandoned producer +// unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), +// TRANSPORT-20 with RETRY-2 (a permanent misconfiguration is classified outside the IoError tree, +// a failed exchange inside it), TRANSPORT-9 (a producer that loses the race cancels the native call +// it raced, so no response is stranded), TRANSPORT-30 +// (no proxy option exists at all), SEAM-30 (no producer is left running for its rejection to reach +// Node's default unhandledRejection policy) +import {describe, expect, test} from 'bun:test'; +import { + byteArrayBody, + Headers, + isIoError, + Request, + streamBody, + TransportFailureError, + type Body, +} from '@dexpace/core'; +import {fetchTransport} from './fetch-transport.js'; + +/** + * Awaits `pending` and hands back its rejection reason. `expect(p).rejects.…` is typed `void` here, + * so this keeps the assertion ordered with whatever the row checks afterwards. + */ +async function rejection(pending: Promise<unknown>): Promise<unknown> { + try { + await pending; + } catch (error) { + return error; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +/** A `fetch` double recording the `RequestInit` it was handed, answering a fixed 200. */ +type RecordedInit = RequestInit & {duplex?: 'half'}; + +function recordingFetch(): { + fetch: (input: string, init: RecordedInit) => Promise<globalThis.Response>; + calls: RecordedInit[]; +} { + const calls: RecordedInit[] = []; + return { + calls, + fetch: (_input, init) => { + calls.push(init); + return Promise.resolve(new globalThis.Response('ok', {status: 200})); + }, + }; +} + +describe('fetchTransport dispatch', () => { + test('TRANSPORT-1/2: redirects are never followed by the native client', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const request = Request.newBuilder() + .url('http://127.0.0.1:1/anything') + .build(); + await (await transport.send(request)).close(); + expect(recorder.calls[0]?.redirect).toBe('manual'); + }); + + test('TRANSPORT-11: the framing headers the client computes are dropped', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const request = Request.newBuilder() + .url('http://127.0.0.1:1/anything') + .headers( + Headers.newBuilder() + .set('Content-Length', '999') + .set('Connection', 'keep-alive') + .set('X-Kept', 'yes') + .build(), + ) + .build(); + await (await transport.send(request)).close(); + const sent = recorder.calls[0]?.headers as globalThis.Headers; + expect(sent.get('content-length')).toBeNull(); + expect(sent.get('connection')).toBeNull(); + expect(sent.get('x-kept')).toBe('yes'); + }); + + test('a small replayable body is materialized rather than streamed', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/anything') + .body( + byteArrayBody(new Uint8Array([1, 2, 3]), 'application/octet-stream'), + ) + .build(); + await (await transport.send(request)).close(); + expect(recorder.calls[0]?.body).toBeInstanceOf(Uint8Array); + expect(recorder.calls[0]?.duplex).toBeUndefined(); + }); + + test('TRANSPORT-17: a single-use body is streamed with duplex declared', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const source = new ReadableStream<Uint8Array>({ + start(controller) { + controller.enqueue(new Uint8Array([7])); + controller.close(); + }, + }); + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/anything') + .body(streamBody(source)) + .build(); + await (await transport.send(request)).close(); + expect(recorder.calls[0]?.body).toBeInstanceOf(ReadableStream); + expect(recorder.calls[0]?.duplex).toBe('half'); + }); +}); + +describe('fetchTransport failure paths', () => { + test('TRANSPORT-22: an adaptation throw cancels the native body before propagating', async () => { + let cancelled = false; + const body = new ReadableStream<Uint8Array>({ + cancel() { + cancelled = true; + }, + }); + // A deliberately hostile Response: the only way to make adaptation fail, since every value a + // conforming one carries is either total (Status.of) or degraded rather than rejected. + const hostile = { + status: 200, + statusText: 'OK', + body, + headers: { + forEach: () => { + throw new Error('adaptation exploded'); + }, + getSetCookie: () => [], + }, + } as unknown as globalThis.Response; + + const transport = fetchTransport({fetch: () => Promise.resolve(hostile)}); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + expect(await rejection(transport.send(request))).toMatchObject({ + message: 'adaptation exploded', + }); + expect(cancelled).toBe(true); + }); + + test('TRANSPORT-19/20: a producer failure fails the send and unwinds the producer', async () => { + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo() { + return Promise.reject(new Error('producer exploded')); + }, + }; + // A fetch that never settles, so the only way this send can finish is the producer's failure + // winning the race -- the regression this guards is sequencing the two instead of racing them. + const transport = fetchTransport({ + fetch: () => new Promise<globalThis.Response>(() => undefined), + }); + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body(failing) + .build(); + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + }); + }); +}); + +describe('fetchTransport request-body failures', () => { + test('a buffered body that cannot be written fails the send the same way', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body({ + kind: 'byte-array', + mediaType: 'text/plain', + contentLength: 3, + replayable: true, + writeTo: () => Promise.reject(new Error('body exploded')), + }) + .build(); + // The materialized branch classifies a body failure exactly as the streaming branch does. + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + cause: {message: 'body exploded'}, + }); + expect(recorder.calls.length).toBe(0); + }); + + test('a header-mapping throw never strands a started body producer (TRANSPORT-19, SEAM-30)', async () => { + // This transport builds its DispatchPlan as one object literal, so the safety here rests on + // property EVALUATION ORDER: `headers` must be computed before `prepared`. `prepareBody` starts + // a streaming producer eagerly and `toNativeHeaders` reads `request.body.mediaType`, a + // caller-supplied getter that may throw -- reversing the two would leave a live producer nobody + // can abandon, whose later rejection reaches Node's default unhandledRejection policy. The + // undici twin had exactly that ordering bug; this row keeps it from appearing here. + let producerStarted = false; + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body({ + kind: 'stream', + get mediaType(): string | undefined { + throw new Error('mediaType getter exploded'); + }, + // -1 / non-replayable forces the streaming branch rather than the buffered one. + contentLength: -1, + replayable: false, + writeTo: () => { + producerStarted = true; + return Promise.resolve(); + }, + }) + .build(); + + await rejection(transport.send(request)); + expect(producerStarted).toBe(false); + expect(recorder.calls.length).toBe(0); + }); + + test('a network failure is wrapped as TransportFailureError with its cause kept', async () => { + const cause = new Error('connect ECONNREFUSED'); + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + cause, + }); + }); +}); + +describe('fetchTransport failure classification (TRANSPORT-20, RETRY-2)', () => { + /** Every shape a runtime's `fetch` uses to say "these arguments can never work". */ + const permanent: readonly (readonly [string, Error])[] = [ + // Node's undici-backed `fetch`, thrown out of the `Request` constructor: no cause, because no + // dispatch was ever attempted. + [ + 'a forbidden method', + new TypeError("'CONNECT' HTTP method is unsupported."), + ], + [ + 'a non-token method', + new TypeError("'BAD METHOD' is not a valid HTTP method."), + ], + // The same runtime's scheme refusal, which it can only report as a network error. + [ + 'an unsupported scheme', + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + ], + // Bun 1.3.14's shape for the same scheme refusal: a code, and no cause at all. + [ + "Bun's coded scheme refusal", + Object.assign(new TypeError('protocol must be http:, https: or s3:'), { + code: 'ERR_INVALID_ARG_VALUE', + }), + ], + ]; + + for (const [what, cause] of permanent) { + test(`${what} is terminal, outside the IoError tree`, async () => { + // A permanent misconfiguration classified as TransportFailureError is an IoError, and + // `classify.ts` returns true for every IoError -- so the caller's whole retry budget goes on + // re-proving it. The undici twin has refused its own equivalents since Phase 8a; this + // transport refused none of them until audit #67 / #82. + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TypeError); + expect(isIoError(error)).toBe(false); + expect((error as Error).cause).toBe(cause); + }); + } + + test('a network failure reported the same way stays retryable', async () => { + // The twin of the rows above: `fetch failed` is also how every genuine connect/DNS/TLS failure + // arrives, so the cause is the only discriminator and narrowing must not swallow this. + const cause = new TypeError('fetch failed', { + cause: Object.assign(new Error('getaddrinfo ENOTFOUND h.invalid'), { + code: 'ENOTFOUND', + }), + }); + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://h.invalid/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(isIoError(error)).toBe(true); + }); +}); + +describe('fetchTransport lifecycle', () => { + test('TRANSPORT-15/16: close is a no-op and send still works afterwards (SEAM-15)', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + await transport.close(); + // Reaching the next line proves the second close neither threw nor hung (TRANSPORT-16). + await transport.close(); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + await (await transport.send(request)).close(); + expect(recorder.calls.length).toBe(1); + }); + + test('asyncDispose is the same teardown as close, where the runtime has it', async () => { + const transport = fetchTransport(); + // Cast rather than a bare `Symbol.asyncDispose` index: on the pinned floor (Node 20.3, which + // predates the symbol's 20.4 arrival) it is `undefined` and the index would read the string key + // `"undefined"`. The install in fetch-transport.ts is guarded to match. + const asyncDispose = (Symbol as {asyncDispose?: symbol}).asyncDispose; + if (typeof asyncDispose === 'symbol') { + const dispose = ( + transport as unknown as Record< + symbol, + (() => Promise<void>) | undefined + > + )[asyncDispose]; + expect(dispose).toBeDefined(); + await dispose?.call(transport); + } + expect( + Object.getOwnPropertyNames(Object.getPrototypeOf(transport)), + ).not.toContain('undefined'); + await transport.close(); + }); + + test('an aborted signal fails the send before any fetch call is made', async () => { + const recorder = recordingFetch(); + const transport = fetchTransport({fetch: recorder.fetch}); + const controller = new AbortController(); + controller.abort(); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + expect( + await rejection(transport.send(request, undefined, controller.signal)), + ).toMatchObject({name: 'CancellationError'}); + expect(recorder.calls.length).toBe(0); + }); + + test('defaultTimeoutMs bounds a call that supplies no timeout of its own', async () => { + // Asserted through the outcome, not through "a signal was handed over": every send dispatches + // with a forked signal since audit #67 / #82, so the presence of one no longer discriminates. + // The double honours its signal the way a real `fetch` does, which is what makes the composed + // deadline observable. + const transport = fetchTransport({ + fetch: (_input, init) => + new Promise<globalThis.Response>((_resolve, reject) => { + init.signal?.addEventListener('abort', () => { + reject(init.signal?.reason as Error); + }); + }), + defaultTimeoutMs: 20, + }); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + // TRANSPORT-4: a timeout is the retryable failure, never a cancellation, and the fork carries + // the source's `TimeoutError` reason through for `isTimeoutSignal` to read. + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + message: 'request timed out', + }); + }); +}); + +describe('fetchTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + test('a producer that loses the race takes the pending fetch down with it', async () => { + // The send rejects the moment `writeTo` fails, while `fetch` is still pending -- and until + // audit #67 / #82 nothing then cancelled it. With no caller signal and no timeout the transport + // dispatched with NO signal at all, so a response arriving afterwards was dropped with its body + // neither read nor cancelled: a connection held for as long as the pool would keep it. + let dispatched: AbortSignal | undefined; + let settled = false; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return new Promise<globalThis.Response>(resolve => { + setTimeout(() => { + settled = true; + resolve(new globalThis.Response('late', {status: 200})); + }, 30); + }); + }, + }); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + // The producer's own classification, not the native table's: `writeTo` failing is a transport + // failure, and the send must not be waiting on `fetch` to say so. + expect(error).toBeInstanceOf(TransportFailureError); + expect(settled).toBe(false); + expect(dispatched?.aborted).toBe(true); + expect((dispatched?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + // The twin: `abort` is latched by `detach`, so the fork's second direction cannot become the + // very violation its first direction exists to prevent. + let dispatched: AbortSignal | undefined; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return Promise.resolve(new globalThis.Response('ok', {status: 200})); + }, + }); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const response = await transport.send(request); + expect(await response.text()).toBe('ok'); + expect(dispatched?.aborted).toBe(false); + }); +}); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts new file mode 100644 index 0000000..784c33b --- /dev/null +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -0,0 +1,398 @@ +// SPDX-License-Identifier: MIT +// packages/transport-fetch/src/fetch-transport.ts +import { + composeSignal, + Protocol, + Response, + Status, + TransportFailureError, + type Body, + type Request, + type RequestOptions, + type Transport, +} from '@dexpace/core'; +import { + abortToSdkError, + createDropLogger, + degradeInboundHeaders, + forkSignal, + hasNoResponseBody, + isMaterializable, + mapOutboundHeaders, + materializeBody, + producerFailure, + pumpBody, + requireValidDefaultTimeoutMs, + toDispatchFailure, + type ForkedSignal, + type HeaderDropLogging, +} from '@dexpace/transport-shared'; + +/** + * TRANSPORT-1's redirect mode. `'manual'` yields the raw 3xx — status, `Location`, body — on every + * runtime this package is tested against (Node and Bun, both `undici`-backed). + * + * On a **browser** the same value yields an *opaque-redirect* filtered response instead: status `0`, + * no headers, a null body. The redirect is still not followed, so TRANSPORT-1 holds, but the + * pipeline above has nothing to redirect *with*. `@dexpace/transport-fetch` is therefore Node/Bun in + * practice even though its dependency list would run anywhere; a browser build needs a redirect + * strategy that does not depend on reading `Location` off the 3xx. + */ +const REDIRECT_MODE = 'manual' as const; + +/** + * TRANSPORT-11's outbound drop set for this transport. + * + * `connection` is in it because WHATWG `fetch` treats it as a forbidden request header and would + * strip it silently — dropping it here makes the removal observable through the drop log instead. + * + * `expect`, `keep-alive` and `upgrade` are in it because the *implementations* do not honour the + * WHATWG forbidden-header list at all. Node's global `fetch` is undici-backed and undici's `Headers` + * deliberately does not implement forbidden names in a non-browser environment, so the three reach + * `lib/core/request.js:398,409` and reject the dispatch; `fetch()` then rejects with a bare + * `TypeError: fetch failed`, which this transport can only classify as the **retryable** + * `TransportFailureError` — a permanent misconfiguration that spends the caller's whole retry budget + * re-proving itself. Bun 1.3.14 diverges again: it forwards `expect` and `keep-alive` to the wire + * and hangs indefinitely on `upgrade`. Measured on both, 2026-09-05, audit #67 / #81. Neither + * outcome is TRANSPORT-12's, and a name in the drop set is the one behaviour that is. + */ +const FETCH_FORBIDDEN_HEADERS = [ + 'content-length', + 'host', + 'transfer-encoding', + 'connection', + 'expect', + 'keep-alive', + 'upgrade', +] as const; + +/** + * Bodies at or below this declared length are materialized into one `Uint8Array` instead of streamed, + * which sidesteps the `duplex: 'half'` corner cases some `fetch` implementations still have. An + * explicit named bound, per the styleguide's "every buffer declares its bound" rule. + */ +const MAX_MATERIALIZED_BODY_BYTES = 1_000_000; + +/** + * Options for {@link fetchTransport}. + * + * There is deliberately **no** `proxy` option: Node's bare global `fetch` exposes no proxy hook that + * does not route through `undici` internals, and depending on `undici` would undo this package's + * entire reason to exist. The absence is the contract — reach for `@dexpace/transport-undici` when + * you need proxying (TRANSPORT-30, scoped out; design doc §6). + * + * @public + */ +export interface FetchTransportOptions { + /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ + readonly headerDropLogging?: HeaderDropLogging; + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link fetchTransport} rather than by the first send (HTTP-35). + */ + readonly defaultTimeoutMs?: number; + /** A custom `fetch` implementation; defaults to `globalThis.fetch`. */ + readonly fetch?: FetchLike; +} + +/** + * The narrow slice of `fetch` this transport calls. Deliberately not `typeof globalThis.fetch`: some + * runtimes hang extra statics off that value (Bun's `fetch.preconnect`), and requiring them would + * reject every reasonable test double while adding nothing this transport uses. + * + * @public + */ +export type FetchLike = ( + input: string, + init: RequestInit & {duplex?: 'half'}, +) => Promise<globalThis.Response>; + +/** A request body prepared for one `fetch` call, plus the teardown its producer may still need. */ +interface PreparedBody { + /** What to hand `RequestInit.body`, or `undefined` for a body-less request. */ + readonly init: BodyInit | undefined; + /** `'half'` when `init` is a stream, which `fetch` requires be declared explicitly. */ + readonly duplex: 'half' | undefined; + /** Settles when the streaming producer finishes; `undefined` for the buffered/no-body cases. */ + readonly done: Promise<void> | undefined; + /** Idempotent teardown for an abandoned producer (TRANSPORT-19); resolves once it has unwound. */ + abandon(cause: unknown): Promise<void>; +} + +const NO_BODY: PreparedBody = { + init: undefined, + duplex: undefined, + done: undefined, + abandon: () => Promise.resolve(), +}; + +async function prepareBody(body: Body | undefined): Promise<PreparedBody> { + if (body === undefined) return NO_BODY; + if (isMaterializable(body, MAX_MATERIALIZED_BODY_BYTES)) { + try { + return {...NO_BODY, init: await materializeBody(body)}; + } catch (error) { + // Same classification the streaming branch gives the same failure: a body that could not be + // produced is a transport failure with its cause intact, not a raw body error on one path and + // a wrapped one on the other (TRANSPORT-18's buffering clause, restated). + throw new TransportFailureError('request body could not be written', { + cause: error, + }); + } + } + const pump = pumpBody(body); + return { + init: pump.readable, + duplex: 'half', + done: pump.done, + abandon: cause => pump.abandon(cause), + }; +} + +/** One entry per VALUE, so a repeated name survives as repeated appends (HTTP-14). */ +function toNativeHeaders( + request: Request, + logDrops: (dropped: readonly string[]) => void, +): globalThis.Headers { + const {sent, dropped} = mapOutboundHeaders( + request.headers, + FETCH_FORBIDDEN_HEADERS, + {bodyDerivedMediaType: request.body?.mediaType}, + ); + logDrops(dropped); + + const native = new globalThis.Headers(); + for (const [name, value] of sent.entries()) { + try { + native.append(name, value); + } catch { + // TRANSPORT-12: a name the WHATWG layer rejects degrades to a drop, never a failed send. + logDrops([name]); + } + } + return native; +} + +function adaptResponse( + request: Request, + fetchResponse: globalThis.Response, + logDrops: (dropped: readonly string[]) => void, +): Response { + const raw: [string, string][] = []; + fetchResponse.headers.forEach((value, name) => { + // Set-Cookie is the one name WHATWG keeps un-joined; every other name arrives comma-joined. + if (name.toLowerCase() !== 'set-cookie') raw.push([name, value]); + }); + for (const cookie of fetchResponse.headers.getSetCookie()) { + raw.push(['set-cookie', cookie]); + } + + const {headers, dropped} = degradeInboundHeaders(raw); + logDrops(dropped); + + return ( + Response.newBuilder() + .request(request) + // A documented best-effort default, not an observed value: the WHATWG `Response` exposes no + // negotiated-HTTP-version field for this transport to read (Deviation Ledger). + .protocol(Protocol.HTTP_1_1) + .status(Status.of(fetchResponse.status)) + .reasonPhrase(fetchResponse.statusText || undefined) + .headers(headers) + // Decided here, not inherited from the runtime. Node's `fetch` returns `null` for 204, 304 + // and HEAD as the spec requires, and Bun 1.3.14's returns a live `ReadableStream` for all + // three (measured 2026-09-05) -- so forwarding `fetchResponse.body` made the SHAPE of a + // body-less response a property of the runtime rather than of this SDK. `#exchange` releases + // whatever handle this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, fetchResponse.status) + ? null + : fetchResponse.body, + ) + .build() + ); +} + +/** Everything one dispatch needs beyond the request itself; keeps `max-params` at three. */ +interface DispatchPlan { + readonly headers: globalThis.Headers; + readonly prepared: PreparedBody; + /** The forked signal handed to `fetch`; detached by `send` the moment the response is delivered. */ + readonly fork: ForkedSignal; +} + +class FetchTransport implements Transport { + readonly #logDrops: (dropped: readonly string[]) => void; + readonly #fetch: FetchLike; + readonly #defaultTimeoutMs: number | undefined; + + constructor(options: FetchTransportOptions) { + // Before anything else: a default no `AbortSignal.timeout()` can take is a caller error, and + // HTTP-35 puts it where it was supplied rather than at the first send (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); + this.#logDrops = createDropLogger( + options.headerDropLogging ?? 'first-per-name', + ); + this.#fetch = options.fetch ?? globalThis.fetch.bind(globalThis); + this.#defaultTimeoutMs = options.defaultTimeoutMs; + } + + async send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + const composed = composeSignal( + signal, + options?.timeoutMs ?? this.#defaultTimeoutMs, + ); + if (composed?.aborted) throw abortToSdkError(composed, composed.reason); + + // Dispatched with a fork the caller cannot reach: cancellation stays live for the whole in-flight + // window and goes inert the moment the response is handed over (SEAM-16). + const plan: DispatchPlan = { + headers: toNativeHeaders(request, this.#logDrops), + prepared: await prepareBody(request.body), + fork: forkSignal(composed), + }; + try { + return await this.#exchange(request, plan, composed); + } finally { + plan.fork.detach(); + } + } + + async #exchange( + request: Request, + plan: DispatchPlan, + composed: AbortSignal | undefined, + ): Promise<Response> { + const fetchResponse = await this.#dispatch(request, plan); + + if (composed?.aborted) { + // TRANSPORT-9 / SEAM-30: this response will never reach a caller, so this producer closes it. + await fetchResponse.body?.cancel().catch(() => undefined); + await plan.prepared.abandon(composed.reason); + throw abortToSdkError(composed, composed.reason); + } + + try { + // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. + const response = adaptResponse(request, fetchResponse, this.#logDrops); + if (response.body === null && fetchResponse.body !== null) { + // A runtime handed a body for a response that cannot have one. Nothing references it any + // more, so releasing it is this transport's, not the caller's (TRANSPORT-25, SEAM-30). + await fetchResponse.body.cancel().catch(() => undefined); + } + return response; + } catch (error) { + await fetchResponse.body?.cancel().catch(() => undefined); + // TRANSPORT-19: nothing is delivered on this path either, so the producer is owed its teardown + // exactly as on the abort branch above. + await plan.prepared.abandon(error); + throw error; + } + } + + async #dispatch( + request: Request, + plan: DispatchPlan, + ): Promise<globalThis.Response> { + const {prepared} = plan; + const {signal} = plan.fork; + const init: RequestInit & {duplex?: 'half'} = { + method: request.method, + headers: plan.headers, + // TRANSPORT-1: the pipeline, not the native client, is the redirect authority. + redirect: REDIRECT_MODE, + }; + if (prepared.init !== undefined) init.body = prepared.init; + if (prepared.duplex !== undefined) init.duplex = prepared.duplex; + init.signal = signal; + + try { + // Raced, not sequenced: a producer failure must surface even while `fetch` is still pending, + // and a producer that never resolves must not outlive the send (TRANSPORT-19). + return await Promise.race([ + this.#fetch(request.url.href, init), + producerFailure(prepared.done), + ]); + } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const abortedByCaller = signal.aborted; + await prepared.abandon(error); + if (abortedByCaller) throw abortToSdkError(signal, error); + // TRANSPORT-9: when the producer lost the race, `fetch` is still pending. Nothing awaits it + // any more, so a response that arrives later would be dropped with its body neither read nor + // cancelled -- a leaked connection for as long as the pool keeps it. Pulling the fork takes + // the native call down instead. On the path where `fetch` itself rejected there is nothing + // left to cancel and this is inert (audit #67 / #82). + plan.fork.abort(error); + // TRANSPORT-20 versus RETRY-2, decided by the table in `@dexpace/transport-shared` rather + // than here: until audit #67 / #82 every native rejection became `TransportFailureError`, + // which `classify.ts` reports retryable for being an `IoError`, so an `ftp://` URL or a + // `CONNECT` method spent the caller's whole retry budget re-proving a permanent + // misconfiguration. The undici twin already refused those; the two must not disagree. + throw toDispatchFailure(error, 'fetch failed'); + } + } + + /** + * Resolves immediately: the global `fetch` owns no resource this package created, so there is + * nothing to release (SEAM-14). `send()` therefore keeps working after `close()` — the documented + * post-close mode this transport picks under SEAM-15. + * + * @returns a promise that resolves once teardown is complete, which is immediately. + */ + close(): Promise<void> { + return Promise.resolve(); + } +} + +// Single teardown path for `await using`, delegating to `FetchTransport.close()` and installed at run +// time only when the symbol exists — the same guarded shape `SseStream` and `Page` use. +// +// DO NOT restore this as a plain `[Symbol.asyncDispose]()` class member. Node 20.3 is this package's +// declared floor (`engines.node`, checked by verify:runtime-floor) and predates the symbol, which +// arrived in 20.4. On the floor the computed key evaluates to `undefined` and binds the method to the +// string key `"undefined"` — a junk prototype entry, and no working disposal. Declaring it on the +// class would also emit it into the `.d.ts` unconditionally, promising consumers on the floor a method +// that is not there. +if (typeof Symbol.asyncDispose === 'symbol') { + Object.defineProperty(FetchTransport.prototype, Symbol.asyncDispose, { + value: function asyncDispose(this: FetchTransport): Promise<void> { + return this.close(); + }, + writable: true, + configurable: true, + }); +} + +/** + * Creates a `Transport` backed by the standard global `fetch` — the zero-dependency option. + * + * `close()` is a sanctioned no-op and `send()` keeps working after it (SEAM-15). There is no proxy + * support at all; see {@link FetchTransportOptions}. + * + * `close()` is the single teardown path `docs/knowledge/harvested/resource-management.md` asks for. A + * `[Symbol.asyncDispose]` delegating to it is installed at run time **when the runtime has the + * symbol**, which this package's declared floor (`engines.node >=20.3`) does not — it arrived in Node + * 20.4. The return type therefore does not promise `AsyncDisposable`: claiming it would type-check + * `await using` for a consumer sitting on the floor, where the method is genuinely absent. Call + * `close()`, or raise your own floor to 20.4+ and reach the symbol through a cast. + * + * @param options - optional transport settings. + * @returns a transport ready to send; release it with `close()`. + * @throws `TypeError` when `defaultTimeoutMs` is not an integer number of milliseconds in + * `1 .. 2**32 - 1` — `AbortSignal.timeout()`'s range, and therefore the only one a per-call + * deadline can be built from. + * + * @public + */ +export function fetchTransport(options: FetchTransportOptions = {}): Transport { + return new FetchTransport(options); +} diff --git a/packages/transport-fetch/src/index.ts b/packages/transport-fetch/src/index.ts new file mode 100644 index 0000000..d8ed16e --- /dev/null +++ b/packages/transport-fetch/src/index.ts @@ -0,0 +1,4 @@ +// SPDX-License-Identifier: MIT +// packages/transport-fetch/src/index.ts +export {fetchTransport} from './fetch-transport.js'; +export type {FetchLike, FetchTransportOptions} from './fetch-transport.js'; diff --git a/packages/transport-fetch/tsconfig.build.json b/packages/transport-fetch/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/transport-fetch/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/transport-fetch/tsconfig.json b/packages/transport-fetch/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/transport-fetch/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/transport-shared/README.md b/packages/transport-shared/README.md new file mode 100644 index 0000000..91242ee --- /dev/null +++ b/packages/transport-shared/README.md @@ -0,0 +1,25 @@ +# @dexpace/transport-shared + +Internal plumbing shared by `@dexpace/transport-fetch` and `@dexpace/transport-undici`. **Not a +package you install directly** — every export is `@internal`, and both transports depend on it so +that the one algorithm they both need exists once rather than twice. + +It is published anyway because `NFR-4` snapshots every published unit regardless of how its exports +are marked, and because a transport's own `dependencies` must resolve for consumers. + +## What lives here, and why it is not in a transport + +Putting any of this in one transport would make the other depend on a sibling transport, which the +Phase 8 segmentation design deliberately avoids — the two adapters must stay independent of each +other, not merely of the rest of the tree. + +| Module | Concern | +|---|---| +| `header-mapping.ts` | `TRANSPORT-10`/`TRANSPORT-12`'s outbound drop-and-degrade pass and `TRANSPORT-14`'s lenient inbound copy, which preserves obs-text values rather than rejecting them | +| `drop-log.ts` | `TRANSPORT-13`'s bounded, case-insensitive, drain-to-cap dedup of already-logged drop names. Names only — never values | +| `abort-mapping.ts` | The single mapping from an aborted signal to a canonical SDK error: `TransportFailureError` on timeout, `CancellationError` otherwise. A raw `DOMException` is never surfaced | +| `dispatch-classification.ts` | The single mapping from a *native rejection* to one: the retryable `TransportFailureError` `TRANSPORT-20` requires for a failed exchange, and a bare `TypeError` outside the `IoError` tree for a request the client refused to make. An allow-list, so an unrecognised rejection stays retryable | +| `body-less.ts` | Which method/status pairs can carry no response body at all, so `Response.body` is `null` for a 204, a 304 or a HEAD on every runtime rather than on whichever ones agree with the spec | +| `default-timeout.ts` | `HTTP-35`'s range check for a transport-wide default timeout: an integer in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s and therefore every transport's | +| `body-pump.ts` | Turning a `Body` into a request stream the transport owns the closing of, plus `TRANSPORT-19`'s idempotent teardown for an abandoned producer, and the classification of a producer's own failure | +| `signal-fork.ts` | `SEAM-16`'s abort-after-delivery rule: both native clients tie a response body's lifetime to the signal they were given, so the transport dispatches over a fork it detaches at delivery — and `TRANSPORT-9`'s other direction, a handle the transport pulls to cancel a native call it has abandoned | diff --git a/packages/transport-shared/api-extractor.json b/packages/transport-shared/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/transport-shared/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md new file mode 100644 index 0000000..4ac3d3d --- /dev/null +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -0,0 +1,115 @@ +## API Report File for "@dexpace/transport-shared" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import { Body as Body_2 } from '@dexpace/core'; +import { DexpaceError } from '@dexpace/core'; +import { Headers as Headers_2 } from '@dexpace/core'; +import type { Method } from '@dexpace/core'; + +// Warning: (ae-internal-missing-underscore) The name "abortToSdkError" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function abortToSdkError(signal: AbortSignal, cause: unknown): DexpaceError; + +// Warning: (ae-internal-missing-underscore) The name "BodyPump" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export interface BodyPump { + abandon(cause: unknown): Promise<void>; + readonly done: Promise<void>; + readonly readable: ReadableStream<Uint8Array>; +} + +// Warning: (ae-internal-missing-underscore) The name "createDropLogger" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function createDropLogger(mode: HeaderDropLogging): (dropped: readonly string[]) => void; + +// Warning: (ae-internal-missing-underscore) The name "degradeInboundHeaders" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function degradeInboundHeaders(raw: Iterable<readonly [string, string]>): { + headers: Headers_2; + dropped: readonly string[]; +}; + +// Warning: (ae-internal-missing-underscore) The name "ForkedSignal" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export interface ForkedSignal { + abort(reason: unknown): void; + detach(): void; + readonly signal: AbortSignal; +} + +// Warning: (ae-internal-missing-underscore) The name "forkSignal" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function forkSignal(source: AbortSignal | undefined): ForkedSignal; + +// Warning: (ae-internal-missing-underscore) The name "hasNoResponseBody" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function hasNoResponseBody(method: Method, status: number): boolean; + +// Warning: (ae-internal-missing-underscore) The name "HeaderDropLogging" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export type HeaderDropLogging = 'all' | 'first-per-name' | 'quiet'; + +// Warning: (ae-internal-missing-underscore) The name "isMaterializable" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function isMaterializable(body: Body_2, maxBytes: number): boolean; + +// Warning: (ae-internal-missing-underscore) The name "isPermanentDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function isPermanentDispatchFailure(error: unknown): boolean; + +// Warning: (ae-internal-missing-underscore) The name "mapOutboundHeaders" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function mapOutboundHeaders(headers: Headers_2, forbidden: readonly string[], opts?: MapOutboundHeadersOptions): { + sent: Headers_2; + dropped: readonly string[]; +}; + +// Warning: (ae-internal-missing-underscore) The name "MapOutboundHeadersOptions" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export interface MapOutboundHeadersOptions { + readonly bodyDerivedMediaType?: string | undefined; +} + +// Warning: (ae-internal-missing-underscore) The name "materializeBody" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function materializeBody(body: Body_2): Promise<Uint8Array<ArrayBuffer>>; + +// Warning: (ae-internal-missing-underscore) The name "producerFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function producerFailure(done: Promise<void> | undefined): Promise<never>; + +// Warning: (ae-internal-missing-underscore) The name "pumpBody" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function pumpBody(body: Body_2): BodyPump; + +// Warning: (ae-internal-missing-underscore) The name "requireValidDefaultTimeoutMs" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function requireValidDefaultTimeoutMs(value: number | undefined): void; + +// Warning: (ae-internal-missing-underscore) The name "toDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function toDispatchFailure(error: unknown, fallbackMessage: string): Error; + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/transport-shared/package.json b/packages/transport-shared/package.json new file mode 100644 index 0000000..dabee4d --- /dev/null +++ b/packages/transport-shared/package.json @@ -0,0 +1,51 @@ +{ + "name": "@dexpace/transport-shared", + "version": "0.0.0", + "description": "Shared transport adaptation and mapping helpers for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/transport-shared" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": {}, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/transport-shared/src/abort-mapping.test.ts b/packages/transport-shared/src/abort-mapping.test.ts new file mode 100644 index 0000000..533b964 --- /dev/null +++ b/packages/transport-shared/src/abort-mapping.test.ts @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/abort-mapping.test.ts +// Exercises: TRANSPORT-3 (cancellation -> CancellationError), TRANSPORT-4 (timeout -> TransportFailureError) +import {describe, expect, test} from 'bun:test'; +import {CancellationError, TransportFailureError} from '@dexpace/core'; +import {abortToSdkError} from './abort-mapping.js'; + +describe('abortToSdkError', () => { + test('maps AbortController abort to CancellationError', () => { + const controller = new AbortController(); + controller.abort(new Error('user abort')); + const err = abortToSdkError(controller.signal, controller.signal.reason); + expect(err).toBeInstanceOf(CancellationError); + expect(err.message).toBe('request cancelled'); + }); + + test('maps AbortSignal.timeout to TransportFailureError', async () => { + const signal = AbortSignal.timeout(5); + await new Promise(r => setTimeout(r, 20)); + const err = abortToSdkError(signal, signal.reason); + expect(err).toBeInstanceOf(TransportFailureError); + expect(err.message).toBe('request timed out'); + }); +}); diff --git a/packages/transport-shared/src/abort-mapping.ts b/packages/transport-shared/src/abort-mapping.ts new file mode 100644 index 0000000..7c06590 --- /dev/null +++ b/packages/transport-shared/src/abort-mapping.ts @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/abort-mapping.ts +import { + CancellationError, + TransportFailureError, + isTimeoutSignal, + type DexpaceError, +} from '@dexpace/core'; + +/** + * Maps an aborted signal to the canonical SDK error type. + * + * @param signal - the aborted AbortSignal + * @param cause - the original reason or error + * @returns a TransportFailureError if the signal was aborted by timeout, or CancellationError otherwise. + * + * @internal + */ +export function abortToSdkError( + signal: AbortSignal, + cause: unknown, +): DexpaceError { + return isTimeoutSignal(signal) + ? new TransportFailureError('request timed out', {cause}) + : new CancellationError('request cancelled', {cause}); +} diff --git a/packages/transport-shared/src/body-less.test.ts b/packages/transport-shared/src/body-less.test.ts new file mode 100644 index 0000000..a7ff678 --- /dev/null +++ b/packages/transport-shared/src/body-less.test.ts @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.test.ts +// Exercises: TRANSPORT-24 (every status is surfaced faithfully, including the ones that carry no +// body), TRANSPORT-25 (a response whose body a transport declines to expose still has its native +// handle released), TRANSPORT-27 (an absent length is the unknown-length case, not a failure) +import {describe, expect, test} from 'bun:test'; +import type {Method} from '@dexpace/core'; +import {hasNoResponseBody} from './body-less.js'; + +describe('hasNoResponseBody', () => { + test('the WHATWG null-body statuses carry none, whatever the method', () => { + for (const status of [101, 103, 204, 205, 304]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + true, + ]); + expect([status, hasNoResponseBody('POST', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('an ordinary status carries one', () => { + for (const status of [200, 201, 206, 302, 400, 404, 500, 520]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + false, + ]); + } + }); + + test('HEAD never carries one, whatever the status', () => { + // The Content-Length of a HEAD response describes the body a GET would have returned, so a + // transport that framed a stream from it would hand the caller a read that never completes. + for (const status of [200, 206, 404, 500]) { + expect([status, hasNoResponseBody('HEAD', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('a 2xx CONNECT is a tunnel, a failed CONNECT is an ordinary error response', () => { + expect(hasNoResponseBody('CONNECT', 200)).toBe(true); + expect(hasNoResponseBody('CONNECT', 299)).toBe(true); + expect(hasNoResponseBody('CONNECT', 407)).toBe(false); + expect(hasNoResponseBody('CONNECT', 502)).toBe(false); + }); + + test('every other method the model admits is decided by the status alone', () => { + const methods: readonly Method[] = [ + 'GET', + 'POST', + 'PUT', + 'DELETE', + 'OPTIONS', + 'TRACE', + 'PATCH', + ]; + for (const method of methods) { + expect([method, hasNoResponseBody(method, 204)]).toEqual([method, true]); + expect([method, hasNoResponseBody(method, 200)]).toEqual([method, false]); + } + }); +}); diff --git a/packages/transport-shared/src/body-less.ts b/packages/transport-shared/src/body-less.ts new file mode 100644 index 0000000..6cbdb5a --- /dev/null +++ b/packages/transport-shared/src/body-less.ts @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.ts +import type {Method} from '@dexpace/core'; + +/** + * The statuses that can never carry a body, whatever the request was: WHATWG fetch's null-body + * status set, which is RFC 9110's own list of body-less statuses (`101`, `103`, `204`, `205`, `304`). + * A `Content-Length` on one of them describes the body a `200` would have had and frames nothing. + */ +const NULL_BODY_STATUSES: ReadonlySet<number> = new Set([ + 101, 103, 204, 205, 304, +]); + +/** The lower and upper bounds of the 2xx range, inside which a `CONNECT` response is body-less. */ +const OK_MIN = 200; +const OK_MAX = 299; + +/** + * Whether the response to `method` with `status` can carry a body at all. + * + * The WHATWG rule, and what `@dexpace/core`'s model already types: `Response.body` is + * `ReadableStream<Uint8Array> | null` (`http/response.ts:18`), and `null` is what a consumer can + * branch on without reading. Both shipped adapters apply this rather than forwarding whatever their + * native client happened to produce, because three of the four combinations disagreed until audit + * #67 / #82: + * + * - undici's dispatcher always hands back a `BodyReadable`, so `@dexpace/transport-undici` wrapped + * an empty stream for 204, 304 and HEAD alike; + * - Node's global `fetch` returns `null` for all three, per the spec; + * - Bun 1.3.14's `fetch` returns a live `ReadableStream` for all three (measured 2026-09-05), so + * `@dexpace/transport-fetch` inherited the runtime's answer rather than the contract's. + * + * A transport that decides here instead reports the same shape on every runtime, which is what a + * conformance row can assert. Whatever native handle it then declines to expose is its own to + * release — an undrained `BodyReadable` holds the pooled connection open (TRANSPORT-25, SEAM-30). + * + * @param method - the request method; always a canonical uppercase token (HTTP-9). + * @param status - the response status code as the server sent it. + * @returns `true` when the adapted response must carry `body === null`. + * + * @internal + */ +export function hasNoResponseBody(method: Method, status: number): boolean { + if (method === 'HEAD') return true; + // A 2xx CONNECT switches the connection to a tunnel; anything after the blank line is tunnelled + // bytes, not a body. A non-2xx CONNECT is an ordinary error response and may carry one. + if (method === 'CONNECT') return status >= OK_MIN && status <= OK_MAX; + return NULL_BODY_STATUSES.has(status); +} diff --git a/packages/transport-shared/src/body-pump.test.ts b/packages/transport-shared/src/body-pump.test.ts new file mode 100644 index 0000000..860af4d --- /dev/null +++ b/packages/transport-shared/src/body-pump.test.ts @@ -0,0 +1,183 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-pump.test.ts +// Exercises: TRANSPORT-17 (a body is written exactly once), TRANSPORT-19 (an abandoned streaming +// producer is unblocked, teardown idempotent), BODY-8 (the sink's creator owns closing it) +import {describe, expect, test} from 'bun:test'; +import {byteArrayBody, type Body} from '@dexpace/core'; +import { + isMaterializable, + materializeBody, + producerFailure, + pumpBody, +} from './body-pump.js'; + +function countingBody(closesSink: boolean): Body & {readonly writes: number[]} { + const writes: number[] = []; + return { + kind: 'stream', + mediaType: 'text/plain', + contentLength: -1, + replayable: false, + writes, + async writeTo(sink) { + writes.push(1); + const writer = sink.getWriter(); + await writer.write(new TextEncoder().encode('ab')); + if (closesSink) await writer.close(); + else writer.releaseLock(); + }, + }; +} + +/** Awaits `pending` and hands back its rejection reason, so the assertion stays ordered. */ +async function rejection(pending: Promise<unknown>): Promise<unknown> { + try { + await pending; + } catch (error) { + return error; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +async function drain(stream: ReadableStream<Uint8Array>): Promise<string> { + const reader = stream.getReader(); + const parts: string[] = []; + for (;;) { + const {done, value} = await reader.read(); + if (done) break; + parts.push(new TextDecoder().decode(value)); + } + return parts.join(''); +} + +describe('pumpBody', () => { + test('terminates the stream for a body that closes the sink it was given', async () => { + const body = countingBody(true); + const pump = pumpBody(body); + expect(await drain(pump.readable)).toBe('ab'); + await pump.done; + expect(body.writes.length).toBe(1); + }); + + test('terminates the stream for a body that leaves the sink open (BODY-8)', async () => { + // @dexpace/body-file's writeTo releases its lock without closing; the pump must still end the + // stream, or the native client waits forever on a request body that never finishes. + const pump = pumpBody(countingBody(false)); + expect(await drain(pump.readable)).toBe('ab'); + await pump.done; + }); + + test('a producer failure rejects `done` rather than hanging the stream', async () => { + const body: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo() { + return Promise.reject(new Error('producer exploded')); + }, + }; + const pump = pumpBody(body); + expect(await rejection(pump.done)).toMatchObject({ + message: 'producer exploded', + }); + expect(await rejection(drain(pump.readable))).toBeDefined(); + }); + + test('abandon unblocks a producer that would otherwise never finish, idempotently', async () => { + let unblocked = false; + const body: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + async writeTo(sink) { + const writer = sink.getWriter(); + try { + // No reader ever drains this, so the second write parks on backpressure forever unless + // abandon() aborts the writer underneath it (TRANSPORT-19). + for (;;) await writer.write(new Uint8Array(64 * 1024)); + } finally { + unblocked = true; + } + }, + }; + const pump = pumpBody(body); + await pump.abandon(new Error('send failed')); + await pump.abandon(new Error('send failed')); + expect(unblocked).toBe(true); + }); +}); + +describe('producerFailure', () => { + /** Settles `pending` against a marker, so "never settles" is observable without hanging the row. */ + async function raceWithTimeout(pending: Promise<never>): Promise<string> { + return Promise.race([ + pending.then( + () => 'resolved', + (error: unknown) => `rejected: ${(error as Error).message}`, + ), + new Promise<string>(resolve => + setTimeout(() => { + resolve('pending'); + }, 50), + ), + ]); + } + + test('never settles when there is no streamed producer', async () => { + expect(await raceWithTimeout(producerFailure(undefined))).toBe('pending'); + }); + + test('never settles when the producer succeeds', async () => { + // A producer finishing says nothing about the response, so this must not win a `Promise.race` + // against a dispatch that is still in flight. + expect(await raceWithTimeout(producerFailure(Promise.resolve()))).toBe( + 'pending', + ); + }); + + test('carries the producer failure onward', async () => { + const done = Promise.reject(new Error('producer exploded')); + expect(await raceWithTimeout(producerFailure(done))).toBe( + 'rejected: producer exploded', + ); + }); + + test('keeps a handler on a rejection that lands after the race settled', async () => { + // The delivery-path guarantee, at its source: once `Promise.race` has attached to this promise, + // a producer that fails later is an observed rejection rather than one that reaches the + // runtime's default `unhandledRejection` policy. A leak here fails the row on both runners. + let fail!: (error: Error) => void; + const done = new Promise<void>((_resolve, reject) => { + fail = reject; + }); + const raced = await Promise.race([ + producerFailure(done), + new Promise<string>(resolve => + setTimeout(() => { + resolve('delivered'); + }, 10), + ), + ]); + expect(raced).toBe('delivered'); + fail(new Error('late producer failure')); + await new Promise(resolve => setTimeout(resolve, 50)); + }); +}); + +describe('materializeBody / isMaterializable', () => { + test('collects every chunk in order', async () => { + const bytes = await materializeBody( + byteArrayBody(new Uint8Array([1, 2, 3])), + ); + expect([...bytes]).toEqual([1, 2, 3]); + }); + + test('classifies by replayability and declared length', () => { + const small = byteArrayBody(new Uint8Array([1])); + expect(isMaterializable(small, 10)).toBe(true); + expect(isMaterializable(small, 0)).toBe(false); + expect(isMaterializable(countingBody(true), 10)).toBe(false); + }); +}); diff --git a/packages/transport-shared/src/body-pump.ts b/packages/transport-shared/src/body-pump.ts new file mode 100644 index 0000000..f657484 --- /dev/null +++ b/packages/transport-shared/src/body-pump.ts @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-pump.ts +import {TransportFailureError, type Body} from '@dexpace/core'; + +/** + * A streaming request body in flight: the stream to hand the native client, the producer's own + * settlement, and the teardown an abandoned send owes it (TRANSPORT-19). + * + * @internal + */ +export interface BodyPump { + /** The bytes `writeTo` produces, ready to hand to the native client. */ + readonly readable: ReadableStream<Uint8Array>; + /** Settles when the producer finishes; rejects with whatever `writeTo` raised. */ + readonly done: Promise<void>; + /** Idempotent teardown: aborts the producer and resolves once it has actually unwound. */ + abandon(cause: unknown): Promise<void>; +} + +/** + * The sink handed to `writeTo`, interposed rather than passing the `TransformStream`'s own writable + * straight through. Closing belongs to whoever created the stream (BODY-8), and the two conventions + * in this tree disagree: every `@dexpace/core` body closes the sink it was given, while + * `@dexpace/body-file`'s deliberately does not. Owning `close` here terminates the request body + * exactly once for both shapes — handing over the raw writable would either double-close (a + * `TypeError` that surfaces as a failed send) or never close at all (the native client waiting + * forever on a stream that never ends). + */ +function interposedSink( + writer: WritableStreamDefaultWriter<Uint8Array>, +): WritableStream<Uint8Array> { + return new WritableStream<Uint8Array>({ + write: chunk => writer.write(chunk), + close: () => undefined, + abort: () => undefined, + }); +} + +/** + * Starts `body`'s producer against a fresh `TransformStream` and returns the read end. + * + * The returned `done` is retained, never floating: a `writeTo` rejection must fail the send rather + * than leave the native client waiting on a stream that never closes. + * + * @param body - the body to stream; written exactly once (TRANSPORT-17). + * @returns the read end, the producer's settlement, and its teardown. + * + * @internal + */ +export function pumpBody(body: Body): BodyPump { + const {readable, writable} = new TransformStream<Uint8Array, Uint8Array>(); + const writer = writable.getWriter(); + const done = (async () => { + try { + await body.writeTo(interposedSink(writer)); + } catch (error) { + await writer.abort(error).catch(() => undefined); + throw error; + } + await writer.close(); + })(); + return { + readable, + done, + abandon: async (cause: unknown) => { + // `abort` is idempotent, satisfying TRANSPORT-19's idempotent-teardown clause; awaiting the + // producer with its rejection swallowed guarantees it has unwound before `send()` returns. + await writer.abort(cause).catch(() => undefined); + await done.catch(() => undefined); + }, + }; +} + +/** + * A promise that rejects when `done` rejects and otherwise never settles, for racing a pending + * dispatch against its own request-body producer. + * + * Racing is not the only reason to call this, and on the delivery path it is not even the main one: + * `Promise.race` attaches a handler to `done` that outlives the race, so a producer that fails + * *after* the native client already delivered a response is an observed rejection rather than an + * unhandled one. Without it that late rejection reaches Node's default `unhandledRejection` policy + * and takes the process down — the exact hazard SEAM-30 names, arriving from the request side. + * + * The rejection is classified **here**, as the retryable `TransportFailureError` both transports + * already reported for it, and not left raw for the caller's catch to guess at. A body that could + * not be written is a failure of this layer, and the catch that receives it is the one that also + * receives the *native* client's rejections — which since audit #67 / #82 go through a table that + * can call a bare `TypeError` a permanent misconfiguration. A producer that happened to throw one + * would have been read as the wire refusing the request. Classifying at the source is what + * `prepareBody`'s buffered branch already does with the same failure. + * + * @param done - the producer settlement from {@link pumpBody}, or `undefined` when the body was not + * streamed. + * @returns a promise that rejects with the producer's failure, wrapped, and never resolves. + * + * @internal + */ +export function producerFailure( + done: Promise<void> | undefined, +): Promise<never> { + if (done === undefined) return new Promise<never>(() => undefined); + return done.then( + // A producer *success* says nothing about the response, so only the failure is carried onward. + () => new Promise<never>(() => undefined), + (cause: unknown) => { + throw new TransportFailureError( + cause instanceof Error + ? cause.message + : 'request body could not be written', + {cause}, + ); + }, + ); +} + +/** + * Collects `body` into one contiguous buffer, for the small-and-replayable case both transports + * prefer over a streamed request body. + * + * The `Uint8Array<ArrayBuffer>` return type is load-bearing, not decoration: `BodyInit` accepts + * `ArrayBufferView<ArrayBuffer>` but not the `ArrayBufferLike`-backed default, which may be a + * `SharedArrayBuffer`. This always allocates a fresh, non-shared buffer, so it says so. + * + * @param body - the body to write. + * @returns every byte the body produced, in order. + * + * @internal + */ +export async function materializeBody( + body: Body, +): Promise<Uint8Array<ArrayBuffer>> { + // Chunks are retained by reference until the merge below, which relies on the Web Streams + // convention that a chunk passed to `write()` belongs to the sink. Every `Body` in this tree + // allocates per chunk (`node:fs` read streams included); a producer that wrote views over one + // reused scratch buffer would need a copy here instead. + const chunks: Uint8Array[] = []; + let total = 0; + await body.writeTo( + new WritableStream<Uint8Array>({ + write(chunk) { + chunks.push(chunk); + total += chunk.byteLength; + }, + }), + ); + const merged = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + merged.set(chunk, offset); + offset += chunk.byteLength; + } + return merged; +} + +/** + * Whether a body is small enough and replayable enough to materialize rather than stream. Streaming + * request bodies still carry `duplex: 'half'` corner cases in some `fetch` implementations, so the + * buffered path is the default wherever it is available. + * + * @param body - the body to classify. + * @param maxBytes - the inclusive upper bound on a materializable body's declared length. + * @returns `true` when {@link materializeBody} should be used instead of {@link pumpBody}. + * + * @internal + */ +export function isMaterializable(body: Body, maxBytes: number): boolean { + return ( + body.replayable && body.contentLength >= 0 && body.contentLength <= maxBytes + ); +} diff --git a/packages/transport-shared/src/default-timeout.test.ts b/packages/transport-shared/src/default-timeout.test.ts new file mode 100644 index 0000000..a4637ef --- /dev/null +++ b/packages/transport-shared/src/default-timeout.test.ts @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.test.ts +// Exercises: HTTP-35 (a timeout outside the range a transport can honour is refused where it was +// supplied, not where it is used), TRANSPORT-5 (a per-call override replaces a transport default, +// so the default is a real configuration value and answerable for its own range) +import {describe, expect, test} from 'bun:test'; +import {isIoError} from '@dexpace/core'; +import {requireValidDefaultTimeoutMs} from './default-timeout.js'; + +describe('requireValidDefaultTimeoutMs', () => { + test('accepts undefined and every integer in the honourable range', () => { + for (const value of [undefined, 1, 50, 30_000, 2 ** 32 - 1]) { + expect(() => { + requireValidDefaultTimeoutMs(value); + }).not.toThrow(); + } + }); + + test('refuses everything AbortSignal.timeout() cannot take', () => { + // The full range, not merely its lower bound. `1.5` and `2**32` are the two Bun 1.3.14 accepts + // and Node rejects with a `RangeError`, which is the divergence that made an unvalidated + // default a per-runtime behaviour rather than a per-caller error. + for (const value of [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, + ]) { + let thrown: unknown; + try { + requireValidDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect([value, thrown instanceof TypeError]).toEqual([value, true]); + // Outside the `IoError` tree, like every other construction-time refusal these transports + // raise: nothing retries a factory, and one class for all of them is easier to catch. + expect([value, isIoError(thrown)]).toEqual([value, false]); + // "Discoverable": the message names the value that was refused, not merely that one was. + expect((thrown as Error).message).toContain(String(value)); + } + }); + + test('the message names the range as well as the value', () => { + expect(() => { + requireValidDefaultTimeoutMs(0); + }).toThrow('1..4294967295'); + }); +}); diff --git a/packages/transport-shared/src/default-timeout.ts b/packages/transport-shared/src/default-timeout.ts new file mode 100644 index 0000000..afa6907 --- /dev/null +++ b/packages/transport-shared/src/default-timeout.ts @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.ts + +/** + * `AbortSignal.timeout()`'s upper bound, and therefore every transport's. Duplicated from + * `@dexpace/core`'s `http/request-options.ts:12`, which is not exported: the two must agree, and + * `RequestOptionsBuilder.timeoutMs`'s own rejection message is the wording copied below so a caller + * who trips either one reads the same sentence. + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + +/** + * Rejects a transport-wide default timeout that no transport could honour. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range the thing this value ends up in accepts. `RequestOptionsBuilder.timeoutMs` has checked + * exactly this since audit #67 / #76, on HTTP-35's reading that a timeout a setter accepted and a + * transport then refused is a failure belonging at the call site. `defaultTimeoutMs` was left + * unchecked on both transports and so became the last path by which `1.5`, `0` or `2**32` reached + * `composeSignal` — where Node throws `RangeError` and Bun 1.3.14 accepts the first two, so the same + * misconfiguration was a failed send on one runtime and a silently different deadline on the other + * (audit #67 / #82). + * + * A `TypeError`, matching the construction-time refusals both transports already raise for a + * caller misconfiguration (`undiciTransport`'s two) and deliberately outside the `IoError` tree — + * though nothing retries a factory, the conformance row asserts the same shape for both, and a + * transport is easier to reason about when every construction-time refusal is one class. + * + * @param value - the configured default, or `undefined` for none. + * @throws `TypeError` when a defined value is zero, negative, not finite, not an integer, or + * greater than `2**32 - 1`. + * + * @internal + */ +export function requireValidDefaultTimeoutMs(value: number | undefined): void { + if ( + value === undefined || + (Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { + return; + } + throw new TypeError( + `defaultTimeoutMs must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, ` + + `got ${String(value)}: it is handed to AbortSignal.timeout(), which accepts nothing else`, + ); +} diff --git a/packages/transport-shared/src/dispatch-classification.test.ts b/packages/transport-shared/src/dispatch-classification.test.ts new file mode 100644 index 0000000..9a61ef1 --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.test.ts @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.test.ts +// Exercises: TRANSPORT-20 (a failure that produced no response is the retryable transport failure), +// RETRY-2 (the retryable set is an allow-list, so a permanent misconfiguration outside the IoError +// tree is non-retryable for free), TRANSPORT-8 (an argument the native client can never accept is +// told apart from an exchange that failed) +import {describe, expect, test} from 'bun:test'; +import {isIoError, TransportFailureError} from '@dexpace/core'; +import { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; + +/** An error carrying a native `code`, the shape undici and Bun both attach one to. */ +function coded(message: string, code: string): Error { + return Object.assign(new Error(message), {code}); +} + +describe('isPermanentDispatchFailure', () => { + test("undici's two argument-validation codes are permanent", () => { + expect( + isPermanentDispatchFailure( + coded( + 'Invalid URL protocol: the URL must start with `http:` or `https:`.', + 'UND_ERR_INVALID_ARG', + ), + ), + ).toBe(true); + expect( + isPermanentDispatchFailure(coded('expect', 'UND_ERR_NOT_SUPPORTED')), + ).toBe(true); + }); + + test("Bun's coded fetch refusal for an unsupported scheme is permanent", () => { + // Bun 1.3.14, measured: `fetch('ftp://…')` rejects with this exact shape, where Node's + // undici-backed `fetch` rejects with `fetch failed` and an `unknown scheme` cause instead. + const error = Object.assign( + new TypeError('protocol must be http:, https: or s3:'), + {code: 'ERR_INVALID_ARG_VALUE'}, + ); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test("a causeless TypeError is undici's argument validation, so it is permanent", () => { + // Node's global `fetch` throws these out of the `Request` constructor, before any dispatch. + for (const message of [ + "'CONNECT' HTTP method is unsupported.", + "'BAD METHOD' is not a valid HTTP method.", + 'Request with GET/HEAD method cannot have body.', + ]) { + expect(isPermanentDispatchFailure(new TypeError(message))).toBe(true); + } + }); + + test('a scheme refusal reported through `fetch failed` is permanent', () => { + const error = new TypeError('fetch failed', { + cause: new Error('unknown scheme'), + }); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test('a network failure reported through `fetch failed` is NOT permanent', () => { + const error = new TypeError('fetch failed', { + cause: coded('getaddrinfo ENOTFOUND example.invalid', 'ENOTFOUND'), + }); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a blocked port stays retryable (TRANSPORT-20 probes one by name)', () => { + // `http://127.0.0.1:1` is the dead-port probe §17 names for TRANSPORT-20, and port 1 is on + // WHATWG's blocked list, so Node's `fetch` refuses it before connecting and says so in the + // cause. Classifying that reason as permanent would turn the SDK's headline retryable case + // terminal, which is why the reason table excludes it explicitly. + const error = new TypeError('fetch failed', {cause: new Error('bad port')}); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a plain connection failure and a non-Error rejection stay retryable', () => { + expect( + isPermanentDispatchFailure( + coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'), + ), + ).toBe(false); + expect(isPermanentDispatchFailure('a string nobody typed')).toBe(false); + }); +}); + +describe('toDispatchFailure', () => { + test('a permanent misconfiguration is a TypeError outside the IoError tree', () => { + const cause = coded('invalid request method', 'UND_ERR_INVALID_ARG'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TypeError); + // RETRY-2's allow-list is what makes this non-retryable; the class is how it stays outside it. + expect(isIoError(mapped)).toBe(false); + expect(mapped.cause).toBe(cause); + }); + + test('an exchange failure is the retryable TransportFailureError, cause intact', () => { + const cause = coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(isIoError(mapped)).toBe(true); + expect(mapped.message).toBe('connect ECONNREFUSED 127.0.0.1:1'); + expect(mapped.cause).toBe(cause); + }); + + test('a permanent verdict taken from the cause names the cause in its message', () => { + // `fetch failed` names nothing; the reason that made the verdict is the useful half. + const mapped = toDispatchFailure( + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + 'fetch failed', + ); + expect(mapped.message).toBe('fetch failed: unknown scheme'); + }); + + test('an error already in the SDK vocabulary is passed through untouched', () => { + // A request-body producer failure racing the dispatch arrives here already classified; the + // table knows nothing about the producer and must not re-answer for it. + const already = new TransportFailureError('producer exploded'); + expect(toDispatchFailure(already, 'fetch failed')).toBe(already); + }); + + test('a non-Error rejection falls back to the caller-supplied message', () => { + const mapped = toDispatchFailure(Symbol('nope'), 'fetch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(mapped.message).toBe('fetch failed'); + }); +}); diff --git a/packages/transport-shared/src/dispatch-classification.ts b/packages/transport-shared/src/dispatch-classification.ts new file mode 100644 index 0000000..cd838de --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.ts @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.ts +import {DexpaceError, TransportFailureError} from '@dexpace/core'; + +/** + * Error codes a native client uses for "these arguments can never work", as opposed to "this + * exchange failed". + * + * `UND_ERR_INVALID_ARG` and `UND_ERR_NOT_SUPPORTED` are undici's two argument-validation codes, + * raised by `Dispatcher.request` before a socket is touched (`lib/core/errors.js` in 6.28.0): a + * non-`http(s)` origin, `CONNECT` as a method, a non-token method, a per-request + * `Proxy-Authorization` on a `ProxyAgent`. `ERR_INVALID_ARG_VALUE`, `ERR_INVALID_ARG_TYPE` and + * `ERR_INVALID_URL` are the Node-style codes Bun's `fetch` sets on the same class of refusal — Bun + * 1.3.14 rejects an `ftp://` URL with a `TypeError` carrying `ERR_INVALID_ARG_VALUE`, where Node's + * undici-backed `fetch` rejects with a causing network error instead. Measured on both, 2026-09-05. + */ +const TERMINAL_ARGUMENT_CODES: ReadonlySet<string> = new Set([ + 'UND_ERR_INVALID_ARG', + 'UND_ERR_NOT_SUPPORTED', + 'ERR_INVALID_ARG_VALUE', + 'ERR_INVALID_ARG_TYPE', + 'ERR_INVALID_URL', +]); + +/** + * WHATWG network-error reasons that describe the *request* rather than the exchange. + * + * undici's `fetch` funnels every failure into one `TypeError('fetch failed', {cause})` + * (`lib/web/fetch/index.js:230`), so the top-level error cannot tell a refused scheme from a + * refused connection — the cause's message is the only discriminator the runtime offers. These + * three are `makeNetworkError` reasons raised before any dispatch (`:620`, `:793`, `:962` in + * 6.28.0); a scheme this SDK cannot speak is the same permanent misconfiguration undici's + * *dispatcher* reports as `UND_ERR_INVALID_ARG`. + * + * `'bad port'` is deliberately **not** here. WHATWG blocks a fixed list of ports, `1` among them, + * so on Node's `fetch` the canonical dead-port probe (`http://127.0.0.1:1`) arrives with that + * reason — and TRANSPORT-20's own conformance sentence is "connect to a dead port; assert the + * retryable type". Adding it would turn that row, and the SDK's headline retryable case, terminal. + */ +const TERMINAL_NETWORK_REASONS: ReadonlySet<string> = new Set([ + 'unknown scheme', + 'URL scheme must be a HTTP(S) scheme', + 'about scheme is not supported', +]); + +function errorCode(error: unknown): string | undefined { + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' ? code : undefined; +} + +function hasTerminalCode(error: unknown): boolean { + const code = errorCode(error); + return code !== undefined && TERMINAL_ARGUMENT_CODES.has(code); +} + +/** + * Whether a native rejection is a permanent misconfiguration rather than a failed exchange. + * + * Three positive recognitions, in one place so the two shipped adapters cannot answer differently + * for the same condition (the `ftp://` row asserts they do not). Everything else falls through to + * retryable, which is both the safe default and the behaviour every adapter had before audit #67 / + * #82 — TRANSPORT-20 makes "no response was produced" a MUST-retryable, so a rejection this table + * does not recognise must stay one. + * + * 1. A **terminal argument code** on the error or its immediate cause, per the + * `TERMINAL_ARGUMENT_CODES` table above. This is the whole undici-dispatcher leg, and Bun's + * `fetch`. + * 2. A **`TypeError` with no `cause`**. undici's `fetch` — which is also Node's global `fetch` — + * builds every *network* rejection with a cause, and every argument rejection as a bare + * `TypeError` thrown out of the `Request`/`Headers` constructors before a dispatch is attempted: + * an unsupported method, a non-token method, a body on a GET. The presence of a cause is + * therefore the runtime's own line between the two, and it needs no message matching. + * 3. A cause whose message is one of the `TERMINAL_NETWORK_REASONS` above — the scheme refusals + * that undici's `fetch` can only report through its fixed `fetch failed` message. + * + * @param error - whatever the native call rejected with. + * @returns `true` when no retry of the same request could succeed. + * + * @internal + */ +export function isPermanentDispatchFailure(error: unknown): boolean { + if (hasTerminalCode(error)) return true; + if (!(error instanceof Error)) return false; + const {cause} = error; + if (error instanceof TypeError && cause === undefined) return true; + if (hasTerminalCode(cause)) return true; + return cause instanceof Error && TERMINAL_NETWORK_REASONS.has(cause.message); +} + +/** + * The message to put on the mapped error: the native message, plus the cause's when the native + * layer's own message is a fixed placeholder. `fetch failed` names nothing on its own, and the + * reason that made the verdict permanent is the only useful thing to say. + */ +function describe(error: unknown, fallbackMessage: string): string { + if (!(error instanceof Error)) return fallbackMessage; + const {cause} = error; + if (!(cause instanceof Error) || error.message.includes(cause.message)) { + return error.message; + } + return `${error.message}: ${cause.message}`; +} + +/** + * Maps one native dispatch rejection onto the SDK's error vocabulary. + * + * A permanent misconfiguration becomes a bare `TypeError` carrying the native error as `cause`, + * deliberately **outside** the `IoError` tree: `retry/classify.ts` is an allow-list that returns + * `true` for every `IoError`, so a condition no retry can fix is non-retryable for free (RETRY-2), + * and `TypeError` is already what both transports raise for a caller misconfiguration caught at + * construction. Anything else becomes the retryable `TransportFailureError` TRANSPORT-20 requires. + * + * An error that already descends from `DexpaceError` is returned unchanged: it was classified at + * its own source — a request-body producer failure racing the dispatch is the live case — and + * re-classifying it here would answer for a layer this table knows nothing about. + * + * @param error - whatever the native call rejected with. + * @param fallbackMessage - the message to use when the rejection is not an `Error` at all. + * @returns the error to throw; the caller always throws it. + * + * @internal + */ +export function toDispatchFailure( + error: unknown, + fallbackMessage: string, +): Error { + if (error instanceof DexpaceError) return error; + if (isPermanentDispatchFailure(error)) { + return new TypeError(describe(error, fallbackMessage), {cause: error}); + } + return new TransportFailureError( + error instanceof Error ? error.message : fallbackMessage, + {cause: error}, + ); +} diff --git a/packages/transport-shared/src/drop-log.test.ts b/packages/transport-shared/src/drop-log.test.ts new file mode 100644 index 0000000..a154f91 --- /dev/null +++ b/packages/transport-shared/src/drop-log.test.ts @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/drop-log.test.ts +// Exercises: TRANSPORT-13 (HeaderDropLogging: all, first-per-name, quiet; bounded case-insensitive dedup), +// OBS-19 (the verbosity policy's LEVELS: warn every occurrence; warn the first drop per name then verbose) +import {afterEach, beforeEach, describe, expect, test} from 'bun:test'; +import { + getGlobalLogger, + setGlobalLogger, + type Logger, + type LogLevel, +} from '@dexpace/core'; +import {createDropLogger} from './drop-log.js'; + +let logged: { + level: LogLevel; + event?: string; + fields: Record<string, unknown>; +}[] = []; +let originalLogger: Logger; + +beforeEach(() => { + logged = []; + originalLogger = getGlobalLogger(); + setGlobalLogger({ + atLevel: (level: LogLevel) => { + const entry: { + level: LogLevel; + event?: string; + fields: Record<string, unknown>; + } = { + level, + fields: {}, + }; + const mockEvent = { + event: (name: string) => { + entry.event = name; + return mockEvent; + }, + field: (key: string, value: unknown) => { + entry.fields[key] = value; + return mockEvent; + }, + cause: () => mockEvent, + emit: () => { + logged.push(entry); + }, + }; + return mockEvent; + }, + withContext: () => originalLogger, + }); +}); + +afterEach(() => { + setGlobalLogger(originalLogger); +}); + +describe('createDropLogger (TRANSPORT-13)', () => { + test('mode quiet logs nothing', () => { + const logger = createDropLogger('quiet'); + logger(['Content-Length', 'Host']); + expect(logged.length).toBe(0); + }); + + test('mode all logs every occurrence, every one loudly (OBS-19)', () => { + const logger = createDropLogger('all'); + logger(['Content-Length']); + logger(['content-length']); + expect(logged.length).toBe(2); + expect(logged.map(l => l.level)).toEqual(['warning', 'warning']); + }); + + test('mode first-per-name warns once per name, then goes verbose (OBS-19)', () => { + const logger = createDropLogger('first-per-name'); + logger(['Content-Length']); + logger(['content-length']); + logger(['X-Custom']); + expect(logged.length).toBe(3); + expect(logged.map(l => l.level)).toEqual(['warning', 'verbose', 'warning']); + expect(logged.map(l => l.fields)).toEqual([ + {header: 'content-length'}, + {header: 'content-length'}, + {header: 'x-custom'}, + ]); + }); + + test('bounded dedup drains to MAX_LOGGED_DROP_NAMES', () => { + const logger = createDropLogger('first-per-name'); + const names = Array.from({length: 150}, (_, i) => `x-header-${String(i)}`); + logger(names); + expect(logged.length).toBe(150); + expect(logged.every(l => l.level === 'warning')).toBe(true); + }); +}); diff --git a/packages/transport-shared/src/drop-log.ts b/packages/transport-shared/src/drop-log.ts new file mode 100644 index 0000000..f586caf --- /dev/null +++ b/packages/transport-shared/src/drop-log.ts @@ -0,0 +1,77 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/drop-log.ts +import {getGlobalLogger, type LogLevel} from '@dexpace/core'; + +/** + * Logging mode for dropped headers (TRANSPORT-13, OBS-19). + * + * @internal + */ +export type HeaderDropLogging = 'all' | 'first-per-name' | 'quiet'; + +/** Bound on the dedup set so an attacker synthesising distinct names cannot grow it (TRANSPORT-13, XCUT-14). */ +const MAX_LOGGED_DROP_NAMES = 128; + +function emitDropLog(key: string, level: LogLevel): void { + try { + getGlobalLogger() + .atLevel(level) + .event('http.header.dropped') + .field('header', key) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request + } +} + +/** + * Evicts the oldest name once the set has outgrown its bound. One eviction per insert is enough + * precisely because this runs on every insert -- the set can only ever be one over the cap. + */ +function trimSeen(seen: Set<string>): void { + if (seen.size > MAX_LOGGED_DROP_NAMES) { + const first = seen.values().next().value; + if (first !== undefined) { + seen.delete(first); + } + } +} + +/** + * Creates a drop logger function adhering to the requested logging mode and bounded dedup policy. + * + * **The mode selects a LEVEL, not merely whether a line is written** (OBS-19). `'all'` warns on + * every occurrence; `'first-per-name'` warns the first drop of each header name and drops the rest + * to `verbose`, which is OBS-19's default and the mode whose conformance text reads "repeatedly + * drop the same name and assert exactly one WARN then verbose lines"; `'quiet'` writes nothing at + * all, which is TRANSPORT-13's own third mode ("all quiet"). + * + * Until 2026-09-02 every mode emitted at `verbose`, so the policy was configurable in name only -- + * a caller-set header vanishing before it reached the wire was indistinguishable, at any level a + * production logger enables, from nothing having happened. + * + * @internal + */ +export function createDropLogger( + mode: HeaderDropLogging, +): (dropped: readonly string[]) => void { + if (mode === 'quiet') { + return () => undefined; + } + const seen = new Set<string>(); + return (dropped: readonly string[]) => { + for (const name of dropped) { + const key = name.toLowerCase(); + if (mode === 'all') { + emitDropLog(key, 'warning'); + continue; + } + const firstOfThisName = !seen.has(key); + if (firstOfThisName) { + seen.add(key); + trimSeen(seen); + } + emitDropLog(key, firstOfThisName ? 'warning' : 'verbose'); + } + }; +} diff --git a/packages/transport-shared/src/header-mapping.test.ts b/packages/transport-shared/src/header-mapping.test.ts new file mode 100644 index 0000000..6d02976 --- /dev/null +++ b/packages/transport-shared/src/header-mapping.test.ts @@ -0,0 +1,142 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/header-mapping.test.ts +// Exercises: TRANSPORT-10 (Content-Type authority), TRANSPORT-11 (framing-header drop set, verbose log), +// TRANSPORT-12 (per-header graceful degradation), TRANSPORT-14 (lenient inbound copy, obs-text preserved, control-byte header dropped) +import {describe, expect, test} from 'bun:test'; +import {Headers} from '@dexpace/core'; +import { + CONTROL_BYTE, + degradeInboundHeaders, + mapOutboundHeaders, +} from './header-mapping.js'; + +describe('mapOutboundHeaders', () => { + test('drops framing headers the native client computes', () => { + const {sent, dropped} = mapOutboundHeaders( + Headers.newBuilder() + .set('Content-Length', '999') + .set('X-Custom', 'v') + .build(), + ['content-length', 'host', 'transfer-encoding'], + ); + expect(sent.get('content-length')).toBeUndefined(); + expect(sent.get('x-custom')).toBe('v'); + expect(dropped).toContain('content-length'); + }); + + test('an explicit Content-Type is never overwritten by a body-derived one', () => { + const {sent} = mapOutboundHeaders( + Headers.newBuilder().set('Content-Type', 'text/plain').build(), + [], + {bodyDerivedMediaType: 'application/json'}, + ); + expect(sent.get('content-type')).toBe('text/plain'); + }); + + test('sets body-derived Content-Type when none is provided', () => { + const {sent} = mapOutboundHeaders( + Headers.newBuilder().set('X-Custom', 'v').build(), + [], + {bodyDerivedMediaType: 'application/json'}, + ); + expect(sent.get('content-type')).toBe('application/json'); + expect(sent.get('x-custom')).toBe('v'); + }); +}); + +describe('mapOutboundHeaders graceful degradation (TRANSPORT-12)', () => { + test('a value the outbound grammar rejects drops that header only', () => { + // `addInbound` is the lenient path (HTTP-19) and admits obs-text; the strict outbound `add` + // does not. A Headers built from a server response and re-sent is the realistic way a + // model-valid, wire-invalid value reaches this function. + const inbound = Headers.newBuilder() + .addInbound('X-Obs-Text', 'caf\u00e9') + .add('X-Kept', 'value') + .build(); + const {sent, dropped} = mapOutboundHeaders(inbound, []); + expect(sent.get('x-obs-text')).toBeUndefined(); + expect(sent.get('x-kept')).toBe('value'); + expect(dropped).toEqual(['x-obs-text']); + }); + + test('an unusable body-derived media type is dropped rather than failing the mapping', () => { + const {sent, dropped} = mapOutboundHeaders( + Headers.newBuilder().set('X-Kept', 'value').build(), + [], + {bodyDerivedMediaType: 'text/plain\u0000'}, + ); + expect(sent.get('content-type')).toBeUndefined(); + expect(sent.get('x-kept')).toBe('value'); + expect(dropped).toEqual(['content-type']); + }); +}); + +describe('CONTROL_BYTE (TRANSPORT-14)', () => { + // Asserted on the character class itself, not through `degradeInboundHeaders`, because the two + // gates that reject an inbound value are redundant by construction: the regex below and + // `Headers.addInbound`'s own `hasForbiddenInboundValueByte`, whose throw the `try`/`catch` turns + // into the same drop. A value only the second one caught looked identical from outside, which is + // how `\x0A` stayed out of this class from Phase 8a to audit #67 / #82. + test('every C0 control byte except HTAB is refused, LF included', () => { + for (let code = 0x00; code <= 0x1f; code += 1) { + const value = `v${String.fromCharCode(code)}alue`; + expect([code, CONTROL_BYTE.test(value)]).toEqual([code, code !== 0x09]); + } + expect(CONTROL_BYTE.test('v\x7falue')).toBe(true); + }); + + test('LF is refused whether or not it is preceded by CR', () => { + // The obs-fold shape (`\r\n ` continuation) and a bare LF are both header injection on the + // inbound path; RFC 9110 5.5 forbids either from reaching a field value. + expect(CONTROL_BYTE.test('one\nvalue')).toBe(true); + expect(CONTROL_BYTE.test('one\r\n two')).toBe(true); + }); + + test('HTAB and obs-text are carried, not refused', () => { + // TRANSPORT-14's own SHOULD: a non-ASCII byte in a value is preserved rather than stripped, and + // HTAB is legal whitespace inside a field value (RFC 9110 5.5). + expect(CONTROL_BYTE.test('one\ttwo')).toBe(false); + expect(CONTROL_BYTE.test('café')).toBe(false); + }); +}); + +describe('degradeInboundHeaders', () => { + test('drops a header whose value carries a line feed, keeps the rest', () => { + const {headers, dropped} = degradeInboundHeaders([ + ['x-injected', 'value\nx-forged: yes'], + ['x-good', 'value'], + ]); + expect(headers.get('x-injected')).toBeUndefined(); + expect(headers.get('x-forged')).toBeUndefined(); + expect(headers.get('x-good')).toBe('value'); + expect(dropped).toEqual(['x-injected']); + }); + + test('drops a header whose value carries a control byte, keeps the rest', () => { + const {headers, dropped} = degradeInboundHeaders([ + ['x-bad', 'v\x01alue'], + ['x-good', 'value'], + ]); + expect(headers.get('x-bad')).toBeUndefined(); + expect(headers.get('x-good')).toBe('value'); + expect(dropped).toEqual(['x-bad']); + }); + + test('drops a header whose name carries non-ASCII or control characters', () => { + const {headers, dropped} = degradeInboundHeaders([ + ['x-bad\x02name', 'value'], + ['x-bad-café', 'value'], + ['x-good', 'value'], + ]); + expect(headers.get('x-bad\x02name')).toBeUndefined(); + expect(headers.get('x-bad-café')).toBeUndefined(); + expect(headers.get('x-good')).toBe('value'); + expect(dropped).toContain('x-bad\x02name'); + expect(dropped).toContain('x-bad-café'); + }); + + test('preserves an obs-text (non-ASCII) byte in a value rather than stripping it', () => { + const {headers} = degradeInboundHeaders([['x-name', 'café']]); + expect(headers.get('x-name')).toBe('café'); + }); +}); diff --git a/packages/transport-shared/src/header-mapping.ts b/packages/transport-shared/src/header-mapping.ts new file mode 100644 index 0000000..d570e4c --- /dev/null +++ b/packages/transport-shared/src/header-mapping.ts @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/header-mapping.ts +import {Headers} from '@dexpace/core'; + +/** + * Every byte TRANSPORT-14 refuses in an *inbound* header value: the C0 controls except HTAB, plus + * DEL. Deliberately the same character class as `@dexpace/core`'s `hasForbiddenInboundValueByte`, + * which `Headers.addInbound` applies a few lines later \u2014 obs-text (\u2265 0x80) is carried, HTAB is + * carried, everything else below 0x20 is not. + * + * `\x0A` was missing from Phase 8a until audit #67 / #82: the class read `\x0B-\x1F`, excepting LF + * alongside the intended HTAB. Nothing observable changed, because `addInbound` rejected the value + * anyway and the `try`/`catch` in {@link degradeInboundHeaders} recorded the same drop \u2014 which is + * exactly why it survived, and why the test for this constant reads the class directly rather than + * going through that function. + * + * Exported for that test only. The package barrel deliberately does not re-export it: it is one + * half of a redundant pair, not plumbing another transport should reach for. + * + * @internal + */ +/* eslint-disable no-control-regex -- RFC 9110 requires testing for ASCII control characters */ +export const CONTROL_BYTE = /[\x00-\x08\x0A-\x1F\x7F]/u; +const NON_ASCII_OR_CONTROL = /[\x00-\x1F\x7F-\uFFFF]/u; +/* eslint-enable no-control-regex -- re-enable */ + +/** + * Options for outbound header mapping. + * + * @internal + */ +export interface MapOutboundHeadersOptions { + /** A media type derived from the request body to use if Content-Type is absent. */ + readonly bodyDerivedMediaType?: string | undefined; +} + +/** + * Filters forbidden framing headers and applies per-header degradation for outbound requests (TRANSPORT-10-12). + * + * @internal + */ +export function mapOutboundHeaders( + headers: Headers, + forbidden: readonly string[], + opts: MapOutboundHeadersOptions = {}, +): {sent: Headers; dropped: readonly string[]} { + const forbiddenSet = new Set(forbidden.map(h => h.toLowerCase())); + const dropped: string[] = []; + const builder = Headers.newBuilder(); + for (const [name, value] of headers.entries()) { + if (forbiddenSet.has(name.toLowerCase())) { + dropped.push(name.toLowerCase()); + continue; + } + try { + builder.add(name, value); + } catch { + dropped.push(name.toLowerCase()); + } + } + if ( + opts.bodyDerivedMediaType !== undefined && + headers.get('content-type') === undefined + ) { + try { + builder.set('Content-Type', opts.bodyDerivedMediaType); + } catch { + dropped.push('content-type'); + } + } + return {sent: builder.build(), dropped}; +} + +/** + * Leniently copies inbound response headers, dropping malformed entries while preserving obs-text (TRANSPORT-14). + * + * @internal + */ +export function degradeInboundHeaders( + raw: Iterable<readonly [string, string]>, +): {headers: Headers; dropped: readonly string[]} { + const dropped: string[] = []; + const builder = Headers.newBuilder(); + for (const [name, value] of raw) { + if (NON_ASCII_OR_CONTROL.test(name) || CONTROL_BYTE.test(value)) { + dropped.push(name); + continue; + } + try { + builder.addInbound(name, value); + } catch { + dropped.push(name); + } + } + return {headers: builder.build(), dropped}; +} diff --git a/packages/transport-shared/src/index.ts b/packages/transport-shared/src/index.ts new file mode 100644 index 0000000..ebfcf58 --- /dev/null +++ b/packages/transport-shared/src/index.ts @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/index.ts +export {abortToSdkError} from './abort-mapping.js'; +export {hasNoResponseBody} from './body-less.js'; +export { + isMaterializable, + materializeBody, + producerFailure, + pumpBody, + type BodyPump, +} from './body-pump.js'; +export {requireValidDefaultTimeoutMs} from './default-timeout.js'; +export { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; +export {createDropLogger, type HeaderDropLogging} from './drop-log.js'; +export { + degradeInboundHeaders, + mapOutboundHeaders, + type MapOutboundHeadersOptions, +} from './header-mapping.js'; +export {forkSignal, type ForkedSignal} from './signal-fork.js'; diff --git a/packages/transport-shared/src/signal-fork.test.ts b/packages/transport-shared/src/signal-fork.test.ts new file mode 100644 index 0000000..37ff026 --- /dev/null +++ b/packages/transport-shared/src/signal-fork.test.ts @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/signal-fork.test.ts +// Exercises: SEAM-16 (an abort after delivery must not reach the native client), SEAM-13/TRANSPORT-7 +// (an abort before delivery must), TRANSPORT-9 (the transport can cancel a native call it abandons, +// with or without a caller signal) +import {describe, expect, test} from 'bun:test'; +import {forkSignal} from './signal-fork.js'; + +describe('forkSignal', () => { + test('still yields a live signal when the caller supplied none', () => { + // The transport's own cancellation handle. Until audit #67 / #82 this returned `undefined`, so + // a send with no caller signal and no timeout dispatched with none -- and a request-body + // producer that failed mid-flight had no way to take the native call down with it. + const fork = forkSignal(undefined); + expect(fork.signal).toBeInstanceOf(AbortSignal); + expect(fork.signal.aborted).toBe(false); + expect(() => { + fork.detach(); + }).not.toThrow(); + }); + + test('forwards an abort that fires while still attached, reason and all', () => { + const controller = new AbortController(); + const fork = forkSignal(controller.signal); + const reason = new Error('caller changed their mind'); + controller.abort(reason); + expect(fork.signal.aborted).toBe(true); + // Carried verbatim because `isTimeoutSignal` reads `reason.name`: a fork that invented its own + // reason would turn every per-call timeout into a CancellationError (TRANSPORT-4). + expect(fork.signal.reason).toBe(reason); + }); + + test('an already-aborted source forks as already aborted', () => { + const controller = new AbortController(); + controller.abort(new Error('too late')); + const fork = forkSignal(controller.signal); + expect(fork.signal.aborted).toBe(true); + }); + + test('an abort after detach never reaches the fork (SEAM-16)', () => { + const controller = new AbortController(); + const fork = forkSignal(controller.signal); + fork.detach(); + fork.detach(); // idempotent + controller.abort(new Error('after delivery')); + expect(fork.signal.aborted).toBe(false); + }); +}); + +describe('forkSignal.abort (TRANSPORT-9)', () => { + test('cancels the native call with the reason the transport gave up for', () => { + const fork = forkSignal(undefined); + const reason = new Error('producer exploded'); + fork.abort(reason); + expect(fork.signal.aborted).toBe(true); + expect(fork.signal.reason).toBe(reason); + }); + + test('cancels a fork that has a source too, without touching the source', () => { + const controller = new AbortController(); + const fork = forkSignal(controller.signal); + fork.abort(new Error('producer exploded')); + expect(fork.signal.aborted).toBe(true); + // The caller's own signal is not the transport's to abort; only the fork it dispatched with. + expect(controller.signal.aborted).toBe(false); + }); + + test('is a no-op after detach, so a delivered body is never torn out (SEAM-16)', () => { + // The latch is what keeps the second direction of the fork from becoming the very violation + // the first direction exists to prevent. + const fork = forkSignal(undefined); + fork.detach(); + fork.abort(new Error('too late to matter')); + expect(fork.signal.aborted).toBe(false); + }); +}); diff --git a/packages/transport-shared/src/signal-fork.ts b/packages/transport-shared/src/signal-fork.ts new file mode 100644 index 0000000..55c84d3 --- /dev/null +++ b/packages/transport-shared/src/signal-fork.ts @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/signal-fork.ts + +/** + * A caller signal, forwarded to the native client only for as long as the transport wants it — and + * a handle the transport can pull itself. + * + * @internal + */ +export interface ForkedSignal { + /** + * Hand this to the native client instead of the caller's own signal. + * + * Always present, even when the caller supplied no signal and no timeout was composed. That is + * not symmetry for its own sake: {@link ForkedSignal.abort} is the only way a transport can + * cancel a native call it has decided to abandon, and a send with no caller signal is exactly + * the case where a failed request-body producer would otherwise leave one running forever + * (TRANSPORT-9, SEAM-30). A controller nobody ever aborts costs one allocation and is + * indistinguishable, to the native client, from no signal at all. + */ + readonly signal: AbortSignal; + /** + * Stops forwarding, and latches the fork: a later {@link ForkedSignal.abort} is a no-op too. + * Idempotent. Called at delivery, which is the moment the response stops being the transport's. + */ + detach(): void; + /** + * Cancels the in-flight native call, so a response that arrives afterwards is refused rather + * than stranded with its body neither read nor released (TRANSPORT-9). + * + * A no-op after {@link ForkedSignal.detach}, which is what keeps this from becoming the SEAM-16 + * violation the fork exists to prevent: once a body has been handed to the caller, nothing in + * this transport may close it. + * + * @param reason - the abort reason; the failure that made the transport give up. + */ + abort(reason: unknown): void; +} + +/** + * Forks `source` into a signal the transport controls. + * + * SEAM-16 forbids a signal abort that fires *after* the send resolved from closing the + * already-delivered response body — the caller still owns it, even when discarding the value. Both + * WHATWG `fetch` and undici tie the response body's lifetime to whatever signal they were handed, so + * passing the caller's signal straight through violates that clause: a later `controller.abort()` + * truncates a body the caller was reading. Forwarding through a fork the transport detaches at + * delivery keeps cancellation live for the whole in-flight window (SEAM-13, TRANSPORT-7) and inert + * afterwards. + * + * The fork is two-way. It carries the caller's abort *in*, and it lets the transport cancel the + * native call *out* — the second direction added by audit #67 / #82, because a request-body + * producer that fails while the native call is still pending has to take that call down with it. + * + * @param source - the composed caller/timeout signal, if any. + * @returns the signal to dispatch with, the detach the transport calls on delivery, and the abort + * it calls when it abandons the exchange. + * + * @internal + */ +export function forkSignal(source: AbortSignal | undefined): ForkedSignal { + const controller = new AbortController(); + let detached = false; + const forward = (): void => { + controller.abort(source?.reason); + }; + if (source !== undefined) { + if (source.aborted) forward(); + else source.addEventListener('abort', forward, {once: true}); + } + return { + signal: controller.signal, + detach: () => { + detached = true; + // removeEventListener is idempotent and a no-op for a listener never added, so a detach on + // both the success and failure path, with or without a source, is safe. + source?.removeEventListener('abort', forward); + }, + abort: (reason: unknown) => { + if (detached) return; + controller.abort(reason); + }, + }; +} diff --git a/packages/transport-shared/tsconfig.build.json b/packages/transport-shared/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/transport-shared/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/transport-shared/tsconfig.json b/packages/transport-shared/tsconfig.json new file mode 100644 index 0000000..6a85a2a --- /dev/null +++ b/packages/transport-shared/tsconfig.json @@ -0,0 +1,17 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "stripInternal": false, + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md new file mode 100644 index 0000000..54edeee --- /dev/null +++ b/packages/transport-undici/README.md @@ -0,0 +1,132 @@ +# @dexpace/transport-undici + +The full-featured `Transport` for the dexpace SDK, built on `undici` — connection-pool control, +proxy routing, and real ownership-aware `close()` semantics. Exactly one external dependency. + +```sh +bun add @dexpace/transport-undici @dexpace/core +``` + +```typescript +import {Request} from '@dexpace/core'; +import {undiciTransport} from '@dexpace/transport-undici'; + +const transport = undiciTransport({ + agentOptions: {connections: 32}, + defaultTimeoutMs: 30_000, +}); + +try { + const response = await transport.send( + Request.newBuilder().url('https://example.com/v1/users').build(), + ); + await response.close(); +} finally { + await transport.close(); // this transport owns a real dispatcher — always close it +} +``` + +`close()` is the teardown, not `await using`. The factory returns a plain `Transport`: the disposal +member is installed only when `Symbol.asyncDispose` exists, which it does not on this package's +declared `engines.node` floor of `>=20.3` (the symbol arrived in 20.4). Declaring `AsyncDisposable` +in the `.d.ts` regardless would be a type that lies on the supported runtime — `NFR-10` forbids it, +and the [`await using` support row](https://github.com/dexpace/nodejs-sdk/blob/main/docs/work/mvp/2026-09-04-open-items-dissolution.md#d-nfr-10-await-using) in the dissolved open-items register +records the decision and the four reasons the floor does not move instead. Unlike `@dexpace/transport-fetch`, closing here is not optional: see below. + +## Dispatcher ownership + +Exactly one decision, made once at construction, fixing both the dispatcher and who closes it: + +| Option supplied | Dispatcher used | Closed by `close()` | +|---|---|---| +| `dispatcher` | yours, as-is | **no** — a caller-supplied client is never touched (`SEAM-14`) | +| `proxy` | a `ProxyAgent` this package constructs, plus an `Agent` for `NO_PROXY` hosts | yes, both | +| neither | an `Agent` this package constructs | yes | + +Supplying **both** `dispatcher` and `proxy` is a construction-time `TypeError`, not a silent win for +one: a bring-your-own dispatcher may already be a `ProxyAgent`, and ignoring either option would +hide which is in force. `close()` is idempotent and concurrent calls share one teardown +(`TRANSPORT-15`/`TRANSPORT-16`). + +`close()` **destroys** the dispatchers it owns rather than draining them: `TRANSPORT-16` requires a +non-blocking shutdown with no unbounded await, and a graceful close would stall teardown for as long +as one in-flight send against a slow peer takes. Sends still in flight therefore reject with the +terminal `CancellationError`, and so does a `send()` issued after `close()` — this transport's +documented `SEAM-15` post-close mode. It cannot succeed over a dispatcher that no longer exists, so +it is not reported as a retryable failure. + +## Proxy support and its two real limits + +`ProxyOptions` routes here in full: address, Basic credentials, and `NO_PROXY`/`nonProxyHosts` +bypass globs, which route over a separate direct `Agent` rather than being tunnelled anyway. + +**`type` must be `http`.** undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and reads its `uri` as +a URL, so `socks4`/`socks5` — which `ProxyType` admits and core resolves from `ALL_PROXY` +(`CFG-22`) — are refused at construction with a `TypeError` naming the type. Before 2026-09-05 they +reached `new ProxyAgent({uri: 'socks5://…'})` and escaped this factory as an undici +`InvalidArgumentError`, untyped and undocumented. Neither shipped transport can carry SOCKS; +`docs/deviations.md` records the gap. + +**A custom `challengeHandler` cannot be dispatched**, and the limitation is surfaced rather than +silently misbehaving (`TRANSPORT-30`): + +- undici's `ProxyAgent` takes its credential **only** from its own constructor and rejects any + per-request `Proxy-Authorization` header with `InvalidArgumentError` — a deliberate security fix + on their side, not an oversight. The constructor runs before any challenge has been seen, so + there is no point at which a handler-minted credential could be applied to the exchange that + provoked it. +- Configuring one therefore emits a WARN at construction, and a second WARN the first time a proxy + actually answers `407`. The `407` is surfaced to the caller unchanged, for its own auth layer. +- Proxy auth falls back to **Basic**: `ProxyOptions.credentials`, which is passed to the + `ProxyAgent` constructor as a token. Credentials are never logged, and are never sent in answer to + an origin-server `401`. +- A per-request `Proxy-Authorization` header is dropped from the outbound pass whenever a proxy is + configured — forwarding one would turn every proxied send into a hard failure. The drop is logged + by name like any other. + +## Behavior worth knowing + +- File bodies (`body.kind === 'file'`, e.g. `@dexpace/body-file`'s `fileBody()`) are written + through the descriptor's own `writeTo`, exactly as in the `fetch` transport — buffered below + 1 MB, streamed above it. Until 2026-09-05 this transport instead handed + `createReadStream(path, {start, end})` to undici: one fewer userspace copy, and no `writeTo`, so + `BODY-13`'s `transferred === count` check never ran and a file truncated between `stat` and + `send` uploaded its remaining bytes and returned 200. `content-length` is dropped outbound, so + the framing could not catch it either. `TRANSPORT-28`'s zero-copy clause is a SHOULD that no + user-space path in either client can honour anyway (Deviation Ledger item 13); its MUSTs — a file + body is replayable, and exactly its declared byte range reaches the wire — are honoured by the + descriptor. Recognition, where it is still needed, stays structural: this package does not depend + on `@dexpace/body-file`. +- Redirects are pinned off (`maxRedirections: 0`) even behind a bring-your-own dispatcher that may + carry a redirect interceptor. The pipeline is the single redirect authority. +- `Connection` is **not** dropped outbound — §17's own note is that an undici-class transport + forwards it — but only with a value undici will carry (`close` or `keep-alive`, matched + case-insensitively). Any other value is dropped, because undici rejects it outright. +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped because undici computes them; + `Expect`, `Keep-Alive` and `Upgrade` because undici refuses them + (`InvalidArgumentError`/`NotSupportedError` out of its own argument validation, before anything + reaches the wire). So is any header name outside RFC 9110 `token` — `@dexpace/core` admits every + printable ASCII byte in a name, undici does not. Every one of these is a drop logged by name + (`TRANSPORT-12`), never a failed send. +- Destroying the dispatcher mid-flight surfaces as the terminal `CancellationError`, while a timeout + on the same path stays the retryable `TransportFailureError` (`TRANSPORT-8`). +- An argument undici refuses outright — a non-`http(s)` origin such as `ftp://`, `CONNECT` as a + method, a per-request `Proxy-Authorization` on a bring-your-own `ProxyAgent` — is a bare + `TypeError` outside the `IoError` tree, so `retry/classify.ts`'s allow-list makes it non-retryable + (`RETRY-2`). A failed *exchange* stays the retryable `TransportFailureError` (`TRANSPORT-20`). The + table that tells them apart moved to `@dexpace/transport-shared` on 2026-09-05 so + `@dexpace/transport-fetch` answers identically (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. undici's dispatcher always hands back a + `BodyReadable`, so this transport `dump()`s the one it declines to expose — returning the + connection to the pool — rather than wrapping an empty stream the caller would have to read to + discover was empty. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `undiciTransport()`, raised + before any dispatcher is allocated, not a failure on the first send (`HTTP-35`). +- `Response.protocol` is always `HTTP_1_1`: undici's `ResponseData` does not surface the negotiated + version. A Deviation Ledger row, not a silent gap. + +## Conformance + +Proven against the shared `TRANSPORT-N` suite in `@dexpace/transport-conformance`, the same one +`@dexpace/transport-fetch` runs, so the two adapters cannot drift. diff --git a/packages/transport-undici/api-extractor.json b/packages/transport-undici/api-extractor.json new file mode 100644 index 0000000..455423c --- /dev/null +++ b/packages/transport-undici/api-extractor.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", + "mainEntryPointFilePath": "<projectFolder>/dist/index.d.ts", + "apiReport": { + "enabled": true, + "reportFolder": "<projectFolder>/etc/" + }, + "docModel": { + "enabled": false + }, + "dtsRollup": { + "enabled": false + }, + "messages": { + "extractorMessageReporting": { + "ae-forgotten-export": { + "logLevel": "error", + "addToApiReportFile": false + } + } + } +} diff --git a/packages/transport-undici/etc/transport-undici.api.md b/packages/transport-undici/etc/transport-undici.api.md new file mode 100644 index 0000000..1b39350 --- /dev/null +++ b/packages/transport-undici/etc/transport-undici.api.md @@ -0,0 +1,27 @@ +## API Report File for "@dexpace/transport-undici" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts + +import type { Agent } from 'undici'; +import type { Dispatcher } from 'undici'; +import { HeaderDropLogging } from '@dexpace/transport-shared'; +import { ProxyOptions } from '@dexpace/core'; +import { Transport } from '@dexpace/core'; + +// @public +export function undiciTransport(options?: UndiciTransportOptions): Transport; + +// @public +export interface UndiciTransportOptions { + readonly agentOptions?: Agent.Options; + readonly defaultTimeoutMs?: number; + readonly dispatcher?: Dispatcher; + readonly headerDropLogging?: HeaderDropLogging; + readonly proxy?: ProxyOptions; +} + +// (No @packageDocumentation comment for this package) + +``` diff --git a/packages/transport-undici/package.json b/packages/transport-undici/package.json new file mode 100644 index 0000000..a285d25 --- /dev/null +++ b/packages/transport-undici/package.json @@ -0,0 +1,55 @@ +{ + "name": "@dexpace/transport-undici", + "version": "0.0.0", + "description": "Undici-based transport adapter with proxy and connection pooling support for the dexpace Node.js SDK.", + "license": "MIT", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "engines": { + "node": ">=20.3" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dexpace/nodejs-sdk.git", + "directory": "packages/transport-undici" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "sideEffects": false, + "dependencies": { + "@dexpace/transport-shared": "workspace:*", + "undici": "^6.21.1" + }, + "peerDependencies": { + "@dexpace/core": "workspace:*" + }, + "peerDependenciesMeta": { + "@dexpace/core": { + "optional": false + } + }, + "devDependencies": { + "@dexpace/core": "workspace:*", + "@dexpace/transport-conformance": "workspace:*", + "@microsoft/api-extractor": "catalog:", + "expect-type": "catalog:", + "fast-check": "catalog:", + "typescript": "catalog:" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "test": "bun test", + "api:local": "api-extractor run --local", + "api:ci": "api-extractor run", + "prepublishOnly": "bun run build && bun run api:ci && publint . && attw --pack . --ignore-rules cjs-resolves-to-esm" + } +} diff --git a/packages/transport-undici/src/challenge-handler.test.ts b/packages/transport-undici/src/challenge-handler.test.ts new file mode 100644 index 0000000..942c5db --- /dev/null +++ b/packages/transport-undici/src/challenge-handler.test.ts @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/challenge-handler.test.ts +// Exercises: TRANSPORT-30 -- an undispatchable custom proxy challenge handler is surfaced with a +// WARN at construction and again the first time a 407 actually arrives, proxy auth falls back to +// Basic, an origin-server 401 is never treated as a proxy challenge, and no credential is ever logged +import {afterEach, beforeEach, describe, expect, test} from 'bun:test'; +import { + createProxyOptions, + getGlobalLogger, + Headers, + Protocol, + Request, + Response, + setGlobalLogger, + Status, + type Logger, + type ProxyOptions, +} from '@dexpace/core'; +import { + createProxyChallengeReporter, + warnIfCustomChallengeHandler, +} from './challenge-handler.js'; + +/** Every field value the global logger saw, so "credentials are never logged" is checkable. */ +let logged: string[] = []; +let previousLogger: Logger; + +beforeEach(() => { + logged = []; + previousLogger = getGlobalLogger(); + const capturing: Logger = { + atLevel: level => { + const entry = { + field: (key: string, value: unknown) => { + logged.push(`${key}=${String(value)}`); + return entry; + }, + event: (name: string) => { + logged.push(`event=${name}@${level}`); + return entry; + }, + cause: (error: unknown) => { + logged.push(`cause=${String(error)}`); + return entry; + }, + emit: () => undefined, + }; + return entry; + }, + withContext: () => capturing, + }; + setGlobalLogger(capturing); +}); + +afterEach(() => { + setGlobalLogger(previousLogger); +}); + +const SECRET = 'hunter2'; + +function proxyWithHandler(): ProxyOptions { + return createProxyOptions({ + type: 'http', + host: 'proxy.internal', + port: 8080, + credentials: {username: 'user', password: SECRET}, + challengeHandler: () => 'Bearer minted-token', + }); +} + +function makeResponse(status: number): Response { + return Response.newBuilder() + .request(Request.newBuilder().url('http://localhost').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(Headers.newBuilder().build()) + .build(); +} + +describe('warnIfCustomChallengeHandler', () => { + test('says nothing without a proxy, or with a proxy carrying no custom handler', () => { + warnIfCustomChallengeHandler(undefined); + warnIfCustomChallengeHandler( + createProxyOptions({type: 'http', host: 'proxy.internal', port: 8080}), + ); + expect(logged).toEqual([]); + }); + + test('warns at construction, naming the proxy address but never its credentials', () => { + warnIfCustomChallengeHandler(proxyWithHandler()); + const rendered = logged.join('|'); + expect(rendered).toContain( + 'event=proxy.challengeHandler.unsupported@warning', + ); + expect(rendered).toContain('proxy.host=proxy.internal'); + expect(rendered).toContain('proxy.port=8080'); + expect(rendered).not.toContain(SECRET); + }); +}); + +describe('createProxyChallengeReporter', () => { + test('is inert when no custom handler is configured', () => { + const report = createProxyChallengeReporter( + createProxyOptions({type: 'http', host: 'proxy.internal', port: 8080}), + ); + report(makeResponse(407)); + expect(logged).toEqual([]); + }); + + test('never treats an origin-server 401 as a proxy challenge', () => { + const report = createProxyChallengeReporter(proxyWithHandler()); + report(makeResponse(401)); + report(makeResponse(200)); + expect(logged).toEqual([]); + }); + + test('warns on the first 407 and stays quiet on every one after it', () => { + const report = createProxyChallengeReporter(proxyWithHandler()); + report(makeResponse(407)); + const afterFirst = logged.length; + report(makeResponse(407)); + report(makeResponse(407)); + expect(logged.length).toBe(afterFirst); + const rendered = logged.join('|'); + expect(rendered).toContain('event=proxy.challenge.unanswered@warning'); + expect(rendered).not.toContain(SECRET); + expect(rendered).not.toContain('minted-token'); + }); + + test('a logger that throws never fails the request it was describing (OBS-20)', () => { + setGlobalLogger({ + atLevel: () => { + throw new Error('logger exploded'); + }, + withContext: () => getGlobalLogger(), + }); + const report = createProxyChallengeReporter(proxyWithHandler()); + expect(() => { + report(makeResponse(407)); + }).not.toThrow(); + expect(() => { + warnIfCustomChallengeHandler(proxyWithHandler()); + }).not.toThrow(); + }); +}); diff --git a/packages/transport-undici/src/challenge-handler.ts b/packages/transport-undici/src/challenge-handler.ts new file mode 100644 index 0000000..778aaa2 --- /dev/null +++ b/packages/transport-undici/src/challenge-handler.ts @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/challenge-handler.ts +import { + getGlobalLogger, + type LogEvent, + type ProxyOptions, + type Response, +} from '@dexpace/core'; + +/** OBS-20: a logger failure must never fail the request it was describing. */ +function safeWarn(event: string, decorate: (entry: LogEvent) => void): void { + try { + const entry = getGlobalLogger().atLevel('warning').event(event); + decorate(entry); + entry.emit(); + } catch { + // Deliberately swallowed -- see OBS-20. + } +} + +/** Names the proxy without ever rendering its credentials (TRANSPORT-30, redaction rules). */ +function describeProxy(entry: LogEvent, proxy: ProxyOptions): LogEvent { + return entry.field('proxy.host', proxy.host).field('proxy.port', proxy.port); +} + +function hasCustomChallengeHandler(proxy: ProxyOptions | undefined): boolean { + return proxy !== undefined && typeof proxy.challengeHandler === 'function'; +} + +/** + * TRANSPORT-30's discoverability clause, at construction: undici cannot dispatch a custom + * (non-Basic) proxy challenge handler at all, so a configured one is surfaced with a WARN rather + * than silently ignored. + * + * The reason is a hard constraint of the native client, not a gap in this package: `ProxyAgent` + * rejects a per-request `Proxy-Authorization` header with `InvalidArgumentError` — it was removed + * deliberately as a security fix — and takes its credential only from its own constructor, which + * runs before any challenge has been seen. There is therefore no point at which a handler-minted + * credential could be applied to the exchange that provoked it. Proxy auth falls back to Basic: + * `ProxyOptions.credentials`, which this transport does pass to the `ProxyAgent` constructor. + * + * @param proxy - the configured proxy, if any. + * + * @internal + */ +export function warnIfCustomChallengeHandler( + proxy: ProxyOptions | undefined, +): void { + if (proxy === undefined || !hasCustomChallengeHandler(proxy)) return; + safeWarn('proxy.challengeHandler.unsupported', entry => { + describeProxy(entry, proxy).field( + 'detail', + 'undici takes proxy credentials only from the ProxyAgent constructor and rejects a ' + + 'per-request Proxy-Authorization header, so a custom challenge handler cannot be ' + + 'dispatched; proxy auth falls back to Basic (ProxyOptions.credentials)', + ); + }); +} + +/** + * Builds the per-transport reporter for TRANSPORT-30's second discoverability moment: the first time + * a proxy actually answers 407 while an undispatchable challenge handler is configured. + * + * Only a 407 is reported. A 401 is an *origin-server* challenge, and nothing about proxy credentials + * belongs anywhere near it — the spec makes that an explicit MUST NOT, so it is a guard here rather + * than an accident of control flow. The credential itself is never logged on any path; the 407 is + * returned to the caller untouched, for its own auth layer to act on. + * + * @param proxy - the configured proxy, if any. + * @returns a reporter to call with each adapted response; warns at most once per transport. + * + * @internal + */ +export function createProxyChallengeReporter( + proxy: ProxyOptions | undefined, +): (response: Response) => void { + if (proxy === undefined || !hasCustomChallengeHandler(proxy)) { + return () => undefined; + } + let reported = false; + return (response: Response) => { + if (response.status.code !== 407 || reported) return; + reported = true; + safeWarn('proxy.challenge.unanswered', entry => { + describeProxy(entry, proxy).field( + 'detail', + 'the proxy issued a 407 and the configured challenge handler cannot be dispatched; ' + + 'the response is surfaced unchanged for the caller’s own auth layer', + ); + }); + }; +} diff --git a/packages/transport-undici/src/index.ts b/packages/transport-undici/src/index.ts new file mode 100644 index 0000000..6a6f233 --- /dev/null +++ b/packages/transport-undici/src/index.ts @@ -0,0 +1,4 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/index.ts +export {undiciTransport} from './undici-transport.js'; +export type {UndiciTransportOptions} from './undici-transport.js'; diff --git a/packages/transport-undici/src/undici-transport.conformance.test.ts b/packages/transport-undici/src/undici-transport.conformance.test.ts new file mode 100644 index 0000000..24f0ef1 --- /dev/null +++ b/packages/transport-undici/src/undici-transport.conformance.test.ts @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/undici-transport.conformance.test.ts +// Runs the shared TRANSPORT-N suite (@dexpace/transport-conformance) against undiciTransport(). +import {createProxyOptions} from '@dexpace/core'; +import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; +import {undiciTransport} from './undici-transport.js'; + +runTransportConformanceSuite('undiciTransport', () => undiciTransport(), { + supportsInternalCancel: true, + supportsProxy: true, + // TRANSPORT-11's own note: an undici-class transport forwards `Connection` rather than dropping it. + dropsConnectionHeader: false, + // TRANSPORT-30: core resolves `ALL_PROXY=socks5://host:1080` to this type (CFG-22), and undici's + // `ProxyAgent` is an HTTP CONNECT tunnel that cannot carry it. + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => + undiciTransport({defaultTimeoutMs: value}), + unsupportedProxy: { + type: 'socks5', + build: () => + undiciTransport({ + proxy: createProxyOptions({ + type: 'socks5', + host: '127.0.0.1', + port: 1080, + }), + }), + }, +}); diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts new file mode 100644 index 0000000..c8fe3b5 --- /dev/null +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -0,0 +1,893 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/undici-transport.test.ts +// Exercises: TRANSPORT-2 (no redirect interceptor is composed), TRANSPORT-8 (a native-internal cancel +// is terminal while a timeout stays retryable), TRANSPORT-11 (undici keeps `Connection`), +// XCUT-22 (the SDK closes only resources it created: a caller-supplied dispatcher is never closed and +// stays usable afterwards), XCUT-13 (close is idempotent -- a second call is a no-op that neither +// throws nor blocks), +// TRANSPORT-15/16 (ownership-aware, idempotent close), TRANSPORT-22 (an adaptation throw destroys the +// native body), TRANSPORT-20 (a permanent argument error is terminal, a no-response failure is +// retryable), TRANSPORT-28 (a file body dispatches its declared byte range), SEAM-14, +// TRANSPORT-19 (a header-mapping throw leaves no started body producer stranded), SEAM-30 (so no +// producer rejection reaches Node's default unhandledRejection policy), TRANSPORT-9 (a producer that +// loses the race cancels the dispatch it raced, so no response is stranded) +import {createRequire} from 'node:module'; +import {mkdtemp, readFile, rm, writeFile} from 'node:fs/promises'; +import {createServer, type Server} from 'node:http'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + createProxyOptions, + getGlobalLogger, + Headers, + Request, + RequestOptions, + IoError, + setGlobalLogger, + TransportFailureError, + type Body, + type FileBodyDescriptor, + type Logger, +} from '@dexpace/core'; +import type {Agent, Dispatcher, ProxyAgent} from 'undici'; +import {undiciTransport} from './undici-transport.js'; + +const require = createRequire(import.meta.url); +const undici = require('undici/index.js') as typeof import('undici'); + +/** + * Awaits `pending` and hands back its rejection reason. `expect(p).rejects.…` is typed `void` here, + * so this keeps the assertion ordered with whatever the row checks afterwards. + */ +async function rejection(pending: Promise<unknown>): Promise<unknown> { + try { + await pending; + } catch (error) { + return error; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +/** Installs a logger that records every dropped header name, and returns the restore function. */ +function captureDroppedHeaders(): { + dropped: string[]; + restore: () => void; +} { + const dropped: string[] = []; + const previous = getGlobalLogger(); + const capturing: Logger = { + atLevel: () => { + let name: string | undefined; + const entry = { + field: (key: string, value: unknown) => { + if (key === 'header') name = String(value); + return entry; + }, + event: () => entry, + cause: () => entry, + emit: () => { + if (name !== undefined) dropped.push(name); + }, + }; + return entry; + }, + withContext: () => capturing, + }; + setGlobalLogger(capturing); + return { + dropped, + restore: () => { + setGlobalLogger(previous); + }, + }; +} + +/** + * DispatcherBase's public `destroyed` getter, which undici's shipped `Dispatcher` and `ProxyAgent` + * types omit even though every concrete dispatcher exposes it. + */ +interface DestroyableDispatcher { + readonly destroyed: boolean; +} + +/** + * A streaming body whose `mediaType` getter throws, recording whether its producer was ever started. + * The shape header mapping trips over: `mediaType` is read during mapping, while `writeTo` only runs + * once `pumpBody` has taken ownership. + */ +function bodyWithThrowingMediaType(): { + body: Body; + producerStarted: () => boolean; +} { + let started = false; + return { + body: { + kind: 'stream', + get mediaType(): string | undefined { + throw new Error('mediaType getter exploded'); + }, + // -1 / non-replayable forces the streaming branch rather than the buffered one. + contentLength: -1, + replayable: false, + writeTo: () => { + started = true; + return Promise.resolve(); + }, + }, + producerStarted: () => started, + }; +} + +/** + * Swaps undici's exported `Agent` binding for a capturing subclass, so the dispatcher a transport + * constructs for *itself* is reachable from the test. `destroyed` is DispatcherBase's own public getter, + * so teardown stays observable without patching `destroy` at all. + */ +function captureOwnedAgents(): { + agents: DestroyableDispatcher[]; + restore: () => void; +} { + const bindings = undici as unknown as Record<string, unknown>; + const RealAgent = undici.Agent; + const agents: DestroyableDispatcher[] = []; + + class CapturingAgent extends RealAgent { + constructor(opts?: Agent.Options) { + super(opts); + // No cast needed here: undici's shipped `Agent` type declares `destroyed`, unlike its bare + // `Dispatcher` and `ProxyAgent` types. + agents.push(this); + } + } + + bindings.Agent = CapturingAgent; + return { + agents, + restore: () => { + bindings.Agent = RealAgent; + }, + }; +} + +/** + * Swaps undici's exported `Agent` / `ProxyAgent` bindings so a transport built afterwards gets a + * direct `Agent` whose `destroy()` rejects, and captures the `ProxyAgent` constructed alongside it. + * + * The exported CLASS BINDINGS are swapped, not `DispatcherBase.prototype.destroy`: the transport + * reads `undici.Agent` / `undici.ProxyAgent` off this exports object at construction time, while + * ProxyAgent's own internal Agent comes from its private `require('./agent')`. Patching the shared + * prototype instead makes the injected failure fire inside ProxyAgent's internals too, which is a + * different bug than the one under test. + * + * The ProxyAgent is captured rather than intercepted: overriding its `destroy` would also catch + * DispatcherBase's internal `this.destroy(err, callback)` re-dispatch and recurse. `destroyed` is + * DispatcherBase's own public getter, so the effect is observable without touching teardown at all. + */ +function explodeDirectAgentDestroy(): { + proxyAgents: DestroyableDispatcher[]; + restore: () => void; +} { + const bindings = undici as unknown as Record<string, unknown>; + const RealAgent = undici.Agent; + const RealProxyAgent = undici.ProxyAgent; + const proxyAgents: DestroyableDispatcher[] = []; + + class ExplodingAgent extends RealAgent { + override destroy(): Promise<void> { + return Promise.reject(new Error('agent destroy exploded')); + } + } + class CapturingProxyAgent extends RealProxyAgent { + constructor(opts: ProxyAgent.Options) { + super(opts); + // Cast because undici's shipped ProxyAgent type omits DispatcherBase's `destroyed` getter. + proxyAgents.push(this as unknown as DestroyableDispatcher); + } + } + + bindings.Agent = ExplodingAgent; + bindings.ProxyAgent = CapturingProxyAgent; + return { + proxyAgents, + restore: () => { + bindings.Agent = RealAgent; + bindings.ProxyAgent = RealProxyAgent; + }, + }; +} + +/** Records every request body the server received, so a file body's byte range is checkable. */ +let server: Server; +let origin: string; +const received: string[] = []; + +beforeAll(async () => { + server = createServer((req, res) => { + const chunks: Buffer[] = []; + req.on('data', (chunk: Buffer) => chunks.push(chunk)); + req.on('end', () => { + received.push(Buffer.concat(chunks).toString('utf8')); + if (req.url === '/slow') return; // never answers -- the in-flight fixture + res.writeHead(200, {'content-type': 'text/plain'}); + res.end('ok'); + }); + }); + await new Promise<void>(done => { + server.listen(0, '127.0.0.1', done); + }); + const address = server.address(); + const port = + typeof address === 'object' && address !== null ? address.port : 0; + origin = `http://127.0.0.1:${String(port)}`; +}); + +afterAll(async () => { + server.closeAllConnections(); + await new Promise<void>(done => { + server.close(() => { + done(); + }); + }); +}); + +describe('undiciTransport construction and ownership', () => { + test('SEAM-14: a bring-your-own dispatcher is never closed by the transport', async () => { + let closed = false; + const byo = { + request: () => Promise.reject(new Error('not dispatched in this test')), + close: () => { + closed = true; + return Promise.resolve(); + }, + destroy: () => Promise.resolve(), + } as unknown as Dispatcher; + + const transport = undiciTransport({dispatcher: byo}); + await transport.close(); + await transport.close(); + expect(closed).toBe(false); + }); + + test('TRANSPORT-15/16: an owned agent is closed, idempotently', async () => { + const transport = undiciTransport({agentOptions: {connections: 1}}); + await transport.close(); + // Reaching the next line proves the second close neither threw nor hung (TRANSPORT-16). + await transport.close(); + }); + + test('a transport-constructed ProxyAgent is owned and released too', async () => { + const transport = undiciTransport({ + proxy: createProxyOptions({type: 'http', host: '127.0.0.1', port: 3128}), + }); + // The ProxyAgent is SDK-created, so close() must release it -- the bug this guards is closing + // only the separately-constructed direct Agent and leaking the ProxyAgent actually in use. + await transport.close(); + await transport.close(); + }); + + test('a failing dispatcher destroy still releases every other owned dispatcher (TRANSPORT-15/16)', async () => { + // owned is [ProxyAgent, Agent] and close() walks it reversed, so the direct Agent is destroyed + // first. When that destroy rejects, a naive `for … await` loop propagates immediately and the + // ProxyAgent -- the dispatcher actually holding the pooled proxy connections -- is never + // released. + const {proxyAgents, restore} = explodeDirectAgentDestroy(); + try { + const transport = undiciTransport({ + proxy: createProxyOptions({ + type: 'http', + host: '127.0.0.1', + port: 3128, + }), + }); + expect(proxyAgents.length).toBe(1); + // The failure is reported -- teardown must not swallow it (TRANSPORT-16) -- and it is reported + // with the underlying cause intact rather than flattened to a message. + const error = await rejection(transport.close()); + expect(error).toBeInstanceOf(TransportFailureError); + expect(error).toMatchObject({ + cause: {message: 'agent destroy exploded'}, + }); + // Idempotent even on the failure path: the rejection is memoized, so a second close reports the + // same failure rather than falsely claiming a clean teardown (TRANSPORT-16, XCUT-13). + expect(await rejection(transport.close())).toBe(error); + // ... but every other owned dispatcher is released regardless. This is the leak: with the + // naive `for … await` loop the direct Agent's rejection aborts the walk and this stays false. + expect(proxyAgents[0]?.destroyed).toBe(true); + } finally { + // Restoration must never be skipped, so this block stays assertion-free. + restore(); + } + }); + + test('supplying both a dispatcher and a proxy fails loudly at construction', () => { + const agent = new undici.Agent(); + expect(() => + undiciTransport({ + dispatcher: agent, + proxy: createProxyOptions({type: 'http', host: 'proxy', port: 8080}), + }), + ).toThrow(TypeError); + void agent.close(); + }); +}); + +describe('undiciTransport disposal (TRANSPORT-15/16)', () => { + test('asyncDispose is the same teardown as close, where the runtime has it', async () => { + // The owned Agent is captured so "same teardown as close" is an assertion about the dispatcher + // this transport constructed, not merely about the member existing: an asyncDispose wired to + // anything other than close() -- or to a bare resolved promise -- leaves `destroyed` false below. + const {agents, restore} = captureOwnedAgents(); + try { + const transport = undiciTransport(); + expect(agents.length).toBe(1); + // Cast rather than a bare `Symbol.asyncDispose` index: on the pinned floor (Node 20.3, which + // predates the symbol's 20.4 arrival) it is `undefined` and the index would read the string key + // `"undefined"`. The install in undici-transport.ts is guarded to match. + const asyncDispose = (Symbol as {asyncDispose?: symbol}).asyncDispose; + if (typeof asyncDispose === 'symbol') { + const dispose = ( + transport as unknown as Record< + symbol, + (() => Promise<void>) | undefined + > + )[asyncDispose]; + expect(dispose).toBeDefined(); + await dispose?.call(transport); + expect(agents[0]?.destroyed).toBe(true); + } + // Both legs: an unguarded `[Symbol.asyncDispose]()` class member would leave this junk key on + // the prototype on the >=20.3 floor, with no working disposal behind it. + expect( + Object.getOwnPropertyNames(Object.getPrototypeOf(transport)), + ).not.toContain('undefined'); + await transport.close(); + } finally { + // Restoration must never be skipped, so this block stays assertion-free. + restore(); + } + }); +}); + +/** + * A `Dispatcher` that records every `request()` it is handed and answers 200 with an empty body, so + * a row can assert what reached undici's argument validation rather than what reached the wire. + * Returns the recording array; the caller builds the transport around it. + */ +function recordingDispatcher(sink: Dispatcher.RequestOptions[]): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + sink.push(options); + return Promise.resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; +} + +/** The flat `[name, value, ...]` array one recorded dispatch carried. */ +function dispatchedHeaders( + sent: Dispatcher.RequestOptions | undefined, +): string[] { + return (sent?.headers ?? []) as string[]; +} + +describe('undiciTransport dispatch', () => { + test('TRANSPORT-2/11: redirects are pinned off and Connection is forwarded, not dropped', async () => { + const dispatched: Dispatcher.RequestOptions[] = []; + const recorder = recordingDispatcher(dispatched); + + const transport = undiciTransport({dispatcher: recorder}); + const request = Request.newBuilder() + .url(`${origin}/anything?q=1`) + .headers( + Headers.newBuilder() + .set('Connection', 'keep-alive') + .set('Content-Length', '999') + .build(), + ) + .build(); + await (await transport.send(request)).close(); + + const sent = dispatched[0]; + expect(sent?.maxRedirections).toBe(0); + expect(sent?.path).toBe('/anything?q=1'); + const headers = dispatchedHeaders(sent); + expect(headers).toContain('Connection'); + expect(headers).not.toContain('Content-Length'); + }); +}); + +describe('undiciTransport outbound header degradation (TRANSPORT-11/12)', () => { + test('every header undici refuses is dropped before dispatch, and logged by name', async () => { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); + try { + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + const headers = dispatchedHeaders(dispatched[0]); + // Asserted against the argument array, not the wire: undici validates in `new Request(...)`, + // so a name that reaches this array is a name that would have thrown (audit #67 / #81). + expect(headers).toEqual(['X-Pass-Through', 'survives']); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + expect(capture.dropped).toContain(name); + } + } finally { + capture.restore(); + } + }); + + test('a Connection value undici cannot carry is dropped, close and keep-alive are not', async () => { + const carried: string[] = []; + const dropped: string[] = []; + for (const value of ['close', 'Keep-Alive', 'upgrade']) { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); + try { + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers(Headers.newBuilder().set('Connection', value).build()) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + if (dispatchedHeaders(dispatched[0]).includes('Connection')) { + carried.push(value); + } + // `createDropLogger` lower-cases the name it logs, so the field is `connection`. + if (capture.dropped.includes('connection')) dropped.push(value); + } finally { + capture.restore(); + } + } + // `Keep-Alive` mixed-cased on purpose: undici lower-cases the value before comparing + // (`lib/core/request.js:401`), so this transport must too or it would drop a header undici + // would have carried. + expect(carried).toEqual(['close', 'Keep-Alive']); + expect(dropped).toEqual(['upgrade']); + }); +}); + +/** + * A `kind: 'file'` descriptor over a real path whose `writeTo` streams its own declared range and + * counts its calls -- the shape `@dexpace/body-file`'s `fileBody()` produces. Built by hand because + * TRANSPORT-28's recognition contract is structural: this package narrows on `kind` alone and must + * never `instanceof` against a package it does not depend on. + */ +function fileDescriptor( + path: string, + range: {start: number; count: number}, + writes: {count: number}, +): FileBodyDescriptor { + return { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: range.count, + replayable: true, + path, + start: range.start, + count: range.count, + async writeTo(sink: WritableStream<Uint8Array>): Promise<void> { + writes.count += 1; + const writer = sink.getWriter(); + if (range.count === 0) { + writer.releaseLock(); + return; + } + const bytes = (await readFile(path)).subarray( + range.start, + range.start + range.count, + ); + await writer.write(new Uint8Array(bytes)); + writer.releaseLock(); + }, + }; +} + +/** Runs `body` against a fresh temporary file holding `ABCDEFGH`, and removes the directory after. */ +async function withPayloadFile<T>( + run: (path: string) => Promise<T>, +): Promise<T> { + const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, 'ABCDEFGH'); + return await run(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + +describe('undiciTransport body and adaptation paths', () => { + test('TRANSPORT-28/BODY-13: a file body dispatches its declared range THROUGH its own writeTo', async () => { + await withPayloadFile(async path => { + const writes = {count: 0}; + const transport = undiciTransport(); + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(fileDescriptor(path, {start: 2, count: 4}, writes)) + .build(); + received.length = 0; + await (await transport.send(request)).close(); + await transport.close(); + expect(received[0]).toBe('CDEF'); + // The `writes` count is the whole point of the row. Until audit #67 / #81 this transport + // handed `createReadStream(path, {start, end})` to undici and left `writeTo` uncalled, so the + // same four bytes reached the wire with BODY-13's `transferred === count` invariant -- + // the only thing that can see a file truncated after its length was captured -- never run. + expect(writes.count).toBe(1); + }); + }); + + test('a zero-count file body dispatches as an empty body, not a stream error', async () => { + await withPayloadFile(async path => { + // `isMaterializable` admits `contentLength === 0`, and `materializeBody` returns an empty + // buffer without opening anything, so the empty range needs no branch of its own. It did when + // the file body went to `createReadStream`, which throws ERR_OUT_OF_RANGE the moment `end` + // (start + count - 1) falls below `start`. + const writes = {count: 0}; + const transport = undiciTransport(); + received.length = 0; + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(fileDescriptor(path, {start: 4, count: 0}, writes)) + .build(), + ); + await response.close(); + await transport.close(); + expect(received[0]).toBe(''); + expect(writes.count).toBe(1); + }); + }); +}); + +describe('undiciTransport request-body failures', () => { + test('a body that cannot be written fails the send as a transport failure', async () => { + const transport = undiciTransport(); + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body({ + kind: 'byte-array', + mediaType: 'text/plain', + contentLength: 3, + replayable: true, + writeTo: () => Promise.reject(new Error('body exploded')), + }) + .build(); + // Classified the same way the streaming branch classifies the same failure, cause intact. + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + cause: {message: 'body exploded'}, + }); + await transport.close(); + }); + + test('a header-mapping throw never strands a started body producer (TRANSPORT-19, SEAM-30)', async () => { + // `pumpBody` starts the producer EAGERLY, and header mapping reads `request.body.mediaType` -- + // a getter on a caller-supplied Body, which may throw. If the producer is started first, that + // throw escapes before anything can abandon it and the producer's own later rejection reaches + // Node's default unhandledRejection policy. Mapping headers first closes the window, which is + // the order the fetch twin already evaluates them in. + const {body, producerStarted} = bodyWithThrowingMediaType(); + const transport = undiciTransport(); + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(body) + .build(); + + await rejection(transport.send(request)); + expect(producerStarted()).toBe(false); + await transport.close(); + }); + + test('TRANSPORT-22: an adaptation throw destroys the native body before propagating', async () => { + let destroyed = false; + const hostile = { + get statusCode(): number { + throw new Error('adaptation exploded'); + }, + headers: {}, + body: { + destroy: () => { + destroyed = true; + }, + }, + } as unknown as Dispatcher.ResponseData; + + const transport = undiciTransport({ + dispatcher: { + request: () => Promise.resolve(hostile), + close: () => Promise.resolve(), + } as unknown as Dispatcher, + }); + const request = Request.newBuilder().url(`${origin}/anything`).build(); + expect(await rejection(transport.send(request))).toMatchObject({ + message: 'adaptation exploded', + }); + expect(destroyed).toBe(true); + }); +}); + +describe('undiciTransport failure classification (TRANSPORT-20)', () => { + test('an argument undici can never accept is terminal, not a retryable failure', async () => { + // The drop set that removes Proxy-Authorization is chosen from `options.proxy`, so a BYO + // ProxyAgent leaves the header in place and ProxyAgent.dispatch rejects it outright. That is a + // permanent misconfiguration: classifying it as TransportFailureError would make it an IoError, + // and classify.ts returns true for every IoError -- a caller's whole retry budget spent + // re-proving the same rejection. It is reported outside the IoError tree instead. + const agent = new undici.ProxyAgent({uri: 'http://127.0.0.1:1/'}); + const transport = undiciTransport({dispatcher: agent}); + try { + const request = Request.newBuilder() + .url('http://example.invalid/') + .headers( + Headers.newBuilder() + .set('Proxy-Authorization', 'Basic Zm9vOmJhcg==') + .build(), + ) + .build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TypeError); + // Outside the IoError tree is the whole point: classify.ts's allow-list returns true for + // every IoError and false for anything it was never opted into (RETRY-2). + expect(error).not.toBeInstanceOf(IoError); + expect((error as {cause?: {code?: string}}).cause?.code).toBe( + 'UND_ERR_INVALID_ARG', + ); + } finally { + await transport.close(); + await agent.close(); + } + }); + + test('a genuine network failure stays the retryable TransportFailureError', async () => { + // The twin of the row above: the catch-all branch must keep classifying a no-response failure + // as retryable, so narrowing it did not turn every dispatch error terminal. + const transport = undiciTransport(); + try { + const request = Request.newBuilder().url('http://127.0.0.1:1/').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(error).toBeInstanceOf(IoError); + } finally { + await transport.close(); + } + }); +}); + +describe('undiciTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + /** + * A `Dispatcher` whose `request()` resolves only after `delayMs`, recording the signal it was + * handed. Nothing awaits that promise once the producer has lost the race, so the signal is the + * only thing that can still stop the exchange. + */ + function lateDispatcher( + seen: {signal?: AbortSignal | null; settled: boolean}, + delayMs: number, + ): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + seen.signal = options.signal as AbortSignal | null; + return new Promise<Dispatcher.ResponseData>(resolve => { + setTimeout(() => { + seen.settled = true; + resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + dump: () => Promise.resolve(), + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, delayMs); + }); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; + } + + test('a producer that loses the race takes the pending dispatch down with it', async () => { + // Until audit #67 / #82 this send dispatched with `signal: null` -- the fork only existed when + // the caller supplied a signal or a timeout was composed -- so undici kept dispatching after + // `send()` rejected and whatever came back was dropped with its `BodyReadable` neither read nor + // destroyed, holding the pooled connection open. + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 30)}); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(seen.settled).toBe(false); + expect(seen.signal?.aborted).toBe(true); + expect((seen.signal?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + await transport.close(); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 0)}); + const request = Request.newBuilder().url(`${origin}/anything`).build(); + const response = await transport.send(request); + await response.close(); + // The fork is latched at delivery, so its abort direction can no longer reach a body the + // caller now owns. + expect(seen.signal?.aborted).toBe(false); + await transport.close(); + }); +}); + +describe('undiciTransport proxy dispatch (TRANSPORT-30)', () => { + test('TRANSPORT-30: a SOCKS proxy is refused at the factory, before any Agent is built', () => { + // Both SOCKS values `ProxyType` admits, because core resolves both from the environment + // (`socks4`/`socks4a` and `socks:`/`socks5`/`socks5h`, `config/proxy.ts:372-380`) and undici's + // `ProxyAgent` carries neither -- it is an HTTP CONNECT tunnel and reads its `uri` as a URL. + const {agents, restore} = captureOwnedAgents(); + try { + for (const type of ['socks4', 'socks5'] as const) { + let thrown: unknown; + try { + undiciTransport({ + proxy: createProxyOptions({type, host: '127.0.0.1', port: 1080}), + }); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // TRANSPORT-30's "discoverable": the refusal names the type, and it is not the raw + // `InvalidArgumentError: Invalid URL protocol: socks5:` undici used to let escape. + expect((thrown as Error).message).toContain(type); + expect(thrown).not.toBeInstanceOf(IoError); + } + // The check runs before `new undici.Agent(...)`, so a refused construction leaks nothing -- + // there is no transport for the caller to have called `close()` on. + expect(agents.length).toBe(0); + } finally { + restore(); + } + }); + + test('a per-request Proxy-Authorization is dropped when a proxy is configured', async () => { + // ProxyAgent.dispatch throws InvalidArgumentError on ANY per-request Proxy-Authorization -- a + // deliberate undici security fix -- so forwarding one would turn every proxied send into a hard + // failure. It is dropped instead, and the drop log is what keeps that discoverable + // (TRANSPORT-11/12/30). + const {dropped, restore} = captureDroppedHeaders(); + const transport = undiciTransport({ + headerDropLogging: 'all', + proxy: createProxyOptions({ + type: 'http', + host: '127.0.0.1', + port: 1, + nonProxyHosts: ['127.0.0.1'], + }), + }); + try { + const response = await transport.send( + Request.newBuilder() + .url(`${origin}/anything`) + .headers( + Headers.newBuilder() + .set('Proxy-Authorization', 'Basic stale') + .build(), + ) + .build(), + ); + await response.close(); + expect(dropped).toContain('proxy-authorization'); + } finally { + restore(); + await transport.close(); + } + }); + + test('a proxied transport routes a NO_PROXY host over its direct agent', async () => { + const transport = undiciTransport({ + proxy: createProxyOptions({ + type: 'http', + host: '127.0.0.1', + port: 1, + nonProxyHosts: ['127.0.0.1'], + }), + }); + // Port 1 is a dead proxy: reaching the fixture at all proves the bypass routed direct. + const response = await transport.send( + Request.newBuilder().url(`${origin}/anything`).build(), + ); + expect(response.status.code).toBe(200); + await response.close(); + await transport.close(); + }); +}); + +describe('undiciTransport cancellation (TRANSPORT-8)', () => { + test('TRANSPORT-16: close does not wait out an in-flight request', async () => { + const transport = undiciTransport(); + const pending = rejection( + transport.send(Request.newBuilder().url(`${origin}/slow`).build()), + ); + await new Promise(resolve => setTimeout(resolve, 25)); + const startedClosing = Date.now(); + await transport.close(); + // The fixture holds /slow open forever; a graceful close would block here until it gave up. + expect(Date.now() - startedClosing).toBeLessThan(1_000); + expect(await pending).toMatchObject({name: 'CancellationError'}); + }); + + test('destroying the dispatcher mid-flight is terminal, not a retryable failure', async () => { + const agent = new undici.Agent(); + const transport = undiciTransport({dispatcher: agent}); + const pending = transport.send( + Request.newBuilder().url(`${origin}/slow`).build(), + ); + // Give the request time to actually reach the socket before tearing the client down. + await new Promise(resolve => setTimeout(resolve, 25)); + await agent.destroy(); + expect(await rejection(pending)).toMatchObject({ + name: 'CancellationError', + }); + }); + + test('a timeout on the same path stays retryable', async () => { + const transport = undiciTransport(); + const pending = transport.send( + Request.newBuilder().url(`${origin}/slow`).build(), + RequestOptions.newBuilder().timeoutMs(40).build(), + ); + expect(await rejection(pending)).toMatchObject({ + name: 'TransportFailureError', + }); + await transport.close(); + }); +}); diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts new file mode 100644 index 0000000..0fc67a2 --- /dev/null +++ b/packages/transport-undici/src/undici-transport.ts @@ -0,0 +1,708 @@ +// SPDX-License-Identifier: MIT +// packages/transport-undici/src/undici-transport.ts +import {createRequire} from 'node:module'; +import {Readable} from 'node:stream'; +import type {ReadableStream as NodeReadableStream} from 'node:stream/web'; +import { + CancellationError, + composeSignal, + Protocol, + Response, + shouldBypassProxy, + Status, + TransportFailureError, + type Body, + type ProxyOptions, + type ProxyType, + type Request, + type RequestOptions, + type Transport, +} from '@dexpace/core'; +import { + abortToSdkError, + createDropLogger, + degradeInboundHeaders, + forkSignal, + hasNoResponseBody, + isMaterializable, + mapOutboundHeaders, + materializeBody, + producerFailure, + pumpBody, + requireValidDefaultTimeoutMs, + toDispatchFailure, + type BodyPump, + type ForkedSignal, + type HeaderDropLogging, +} from '@dexpace/transport-shared'; +import type {Agent, Dispatcher, ProxyAgent} from 'undici'; +import { + createProxyChallengeReporter, + warnIfCustomChallengeHandler, +} from './challenge-handler.js'; + +/** + * `undici` is loaded through `createRequire`, not a static `import`, because Bun resolves the bare + * specifier `undici` to its own built-in shim: the shim's `Agent` constructs but has no `request` + * method, so every dispatch under `bun test` would fail with a `TypeError` instead of reaching the + * wire. Requiring the real package's entry file by path bypasses that alias and resolves identically + * under plain Node. The types still come from the static `import type` above, so this stays fully + * checked. Revisit when Bun's shim implements `Dispatcher.request`, or if undici ever adds an + * `exports` map that hides `index.js` (this package pins `^6`, which has neither). + */ +const require = createRequire(import.meta.url); +const undici = require('undici/index.js') as typeof import('undici'); + +/** + * TRANSPORT-11's outbound drop set for this transport: the three framing headers undici computes + * itself, plus the three it refuses outright. + * + * `expect`, `keep-alive` and `upgrade` are undici's unconditional rejections + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`). Until 2026-09-05 they were absent, so a caller who set any of + * them got a failed send out of `undiciTransport().send()` where the fetch twin dispatched + * (audit #67 / #81). TRANSPORT-12 says the header is what gives way, not the request. + * + * `connection` is deliberately absent — §17's own note is that an undici-class transport forwards + * it rather than dropping it. undici carries only `close` and `keep-alive`, so a third value is + * dropped per-request by {@link isUnsendableHeader} rather than by name here. + */ +const UNDICI_FORBIDDEN_HEADERS: readonly string[] = [ + 'content-length', + 'host', + 'transfer-encoding', + 'expect', + 'keep-alive', + 'upgrade', +]; + +/** + * The drop set when this transport owns a `ProxyAgent`. `ProxyAgent.dispatch` throws + * `InvalidArgumentError` on *any* per-request `Proxy-Authorization` — a deliberate undici security + * fix, not an oversight — so forwarding one turns every proxied send into a hard failure. Dropping + * it degrades one header instead (TRANSPORT-12) and, because every drop is logged by name, keeps the + * limitation discoverable rather than silent (TRANSPORT-11/13, TRANSPORT-30). + */ +const UNDICI_PROXIED_FORBIDDEN_HEADERS: readonly string[] = [ + ...UNDICI_FORBIDDEN_HEADERS, + 'proxy-authorization', +]; + +/** Bodies at or below this declared length are buffered rather than streamed; see the fetch twin. */ +const MAX_MATERIALIZED_BODY_BYTES = 1_000_000; + +/** + * Options for {@link undiciTransport}. + * + * @public + */ +export interface UndiciTransportOptions { + /** + * A bring-your-own `Dispatcher`. It is used as-is and **never** closed by this transport + * (SEAM-14); supplying it together with `proxy` is a construction-time error. + */ + readonly dispatcher?: Dispatcher; + /** + * Proxy configuration; the transport constructs and owns the resulting `ProxyAgent`. + * + * `type` must be `http`. undici's `ProxyAgent` is an HTTP `CONNECT` tunnel, so the `socks4` and + * `socks5` values `ProxyType` also admits — which core resolves from `ALL_PROXY` — are refused at + * construction rather than at the first send (TRANSPORT-30). + */ + readonly proxy?: ProxyOptions; + /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ + readonly headerDropLogging?: HeaderDropLogging; + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link undiciTransport} rather than by the first send (HTTP-35). + */ + readonly defaultTimeoutMs?: number; + /** `Agent` options, used only when no `dispatcher` is supplied. */ + readonly agentOptions?: Agent.Options; +} + +/** The dispatcher pair one transport routes over, plus the subset it owns and must close. */ +interface DispatcherSet { + /** Where a non-bypassed request goes; identical to `direct` when no proxy is configured. */ + readonly proxied: Dispatcher; + /** Where a `shouldBypassProxy` host goes, so `NO_PROXY` is honored rather than tunnelled. */ + readonly direct: Dispatcher; + /** Dispatchers this transport constructed; empty for a caller-supplied one (SEAM-14). */ + readonly owned: readonly Dispatcher[]; +} + +/** + * The one `ProxyOptions.type` this transport can build a dispatcher for. + * + * undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and takes its `uri` as a URL, so a `socks5://` + * or `socks4://` one throws `InvalidArgumentError('Invalid URL protocol: …')` out of its + * constructor. Core resolves `ALL_PROXY=socks5://host:1080` to `type: 'socks5'` quite legitimately + * (`config/proxy.ts:372-380`, CFG-22), so the configuration can express a proxy neither shipped + * transport can honour, and until 2026-09-05 the way you found out was an undici error escaping a + * public factory untyped and undocumented (audit #67 / #81, `docs/deviations.md`). + */ +const SUPPORTED_PROXY_TYPE: ProxyType = 'http'; + +/** + * TRANSPORT-30's "make the limitation discoverable rather than silently misbehaving", applied at + * the earliest point that can: construction, before any dispatcher is allocated. + * + * A `TypeError`, matching {@link selectDispatchers}' other construction-time refusal and + * deliberately outside the `IoError` tree — `retry/classify.ts` is an allow-list, so a + * misconfiguration that no retry can fix is non-retryable for free (RETRY-2). + * + * @param proxy - the configured proxy. + * @throws `TypeError` when `proxy.type` is anything but `http`. + */ +function requireSupportedProxyType(proxy: ProxyOptions): void { + if (proxy.type === SUPPORTED_PROXY_TYPE) return; + throw new TypeError( + `unsupported proxy type \`${proxy.type}\`: undici's ProxyAgent is an HTTP CONNECT tunnel and ` + + 'cannot carry a SOCKS proxy, and @dexpace/transport-fetch has no proxy support at all. ' + + 'Configure an http proxy, or route SOCKS outside this SDK.', + ); +} + +/** + * The proxy URI plus its Basic credential, kept apart. `formatProxyOptions` is deliberately *not* + * used here: it masks credentials as `***:***` for logging, and feeding that to `ProxyAgent` would + * authenticate with the literal mask (TRANSPORT-30 — credentials must not leak, and must still work). + */ +function toProxyAgentOptions(proxy: ProxyOptions): ProxyAgent.Options { + // `host` is stored bare, so an IPv6 literal needs its brackets back before it can be a URL authority. + const host = proxy.host.includes(':') ? `[${proxy.host}]` : proxy.host; + const uri = `${proxy.type}://${host}:${String(proxy.port)}`; + if (proxy.credentials === undefined) return {uri}; + const raw = `${proxy.credentials.username}:${proxy.credentials.password}`; + return {uri, token: `Basic ${Buffer.from(raw).toString('base64')}`}; +} + +/** + * One exclusive decision, made once, fixing both the dispatcher pair and its ownership. Supplying + * both `dispatcher` and `proxy` fails loudly rather than silently picking one: a BYO dispatcher may + * already be a `ProxyAgent`, and ignoring either option hides which is in force. + */ +function selectDispatchers(options: UndiciTransportOptions): DispatcherSet { + if (options.dispatcher !== undefined && options.proxy !== undefined) { + throw new TypeError( + 'supply either `dispatcher` or `proxy`, not both: a bring-your-own dispatcher may already be ' + + 'a ProxyAgent, and silently ignoring one of the two hides which is in force', + ); + } + if (options.dispatcher !== undefined) { + const byo = options.dispatcher; + return {proxied: byo, direct: byo, owned: []}; + } + // Before anything is allocated: a refusal after `new undici.Agent(...)` would leak the direct + // agent on the way out, with no transport for the caller to close it through. + if (options.proxy !== undefined) requireSupportedProxyType(options.proxy); + // Agent, not Pool: a Pool is bound to one origin at construction, but a general-purpose Transport + // must reach whatever origin each Request names. + const direct = new undici.Agent(options.agentOptions); + if (options.proxy === undefined) + return {proxied: direct, direct, owned: [direct]}; + const proxied = new undici.ProxyAgent(toProxyAgentOptions(options.proxy)); + return {proxied, direct, owned: [proxied, direct]}; +} + +/** + * RFC 9110 `token`, which is what undici's `isValidHTTPToken` enforces on every header name + * (`lib/core/util.js:547-587`): VCHAR minus the delimiters `"(),/:;<=>?@[\]{}`. Anchored and + * one-or-more, so the empty name undici also rejects fails here too. + * + * `@dexpace/core` is deliberately laxer: `hasForbiddenNameByte` admits every printable ASCII byte + * (`http/ascii-validation.ts:29-33`), so `X Custom` is a model-valid name that no native client on + * this platform will carry. + */ +const RFC9110_TOKEN = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/u; + +/** The only two `Connection` values undici carries; anything else is `InvalidArgumentError`. */ +const FORWARDABLE_CONNECTION_VALUES: ReadonlySet<string> = new Set([ + 'close', + 'keep-alive', +]); + +/** + * Whether undici's own request validation would reject this header, which for TRANSPORT-12 means it + * is dropped rather than allowed to fail the send. + * + * Only the *name* grammar and the `connection` value are checked. undici's value grammar + * (`headerCharRegex`, `lib/core/util.js:598`) admits HTAB, 0x20-0x7E **and** obs-text 0x80-0xFF, + * which is strictly wider than the outbound value rule `mapOutboundHeaders` has already applied + * (HTTP-18), so no value that reaches here can fail there. + * + * @param name - the header name, as the caller spelled it. + * @param value - the header value. + * @returns `true` when the header must be dropped instead of dispatched. + */ +function isUnsendableHeader(name: string, value: string): boolean { + if (!RFC9110_TOKEN.test(name)) return true; + return ( + name.toLowerCase() === 'connection' && + !FORWARDABLE_CONNECTION_VALUES.has(value.toLowerCase()) + ); +} + +/** + * undici's flat `[name, value, name, value, ...]` form -- the only shape that keeps a repeated name + * repeated (HTTP-14) -- with every header undici would refuse degraded to a logged drop. + * + * The per-header guard is the same degrade `@dexpace/transport-fetch` gets for free from wrapping + * `Headers.append` in a `try`/`catch` (`fetch-transport.ts:159-166`): undici validates at dispatch + * rather than at construction, so this transport has to ask the question itself (TRANSPORT-12). + */ +function toUndiciHeaders( + request: Request, + forbidden: readonly string[], + logDrops: (dropped: readonly string[]) => void, +): string[] { + const {sent, dropped} = mapOutboundHeaders(request.headers, forbidden, { + bodyDerivedMediaType: request.body?.mediaType, + }); + const degraded = [...dropped]; + const flat: string[] = []; + for (const [name, value] of sent.entries()) { + if (isUnsendableHeader(name, value)) degraded.push(name); + else flat.push(name, value); + } + logDrops(degraded); + return flat; +} + +/** + * undici's own codes for "this exchange was torn down from inside the client", as opposed to a + * network failure. TRANSPORT-8 requires the two be told apart: a destroyed dispatcher is terminal + * (nothing about retrying it can succeed — the client is gone), while a timeout on the same code + * path stays retryable. Reached only after the caller-signal branch, so a caller abort and a + * per-call timeout are already classified by then. + */ +const NATIVE_CANCEL_CODES: ReadonlySet<string> = new Set([ + 'UND_ERR_DESTROYED', + 'UND_ERR_ABORTED', + 'UND_ERR_CLOSED', +]); + +function isNativeCancel(error: unknown): boolean { + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' && NATIVE_CANCEL_CODES.has(code); +} + +/** + * Maps one dispatch failure onto the SDK's error vocabulary. Extracted from `#dispatch` so the + * branches read as one classification table rather than as control flow wrapped around a call. + * + * Only the first two branches are this transport's own. The permanent-versus-retryable question the + * third asks is `@dexpace/transport-shared`'s {@link toDispatchFailure}, because the two shipped + * adapters answered it oppositely for the same condition until audit #67 / #82: `ftp://` is + * `UND_ERR_INVALID_ARG` here and a `fetch failed` with an `unknown scheme` cause over there, and + * only this transport treated it as permanent. A shared table is what keeps that from recurring — + * the same reason `abort-mapping.ts` exists. + * + * @param error - whatever the dispatch rejected with. + * @param signal - the forked signal the dispatch was given. + * @returns the error to throw; never returns normally without one. + */ +function toDispatchError(error: unknown, signal: AbortSignal): Error { + if (signal.aborted) return abortToSdkError(signal, error); + if (isNativeCancel(error)) { + // TRANSPORT-8: terminal, never retryable -- the dispatcher this send was routed over no longer + // exists, so a retry over it cannot succeed. + return new CancellationError('undici dispatcher was destroyed', { + cause: error, + }); + } + return toDispatchFailure(error, 'undici dispatch failed'); +} + +/** What undici accepts as a request body; `undefined` is not one of them, `null` is. */ +type UndiciBody = Exclude<Dispatcher.RequestOptions['body'], undefined>; + +/** A request body prepared for one dispatch, plus the teardown an abandoned producer is owed. */ +interface PreparedBody { + readonly init: UndiciBody; + readonly pump: BodyPump | undefined; +} + +/** + * Prepares one request body for dispatch: buffer it if it is small and replayable, stream it + * otherwise. Exactly the two decisions the fetch twin makes, in the same order, and deliberately so + * — there is **no** `kind === 'file'` branch here. + * + * There was one until 2026-09-05. It handed `createReadStream(path, {start, end})` to undici, which + * is a genuinely shorter path to the wire — one fewer userspace copy — and it bypassed the + * descriptor's own `writeTo`, so `@dexpace/body-file`'s `transferred === count` invariant never ran + * (BODY-13). `content-length` is dropped outbound, so undici framed the body chunked and the wire + * could not detect the short write either: a file truncated between `stat` and `send` POSTed its + * remaining bytes and resolved 200, where the fetch transport raised `TransportFailureError` + * (audit #67 / #81). A file body is now written through `writeTo` like any other, which is what + * makes BODY-13 hold on both transports and what the shared conformance row asserts. + * + * TRANSPORT-28's SHOULD is not thereby abandoned so much as re-described: no user-space path in + * either client reaches `sendfile(2)`, which `docs/deviations.md` item 13 has recorded since Phase + * 8a, and the clause's MUST — a file body is replayable, and exactly its declared byte range reaches + * the wire — is honoured by the descriptor itself, on both transports, by the same code. + * + * The zero-count case needs no branch either: `isMaterializable` admits `contentLength === 0`, and + * `materializeBody` returns an empty buffer without ever opening a read stream. + */ +async function prepareBody(body: Body | undefined): Promise<PreparedBody> { + if (body === undefined) return {init: null, pump: undefined}; + if (isMaterializable(body, MAX_MATERIALIZED_BODY_BYTES)) { + try { + return {init: await materializeBody(body), pump: undefined}; + } catch (error) { + // Same classification the streaming branch gives the same failure -- see the fetch twin. + throw new TransportFailureError('request body could not be written', { + cause: error, + }); + } + } + const pump = pumpBody(body); + return { + init: Readable.fromWeb(pump.readable as unknown as NodeReadableStream), + pump, + }; +} + +/** + * Wraps undici's body in a web stream that reads only when pulled. + * + * Deliberately not `Readable.toWeb`: Bun's adapter keeps enqueuing after the controller closes and + * throws `ERR_INVALID_STATE` the moment a response is closed without being fully read — which is + * exactly TRANSPORT-25's close-without-reading path. Deliberately not a `start()` that attaches a + * `'data'` listener either: that switches the Node stream into flowing mode and buffers the whole + * body eagerly, defeating the same requirement from the other side. Async iteration is pull-based, + * so a chunk is read only when the consumer asks, and `cancel` destroys the underlying body, which + * is what returns the connection to the pool. + */ +function toDemandDrivenStream(body: Readable): ReadableStream<Uint8Array> { + // `undefined` as the return type, not the default `any`: the done-result's `value` would + // otherwise destructure as `any` and defeat the type-aware lint rules. + const chunks = body[Symbol.asyncIterator]() as AsyncIterator< + Uint8Array, + undefined + >; + return new ReadableStream<Uint8Array>({ + async pull(controller) { + try { + const {done, value} = await chunks.next(); + if (done) controller.close(); + else controller.enqueue(value); + } catch (error) { + controller.error(error); + } + }, + cancel(reason) { + body.destroy(reason instanceof Error ? reason : undefined); + }, + }); +} + +function adaptResponse( + request: Request, + result: Dispatcher.ResponseData, + logDrops: (dropped: readonly string[]) => void, +): Response { + const raw: [string, string][] = []; + for (const [name, value] of Object.entries(result.headers)) { + if (value === undefined) continue; + // An array means a genuinely repeated header -- undici arrays ANY name it saw more than once, + // not just `Set-Cookie`. Keep each value its own entry: `WWW-Authenticate` and + // `Proxy-Authenticate` arrive this way from a server following RFC 7616 3.3, and collapsing them + // to the first would hide every challenge after it (audit #67 / #74). `@dexpace/transport-fetch` + // comma-joins the same response, which RFC 9110 5.3 makes equivalent; the conformance row + // `a repeated inbound header keeps every value` asserts the two agree on the list. + if (Array.isArray(value)) for (const each of value) raw.push([name, each]); + else raw.push([name, value]); + } + const {headers, dropped} = degradeInboundHeaders(raw); + logDrops(dropped); + + return ( + Response.newBuilder() + .request(request) + // A documented best-effort default: undici's ResponseData does not surface the negotiated HTTP + // version any more than the WHATWG Response does (Deviation Ledger). + .protocol(Protocol.HTTP_1_1) + .status(Status.of(result.statusCode)) + .headers(headers) + // undici's dispatcher always hands back a `BodyReadable`, even for a 204, a 304 or a HEAD -- + // so wrapping it unconditionally gave a caller an empty stream it had to read to discover + // was empty, where the fetch twin on Node gave `null`. The two adapters now decide by the + // same rule; `#exchange` dumps whatever this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, result.statusCode) + ? null + : toDemandDrivenStream(result.body), + ) + .build() + ); +} + +/** Everything one dispatch needs that is not the request itself; keeps `max-params` at three. */ +interface DispatchContext { + readonly headers: string[]; + readonly body: UndiciBody; + /** The forked signal handed to undici; detached by `send` the moment the response is delivered. */ + readonly fork: ForkedSignal; +} + +/** + * Destroys every dispatcher in reverse acquisition order and returns whatever failed, rather than + * stopping at the first rejection. Teardown is best-effort by definition: a dispatcher that cannot be + * released is not a reason to leak the ones behind it (TRANSPORT-15/16). + */ +async function releaseAll( + dispatchers: readonly Dispatcher[], +): Promise<unknown[]> { + const failures: unknown[] = []; + for (const dispatcher of [...dispatchers].reverse()) { + try { + await dispatcher.destroy(); + } catch (error) { + failures.push(error); + } + } + return failures; +} + +class UndiciTransport implements Transport { + readonly #dispatchers: DispatcherSet; + readonly #proxy: ProxyOptions | undefined; + readonly #logDrops: (dropped: readonly string[]) => void; + readonly #defaultTimeoutMs: number | undefined; + readonly #forbiddenHeaders: readonly string[]; + readonly #reportProxyChallenge: (response: Response) => void; + #closing: Promise<void> | undefined; + + constructor(options: UndiciTransportOptions) { + // Before `selectDispatchers` allocates anything: a refusal afterwards would leak the agents it + // built, with no transport for the caller to close them through (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); + this.#dispatchers = selectDispatchers(options); + this.#proxy = options.proxy; + this.#logDrops = createDropLogger( + options.headerDropLogging ?? 'first-per-name', + ); + this.#defaultTimeoutMs = options.defaultTimeoutMs; + this.#forbiddenHeaders = + options.proxy === undefined + ? UNDICI_FORBIDDEN_HEADERS + : UNDICI_PROXIED_FORBIDDEN_HEADERS; + this.#reportProxyChallenge = createProxyChallengeReporter(options.proxy); + // TRANSPORT-30: undici cannot dispatch a custom challenge handler at all, so the limitation is + // surfaced up front rather than discovered on a 407. + warnIfCustomChallengeHandler(options.proxy); + } + + async send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + const composed = composeSignal( + signal, + options?.timeoutMs ?? this.#defaultTimeoutMs, + ); + if (composed?.aborted) throw abortToSdkError(composed, composed.reason); + + // Headers BEFORE the body, deliberately -- the fetch twin evaluates them in this order too. + // `prepareBody` starts a streaming producer eagerly, while `toUndiciHeaders` reads + // `request.body.mediaType`, a getter on a caller-supplied Body that may throw. Preparing the + // body first leaves such a throw with a live producer nobody can abandon, whose own later + // rejection then reaches Node's default unhandledRejection policy (TRANSPORT-19, SEAM-30). + const headers = toUndiciHeaders( + request, + this.#forbiddenHeaders, + this.#logDrops, + ); + const prepared = await prepareBody(request.body); + // Dispatched with a fork the caller cannot reach: cancellation stays live for the whole in-flight + // window and goes inert the moment the response is handed over (SEAM-16). + const context: DispatchContext = { + headers, + body: prepared.init, + fork: forkSignal(composed), + }; + try { + return await this.#exchange(request, context, prepared.pump); + } finally { + context.fork.detach(); + } + } + + async #exchange( + request: Request, + context: DispatchContext, + pump: BodyPump | undefined, + ): Promise<Response> { + const result = await this.#dispatch(request, context, pump); + // The fork, not the caller's signal: it mirrors the source for as long as it stays attached, + // which is exactly the in-flight window this check is about. + const dispatched = context.fork.signal; + + if (dispatched.aborted) { + // TRANSPORT-9 / SEAM-30: this response will never reach a caller, so this producer closes it. + await result.body.dump().catch(() => undefined); + await pump?.abandon(dispatched.reason); + throw abortToSdkError(dispatched, dispatched.reason); + } + + try { + // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. + const response = adaptResponse(request, result, this.#logDrops); + if (response.body === null) { + // Nothing references the `BodyReadable` any more, and an undrained one holds the pooled + // connection open until the dispatcher times it out (TRANSPORT-25, SEAM-30). `dump` reads + // and discards, which is what returns the socket to the pool. + await result.body.dump().catch(() => undefined); + } + this.#reportProxyChallenge(response); + return response; + } catch (error) { + result.body.destroy(); + // TRANSPORT-19: nothing is delivered on this path either, so the producer is owed its teardown + // exactly as on the abort branch above. + await pump?.abandon(error); + throw error; + } + } + + async #dispatch( + request: Request, + context: DispatchContext, + pump: BodyPump | undefined, + ): Promise<Dispatcher.ResponseData> { + const dispatcher = + this.#proxy !== undefined && + shouldBypassProxy(this.#proxy, request.url.hostname) + ? this.#dispatchers.direct + : this.#dispatchers.proxied; + try { + // Raced, not sequenced, for the same two reasons as the fetch twin: a producer failure must + // surface even while undici is still pending, and -- because the race keeps a handler on + // `done` after it settles -- a producer that fails *after* delivery is an observed rejection + // rather than one that reaches Node's default `unhandledRejection` policy (TRANSPORT-19). + return await Promise.race([ + dispatcher.request({ + origin: request.url.origin, + path: `${request.url.pathname}${request.url.search}`, + method: request.method, + headers: context.headers, + body: context.body, + // Always a real signal since audit #67 / #82: the fork is this transport's own + // cancellation handle, not merely a relay for the caller's. + signal: context.fork.signal, + // TRANSPORT-1: pinned explicitly rather than inherited -- a BYO dispatcher may carry a + // redirect interceptor, and the pipeline is the single redirect authority. + maxRedirections: 0, + }), + producerFailure(pump?.done), + ]); + } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const mapped = toDispatchError(error, context.fork.signal); + await pump?.abandon(error); + // TRANSPORT-9: when the producer lost the race, undici is still dispatching. Nothing awaits + // it any more, so a response that arrives later would be dropped with its `BodyReadable` + // neither read nor destroyed, holding the pooled connection. Pulling the fork takes the + // dispatch down instead; on the path where undici itself rejected it is inert. + context.fork.abort(error); + throw mapped; + } + } + + /** + * Releases every dispatcher this transport constructed, in reverse acquisition order, and never a + * caller-supplied one (SEAM-14, TRANSPORT-15). Idempotent, and concurrent calls share one + * teardown (TRANSPORT-16). + * + * `destroy()`, not undici's graceful `close()`: TRANSPORT-16 requires a non-blocking shutdown with + * no unbounded await, and `close()` waits for every enqueued request to finish — one in-flight send + * against a slow peer would stall teardown for that peer's whole timeout. Sends still in flight + * therefore reject with the terminal `CancellationError`, which is also this transport's documented + * SEAM-15 post-close mode: a send issued after `close()` cannot succeed over a dispatcher that no + * longer exists, so it is not reported as a retryable failure. + * + * A dispatcher that fails to release does not strand the rest: every owned dispatcher is destroyed + * before the failure is reported, so one bad pool cannot leak the others. + * + * @returns a promise that resolves once the owned dispatchers are released. + * @throws `TransportFailureError` when one or more owned dispatchers failed to release. The + * rejection is memoized like the success path, so a later `close()` reports the same failure rather + * than falsely claiming a clean teardown. + */ + close(): Promise<void> { + this.#closing ??= (async () => { + // Every owned dispatcher is destroyed even when an earlier one rejects. A bare `for … await` + // loop propagates on the first failure and leaks the pooled connections of every dispatcher + // after it -- and `owned` is walked in reverse, so with a proxy configured the ProxyAgent + // actually holding those connections is the one destroyed last. + const failures = await releaseAll(this.#dispatchers.owned); + if (failures.length > 0) { + // A raw undici error would otherwise escape a public method untyped (NFR-7); the causes are + // preserved rather than flattened to a message. + throw new TransportFailureError( + 'one or more owned dispatchers failed to release', + { + cause: + failures.length === 1 + ? failures[0] + : new AggregateError(failures), + }, + ); + } + })(); + return this.#closing; + } +} + +// Single teardown path for `await using`, delegating to `UndiciTransport.close()` and installed at run +// time only when the symbol exists — the same guarded shape `SseStream` and `Page` use. +// +// DO NOT restore this as a plain `[Symbol.asyncDispose]()` class member. Node 20.3 is this package's +// declared floor (`engines.node`, checked by verify:runtime-floor) and predates the symbol, which +// arrived in 20.4. On the floor the computed key evaluates to `undefined` and binds the method to the +// string key `"undefined"` — a junk prototype entry, and no working disposal. Declaring it on the +// class would also emit it into the `.d.ts` unconditionally, promising consumers on the floor a method +// that is not there. +if (typeof Symbol.asyncDispose === 'symbol') { + Object.defineProperty(UndiciTransport.prototype, Symbol.asyncDispose, { + value: function asyncDispose(this: UndiciTransport): Promise<void> { + return this.close(); + }, + writable: true, + configurable: true, + }); +} + +/** + * Creates a `Transport` backed by `undici` — the full-featured option, with connection-pool control, + * proxy support, and real `close()` semantics over the dispatchers it owns. + * + * `close()` is the single teardown path `docs/knowledge/harvested/resource-management.md` asks for, and the one + * that actually destroys the dispatchers this transport owns. A `[Symbol.asyncDispose]` delegating to + * it is installed at run time **when the runtime has the symbol**, which this package's declared floor + * (`engines.node >=20.3`) does not — it arrived in Node 20.4. The return type therefore does not + * promise `AsyncDisposable`: claiming it would type-check `await using` for a consumer sitting on the + * floor, where the method is genuinely absent, and leak every pooled connection. Call `close()`, or + * raise your own floor to 20.4+ and reach the symbol through a cast. + * + * @param options - optional transport settings. + * @returns a transport ready to send; release it with `close()`. + * @throws `TypeError` when both `dispatcher` and `proxy` are supplied; when `proxy.type` is + * anything but `http` — undici's `ProxyAgent` cannot carry a SOCKS proxy, and neither can + * `@dexpace/transport-fetch`, which has no proxy option at all; or when `defaultTimeoutMs` is not + * an integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s range. + * + * @public + */ +export function undiciTransport( + options: UndiciTransportOptions = {}, +): Transport { + return new UndiciTransport(options); +} diff --git a/packages/transport-undici/tsconfig.build.json b/packages/transport-undici/tsconfig.build.json new file mode 100644 index 0000000..d39dc7e --- /dev/null +++ b/packages/transport-undici/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "exclude": [ + "src/**/*.test.ts" + ] +} diff --git a/packages/transport-undici/tsconfig.json b/packages/transport-undici/tsconfig.json new file mode 100644 index 0000000..5c03021 --- /dev/null +++ b/packages/transport-undici/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ] + }, + "include": [ + "src/**/*.ts" + ] +} diff --git a/scripts/changeset.mjs b/scripts/changeset.mjs new file mode 100644 index 0000000..d1d9493 --- /dev/null +++ b/scripts/changeset.mjs @@ -0,0 +1,122 @@ +// SPDX-License-Identifier: MIT +// scripts/changeset.mjs +// +// Wrapper around the changesets CLI that renames a newly created changeset +// from `@changesets/write`'s random `human-id` name (`silly-pandas-jump.md`) +// to this repo's convention: `YYYY-MM-DD-<kebab-slug>.md`, the same name shape +// every document under `docs/work/mvp/` carries. +// +// The name is not a config knob — the ID comes from a hardcoded `humanId()` +// call inside `@changesets/write`, and `.changeset/config.json`'s schema has +// no filename field. Renaming afterwards is safe because nothing reads the +// filename back: the CLI globs `.changeset/*.md` (skipping `README.md` and +// `config.json`) and takes every decision from the frontmatter. +// +// Every argument is forwarded to `changeset` verbatim. Only the invocations +// that can create a changeset (`add`, or no subcommand) are renamed; +// `version`, `status`, `publish`, `tag`, `pre` and `init` pass through +// untouched. +import {spawnSync} from 'node:child_process'; +import {existsSync, readFileSync, readdirSync, renameSync} from 'node:fs'; +import {join} from 'node:path'; +import {createInterface} from 'node:readline/promises'; +import {stdin, stdout} from 'node:process'; +import {fileURLToPath} from 'node:url'; + +const CHANGESET_DIR = fileURLToPath(new URL('../.changeset', import.meta.url)); +const NON_CHANGESET_FILES = new Set(['README.md']); +const PASSTHROUGH_SUBCOMMANDS = new Set([ + 'version', + 'status', + 'publish', + 'tag', + 'pre', + 'init', +]); +const MAX_SLUG_LENGTH = 48; + +function listChangesets() { + return readdirSync(CHANGESET_DIR).filter( + name => name.endsWith('.md') && !NON_CHANGESET_FILES.has(name), + ); +} + +function today() { + const now = new Date(); + const month = String(now.getMonth() + 1).padStart(2, '0'); + const day = String(now.getDate()).padStart(2, '0'); + return `${now.getFullYear()}-${month}-${day}`; +} + +function toSlug(text) { + const words = text + .toLowerCase() + .replace(/[^a-z0-9]+/g, '-') + .replace(/^-+|-+$/g, '') + .split('-') + .filter(Boolean); + + const kept = []; + let length = 0; + for (const word of words) { + const next = length === 0 ? word.length : length + 1 + word.length; + if (kept.length > 0 && next > MAX_SLUG_LENGTH) break; + kept.push(word); + length = next; + } + return kept.join('-').slice(0, MAX_SLUG_LENGTH); +} + +// The summary a changeset was written with is the best default name for it. +// Frontmatter is delimited by the first two `---` lines; the summary is the +// first non-empty line after that, cut at its first sentence — these summaries +// open with a title-like clause and then keep going for paragraphs. +function summaryOf(fileName) { + const lines = readFileSync(join(CHANGESET_DIR, fileName), 'utf8').split('\n'); + const closing = lines.indexOf('---', lines.indexOf('---') + 1); + const summary = lines.slice(closing + 1).find(line => line.trim() !== ''); + return (summary ?? '').trim().split(/(?<=\.)\s/)[0]; +} + +function uniqueName(date, slug) { + const base = `${date}-${slug}`; + if (!existsSync(join(CHANGESET_DIR, `${base}.md`))) return `${base}.md`; + for (let suffix = 2; ; suffix++) { + const candidate = `${base}-${suffix}.md`; + if (!existsSync(join(CHANGESET_DIR, candidate))) return candidate; + } +} + +async function askSlug(fallback) { + // Non-interactive callers (CI, a scripted `--empty`) get the derived slug + // rather than a hang on a prompt nobody can answer. + if (!stdin.isTTY) return fallback; + const rl = createInterface({input: stdin, output: stdout}); + try { + const answer = await rl.question(`Changeset slug (${fallback}): `); + const slug = toSlug(answer); + return slug === '' ? fallback : slug; + } finally { + rl.close(); + } +} + +async function rename(fileName) { + const derived = toSlug(summaryOf(fileName)) || 'changeset'; + const target = uniqueName(today(), await askSlug(derived)); + renameSync(join(CHANGESET_DIR, fileName), join(CHANGESET_DIR, target)); + console.log(`Renamed ${fileName} -> ${target}`); +} + +const args = process.argv.slice(2); +const creates = !PASSTHROUGH_SUBCOMMANDS.has(args[0] ?? 'add'); +const before = creates ? new Set(listChangesets()) : new Set(); + +const result = spawnSync('bunx', ['changeset', ...args], {stdio: 'inherit'}); +if (result.status !== 0) process.exit(result.status ?? 1); + +if (creates) { + for (const fileName of listChangesets().filter(name => !before.has(name))) { + await rename(fileName); + } +} diff --git a/scripts/knowledge-drift.mjs b/scripts/knowledge-drift.mjs new file mode 100644 index 0000000..50f12d8 --- /dev/null +++ b/scripts/knowledge-drift.mjs @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: MIT +// scripts/knowledge-drift.mjs +// +// Reports what has gone stale in `docs/knowledge/`, in two dimensions: +// +// sources — every sha256 recorded in `harvested/SOURCES.md` against the file +// on disk, so you can see which harvested entries describe a +// document that has since changed. +// keys — every `<topic>/<8 hex>` a note cites, against the corpus, so you +// can see which notes name a rule whose text no longer exists. A +// key digests entry text; a re-harvest that rewords a rule breaks +// the citation, and that is precisely when the note needs revisiting. +// +// Named for its subject rather than a verb, like `knowledge.mjs` and +// `changeset.mjs`: the `verify-*.mjs` prefix in this directory belongs to the +// blocking gates, and this is a report. +// +// A report, not a gate, and deliberately not in CI: no drift state fails it. +// (A manifest that is missing or malformed still exits 2 — that is the report +// being unable to run, not a state it reports.) Two reasons. The styleguide +// root is a sibling repository addressed by an absolute path on the harvest +// machine, so 16 of the sources simply do not exist in a CI checkout — they are +// NOT VERIFIABLE, never a failure. And drift is normal: a design chapter that a +// phase edits to record an outcome SHOULD drift, and the fix is a re-harvest, +// which is a user-invoked skill rather than something CI can do. +// +// The states are OK, DRIFT, NOT VERIFIABLE and UNREADABLE. +import {createHash} from 'node:crypto'; +import {readFileSync} from 'node:fs'; +import {join} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +import { + danglingKeys, + derivePrefixes, + loadCanonicalIds, + loadCorpus, +} from './knowledge.mjs'; + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const sourcesPath = join( + repoRoot, + 'docs', + 'knowledge', + 'harvested', + 'SOURCES.md', +); + +// `| \`path\` | role | \`sha\` | date |` — the sha cell sometimes carries an +// annotation after the digest, so take the first backticked token in it. +const SOURCE_ROW = /^\|\s*`([^`]+)`\s*\|\s*([^|]*?)\s*\|\s*`([0-9a-f]+)`/; +// Any data row of the manifest table, parseable or not. Counting these is what +// turns "47 sources OK" from a count of rows that happened to match into a +// statement about the table: a row with an upper-case or empty digest used to +// vanish, and the summary line reported the smaller number as if it were all. +const ANY_ROW = /^\|\s*`([^`]+)`\s*\|/; +// A digest short enough to collide by accident is not a pin. The manifest +// records 12 hex; comparing at the recorded width alone would let a truncated +// row compare clean forever. +const MIN_SHA_LENGTH = 12; + +function parseSources(text) { + const rows = []; + let listed = 0; + for (const line of text.split('\n')) { + if (!ANY_ROW.test(line)) continue; + listed += 1; + const match = SOURCE_ROW.exec(line); + if (match) rows.push({path: match[1], role: match[2], sha: match[3]}); + } + if (rows.length === 0) { + throw new Error( + `parsed zero source rows out of ${sourcesPath}; its table format changed`, + ); + } + if (rows.length < listed) { + throw new Error( + `${sourcesPath} lists ${listed} sources but only ${rows.length} parse; ` + + 'the rest carry a malformed digest cell and would be silently skipped', + ); + } + for (const row of rows) { + if (row.sha.length >= MIN_SHA_LENGTH) continue; + throw new Error( + `${row.path} is pinned to a ${row.sha.length}-character digest; at least ` + + `${MIN_SHA_LENGTH} are needed for the comparison to mean anything`, + ); + } + return rows; +} + +// The manifest records a truncated digest, so compare at the recorded width. +// +// Only ENOENT is NOT VERIFIABLE. An unreadable or wrong-typed path reported as +// "not present" would hide inside the one state this report teaches the reader +// to ignore — off the harvest machine, 16 sources are legitimately absent. +function stateOf(row) { + let bytes; + try { + bytes = readFileSync( + row.path.startsWith('/') ? row.path : join(repoRoot, row.path), + ); + } catch (error) { + if (error.code === 'ENOENT') return {state: 'NOT VERIFIABLE', actual: null}; + return {state: 'UNREADABLE', actual: null, detail: error.code}; + } + const actual = createHash('sha256') + .update(bytes) + .digest('hex') + .slice(0, row.sha.length); + return {state: actual === row.sha ? 'OK' : 'DRIFT', actual}; +} + +function detailFor(row, result) { + if (result.state === 'DRIFT') { + return `recorded ${row.sha}, actual ${result.actual}`; + } + if (result.state === 'UNREADABLE') return `read failed: ${result.detail}`; + return 'file not present in this checkout'; +} + +// A note names the rule it overrides by key. Report every citation that no +// longer resolves — the rule was reworded, so the note is describing something +// that is not there any more. +function reportKeys() { + const entries = loadCorpus(derivePrefixes(loadCanonicalIds())); + const dangling = danglingKeys(entries); + for (const {note, cited} of dangling) { + process.stdout.write( + `STALE KEY\t${note}\tcites ${cited}, which no entry carries\n`, + ); + } + const cited = entries + .filter(entry => entry.origin === 'note') + .reduce((total, note) => total + note.overrides.length, 0); + process.stdout.write( + `\n${cited} note citation(s) resolve, ${dangling.length} do not.\n`, + ); + if (dangling.length > 0) { + process.stdout.write( + 'A stale key means the harvested rule was reworded or re-harvested. ' + + 'Re-read the rule, then update the note to the key it prints now.\n', + ); + } +} + +function main() { + const rows = parseSources(readFileSync(sourcesPath, 'utf8')); + const counts = {OK: 0, DRIFT: 0, 'NOT VERIFIABLE': 0, UNREADABLE: 0}; + + for (const row of rows) { + const result = stateOf(row); + counts[result.state] += 1; + if (result.state === 'OK') continue; + process.stdout.write( + `${result.state}\t${row.path}\t${detailFor(row, result)}\n`, + ); + } + + process.stdout.write( + `\n${rows.length} harvested sources: ${counts.OK} OK, ` + + `${counts.DRIFT} DRIFT, ${counts['NOT VERIFIABLE']} NOT VERIFIABLE, ` + + `${counts.UNREADABLE} UNREADABLE.\n`, + ); + if (counts.DRIFT > 0) { + process.stdout.write( + 'A drifted source means the harvested entries derived from it describe an ' + + 'older revision. Re-harvest that source, or record what changed as a ' + + 'note under docs/knowledge/notes/. This check never fails the build.\n', + ); + } + if (counts['NOT VERIFIABLE'] > 0) { + process.stdout.write( + 'NOT VERIFIABLE is expected off the harvest machine: the styleguide root ' + + 'is a sibling repository at an absolute path. It is not a failure.\n', + ); + } + reportKeys(); + return 0; +} + +export {parseSources, stateOf, detailFor}; + +if (process.argv[1] === fileURLToPath(import.meta.url)) { + try { + process.exitCode = main(); + } catch (error) { + const messages = []; + for (let current = error; current; current = current.cause) { + messages.push(current.message); + } + process.stderr.write(`${messages.join('\n caused by: ')}\n`); + process.exitCode = 2; + } +} diff --git a/scripts/knowledge.mjs b/scripts/knowledge.mjs new file mode 100644 index 0000000..b550ac2 --- /dev/null +++ b/scripts/knowledge.mjs @@ -0,0 +1,1186 @@ +// SPDX-License-Identifier: MIT +// scripts/knowledge.mjs +// +// Query surface over `docs/knowledge/`. The corpus is ~1.5k entries across two +// trees; without a filter, answering "what do we already know about RETRY-12" +// means reading a 20 KB file. This turns that into a query that returns the +// handful of entries that actually cite the requirement. +// +// Two trees, one query surface: +// harvested/ generated by `knowledge-harvest`, never hand-edited. A `<sub>` +// sha there digests the WHOLE source file, so a hand edit inside +// an entry is invisible to the next harvest — it is regenerated +// or duplicated. Hence: no hand edits, ever. +// notes/ hand-written during implementation. Role `review`, a manual +// `sha:` marker. What the implementation found, which outranks +// what the documents say. +// +// `verify-knowledge-structure.mjs` is the gate that keeps the two apart. This +// file is not a gate — `--coverage` is a report you run by hand. +// +// Zero dependencies, plain Node ESM, same shape as the `verify-*.mjs` scripts. +import {createHash} from 'node:crypto'; +import {readFileSync, readdirSync} from 'node:fs'; +import {join, basename} from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {parseArgs} from 'node:util'; + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const knowledgeDir = join(repoRoot, 'docs', 'knowledge'); +const harvestedDir = join(knowledgeDir, 'harvested'); +const notesDir = join(knowledgeDir, 'notes'); +const appendixCPath = join( + repoRoot, + 'docs', + 'product-spec', + 'appendix-c-consolidated-normative-requirement-index.md', +); + +// `INDEX.md` is a generated topic table, `SOURCES.md` a provenance manifest and +// `README.md` the two-tree contract; none holds entries, and all would parse as +// noise. +const NON_TOPIC_FILES = new Set(['INDEX.md', 'SOURCES.md', 'README.md']); + +// Appendix B is the conformance-test checklist. Its entries roll several +// requirement IDs into one "the suite verifies X, Y, Z" sentence, so they make +// an ID look cited while carrying none of its content. 256 of the 641 cited IDs +// resolve ONLY to a roll-up — a silent wrong answer unless it is called out. +const ROLLUP_SOURCE = 'appendix-b-conformance-test-checklist'; + +// Provenance roles the corpus uses, most common first. `review` is the notes +// tree's role and appears nowhere under `harvested/`. +const ROLES = ['spec', 'design', 'styleguide', 'review']; + +// The two trees, and the value of an entry's `origin` field for each. +const ORIGINS = ['harvested', 'note']; +const TREES = [ + {origin: 'harvested', dir: harvestedDir, required: true}, + {origin: 'note', dir: notesDir, required: false}, +]; + +// The six section names the parser recognises, in emission order. No file +// carries all six: `Superseded` exists only under `notes/`, where an override +// is recorded — `verify-knowledge-structure.mjs` rejects one under `harvested/`. +const SECTIONS = [ + 'Rules', + 'Constraints', + 'Conclusions', + 'Reference', + 'Conflicts', + 'Superseded', +]; + +// --------------------------------------------------------------------------- +// Canonical requirement IDs +// --------------------------------------------------------------------------- + +// A bare `\b[A-Z]{2,12}-\d+\b` is not a requirement-ID matcher: it also claims +// `UTF-8` (10 hits in the corpus), `SHA-256`, `ISO-8601` and `RFC-3986`. The +// only authority on what a requirement ID looks like is appendix C, so the +// prefix allowlist is derived from it at runtime and never hardcoded — a spec +// revision that adds a subsystem is picked up without editing this file. +const ID_TOKEN = /\b[A-Z][A-Z0-9]{1,11}-\d+\b/g; +const APPENDIX_C_ROW = /^\|\s*([A-Z][A-Z0-9]{1,11}-\d+)\s*\|/; + +function loadCanonicalIds() { + let text; + try { + text = readFileSync(appendixCPath, 'utf8'); + } catch (cause) { + throw new Error( + `cannot read the canonical requirement index at ${appendixCPath}; ` + + 'the requirement-ID allowlist is derived from it and there is no fallback', + {cause}, + ); + } + + const ids = new Map(); + for (const line of text.split('\n')) { + const match = APPENDIX_C_ROW.exec(line); + if (!match) continue; + const cells = line.split('|').map(cell => cell.trim()); + // `| ID | Level | Subsystem | Requirement |` — leading/trailing empties. + ids.set(match[1], { + id: match[1], + level: cells[2] ?? '', + subsystem: cells[3] ?? '', + }); + } + + if (ids.size === 0) { + throw new Error( + `parsed zero requirement IDs out of ${appendixCPath}; its table format ` + + 'changed and the allowlist cannot be derived — refusing to fall back ' + + 'to a bare regex, which false-positives on UTF-8 and SHA-256', + ); + } + return ids; +} + +function derivePrefixes(canonicalIds) { + const prefixes = new Set(); + for (const id of canonicalIds.keys()) { + prefixes.add(id.slice(0, id.lastIndexOf('-'))); + } + return prefixes; +} + +// Tokenize, then compare whole tokens. Never substring-match: `HTTP-7` and +// `HTTP-70` are different requirements and a substring test conflates them. +function extractIds(text, prefixes) { + const found = []; + for (const [token] of text.matchAll(ID_TOKEN)) { + if (!prefixes.has(token.slice(0, token.lastIndexOf('-')))) continue; + if (!found.includes(token)) found.push(token); + } + return found; +} + +// --------------------------------------------------------------------------- +// Corpus parsing +// --------------------------------------------------------------------------- + +const SECTION_HEADING = /^##\s+(.+?)\s*$/; +const BULLET_START = /^-\s+(.*)$/; +const SUB_LINE = /^\s+<sub>(.*)<\/sub>\s*$/; +const ROLE_AND_SOURCE = /^(\S+)\s+`(.+)`$/; +const BARE_SOURCE = /^`(.+)`$/; + +// Two `<sub>` shapes are in the corpus: +// role · `path:lines` · confidence · sha:xxxx (the common one) +// roleA `pathA` · roleB `pathB` · resolution-status (Conflicts entries) +// so role and path are sometimes separate ` · ` fields and sometimes one. Walk +// the fields and classify each rather than reading them positionally. +function parseSub(inner) { + const roles = []; + const sources = []; + let confidence = null; + let sha = null; + let pendingRole = null; + + for (const raw of inner.split(' · ')) { + const field = raw.trim(); + + const pair = ROLE_AND_SOURCE.exec(field); + if (pair) { + roles.push(pair[1]); + sources.push(pair[2]); + pendingRole = null; + continue; + } + + const bare = BARE_SOURCE.exec(field); + if (bare) { + roles.push(pendingRole ?? 'unknown'); + sources.push(bare[1]); + pendingRole = null; + continue; + } + + if (field.startsWith('sha:')) { + sha = field.slice('sha:'.length); + continue; + } + + // A lone word that is not a confidence level is the role of the source + // field that follows it; anything else is the confidence / status. + if (pendingRole !== null) confidence = pendingRole; + pendingRole = /^\S+$/.test(field) ? field : null; + if (pendingRole === null) confidence = field; + } + if (pendingRole !== null) confidence = pendingRole; + + return { + role: roles[0] ?? null, + roles, + source: sources[0] ?? null, + sources, + confidence, + sha, + }; +} + +// A citable name for one entry, stable across a re-order and a re-harvest. +// The line number is not that name: an entry moves whenever a neighbour is +// added. The `<sub>` sha is not either — it digests the whole source file, so +// every entry harvested from one file shares it. The digest of the entry's own +// text changes when, and only when, the rule changes, which is what a note +// naming a broken rule wants: a changed rule needs a fresh check. +function entryKey(topic, text) { + const hash = createHash('sha256').update(text.trimEnd()).digest('hex'); + return `${topic}/${hash.slice(0, 8)}`; +} + +// A topic file authored on Windows, or saved with a BOM, is the one input that +// fails silently rather than loudly: `\r` defeats BULLET_START's `$` and a BOM +// defeats SECTION_HEADING, so the file parses to zero entries or to entries +// with no section, and every downstream gate then reports OK over a hole. The +// repo has no `.gitattributes`, so `core.autocrlf=true` is one contributor +// away. Normalize both at the door. +function readTopicFile(path) { + let text; + try { + text = readFileSync(path, 'utf8'); + } catch (cause) { + throw new Error(`cannot read the topic file ${path}`, {cause}); + } + return text.replace(/^\uFEFF/, '').split(/\r?\n/); +} + +// One bullet carrying two `<sub>` lines is malformed, but overwriting the first +// with the second drops its sources — and a source is what the structural gate +// checks, so the malformed half would escape the check. Accumulate instead, and +// let the gate see everything the entry cites. +function mergeSub(entry, parsed, subLine) { + if (entry.subLine === null) { + Object.assign(entry, parsed, {subLine}); + return; + } + entry.roles.push(...parsed.roles); + entry.sources.push(...parsed.sources); + entry.subLine += ` ${subLine}`; +} + +function parseFile(path, prefixes, origin = 'harvested') { + const entries = []; + const lines = readTopicFile(path); + const file = basename(path); + const topic = file.replace(/\.md$/, ''); + let section = null; + let current = null; + + const flush = () => { + if (!current) return; + current.reqs = extractIds(current.text, prefixes); + current.key = entryKey(topic, current.text); + entries.push(current); + current = null; + }; + + for (const [index, line] of lines.entries()) { + const heading = SECTION_HEADING.exec(line); + if (heading) { + flush(); + section = heading[1]; + continue; + } + + const sub = SUB_LINE.exec(line); + if (sub && current) { + mergeSub(current, parseSub(sub[1]), line.trim()); + continue; + } + + const bullet = BULLET_START.exec(line); + if (bullet) { + flush(); + current = { + file, + topic, + origin, + key: null, + overriddenBy: [], + overrides: [], + line: index + 1, + section, + text: bullet[1], + role: null, + roles: [], + source: null, + sources: [], + confidence: null, + sha: null, + subLine: null, + }; + continue; + } + + // A continuation line: any non-`<sub>` line before the open bullet's + // provenance line, blank lines included — a few Conflicts entries run to + // several paragraphs, and dropping the tail silently loses the requirement + // IDs it cites. An entry therefore ends only at its `<sub>`, at the next + // bullet, at the next heading, or at end of file. + if (current && !current.subLine && line.trim() !== '') { + current.text += ` ${line.trim()}`; + } + } + + flush(); + return entries; +} + +function loadCorpus(prefixes) { + const entries = topicFiles().flatMap(({path, origin}) => + parseFile(path, prefixes, origin), + ); + return linkOverrides(entries); +} + +// A note names the harvested rule it overrides by that rule's key. Resolve those +// references once, at load, and hang the answer on both ends. +// +// Without this the override is prose inside one note and nothing else: a query +// that lands on the harvested entry — by `--section reference`, by a bare word, +// by anything that does not also return the note — hands back a rule the +// implementation has already overruled, with no sign that it did. The corpus's +// highest-stakes entry is exactly that case: the pagination snippet whose note +// records that following it "would have shipped a MUST violation the checklist +// could not catch". +// +// `overriddenBy` is also how a stale reference becomes visible. A key digests +// entry text, so a re-harvest that rewords a rule leaves the citing note +// pointing at nothing; `danglingKeys` is that list, and `--key` reports it. +function linkOverrides(entries) { + const byKey = new Map(entries.map(entry => [entry.key, entry])); + for (const note of entries) { + if (note.origin !== 'note') continue; + for (const [, cited] of note.text.matchAll(CITED_KEY)) { + const target = byKey.get(cited); + if (target === undefined || target === note) continue; + target.overriddenBy.push(entryLocation(note)); + note.overrides.push(cited); + } + } + return entries; +} + +// A key inside an entry's prose, backticked: `pagination/81881061`. +const CITED_KEY = /`([a-z0-9-]+\/[0-9a-f]{8})`/g; + +function danglingKeys(entries) { + const known = new Set(entries.map(entry => entry.key)); + const dangling = []; + for (const note of entries) { + if (note.origin !== 'note') continue; + for (const [, cited] of note.text.matchAll(CITED_KEY)) { + if (!known.has(cited)) dangling.push({note: entryLocation(note), cited}); + } + } + return dangling; +} + +// Every topic file in both trees, harvested first, as records rather than bare +// names: a caller needs the tree an entry came from, and `pagination.md` exists +// in both. +function topicFiles() { + return TREES.flatMap(({origin, dir, required}) => + readTree(dir, required).map(name => ({ + file: name, + topic: name.replace(/\.md$/, ''), + path: join(dir, name), + origin, + })), + ); +} + +function readTree(dir, required) { + let names; + try { + names = readdirSync(dir); + } catch (cause) { + if (required) { + throw new Error( + `cannot read the harvested corpus at ${dir}; docs/knowledge/ is two ` + + 'trees (harvested/ and notes/) and the harvested one is not optional', + {cause}, + ); + } + // The notes tree holds a file only where a note exists. None yet is a + // legitimate state, not a broken checkout. + return []; + } + return names + .filter(name => name.endsWith('.md') && !NON_TOPIC_FILES.has(name)) + .sort(); +} + +// True when every source this entry cites is the conformance checklist, i.e. +// the entry names requirement IDs without saying anything about them. +function isRollup(entry) { + return ( + entry.sources.length > 0 && + entry.sources.every(source => source.includes(ROLLUP_SOURCE)) + ); +} + +// Styleguide `<sub>` paths carry a numbered chapter file +// (`.../typescript/06-classes-and-data-modeling.md:168-183`), so "styleguide +// 6.7" is answerable by matching the chapter number — no hardcoded chapter → +// topic table that could drift from the styleguide itself. +// Only styleguide-role sources count: `docs/product-spec/04-…md` is a numbered +// chapter too, and conflating the two would answer "styleguide 4" with spec +// chapter 4. `roles[i]` pairs with `sources[i]`. +const STYLEGUIDE_CHAPTER = /\/(\d{2})-[^/]*\.md(?::|$)/; + +function chaptersOf(entry) { + const chapters = []; + entry.sources.forEach((source, index) => { + if (entry.roles[index] !== 'styleguide') return; + const match = STYLEGUIDE_CHAPTER.exec(source); + if (match) chapters.push(String(Number(match[1]))); + }); + return chapters; +} + +// --------------------------------------------------------------------------- +// Filtering +// --------------------------------------------------------------------------- + +// `--req A,B --req C` and `--req A --req B` mean the same thing: values inside +// one filter OR. +// +// An empty value is dropped, and a filter whose values were ALL empty throws. +// `--topic ''` and the one-character typo `--topic 'a,b,'` would otherwise match +// every file — `''` is a substring of everything and a regex that matches +// everything — so a trailing comma in the skill's own comma-separated audit form +// silently turns a 55-entry query into the whole corpus, which is the one thing +// this tool exists to prevent. +function splitValues(values, flag) { + const supplied = (values ?? []).flatMap(value => value.split(',')); + const kept = supplied.filter(value => value.trim() !== ''); + if (supplied.length > 0 && kept.length === 0) { + throw new Error( + `${flag} was given only empty values; an empty value matches every ` + + 'entry, so this would print the whole corpus. Drop the flag, or a ' + + 'stray comma.', + ); + } + return kept; +} + +function warnUncanonicalIds(reqs, canonicalIds) { + for (const id of reqs) { + if (canonicalIds.has(id)) continue; + process.stderr.write( + `warning: ${id} is not in appendix C — it is not a canonical ` + + 'requirement ID, so no entry can legitimately cite it\n', + ); + } +} + +function parseRoles(options) { + return splitValues(options.role, '--role').map(value => { + if (!ROLES.includes(value)) { + throw new Error( + `unknown role '${value}'; the roles are ${ROLES.join(', ')}`, + ); + } + return value; + }); +} + +function parseOrigins(options) { + return splitValues(options.origin, '--origin').map(value => { + if (!ORIGINS.includes(value)) { + throw new Error( + `unknown origin '${value}'; the origins are ${ORIGINS.join(', ')} — ` + + 'harvested/ is what the documents say, notes/ what the ' + + 'implementation found', + ); + } + return value; + }); +} + +// An ID family, for the audit query: `--prefix HTTP` is every HTTP-N entry, +// which `--req` cannot express without listing all 92 of them. Validated +// against appendix C for the same reason `--req` warns — `--prefix UTF` would +// otherwise be a silent empty result rather than a typo. +function parseIdPrefixes(options, known) { + return splitValues(options.prefix, '--prefix').map(value => { + const prefix = value.trim().toUpperCase(); + if (!known.has(prefix)) { + throw new Error( + `'${prefix}' is not a requirement-ID prefix in appendix C, so no ` + + 'entry can legitimately cite one. The prefixes are ' + + `${[...known].sort().join(' ')}`, + ); + } + return prefix; + }); +} + +// A key is `<topic>/<8 hex>`. Not validated against the corpus here: a key that +// resolves to nothing is exactly what a reader needs told — the note citing it +// is describing a rule whose text has changed — so it flows through to +// renderNoMatches, which says so. +const ENTRY_KEY = /^[a-z0-9-]+\/[0-9a-f]{8}$/; + +function parseKeys(options) { + return splitValues(options.key, '--key').map(value => { + const key = value.trim(); + if (!ENTRY_KEY.test(key)) { + throw new Error( + `'${key}' is not an entry key; a key is <topic>/<8 hex>, as printed ` + + 'after the section name on every result', + ); + } + return key; + }); +} + +// "styleguide 6.7" — the chapter is queryable, the sub-section number is not, +// so take the chapter and say plainly that the rest was dropped. +function parseChapters(options) { + return splitValues(options.chapter, '--chapter').map(value => { + const match = /^(\d{1,2})(?:\.(\d+))?$/.exec(value.trim()); + if (!match) { + throw new Error( + `unknown chapter '${value}'; expected a styleguide chapter like 6 or 6.7`, + ); + } + if (match[2] !== undefined) { + process.stderr.write( + 'note: entries record a chapter file and line range, not section ' + + `numbers — querying chapter ${match[1]}, ignoring .${match[2]}. ` + + 'Narrow with bare words.\n', + ); + } + return String(Number(match[1])); + }); +} + +function parseSections(options) { + return splitValues(options.section, '--section').map(value => { + const resolved = SECTIONS.find( + name => name.toLowerCase() === value.toLowerCase(), + ); + if (!resolved) { + throw new Error( + `unknown section '${value}'; the six sections are ${SECTIONS.join(', ')}`, + ); + } + return resolved; + }); +} + +// An empty pattern matches every entry, exactly as an empty `--topic` does, so +// it gets the same refusal rather than printing the corpus. +function parsePatterns(options, positionals) { + const sources = [...(options.grep ?? []), ...positionals]; + if (sources.length > 0 && sources.every(value => value.trim() === '')) { + throw new Error( + '--grep was given only empty values; an empty pattern matches every ' + + 'entry, so this would print the whole corpus.', + ); + } + const patterns = []; + for (const source of options.grep ?? []) { + if (source.trim() !== '') patterns.push(new RegExp(source, 'i')); + } + for (const word of positionals) { + if (word.trim() !== '') patterns.push(new RegExp(escapeRegExp(word), 'i')); + } + return patterns; +} + +function buildFilters(options, positionals, canonicalIds) { + const reqs = splitValues(options.req, '--req'); + warnUncanonicalIds(reqs, canonicalIds); + + return { + reqs, + topics: splitValues(options.topic, '--topic'), + keys: parseKeys(options), + origins: parseOrigins(options), + prefixes: parseIdPrefixes(options, derivePrefixes(canonicalIds)), + roles: parseRoles(options), + chapters: parseChapters(options), + sections: parseSections(options), + patterns: parsePatterns(options, positionals), + }; +} + +function escapeRegExp(value) { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +// Every supplied filter must hold — they AND, never OR. Within one filter, +// multiple values OR (`--req A --req B` is "cites A or B"). +function matches(entry, filters) { + const { + reqs, + topics, + keys, + origins, + prefixes, + roles, + chapters, + sections, + patterns, + } = filters; + if (keys.length > 0 && !keys.includes(entry.key)) return false; + if (reqs.length > 0 && !reqs.some(id => entry.reqs.includes(id))) + return false; + if ( + prefixes.length > 0 && + !entry.reqs.some(id => prefixes.includes(splitId(id)[0])) + ) { + return false; + } + if (topics.length > 0 && !topics.some(t => entry.file.includes(t))) { + return false; + } + if (origins.length > 0 && !origins.includes(entry.origin)) return false; + if (sections.length > 0 && !sections.includes(entry.section)) return false; + if (roles.length > 0 && !roles.some(r => entry.roles.includes(r))) { + return false; + } + if (chapters.length > 0) { + const entryChapters = chaptersOf(entry); + if (!chapters.some(c => entryChapters.includes(c))) return false; + } + if (!patterns.every(pattern => pattern.test(entry.text))) return false; + return true; +} + +function isEmptyFilter(filters) { + return ( + filters.reqs.length === 0 && + filters.keys.length === 0 && + filters.prefixes.length === 0 && + filters.origins.length === 0 && + filters.topics.length === 0 && + filters.roles.length === 0 && + filters.chapters.length === 0 && + filters.sections.length === 0 && + filters.patterns.length === 0 + ); +} + +// --------------------------------------------------------------------------- +// Reports +// --------------------------------------------------------------------------- + +function citationIndex(entries) { + const index = new Map(); + for (const entry of entries) { + for (const id of entry.reqs) { + if (!index.has(id)) index.set(id, []); + index.get(id).push(entry); + } + } + return index; +} + +function compareIds(a, b) { + const [prefixA, numberA] = splitId(a); + const [prefixB, numberB] = splitId(b); + return prefixA === prefixB + ? numberA - numberB + : prefixA.localeCompare(prefixB); +} + +function splitId(id) { + const cut = id.lastIndexOf('-'); + return [id.slice(0, cut), Number(id.slice(cut + 1))]; +} + +function renderListReqs(index) { + const out = []; + for (const id of [...index.keys()].sort(compareIds)) { + const locations = index + .get(id) + .map(entry => entryLocation(entry)) + .join(' '); + out.push(`${id}\t${locations}`); + } + out.push(''); + out.push(`${index.size} requirement IDs cited across the corpus`); + return out.join('\n'); +} + +function renderListTopics(entries) { + const stats = new Map(); + for (const entry of entries) { + const name = entry.topic; + if (!stats.has(name)) { + stats.set(name, {entries: 0, notes: 0, ids: new Set()}); + } + const row = stats.get(name); + if (entry.origin === 'note') row.notes += 1; + else row.entries += 1; + entry.reqs.forEach(id => row.ids.add(id)); + } + const out = ['topic\tentries\tdistinct IDs\tnotes']; + for (const [name, {entries: count, notes, ids}] of [...stats].sort()) { + out.push(`${name}\t${count}\t${ids.size}\t${notes}`); + } + // Count only harvested topics as styleguide-derived: a note-only topic has no + // requirement ID either, and calling it styleguide-derived is simply wrong. + const idless = [...stats].filter( + ([, v]) => v.ids.size === 0 && v.entries > 0, + ).length; + const noted = [...stats].filter(([, v]) => v.notes > 0).length; + const harvested = [...stats].filter(([, v]) => v.entries > 0).length; + out.push(''); + out.push( + `${stats.size} topics, ${harvested} of them harvested. ${idless} ` + + 'harvested topics carry no requirement ID at all — those are ' + + 'styleguide-derived and are only reachable topic-first. ' + + `${noted} topics carry a hand-written note (\`--origin note\`), which ` + + 'states what the implementation found and overrides the harvested entry.', + ); + return out.join('\n'); +} + +function renderCoverage(canonicalIds, index) { + const uncovered = [...canonicalIds.keys()] + .filter(id => !index.has(id)) + .sort(compareIds); + + const byPrefix = new Map(); + let rollupOnlyTotal = 0; + for (const id of canonicalIds.keys()) { + const [prefix] = splitId(id); + if (!byPrefix.has(prefix)) { + byPrefix.set(prefix, {total: 0, missing: [], rollupOnly: []}); + } + const row = byPrefix.get(prefix); + row.total += 1; + const hits = index.get(id); + if (!hits) { + row.missing.push(id); + } else if (hits.every(isRollup)) { + // Cited, but only by a conformance-checklist sentence that names it. + row.rollupOnly.push(id); + rollupOnlyTotal += 1; + } + } + + const out = ['requirement-ID coverage of docs/knowledge/', '']; + out.push('prefix\tsubstantive\troll-up only\tuncited\ttotal\tuncited IDs'); + for (const prefix of [...byPrefix.keys()].sort()) { + const {total, missing, rollupOnly} = byPrefix.get(prefix); + const substantive = total - missing.length - rollupOnly.length; + out.push( + `${prefix}\t${substantive}\t${rollupOnly.length}\t${missing.length}\t` + + `${total}\t${missing.length === 0 ? '-' : missing.join(' ')}`, + ); + } + const substantiveTotal = + canonicalIds.size - uncovered.length - rollupOnlyTotal; + out.push(''); + out.push( + `${substantiveTotal}/${canonicalIds.size} canonical IDs have a substantive ` + + `entry. ${rollupOnlyTotal} more are named only by an appendix-B ` + + `conformance roll-up (cited, but no content). ${uncovered.length} are ` + + 'cited nowhere.', + ); + return out.join('\n'); +} + +// --------------------------------------------------------------------------- +// Output +// --------------------------------------------------------------------------- + +// `pagination.md` exists in both trees, so a bare basename is ambiguous. A +// harvested entry prints as `pagination.md:134`, a note as +// `notes/pagination.md:5`. +function entryLocation(entry) { + const prefix = entry.origin === 'note' ? 'notes/' : ''; + return `${prefix}${entry.file}:${entry.line}`; +} + +function plural(count, noun) { + return `${count} ${noun}${count === 1 ? '' : 's'}`; +} + +// The tags an entry can carry, in the order that matters to a reader: an +// override first, because it changes whether the entry is still true. +function tagsOf(entry) { + const tags = (entry.overriddenBy ?? []).map(at => ` [overridden by ${at}]`); + if (isRollup(entry)) tags.push(' [appendix-B roll-up]'); + return tags.join(''); +} + +function renderEntries(results, brief, filters) { + const out = []; + for (const entry of results) { + const key = entry.key ? ` ${entry.key}` : ''; + out.push( + `${entryLocation(entry)} (${entry.section})${key}${tagsOf(entry)}`, + ); + out.push(`- ${entry.text}`); + if (!brief && entry.subLine) out.push(` ${entry.subLine}`); + out.push(''); + } + const files = new Set( + results.map(entry => entryLocation(entry).split(':')[0]), + ); + const notes = results.filter(entry => entry.origin === 'note').length; + out.push( + `${plural(results.length, 'entry').replace('entrys', 'entries')} across ` + + `${plural(files.size, 'topic file')}` + + (notes > 0 + ? `, ${notes} of them notes — a note states what the implementation ` + + 'found and overrides the harvested entry where it names one' + : ''), + ); + + // The silent wrong answer this tool can give: a `--req` that "hits" but whose + // every hit merely names the ID in a conformance-checklist sentence. Exit 0 + // makes it look answered, so say so loudly instead. + if (results.every(isRollup) && (filters?.reqs.length ?? 0) > 0) { + out.push( + '', + 'WARNING: every result is an appendix-B conformance roll-up — it names ' + + `${filters.reqs.join(', ')} without stating the requirement. The corpus ` + + 'has no substantive entry. Read the canonical text in appendix C and ' + + 'the owning docs/product-spec/NN chapter instead.', + ); + } + return out.join('\n'); +} + +// A zero-result query must never look like "the corpus has nothing to say" when +// it is really a typo, the wrong topic name, or two filters that cannot both +// hold. Spend the tokens on saying which of those it was. +// +// The dimension a hint blames has to be the dimension that emptied the result. +// A per-filter hint that fires unconditionally states falsehoods: with +// `--prefix HTTP --req PAGE-11` it used to report "PAGE-11 is canonical but no +// entry cites it yet", which is wrong — three entries cite it, none of them an +// HTTP one. So each dimension is first tested alone, and only a dimension that +// matches nothing on its own gets to explain itself. +const DIMENSIONS = [ + 'reqs', + 'keys', + 'prefixes', + 'origins', + 'topics', + 'roles', + 'sections', + 'chapters', + 'patterns', +]; + +function soloFilters(filters, dimension) { + const solo = Object.fromEntries(DIMENSIONS.map(name => [name, []])); + solo[dimension] = filters[dimension]; + return solo; +} + +function renderNoMatches(filters, entries, index, canonicalIds) { + const out = ['no matching entries.']; + const used = DIMENSIONS.filter(name => filters[name].length > 0); + const barren = used.filter( + name => !entries.some(entry => matches(entry, soloFilters(filters, name))), + ); + + if (barren.length === 0) { + const alone = used + .map( + name => + `${name} ${entries.filter(entry => matches(entry, soloFilters(filters, name))).length}`, + ) + .join(', '); + out.push( + ' every filter matches something on its own; no entry satisfies all ' + + `of them at once (matching alone: ${alone}). Filters AND together — ` + + 'drop one.', + ); + return out.join('\n'); + } + + for (const name of barren) { + out.push(...HINTS[name](filters, entries, index, canonicalIds)); + } + return out.join('\n'); +} + +function hintForReqs(filters, entries, index, canonicalIds) { + const out = []; + for (const id of filters.reqs) { + const [prefix, number] = splitId(id); + out.push( + canonicalIds.has(id) + ? ` ${id} is canonical but no entry cites it yet.` + : ` ${id} is not a canonical requirement ID (not in appendix C).`, + ); + // Never offer the queried ID back as its own nearest neighbour. + const nearest = [...index.keys()] + .filter(other => other !== id && splitId(other)[0] === prefix) + .sort((a, b) => { + const distance = + Math.abs(splitId(a)[1] - number) - Math.abs(splitId(b)[1] - number); + return distance === 0 ? compareIds(a, b) : distance; + }) + .slice(0, 5); + out.push( + nearest.length > 0 + ? ` nearest cited ${prefix} IDs: ${nearest.join(' ')}` + : ` no ${prefix} ID is cited anywhere. cited prefixes: ${[ + ...new Set([...index.keys()].map(other => splitId(other)[0])), + ] + .sort() + .join(' ')}`, + ); + const topics = topicsForPrefix(entries, prefix); + if (topics.length > 0) { + out.push(` topics carrying ${prefix} knowledge: ${topics.join(' ')}`); + } + } + return out; +} + +// A key that resolves to nothing is not a typo, it is news: the entry it named +// has been reworded, so whatever cites it — a note, an audit — is describing a +// rule that no longer exists in that form and needs a fresh check. +function hintForKeys(filters, entries) { + return filters.keys.map(key => { + const [topic] = key.split('/'); + const citedBy = entries + .filter(entry => entry.text.includes(`\`${key}\``)) + .map(entry => entryLocation(entry)); + const known = entries.some(entry => entry.topic === topic); + const where = + citedBy.length > 0 ? ` It is cited by ${citedBy.join(' ')}` : ''; + return known + ? ` no entry carries the key ${key}. A key digests the entry's text, ` + + `so a reworded or re-harvested rule gets a new one.${where}` + + (citedBy.length > 0 ? ', which needs updating.' : '') + : ` no entry carries the key ${key}, and no topic '${topic}' exists.`; + }); +} + +function hintForPrefixes(filters) { + return filters.prefixes.map(prefix => ` no entry cites any ${prefix} ID.`); +} + +function hintForOrigins(filters) { + return filters.origins.map(origin => + origin === 'note' + ? ' the notes tree carries only what an implementation found that ' + + 'overrides a harvested rule; it is meant to be small.' + : ' no harvested entry matches. harvested/ is the whole corpus bar the ' + + 'notes, so this is a filter combination, not an empty tree.', + ); +} + +function hintForTopics(filters, entries) { + const known = [...new Set(entries.map(entry => entry.topic))].sort(); + return filters.topics.map( + topic => + ` no topic file matches '${topic}'. available: ${known.join(' ')}`, + ); +} + +function hintForRoles(filters) { + return filters.roles.map(role => + role === 'review' + ? " no review-role entry matches. review is the notes tree's role; " + + 'under harvested/ it is a structural violation, so there are none.' + : ` no entry carries the role ${role}.`, + ); +} + +function hintForSections(filters) { + return filters.sections.map( + section => + ` the ${section} section holds no entry in either tree` + + (section === 'Superseded' + ? ' — it exists only under notes/, where an override is recorded.' + : '.'), + ); +} + +function hintForChapters(filters, entries) { + const known = [...new Set(entries.flatMap(entry => chaptersOf(entry)))].sort( + (a, b) => Number(a) - Number(b), + ); + return filters.chapters.map( + chapter => + ` no entry cites styleguide chapter ${chapter}. harvested chapters: ` + + known.join(' '), + ); +} + +function hintForPatterns() { + return [ + ' text filters are applied to entry text only; try --grep with a ' + + 'looser pattern, or fewer bare words.', + ]; +} + +const HINTS = { + reqs: hintForReqs, + keys: hintForKeys, + prefixes: hintForPrefixes, + origins: hintForOrigins, + topics: hintForTopics, + roles: hintForRoles, + sections: hintForSections, + chapters: hintForChapters, + patterns: hintForPatterns, +}; + +function topicsForPrefix(entries, prefix) { + const counts = new Map(); + for (const entry of entries) { + for (const id of entry.reqs) { + if (splitId(id)[0] !== prefix) continue; + counts.set(entry.topic, (counts.get(entry.topic) ?? 0) + 1); + } + } + return [...counts.entries()] + .sort((a, b) => b[1] - a[1]) + .slice(0, 3) + .map(([topic]) => topic); +} + +const USAGE = `Usage: bun run knowledge [options] [words...] + +Query docs/knowledge/ — both harvested/ (what the documents say) and notes/ +(what the implementation found; it overrides). Different filters AND together; +values within one filter OR. + + --req <ID> entries citing that requirement ID (repeatable, comma-ok). + Comma form is the whole-task query: --req HTTP-13,HTTP-14 + --key <topic/hex> the one entry with that key — how a note's citation is + resolved. An unknown key is reported, not an error: it + means the entry was reworded and the citation is stale. + --prefix <name> a whole ID family: --prefix HTTP. The audit-scale filter. + --origin <names> ${ORIGINS.join(' | ')} + --topic <names> topic files whose name contains any of these (substring) + --section <names> ${SECTIONS.map(s => s.toLowerCase()).join(' | ')} + --role <names> ${ROLES.join(' | ')} + --chapter <n> styleguide chapter, e.g. 6 (a "6.7" drops the .7) + --grep <regex> case-insensitive regex over entry text (repeatable) + <words...> bare words: case-insensitive substrings, all must match + --brief drop <sub> provenance lines (~30% less output) + --json machine-readable records + --list-topics every topic with entry, distinct-ID and note counts + --list-reqs requirement-ID -> location map (~6k tokens; prefer --coverage) + --coverage substantive vs roll-up-only vs uncited, per prefix + --help + +Every result carries a stable key, <topic>/<8 hex>, digested from the entry +text. Name a rule by that key in a note: it survives a re-order, and it changes +exactly when the rule's text does — including on a re-harvest that rewords it, +which is when the note needs revisiting. Resolve one with --key. A harvested +entry a note overrides prints [overridden by notes/...]. + +An unknown --role, --section, --chapter, --origin or --prefix exits 2: a typo +there is a silent empty result. An unknown --req only warns, because a +not-yet-canonical ID is a legitimate thing to ask about. + +Exits 1 when a query matches nothing. + +Examples: + bun run knowledge --req HTTP-13,HTTP-14,HTTP-15 # one task's whole ID set + bun run knowledge --origin note --brief # start of a phase: what we found + bun run knowledge --prefix HTTP --section rules # an audit group, by ID family + bun run knowledge --chapter 6 interface class # "styleguide 6.7" + bun run knowledge --topic pipeline --section rules --brief cursor fork +`; + +function main(argv) { + const {values, positionals} = parseArgs({ + args: argv, + allowPositionals: true, + options: { + req: {type: 'string', multiple: true}, + key: {type: 'string', multiple: true}, + prefix: {type: 'string', multiple: true}, + origin: {type: 'string', multiple: true}, + topic: {type: 'string', multiple: true}, + section: {type: 'string', multiple: true}, + role: {type: 'string', multiple: true}, + chapter: {type: 'string', multiple: true}, + grep: {type: 'string', multiple: true}, + brief: {type: 'boolean', default: false}, + json: {type: 'boolean', default: false}, + 'list-topics': {type: 'boolean', default: false}, + 'list-reqs': {type: 'boolean', default: false}, + coverage: {type: 'boolean', default: false}, + help: {type: 'boolean', default: false}, + }, + }); + + if (values.help) { + process.stdout.write(USAGE); + return 0; + } + + const canonicalIds = loadCanonicalIds(); + const entries = loadCorpus(derivePrefixes(canonicalIds)); + const index = citationIndex(entries); + + if (values['list-topics']) { + process.stdout.write(`${renderListTopics(entries)}\n`); + return 0; + } + if (values['list-reqs']) { + process.stdout.write(`${renderListReqs(index)}\n`); + return 0; + } + if (values.coverage) { + process.stdout.write(`${renderCoverage(canonicalIds, index)}\n`); + return 0; + } + + const filters = buildFilters(values, positionals, canonicalIds); + if (isEmptyFilter(filters)) { + process.stdout.write(USAGE); + return 0; + } + + const results = entries.filter(entry => matches(entry, filters)); + + if (values.json) { + const annotated = results.map(entry => ({ + ...entry, + rollup: isRollup(entry), + })); + process.stdout.write(`${JSON.stringify(annotated, null, 2)}\n`); + return results.length === 0 ? 1 : 0; + } + + if (results.length === 0) { + process.stdout.write( + `${renderNoMatches(filters, entries, index, canonicalIds)}\n`, + ); + return 1; + } + + process.stdout.write(`${renderEntries(results, values.brief, filters)}\n`); + return 0; +} + +export { + ROLES, + SECTIONS, + loadCanonicalIds, + derivePrefixes, + extractIds, + entryKey, + parseSub, + parseFile, + loadCorpus, + citationIndex, + buildFilters, + matches, + renderCoverage, + renderEntries, + renderListTopics, + renderNoMatches, + isRollup, + chaptersOf, + danglingKeys, + entryLocation, + topicFiles, + compareIds, + main, +}; + +// Only run the CLI when invoked directly, so the test file can import the +// parsing helpers without the process exiting underneath it. +if (process.argv[1] === fileURLToPath(import.meta.url)) { + try { + process.exitCode = main(process.argv.slice(2)); + } catch (error) { + process.stderr.write(`${error.message}\n`); + process.exitCode = 2; + } +} diff --git a/scripts/knowledge.test.mjs b/scripts/knowledge.test.mjs new file mode 100644 index 0000000..c8145bb --- /dev/null +++ b/scripts/knowledge.test.mjs @@ -0,0 +1,790 @@ +// SPDX-License-Identifier: MIT +// scripts/knowledge.test.mjs +// +// Run with `bun run test:scripts` (`node --test 'scripts/*.test.mjs'` — Node +// 26 no longer accepts a bare directory there). Deliberately outside `bun test`, +// which `bunfig.toml` scopes to `packages`: the 80% line-coverage floor is a +// statement about `packages/core`, not about repo tooling. +import assert from 'node:assert/strict'; +import {mkdtempSync, writeFileSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {test} from 'node:test'; + +import { + buildFilters, + citationIndex, + compareIds, + danglingKeys, + derivePrefixes, + entryKey, + entryLocation, + extractIds, + loadCanonicalIds, + loadCorpus, + matches, + parseFile, + parseSub, + renderCoverage, + renderEntries, + renderListTopics, + renderNoMatches, + isRollup, + chaptersOf, + topicFiles, +} from './knowledge.mjs'; + +const canonicalIds = loadCanonicalIds(); +const prefixes = derivePrefixes(canonicalIds); + +function fixture(body) { + const dir = mkdtempSync(join(tmpdir(), 'knowledge-test-')); + const path = join(dir, 'topic.md'); + writeFileSync(path, body); + return path; +} + +// --- canonical IDs and the derived allowlist ------------------------------- + +test('appendix C parses into the full canonical requirement set', () => { + assert.equal(canonicalIds.size, 645); + assert.equal(canonicalIds.get('SEAM-1').level, 'MUST'); + assert.equal(canonicalIds.get('RETRY-12').level, 'SHOULD'); + assert.ok(canonicalIds.has('HTTP-7')); +}); + +test('the prefix allowlist is derived from appendix C, not hardcoded', () => { + assert.equal(prefixes.size, 19); + for (const prefix of ['HTTP', 'SEAM', 'RETRY', 'CTX', 'NFR']) { + assert.ok(prefixes.has(prefix), `${prefix} should be a canonical prefix`); + } +}); + +test('the allowlist rejects the shapes a bare regex false-positives on', () => { + const text = + 'Encode as UTF-8, hash with SHA-256, per RFC-3986 and ISO-8601, see HTTP-7.'; + assert.deepEqual(extractIds(text, prefixes), ['HTTP-7']); + for (const prefix of ['UTF', 'SHA', 'RFC', 'ISO']) { + assert.ok(!prefixes.has(prefix), `${prefix} must not be an ID prefix`); + } +}); + +test('ID extraction is exact-token, so HTTP-7 does not match HTTP-70', () => { + const found = extractIds( + 'Covers HTTP-70 and HTTP-700 but not the short one.', + prefixes, + ); + assert.deepEqual(found, ['HTTP-70', 'HTTP-700']); + assert.ok(!found.includes('HTTP-7')); +}); + +test('ID extraction de-duplicates and preserves first-seen order', () => { + assert.deepEqual( + extractIds('SEAM-29 then HTTP-2 then SEAM-29 again.', prefixes), + ['SEAM-29', 'HTTP-2'], + ); +}); + +// --- entry parsing --------------------------------------------------------- + +test('entries are attributed to the section heading above them', () => { + const path = fixture( + [ + '# topic', + '', + '## Rules', + '- First rule (HTTP-1).', + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc123</sub>', + '', + '## Constraints', + '- A constraint (SEAM-1).', + ' <sub>design · `docs/sdk-design-nodejs/02-package-and-workspace-layout.md:3` · high · sha:def456</sub>', + '', + ].join('\n'), + ); + const entries = parseFile(path, prefixes); + + assert.equal(entries.length, 2); + assert.deepEqual( + entries.map(entry => [entry.section, entry.line, entry.reqs]), + [ + ['Rules', 4, ['HTTP-1']], + ['Constraints', 8, ['SEAM-1']], + ], + ); + assert.equal(entries[0].role, 'spec'); + assert.equal( + entries[0].source, + 'docs/product-spec/04-core-http-domain-model.md:9', + ); + assert.equal(entries[0].confidence, 'high'); + assert.equal(entries[0].sha, 'abc123'); +}); + +test('a multi-paragraph bullet keeps its tail, blank lines included', () => { + const path = fixture( + [ + '## Conflicts', + '- Opening claim.', + '', + ' Continuation paragraph citing PAGE-11 after a blank line.', + ' <sub>review · `docs/superpowers/specs/x.md` · high · sha:manual</sub>', + '', + ].join('\n'), + ); + const [entry] = parseFile(path, prefixes); + + assert.match(entry.text, /Continuation paragraph/); + assert.deepEqual(entry.reqs, ['PAGE-11']); + assert.equal(entry.section, 'Conflicts'); +}); + +test('a two-source Conflicts <sub> yields both role/source pairs', () => { + const parsed = parseSub( + 'design `docs/sdk-design-nodejs/04.md:7-14` · styleguide `/abs/06-classes.md:168-183` · unresolved 2026-07-25', + ); + assert.deepEqual(parsed.roles, ['design', 'styleguide']); + assert.deepEqual(parsed.sources, [ + 'docs/sdk-design-nodejs/04.md:7-14', + '/abs/06-classes.md:168-183', + ]); + assert.equal(parsed.confidence, 'unresolved 2026-07-25'); + assert.equal(parsed.sha, null); +}); + +test('the standard four-field <sub> splits role from source correctly', () => { + const parsed = parseSub( + 'spec · `docs/product-spec/09-retry-and-resilience.md:28` · high · sha:9efbe276001e', + ); + assert.deepEqual(parsed.roles, ['spec']); + assert.equal( + parsed.source, + 'docs/product-spec/09-retry-and-resilience.md:28', + ); + assert.equal(parsed.confidence, 'high'); + assert.equal(parsed.sha, '9efbe276001e'); +}); + +test('the real corpus parses with one <sub> per bullet and no orphans', () => { + const entries = loadCorpus(prefixes); + const harvested = entries.filter(entry => entry.origin === 'harvested'); + assert.equal(harvested.length, 1457); + assert.ok(entries.length > harvested.length, 'the notes tree is non-empty'); + for (const entry of entries) { + assert.ok(entry.subLine, `${entry.file}:${entry.line} lost its <sub> line`); + assert.ok( + entry.sources.length > 0, + `${entry.file}:${entry.line} has no source`, + ); + assert.ok(entry.section, `${entry.file}:${entry.line} has no section`); + } +}); + +// --- filtering ------------------------------------------------------------- + +const sampleEntry = { + file: 'retry-and-resilience.md', + line: 8, + section: 'Rules', + text: 'The retryable-status classifier MUST be single-sourced (RETRY-1).', + roles: ['spec'], + reqs: ['RETRY-1'], +}; + +function filtersFor(values, positionals = []) { + return buildFilters(values, positionals, canonicalIds); +} + +test('--req matches on the exact token only', () => { + assert.ok(matches(sampleEntry, filtersFor({req: ['RETRY-1']}))); + assert.ok(!matches(sampleEntry, filtersFor({req: ['RETRY-10']}))); +}); + +test('filters AND together across dimensions', () => { + const both = filtersFor({req: ['RETRY-1'], section: ['rules']}); + assert.ok(matches(sampleEntry, both)); + + const sectionMiss = filtersFor({req: ['RETRY-1'], section: ['reference']}); + assert.ok(!matches(sampleEntry, sectionMiss)); + + const roleMiss = filtersFor({req: ['RETRY-1'], role: ['styleguide']}); + assert.ok(!matches(sampleEntry, roleMiss)); + + const topicMiss = filtersFor({req: ['RETRY-1'], topic: ['pagination']}); + assert.ok(!matches(sampleEntry, topicMiss)); +}); + +test('multiple values within one filter OR together', () => { + const either = filtersFor({req: ['RETRY-99', 'RETRY-1']}); + assert.ok(matches(sampleEntry, either)); +}); + +test('bare words AND together and are case-insensitive', () => { + assert.ok( + matches(sampleEntry, filtersFor({}, ['CLASSIFIER', 'single-sourced'])), + ); + assert.ok(!matches(sampleEntry, filtersFor({}, ['classifier', 'redirect']))); +}); + +test('an unknown --section name fails loudly rather than matching nothing', () => { + assert.throws( + () => filtersFor({section: ['rulez']}), + /unknown section 'rulez'/, + ); +}); + +// --- reports --------------------------------------------------------------- + +test('a cited ID never appears in the uncited column', () => { + const index = citationIndex(loadCorpus(prefixes)); + const uncited = [...canonicalIds.keys()].filter(id => !index.has(id)); + for (const id of uncited) { + assert.equal(index.get(id), undefined); + } + assert.ok(index.has('RETRY-13'), 'RETRY-13 should be cited after annotation'); +}); + +test('IDs sort by prefix then numerically, not lexically', () => { + const sorted = ['HTTP-70', 'HTTP-7', 'AUTH-2', 'HTTP-100'].sort(compareIds); + assert.deepEqual(sorted, ['AUTH-2', 'HTTP-7', 'HTTP-70', 'HTTP-100']); +}); + +// --- roll-up detection ------------------------------------------------------ + +test('an entry sourced only from appendix B is a roll-up', () => { + const rollup = { + sources: ['docs/product-spec/appendix-b-conformance-test-checklist.md:91'], + }; + const substantive = { + sources: ['docs/product-spec/09-retry-and-resilience.md:28'], + }; + const mixed = {sources: [...rollup.sources, ...substantive.sources]}; + + assert.ok(isRollup(rollup)); + assert.ok(!isRollup(substantive)); + assert.ok(!isRollup(mixed), 'one real source is enough to be substantive'); + assert.ok(!isRollup({sources: []})); +}); + +test('a --req answered only by roll-ups warns instead of exiting quietly', () => { + const entries = loadCorpus(prefixes); + const index = citationIndex(entries); + const hits = index.get('NFR-13'); + + assert.ok(hits.every(isRollup), 'NFR-13 is roll-up-only in this corpus'); + const rendered = renderEntries(hits, false, {reqs: ['NFR-13']}); + assert.match(rendered, /\[appendix-B roll-up\]/); + assert.match(rendered, /WARNING: every result is an appendix-B/); + assert.match(rendered, /NFR-13/); +}); + +test('a substantive result carries no roll-up warning', () => { + const index = citationIndex(loadCorpus(prefixes)); + const rendered = renderEntries(index.get('RETRY-13'), false, { + reqs: ['RETRY-13'], + }); + assert.ok(!rendered.includes('WARNING')); + assert.ok(!rendered.includes('[appendix-B roll-up]')); +}); + +test('coverage separates substantive from roll-up-only from uncited', () => { + const index = citationIndex(loadCorpus(prefixes)); + const report = renderCoverage(canonicalIds, index); + + const rows = report + .split('\n') + .map(line => /^([A-Z][A-Z0-9]*)\t(\d+)\t(\d+)\t(\d+)\t(\d+)\t/.exec(line)) + .filter(Boolean); + assert.equal(rows.length, prefixes.size); + + let total = 0; + for (const [, prefix, sub, rollup, uncited, rowTotal] of rows) { + assert.equal( + Number(sub) + Number(rollup) + Number(uncited), + Number(rowTotal), + `${prefix}: the three buckets must partition the total`, + ); + total += Number(rowTotal); + } + assert.equal(total, canonicalIds.size); + assert.match(report, /\d+\/645 canonical IDs have a substantive entry/); +}); + +// --- styleguide chapters ---------------------------------------------------- + +test('chapters come from styleguide sources only, never spec chapters', () => { + const specOnly = { + roles: ['spec'], + sources: ['docs/product-spec/04-core-http-domain-model.md:22-22'], + }; + assert.deepEqual( + chaptersOf(specOnly), + [], + 'spec chapter 04 is not chapter 4', + ); + + const styleguide = { + roles: ['styleguide'], + sources: [ + '/home/u/styleguide/typescript/06-classes-and-data-modeling.md:168-183', + ], + }; + assert.deepEqual(chaptersOf(styleguide), ['6'], 'leading zero is stripped'); + + const conflict = { + roles: ['design', 'styleguide'], + sources: [ + 'docs/sdk-design-nodejs/09-toolchain-and-quality-gates.md:12', + '/home/u/styleguide/typescript/11-testing.md:47-48', + ], + }; + assert.deepEqual(chaptersOf(conflict), ['11'], 'only the styleguide side'); +}); + +test('--chapter 6 reaches data-modeling, which carries no requirement ID', () => { + const entries = loadCorpus(prefixes); + const filters = filtersFor({chapter: ['6.7']}); + assert.deepEqual( + filters.chapters, + ['6'], + 'the sub-section number is dropped', + ); + + const hits = entries.filter(entry => matches(entry, filters)); + assert.ok(hits.length > 0); + assert.ok(hits.some(entry => entry.file === 'data-modeling.md')); + assert.ok( + hits.every( + entry => entry.reqs.length === 0 || entry.roles.includes('styleguide'), + ), + ); +}); + +test('an unknown chapter or role fails loudly, like an unknown section', () => { + assert.throws(() => filtersFor({chapter: ['six']}), /unknown chapter 'six'/); + assert.throws(() => filtersFor({role: ['spek']}), /unknown role 'spek'/); +}); + +// --- topic listing ---------------------------------------------------------- + +test('--list-topics covers every topic file in both trees', () => { + const entries = loadCorpus(prefixes); + const report = renderListTopics(entries); + + const files = topicFiles(); + assert.equal( + files.filter(file => file.origin === 'harvested').length, + 38, + 'the harvested corpus is 38 topic files, the register having been dropped', + ); + assert.ok( + files.some(file => file.origin === 'note'), + 'the notes tree is discovered too', + ); + for (const {topic} of files) { + assert.ok(report.includes(topic), `${topic} missing from --list-topics`); + } + const topics = new Set(files.map(file => file.topic)); + const harvested = new Set( + files.filter(file => file.origin === 'harvested').map(file => file.topic), + ); + assert.match( + report, + new RegExp( + `^${topics.size} topics, ${harvested.size} of them harvested\\.`, + 'm', + ), + 'the topic count is the union of both trees — deliberate-deviations is note-only', + ); + // The prose count must agree with the table it summarises. This half tests the renderer, and it + // holds whatever the corpus says. The count is scoped to harvested topics: a note-only topic is + // not styleguide-derived, so its row (entries 0) is not an ID-less harvested topic. + const stated = Number( + /(\d+) harvested topics carry no requirement ID at all/.exec(report)?.[1], + ); + const zeroIdRows = report + .split('\n') + .filter(line => /^\S+\t[1-9]\d*\t0\t\d+$/.test(line)).length; + assert.equal( + stated, + zeroIdRows, + "--list-topics' summary line disagrees with its own table", + ); + + // Corpus-shape canary, hardcoded on purpose. It fires when a topic that carried no requirement ID + // gains its first one, which is a real event rather than noise: ID-less topics are reachable only + // via `--topic`/`--chapter`, so the count is quoted to readers in two documents outside this file. + // When it fires, confirm the corpus edit was intended, then move this assertion together with + // CLAUDE.md's "Querying `docs/knowledge/`" section and `.claude/skills/knowledge-lookup/SKILL.md`. + // Last moved 16 -> 15 by 36c3f96 (PR #59), whose Phase 10 correction to + // `docs/knowledge/deliberate-deviations.md` cites CFG-1. + assert.equal( + stated, + 15, + 'ID-less topic count changed — update CLAUDE.md and knowledge-lookup/SKILL.md with it', + ); + assert.match(report, /carry a hand-written note/); +}); + +test('--coverage pins the substantive / roll-up / uncited split the docs quote', () => { + // Same canary, for the other three numbers in CLAUDE.md's "Querying `docs/knowledge/`" paragraph. + // Those had drifted by one apiece and nothing noticed, because the only assertion in this file + // covering that sentence was the topic count above. A count quoted to a reader and re-verified by + // nothing is how the corpus and its documentation part company. + const report = renderCoverage( + canonicalIds, + citationIndex(loadCorpus(prefixes)), + ); + const numbers = + /(\d+)\/(\d+) canonical IDs have a substantive entry\. (\d+) more are named only by an appendix-B conformance roll-up[^.]*\. (\d+) are cited nowhere/.exec( + report, + ); + assert.ok( + numbers, + `--coverage summary line not found in:\n${report.slice(-400)}`, + ); + const [, substantive, total, rollup, uncited] = numbers.map(Number); + assert.equal( + total, + canonicalIds.size, + 'appendix C and --coverage disagree on the ID total', + ); + assert.equal( + substantive + rollup + uncited, + total, + 'the three buckets do not account for every canonical ID', + ); + assert.deepEqual( + {substantive, rollup, uncited}, + {substantive: 385, rollup: 256, uncited: 4}, + 'corpus coverage changed — update CLAUDE.md and knowledge-lookup/SKILL.md with the new numbers', + ); +}); + +test('a note and its harvested topic share a name but not a tree', () => { + const files = topicFiles(); + const pagination = files.filter(file => file.topic === 'pagination'); + assert.deepEqual( + pagination.map(file => file.origin).sort(), + ['harvested', 'note'], + 'pagination.md exists in both trees', + ); + assert.ok(pagination.every(file => file.path.endsWith('pagination.md'))); +}); + +test('an entry location names the tree, so the two are never confused', () => { + const entries = loadCorpus(prefixes); + const note = entries.find(entry => entry.origin === 'note'); + const harvested = entries.find(entry => entry.origin === 'harvested'); + assert.match(entryLocation(note), /^notes\/[a-z-]+\.md:\d+$/); + assert.match(entryLocation(harvested), /^[a-z-]+\.md:\d+$/); +}); + +// --- the two trees ---------------------------------------------------------- + +test('an entry records which tree it came from', () => { + const body = [ + '## Superseded', + '- A hand-written note.', + ' <sub>review · `docs/superpowers/specs/x.md` · high · sha:manual-note</sub>', + '', + ].join('\n'); + + const [harvested] = parseFile(fixture(body), prefixes, 'harvested'); + const [note] = parseFile(fixture(body), prefixes, 'note'); + + assert.equal(harvested.origin, 'harvested'); + assert.equal(note.origin, 'note'); +}); + +test('--origin selects one tree and rejects an unknown name', () => { + const harvested = {origin: 'harvested', roles: ['spec'], reqs: []}; + const note = {origin: 'note', roles: ['review'], reqs: []}; + + assert.ok(matches(note, filtersFor({origin: ['note']}))); + assert.ok(!matches(harvested, filtersFor({origin: ['note']}))); + assert.ok(matches(harvested, filtersFor({origin: ['harvested']}))); + assert.throws( + () => filtersFor({origin: ['notes']}), + /unknown origin 'notes'/, + ); +}); + +test('the corpus is both trees, and only notes carry the review role', () => { + const entries = loadCorpus(prefixes); + const notes = entries.filter(entry => entry.origin === 'note'); + + assert.ok(notes.length > 0, 'the notes tree should hold entries'); + assert.ok( + notes.every(entry => entry.roles.includes('review')), + 'every note is a review-role entry', + ); + assert.ok( + entries + .filter(entry => entry.origin === 'harvested') + .every(entry => !entry.roles.includes('review')), + 'no harvested entry carries the review role', + ); +}); + +// --- --prefix --------------------------------------------------------------- + +test('--prefix selects a whole requirement family', () => { + const http = {reqs: ['HTTP-7'], roles: ['spec']}; + const retry = {reqs: ['RETRY-1'], roles: ['spec']}; + const idless = {reqs: [], roles: ['styleguide']}; + + assert.ok(matches(http, filtersFor({prefix: ['HTTP']}))); + assert.ok(!matches(retry, filtersFor({prefix: ['HTTP']}))); + assert.ok(!matches(idless, filtersFor({prefix: ['HTTP']}))); + assert.ok(matches(retry, filtersFor({prefix: ['HTTP,RETRY']}))); +}); + +test('--prefix rejects a name appendix C does not define', () => { + assert.throws( + () => filtersFor({prefix: ['UTF']}), + /'UTF' is not a requirement-ID prefix in appendix C/, + ); +}); + +test('--prefix beats a --req list at reaching a whole family', () => { + const entries = loadCorpus(prefixes); + const hits = entries.filter(entry => + matches(entry, filtersFor({prefix: ['PAGE']})), + ); + assert.ok(hits.length > 0); + assert.ok( + hits.every(entry => entry.reqs.some(id => id.startsWith('PAGE-'))), + 'every hit cites a PAGE id', + ); +}); + +// --- the stable entry key --------------------------------------------------- + +test('an entry key is <topic>/<8 hex> derived from the entry text alone', () => { + const [entry] = parseFile( + fixture( + [ + '## Rules', + '- A rule about HTTP-1.', + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc123</sub>', + '', + ].join('\n'), + ), + prefixes, + 'harvested', + ); + + assert.match(entry.key, /^topic\/[0-9a-f]{8}$/); + assert.equal(entry.key, entryKey('topic', 'A rule about HTTP-1.')); +}); + +test('an entry key survives a re-order but not a re-wording', () => { + const rule = '- A rule about HTTP-1.'; + const sub = + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc123</sub>'; + + const first = parseFile( + fixture(['## Rules', rule, sub, '- Another rule.', sub, ''].join('\n')), + prefixes, + 'harvested', + ); + const moved = parseFile( + fixture(['## Rules', '- Another rule.', sub, rule, sub, ''].join('\n')), + prefixes, + 'harvested', + ); + + assert.notEqual(first[0].line, moved[1].line, 'the entry did move'); + assert.equal(first[0].key, moved[1].key, 'the key does not follow the line'); + assert.notEqual( + first[0].key, + first[1].key, + 'a different rule, a different key', + ); + assert.notEqual( + entryKey('topic', 'A rule about HTTP-1.'), + entryKey('topic', 'A rule about HTTP-2.'), + ); +}); + +test('the key ignores trailing whitespace and the topic scopes it', () => { + assert.equal(entryKey('t', 'A rule. '), entryKey('t', 'A rule.')); + assert.notEqual(entryKey('a', 'A rule.'), entryKey('b', 'A rule.')); +}); + +// --- parser robustness ------------------------------------------------------ + +test('a CRLF topic file parses, rather than silently yielding nothing', () => { + const body = [ + '## Rules', + '- A rule about HTTP-1.', + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc123</sub>', + '', + ].join('\r\n'); + const entries = parseFile(fixture(body), prefixes, 'harvested'); + + assert.equal(entries.length, 1, 'CRLF must not empty the file'); + assert.equal(entries[0].section, 'Rules'); + assert.equal(entries[0].sources.length, 1); + assert.equal(entries[0].text, 'A rule about HTTP-1.'); +}); + +test('a BOM does not orphan every entry from its section', () => { + const body = + '' + + [ + '## Rules', + '- A rule.', + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc</sub>', + '', + ].join('\n'); + const [entry] = parseFile(fixture(body), prefixes, 'harvested'); + assert.equal(entry.section, 'Rules'); +}); + +test('a second <sub> adds its sources instead of hiding the first', () => { + const [entry] = parseFile( + fixture( + [ + '## Rules', + '- A rule.', + ' <sub>review · `docs/superpowers/specs/x.md` · high · sha:manual</sub>', + ' <sub>spec · `docs/product-spec/04-core-http-domain-model.md:9` · high · sha:abc</sub>', + '', + ].join('\n'), + ), + prefixes, + 'harvested', + ); + assert.deepEqual(entry.roles, ['review', 'spec']); + assert.equal(entry.sources.length, 2); +}); + +test('an unreadable topic file names itself in the error', () => { + assert.throws( + () => parseFile(join(tmpdir(), 'knowledge-test-nonexistent.md'), prefixes), + /cannot read the topic file .*knowledge-test-nonexistent\.md/, + ); +}); + +// --- empty filter values ---------------------------------------------------- + +test('an empty filter value is rejected, not treated as "match everything"', () => { + assert.throws( + () => filtersFor({topic: ['']}), + /--topic was given only empty/, + ); + assert.throws(() => filtersFor({}, ['']), /--grep was given only empty/); + assert.throws(() => filtersFor({req: [',']}), /--req was given only empty/); +}); + +test('a trailing comma is dropped, not turned into a whole-corpus query', () => { + const filters = filtersFor({topic: ['pipeline,']}); + assert.deepEqual(filters.topics, ['pipeline']); +}); + +// --- --key and note overrides ---------------------------------------------- + +test('--key selects the single entry with that key', () => { + const entries = loadCorpus(prefixes); + const target = entries.find(entry => entry.origin === 'harvested'); + const hits = entries.filter(entry => + matches(entry, filtersFor({key: [target.key]})), + ); + assert.deepEqual( + hits.map(entry => entry.key), + [target.key], + ); +}); + +test('--key rejects anything that is not <topic>/<8 hex>', () => { + assert.throws(() => filtersFor({key: ['pagination']}), /is not an entry key/); + assert.throws( + () => filtersFor({key: ['pagination/xyz']}), + /is not an entry key/, + ); +}); + +test('a note links to the harvested entry it names, in both directions', () => { + const entries = loadCorpus(prefixes); + const note = entries.find( + entry => entry.origin === 'note' && entry.section === 'Superseded', + ); + assert.ok(note.overrides.length > 0, 'the note cites at least one key'); + + for (const key of note.overrides) { + const target = entries.find(entry => entry.key === key); + assert.ok(target, `${key} resolves`); + assert.ok( + target.overriddenBy.includes(entryLocation(note)), + 'the harvested entry points back at the note', + ); + } +}); + +test('every key a note cites resolves — a dangling one is a stale note', () => { + assert.deepEqual(danglingKeys(loadCorpus(prefixes)), []); +}); + +test('danglingKeys reports a citation whose entry has been reworded', () => { + const note = { + origin: 'note', + file: 'pagination.md', + line: 8, + text: 'Supersedes `pagination/deadbeef`.', + }; + assert.deepEqual(danglingKeys([note]), [ + {note: 'notes/pagination.md:8', cited: 'pagination/deadbeef'}, + ]); +}); + +test('an overridden harvested entry says so in its rendered header', () => { + const entries = loadCorpus(prefixes); + const overridden = entries.find(entry => entry.overriddenBy.length > 0); + const rendered = renderEntries([overridden], true, {reqs: []}); + assert.match(rendered, /\[overridden by notes\//); +}); + +// --- zero-result diagnosis -------------------------------------------------- + +test('an empty intersection is reported as one, not blamed on a filter', () => { + const entries = loadCorpus(prefixes); + const filters = filtersFor({prefix: ['HTTP'], req: ['PAGE-11']}); + const index = citationIndex(entries); + + assert.equal(entries.filter(entry => matches(entry, filters)).length, 0); + const rendered = renderNoMatches(filters, entries, index, canonicalIds); + assert.match(rendered, /every filter matches something on its own/); + assert.ok( + !rendered.includes('no entry cites it yet'), + 'PAGE-11 is cited; blaming it would be a false statement', + ); +}); + +test('a genuinely uncited ID is still named, and never as its own neighbour', () => { + const entries = loadCorpus(prefixes); + const filters = filtersFor({req: ['PAGE-9999']}); + const rendered = renderNoMatches( + filters, + entries, + citationIndex(entries), + canonicalIds, + ); + assert.match(rendered, /PAGE-9999 is not a canonical requirement ID/); + assert.match(rendered, /nearest cited PAGE IDs/); + assert.ok(!/nearest cited PAGE IDs:.*PAGE-9999/.test(rendered)); +}); + +test('a stale key is diagnosed as a reworded rule, not as a typo', () => { + const entries = loadCorpus(prefixes); + const filters = filtersFor({key: ['pagination/deadbeef']}); + const rendered = renderNoMatches( + filters, + entries, + citationIndex(entries), + canonicalIds, + ); + assert.match(rendered, /no entry carries the key pagination\/deadbeef/); + assert.match(rendered, /a key digests the entry's text/i); +}); diff --git a/scripts/verify-consumer-types.mjs b/scripts/verify-consumer-types.mjs new file mode 100644 index 0000000..48a10ed --- /dev/null +++ b/scripts/verify-consumer-types.mjs @@ -0,0 +1,610 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-consumer-types.mjs +// +// Compiles a throwaway consumer against the BUILT `.d.ts` using the same `lib` and `target` this +// workspace declares, with `types: []` so nothing from devDependencies leaks in. +// +// This gate exists because a real defect got all the way through every other one. `Response` shipped +// an `async [Symbol.asyncDispose]()` that type-checked in-repo only because `@types/bun` — a +// dev-only global — supplies the symbol. A consumer on `lib: ["ES2022", "DOM"]`, which is what this +// workspace itself declares, got `TS2550: Property 'asyncDispose' does not exist on type +// 'SymbolConstructor'` and could not build at all. `typecheck` passed (dev types present), `build` +// passed, `api` passed, `lint:publish` passed (publint and attw check resolution and export shape, +// not whether the declarations resolve), and `verify:dual-consumption` passed because it runs `node`, +// not `tsc`. +// +// The `lib`/`target` are read from tsconfig.base.json rather than hardcoded, so the gate tracks the +// declared baseline instead of drifting away from it. +import assert from 'node:assert/strict'; +import {execFileSync} from 'node:child_process'; +import { + existsSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join, resolve} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +const repoRoot = resolve(fileURLToPath(new URL('..', import.meta.url))); +const base = JSON.parse( + readFileSync(join(repoRoot, 'tsconfig.base.json'), 'utf8'), +); +const {lib, target} = base.compilerOptions; +assert.ok( + Array.isArray(lib) && lib.length > 0, + 'tsconfig.base.json must declare a lib array', +); + +const built = join(repoRoot, 'packages', 'core', 'dist', 'index.js'); +const builtCodecJson = join( + repoRoot, + 'packages', + 'codec-json', + 'dist', + 'index.js', +); +const builtLoggingPino = join( + repoRoot, + 'packages', + 'logging-pino', + 'dist', + 'index.js', +); +const builtLoggingDebug = join( + repoRoot, + 'packages', + 'logging-debug', + 'dist', + 'index.js', +); +const builtBodyFile = join( + repoRoot, + 'packages', + 'body-file', + 'dist', + 'index.js', +); +const builtTransportShared = join( + repoRoot, + 'packages', + 'transport-shared', + 'dist', + 'index.js', +); +const builtTransportFetch = join( + repoRoot, + 'packages', + 'transport-fetch', + 'dist', + 'index.js', +); +const builtTransportUndici = join( + repoRoot, + 'packages', + 'transport-undici', + 'dist', + 'index.js', +); +const builtRx = join(repoRoot, 'packages', 'rx', 'dist', 'index.js'); +const tsc = join(repoRoot, 'node_modules', '.bin', 'tsc'); + +// Checked up front, not left to the catch below. A missing prerequisite reported through the +// type-failure path would read as "the published .d.ts is broken", which is the one message this +// gate must never send falsely. +assert.ok( + existsSync(tsc), + `tsc not found at ${tsc} — run \`bun install\` before this gate`, +); +for (const artifact of [ + built, + builtCodecJson, + builtLoggingPino, + builtLoggingDebug, + builtBodyFile, + builtTransportShared, + builtTransportFetch, + builtTransportUndici, + builtRx, +]) { + assert.ok( + existsSync(artifact), + `built package not found at ${artifact} — run \`bun run build\` before this gate`, + ); +} +const workDir = mkdtempSync(join(tmpdir(), 'dexpace-consumer-types-')); + +// Exercises the surface most likely to reference a declaration the consumer's lib cannot resolve: +// the resource-owning class, an async iterable/stream type, a generic, and a factory. +// +// It ALSO names every type the pillar-authoring surface promoted in Phase 5c, because a second defect +// got through every other gate too: an `@internal` token inside a prose comment above the barrel's +// context-family export made `stripInternal` delete that export from the emitted `.d.ts`. `typecheck` +// passed (the source says it is exported), `build` passed (tsc emitted happily), and `api:ci` passed +// because api-extractor recorded the resulting `ae-forgotten-export` as report TEXT. Nothing compiled +// the promoted names from outside the package, so nothing noticed. Naming them here is what makes +// that class of silent elision loud. +const consumer = ` +import { + absent, + ApiKeyCredential, + type ApiKeyCredentialConfig, + type AuthCredentialSet, + type AuthDescriptor, + type AuthRequirement, + authRequirementsEqual, + AuthResolutionError, + type AuthScheme, + authStep, + type AuthStepSettings, + type AuthTiers, + type BackoffSettings, + BasicCredential, + type BearerCredential, + BearerToken, + bearerTokensEqual, + type Body, + byteArrayBody, + type ChallengeHook, + type Clock, + createAuthDescriptor, + createAuthRequirement, + createBearerToken, + decodeResponse, + decodeSuccessResponse, + type DecodeTarget, + DeserializationError, + type DeserializationErrorOptions, + type Deserializer, + type DigestAlgorithm, + DigestCredential, + type DispatchContext, + type ExchangeContext, + type ExecutionContext, + foldTristate, + type InstrumentationBundle, + isAbsent, + isNull, + isPresent, + isSerdeError, + isTristate, + materialize, + NameKeyCredential, + type Next, + nullValue, + ofNullable, + PILLAR_STAGES, + PipelineBuilder, + PlaintextCredentialError, + present, + type RedirectCondition, + type RedirectPredicate, + type RedirectSettings, + redirectStep, + Request, + type RequestContext, + Response, + type RetrySettings, + retryStep, + type RetryStepOptions, + Runtime, + type Schema, + type Serde, + serdeBody, + type SerdeErrorOptions, + SerializationError, + type Serializer, + type Stage, + STAGE_ORDER, + standardResilience, + type StandardResilienceOptions, + Status, + type Step, + type StepContext, + type StepDescriptor, + toHttpError, + type TokenProvider, + type Transport, + type Tristate, + TRISTATE_BRAND, + type TristateBranches, + tristateToString, + TypedResponse, + valueOrNull, + type Counter, + type Histogram, + type Meter, + NOOP_METER, + type CreateLoggerOptions, + type LogEvent, + type LogLevel, + type Logger, + NOOP_LOGGER, + createLogger, + getGlobalLogger, + setGlobalLogger, + type Scope, + type Span, + type SpanContext, + type Tracer, + NOOP_SPAN, + NOOP_TRACER, + activateSpan, + activateSpanForCorrelation, + createInstrumentationBundle, + getActiveSpan, + type DroppedHeaderPolicy, + type LoggingGranularity, + type LoggingStepSettings, + LOGGING_STEP_TYPE, + loggingStep, + IoError, + TransportFailureError, + type FileBodyDescriptor, +} from ${JSON.stringify(built)}; +import { + jsonSerde, + type JsonSerdeOptions, + tristate, + tristateObject, + tristateReplacer, +} from ${JSON.stringify(builtCodecJson)}; +import { + createPinoLogger, + type PinoLike, +} from ${JSON.stringify(builtLoggingPino)}; +import { + createDebugLogger, + type DebugLike, + type DebugFactory, +} from ${JSON.stringify(builtLoggingDebug)}; +import { + fileBody, + type FileBodyOptions, +} from ${JSON.stringify(builtBodyFile)}; +import { + fetchTransport, + type FetchTransportOptions, +} from ${JSON.stringify(builtTransportFetch)}; +import { + undiciTransport, + type UndiciTransportOptions, +} from ${JSON.stringify(builtTransportUndici)}; +import { + pageItems$, + pages$, + sseEvents$, + typedSse$, +} from ${JSON.stringify(builtRx)}; +import type {Paginator, SseMapper, SseStream} from ${JSON.stringify(built)}; + + +export function readBody(response: Response): Promise<string> { + return response.text(); +} +export function release(response: Response): Promise<void> { + return response.close(); +} +export function stream(response: Response): ReadableStream<Uint8Array> | null { + return response.body; +} +export function replay(body: Body): Promise<Body> { + return materialize(body); +} +export function typed(wrapper: TypedResponse<number>): Promise<number> { + return wrapper.value(); +} +export const bytes: Body = byteArrayBody(new Uint8Array([1]), 'application/octet-stream'); +export const errorOf = toHttpError; +export const ok: number = Status.of(200).code; + +// --- the pillar-authoring surface promoted in Phase 5c --- +export function kindOf(context: ExecutionContext): string { + return context.kind; +} +export function dispatchKey(context: DispatchContext): symbol { + return context.key; +} +export function requestOf(context: RequestContext): Request { + return context.request; +} +export function responseOf(context: ExchangeContext): Response { + return context.response; +} +export function traceOf(bundle: InstrumentationBundle): string { + return bundle.traceId; +} +export const customStep: Step = async (request, ctx: StepContext) => { + const advance: Next = ctx.fork?.() ?? ctx.next; + kindOf(ctx.context); + return advance(request); +}; +export const descriptor: StepDescriptor = { + type: Symbol('consumer.custom'), + stage: 'PRE_AUTH' satisfies Stage, + fn: customStep, +}; +export const stageCount: number = STAGE_ORDER.length + PILLAR_STAGES.size; + +export function assemble(transport: Transport): Runtime { + const provider: TokenProvider = async () => createBearerToken('t', Date.now() + 60_000); + const settings: AuthStepSettings = { + credentials: { + apiKey: {credential: new ApiKeyCredential('k'), prefix: 'ApiKey'}, + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p', ['SHA-256' satisfies DigestAlgorithm]), + bearer: {provider, marginMs: 5_000}, + }, + tiers: { + client: createAuthDescriptor([ + createAuthRequirement('OAUTH2' satisfies AuthScheme, ['scope.read']), + createAuthRequirement('NO_AUTH'), + ]), + } satisfies AuthTiers, + challengeHook: (async () => undefined) satisfies ChallengeHook, + bearerMarginMs: 30_000, + clock: {now: () => Date.now()}, + }; + const options: StandardResilienceOptions = { + auth: settings, + retry: {settings: {maxAttempts: 3}} satisfies RetryStepOptions, + redirect: {maxHops: 2}, + }; + const hand = new PipelineBuilder(transport) + .append(redirectStep({maxHops: 2})) + .append(retryStep()) + .append(authStep(settings)) + .append(descriptor) + .build(); + const seeded = PipelineBuilder.seedFrom(hand, 'nest').build(); + return standardResilience(seeded, options); +} + +export function requirementEquality(a: AuthRequirement, b: AuthRequirement): boolean { + return authRequirementsEqual(a, b); +} +export function tokenEquality(a: BearerToken, b: BearerToken): boolean { + return bearerTokensEqual(a, b); +} +export function describeDescriptor(d: AuthDescriptor): boolean { + return d.allowsAnonymous; +} +export function nameKey(): NameKeyCredential { + return new NameKeyCredential('x-api-key', 'k'); +} +export function narrow(error: unknown): string | undefined { + if (error instanceof PlaintextCredentialError) return error.scheme; + if (error instanceof AuthResolutionError) return error.requiredSchemes?.[0]; + return undefined; +} +export function credentialSet(set: AuthCredentialSet): string | undefined { + return set.basic?.username; +} +export function bearerCredential(c: BearerCredential): TokenProvider { + return c.provider; +} +export function digestCredential(c: DigestCredential): string { + return c.username; +} +export function apiKeyConfig(c: ApiKeyCredentialConfig): string | undefined { + return c.headerName; +} +export function redirectPolicy(s: RedirectSettings, p: RedirectPredicate): boolean { + return p({response: undefined as unknown as Response, redirectsFollowed: s.maxHops, visited: new Set()}); +} +export function retryPolicy(s: RetrySettings, b: BackoffSettings): number { + return s.maxAttempts + b.initialDelayMs; +} +export function clockNow(c: Clock): number { + return c.now(); +} +export function conditionOf(c: RedirectCondition): number { + return c.redirectsFollowed; +} + +// --- the serde seam promoted in Phase 6a --- +export function mediaTypeOf(serde: Serde): string { + return serde.mediaType; +} +export function encodeInto(s: Serializer, value: unknown, buf: Uint8Array): number { + return s.serializeInto(value, buf, 0); +} +export function decodeOne<T>(d: Deserializer, data: Uint8Array, schema: Schema<T>): T { + return d.deserialize(data, {schema, typeName: 'T'}); +} +export function decodeTarget<T>(schema: Schema<T>): DecodeTarget<T> { + return {schema, typeName: 'T'}; +} +export function decodeBoth<T>( + response: Response, + d: Deserializer, + target: DecodeTarget<T>, +): [Promise<T>, Promise<T>] { + return [decodeResponse(response, d, target), decodeSuccessResponse(response, d, target)]; +} +export function bodyFromSerde(serde: Serde): Body { + return serdeBody({a: 1}, serde, 'application/merge-patch+json'); +} +export function serdeErrorContext(e: unknown): number | undefined { + // Direction is narrowed first: response context lives on the read leaf, not on the union. + if (!isSerdeError(e)) return undefined; + return e instanceof DeserializationError ? e.status : undefined; +} +export function newSerdeErrors( + write: SerdeErrorOptions, + read: DeserializationErrorOptions, +): [SerializationError, DeserializationError] { + return [new SerializationError('w', write), new DeserializationError('r', read)]; +} +export function readLeafContext(e: DeserializationError): [number | undefined, string | null] { + return [e.status, e.etag]; +} +export function tristateBranches<T>(t: Tristate<T>): string { + const branches: TristateBranches<T, string> = { + onAbsent: () => 'absent', + onNull: () => 'null', + onPresent: (value) => tristateToString(present<T>(value as NonNullable<T>)), + }; + return foldTristate(t, branches); +} +export function tristateStates(): readonly [Tristate<number>, Tristate<number>, Tristate<number>] { + return [absent(), nullValue(), present(1)]; +} +export function tristateNarrowing(t: Tristate<string>): string | null { + if (isAbsent(t) || isNull(t)) return valueOrNull(t); + if (isPresent(t)) return t.value; + return null; +} +export function tristateFromNullable(v: string | null): Tristate<string> { + return ofNullable(v); +} +export function brandedRecognition(v: unknown): boolean { + return isTristate(v) && TRISTATE_BRAND in (v as object); +} + +// --- @dexpace/codec-json, the workspace's second publishable package --- +export function jsonBundle(options: JsonSerdeOptions): Serde { + return jsonSerde(options); +} +export function defaultJsonBundle(): Serde { + return jsonSerde(); +} +export function tristateField(inner: Schema<number>): Schema<Tristate<number>> { + return tristate(inner); +} +export function tristateShape(inner: Schema<number>): Tristate<number> { + return tristateObject({age: inner}).parse({}).age; +} +export function replacerRoundTrip(value: unknown): string { + return JSON.stringify(value, tristateReplacer); +} + +// --- Phase 7b Observability and Logging --- +export function loggingSeam(logger: Logger, meter: Meter, tracer: Tracer): void { + const event: LogEvent = logger.atLevel('info' satisfies LogLevel); + event.event('test').field('k', 'v').cause(new Error('err')).emit(); + const derived = logger.withContext({global: 'val'}); + setGlobalLogger(derived); + getGlobalLogger(); + NOOP_LOGGER.atLevel('verbose').emit(); + + const c: Counter = meter.createCounter('c', {unit: '{req}'}); + c.add(1, {k: 'v'}); + const h: Histogram = meter.createHistogram('h', {description: 'd'}); + h.record(1.5); + NOOP_METER.createCounter('c').add(1); + + const span: Span = tracer.startSpan('op'); + const scope: Scope = activateSpan(span); + scope.close(); + const correlatedScope = activateSpanForCorrelation(span); + correlatedScope.close(); + getActiveSpan(); + NOOP_TRACER.startSpan('op').end(); + const bundle = createInstrumentationBundle(() => tracer); + traceOf(bundle); + const spanCtx: SpanContext | undefined = span.spanContext?.(); + void spanCtx; + + const stepOpts: LoggingStepSettings = { + logger, + meter, + severity: 'info', + granularity: 'body' satisfies LoggingGranularity, + previewSizeBytes: 4096, + tracerFactory: () => tracer, + droppedHeaderPolicy: 'mark', + }; + const step = loggingStep(stepOpts); + void step; + void LOGGING_STEP_TYPE; +} + +export function bridgeAdapters(pino: PinoLike, debug: DebugLike, debugFactory: DebugFactory): [Logger, Logger] { + return [createPinoLogger(pino), createDebugLogger(debugFactory, 'custom')]; +} + +// Every symbol Phase 8a promotes, referenced from a consumer's own .d.ts on the declared lib with +// types: []. @dexpace/transport-shared is deliberately absent: its exports are @internal and no +// consumer is meant to import them, so only its build artifact's existence is asserted above. +export function transportErrors(failure: TransportFailureError, io: IoError): string[] { + return [failure.name, failure.message, io.name]; +} + +export function transportAdapters( + descriptor: FileBodyDescriptor, + fileOptions: FileBodyOptions, + fetchOptions: FetchTransportOptions, +): [Transport, FileBodyDescriptor] { + return [fetchTransport(fetchOptions), fileBody(descriptor.path, fileOptions)]; +} + +export function undiciAdapter(options: UndiciTransportOptions): Transport { + return undiciTransport(options); +} + +export function rxBridge( + stream: SseStream, + mapper: SseMapper<number>, + paginator: Paginator<string>, +): void { + const _e = sseEvents$(stream); + const _t = typedSse$(stream, mapper); + const _i = pageItems$(paginator); + const _p = pages$(paginator); + void _e; + void _t; + void _i; + void _p; +} +`; + +const tsconfig = { + compilerOptions: { + target, + lib, + module: 'nodenext', + moduleResolution: 'nodenext', + strict: true, + noEmit: true, + // The whole point: no ambient globals from devDependencies. A consumer installing this package + // gets exactly `lib` plus whatever they install themselves. + types: [], + skipLibCheck: false, + }, + include: ['consumer.ts'], +}; + +try { + writeFileSync(join(workDir, 'consumer.ts'), consumer); + writeFileSync( + join(workDir, 'tsconfig.json'), + JSON.stringify(tsconfig, null, 2), + ); + + execFileSync(tsc, ['-p', join(workDir, 'tsconfig.json')], { + stdio: 'pipe', + encoding: 'utf8', + }); +} catch (error) { + const detail = `${error.stdout ?? ''}${error.stderr ?? ''}`.trim(); + console.error( + "consumer-types check FAILED: the published .d.ts does not compile against this workspace's\n" + + `own declared lib (${lib.join(', ')}) with types: [].\n\n${detail}\n\n` + + 'Either a declaration is reaching for a global that only a devDependency supplies (drop it, or\n' + + 'add the lib entry to tsconfig.base.json and raise engines.node to a runtime that has it), or\n' + + 'the barrel claims an export the emitted .d.ts does not actually carry -- check for an\n' + + '`@internal` token inside a comment above the export, which `stripInternal` deletes it for.', + ); + process.exit(1); +} finally { + rmSync(workDir, {recursive: true, force: true}); +} + +console.log( + `consumer-types check passed: dist/*.d.ts compiles on lib [${lib.join(', ')}] with types: [],\n` + + 'including every symbol the pillar-authoring surface and the Phase 6a serde seam promote, plus\n' + + "@dexpace/codec-json's own entry point.", +); diff --git a/scripts/verify-dual-consumption.mjs b/scripts/verify-dual-consumption.mjs index ebf6195..8e5c2c0 100644 --- a/scripts/verify-dual-consumption.mjs +++ b/scripts/verify-dual-consumption.mjs @@ -1,10 +1,155 @@ // SPDX-License-Identifier: MIT // scripts/verify-dual-consumption.mjs +// +// Every publishable package must be importable and executable by plain `node` against its BUILT +// artifact, through its package name and its `exports` map. Generalized from a core-only check in +// Phase 6a, when `@dexpace/codec-json` became the workspace's second package -- a check hard-coded +// to one package silently stops covering the workspace the moment it grows. import assert from 'node:assert/strict'; -import {Status} from '@dexpace/core'; +import { + absent, + Headers, + present, + Protocol, + Request, + Response, + serdeBody, + sseStreamFrom, + Status, +} from '@dexpace/core'; +import {jsonSerde} from '@dexpace/codec-json'; + +import {createPinoLogger} from '@dexpace/logging-pino'; +import {createDebugLogger} from '@dexpace/logging-debug'; +import {fileBody} from '@dexpace/body-file'; +import { + mapOutboundHeaders, + degradeInboundHeaders, +} from '@dexpace/transport-shared'; +import {fetchTransport} from '@dexpace/transport-fetch'; +import {undiciTransport} from '@dexpace/transport-undici'; +import {pageItems$, pages$, sseEvents$, typedSse$} from '@dexpace/rx'; +import {firstValueFrom, toArray} from 'rxjs'; assert.equal(Status.of(200).code, 200); assert.equal(Status.of(200).name, 'OK'); + +// The second package, exercised end to end rather than merely imported: a bundle is built, a value +// round-trips through both halves of the seam, and the Tristate wiring that crosses the package +// boundary is the thing being encoded -- so a broken `exports` map, a missing build, or a +// dual-package brand mismatch all surface here rather than at a consumer's call time. +const serde = jsonSerde(); +assert.equal(serde.mediaType, 'application/json'); + +const encoded = serde.serializer.serializeToString({ + keep: absent(), + set: present('v'), +}); +assert.equal(encoded, '{"set":"v"}'); + +const decoded = serde.deserializer.deserialize( + serde.serializer.serialize({id: 7}), + {schema: {parse: input => input}, typeName: 'Probe'}, +); +assert.deepEqual(decoded, {id: 7}); + +// The core-to-codec direction of the same boundary: core's body factory driving the codec's +// serializer and stamping the codec's own declared media type (SERDE-2). +const body = serdeBody({id: 7}, serde); +assert.equal(body.mediaType, 'application/json'); +assert.equal(body.replayable, true); + +// Exercise logging-pino bridge +const pinoEvents = []; +const fakePino = { + isLevelEnabled: () => true, + error: obj => pinoEvents.push({level: 'error', obj}), + warn: obj => pinoEvents.push({level: 'warn', obj}), + info: obj => pinoEvents.push({level: 'info', obj}), + debug: obj => pinoEvents.push({level: 'debug', obj}), + trace: obj => pinoEvents.push({level: 'trace', obj}), +}; +const pinoLogger = createPinoLogger(fakePino); +pinoLogger.atLevel('info').event('dual.pino').field('k', 'v').emit(); +assert.equal(pinoEvents.length, 1); +assert.equal(pinoEvents[0].obj.event, 'dual.pino'); + +// Exercise logging-debug bridge +const debugEvents = []; +const fakeDebug = Object.assign( + (formatter, ...args) => debugEvents.push(args.join(' ')), + {enabled: true}, +); +const debugLogger = createDebugLogger(fakeDebug); +debugLogger.atLevel('info').event('dual.debug').field('k', 'v').emit(); +assert.equal(debugEvents.length, 1); +assert.ok(debugEvents[0].includes('event=dual.debug')); + +// Exercise body-file +const fb = fileBody('package.json'); +assert.equal(fb.kind, 'file'); +assert.equal(fb.replayable, true); + +// Exercise transport-shared: the outbound drop pass and the lenient inbound copy. +const outbound = mapOutboundHeaders( + Headers.newBuilder().set('Content-Length', '10').set('X-Kept', 'v').build(), + ['content-length'], +); +assert.ok(outbound.dropped.includes('content-length')); +assert.equal(outbound.sent.get('x-kept'), 'v'); +const inbound = degradeInboundHeaders([['Content-Type', 'text/plain']]); +assert.equal(inbound.headers.get('content-type'), 'text/plain'); + +// Exercise both transports far enough to prove the module graph resolved and construction runs -- +// not far enough to need a network. `close()` is the one lifecycle call that is safe with no peer. +// Never index with a bare `Symbol.asyncDispose` here. This gate runs on whatever `node` is on PATH, +// which includes the declared `engines.node` floor of >=20.3 -- and the symbol arrived in 20.4. On the +// floor the computed key is `undefined`, so `transport[Symbol.asyncDispose]` reads the STRING key +// `"undefined"`. That used to resolve to the junk prototype entry left by an unguarded +// `[Symbol.asyncDispose]()` class member, so this assertion passed over a transport that could not be +// disposed; since the guarded install it resolves to `undefined` and the assertion fails outright. +// Branch on the symbol, and assert the junk key's absence on BOTH legs -- the same shape +// `packages/transport-fetch/src/fetch-transport.test.ts` uses. +const asyncDispose = Symbol.asyncDispose; +for (const transport of [fetchTransport(), undiciTransport()]) { + assert.equal(typeof transport.send, 'function'); + if (typeof asyncDispose === 'symbol') { + assert.equal(typeof transport[asyncDispose], 'function'); + await transport[asyncDispose](); + } + assert.ok( + !Object.getOwnPropertyNames(Object.getPrototypeOf(transport)).includes( + 'undefined', + ), + 'transport prototype carries an "undefined" key: [Symbol.asyncDispose] was declared as a plain class member ahead of the floor bump', + ); + await transport.close(); +} +// Exercise @dexpace/rx bridge +const req = Request.newBuilder() + .method('GET') + .url('https://api.test/events') + .build(); +const sseBody = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: hello\n\n')); + controller.close(); + }, +}); +const sseResp = Response.newBuilder() + .request(req) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(sseBody) + .build(); +const sseStream = sseStreamFrom(sseResp); +const events = await firstValueFrom(sseEvents$(sseStream).pipe(toArray())); +assert.equal(events.length, 1); +assert.deepEqual(events[0].data, ['hello']); +assert.equal(typeof typedSse$, 'function'); +assert.equal(typeof pageItems$, 'function'); +assert.equal(typeof pages$, 'function'); + console.log( - 'dual-consumption check passed: plain Node import resolved and executed @dexpace/core', + 'dual-consumption check passed: plain Node import resolved and executed all packages in workspace', ); diff --git a/scripts/verify-import-cycles.mjs b/scripts/verify-import-cycles.mjs new file mode 100644 index 0000000..64f43c2 --- /dev/null +++ b/scripts/verify-import-cycles.mjs @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-import-cycles.mjs +import {readdirSync, readFileSync, statSync} from 'node:fs'; +import {dirname, join, relative, resolve} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +/** + * `docs/knowledge/harvested/module-organization.md:20` treats an import cycle as a bug rather than a style nit, + * and `:22` requires it be gated in CI. Until 2026-09-04 neither `madge --circular` nor + * `eslint-plugin-import/no-cycle` appeared anywhere in this repository, so twelve-plus source folders per + * package relied on review alone (`docs/work/mvp/2026-09-04-open-items-dissolution.md` K12). + * + * Hand-written rather than `madge` for the reason every other gate here is: `verify:seam-1` asserts zero runtime + * dependencies per package and this repo keeps its gates dependency-free, so a gate that needs an install is a + * gate that can be skipped. The graph this walks is small — relative specifiers inside one package's `src/` — + * and the whole traversal is a depth-first search with a colour map. + * + * **Type-only edges count.** `import type {X} from './y.js'` is erased at runtime and cannot deadlock a module + * initialization, but a type cycle is still the design smell the requirement is about, and `verbatimModuleSyntax` + * means the distinction is spelled consistently enough to make excluding them a deliberate choice rather than an + * accident. If a type-only cycle is ever judged acceptable, exclude it here with a stated reason — do not widen + * the whole gate. + * + * Test files are skipped: a `*.test.ts` is a leaf nothing imports, and a test reaching sideways for a fixture is + * not the failure this guards. + */ + +const ROOT = resolve(fileURLToPath(new URL('..', import.meta.url))); +const PACKAGES = join(ROOT, 'packages'); + +/** Matches the specifier of a static import/export, a side-effect import, or a dynamic import. */ +const SPECIFIER_PATTERNS = [ + /(?:^|[;\n])\s*(?:import|export)[\s\S]*?from\s*['"]([^'"]+)['"]/g, + /(?:^|[;\n])\s*import\s*['"]([^'"]+)['"]/g, + /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g, +]; + +function sourceFilesUnder(dir) { + const out = []; + for (const entry of readdirSync(dir)) { + const full = join(dir, entry); + if (statSync(full).isDirectory()) { + out.push(...sourceFilesUnder(full)); + } else if (entry.endsWith('.ts') && !entry.endsWith('.test.ts')) { + out.push(full); + } + } + return out; +} + +/** + * The relative specifiers `file` imports, resolved to absolute `.ts` paths. + * + * ESM-only/NodeNext means every relative specifier carries a `.js` extension even in `.ts` source, so the + * mapping back is `.js` → `.ts`. A specifier that does not resolve to a file on disk is skipped rather than + * reported: `tsc` already fails on an unresolvable import, and duplicating that check here would make this gate + * fail for a reason that has nothing to do with cycles. + */ +function edgesFrom(file) { + const text = readFileSync(file, 'utf8'); + const edges = new Set(); + for (const pattern of SPECIFIER_PATTERNS) { + pattern.lastIndex = 0; + let match; + while ((match = pattern.exec(text)) !== null) { + const specifier = match[1]; + if (!specifier.startsWith('.')) continue; + const target = resolve(dirname(file), specifier.replace(/\.js$/, '.ts')); + try { + if (statSync(target).isFile()) edges.add(target); + } catch { + // Unresolvable here is `tsc`'s to report, not this gate's. See the doc comment above. + } + } + } + return [...edges]; +} + +/** + * Depth-first search reporting the first cycle it closes, as the list of files that form it with the + * entry point repeated at the end. `null` when the graph is acyclic. + * + * `edgesOf` is injectable so the gate's own suite can drive a synthetic graph without writing files — + * the same shape `verify-sse-37.mjs` uses for its scanner. + * + * @param {string[]} files + * @param {(file: string) => string[]} [edgesOf] + * @returns {string[] | null} + */ +export function findCycle(files, edgesOf = edgesFrom) { + const graph = new Map(files.map(file => [file, edgesOf(file)])); + const state = new Map(); // undefined = unvisited, 1 = on the current path, 2 = finished + const path = []; + + function visit(file) { + state.set(file, 1); + path.push(file); + for (const next of graph.get(file) ?? []) { + if (state.get(next) === 1) + return [...path.slice(path.indexOf(next)), next]; + if (state.get(next) === undefined) { + const found = visit(next); + if (found !== null) return found; + } + } + path.pop(); + state.set(file, 2); + return null; + } + + for (const file of files) { + if (state.get(file) === undefined) { + const found = visit(file); + if (found !== null) return found; + } + } + return null; +} + +const isDirect = + process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]; + +if (isDirect) { + const failures = []; + + let scanned = 0; + for (const pkg of readdirSync(PACKAGES).sort()) { + const src = join(PACKAGES, pkg, 'src'); + try { + if (!statSync(src).isDirectory()) continue; + } catch { + continue; + } + const files = sourceFilesUnder(src); + scanned += files.length; + const cycle = findCycle(files); + if (cycle !== null) { + failures.push( + `@dexpace/${pkg}: import cycle\n ${cycle + .map(file => relative(ROOT, file)) + .join('\n -> ')}`, + ); + } + } + + if (failures.length > 0) { + console.error('verify:import-cycles FAILED\n'); + for (const failure of failures) console.error(` ${failure}\n`); + console.error( + 'An import cycle is a bug, not a style nit (docs/knowledge/harvested/module-organization.md:20).\n' + + 'Break it by moving the shared declaration into a module both sides can import, not by making one\n' + + 'edge type-only -- a type cycle still fails this gate, deliberately.', + ); + process.exit(1); + } + + console.log( + `verify:import-cycles OK -- no cycles across ${String(scanned)} source file(s)`, + ); +} diff --git a/scripts/verify-import-cycles.test.mjs b/scripts/verify-import-cycles.test.mjs new file mode 100644 index 0000000..affd78d --- /dev/null +++ b/scripts/verify-import-cycles.test.mjs @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-import-cycles.test.mjs +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {findCycle} from './verify-import-cycles.mjs'; + +/** Drives the search over a literal adjacency map, so no file is written. */ +function edgesOf(graph) { + return file => graph[file] ?? []; +} + +test('an acyclic graph reports no cycle', () => { + const graph = {a: ['b', 'c'], b: ['c'], c: []}; + assert.equal(findCycle(Object.keys(graph), edgesOf(graph)), null); +}); + +test('a two-node cycle is caught', () => { + const graph = {a: ['b'], b: ['a']}; + assert.deepEqual(findCycle(Object.keys(graph), edgesOf(graph)), [ + 'a', + 'b', + 'a', + ]); +}); + +test('a self-import is caught', () => { + const graph = {a: ['a']}; + assert.deepEqual(findCycle(Object.keys(graph), edgesOf(graph)), ['a', 'a']); +}); + +test('a longer cycle is reported with every file on it', () => { + const graph = {a: ['b'], b: ['c'], c: ['a']}; + assert.deepEqual(findCycle(Object.keys(graph), edgesOf(graph)), [ + 'a', + 'b', + 'c', + 'a', + ]); +}); + +test('a cycle reachable only from an unrelated entry point is still caught', () => { + // `entry` is acyclic itself; the cycle sits two hops in. A search that stopped at the first + // finished component would miss it. + const graph = {entry: ['a'], a: ['b'], b: ['c'], c: ['b']}; + assert.deepEqual(findCycle(Object.keys(graph), edgesOf(graph)), [ + 'b', + 'c', + 'b', + ]); +}); + +test('a diamond is not a cycle', () => { + // Two paths reaching the same node is re-convergence, not recursion. A search that marked a node + // "seen" without distinguishing "on the current path" from "finished" would report this. + const graph = {a: ['b', 'c'], b: ['d'], c: ['d'], d: []}; + assert.equal(findCycle(Object.keys(graph), edgesOf(graph)), null); +}); + +test('an edge to a file outside the scanned set is ignored', () => { + const graph = {a: ['outside']}; + assert.equal(findCycle(['a'], edgesOf(graph)), null); +}); diff --git a/scripts/verify-knowledge-structure.mjs b/scripts/verify-knowledge-structure.mjs new file mode 100644 index 0000000..924d359 --- /dev/null +++ b/scripts/verify-knowledge-structure.mjs @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-knowledge-structure.mjs +// +// `docs/knowledge/` is two trees, and this is the gate that keeps them apart. +// +// harvested/ what the source documents say. Generated by `knowledge-harvest`, +// never hand-edited. +// notes/ what the implementation found. Hand-written, role `review`, a +// manual `sha:` marker, and it outranks the harvested entry it +// names. +// +// Why a gate rather than a convention: a `<sub>` sha digests the WHOLE source +// file, not the entry, so every entry harvested from one file carries the same +// value and an edit to an entry's text does not change it. The next harvest +// cannot see the edit — it regenerates the original text or writes a duplicate. +// Hand-written knowledge under `harvested/` is therefore not just untidy, it is +// scheduled for silent deletion. The three rules below are the smallest set +// that catches it, and none of them needs a file outside the repository. +// +// Structural only. It says nothing about whether an entry is TRUE — that is +// what the drift check (`bun run knowledge:drift`) and a re-harvest are for. +import {readFileSync, readdirSync} from 'node:fs'; +import {dirname, join, normalize} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +import { + ROLES, + derivePrefixes, + entryLocation, + loadCanonicalIds, + loadCorpus, +} from './knowledge.mjs'; + +// The roles a harvested entry may carry: everything but `review`, which is the +// notes tree's. Derived from the CLI's list so a new role is a one-line change +// there, not two. +const HARVESTED_ROLES = ROLES.filter(role => role !== 'review'); + +// Below this the corpus is not small, it is broken — a parse that silently +// yields nothing (a CRLF topic file, a moved directory) would otherwise let this +// gate print OK over an empty tree, which is worse than failing. The floor is +// deliberately far below the real count (1457) so ordinary editing never trips +// it. +const MIN_HARVESTED_ENTRIES = 1000; + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const knowledgeDir = join(repoRoot, 'docs', 'knowledge'); +const sourcesPath = join( + repoRoot, + 'docs', + 'knowledge', + 'harvested', + 'SOURCES.md', +); + +const SOURCE_ROW = /^\|\s*`([^`]+)`\s*\|/; +// A `<sub>` source is `path/to/file.md:12-18`, `…:12`, or a bare path. +const LINE_SUFFIX = /:[\d,-]+$/; + +// The three source roots are not hardcoded: they are the directories the +// harvest manifest itself names. One of them is an absolute path on the harvest +// machine (the styleguide is a sibling repository), so hardcoding it would make +// this gate machine-specific — and deriving it means a fourth root is a +// SOURCES.md edit, reviewable in the same diff as the entries that use it. +function sourceRoots(text) { + const roots = new Set(); + for (const line of text.split('\n')) { + const match = SOURCE_ROW.exec(line); + if (match) roots.add(dirname(match[1])); + } + if (roots.size === 0) { + throw new Error( + `parsed zero source rows out of ${sourcesPath}; its table format changed ` + + 'and the allowlist of source roots cannot be derived', + ); + } + + // One row at a root's parent would widen the allowlist to everything beneath + // it — a row naming a file at the `docs/` root makes `docs` a root, and then every + // `docs/work/...` citation passes. Refuse rather than silently widen. + const sorted = [...roots].sort(); + for (const root of sorted) { + const swallowed = sorted.find( + other => other !== root && other.startsWith(`${root}/`), + ); + if (swallowed !== undefined) { + throw new Error( + `${sourcesPath} derives the source root '${root}', which contains ` + + `'${swallowed}'. A root that contains another admits everything ` + + 'beneath it; list sources at one level, not two.', + ); + } + } + return sorted; +} + +// `normalize` first: a `..` segment makes a string prefix test meaningless, and +// the point of this check is where the path actually lands. +function isUnderRoot(source, roots) { + const path = normalize(source.replace(LINE_SUFFIX, '')); + return roots.some(root => path.startsWith(`${normalize(root)}/`)); +} + +// Returns one line per violation, empty when the trees are clean. +function structuralViolations(entries, roots) { + const violations = []; + for (const entry of entries) { + if (entry.origin !== 'harvested') continue; + const at = `${entryLocation(entry)} (${entry.section})`; + + if (entry.roles.includes('review')) { + violations.push( + `${at}: role \`review\` under harvested/. A review-role entry is ` + + 'hand-written knowledge; move it to docs/knowledge/notes/.', + ); + } + // `review` is the label an honest hand edit wears. An invented role is what + // a careless one wears, and it used to pass every rule here. + for (const role of entry.roles) { + if (role === 'review' || HARVESTED_ROLES.includes(role)) continue; + violations.push( + `${at}: role \`${role}\` under harvested/, which is not one of ` + + `${HARVESTED_ROLES.join(', ')}. A harvested entry carries the role of ` + + 'the document it came from.', + ); + } + // The per-source loop below iterates zero times on an entry with no + // provenance line at all, so the strongest rule here never ran on the one + // shape a hand-written bullet is most likely to have. + if (entry.sources.length === 0) { + violations.push( + `${at}: no source. Every harvested entry carries a \`<sub>\` naming ` + + 'the document it came from; a bullet without one was written by hand.', + ); + } + if (entry.section === 'Superseded') { + violations.push( + `${at}: a Superseded entry under harvested/. Superseding is a ` + + 'judgement the implementation made; it belongs in docs/knowledge/notes/.', + ); + } + for (const source of entry.sources) { + if (isUnderRoot(source, roots)) continue; + violations.push( + `${at}: cites \`${source}\`, which is under none of the harvested ` + + `source roots (${roots.join(', ')}). Only a harvest of those roots ` + + 'belongs in harvested/.', + ); + } + } + return violations; +} + +// A fourth rule, beyond the three the trees themselves imply, because the +// accident it catches is silent: `knowledge-harvest` defaults its `--corpus` to +// `<cwd>/docs/knowledge/`, so a run that forgets `--corpus docs/knowledge/harvested` +// writes a third copy of the corpus at the root. No query reads it — the CLI +// walks harvested/ and notes/ — so the knowledge is not wrong, it is invisible. +function strayTopicFiles(dir = knowledgeDir) { + return readdirSync(dir, {withFileTypes: true}) + .filter( + entry => + entry.isFile() && + entry.name.endsWith('.md') && + entry.name !== 'README.md', + ) + .map( + entry => + `docs/knowledge/${entry.name}: a topic file at the root of ` + + 'docs/knowledge/, which is neither tree — no query reads it. Move it ' + + 'into harvested/ or notes/. (A `knowledge-harvest` run without ' + + '`--corpus docs/knowledge/harvested` writes here.)', + ); +} + +// The mirror of rule 1, and fatal for the same reason rather than advisory: a +// note that does not say `review` reads, in a query result, exactly as though a +// source document had said it — and a warning inside a step that exits 0 is +// invisible in a green log. +function noteViolations(entries) { + return entries + .filter(entry => entry.origin === 'note' && !entry.roles.includes('review')) + .map( + entry => + `${entryLocation(entry)}: a note whose role is ` + + `\`${entry.role ?? 'none'}\`, not \`review\`. A note states what the ` + + 'implementation found; that is what the role says.', + ); +} + +/** A backticked `<topic>/<8 hex>` key, the form a note uses to name the rule it overrides. */ +const NOTE_KEY = /`([a-z0-9-]+\/[0-9a-f]{8})`/g; + +/** + * Every backticked key a note cites that no harvested entry carries. + * + * A note names the harvested rule it overrides by that rule's stable key, which is digested from the + * entry's *text* — so a re-harvest that rewords the rule changes its key and silently orphans every + * note citing it. `bun run knowledge:drift` reported that, but nothing blocked on it, which made the + * rot invisible until somebody ran a hand tool. Failing here means a re-harvest cannot land until + * the notes it invalidates are updated in the same commit, which is the point (docs/work/mvp/2026-09-04-open-items-dissolution.md + * O2, mechanism 1 of the two reviewed). + */ +function orphanedNoteKeys(entries) { + const live = new Set( + entries.filter(entry => entry.origin !== 'note').map(entry => entry.key), + ); + const violations = []; + for (const entry of entries) { + if (entry.origin !== 'note') continue; + for (const [, key] of entry.text.matchAll(NOTE_KEY)) { + if (live.has(key)) continue; + violations.push( + `${entryLocation(entry)}: cites \`${key}\`, which no harvested entry ` + + 'carries. A key is digested from the entry text, so a re-harvest that ' + + 'rewords the rule changes it — update the note to the new key ' + + '(`bun run knowledge --topic <topic> --section <section>` prints it).', + ); + } + } + return violations; +} + +// The scripts here wrap-and-rethrow with `{cause}`; printing only the outer +// message throws away the half that says which file. +function formatCauses(error) { + const messages = []; + for (let current = error; current; current = current.cause) { + messages.push(current.message); + } + return messages.join('\n caused by: '); +} + +function main() { + const roots = sourceRoots(readFileSync(sourcesPath, 'utf8')); + const entries = loadCorpus(derivePrefixes(loadCanonicalIds())); + const violations = [ + ...structuralViolations(entries, roots), + ...orphanedNoteKeys(entries), + ...noteViolations(entries), + ...strayTopicFiles(), + ]; + + const harvested = entries.filter(entry => entry.origin === 'harvested'); + if (harvested.length < MIN_HARVESTED_ENTRIES) { + throw new Error( + `only ${harvested.length} harvested entries parsed, below the floor of ` + + `${MIN_HARVESTED_ENTRIES}. The corpus did not shrink by hand — a parse ` + + 'is failing (a CRLF or BOM topic file, a moved directory), and every ' + + 'rule below would pass vacuously over the hole.', + ); + } + + if (violations.length > 0) { + for (const violation of violations) { + process.stderr.write(`knowledge-structure violation: ${violation}\n`); + } + process.stderr.write( + `${violations.length} violation(s). docs/knowledge/harvested/ carries ` + + 'only harvested entries; hand-written knowledge lives in ' + + 'docs/knowledge/notes/ with role `review` and a manual sha marker.\n', + ); + return 1; + } + + const notes = entries.length - harvested.length; + process.stdout.write( + `knowledge structure OK: ${harvested.length} harvested entries, each with ` + + `a source under one of ${roots.length} roots and a role among ` + + `${HARVESTED_ROLES.join('/')}, none Superseded; ${notes} notes, all ` + + 'review-role, every cited key live; nothing stranded at the root.\n', + ); + return 0; +} + +export { + sourceRoots, + structuralViolations, + noteViolations, + orphanedNoteKeys, + strayTopicFiles, +}; + +if (process.argv[1] === fileURLToPath(import.meta.url)) { + try { + process.exitCode = main(); + } catch (error) { + process.stderr.write(`${formatCauses(error)}\n`); + process.exitCode = 2; + } +} diff --git a/scripts/verify-knowledge-structure.test.mjs b/scripts/verify-knowledge-structure.test.mjs new file mode 100644 index 0000000..55f1a0b --- /dev/null +++ b/scripts/verify-knowledge-structure.test.mjs @@ -0,0 +1,266 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-knowledge-structure.test.mjs +// +// Run with `bun run test:scripts`. Tests the detector, not the corpus: the +// corpus being clean today is what `bun run verify:knowledge-structure` says in +// CI, and a gate nobody has seen fail is a gate nobody trusts. +import assert from 'node:assert/strict'; +import {mkdirSync, mkdtempSync, writeFileSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {test} from 'node:test'; + +import { + noteViolations, + orphanedNoteKeys, + sourceRoots, + strayTopicFiles, + structuralViolations, +} from './verify-knowledge-structure.mjs'; + +const ROOTS = [ + '/home/u/styleguide/typescript', + 'docs/product-spec', + 'docs/sdk-design-nodejs', +]; + +function entry(overrides) { + return { + file: 'topic.md', + topic: 'topic', + origin: 'harvested', + line: 4, + section: 'Rules', + roles: ['spec'], + role: 'spec', + sources: ['docs/product-spec/04-core-http-domain-model.md:9'], + ...overrides, + }; +} + +test('the source roots are derived from the manifest, not hardcoded', () => { + const roots = sourceRoots( + [ + '# Harvested Sources', + '', + '| source | role | sha256 | last harvest |', + '| --- | --- | --- | --- |', + '| `/home/u/styleguide/typescript/01-formatting.md` | styleguide | `aaa` | 2026-07-25 |', + '| `docs/product-spec/04-core-http-domain-model.md` | spec | `bbb` | 2026-07-25 |', + '| `docs/product-spec/12-pagination.md` | spec | `ccc` | 2026-07-25 |', + ].join('\n'), + ); + assert.deepEqual(roots, [ + '/home/u/styleguide/typescript', + 'docs/product-spec', + ]); +}); + +test('a manifest that parses to nothing fails loudly', () => { + assert.throws(() => sourceRoots('# Harvested Sources\n'), /zero source rows/); +}); + +test('a clean pair of trees reports no violation', () => { + const entries = [ + entry({}), + entry({ + origin: 'note', + section: 'Superseded', + roles: ['review'], + role: 'review', + sources: [ + 'docs/superpowers/specs/2026-07-28-phase6c-pagination-design.md', + ], + }), + ]; + assert.deepEqual(structuralViolations(entries, ROOTS), []); + assert.deepEqual(noteViolations(entries), []); +}); + +test('a review-role entry under harvested/ is a violation', () => { + const [violation, ...rest] = structuralViolations( + [entry({roles: ['review'], role: 'review'})], + ROOTS, + ); + assert.equal(rest.length, 0); + assert.match(violation, /role `review` under harvested\//); + assert.match(violation, /topic\.md:4/); +}); + +test('a Superseded entry under harvested/ is a violation', () => { + const [violation] = structuralViolations( + [entry({section: 'Superseded'})], + ROOTS, + ); + assert.match(violation, /Superseded entry under harvested\//); +}); + +test('a harvested <sub> citing outside the three roots is a violation', () => { + const [violation] = structuralViolations( + [entry({sources: ['docs/superpowers/plans/2026-07-28-phase9.md:1097']})], + ROOTS, + ); + assert.match(violation, /cites `docs\/superpowers\/plans/); + assert.match(violation, /under none of the harvested source roots/); +}); + +test('a line range or a bare path both resolve to their root', () => { + const ranged = entry({ + sources: ['/home/u/styleguide/typescript/06-classes.md:168-183'], + }); + const bare = entry({sources: ['docs/sdk-design-nodejs/04-domain.md']}); + assert.deepEqual(structuralViolations([ranged, bare], ROOTS), []); +}); + +test('a prefix match is on a path segment, not on characters', () => { + const sibling = entry({sources: ['docs/product-spec-draft/04-core.md:9']}); + const [violation] = structuralViolations([sibling], ROOTS); + assert.match(violation, /docs\/product-spec-draft/); +}); + +test('the same shapes are allowed under notes/, which is hand-written', () => { + const note = entry({ + origin: 'note', + section: 'Superseded', + roles: ['review'], + role: 'review', + sources: ['docs/superpowers/plans/2026-07-28-phase9.md:1097'], + }); + assert.deepEqual(structuralViolations([note], ROOTS), []); +}); + +test('a topic file stranded at the root is caught', () => { + const dir = mkdtempSync(join(tmpdir(), 'knowledge-stray-')); + mkdirSync(join(dir, 'harvested')); + writeFileSync(join(dir, 'harvested', 'api-design.md'), '## Rules\n'); + assert.deepEqual(strayTopicFiles(dir), [], 'a tree is not a stray file'); + + writeFileSync(join(dir, 'README.md'), '# the two trees\n'); + assert.deepEqual( + strayTopicFiles(dir), + [], + 'README.md is the contract, not a topic', + ); + + writeFileSync(join(dir, 'pagination.md'), '## Rules\n'); + const [violation, ...rest] = strayTopicFiles(dir); + assert.equal(rest.length, 0); + assert.match(violation, /pagination\.md: a topic file at the root/); + assert.match(violation, /--corpus docs\/knowledge\/harvested/); +}); + +test('the live corpus has nothing stranded at its root', () => { + assert.deepEqual(strayTopicFiles(), []); +}); + +test('a harvested entry with no provenance line at all is caught', () => { + const [violation, ...rest] = structuralViolations( + [entry({sources: [], roles: [], role: null, subLine: null})], + ROOTS, + ); + assert.equal(rest.length, 0); + assert.match(violation, /no source/); + assert.match(violation, /written by hand/); +}); + +test('an invented role under harvested/ is caught, not just `review`', () => { + const [violation, ...rest] = structuralViolations( + [entry({roles: ['impl'], role: 'impl'})], + ROOTS, + ); + assert.equal(rest.length, 0); + assert.match(violation, /role `impl` under harvested\//); + assert.match(violation, /spec, design, styleguide/); +}); + +test('a source root that contains another root is refused, not widened', () => { + assert.throws( + () => + sourceRoots( + [ + '| `docs/product-spec/04-core.md` | spec | `aaa` | 2026-07-25 |', + '| `docs/README.md` | design | `bbb` | 2026-07-25 |', + ].join('\n'), + ), + /which contains/, + ); +}); + +test('a `..` segment cannot walk out of a source root', () => { + const escaping = entry({ + sources: ['docs/product-spec/../../etc/passwd'], + }); + const [violation] = structuralViolations([escaping], ROOTS); + assert.match(violation, /under none of the harvested source roots/); +}); + +test('a note that is not review-role is a violation, not a warning', () => { + const note = entry({origin: 'note', roles: ['design'], role: 'design'}); + assert.deepEqual( + structuralViolations([note], ROOTS), + [], + 'the harvested rules do not apply to a note', + ); + const [violation, ...rest] = noteViolations([note]); + assert.equal(rest.length, 0); + assert.match(violation, /notes\/topic\.md:4/); + assert.match(violation, /`design`, not `review`/); +}); + +test('a note citing a key no harvested entry carries is a violation (O2)', () => { + const entries = [ + {origin: 'harvested', key: 'pipeline/e66ace13', text: 'the rule'}, + { + origin: 'note', + topic: 'pipeline', + line: 8, + key: 'pipeline/dd68351a', + text: 'Resolves `pipeline/deadbeef`, which no longer exists.', + }, + ]; + const found = orphanedNoteKeys(entries); + assert.equal(found.length, 1); + assert.match(found[0], /pipeline\/deadbeef/); +}); + +test('a note citing a live key is not a violation', () => { + const entries = [ + {origin: 'harvested', key: 'pipeline/e66ace13', text: 'the rule'}, + { + origin: 'note', + topic: 'pipeline', + line: 8, + key: 'pipeline/dd68351a', + text: 'Resolves `pipeline/e66ace13`.', + }, + ]; + assert.deepEqual(orphanedNoteKeys(entries), []); +}); + +test('a note citing no key at all is not a violation', () => { + const entries = [ + { + origin: 'note', + topic: 'x', + line: 1, + key: 'x/00000000', + text: 'no keys here', + }, + ]; + assert.deepEqual(orphanedNoteKeys(entries), []); +}); + +test("a note's own key does not count as a citation of itself", () => { + // The note's key is derived from its text, not written into it, so a note whose body happens to + // contain no backticked key cites nothing — including itself. + const entries = [ + { + origin: 'note', + topic: 'x', + line: 1, + key: 'x/abcdef01', + text: 'plain prose', + }, + ]; + assert.deepEqual(orphanedNoteKeys(entries), []); +}); diff --git a/scripts/verify-node-floor.mjs b/scripts/verify-node-floor.mjs deleted file mode 100644 index 1344a07..0000000 --- a/scripts/verify-node-floor.mjs +++ /dev/null @@ -1,26 +0,0 @@ -// SPDX-License-Identifier: MIT -// scripts/verify-node-floor.mjs -// -// NFR-10/NFR-17 residual pulled forward from Phase 3: CI must run the *built artifact* against the -// declared minimum Node version, not just the runner default. This forces the two-signal branch of -// composeSignal() — the one that calls AbortSignal.any(), the API that landed in exactly Node -// 18.17.0, the repo's declared floor (engines.node ">=18.17"). -import assert from 'node:assert/strict'; -import {composeSignal} from '@dexpace/core'; - -const controller = new AbortController(); -const combined = composeSignal(controller.signal, 50); - -assert.ok( - combined instanceof AbortSignal, - 'composeSignal() must return an AbortSignal when both a user signal and a timeout are supplied', -); -assert.notEqual( - combined, - controller.signal, - 'the combined signal must be a distinct AbortSignal.any() result, not the raw user signal', -); - -console.log( - `node-floor check passed: AbortSignal.any() resolved correctly on Node ${process.version}`, -); diff --git a/scripts/verify-reproducible-build.mjs b/scripts/verify-reproducible-build.mjs new file mode 100644 index 0000000..355c22a --- /dev/null +++ b/scripts/verify-reproducible-build.mjs @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-reproducible-build.mjs +// +// NFR-12: an identical source tree MUST produce a byte-identical build. +// +// This row sat open through Phase 10 on the stated grounds that it "cannot execute without a real +// build artifact" — true while the repository was docs-only, and false from Phase 1 on. The check is +// mechanical: build the workspace twice from a swept tree and compare a SHA-256 of every emitted +// file. Asserting reproducibility without running it is exactly the kind of claimed-but-unverified +// conformance the open-items register existed to catch. +// +// Both builds sweep `dist/` and every `*.tsbuildinfo` first. Without the sweep the second `tsc` is +// incremental and rewrites nothing, so the comparison passes by not having run — the failure mode +// that makes a naive version of this gate worthless. +// +// Non-determinism this would catch: a timestamp or absolute path baked into emitted output, a +// `Math.random()`/`Date.now()` reaching a build-time codegen step +// (`packages/core/scripts/gen-version.mjs` is the one such step today, and injecting a `Date.now()` +// there is this gate's negative test), or a `tsc` upgrade that starts emitting map keys in hash +// order. +// +// TWO LEGS, because "the artifact" means two different things. The emit leg compares every file under +// each package's `dist/`. The pack leg then runs `npm pack` on every publishable package and compares +// the tarball digests — that is the byte sequence a consumer actually installs, and it is what the +// Phase 10 commit message asserted was reproducible on the strength of a by-hand check. Gating it is +// the difference between an asserted property and a verified one. +// +// The pack leg is deliberately kept: it is deterministic here because `npm pack` normalizes tar +// entries (fixed mtime, sorted order, portable mode bits) rather than stamping wall-clock time into +// the header — verified by packing `@dexpace/core` twice, seconds apart, on npm 12.0.1 and getting one +// digest. Both packs also happen inside a single run on a single npm, so an npm upgrade cannot make +// this flap. It adds ~7s and a dependency on `npm` being on PATH, which the Node toolchain CI already +// installs; a missing `npm` fails the gate loudly rather than skipping the leg. +// +// What the pack leg does NOT add much of, stated so nobody over-reads it: with `files: ["dist"]` on +// every publishable package, the tarball is a pure function of the `dist/` bytes the first leg already +// compared plus static manifest files. Its real job is to pin `npm pack`'s own normalization, and to +// catch a future `files`/`.npmignore` change that starts shipping something time-varying from outside +// `dist/`. +import assert from 'node:assert/strict'; +import {execFileSync, spawnSync} from 'node:child_process'; +import {createHash} from 'node:crypto'; +import { + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + statSync, +} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join, relative} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const packagesDir = join(repoRoot, 'packages'); + +/** Every file under a package's `dist/`, repo-relative, sorted for a stable comparison order. */ +function collectArtifacts() { + const files = []; + const walk = dir => { + for (const entry of readdirSync(dir, {withFileTypes: true})) { + const full = join(dir, entry.name); + if (entry.isDirectory()) walk(full); + else if (entry.isFile()) files.push(full); + } + }; + for (const pkg of readdirSync(packagesDir, {withFileTypes: true})) { + if (!pkg.isDirectory()) continue; + const dist = join(packagesDir, pkg.name, 'dist'); + try { + if (statSync(dist).isDirectory()) walk(dist); + } catch { + // No dist/ for this package (private, or source-resolved) — nothing to compare. + } + } + return files.sort(); +} + +/** Sweep every build output so the next build starts from the tree CI checks out, not a warm one. */ +function sweep() { + const walkAndDelete = dir => { + for (const entry of readdirSync(dir, {withFileTypes: true})) { + if (entry.name === 'node_modules') continue; + const full = join(dir, entry.name); + if (entry.isDirectory()) { + if (entry.name === 'dist') rmSync(full, {recursive: true, force: true}); + else walkAndDelete(full); + } else if (entry.name.endsWith('.tsbuildinfo')) { + rmSync(full, {force: true}); + } + } + }; + walkAndDelete(packagesDir); +} + +function build(label) { + process.stdout.write(`verify-reproducible-build: ${label} build…\n`); + execFileSync('bun', ['run', 'build'], {cwd: repoRoot, stdio: 'inherit'}); +} + +/** Map of repo-relative path → SHA-256 of the file's bytes. */ +function digestArtifacts() { + const digests = new Map(); + for (const file of collectArtifacts()) { + digests.set( + relative(repoRoot, file), + createHash('sha256').update(readFileSync(file)).digest('hex'), + ); + } + return digests; +} + +/** + * Every package `npm pack` produces a publishable tarball for. `private: true` packages + * (`shrink-test`, `transport-conformance`) never ship, so their bytes are not an artifact. + */ +function publishablePackages() { + const names = []; + for (const pkg of readdirSync(packagesDir, {withFileTypes: true})) { + if (!pkg.isDirectory()) continue; + const manifest = join(packagesDir, pkg.name, 'package.json'); + try { + if (JSON.parse(readFileSync(manifest, 'utf8')).private !== true) { + names.push(pkg.name); + } + } catch { + // No manifest — not a package, so nothing to pack. + } + } + return names.sort(); +} + +/** + * Map of `npm-pack:<tarball>` → SHA-256 of the tarball, packed into a temp dir this owns. + * + * Packing outside the repo keeps the tarballs out of `collectArtifacts()`'s walk and out of + * `git status`; the dir is removed even when a pack throws. + */ +function digestTarballs(label) { + process.stdout.write(`verify-reproducible-build: ${label} pack…\n`); + const dest = mkdtempSync(join(tmpdir(), `dexpace-repro-${label}-`)); + try { + for (const name of publishablePackages()) { + execFileSync('npm', ['pack', '--pack-destination', dest], { + cwd: join(packagesDir, name), + // `npm pack` narrates the whole tarball manifest on stderr; the digests are the signal. + stdio: ['ignore', 'ignore', 'ignore'], + }); + } + const digests = new Map(); + for (const file of readdirSync(dest).sort()) { + digests.set( + `npm-pack:${file}`, + createHash('sha256') + .update(readFileSync(join(dest, file))) + .digest('hex'), + ); + } + return digests; + } finally { + rmSync(dest, {recursive: true, force: true}); + } +} + +/** The three ways two digest maps can disagree, rendered for the assertion message. */ +function diffDigests(a, b) { + return [ + ...[...a.keys()] + .filter(path => !b.has(path)) + .map(path => ` only in build 1: ${path}`), + ...[...b.keys()] + .filter(path => !a.has(path)) + .map(path => ` only in build 2: ${path}`), + ...[...a.entries()] + .filter(([path, hash]) => b.has(path) && b.get(path) !== hash) + .map(([path]) => ` differing bytes: ${path}`), + ]; +} + +// Fail here rather than inside the first `npm pack`, where an ENOENT from execFileSync reads as a +// packaging defect instead of a missing tool. +assert.equal( + spawnSync('npm', ['--version'], {stdio: 'ignore'}).status, + 0, + 'NFR-12: `npm` is not on PATH, so the pack leg cannot run. Install Node’s npm, or run the' + + ' emit leg alone by hand.', +); + +sweep(); +build('first'); +const first = digestArtifacts(); +const firstTarballs = digestTarballs('first'); + +assert.ok( + first.size > 0, + 'NFR-12: the build emitted no artifacts at all — nothing to compare', +); +assert.ok( + firstTarballs.size > 0, + 'NFR-12: no publishable package produced a tarball — nothing to compare', +); + +sweep(); +build('second'); +const second = digestArtifacts(); +const secondTarballs = digestTarballs('second'); + +const problems = [ + ...diffDigests(first, second), + ...diffDigests(firstTarballs, secondTarballs), +]; + +assert.equal( + problems.length, + 0, + `NFR-12 violation: two clean builds of an identical source tree differed.\n${problems.join('\n')}`, +); + +process.stdout.write( + `verify-reproducible-build: OK — ${String(first.size)} emitted files and ` + + `${String(firstTarballs.size)} npm-pack tarballs byte-identical across two clean builds (NFR-12)\n`, +); diff --git a/scripts/verify-runtime-floor.mjs b/scripts/verify-runtime-floor.mjs index 9272dbc..ec744cf 100644 --- a/scripts/verify-runtime-floor.mjs +++ b/scripts/verify-runtime-floor.mjs @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: MIT // scripts/verify-runtime-floor.mjs // // NFR-10 / the design doc's "Runtime-floor discipline" gate: a publishable @@ -19,13 +20,17 @@ import {join} from 'node:path'; import {fileURLToPath} from 'node:url'; // The agreed pairings for this project. These are deliberate project decisions, -// not a general ES-to-Node compatibility matrix: ES2022 syntax runs on Node -// 16.11+, but the SDK declares a 18.17 floor. Adding a row here is a reviewed -// choice about what runtimes the SDK supports, never a mechanical bump. +// not a general ES-to-Node compatibility matrix, and the floor is set by the +// runtime built-ins the SDK calls rather than by the syntax it emits: ES2023 +// syntax runs on Node 20.0, but `globalThis.crypto` (which `MultipartBody` +// reads synchronously at construction) is exposed unflagged only from 19.0 and +// is absent from ESM on every Node 18 release, and `AbortSignal.any()` (which +// `composeSignal` calls) landed in 20.3.0. Adding or moving a row here is a +// reviewed choice about what runtimes the SDK supports, never a mechanical bump. const LANGUAGE_LEVEL_TO_NODE_FLOOR = { es2021: '>=16.11', es2022: '>=18.17', - es2023: '>=20.0', + es2023: '>=20.3', }; const repoRoot = fileURLToPath(new URL('..', import.meta.url)); diff --git a/scripts/verify-seam-1.mjs b/scripts/verify-seam-1.mjs index 55ceeab..4a69328 100644 --- a/scripts/verify-seam-1.mjs +++ b/scripts/verify-seam-1.mjs @@ -1,16 +1,80 @@ +// SPDX-License-Identifier: MIT // scripts/verify-seam-1.mjs +// +// SEAM-1 / NFR-1: no shipped package carries a runtime dependency it was not explicitly granted. +// Generalized from a core-only check in Phase 6a, when `@dexpace/codec-json` became the workspace's +// second package — a check hard-coded to one package silently stops covering the workspace the moment +// it grows. Phase 8a turned the blanket ban into an allow-list, because NFR-2 grants each optional +// capability core plus at most one external library: `ALLOWED_RUNTIME_DEPENDENCIES` below is that +// grant, written out per package. Every package absent from it is still held to a hard-committed +// empty `dependencies` object — an omitted field is a violation too, so the manifest states the +// invariant rather than merely failing to contradict it. +// +// It also asserts the peer-dependency pairing `sdk-design-nodejs/02` §2 prescribes for every adapter +// package. That is not a style rule: without it npm's nested resolution can install two +// non-identical copies of @dexpace/core, and the branded-identity checks that distinguish core types +// — `Tristate`'s discriminant among them — break exactly the way two JVM classloaders break +// `instanceof`. import assert from 'node:assert/strict'; -import {readFileSync} from 'node:fs'; +import {existsSync, readFileSync, readdirSync} from 'node:fs'; +import {join} from 'node:path'; import {fileURLToPath} from 'node:url'; -const manifestPath = fileURLToPath( - new URL('../packages/core/package.json', import.meta.url), -); -const corePackageJson = JSON.parse(readFileSync(manifestPath, 'utf8')); +const packagesDir = fileURLToPath(new URL('../packages', import.meta.url)); + +const packageDirs = readdirSync(packagesDir, {withFileTypes: true}) + .filter(entry => entry.isDirectory()) + .map(entry => join(packagesDir, entry.name)) + .filter(dir => existsSync(join(dir, 'package.json'))); + +assert.ok(packageDirs.length > 0, 'no packages found under packages/'); + +const ALLOWED_RUNTIME_DEPENDENCIES = { + '@dexpace/transport-fetch': ['@dexpace/transport-shared'], + '@dexpace/transport-undici': ['@dexpace/transport-shared', 'undici'], +}; + +let checkedCount = 0; + +for (const dir of packageDirs) { + const manifest = JSON.parse(readFileSync(join(dir, 'package.json'), 'utf8')); + + // A private package is never published, so neither the dependency budget nor the dual-package + // hazard below can reach a consumer through it. + if (manifest.private === true) continue; + checkedCount++; + + const allowedDeps = ALLOWED_RUNTIME_DEPENDENCIES[manifest.name]; + + if (allowedDeps === undefined) { + assert.deepEqual( + manifest.dependencies, + {}, + `SEAM-1 violation: ${manifest.name} must declare zero runtime dependencies (a hard-committed empty object)`, + ); + } else { + const unexpected = Object.keys(manifest.dependencies ?? {}).filter( + dep => !allowedDeps.includes(dep), + ); + assert.equal( + unexpected.length, + 0, + `SEAM-1 / NFR-2 violation: ${manifest.name} declared unexpected runtime dependencies: ${unexpected.join(', ')}`, + ); + } + + if (manifest.name === '@dexpace/core') continue; + + assert.ok( + manifest.peerDependencies?.['@dexpace/core'], + `dual-package hazard: ${manifest.name} must declare @dexpace/core as a peerDependency, not a regular dependency`, + ); + assert.ok( + manifest.peerDependenciesMeta?.['@dexpace/core'], + `dual-package hazard: ${manifest.name} must carry a peerDependenciesMeta entry for @dexpace/core`, + ); +} -assert.deepEqual( - corePackageJson.dependencies, - {}, - 'SEAM-1 violation: @dexpace/core must declare zero runtime dependencies', +console.log( + `SEAM-1 check passed: ${String(checkedCount)} package(s) verified against dependency boundaries`, ); -console.log('SEAM-1 check passed: @dexpace/core has zero runtime dependencies'); diff --git a/scripts/verify-seam-1.test.mjs b/scripts/verify-seam-1.test.mjs new file mode 100644 index 0000000..8d3d6f7 --- /dev/null +++ b/scripts/verify-seam-1.test.mjs @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-seam-1.test.mjs +// Exercises: SEAM-1 / NFR-1 (no runtime dependencies in any shipped package), and the +// peer-dependency rule from sdk-design-nodejs/02 §2 that guards the dual-package hazard. +// +// Tests the GATE, not a copy of its logic. An earlier draft re-read the same manifests and asserted +// the same invariants, which passes just as happily when `verify-seam-1.mjs` has stopped checking +// anything -- a bad glob, a swallowed assertion, an early `continue`. So this spawns the script and +// reads its exit code and output instead. +// +// Lives in `scripts/` and runs under `node --test` via `bun run test:scripts`, so `node:fs` and +// `node:child_process` are permitted here -- the zero-`node:` invariant governs `packages/*/src`, +// not build tooling. +import assert from 'node:assert/strict'; +import {execFileSync} from 'node:child_process'; +import { + copyFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {test} from 'node:test'; +import {fileURLToPath} from 'node:url'; + +// Absolute, never cwd-relative: `node --test` is run from the repo root today, and a relative +// `readdirSync('packages')` silently starts checking the wrong tree the day it is not. +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const script = join(repoRoot, 'scripts', 'verify-seam-1.mjs'); +const packagesDir = join(repoRoot, 'packages'); + +const PACKAGES = readdirSync(packagesDir, {withFileTypes: true}) + .filter(entry => entry.isDirectory()) + .filter(entry => { + const pkgJson = join(packagesDir, entry.name, 'package.json'); + return ( + existsSync(pkgJson) && + JSON.parse(readFileSync(pkgJson, 'utf8')).private !== true + ); + }) + .map(entry => entry.name); + +test('the workspace has more than one package, so these checks are not vacuous', () => { + assert.ok( + PACKAGES.includes('core'), + `expected packages/core, found ${PACKAGES.join(', ')}`, + ); + assert.ok( + PACKAGES.includes('codec-json'), + `expected packages/codec-json, found ${PACKAGES.join(', ')}`, + ); +}); + +test('verify-seam-1.mjs exits 0 and reports covering every package', () => { + const output = execFileSync(process.execPath, [script], { + encoding: 'utf8', + cwd: repoRoot, + }); + + // The count is the part worth asserting: it is what proves the gate widened with the workspace + // rather than staying pinned to core. A hard-coded `1` here would defeat the point. + assert.match( + output, + new RegExp( + `SEAM-1 check passed: ${String(PACKAGES.length)} package\\(s\\) verified against dependency boundaries`, + ), + `unexpected output from verify-seam-1.mjs:\n${output}`, + ); +}); + +// Copies the real script into a throwaway tree with fixture manifests beside it, so the failure +// paths are driven through the ACTUAL script rather than through a restatement of its assertions. +// The script resolves `packages/` relative to its own location, which is what makes this possible. +function runAgainstFixture(manifests) { + const dir = mkdtempSync(join(tmpdir(), 'dexpace-seam-1-')); + mkdirSync(join(dir, 'scripts'), {recursive: true}); + copyFileSync(script, join(dir, 'scripts', 'verify-seam-1.mjs')); + for (const [name, manifest] of Object.entries(manifests)) { + mkdirSync(join(dir, 'packages', name), {recursive: true}); + writeFileSync( + join(dir, 'packages', name, 'package.json'), + JSON.stringify(manifest), + ); + } + try { + return execFileSync( + process.execPath, + [join(dir, 'scripts', 'verify-seam-1.mjs')], + {encoding: 'utf8', stdio: 'pipe'}, + ); + } finally { + rmSync(dir, {recursive: true, force: true}); + } +} + +const CLEAN_CORE = {name: '@dexpace/core', dependencies: {}}; +const CLEAN_ADAPTER = { + name: '@dexpace/codec-fake', + dependencies: {}, + peerDependencies: {'@dexpace/core': 'workspace:*'}, + peerDependenciesMeta: {'@dexpace/core': {optional: false}}, +}; + +test('the fixture harness itself passes on a well-formed tree', () => { + assert.match( + runAgainstFixture({core: CLEAN_CORE, 'codec-fake': CLEAN_ADAPTER}), + /SEAM-1 check passed: 2 package\(s\)/, + ); +}); + +test('verify-seam-1.mjs fails when any package declares a runtime dependency', () => { + assert.throws( + () => + runAgainstFixture({ + core: CLEAN_CORE, + 'codec-fake': {...CLEAN_ADAPTER, dependencies: {lodash: '^4'}}, + }), + /SEAM-1 violation: @dexpace\/codec-fake/, + 'a non-empty dependencies map on a non-core package did not fail the gate', + ); +}); + +test('verify-seam-1.mjs fails when a package omits `dependencies` instead of committing to {}', () => { + // An omitted field is not the same as a declared empty one: the manifest has to state the + // invariant, not merely fail to contradict it. Phase 8a's allow-list rewrite briefly accepted + // `dependencies: undefined`, which is exactly how the blanket ban would erode in practice. + const omitted = {...CLEAN_ADAPTER}; + delete omitted.dependencies; + assert.throws( + () => runAgainstFixture({core: CLEAN_CORE, 'codec-fake': omitted}), + /SEAM-1 violation: @dexpace\/codec-fake/, + 'an omitted dependencies field did not fail the gate', + ); +}); + +test('verify-seam-1.mjs allows only the dependencies NFR-2 grants a package by name', () => { + // The allow-listed transports may take their sanctioned dependency and nothing else. Keyed by + // package name, so the grant cannot be inherited by a package that merely looks similar. + const granted = { + ...CLEAN_ADAPTER, + name: '@dexpace/transport-undici', + dependencies: {'@dexpace/transport-shared': 'workspace:*', undici: '^6'}, + }; + assert.match( + runAgainstFixture({core: CLEAN_CORE, 'transport-undici': granted}), + /SEAM-1 check passed: 2 package\(s\)/, + ); + assert.throws( + () => + runAgainstFixture({ + core: CLEAN_CORE, + 'transport-undici': { + ...granted, + dependencies: {...granted.dependencies, lodash: '^4'}, + }, + }), + /NFR-2 violation: @dexpace\/transport-undici declared unexpected runtime dependencies: lodash/, + 'a dependency outside the grant did not fail the gate', + ); +}); + +test('verify-seam-1.mjs fails when core itself grows a runtime dependency', () => { + assert.throws( + () => + runAgainstFixture({ + core: {...CLEAN_CORE, dependencies: {lodash: '^4'}}, + 'codec-fake': CLEAN_ADAPTER, + }), + /SEAM-1 violation: @dexpace\/core/, + ); +}); + +test('verify-seam-1.mjs fails when an adapter declares no @dexpace/core peerDependency', () => { + const noPeer = {...CLEAN_ADAPTER}; + delete noPeer.peerDependencies; + assert.throws( + () => runAgainstFixture({core: CLEAN_CORE, 'codec-fake': noPeer}), + /dual-package hazard: @dexpace\/codec-fake must declare @dexpace\/core as a peerDependency/, + ); +}); + +test('verify-seam-1.mjs fails when the peerDependenciesMeta entry is missing', () => { + const noMeta = {...CLEAN_ADAPTER}; + delete noMeta.peerDependenciesMeta; + assert.throws( + () => runAgainstFixture({core: CLEAN_CORE, 'codec-fake': noMeta}), + /must carry a peerDependenciesMeta entry/, + ); +}); diff --git a/scripts/verify-sse-37.mjs b/scripts/verify-sse-37.mjs new file mode 100644 index 0000000..316f87d --- /dev/null +++ b/scripts/verify-sse-37.mjs @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-sse-37.mjs +import {readdirSync, readFileSync, statSync} from 'node:fs'; +import {join, relative} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +const FORBIDDEN = [ + /^\.\.\/serde\//, + /^\.\.\/seams\/serde\.js$/, + /^@dexpace\/codec-json/, +]; + +const IMPORT_PATTERNS = [ + // Standard static import/export: import ... from '...' or export ... from '...' + /(?:^|[;\n])\s*(?:import|export)[\s\S]*?from\s*['"]([^'"]+)['"]/g, + // Side-effect import: import '...' + /(?:^|[;\n])\s*import\s*['"]([^'"]+)['"]/g, + // Dynamic import: import('...') + /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g, +]; + +/** + * SSE-38: reconnection and last-event-id continuity are the caller's responsibility. Core must contain no path + * that re-opens a connection or writes a `Last-Event-ID` header. Checked as a literal scan because the failure + * mode is somebody "helpfully" adding one — there is no type or import that would give it away. + * + * Scanned against **code with comments stripped**. The requirement forbids the code path, not the documentation + * of its absence — and "this subsystem never reconnects; that is the caller's job" is the single most likely + * sentence to appear in a TSDoc under `src/sse/`. A gate that fails on its own requirement's explanation is a + * gate the next person deletes instead of the comment, so it has to tolerate prose to be worth installing. + */ +const RECONNECT_MARKERS = [/Last-Event-ID/i, /\breconnect/i, /\bfetch\s*\(/]; + +/** Blank out block and line comments, preserving line count so reported positions stay meaningful. */ +function stripComments(source) { + return source + .replace(/\/\*[\s\S]*?\*\//g, match => match.replace(/[^\n]/g, ' ')) + .replace(/\/\/[^\n]*/g, match => ' '.repeat(match.length)); +} + +/** Recursively collect all .ts files in dir. */ +function collectFiles(dir) { + const entries = []; + for (const name of readdirSync(dir)) { + const fullPath = join(dir, name); + const stat = statSync(fullPath); + if (stat.isDirectory()) { + entries.push(...collectFiles(fullPath)); + } else if (name.endsWith('.ts')) { + entries.push(fullPath); + } + } + return entries; +} + +/** + * SSE-37: core SSE parsing and streaming must carry no serialization dependency. + * + * @param {string} [dir] directory to scan + * @param {{file: string, source: string}[]} [injected] in-memory files, for testing the detector itself + * @returns {{file: string, specifier: string}[]} + */ +export function findForbiddenSerdeImports(dir, injected) { + const scanDir = + dir ?? fileURLToPath(new URL('../packages/core/src/sse', import.meta.url)); + + const files = + injected ?? + collectFiles(scanDir).map(fullPath => ({ + file: relative(scanDir, fullPath), + source: readFileSync(fullPath, 'utf8'), + })); + + const violations = []; + for (const {file, source} of files) { + const code = stripComments(source); + + for (const pattern of IMPORT_PATTERNS) { + pattern.lastIndex = 0; + for (const match of code.matchAll(pattern)) { + const specifier = match[1]; + if (FORBIDDEN.some(forbidden => forbidden.test(specifier))) { + violations.push({file, specifier}); + } + } + } + + // Reconnect markers are checked on shipped source only. A test double is entitled to say `fetch(` or name a + // reconnect scenario it is asserting the absence of; SSE-38 constrains what core *does*, not what the suite + // describes. + if (file.endsWith('.test.ts')) continue; + for (const marker of RECONNECT_MARKERS) { + if (marker.test(code)) { + violations.push({ + file, + specifier: `SSE-38 reconnect marker ${String(marker)}`, + }); + } + } + } + return violations; +} + +const isDirect = + process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]; + +if (isDirect) { + const violations = findForbiddenSerdeImports(); + if (violations.length > 0) { + for (const {file, specifier} of violations) { + console.error(`SSE-37 violation: ${file} imports ${specifier}`); + } + console.error( + 'Core SSE parsing and streaming MUST carry no serialization dependency (SSE-37) and no reconnection or Last-Event-ID path (SSE-38). Move conversions into a caller-supplied mapper; leave reconnection to the caller.', + ); + process.exit(1); + } + console.log( + 'SSE-37/SSE-38 OK: no serde imports and no reconnect path under packages/core/src/sse', + ); +} diff --git a/scripts/verify-sse-37.test.mjs b/scripts/verify-sse-37.test.mjs new file mode 100644 index 0000000..9b44cd7 --- /dev/null +++ b/scripts/verify-sse-37.test.mjs @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-sse-37.test.mjs +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {findForbiddenSerdeImports} from './verify-sse-37.mjs'; + +test('a clean sse/ tree reports no violations', () => { + assert.deepEqual(findForbiddenSerdeImports('packages/core/src/sse'), []); +}); + +test('a relative serde import is caught', () => { + const found = findForbiddenSerdeImports('packages/core/src/sse', [ + {file: 'fake.ts', source: "import {Tristate} from '../serde/tristate.js';"}, + ]); + assert.equal(found.length, 1); + assert.equal(found[0].specifier, '../serde/tristate.js'); +}); + +test('the serde seam and the codec package are both caught', () => { + const found = findForbiddenSerdeImports('packages/core/src/sse', [ + {file: 'a.ts', source: "import type {Serde} from '../seams/serde.js';"}, + {file: 'b.ts', source: "import {jsonSerde} from '@dexpace/codec-json';"}, + ]); + assert.equal(found.length, 2); +}); + +test('an unrelated import is not caught', () => { + const found = findForbiddenSerdeImports('packages/core/src/sse', [ + {file: 'a.ts', source: "import {IoError} from '../io/errors.js';"}, + ]); + assert.deepEqual(found, []); +}); + +test('a reconnect path or Last-Event-ID header is caught (SSE-38)', () => { + assert.equal( + findForbiddenSerdeImports('packages/core/src/sse', [ + {file: 'a.ts', source: "headers.set('Last-Event-ID', event.id);"}, + ]).length, + 1, + ); + assert.equal( + findForbiddenSerdeImports('packages/core/src/sse', [ + { + file: 'b.ts', + source: 'async function reconnect() { return fetch(url); }', + }, + ]).length, + 2, + ); +}); + +test('documenting the ABSENCE of reconnection is not a violation (SSE-38)', () => { + // The gate has to survive its own requirement being explained, or the first TSDoc that says so gets the gate + // deleted instead of the sentence. Comments are stripped before the marker scan. + assert.deepEqual( + findForbiddenSerdeImports('packages/core/src/sse', [ + { + file: 'stream.ts', + source: [ + '/**', + ' * This subsystem never reconnects and never sets a Last-Event-ID header (SSE-38);', + ' * reconnection is the caller`s job, as is any call to fetch(...) that resumes a stream.', + ' */', + 'export class SseStream {}', + ].join('\n'), + }, + ]), + [], + ); +}); + +test('a commented-out serde import is not a violation either', () => { + assert.deepEqual( + findForbiddenSerdeImports('packages/core/src/sse', [ + { + file: 'a.ts', + source: "// import {Tristate} from '../serde/tristate.js';", + }, + ]), + [], + ); +}); + +test('reconnect markers are not scanned in test files, but serde imports still are', () => { + assert.deepEqual( + findForbiddenSerdeImports('packages/core/src/sse', [ + { + file: 'stream.test.ts', + source: 'const stub = () => fetch(url); // a double may say this', + }, + ]), + [], + ); + assert.equal( + findForbiddenSerdeImports('packages/core/src/sse', [ + { + file: 'stream.test.ts', + source: "import {jsonSerde} from '@dexpace/codec-json';", + }, + ]).length, + 1, + ); +}); + +test('side-effect and dynamic serde imports are caught (SSE-37)', () => { + const found = findForbiddenSerdeImports('packages/core/src/sse', [ + {file: 'a.ts', source: "import '@dexpace/codec-json';"}, + {file: 'b.ts', source: "const m = await import('../serde/tristate.js');"}, + ]); + assert.equal(found.length, 2); +}); diff --git a/scripts/verify-test-partition.mjs b/scripts/verify-test-partition.mjs new file mode 100644 index 0000000..1219b10 --- /dev/null +++ b/scripts/verify-test-partition.mjs @@ -0,0 +1,547 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-test-partition.mjs +// +// Guards the separation between the two suites under `tests/`. The rule itself, and the reasoning +// behind it, live in ONE place: CLAUDE.md, "HARD RULE -- the `tests/` partition". This file is the +// enforcement, not a second copy of the argument. +// +// In one sentence: `tests/conformance/` runs on Bun as part of `bun run test`, +// `tests/node-conformance/` runs on `node --test` against the built `dist/`, and nothing may make +// the second run on the first's runner. Until Phase 10 the file system held that -- the Node tree +// lived at `test/`, where no Bun command could reach it. It now lives inside `tests/`, so a path +// written into five files holds it instead, and those five must agree. +// +// Reads files only. Runs neither suite. + +import {readdirSync, readFileSync} from 'node:fs'; +import {join} from 'node:path'; +import {fileURLToPath} from 'node:url'; + +const REPO_ROOT = fileURLToPath(new URL('..', import.meta.url)); + +const BUNFIG = 'bunfig.toml'; +const PACKAGE_JSON = 'package.json'; +const ESLINT_CONFIG = 'eslint.config.js'; +const RUN_CI = '.claude/skills/ci-preflight/run-ci.mjs'; +const README = 'tests/node-conformance/README.md'; + +const TESTS_ROOT = 'tests'; +const NODE_TREE = 'tests/node-conformance'; +const PACKAGES_ROOT = 'packages'; + +// Bun ignores an unrecognized `[test]` key in silence -- no warning, no error, no effect. This is +// the key that works; the decoy is the near-miss that reads as configured and is not. +const IGNORE_KEY = 'pathIgnorePatterns'; +const DECOY_IGNORE_KEY = 'testPathIgnorePatterns'; +const EXPECTED_BUN_ROOT = 'packages'; + +// `run-ci.mjs`'s `--node-floor` leg repeats the runner glob once per version manager it supports +// (mise, fnm, nvm). A minimum rather than an exact count: adding a fourth manager is fine, losing +// one silently is not. +const RUN_CI_MIN_GLOBS = 3; + +// --- path and glob primitives ------------------------------------------------------------------- + +/** + * @param {string} segment one path segment, no separators + * @returns {string} regexp source + */ +function segmentToRegExp(segment) { + let out = ''; + for (const ch of segment) { + if (ch === '*') out += '[^/]*'; + else if (ch === '?') out += '[^/]'; + else out += ch.replace(/[.+^${}()|[\]\\]/g, '\\$&'); + } + return out; +} + +/** + * Translate a shell-style glob to an anchored RegExp over POSIX-separated, repo-relative paths. + * + * `**` spans whole path segments and only as a whole segment, which is what Bun does: Bun does not + * ignore `tests/wideZa.test.mjs` for a pattern whose last segment is `Za.test.mjs` behind a `**`. + * An earlier draft compiled `**` to a bare `.*` everywhere and matched it -- the dangerous + * direction for a gate, green-lighting a config Bun reads differently. Within a segment, `*` and + * `?` stop at the separator; everything else is a literal. + * + * Deliberately small. No brace expansion, no character classes, no extglob -- Bun expands those and + * this does not, so such a pattern fails the gate rather than passing it wrongly. If one is ever + * needed, that is the signal to reach for a real matcher, not to grow this. + * + * @param {string} glob + * @returns {RegExp} + */ +function globToRegExp(glob) { + const segments = glob.split('/'); + let out = '^'; + for (let i = 0; i < segments.length; i++) { + const isLast = i === segments.length - 1; + if (segments[i] === '**') { + // Zero or more whole segments. A trailing `**` also matches an empty remainder. + out += isLast ? '(?:[^/]+(?:/[^/]+)*)?' : '(?:[^/]+/)*'; + continue; + } + out += segmentToRegExp(segments[i]); + if (!isLast) out += '/'; + } + return new RegExp(`${out}$`); +} + +/** + * `a/b/c.mjs` -> `['a', 'a/b', 'a/b/c.mjs']`. + * + * @param {string} file + * @returns {string[]} + */ +function pathPrefixes(file) { + const parts = file.split('/'); + return parts.map((_, i) => parts.slice(0, i + 1).join('/')); +} + +/** + * Would Bun skip this file for these patterns? + * + * Every directory prefix counts, not just the full path. Bun applies `pathIgnorePatterns` while + * WALKING, so a pattern naming a directory prunes that whole subtree without matching any file path + * -- measured: adding `tests/conformance/fixtures` to the list dropped `fixtures/settle.test.mjs` + * from the run. Testing full paths alone left the widening check below blind to exactly the pattern + * shape a maintainer reaches for first. It is also what makes a bare `tests/node-conformance` (no + * `/**`) read as covering the tree, which is what Bun does with it. + * + * @param {string} file repo-relative POSIX path + * @param {RegExp[]} matchers + * @returns {boolean} + */ +function isIgnored(file, matchers) { + return pathPrefixes(file).some(prefix => + matchers.some(matcher => matcher.test(prefix)), + ); +} + +// --- source readers ----------------------------------------------------------------------------- + +/** + * Every file under `dir`, as repo-relative POSIX paths. + * + * `withFileTypes` rather than a `statSync` per entry: `statSync` throws on a broken symlink, and an + * uncaught `ENOENT` stack trace is the least useful thing a gate can emit. A directory that is + * genuinely absent returns `[]`, and the caller reports that as its own violation; anything else -- + * a permission error, a descriptor limit -- propagates, because "unreadable" recovered to "empty" + * is a wrong diagnosis rather than a known-good state. + * + * @param {string} root + * @param {string} dir repo-relative + * @returns {string[]} + */ +function listFiles(root, dir) { + let entries; + try { + entries = readdirSync(join(root, dir), {withFileTypes: true}); + } catch (error) { + if (error.code === 'ENOENT' || error.code === 'ENOTDIR') return []; + throw error; + } + const found = []; + for (const entry of entries) { + const child = `${dir}/${entry.name}`; + if (entry.isDirectory()) found.push(...listFiles(root, child)); + else found.push(child); + } + return found.sort(); +} + +/** + * Blank out a `#` comment, respecting quoted strings. + * + * A naive `/#.*$/` corrupts `pathIgnorePatterns = ["tests/#node/**"]` into an unterminated line, + * whose array then swallows the rest of the section. + * + * @param {string} line + * @returns {string} + */ +function stripTomlComment(line) { + let out = ''; + let quote = null; + for (const ch of line) { + if (quote) { + out += ch; + if (ch === quote) quote = null; + } else if (ch === '"' || ch === "'") { + quote = ch; + out += ch; + } else if (ch === '#') { + break; + } else { + out += ch; + } + } + return out; +} + +/** + * Is the offset inside a JS comment? + * + * Line-scoped on purpose. The obvious implementation -- blank out `/*...*\/` and `//...` over the + * whole source, as `verify-sse-37.mjs` does -- is wrong for THESE files specifically, because the + * strings they hold are globs: `packages/*\/scripts/*.mjs` contains `/*` and then `*\/`, so a + * block-comment matcher treats the middle of the `files:` array as a comment and deletes the very + * entry this gate exists to find. Looking only at what precedes the match on its own line cannot + * make that mistake. + * + * @param {string} source + * @param {number} index + * @returns {boolean} + */ +function isCommented(source, index) { + const prefix = source.slice(source.lastIndexOf('\n', index) + 1, index); + return ( + prefix.includes('//') || prefix.includes('/*') || /^\s*\*/.test(prefix) + ); +} + +/** + * The array `key` holds inside TOML section `[section]`. + * + * Hand-rolled rather than a TOML dependency: the gate must run with none, and the question is + * narrow -- is this exact key declared in this exact section, and what does it hold. + * + * @param {string} source + * @param {string} section + * @param {string} key + * @returns {{declared: boolean, malformed: boolean, values: string[]}} + */ +function readTomlStringArray(source, section, key) { + const lines = source.split('\n').map(line => stripTomlComment(line).trim()); + let current = ''; + for (let i = 0; i < lines.length; i++) { + const header = /^\[([^\]]+)\]$/.exec(lines[i]); + if (header) { + current = header[1]; + continue; + } + if (current !== section) continue; + const assignment = new RegExp(`^${key}\\s*=\\s*(.*)$`).exec(lines[i]); + if (!assignment) continue; + let raw = assignment[1]; + // Tolerate an array spread over several lines, but stop at anything starting a new key or + // section -- otherwise an unterminated array silently absorbs the next key's value. + while (!raw.includes(']') && i + 1 < lines.length) { + const next = lines[i + 1]; + if (/^\[/.test(next) || /^[\w.-]+\s*=/.test(next)) break; + raw += lines[++i]; + } + if (!raw.includes(']')) + return {declared: true, malformed: true, values: []}; + return { + declared: true, + malformed: false, + values: [...raw.matchAll(/["']([^"']*)["']/g)].map(match => match[1]), + }; + } + return {declared: false, malformed: false, values: []}; +} + +/** + * The string `key` holds inside TOML section `[section]`, or null. + * + * @param {string} source + * @param {string} section + * @param {string} key + * @returns {string | null} + */ +function readTomlString(source, section, key) { + let current = ''; + for (const line of source.split('\n').map(l => stripTomlComment(l).trim())) { + const header = /^\[([^\]]+)\]$/.exec(line); + if (header) { + current = header[1]; + continue; + } + if (current !== section) continue; + const found = new RegExp(`^${key}\\s*=\\s*["']([^"']*)["']`).exec(line); + if (found) return found[1]; + } + return null; +} + +/** + * Path-shaped globs naming the Node tree, lifted out of an arbitrary source. Occurrences, not a + * set: the caller decides whether repetition is meaningful. + * + * Requires a `*`, which is what separates a glob from prose -- `run-ci.mjs` carries + * `node-conformance (matrix)` as a CI step label, and that is not a pattern anything matches + * against. The consequence is that a hard-coded, star-free path slips past, which is acceptable: a + * literal path either resolves or visibly does not, whereas a glob matching nothing fails open. + * + * `skipComments` is what stops a source file from satisfying the check with prose ABOUT its own + * glob: `eslint.config.js`'s comment quotes the `files:` entry beneath it, and without this the + * check stayed green after that entry was deleted -- the sentence explaining the guarantee was what + * voided it. Markdown passes `false`, since there the prose IS the artifact. + * + * @param {string} source + * @param {boolean} [skipComments] + * @returns {string[]} + */ +function extractNodeTreeGlobs(source, skipComments = false) { + const found = []; + for (const match of source.matchAll( + /[\w.*/-]*\/node-conformance\/[\w.*/-]+/g, + )) { + if (!match[0].includes('*')) continue; + if (skipComments && isCommented(source, match.index)) continue; + found.push(match[0]); + } + return found; +} + +/** + * @typedef {object} PartitionSources + * @property {string} bunfig + * @property {string} packageJson + * @property {string} eslintConfig + * @property {string} runCi + * @property {string} readme + * @property {string[]} nodeTreeFiles repo-relative POSIX paths under tests/node-conformance/ + * @property {string[]} bunTreeFiles everything else under tests/ that a runner would collect + * @property {string[]} packageFiles colocated unit tests under packages/ + */ + +/** + * Read every file and tree the checks operate on. + * + * Separated from the checks so `findPartitionViolations` is pure and TOTAL: a caller supplies the + * whole world or none of it, never a mixture that silently reads live repo state it never named. + * + * @param {string} [root] + * @returns {PartitionSources} + */ +export function readPartitionSources(root = REPO_ROOT) { + const read = name => readFileSync(join(root, name), 'utf8'); + const underTests = listFiles(root, TESTS_ROOT); + const inNodeTree = file => file.startsWith(`${NODE_TREE}/`); + return { + bunfig: read(BUNFIG), + packageJson: read(PACKAGE_JSON), + eslintConfig: read(ESLINT_CONFIG), + runCi: read(RUN_CI), + readme: read(README), + nodeTreeFiles: underTests.filter(inNodeTree), + // Everything else under `tests/`, rather than a hardcoded sibling: check 4 has to stay + // meaningful if `tests/conformance/` is renamed, and it covers any future `tests/<other>/` + // for free. + bunTreeFiles: underTests.filter( + file => !inNodeTree(file) && /\.(?:ts|tsx|mjs|cjs|js)$/.test(file), + ), + packageFiles: listFiles(root, PACKAGES_ROOT).filter(file => + file.endsWith('.test.ts'), + ), + }; +} + +// --- the checks --------------------------------------------------------------------------------- + +/** Check 1 — the bunfig key exists, under [test], and is not the silent near-miss. */ +function checkIgnoreKeyDeclared(sources, ignore, fail) { + // Declaration-only, so `bunfig.toml` stays free to NAME the near-miss in a comment. A raw + // substring scan made the one file where that warning belongs the one file forbidden to carry it. + if (readTomlStringArray(sources.bunfig, 'test', DECOY_IGNORE_KEY).declared) { + fail( + 1, + `${BUNFIG} declares \`${DECOY_IGNORE_KEY}\`. Bun does not read that key and does not warn` + + ` about it; the run then collects ${NODE_TREE}/ and reports it passing. The key is` + + ` \`${IGNORE_KEY}\`.`, + ); + } + if (ignore.malformed) { + fail(1, `${BUNFIG}'s \`[test] ${IGNORE_KEY}\` array is not terminated.`); + } else if (!ignore.declared) { + fail( + 1, + `${BUNFIG} has no \`${IGNORE_KEY}\` key under [test]. Without it, \`bun test ./tests\`` + + ` collects ${NODE_TREE}/ and reports node:test files as passing.`, + ); + } else if (ignore.values.length === 0) { + fail(1, `${BUNFIG}'s \`[test] ${IGNORE_KEY}\` is empty.`); + } +} + +/** Check 2 — every file in the Node tree is kept out of `bun test`. */ +function checkNodeTreeIgnored(sources, globs, matchers, fail) { + if (sources.nodeTreeFiles.length === 0) { + fail( + 2, + `${NODE_TREE}/ holds no files. The Node suite is the only thing that runs on Node.`, + ); + } + for (const file of sources.nodeTreeFiles) { + if (!isIgnored(file, matchers)) { + fail( + 2, + `${file} is not matched by \`[test] ${IGNORE_KEY}\` (${globs.join(', ')}), so` + + ' `bun run test` collects it.', + ); + } + } +} + +/** Check 4 — nothing Bun is supposed to run is caught by the ignore glob. */ +function checkOtherTreesNotIgnored(sources, globs, matchers, fail) { + if (sources.bunTreeFiles.length === 0) { + fail( + 4, + `${TESTS_ROOT}/ holds no Bun-runner files outside ${NODE_TREE}/. Either the Bun suite moved` + + ' or it is gone; either way this check has stopped meaning anything.', + ); + } + for (const file of [...sources.bunTreeFiles, ...sources.packageFiles]) { + if (isIgnored(file, matchers)) { + fail( + 4, + `${file} IS matched by \`[test] ${IGNORE_KEY}\` (${globs.join(', ')}), so \`bun run test\`` + + ' silently skips it.', + ); + } + } +} + +/** Check 3 — every file in the Node tree is either the README or reached by `test:node`. */ +function checkRunnerReachesEveryCase(sources, fail) { + const testNode = JSON.parse(sources.packageJson).scripts?.['test:node']; + if (typeof testNode !== 'string') { + fail(3, `${PACKAGE_JSON} has no \`test:node\` script.`); + return; + } + const globs = testNode.split(/\s+/).filter(token => token.endsWith('.mjs')); + if (globs.length === 0) { + fail(3, `\`test:node\` names no .mjs path: ${testNode}`); + return; + } + const matchers = globs.map(globToRegExp); + for (const file of sources.nodeTreeFiles) { + if (file === README || matchers.some(matcher => matcher.test(file))) { + continue; + } + fail( + 3, + `${file} is not matched by \`test:node\` (${globs.join(', ')}), so no command in the repo` + + ' runs it. `node --test` over a glob matching nothing exits 0, so this is silent.', + ); + } +} + +/** Check 5 — every Node-tree glob the other four files carry still reaches the whole suite. */ +function checkDocumentedGlobs(sources, fail) { + const cases = sources.nodeTreeFiles.filter(file => + file.endsWith('.test.mjs'), + ); + const sites = [ + [RUN_CI, sources.runCi, true, RUN_CI_MIN_GLOBS], + [ESLINT_CONFIG, sources.eslintConfig, true, 1], + // Prose, not code: the README names the tree to its reader, so its comments are not skipped. + [README, sources.readme, false, 1], + ]; + for (const [name, source, skipComments, minimum] of sites) { + const globs = extractNodeTreeGlobs(source, skipComments); + if (globs.length < minimum) { + fail( + 5, + `${name} carries ${globs.length} ${NODE_TREE}/ glob(s), expected at least ${minimum}.` + + ' Losing one fails open.', + ); + } + for (const glob of new Set(globs)) { + const matcher = globToRegExp(glob); + const missed = cases.filter(file => !matcher.test(file)); + if (missed.length > 0) { + fail( + 5, + `${name}'s glob \`${glob}\` misses ${missed.length} of ${cases.length} cases under` + + ` ${NODE_TREE}/, starting with ${missed[0]}.`, + ); + } + } + } +} + +/** Checks 6 and 7 — the two further things CLAUDE.md's hard rule tells the reader to keep. */ +function checkRootScriptAndBunRoot(sources, fail) { + const test = JSON.parse(sources.packageJson).scripts?.test; + // Whole arguments, never a substring: `bun test ./packages ./tests/conformance` CONTAINS + // `./tests` while being precisely the narrowing the hard rule forbids -- it protects the root + // script and leaves a hand-typed `bun test ./tests` collecting the Node suite. + const args = typeof test === 'string' ? test.split(/\s+/) : []; + if (typeof test !== 'string') { + fail(6, `${PACKAGE_JSON} has no \`test\` script.`); + } else if ( + !args.includes(`./${PACKAGES_ROOT}`) || + !args.includes(`./${TESTS_ROOT}`) + ) { + fail( + 6, + `\`test\` must pass both trees whole (\`./${PACKAGES_ROOT} ./${TESTS_ROOT}\`); it is` + + ` \`${test}\`. A bare \`bun test\` never visits ${TESTS_ROOT}/, and narrowing to a subtree` + + ` protects only this script -- a hand-typed \`bun test ./${TESTS_ROOT}\` still collects` + + ` ${NODE_TREE}/.`, + ); + } + const root = readTomlString(sources.bunfig, 'test', 'root'); + if (root !== EXPECTED_BUN_ROOT) { + fail( + 7, + `${BUNFIG}'s \`[test] root\` is ${root === null ? 'absent' : `"${root}"`}, expected` + + ` "${EXPECTED_BUN_ROOT}". It scopes a bare \`bun test\` and keeps scripts/*.test.mjs out of` + + " that run's coverage floor.", + ); + } +} + +/** + * Checks 1-5 are issue #55's. 6 and 7 guard two further rules CLAUDE.md's hard rule states and + * nothing enforced: the root script must name both trees, and `[test] root` must stay `"packages"`. + * + * @param {PartitionSources} sources + * @returns {{check: number, message: string}[]} empty when the partition holds + */ +export function findPartitionViolations(sources) { + const violations = []; + const fail = (check, message) => violations.push({check, message}); + + const ignore = readTomlStringArray(sources.bunfig, 'test', IGNORE_KEY); + checkIgnoreKeyDeclared(sources, ignore, fail); + if (ignore.declared && !ignore.malformed) { + const matchers = ignore.values.map(globToRegExp); + checkNodeTreeIgnored(sources, ignore.values, matchers, fail); + checkOtherTreesNotIgnored(sources, ignore.values, matchers, fail); + } + checkRunnerReachesEveryCase(sources, fail); + checkDocumentedGlobs(sources, fail); + checkRootScriptAndBunRoot(sources, fail); + + return violations; +} + +// --- CLI ----------------------------------------------------------------------------------------- + +const isDirect = + process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]; + +if (isDirect) { + const violations = findPartitionViolations(readPartitionSources()); + if (violations.length > 0) { + for (const {check, message} of violations) { + console.error(`test-partition violation (check ${check}): ${message}`); + } + console.error( + `\nThe ${TESTS_ROOT}/conformance/ and ${NODE_TREE}/ suites must never run together. Five` + + ` files hold that apart and they must agree: ${BUNFIG}, ${PACKAGE_JSON},` + + ` ${ESLINT_CONFIG}, ${RUN_CI}, and ${README}. Change one, then change all of them. See` + + ' CLAUDE.md, "HARD RULE -- the `tests/` partition".', + ); + process.exit(1); + } + console.log( + `test-partition OK: ${BUNFIG}, ${PACKAGE_JSON}, ${ESLINT_CONFIG}, ${RUN_CI} and ${README}` + + ` agree on ${NODE_TREE}/, and nothing else under ${TESTS_ROOT}/ is caught by the ignore glob.`, + ); +} diff --git a/scripts/verify-test-partition.test.mjs b/scripts/verify-test-partition.test.mjs new file mode 100644 index 0000000..89d82e0 --- /dev/null +++ b/scripts/verify-test-partition.test.mjs @@ -0,0 +1,534 @@ +// SPDX-License-Identifier: MIT +// scripts/verify-test-partition.test.mjs +// +// Tests the GATE, not a copy of its logic. The CLI half spawns the real script against throwaway +// fixture trees and reads its exit code and output, following `verify-seam-1.test.mjs`: a suite that +// only calls the detector passes just as happily when the CLI has stopped exiting non-zero, which is +// the one failure that would leave CI green over a dead gate. The detector half then drives the +// individual checks, which is where the interesting inputs are. +// +// Lives in `scripts/` and runs under `node --test` via `bun run test:scripts`, so `node:fs` and +// `node:child_process` are permitted here -- the zero-`node:` invariant governs `packages/*/src`, +// not build tooling. +import assert from 'node:assert/strict'; +import {spawnSync} from 'node:child_process'; +import { + copyFileSync, + mkdirSync, + mkdtempSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {dirname, join} from 'node:path'; +import {test} from 'node:test'; +import {fileURLToPath} from 'node:url'; + +import {findPartitionViolations} from './verify-test-partition.mjs'; + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)); +const script = join(repoRoot, 'scripts', 'verify-test-partition.mjs'); + +// --- the shape of a well-formed repo, as file contents and as a sources object -------------------- + +const BUNFIG = [ + '[test]', + 'root = "packages"', + 'pathIgnorePatterns = ["tests/node-conformance/**"]', + 'coverage = true', +].join('\n'); + +const PACKAGE_JSON = JSON.stringify({ + scripts: { + test: 'bun test ./packages ./tests', + 'test:node': 'node --test tests/node-conformance/*.test.mjs', + }, +}); + +const ESLINT_CONFIG = + "export default [{files: ['tests/node-conformance/*.mjs']}];"; + +// Three, because `run-ci.mjs`'s `--node-floor` leg carries one per version manager. +const RUN_CI = [ + '`mise x node@20.3.0 -- node --test tests/node-conformance/*.test.mjs`,', + '`fnm exec --using=20.3.0 node --test tests/node-conformance/*.test.mjs`,', + "`bash -lc 'nvm exec 20.3.0 node --test tests/node-conformance/*.test.mjs'`,", +].join('\n'); + +const README = + 'Run by `bun run test:node` (`node --test tests/node-conformance/*.test.mjs`).'; + +const NODE_CASES = [ + 'tests/node-conformance/retry.test.mjs', + 'tests/node-conformance/seams.test.mjs', +]; + +/** A complete, well-formed sources object; overrides replace individual fields. */ +function sources(overrides = {}) { + return { + bunfig: BUNFIG, + packageJson: PACKAGE_JSON, + eslintConfig: ESLINT_CONFIG, + runCi: RUN_CI, + readme: README, + nodeTreeFiles: [...NODE_CASES, 'tests/node-conformance/README.md'], + bunTreeFiles: ['tests/conformance/xcut/retry-safety.conformance.test.ts'], + packageFiles: ['packages/core/src/http/headers.test.ts'], + ...overrides, + }; +} + +/** Which checks fired, each once — one drifted string usually trips its check per file. */ +const checks = violations => [...new Set(violations.map(v => v.check))].sort(); + +/** The message a given check produced, so no assertion depends on array position. */ +const messageFor = (violations, check) => + violations.find(v => v.check === check)?.message ?? ''; + +// --- the CLI, driven end to end ------------------------------------------------------------------- + +// Copies the real script into a throwaway tree and runs it there. `REPO_ROOT` is resolved from the +// script's own location, so the copy reads the fixture's files rather than this repository's -- the +// same trick `verify-seam-1.test.mjs` uses, and what makes the failure path reachable through the +// ACTUAL CLI, exit code included. +function runAgainstFixture(files) { + const dir = mkdtempSync(join(tmpdir(), 'dexpace-partition-')); + mkdirSync(join(dir, 'scripts'), {recursive: true}); + copyFileSync(script, join(dir, 'scripts', 'verify-test-partition.mjs')); + for (const [name, contents] of Object.entries(files)) { + const target = join(dir, name); + mkdirSync(dirname(target), {recursive: true}); + writeFileSync(target, contents); + } + try { + const result = spawnSync( + process.execPath, + [join(dir, 'scripts', 'verify-test-partition.mjs')], + {encoding: 'utf8'}, + ); + return {status: result.status, output: `${result.stdout}${result.stderr}`}; + } finally { + rmSync(dir, {recursive: true, force: true}); + } +} + +/** A fixture tree the gate should accept. */ +function cleanFixture(overrides = {}) { + return { + 'bunfig.toml': BUNFIG, + 'package.json': PACKAGE_JSON, + 'eslint.config.js': ESLINT_CONFIG, + '.claude/skills/ci-preflight/run-ci.mjs': RUN_CI, + 'tests/node-conformance/README.md': README, + 'tests/node-conformance/retry.test.mjs': '// a case', + 'tests/node-conformance/seams.test.mjs': '// a case', + 'tests/conformance/xcut/a.conformance.test.ts': '// a case', + 'packages/core/src/headers.test.ts': '// a case', + ...overrides, + }; +} + +test('the CLI exits 0 and names all five files when the partition holds', () => { + const {status, output} = runAgainstFixture(cleanFixture()); + assert.equal(status, 0, output); + assert.match(output, /test-partition OK:/); + for (const named of [ + 'bunfig.toml', + 'package.json', + 'eslint.config.js', + 'run-ci.mjs', + 'README.md', + ]) { + assert.ok( + output.includes(named), + `${named} missing from the OK line:\n${output}`, + ); + } +}); + +test('the CLI exits 1 and names the failing check when a string has drifted', () => { + const {status, output} = runAgainstFixture( + cleanFixture({'bunfig.toml': '[test]\nroot = "packages"\n'}), + ); + assert.equal(status, 1, `expected a non-zero exit:\n${output}`); + assert.match(output, /test-partition violation \(check 1\)/); + assert.match(output, /Change one, then change all of them/); +}); + +test('the CLI exits 0 on this repository as committed', () => { + const result = spawnSync(process.execPath, [script], { + encoding: 'utf8', + cwd: repoRoot, + }); + assert.equal(result.status, 0, `${result.stdout}${result.stderr}`); +}); + +// --- check 1: the bunfig key ---------------------------------------------------------------------- + +test('the detector fails when bunfig declares testPathIgnorePatterns in place of pathIgnorePatterns', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\ntestPathIgnorePatterns = ["tests/node-conformance/**"]', + }), + ); + assert.match(messageFor(found, 1), /declares `testPathIgnorePatterns`/); +}); + +test('the detector fails when the decoy key is declared alongside the correct one', () => { + const found = findPartitionViolations( + sources({bunfig: `${BUNFIG}\ntestPathIgnorePatterns = ["x/**"]`}), + ); + assert.match(messageFor(found, 1), /declares `testPathIgnorePatterns`/); +}); + +test('the detector stays quiet when the decoy key is only NAMED in a comment', () => { + // bunfig.toml must be free to explain the hazard it is configured against. A gate that fails on + // its own requirement's explanation is a gate the next person deletes instead of the comment. + const found = findPartitionViolations( + sources({ + bunfig: `${BUNFIG}\n# The key is not testPathIgnorePatterns; Bun would ignore that in silence.`, + }), + ); + assert.deepEqual(found, []); +}); + +test('the detector fails when the ignore key is absent', () => { + const found = findPartitionViolations( + sources({bunfig: '[test]\nroot = "packages"\n'}), + ); + assert.ok(checks(found).includes(1)); +}); + +test('the detector fails when the ignore key holds an empty array', () => { + const found = findPartitionViolations( + sources({bunfig: '[test]\nroot = "packages"\npathIgnorePatterns = []'}), + ); + assert.ok(checks(found).includes(1)); +}); + +test('the detector fails when the ignore key sits under a section other than [test]', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\n[install]\npathIgnorePatterns = ["tests/node-conformance/**"]', + }), + ); + assert.ok(checks(found).includes(1)); +}); + +test('the detector fails when the ignore array is left unterminated', () => { + // The continuation reader must not swallow the next key's value and report a plausible array. + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = [\n"tests/node-conformance/**"\ncoverage = true', + }), + ); + assert.match(messageFor(found, 1), /not terminated/); +}); + +test('the detector accepts a # inside a quoted pattern rather than truncating the line', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/node-conformance/**", "a/#b/**"]', + }), + ); + assert.deepEqual(found, []); +}); + +// --- check 2: the Node tree stays out of `bun test` ------------------------------------------------ + +test('the detector fails when a Node file escapes the ignore glob', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/node-conformance/*.test.mjs"]', + }), + ); + assert.match(messageFor(found, 2), /README\.md is not matched/); +}); + +test('the detector accepts a bare directory pattern, which is what Bun prunes on', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/node-conformance"]', + }), + ); + assert.deepEqual(found, []); +}); + +test('the detector fails when the Node tree holds no files', () => { + const found = findPartitionViolations(sources({nodeTreeFiles: []})); + assert.ok(checks(found).includes(2)); +}); + +test('the detector treats ** as spanning whole segments, the way Bun does', () => { + // `tests/**/s.test.mjs` must not cover `tests/node-conformance/wideSs.test.mjs`. Compiling `**` + // to a bare `.*` matched it and Bun does not -- the direction that green-lights a config Bun + // reads differently. + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/**/s.test.mjs"]', + nodeTreeFiles: ['tests/node-conformance/wideSs.test.mjs'], + }), + ); + assert.ok(checks(found).includes(2)); +}); + +// --- check 3: every case is reachable by the runner ------------------------------------------------ + +test('the detector fails when a Node case sits in a subdirectory the runner glob cannot reach', () => { + const found = findPartitionViolations( + sources({ + nodeTreeFiles: [ + ...NODE_CASES, + 'tests/node-conformance/io/byte-stream.test.mjs', + ], + }), + ); + assert.match( + messageFor(found, 3), + /io\/byte-stream\.test\.mjs is not matched/, + ); +}); + +test('the detector fails when a case is misnamed so no runner glob reaches it', () => { + // Ignored by Bun, unmatched by `test:node`, run by nothing — and `node --test` over a glob that + // matches nothing exits 0, so without this the file is simply never mentioned again. + for (const orphan of [ + 'tests/node-conformance/retry.mjs', + 'tests/node-conformance/orphan.test.ts', + ]) { + const found = findPartitionViolations( + sources({nodeTreeFiles: [...NODE_CASES, orphan]}), + ); + assert.ok(checks(found).includes(3), `${orphan} slipped through`); + } +}); + +test('the detector exempts the README from the runner glob', () => { + assert.deepEqual(findPartitionViolations(sources()), []); +}); + +test('the detector fails when test:node points at the pre-Phase-10 tree', () => { + const found = findPartitionViolations( + sources({ + packageJson: JSON.stringify({ + scripts: { + test: 'bun test ./packages ./tests', + 'test:node': 'node --test test/node-conformance/*.test.mjs', + }, + }), + }), + ); + assert.ok(checks(found).includes(3)); +}); + +test('the detector fails when package.json carries no test:node script at all', () => { + const found = findPartitionViolations( + sources({ + packageJson: JSON.stringify({ + scripts: {test: 'bun test ./packages ./tests'}, + }), + }), + ); + assert.match(messageFor(found, 3), /no `test:node` script/); +}); + +test('the detector fails when test:node names no .mjs path', () => { + const found = findPartitionViolations( + sources({ + packageJson: JSON.stringify({ + scripts: { + test: 'bun test ./packages ./tests', + 'test:node': 'node --test', + }, + }), + }), + ); + assert.match(messageFor(found, 3), /names no \.mjs path/); +}); + +// --- check 4: nothing else is caught by the ignore glob -------------------------------------------- + +test('the detector fails when the ignore glob widens over the Bun tree', () => { + const found = findPartitionViolations( + sources({ + bunfig: '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/**"]', + }), + ); + assert.ok(checks(found).includes(4)); +}); + +test('the detector fails when a bare directory prunes part of the Bun tree', () => { + // Bun applies these patterns while walking, so naming a directory drops everything beneath it + // without matching any file path. Measured: adding `tests/conformance/fixtures` silently removed + // `fixtures/settle.test.mjs` from the run while every file-path check stayed green. + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/node-conformance/**", "tests/conformance/xcut"]', + }), + ); + assert.match(messageFor(found, 4), /silently skips it/); +}); + +test('the detector fails when the ignore glob reaches the packages tree', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/node-conformance/**", "packages/**"]', + }), + ); + assert.ok(checks(found).includes(4)); +}); + +test('the detector fails when the Bun tree has vanished, rather than passing vacuously', () => { + const found = findPartitionViolations( + sources({ + bunfig: '[test]\nroot = "packages"\npathIgnorePatterns = ["tests/**"]', + bunTreeFiles: [], + }), + ); + assert.ok(checks(found).includes(4)); +}); + +// --- check 5: the globs the other files carry ------------------------------------------------------ + +test('the detector fails when run-ci.mjs still names the pre-Phase-10 tree', () => { + const found = findPartitionViolations( + sources({runCi: '`node --test test/node-conformance/*.test.mjs`'}), + ); + assert.ok(checks(found).includes(5)); +}); + +test('the detector fails when run-ci.mjs loses one of its three globs', () => { + const found = findPartitionViolations( + sources({runCi: RUN_CI.split('\n').slice(0, 2).join('\n')}), + ); + assert.match(messageFor(found, 5), /expected at least 3/); +}); + +test('the detector fails when a glob reaches only some of the cases', () => { + const narrowed = RUN_CI.replaceAll( + 'tests/node-conformance/*.test.mjs', + 'tests/node-conformance/seams*.test.mjs', + ); + const found = findPartitionViolations(sources({runCi: narrowed})); + assert.match(messageFor(found, 5), /misses 1 of 2 cases/); +}); + +test('the detector fails when the eslint override glob goes stale', () => { + const found = findPartitionViolations( + sources({eslintConfig: "files: ['tests/node-conformance/*.cjs'],"}), + ); + assert.ok(checks(found).includes(5)); +}); + +test('the detector fails when the eslint override is deleted, leaving only prose about it', () => { + // The comment above that entry quotes the glob. Scanning source and comments alike kept this + // green after the real entry was gone: the sentence explaining the guarantee was what voided it. + const found = findPartitionViolations( + sources({ + eslintConfig: [ + '// The `tests/node-conformance/*.mjs` entry is one of the five strings.', + "export default [{files: ['scripts/*.mjs']}];", + ].join('\n'), + }), + ); + assert.match( + messageFor(found, 5), + /carries 0 tests\/node-conformance\/ glob/, + ); +}); + +test('the detector fails when the README stops naming the tree it documents', () => { + const found = findPartitionViolations( + sources({readme: 'This suite runs on Node. Nothing here names a path.'}), + ); + assert.ok(checks(found).includes(5)); +}); + +test('the detector does not mistake a run-ci step label for a glob', () => { + // `node-conformance (matrix)` carries no slashes, so it never reaches the star filter. + const found = findPartitionViolations( + sources({ + runCi: [ + " {ci: 'node-conformance (matrix)', cmd: 'bun run test:node'},", + RUN_CI, + ].join('\n'), + }), + ); + assert.deepEqual(found, []); +}); + +// --- checks 6 and 7: the two further rules the hard rule states ------------------------------------ + +test('the detector fails when the root test script stops naming both trees', () => { + const found = findPartitionViolations( + sources({ + packageJson: JSON.stringify({ + scripts: { + test: 'bun test', + 'test:node': 'node --test tests/node-conformance/*.test.mjs', + }, + }), + }), + ); + assert.match(messageFor(found, 6), /must pass both trees whole/); +}); + +test('the detector fails when the root test script is narrowed to one subtree', () => { + const found = findPartitionViolations( + sources({ + packageJson: JSON.stringify({ + scripts: { + test: 'bun test ./packages ./tests/conformance', + 'test:node': 'node --test tests/node-conformance/*.test.mjs', + }, + }), + }), + ); + assert.ok(checks(found).includes(6)); +}); + +test('the detector fails when bunfig loses its [test] root key', () => { + const found = findPartitionViolations( + sources({ + bunfig: '[test]\npathIgnorePatterns = ["tests/node-conformance/**"]', + }), + ); + assert.match(messageFor(found, 7), /`\[test\] root` is absent/); +}); + +test('the detector fails when [test] root is repointed away from packages', () => { + const found = findPartitionViolations( + sources({ + bunfig: + '[test]\nroot = "."\npathIgnorePatterns = ["tests/node-conformance/**"]', + }), + ); + assert.match(messageFor(found, 7), /expected "packages"/); +}); + +// --- several at once ------------------------------------------------------------------------------- + +test('the detector reports every drifted string, not only the first', () => { + const found = findPartitionViolations( + sources({ + bunfig: '[test]\npathIgnorePatterns = ["tests/node-conformance/**"]', + packageJson: JSON.stringify({ + scripts: { + test: 'bun test', + 'test:node': 'node --test test/node-conformance/*.test.mjs', + }, + }), + eslintConfig: "files: ['test/node-conformance/*.mjs'],", + }), + ); + assert.deepEqual(checks(found), [3, 5, 6, 7]); +}); diff --git a/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts new file mode 100644 index 0000000..a97008a --- /dev/null +++ b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts @@ -0,0 +1,254 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts +// Exercises: XCUT-1 (cancellation is terminal, non-retryable, and the ambient cancel flag survives), +// XCUT-3 (an inter-attempt retry wait is promptly cancellable and surfaces the cancellation signal, +// not a spurious timeout). +// XCUT-2 stays a retrofit citation at its Phase 2 source (packages/core/src/seams/transport.test.ts), +// where isTimeoutSignal's two branches are asserted directly. +// +// These run the invariants through the fully composed retry+redirect+auth+logging pipeline over a +// real socket, which is what this suite adds over 5a's and Phase 2's own unit-level coverage. +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + CancellationError, + HttpStatusError, + Request, + retryAttempts, +} from '@dexpace/core'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; + +let server: XcutFixtureServer; + +/** + * Walks everything a surfaced failure can nest a prior error under, visiting by identity so a cyclic + * chain terminates -- the same discipline `XCUT-9` puts on the classifier. + * + * Kept after #72 made the top-level assertion possible, because the two answer different questions. + * `CancellationError` carries the raw `AbortSignal.reason` as its `cause`, so the walk is what + * proves the ambient abort was not swallowed on the way out, and it is also what catches a + * `TimeoutError` hiding one hop down where `XCUT-3` forbids one. What it must no longer be is the + * ONLY assertion: 5a's engine folded the retry trail into a `SuppressedError` (`RETRY-34`), the + * cancellation arrived as `.error` beneath that wrapper, and a chain walk was the only way to find + * it -- which is precisely the defect, since a caller writing `catch (e) { e instanceof + * CancellationError }` has no walk. Every row below asserts the top level too. + */ +function* chainOf(error: unknown): Generator { + const seen = new Set<unknown>(); + const queue: unknown[] = [error]; + while (queue.length > 0) { + const current = queue.shift(); + if (current === null || current === undefined || seen.has(current)) + continue; + seen.add(current); + yield current; + if (typeof current !== 'object') continue; + const node = current as Record<string, unknown>; + queue.push(node.cause, node.error, node.suppressed); + } +} + +/** True when anything in the chain is a caller cancellation, by type or by `AbortSignal` reason name. */ +function carriesCancellation(error: unknown): boolean { + for (const link of chainOf(error)) { + if (link instanceof CancellationError) return true; + if ((link as {name?: unknown} | null)?.name === 'AbortError') return true; + } + return false; +} + +/** True when the chain carries the SDK's own terminal cancellation type, not merely an abort. */ +function carriesSdkCancellationError(error: unknown): boolean { + for (const link of chainOf(error)) { + if (link instanceof CancellationError) return true; + } + return false; +} + +/** True when anything in the chain is a timeout -- the classification `XCUT-3` forbids here. */ +function carriesTimeout(error: unknown): boolean { + for (const link of chainOf(error)) { + if ((link as {name?: unknown} | null)?.name === 'TimeoutError') return true; + } + return false; +} + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +describe('XCUT-1: cancellation is terminal and never retried', () => { + test('surfaces CancellationError when an in-flight request is aborted', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 3, initialDelayMs: 1}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/slow?ms=2000`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 25).unref(); + + expect(await rejectionOf(pending)).toBeInstanceOf(CancellationError); + + await pipeline.close(); + }); + + test('does not re-dispatch a cancelled request when retries remain', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 3, initialDelayMs: 1}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/slow?ms=2000`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 25).unref(); + await pending.catch(() => undefined); + + // The retry pillar had two attempts left and must not have spent them: XCUT-1 makes + // cancellation terminal at the condition level, distinct from the safety gate. + expect(pipeline.dispatches()).toBe(1); + + await pipeline.close(); + }); + + test('leaves the ambient cancellation flag set after the error surfaces', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/slow?ms=2000`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 25).unref(); + await pending.catch(() => undefined); + + // The JVM reference re-asserts the interrupt flag; the port's equivalent is that the signal it + // was handed is still aborted, never reset on the way out (deviation ledger item 11). + expect(controller.signal.aborted).toBe(true); + + await pipeline.close(); + }); +}); + +describe('XCUT-3: an inter-attempt wait is promptly cancellable', () => { + test('aborts a 60s backoff near-immediately instead of waiting it out', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const startedAt = Date.now(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + await pending.catch(() => undefined); + + // Nowhere near the 60s backoff: the wait aborted rather than expiring. + expect(Date.now() - startedAt).toBeLessThan(5_000); + + await pipeline.close(); + }); + + test('surfaces a cancellation, not a spurious timeout, from inside the wait', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + const surfaced = await rejectionOf(pending); + + // XCUT-1's conformance clause, at the top level and unqualified: "assert the surfaced error is + // the cancellation type". Two separate defects had to be fixed for this line to hold. Until + // 2026-09-02 the engine surfaced `signal.reason` verbatim, so a cancelled backoff arrived as a + // bare DOMException `AbortError` while the transport mapped the identical abort to + // `CancellationError` -- one requirement, two types, depending on which layer noticed. Until + // 2026-09-05 the mapping was then undone one line later by the retry trail's `SuppressedError` + // wrapper, and a cancelled backoff ALWAYS has a non-empty trail, so this row was false for + // every reachable case. + expect(surfaced).toBeInstanceOf(CancellationError); + // The chain still has to carry the raw abort (the ambient flag survived the mapping) and must + // not carry a timeout, which is XCUT-3's own letter. + expect(carriesCancellation(surfaced)).toBe(true); + expect(carriesSdkCancellationError(surfaced)).toBe(true); + expect(carriesTimeout(surfaced)).toBe(false); + + await pipeline.close(); + }); + + test('does not dispatch a further attempt once the wait is cancelled', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + await pending.catch(() => undefined); + + // One dispatch produced the 500; the cancelled wait must not produce a second. + expect(pipeline.dispatches()).toBe(1); + + await pipeline.close(); + }); +}); + +describe('RETRY-34: the trail survives the unwrapped cancellation', () => { + test('the attempt the cancelled wait was scheduled for stays reachable', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + const surfaced = await rejectionOf(pending); + + // Surfacing the cancellation unwrapped is not allowed to LOSE the 500 that provoked the retry + // in the first place -- that was the one thing the `SuppressedError` wrapper did buy. It rides + // in the trail instead, and the retired response is a buffered `HttpStatusError` (RECOV-16). + const priors = retryAttempts(surfaced); + expect(priors).toHaveLength(1); + expect(priors[0]).toBeInstanceOf(HttpStatusError); + expect((priors[0] as HttpStatusError).status).toBe(500); + + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/concurrency-and-lifecycle.conformance.test.ts b/tests/conformance/xcut/concurrency-and-lifecycle.conformance.test.ts new file mode 100644 index 0000000..0d83eb9 --- /dev/null +++ b/tests/conformance/xcut/concurrency-and-lifecycle.conformance.test.ts @@ -0,0 +1,142 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/concurrency-and-lifecycle.conformance.test.ts +// Exercises: XCUT-11 (a shared, reusable pipeline instance is safe under concurrent invocation and +// keeps per-call state on the call, not the instance), XCUT-13 (close is idempotent and does not +// block). +// +// XCUT-12, XCUT-14 and XCUT-22 stay retrofit citations at their own phases' tests, which assert them +// better than anything reachable from here could: +// XCUT-12 -> packages/core/src/auth/bearer-cache.test.ts (N concurrent callers coalesce to ONE +// provider invocation, in both the expired and post-eviction zones) +// XCUT-14 -> packages/core/src/context/store.test.ts ("a burst of inserts past the cap converges +// the store to at or under the cap") and packages/core/src/auth/digest.test.ts (the +// 1024-entry nonce counter, drain-to-cap under a long run of fresh nonces) +// XCUT-22 -> packages/transport-undici/src/undici-transport.test.ts ("a bring-your-own dispatcher +// is never closed by the transport") +// Neither bounded map is reachable from a consumer-shaped test -- `contextStore` and +// `NonceCountStore` are both absent from core's barrel -- so a burst driven from out here could only +// assert that the stack stays alive, never that a cap held. Asserting the cap where it is observable +// and citing it from here is the honest split. +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import {Headers, Request} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; + +let server: XcutFixtureServer; + +/** Enough concurrency to interleave, small enough not to pace the suite. */ +const CONCURRENCY = 24; + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +describe('XCUT-11: one shared pipeline instance is safe under concurrent use', () => { + test('answers 24 interleaved requests without pairing any response to the wrong request', async () => { + const pipeline = buildComposedPipeline(); + const requests = Array.from({length: CONCURRENCY}, (_, index) => + Request.newBuilder() + .url(`${server.url}/echo?n=${String(index)}`) + .headers( + Headers.newBuilder() + .set('x-correlation', `call-${String(index)}`) + .build(), + ) + .build(), + ); + + const bodies = await Promise.all( + requests.map(async request => { + const response = await pipeline.runtime.send(request); + const text = await response.text(); + await response.close(); + return JSON.parse(text) as {query: Record<string, string>}; + }), + ); + + // Cross-talk would show up as a response carrying another call's correlation value: per-call + // state (attempt counters, deadlines, seen-URI sets) has to live on the call, not the instance. + expect(bodies.map(body => body.query.n)).toEqual( + Array.from({length: CONCURRENCY}, (_, index) => String(index)), + ); + + await pipeline.close(); + }); + + test('dispatches exactly one attempt per concurrent call, with no double-sends', async () => { + const pipeline = buildComposedPipeline(); + const requests = Array.from({length: CONCURRENCY}, (_, index) => + Request.newBuilder() + .url(`${server.url}/ok?n=${String(index)}`) + .build(), + ); + + const responses = await Promise.all( + requests.map(request => pipeline.runtime.send(request)), + ); + await Promise.all(responses.map(response => response.close())); + + expect(pipeline.dispatches()).toBe(CONCURRENCY); + await pipeline.close(); + }); +}); + +describe('XCUT-13: close is idempotent and non-blocking', () => { + test('closing a real transport twice makes the second call a no-op', async () => { + const transport = fetchTransport(); + + await transport.close(); + await transport.close(); + + // Reaching this line is the assertion: the second close neither threw nor hung. + expect(true).toBe(true); + }); + + test('closing a composed pipeline twice makes the second call a no-op', async () => { + const pipeline = buildComposedPipeline(); + + await pipeline.runtime.close(); + await pipeline.runtime.close(); + + // Reaching this line is the assertion: the second close neither threw nor hung -- the same + // shape the transports' own TRANSPORT-16 rows use. + expect(pipeline.dispatches()).toBe(0); + + await pipeline.close(); + }); + + test('closing the pipeline leaves the caller-supplied transport usable (XCUT-22 at pipeline level)', async () => { + const pipeline = buildComposedPipeline(); + + await pipeline.runtime.close(); + + // PIPE-27: the pipeline never OWNS its terminal transport, so `Runtime.close()` is deliberately + // a no-op and the transport a caller handed it stays usable. That is XCUT-22's "close only what + // you created" applied one level up -- and the trap it implies is real: a consumer who only ever + // calls `runtime.close()` never closes the transport. Asserted, not assumed. + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/ok`).build(), + ); + expect(response.status.code).toBe(200); + + await response.close(); + await pipeline.close(); + }); + + test('does not clear an already-aborted signal on the way out', async () => { + const pipeline = buildComposedPipeline(); + const controller = new AbortController(); + controller.abort(); + + await pipeline.runtime.close(); + + // XCUT-13's "preserves the ambient interrupt/cancel flag as-is" half. + expect(controller.signal.aborted).toBe(true); + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/diagnostic-previews.conformance.test.ts b/tests/conformance/xcut/diagnostic-previews.conformance.test.ts new file mode 100644 index 0000000..3a3bfb6 --- /dev/null +++ b/tests/conformance/xcut/diagnostic-previews.conformance.test.ts @@ -0,0 +1,122 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/diagnostic-previews.conformance.test.ts +// Exercises: XCUT-24 (a diagnostic preview of a caller- or server-controlled payload is byte-capped +// and non-consuming -- it must not materialize an unbounded payload, and must not disturb the +// primary read path the consumer will use). +// +// Diagnostic previews are not a standalone Response method in this port: they surface through 7b's +// LOGGING step at `granularity: 'body'`, which tees the body bounded to `previewSizeBytes` into the +// emitted `http.response` event (OBS-36). 7b's own logging-step.test.ts asserts that against a fake +// transport and a 50 KB in-memory string; this file runs XCUT-24's own conformance clause verbatim -- +// "a 10 MB response with a small cap" -- over a real socket through the whole composed pipeline, +// which is where a tee that buffered the entire body would actually show up. +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import {Request, createLogger, type LogLevel} from '@dexpace/core'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; + +let server: XcutFixtureServer; + +/** XCUT-24's own figure: "take a body snapshot/preview of a 10 MB response with a small cap". */ +const BODY_BYTES = 10 * 1024 * 1024; +const PREVIEW_CAP = 1024; + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +/** Collects every emitted event as a plain field map, the same shape 7b's own spy logger uses. */ +function spyLogger(): { + logger: ReturnType<typeof createLogger>; + events: Record<string, unknown>[]; +} { + const events: Record<string, unknown>[] = []; + const logger = createLogger((_level: LogLevel, fields) => { + events.push(Object.fromEntries(fields)); + }); + return {logger, events}; +} + +/** Drives a 10 MB response through the composed pipeline with body logging capped low. */ +async function captureLargeBody( + mediaType = 'application/octet-stream', +): Promise<{ + events: Record<string, unknown>[]; + bodyLength: number; +}> { + const {logger, events} = spyLogger(); + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + logging: {logger, granularity: 'body', previewSizeBytes: PREVIEW_CAP}, + }); + try { + const response = await pipeline.runtime.send( + Request.newBuilder() + .url( + `${server.url}/large-body?bytes=${String(BODY_BYTES)}&type=${encodeURIComponent(mediaType)}`, + ) + .build(), + ); + const body = await response.bytes(); + await response.close(); + return {events, bodyLength: body.byteLength}; + } finally { + await pipeline.close(); + } +} + +describe('XCUT-24: a diagnostic preview is byte-capped', () => { + test('caps a decoded text preview at previewSizeBytes across a 10 MB body', async () => { + const {events} = await captureLargeBody('text/plain'); + + const responseEvent = events.find(event => event.event === 'http.response'); + expect(String(responseEvent?.['http.response.body.preview'])).toHaveLength( + PREVIEW_CAP, + ); + }); + + test('caps a binary body at the same figure, reported as a size-only marker', async () => { + const {events} = await captureLargeBody('application/octet-stream'); + + const responseEvent = events.find(event => event.event === 'http.response'); + // OBS-38: a binary payload is never decoded into the log. The marker still has to report a + // capped capture, which is the half XCUT-24 cares about. + expect(responseEvent?.['http.response.body.preview']).toBe( + `[binary ${String(PREVIEW_CAP)} bytes captured]`, + ); + }); + + test('reports the captured size as the cap, not the payload size', async () => { + const {events} = await captureLargeBody(); + + const responseEvent = events.find(event => event.event === 'http.response'); + // Had the tee buffered the whole body to slice a preview off the end, this would read 10485760 -- + // which is the memory-exhaustion shape XCUT-24 exists to forbid, not merely a wrong number. + expect(responseEvent?.['http.response.body.size']).toBe(PREVIEW_CAP); + }); + + test('emits no field carrying more than the cap', async () => { + const {events} = await captureLargeBody(); + + const responseEvent = events.find(event => event.event === 'http.response'); + const oversized = Object.entries(responseEvent ?? {}).filter( + ([, value]) => typeof value === 'string' && value.length > PREVIEW_CAP, + ); + // Guards the whole event, not just the field key this port happens to use today. + expect(oversized).toEqual([]); + }); +}); + +describe('XCUT-24: a diagnostic preview is non-consuming', () => { + test('leaves the caller reading every one of the 10485760 bytes', async () => { + const {bodyLength} = await captureLargeBody(); + + // The primary read path must be undisturbed: the consumer sees the full body, not the truncation + // the log saw. + expect(bodyLength).toBe(BODY_BYTES); + }); +}); diff --git a/tests/conformance/xcut/error-taxonomy.conformance.test.ts b/tests/conformance/xcut/error-taxonomy.conformance.test.ts new file mode 100644 index 0000000..a434b0c --- /dev/null +++ b/tests/conformance/xcut/error-taxonomy.conformance.test.ts @@ -0,0 +1,201 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/error-taxonomy.conformance.test.ts +// Exercises: XCUT-4 (two-branch taxonomy -- response-carrying protocol errors vs. response-less +// I/O-family transport errors), XCUT-6 (a custom error type participates in retry with no edit to +// the classifier), XCUT-7 (the CONFIGURED retryable-status set is authoritative and both widens and +// narrows), XCUT-9 (a cyclic cause chain terminates instead of hanging). +// XCUT-5 and XCUT-8 stay retrofit citations at their own phases' tests -- see this file's closing note. +// +// Every row drives the composed pipeline rather than calling `isRetryableFailure` directly. That is +// deliberate on two counts: the classifier is `@internal` and absent from core's barrel, so a +// consumer-shaped test cannot reach it at all; and calling it directly would restate 5a's own +// classify.test.ts, which this suite is explicitly not for. +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + IoError, + Request, + toHttpError, + type Response, + type Transport, +} from '@dexpace/core'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; + +let server: XcutFixtureServer; + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +/** A transport that always throws whatever it was handed, to drive classification from the inside. */ +class ThrowingTransport implements Transport { + readonly #error: unknown; + + constructor(error: unknown) { + this.#error = error; + } + + send(): Promise<Response> { + // This transport exists to reject with values that are deliberately NOT Errors, so XCUT-9's + // cyclic-cause row and XCUT-6's opted-out row can drive the classifier with whatever they like. + // An `async` + `throw` rewrite only trades this rule for `require-await`. + /* eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- rejecting with a non-Error IS the behavior under test; re-enable if XCUT-6/XCUT-9 stop needing non-Error rejections */ + return Promise.reject(this.#error); + } + + async close(): Promise<void> { + // Nothing to release: this transport never opens anything. + } +} + +describe('XCUT-4: the taxonomy has exactly two branches', () => { + test('a 5xx arrives as a protocol failure carrying its fully-received response', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + ); + + expect(response.status.code).toBe(500); + await response.close(); + await pipeline.close(); + }); + + test('that response converts to the response-carrying error exposing status and body', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + ); + + const error = await toHttpError(response); + + expect(error?.status).toBe(500); + expect(error?.preview()).toContain('server error'); + await pipeline.close(); + }); + + test('a connection failure arrives as the response-less I/O-family error', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + + const pending = pipeline.runtime.send( + // Port 1: nothing listens, so the connection is refused rather than merely slow. + Request.newBuilder().url('http://127.0.0.1:1/').build(), + ); + + // Catchable as the generic I/O family, which is XCUT-4's "existing I/O catch sites keep matching". + expect(await rejectionOf(pending)).toBeInstanceOf(IoError); + await pipeline.close(); + }); +}); + +describe('XCUT-6: a custom error type participates without editing the classifier', () => { + test('retries an error type declared in this test file, unknown to classify.ts', async () => { + // The port's retryability capability is subtyping, not a duck-typed `isRetryable` flag: the + // cause-walk returns true for anything `instanceof IoError`, so extending it is what opts a new + // failure in with no classifier edit (deviation ledger item 17, docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md). + class CustomTransientError extends IoError {} + const pipeline = buildComposedPipeline({ + transport: new ThrowingTransport(new CustomTransientError('transient')), + retry: {settings: {maxAttempts: 3, initialDelayMs: 1}}, + }); + + await pipeline.runtime + .send(Request.newBuilder().url(`${server.url}/ok`).build()) + .catch(() => undefined); + + expect(pipeline.dispatches()).toBe(3); + await pipeline.close(); + }); + + test('does not retry a plain Error, which opted into nothing', async () => { + const pipeline = buildComposedPipeline({ + transport: new ThrowingTransport(new Error('not opted in')), + retry: {settings: {maxAttempts: 3, initialDelayMs: 1}}, + }); + + await pipeline.runtime + .send(Request.newBuilder().url(`${server.url}/ok`).build()) + .catch(() => undefined); + + // The allow-list shape is the whole point: unknown failures are terminal by default. + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); + +describe('XCUT-7: the configured retryable-status set is authoritative', () => { + test('widening it to include 501 retries a status the built-in classifier excludes', async () => { + const pipeline = buildComposedPipeline({ + retry: { + settings: { + maxAttempts: 3, + initialDelayMs: 1, + retryableStatuses: new Set([501]), + }, + }, + }); + + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/status?code=501`).build(), + ); + + expect(pipeline.dispatches()).toBe(3); + await response.close(); + await pipeline.close(); + }); + + test('narrowing it to exclude 500 stops a status the built-in classifier includes', async () => { + const pipeline = buildComposedPipeline({ + retry: { + settings: { + maxAttempts: 3, + initialDelayMs: 1, + retryableStatuses: new Set([503]), + }, + }, + }); + + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/status?code=500`).build(), + ); + + // 500's built-in classification is retryable; the configured set overrides it rather than + // being AND-ed with it (RETRY-37). + expect(pipeline.dispatches()).toBe(1); + await response.close(); + await pipeline.close(); + }); +}); + +describe('XCUT-9: a cyclic cause chain terminates', () => { + test('classifies a self-referential error without hanging', async () => { + const cyclic = new Error('cyclic'); + cyclic.cause = cyclic; + const pipeline = buildComposedPipeline({ + transport: new ThrowingTransport(cyclic), + retry: {settings: {maxAttempts: 3, initialDelayMs: 1}}, + }); + + const surfaced = await rejectionOf( + pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/ok`).build(), + ), + ); + + // Reaching this line at all is the assertion: an identity-tracking walk terminates, a naive + // recursive one would have spun until the test timed out. + expect(surfaced).toBe(cyclic); + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/fixtures/composed-pipeline.ts b/tests/conformance/xcut/fixtures/composed-pipeline.ts new file mode 100644 index 0000000..9303d68 --- /dev/null +++ b/tests/conformance/xcut/fixtures/composed-pipeline.ts @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/fixtures/composed-pipeline.ts +import { + standardResilience, + type AuthStepSettings, + type LoggingStepSettings, + type RedirectSettings, + type Request, + type RequestOptions, + type Response, + type RetryStepOptions, + type Runtime, + type Transport, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +/** + * Per-pillar overrides, mirroring `StandardResilienceOptions` exactly rather than restating a + * narrowed copy of it -- `retry` is a `RetryStepOptions` (settings nest under `.settings`), not a + * `Partial<RetrySettings>`. + */ +export interface ComposedPipelineOverrides { + readonly retry?: RetryStepOptions | undefined; + readonly redirect?: Partial<RedirectSettings> | undefined; + readonly auth?: AuthStepSettings | undefined; + readonly logging?: LoggingStepSettings | undefined; + /** Swap the terminal transport, e.g. for `undiciTransport()`. Defaults to `fetchTransport()`. */ + readonly transport?: Transport | undefined; +} + +/** + * Counts dispatches to the terminal transport. + * + * Wrapping the TRANSPORT is the only placement that answers "was this retried?". Wrapping + * `Runtime.send` -- one call in, one call out -- counts the caller's own invocations and would read + * 1 whether the retry pillar re-issued four times or none, which is the opposite of what every + * `XCUT-10` row asserts. + */ +class CountingTransport implements Transport { + #dispatches = 0; + readonly #inner: Transport; + + // Not a constructor parameter property: `erasableSyntaxOnly` bans those repo-wide. + constructor(inner: Transport) { + this.#inner = inner; + } + + get dispatches(): number { + return this.#dispatches; + } + + async send( + request: Request, + options?: RequestOptions, + signal?: AbortSignal, + ): Promise<Response> { + this.#dispatches += 1; + return this.#inner.send(request, options, signal); + } + + async close(): Promise<void> { + return this.#inner.close(); + } +} + +/** A built pipeline plus the two things a conformance row needs around it. */ +export interface ComposedPipeline { + /** The composed runtime: redirect wraps retry wraps auth wraps logging (AUTH-27). */ + readonly runtime: Runtime; + /** Dispatches that actually reached the terminal transport, i.e. attempts including retries. */ + readonly dispatches: () => number; + /** Closes the terminal transport. `Runtime.close()` is a documented no-op (PIPE-27). */ + close(): Promise<void>; +} + +/** + * The one real, fully composed pipeline every `XCUT-N` test in this directory drives -- + * retry + redirect + auth + logging via 5c/7b's `standardResilience()` over a real + * `fetchTransport()`. Never a per-test hand-rolled subset: the value this suite adds over each + * pillar's own unit tests is proving the invariants still hold when all of them run together. + * + * @param overrides - per-pillar settings; omitted pillars take their shipped defaults. + * @returns the runtime, its dispatch counter, and a close that reaches the transport. + */ +export function buildComposedPipeline( + overrides: ComposedPipelineOverrides = {}, +): ComposedPipeline { + const counting = new CountingTransport( + overrides.transport ?? fetchTransport(), + ); + const runtime = standardResilience(counting, { + retry: overrides.retry, + redirect: overrides.redirect, + auth: overrides.auth, + logging: overrides.logging, + }); + + return { + runtime, + dispatches: () => counting.dispatches, + close: () => counting.close(), + }; +} diff --git a/tests/conformance/xcut/fixtures/server.ts b/tests/conformance/xcut/fixtures/server.ts new file mode 100644 index 0000000..754cb65 --- /dev/null +++ b/tests/conformance/xcut/fixtures/server.ts @@ -0,0 +1,178 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/fixtures/server.ts +import { + createServer, + type IncomingMessage, + type Server, + type ServerResponse, +} from 'node:http'; + +/** + * A pair of running fixture origins. Two real listeners, not one listener addressed by two hostnames: + * the server binds `127.0.0.1` explicitly, so a second name for the same port is not reliably + * resolvable (`localhost` may resolve to `::1` first), and `XCUT-17`'s cross-origin hop has to be a + * genuinely different origin for the assertion to mean anything. + */ +export interface XcutFixtureServer { + /** The primary origin every path is resolved against, e.g. `http://127.0.0.1:38211`. */ + readonly url: string; + /** A second, independently-listening origin -- a different port, so a different origin. */ + readonly crossOriginUrl: string; + /** Stops both listeners and resolves once each has released its port. */ + close(): Promise<void>; +} + +/** `/large-body`'s default payload: comfortably past any preview cap `XCUT-24` would configure. */ +const LARGE_BODY_BYTES = 10 * 1024 * 1024; +/** `/slow`'s default stall, long enough that no cancellation under test wins its race by luck. */ +const SLOW_RESPONSE_MS = 5_000; + +/** Reflects back what actually arrived, so a test can prove which credentials survived a hop. */ +function echo(req: IncomingMessage, res: ServerResponse, url: URL): void { + res.writeHead(200, {'content-type': 'application/json'}); + res.end( + JSON.stringify({ + path: url.pathname, + query: Object.fromEntries(url.searchParams), + method: req.method ?? null, + authorization: req.headers.authorization ?? null, + cookie: req.headers.cookie ?? null, + proxyAuthorization: req.headers['proxy-authorization'] ?? null, + }), + ); +} + +/** The routes shared by both origins. `crossOrigin` is the OTHER origin, for the two-hop redirect. */ +function route( + req: IncomingMessage, + res: ServerResponse, + crossOrigin: string, +): void { + const url = new URL(req.url ?? '/', 'http://127.0.0.1'); + switch (url.pathname) { + case '/ok': + res.writeHead(200, {'content-type': 'text/plain'}); + res.end('ok'); + return; + case '/echo': + echo(req, res, url); + return; + case '/slow': { + const delayMs = Number( + url.searchParams.get('ms') ?? String(SLOW_RESPONSE_MS), + ); + // `unref` so a stalled timer can never hold the suite open past its own afterAll. + setTimeout(() => { + res.writeHead(200); + res.end('done'); + }, delayMs).unref(); + return; + } + case '/large-body': { + const size = Number( + url.searchParams.get('bytes') ?? String(LARGE_BODY_BYTES), + ); + // content-length is declared, not derived. Writing the body without it leaves Node no length + // to precompute and it falls back to chunked -- and OBS-37 deliberately skips preview capture + // on an unknown-length body, so the XCUT-24 rows would silently assert against no preview + // at all rather than against a capped one. + // The media type is selectable because OBS-38 forks on it: a text body is previewed as + // decoded text, a binary one as a size-only `[binary N bytes captured]` marker. XCUT-24's cap + // has to hold on both paths, so both are driven. + res.writeHead(200, { + 'content-type': + url.searchParams.get('type') ?? 'application/octet-stream', + 'content-length': String(size), + }); + res.end(Buffer.alloc(size, 'x')); + return; + } + case '/redirect-same-origin': + res.writeHead(302, {location: '/echo'}); + res.end(); + return; + case '/redirect-cross-origin': + res.writeHead(302, {location: `${crossOrigin}/echo`}); + res.end(); + return; + case '/redirect-secret-target': { + // A method-preserving 307 whose Location carries a credential-shaped query value, echoed back + // from the caller's own `?secret=` so the test owns the string it then greps the log for. + // XCUT-19's rejected-redirect row needs the secret in the redirect TARGET specifically: that is + // the URL `NonReplayableBodyError` names, and the seed URL alone never reaches that message. + const secret = url.searchParams.get('secret') ?? 'secret'; + res.writeHead(307, {location: `/echo?access_token=${secret}`}); + res.end(); + return; + } + case '/fail-500': + res.writeHead(500, {'content-type': 'text/plain'}); + res.end('server error'); + return; + case '/status': { + // Any status on demand, for XCUT-7's widen/narrow rows: 501 is excluded from the built-in + // retryable set and 500 is in it, so both directions need a live endpoint to prove against. + const code = Number(url.searchParams.get('code') ?? '500'); + res.writeHead(code, {'content-type': 'text/plain'}); + res.end(`status ${String(code)}`); + return; + } + default: + res.writeHead(404, {'content-length': '0'}); + res.end(); + } +} + +/** Starts one listener on an ephemeral port and resolves its origin alongside the handle. */ +function listen( + crossOrigin: () => string, +): Promise<{origin: string; server: Server}> { + return new Promise(resolve => { + const server = createServer((req, res) => { + route(req, res, crossOrigin()); + }); + server.listen(0, '127.0.0.1', () => { + const address = server.address(); + const port = + typeof address === 'object' && address !== null ? address.port : 0; + resolve({origin: `http://127.0.0.1:${String(port)}`, server}); + }); + }); +} + +/** Closes one listener, dropping keep-alive sockets a transport may still be holding. */ +function shutdown(server: Server): Promise<void> { + return new Promise<void>(done => { + // closeAllConnections, not close alone: a pooled socket would otherwise stall this for the + // server's whole idle timeout (the same reason 8a's own fixture does it). + server.closeAllConnections(); + server.close(() => { + done(); + }); + }); +} + +/** + * Starts the two fixture origins every `XCUT-N` suite in this directory shares, each on an ephemeral + * port so parallel test files never collide. + * + * The secondary comes up first so the primary's `/redirect-cross-origin` can name it; the secondary's + * own cross-origin route points back at the primary, which is why both are handed a late-bound + * getter rather than a string. + * + * @returns both origins; the caller closes them in its own `afterAll`. + */ +export async function startFixtureServer(): Promise<XcutFixtureServer> { + let primaryOrigin = ''; + const secondary = await listen(() => primaryOrigin); + const primary = await listen(() => secondary.origin); + primaryOrigin = primary.origin; + + return { + url: primary.origin, + crossOriginUrl: secondary.origin, + close: async (): Promise<void> => { + await Promise.all([shutdown(primary.server), shutdown(secondary.server)]); + }, + }; +} diff --git a/tests/conformance/xcut/fixtures/settle.test.ts b/tests/conformance/xcut/fixtures/settle.test.ts new file mode 100644 index 0000000..821462a --- /dev/null +++ b/tests/conformance/xcut/fixtures/settle.test.ts @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/fixtures/settle.test.ts +// Exercises: the shared rejection-capturing helper this directory's XCUT-N suites assert through. +// Both branches matter -- a `rejectionOf` that reported `undefined` for a REJECTED promise would make +// every `expect(await rejectionOf(p)).toBeInstanceOf(...)` row in this directory vacuously wrong. +import {describe, expect, test} from 'bun:test'; +import {rejectionOf} from './settle.js'; + +describe('rejectionOf', () => { + test('hands back the reason a rejected promise carried', async () => { + const reason = new TypeError('boom'); + + expect(await rejectionOf(Promise.reject(reason))).toBe(reason); + }); + + test('hands back a non-Error rejection reason unchanged', async () => { + // The XCUT-9 row rejects with a cyclic plain object, so the helper must not coerce or wrap. + const cyclic: {self?: unknown} = {}; + cyclic.self = cyclic; + + /* eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- a non-Error rejection is exactly the case under test; re-enable if XCUT-9 stops rejecting with a bare cyclic object */ + expect(await rejectionOf(Promise.reject(cyclic))).toBe(cyclic); + }); + + test('reports undefined when the promise resolved instead', async () => { + expect(await rejectionOf(Promise.resolve('fulfilled'))).toBeUndefined(); + }); +}); diff --git a/tests/conformance/xcut/fixtures/settle.ts b/tests/conformance/xcut/fixtures/settle.ts new file mode 100644 index 0000000..1c99681 --- /dev/null +++ b/tests/conformance/xcut/fixtures/settle.ts @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/fixtures/settle.ts + +/** + * Awaits `promise` and hands back the reason it rejected with, or `undefined` if it resolved. + * + * Used instead of `await expect(p).rejects.toBeInstanceOf(...)` throughout this suite. Bun types the + * `.rejects` matchers as returning `void`, so awaiting one trips `await-thenable` and + * `no-confusing-void-expression` under this repo's type-aware lint tier, and the idiom the packages + * settled on -- dropping the `await` -- makes the assertion fire-and-forget: the matcher's own + * failure surfaces after the test has already returned, if at all. + * + * Capturing the rejection and asserting on the value synchronously is both lint-clean and genuinely + * awaited, which matters here because every row in this directory is asserting on WHICH error came + * back, not merely that one did. + * + * @param promise - the operation under test. + * @returns the rejection reason, or `undefined` when the promise resolved. + */ +export async function rejectionOf(promise: Promise<unknown>): Promise<unknown> { + return promise.then( + () => undefined, + (reason: unknown) => reason, + ); +} diff --git a/tests/conformance/xcut/retry-safety.conformance.test.ts b/tests/conformance/xcut/retry-safety.conformance.test.ts new file mode 100644 index 0000000..0be9dd0 --- /dev/null +++ b/tests/conformance/xcut/retry-safety.conformance.test.ts @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/retry-safety.conformance.test.ts +// Exercises: XCUT-10 (retry-SAFETY is decided at the retry step independently of retryability, and +// applies uniformly to protocol AND transport failures -- the gate must not special-case a transport +// error that never reached the server), XCUT-1 (the class of the surfaced error does not depend on +// how many attempts the pillar spent), RETRY-34 (the earlier attempts stay reachable beside it). +// +// The five XCUT-10 rows are the ones its own conformance clause names, run for the first time against +// the composed pipeline rather than 5a's unit-level harness. Each asserts on dispatches that actually +// reached the terminal transport, which is the only vantage point where "was it re-sent?" is visible. +// The two XCUT-1/RETRY-34 rows below need the same vantage point for the opposite reason: the budget +// they vary is only observable as a dispatch count. +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + Request, + retryAttempts, + streamBody, + stringBody, + TransportFailureError, +} from '@dexpace/core'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; + +let server: XcutFixtureServer; + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +/** Three attempts with a negligible backoff, so a retried row is unmistakable from a non-retried one. */ +function retrying(): {settings: {maxAttempts: number; initialDelayMs: number}} { + return {settings: {maxAttempts: 3, initialDelayMs: 1}}; +} + +/** + * A GET at a closed port. Retry-SAFE (idempotent, body-less) and retryABLE (the transports map a + * refused connection to `TransportFailureError`, an `IoError`), so the pillar spends its whole + * budget and every attempt fails the same way -- which is what makes the surfaced CLASS the only + * variable between the two budgets below. + */ +function unreachable(): Request { + return Request.newBuilder().url('http://127.0.0.1:1/').method('GET').build(); +} + +describe('XCUT-10: retry-safety on a body-less request follows method idempotence', () => { + test('retries a body-less GET against a retryable protocol failure', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).method('GET').build(), + ); + + expect(pipeline.dispatches()).toBe(3); + await response.close(); + await pipeline.close(); + }); + + test('does not retry a body-less POST failing with a protocol error', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const response = await pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).method('POST').build(), + ); + + // The 500 is retryABLE; the bare POST is not retry-SAFE. Two orthogonal axes, and safety wins. + expect(pipeline.dispatches()).toBe(1); + await response.close(); + await pipeline.close(); + }); + + test('still does not retry a body-less POST failing with a transport error', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('http://127.0.0.1:1/').method('POST').build(), + ); + await pending.catch(() => undefined); + + // The row XCUT-10 calls out explicitly: the request demonstrably never reached the server, and + // the gate MUST still refuse. A safety gate that special-cased transport errors would read 3. + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); + +describe('XCUT-10: retry-safety on a body-bearing request follows body replayability', () => { + test('retries a POST whose body is replayable', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const response = await pipeline.runtime.send( + Request.newBuilder() + .url(`${server.url}/fail-500`) + .method('POST') + .body(stringBody('payload')) + .build(), + ); + + // A replayable body makes a non-idempotent method safe to re-send: the body clause governs + // once a body is present, rather than being AND-ed with method idempotence. + expect(pipeline.dispatches()).toBe(3); + await response.close(); + await pipeline.close(); + }); + + test('does not retry a POST whose body is a single-use stream', async () => { + const {readable} = new TransformStream<Uint8Array, Uint8Array>(); + const pipeline = buildComposedPipeline({retry: retrying()}); + + const pending = pipeline.runtime.send( + Request.newBuilder() + .url(`${server.url}/fail-500`) + .method('POST') + .body(streamBody(readable)) + .build(), + ); + await pending.catch(() => undefined).then(r => r?.close()); + + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); + +describe('XCUT-1/RETRY-34: what a retrying pipeline surfaces when it gives up', () => { + test('the same failure surfaces as TransportFailureError for maxAttempts 1 and for 3', async () => { + const once = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + const thrice = buildComposedPipeline({retry: retrying()}); + + const afterOne = await rejectionOf(once.runtime.send(unreachable())); + const afterThree = await rejectionOf(thrice.runtime.send(unreachable())); + + expect(once.dispatches()).toBe(1); + expect(thrice.dispatches()).toBe(3); + // The row #72 exists for. Until 2026-09-05 the three-attempt case surfaced a `SuppressedError` + // holding the transport failure at `.error`, so one condition had two surfaced classes and the + // discriminator was the attempt budget -- something no caller writing `catch` can see. + expect(afterOne).toBeInstanceOf(TransportFailureError); + expect(afterThree).toBeInstanceOf(TransportFailureError); + + await once.close(); + await thrice.close(); + }); + + test('the earlier attempts are reachable beside it, oldest first, self excluded', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const surfaced = await rejectionOf(pipeline.runtime.send(unreachable())); + + // RETRY-34 through the composed pipeline: three sends, so two priors, and the surfaced instance + // is not a member of its own trail. + const priors = retryAttempts(surfaced); + expect(pipeline.dispatches()).toBe(3); + expect(priors).toHaveLength(2); + expect(priors.every(prior => prior instanceof TransportFailureError)).toBe( + true, + ); + expect(priors).not.toContain(surfaced); + + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/security-by-default.conformance.test.ts b/tests/conformance/xcut/security-by-default.conformance.test.ts new file mode 100644 index 0000000..93572c6 --- /dev/null +++ b/tests/conformance/xcut/security-by-default.conformance.test.ts @@ -0,0 +1,330 @@ +// SPDX-License-Identifier: MIT +// tests/conformance/xcut/security-by-default.conformance.test.ts +// Exercises: XCUT-17 (redirect credential hygiene -- Authorization stripped before EVERY re-issue, +// origin-scoped credentials additionally stripped cross-origin), XCUT-16 (no credential is ever +// stamped over a non-HTTPS transport, and the refusal lands BEFORE any token fetch). +// Exercises: XCUT-19 (default-deny log redaction, clause (a) userinfo and clause (b) query values) on +// the rejected-redirect path, with OBS-11, OBS-12 and REDIR-28 as the requirements it lands under. +// +// These run over a real two-origin socket pair through the composed retry+redirect+auth+logging +// pipeline. 5b's own tests decide the hop in isolation against constructed inputs; this is the first +// place the decision runs with a live auth step installed behind it. +// +// Clauses that stay retrofit citations at their own phases' tests, because a plaintext fixture cannot +// reach them and XCUT-16 is precisely why: +// XCUT-17(c) userinfo dropped -> packages/core/src/redirect/decide.test.ts (REDIR-12) +// XCUT-17(d) HTTPS->HTTP denied -> packages/core/src/redirect/decide.test.ts (REDIR-14/15) and +// redirect-step.test.ts +// XCUT-16 unit-level -> packages/core/src/auth/auth-step.test.ts (AUTH-28) +// XCUT-18 header splitting -> packages/core/src/http/headers.test.ts +// XCUT-19 default-deny redaction-> packages/core/src/observability/redaction.test.ts +// XCUT-20 observability never throws -> packages/core/src/observability/logging-step.test.ts +// XCUT-21 CSPRNG cnonce -> packages/core/src/auth/digest.test.ts (AUTH-20) +// +// Also exercises: XCUT-16/AUTH-28 on the CHALLENGE-REPLAY path -- a hop the outbound pass guarded +// stays guarded, so a `challengeHook` that answers a 401 by downgrading to `http://` is refused +// whatever header it carries the credential in (audit #67 / #71). +import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; +import { + createAuthDescriptor, + createAuthRequirement, + createBearerToken, + createLogger, + Headers, + NonReplayableBodyError, + NOOP_LOGGER, + NameKeyCredential, + PlaintextCredentialError, + Protocol, + Request, + setGlobalLogger, + streamBody, + type AuthStepSettings, + type Method, + Response, + Status, + type Transport, +} from '@dexpace/core'; +import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; +import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; + +let server: XcutFixtureServer; + +beforeAll(async () => { + server = await startFixtureServer(); +}); + +afterAll(async () => { + await server.close(); +}); + +/** The credentials a caller sets by hand, which the redirect pillar must police on every hop. */ +function callerCredentials(): Headers { + return Headers.newBuilder() + .set('authorization', 'Bearer caller-set') + .set('cookie', 'sid=abc') + .build(); +} + +/** Reads the fixture's echo of what actually arrived at the final hop. */ +async function followAndEcho( + path: string, +): Promise<{authorization: string | null; cookie: string | null}> { + const pipeline = buildComposedPipeline(); + try { + const response = await pipeline.runtime.send( + Request.newBuilder() + .url(`${server.url}${path}`) + .headers(callerCredentials()) + .build(), + ); + const body = JSON.parse(await response.text()) as { + authorization: string | null; + cookie: string | null; + }; + await response.close(); + return body; + } finally { + await pipeline.close(); + } +} + +describe('XCUT-17: Authorization is stripped before every redirect re-issue', () => { + test('drops Authorization even on a same-origin hop', async () => { + const echoed = await followAndEcho('/redirect-same-origin'); + + // "even same-origin" is the clause that catches the tempting optimisation. + expect(echoed.authorization).toBeNull(); + }); + + test('keeps an origin-scoped Cookie on a same-origin hop', async () => { + const echoed = await followAndEcho('/redirect-same-origin'); + + // Cookie is origin-scoped, and this hop has not left the origin: stripping it here would be + // over-broad, and XCUT-17 scopes the extra stripping to the cross-origin case. + expect(echoed.cookie).toBe('sid=abc'); + }); +}); + +describe('XCUT-17: origin-scoped credentials are additionally stripped cross-origin', () => { + test('drops Authorization on a cross-origin hop', async () => { + const echoed = await followAndEcho('/redirect-cross-origin'); + + expect(echoed.authorization).toBeNull(); + }); + + test('drops the Cookie on a cross-origin hop', async () => { + const echoed = await followAndEcho('/redirect-cross-origin'); + + // Judged against the seed origin, not the previous hop -- the two servers are genuinely + // different origins (different ports), not one origin under two names. + expect(echoed.cookie).toBeNull(); + }); +}); + +describe('XCUT-19: a rejected redirect logs no raw URL (OBS-11, OBS-12, REDIR-28)', () => { + const SECRET = 'SUPERSECRETTOKEN'; + + test('redacts the redirect target inside the rejection cause', async () => { + // A one-shot body makes the 307 unfollowable, so `decide()` fails with the target interpolated + // into the error message -- the one field on this path that `redactUrl` did not already cover. + const seed = Request.newBuilder() + .method('POST') + .url(`${server.url}/redirect-secret-target?secret=${SECRET}`) + .body( + streamBody( + new ReadableStream<Uint8Array>({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ), + ) + .build(); + const pipeline = buildComposedPipeline({ + redirect: {allowedMethods: new Set<Method>(['GET', 'HEAD', 'POST'])}, + }); + const records: Map<string, unknown>[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + records.push(new Map(fields)); + }), + ); + + try { + const rejected = await rejectionOf(pipeline.runtime.send(seed)); + + expect(rejected).toBeInstanceOf(NonReplayableBodyError); + const rejections = records.filter( + r => r.get('event') === 'http.redirect.rejected', + ); + expect(rejections).toHaveLength(1); + expect(String(rejections[0]?.get('cause'))).toContain('access_token=***'); + // Nothing the whole composed pipeline emitted -- not the redirect events, not the + // request/response pair around them -- carries the secret in clear text. + for (const record of records) { + for (const field of record.values()) { + expect(String(field)).not.toContain(SECRET); + } + } + } finally { + setGlobalLogger(NOOP_LOGGER); + await pipeline.close(); + } + }); +}); + +describe('XCUT-16: a credential is never stamped over a non-HTTPS transport', () => { + test('refuses a bearer credential over http:// before fetching the token', async () => { + let providerInvocations = 0; + const auth: AuthStepSettings = { + credentials: { + bearer: { + provider: () => { + providerInvocations += 1; + return Promise.resolve( + createBearerToken('secret', Date.now() + 60_000), + ); + }, + }, + }, + tiers: { + operation: createAuthDescriptor([createAuthRequirement('OAUTH2')]), + }, + }; + const pipeline = buildComposedPipeline({auth}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/echo`).build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + // "fail loudly BEFORE any token fetch or header write" -- a guard that ran after the fetch would + // already have pulled a live secret over the wire, which is the leak the ordering prevents. + expect(providerInvocations).toBe(0); + await pipeline.close(); + }); + + test('never dispatches the credentialed request at all', async () => { + const auth: AuthStepSettings = { + credentials: { + bearer: { + provider: () => + Promise.resolve(createBearerToken('secret', Date.now() + 60_000)), + }, + }, + tiers: { + operation: createAuthDescriptor([createAuthRequirement('OAUTH2')]), + }, + }; + const pipeline = buildComposedPipeline({auth}); + + await pipeline.runtime + .send(Request.newBuilder().url(`${server.url}/echo`).build()) + .catch(() => undefined); + + expect(pipeline.dispatches()).toBe(0); + await pipeline.close(); + }); +}); + +/** + * A transport that answers everything with the same 401 challenge, so the replay path can be driven + * without a TLS fixture. `XCUT-16`'s replay clause needs an outbound hop that is HTTPS — the guard + * cannot have run otherwise — and the plaintext fixture server above cannot provide one. The stubbed + * transport is the same device `error-taxonomy.conformance.test.ts` uses for the inputs a live socket + * cannot produce; everything above the transport is still the real composed pipeline. + */ +class ChallengingTransport implements Transport { + send(request: Request): Promise<Response> { + return Promise.resolve( + Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(401)) + .headers( + Headers.newBuilder() + .setInbound('WWW-Authenticate', 'Basic realm="x"') + .build(), + ) + .build(), + ); + } + + async close(): Promise<void> { + // Nothing to release: this transport never opens anything. + } +} + +describe('XCUT-16: a guarded hop stays guarded across a challenge replay', () => { + /** `X-Api-Key`, not `Authorization`: the header this step is configured to stamp. */ + function apiKeyAuth(replacement: (request: Request) => Request): { + auth: AuthStepSettings; + transport: Transport; + } { + return { + auth: { + credentials: { + apiKey: { + credential: new NameKeyCredential('x-api-key', 'SECRET'), + headerName: 'X-Api-Key', + }, + }, + tiers: { + operation: createAuthDescriptor([createAuthRequirement('API_KEY')]), + }, + challengeHook: (_response, request) => + Promise.resolve(replacement(request)), + }, + transport: new ChallengingTransport(), + }; + } + + test('refuses a replacement that downgrades to http:// and carries the key in X-Api-Key', async () => { + // The reported hole: the replay guard tested two header NAMES, and neither of them is the one + // `ApiKeyCredentialConfig.headerName` told this step to stamp. The credential went out in clear + // text with the whole suite green. + const {auth, transport} = apiKeyAuth(request => + request + .newBuilder() + .url('http://example.com/echo') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + // One dispatch: the guarded outbound pass. The replay never reached the transport. + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); + + test('refuses a downgraded replacement even with no credential header on it at all', async () => { + // The rule is "this hop was guarded", not "this replacement looks credentialed" — a hook is free + // to invent a carrier no enumeration of header names would know to look for. + const {auth, transport} = apiKeyAuth(request => + Request.newBuilder() + .url('http://example.com/echo') + .method(request.method) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); diff --git a/tests/node-conformance/README.md b/tests/node-conformance/README.md new file mode 100644 index 0000000..ef802e0 --- /dev/null +++ b/tests/node-conformance/README.md @@ -0,0 +1,74 @@ +# Node-runtime conformance suite + +`tests/node-conformance/` — run by `bun run test:node` (`node --test tests/node-conformance/*.test.mjs`), +never by `bun test`. + +Closes checkpoint §5.9 (`docs/work/mvp/2026-07-25-checkpoint-scaffold-through-phase3a.md:341`). + +> **This tree must not run on Bun.** That is the only reason it exists. Until Phase 10 it lived at +> `test/node-conformance/`, outside anything `bun test` could reach; it now sits inside `tests/`, so +> `bunfig.toml`'s `[test] pathIgnorePatterns` holds the line instead — along with four other files that +> carry the same path, which `scripts/verify-test-partition.mjs` blocks CI on. **Read CLAUDE.md's "HARD RULE +> — the `tests/` partition" before moving, renaming, or nesting anything here.** It is the one place that +> rule and its reasoning are written down; this file only states what is local to the tree. + +`bun test` runs the whole unit suite on **Bun's** runtime and proves nothing about the runtime this SDK +actually ships to. Bun's Web Streams, `AbortSignal`, and `Uint8Array`/async-iteration behavior are independent +implementations of Node's, and `packages/core/src/io/` — chunk boundaries, backpressure timing, reader-lock +discipline, `queueMicrotask` ordering — is exactly the kind of code where they diverge. The `no node: imports` +grep proves the code is runtime-*agnostic in its imports*, which is a much weaker claim than runtime-*correct +on Node*. + +This layer is **thin and additive**, not a second unit suite. `bun test` stays the unit-test runner, unchanged +— `docs/knowledge/harvested/testing.md` mandates `bun:test` symbol imports, `setSystemTime`, and `--concurrent`, so +migrating the suite to `node:test` would be a styleguide deviation plus a whole-suite rewrite, and it buys +nothing for the pure-logic majority (`Headers`/`MediaType`/`QueryParams` parsing cannot behave differently on +Node). + +## Rules + +- **Name every case `*.test.mjs`, flat in this directory.** `test:node`'s glob does not descend, and + `node --test` over a glob that matches nothing exits **0** — a case parked in a subdirectory is not a + failure, it is a silence. `verify:test-partition` turns that silence into a red CI step. +- **Import the built artifact, never `src/`.** Public surface comes in through the `@dexpace/core` specifier; + `io/` is `@internal` with no public subpath in `exports`, so it is reached by direct `dist/` file path. Run + `bun run build` first — `test:node` does not build for you, because the CI job builds once and then runs the + matrix. +- **Assert runtime-divergent behavior only.** Anything that is pure logic belongs in `bun test`, where it runs + faster and closer to the code. A case here should be one you could imagine failing on one runtime and passing + on the other. +- **Must pass on the declared floor.** `package.json` `engines.node` is the contract; CI runs this suite as a + matrix over that floor and current LTS. Do not reach for an API newer than the floor without moving the floor + in the same change. The floor is set by the *built-ins the code calls*, not by the syntax it emits — it reads + `>=20.3` because `globalThis.crypto` is absent from ESM on every Node 18 release and `AbortSignal.any()` + reached the 20.x line in 20.3.0, not because of anything ES2023. +- **Do not await a timer the runtime does not ref.** `AbortSignal.timeout()`'s timer is unref'd everywhere by + design, so awaiting its `abort` event with nothing else scheduled lets the loop drain and the runner report + `Promise resolution is still pending but the event loop has already resolved`. Hold the loop open with a ref'd + deadline that also fails the case if the event never arrives. + +## Membership rule + +**A phase that touches a runtime-divergent surface adds a case here, not only to `bun test`** (§5.9:378). +Since Phase 4 that has meant most phases — pipelines, retry, redirect, auth, serde, SSE, pagination, +configuration, observability, the two concrete transports, and the RxJS bridge all have cases here — as, since +audit #67's #77, do the five Web Streams bridges that had none: `BufferedSource.toReadableStream`, +`BufferedSink.toWritableStream`, `TeeSink.toWritableStream` in `io-byte-stream.test.mjs`, and the +`withRequestLogging` / `withResponseLogging` body taps in `body-lifecycle.test.mjs`. Every one of them is a +hand-written underlying source or sink object, so what they exercise is the runtime's own pull scheduling, +cancel dispatch and reader-lock bookkeeping. Three are worth naming as the shape to aim for: 8a's +`fetch`/`undici` transports, where this stops being precautionary and becomes the point; 8b's RxJS bridge, +whose reason for being hand-written is a cancellation path the runtime decides; and #77's multipart +`Content-Type`, where Bun's `Response.formData()` accepted a header Node's `Response.formData()` rejected — +the Bun rows were green over a body no Node peer could parse, and only the case here reproduced it. + +## Which cases exist + +`ls` this directory. An earlier revision kept a table of file-to-surface descriptions here; it listed 6 of +14 by the time anyone checked, because nothing regenerated it. What each case covers, and which requirement +IDs it discharges, is recorded once — in that phase's checklist under `docs/work/mvp/phaseN/`. + +One piece of provenance the tree cannot show: `seams.test.mjs` absorbed the retired +`scripts/verify-node-floor.mjs`, whose two `AbortSignal.any()` assertions were the only Node coverage that +existed before this suite. Its `globalThis.crypto` assertion is made from ESM on purpose — Node 18 exposed +`crypto` to CommonJS while leaving it undefined in ES modules, which is what sets the 20.3 floor. diff --git a/tests/node-conformance/auth.test.mjs b/tests/node-conformance/auth.test.mjs new file mode 100644 index 0000000..e29c28c --- /dev/null +++ b/tests/node-conformance/auth.test.mjs @@ -0,0 +1,348 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/auth.test.mjs +// +// Phase 5c reaches three runtime-provided globals that Bun implements independently of Node, and every +// one of them fails SILENTLY rather than loudly if the two disagree: +// +// 1. `globalThis.crypto.subtle.digest('SHA-256', ...)` -- the SHA-256/SHA-256-sess Digest algorithms +// (AUTH-15/AUTH-17). A wrong digest is still a well-formed hex string, so a divergence produces a +// header the server rejects rather than an exception a test would catch. The RFC 7616 vectors here +// are the only thing that pins it. `bun test` covers the same vectors on Bun's implementation; this +// file covers Node's. +// 2. `globalThis.crypto.getRandomValues()` -- the >=128-bit client nonce (AUTH-20). Its absence from +// ESM on every Node 18 release is one of the two reasons `engines.node` reads `>=20.3`, so this is +// also the floor assertion for that global. +// 3. `globalThis.btoa` -- Basic stamping (AUTH-14). A Latin-1/UTF-8 mismatch on a non-ASCII password +// produces a valid-looking base64 blob that authenticates against nothing. +// +// A fourth surface is structural rather than platform-specific but is only observable through Web +// Streams: AUTH-30/AUTH-31/AUTH-32's response-lifecycle discipline is observed through +// `countingResponse()`'s `cancel()`/`pull()` hooks, and Node's timing there is an independent +// implementation of Bun's. +// +// A fifth was added at 5c's adversarial review: AUTH-34's single-flight fetch is shared, so it carries +// no caller signal and each caller instead races its own wait against its own `AbortSignal`. That rests +// on `AbortController`/`AbortSignal` listener add-and-remove semantics and on `Promise.race` settling +// order, both of which Bun implements independently of Node. A divergence here does not throw -- it +// either hangs a caller that aborted or rejects one that did not. +// +// The listener ACCOUNTING that shape depends on is asserted here rather than in `bearer-cache.test.ts` +// for two reasons: `node:events`' `getEventListeners` is the only portable way to count listeners on +// an `AbortSignal` without spying on `removeEventListener`, and no colocated unit test under +// `packages/core/src/` imports a `node:` builtin -- that is the portability posture `basic.ts` and +// `digest.ts` keep by reaching for Web Crypto and `btoa` instead. The Bun side asserts the leak's +// behavioural consequence instead. +// +// `auth/` is `@internal` apart from the barrel-promoted configuration surface, so the handler internals +// are reached by direct `dist/` file path, per this suite's import rule. +import assert from 'node:assert/strict'; +import {getEventListeners} from 'node:events'; +import {describe, it} from 'node:test'; +import { + BasicCredential, + CancellationError, + Request, + authStep, + createAuthDescriptor, + createAuthRequirement, +} from '@dexpace/core'; +import {basicHandler} from '../../packages/core/dist/auth/basic.js'; +import {BearerTokenCache} from '../../packages/core/dist/auth/bearer-cache.js'; +import {createBearerToken} from '../../packages/core/dist/auth/credential.js'; +import { + computeDigestResponse, + digestHandler, +} from '../../packages/core/dist/auth/digest.js'; +import {md5, toHex} from '../../packages/core/dist/auth/md5.js'; +import {createRequestContext} from '../../packages/core/dist/context/context.js'; +import {Cursor} from '../../packages/core/dist/pipeline/cursor.js'; +import { + FakeTransport, + countingResponse, +} from '../../packages/core/dist/testing/fake-transport.js'; + +const REALM = 'testrealm@host.com'; +const NONCE = 'dcd98b7102dd2f0e8b11d0f600bfb0c093'; +const VECTOR = { + realm: REALM, + nonce: NONCE, + isUtf8: true, + method: 'GET', + uri: '/dir/index.html', + username: 'Mufasa', + password: 'Circle Of Life', + cnonce: '0a4f113b', + nc: '00000001', +}; + +function digestChallenge(params) { + return {scheme: 'digest', params: new Map(Object.entries(params))}; +} + +function aRequest(url = 'https://example.com/a') { + return Request.newBuilder().url(url).build(); +} + +function runThrough(descriptor, transport, request = aRequest()) { + return new Cursor({ + steps: [descriptor], + transport, + request, + context: createRequestContext(request), + }).advance(); +} + +function challengeResponse(status, headerName, headerValue) { + const base = countingResponse(status); + const response = base.response + .newBuilder() + .headers( + base.response.headers + .newBuilder() + .setInbound(headerName, headerValue) + .build(), + ) + .build(); + return {response, cancelCount: base.cancelCount}; +} + +describe('Web Crypto SHA-256 under Node (AUTH-15/AUTH-17)', () => { + it('computes the RFC 7616 SHA-256 response, qop=auth', async () => { + assert.equal( + await computeDigestResponse({ + ...VECTOR, + algorithm: 'SHA-256', + hasQopAuth: true, + }), + '5abdd07184ba512a22c53f41470e5eea7dcaa3a93a59b630c13dfe0a5dc6e38b', + ); + }); + + it('computes the RFC 7616 SHA-256-sess response, qop=auth', async () => { + assert.equal( + await computeDigestResponse({ + ...VECTOR, + algorithm: 'SHA-256-sess', + hasQopAuth: true, + }), + 'b8822e12417cb7750f4e2b8515f0dcf25b7dd26993e80bee1426201446a7f59b', + ); + }); + + it('computes the RFC 7616 MD5 response, qop=auth, through the hand-rolled digest', async () => { + assert.equal( + await computeDigestResponse({ + ...VECTOR, + algorithm: 'MD5', + hasQopAuth: true, + }), + '6629fae49393a05397450978507c4ef1', + ); + }); + + it('pins the hand-rolled MD5 primitive itself against the RFC 1321 "abc" vector', async () => { + assert.equal( + toHex(md5(new TextEncoder().encode('abc'))), + '900150983cd24fb0d6963f7d28e17f72', + ); + }); + + it('hashes UTF-8 and ISO-8859-1 inputs differently for a non-ASCII password (AUTH-21)', async () => { + const utf8 = await computeDigestResponse({ + ...VECTOR, + password: 'pässwörd', + algorithm: 'SHA-256', + hasQopAuth: true, + isUtf8: true, + }); + const latin1 = await computeDigestResponse({ + ...VECTOR, + password: 'pässwörd', + algorithm: 'SHA-256', + hasQopAuth: true, + isUtf8: false, + }); + assert.notEqual(utf8, latin1); + }); +}); + +describe('crypto.getRandomValues under Node (AUTH-20)', () => { + it('draws a fresh 128-bit client nonce per stamp', async () => { + const handler = digestHandler('u', 'p'); + const challenge = digestChallenge({ + realm: REALM, + nonce: NONCE, + qop: 'auth', + }); + const request = {method: 'GET', requestTarget: '/x'}; + + const first = /cnonce="([0-9a-f]+)"/u.exec( + await handler.stamp(challenge, request), + ); + const second = /cnonce="([0-9a-f]+)"/u.exec( + await handler.stamp(challenge, request), + ); + + assert.equal(first[1].length, 32); // 16 bytes as hex + assert.notEqual(first[1], second[1]); + }); +}); + +describe('globalThis.btoa under Node (AUTH-14)', () => { + it('base64-encodes the UTF-8 bytes of an ASCII credential', async () => { + const value = await basicHandler('Aladdin', 'open sesame').stamp({ + scheme: 'basic', + params: new Map(), + }); + assert.equal(value, 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='); + }); + + it('base64-encodes the UTF-8 bytes -- not the Latin-1 code units -- of a non-ASCII credential', async () => { + const value = await basicHandler('üser', 'päss').stamp({ + scheme: 'basic', + params: new Map(), + }); + const utf8 = new TextEncoder().encode('üser:päss'); + assert.equal(value, `Basic ${btoa(String.fromCharCode(...utf8))}`); + // A naive `btoa('üser:päss')` would produce a different, shorter string on any runtime that + // accepted it at all -- this is the assertion that catches an encoder swap. + assert.notEqual( + value, + `Basic ${Buffer.from('üser:päss', 'latin1').toString('base64')}`, + ); + }); +}); + +describe('challenge response lifecycle over Node Web Streams (AUTH-30/AUTH-31/AUTH-32)', () => { + const tiers = { + client: createAuthDescriptor([createAuthRequirement('BASIC')]), + }; + const credentials = {basic: new BasicCredential('u', 'p')}; + + it('closes the original 401 before re-driving, and leaves the replacement response open', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const success = countingResponse(200); + const transport = new FakeTransport([ + challenged.response, + success.response, + ]); + + const response = await runThrough( + authStep({credentials, tiers}), + transport, + ); + + assert.equal(transport.sendCount, 2); + assert.equal(challenged.cancelCount(), 1); + assert.equal(success.cancelCount(), 0); + assert.equal(response, success.response); + }); + + it('closes the 401 before propagating a throwing challenge hook (AUTH-32)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([challenged.response]); + const descriptor = authStep({ + credentials: {}, + tiers: {client: createAuthDescriptor([createAuthRequirement('NO_AUTH')])}, + challengeHook: () => Promise.reject(new Error('hook exploded')), + }); + + await assert.rejects(runThrough(descriptor, transport), /hook exploded/u); + assert.equal(challenged.cancelCount(), 1); + }); + + it('leaves an unanswerable 401 open and unclosed -- the caller owns it (AUTH-33)', async () => { + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Negotiate abc123', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + + const response = await runThrough( + authStep({credentials, tiers}), + transport, + ); + + assert.equal(transport.sendCount, 1); + assert.equal(challenged.cancelCount(), 0); + assert.equal(response, challenged.response); + }); +}); + +describe('single-flight cancellation over Node AbortSignal (AUTH-34)', () => { + it('leaves no abort listener behind on a signal reused across many token fetches', async () => { + // `raceAbort` adds one `abort` listener per WAIT and removes it in a `finally`. Drop that + // removal and nothing in the suite fails, but a caller signal outliving many fetches -- one + // request driving a long paginated sweep -- accumulates a dead listener per fetch until Node's + // MaxListenersExceededWarning fires. The signal is never aborted here, so `{once: true}` cannot + // do the cleanup for us: only the explicit removal can. + const cache = new BearerTokenCache(); + const controller = new AbortController(); + + for (let round = 0; round < 12; round += 1) { + const token = createBearerToken(`t${round}`, 10_000); + await cache.stamp({ + provider: () => Promise.resolve(token), + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }); + cache.evict(`Bearer t${round}`); // send the next round back down the fetch path + } + + assert.equal(getEventListeners(controller.signal, 'abort').length, 0); + }); + + it("an aborting caller stops waiting without cancelling a coalesced caller's fetch", async () => { + let release; + let invocations = 0; + const provider = () => { + invocations += 1; + return new Promise(resolve => { + release = resolve; + }); + }; + const cache = new BearerTokenCache(); + const controller = new AbortController(); + + const aborting = cache.stamp({ + provider, + marginMs: 0, + nowMs: 0, + signal: controller.signal, + }); + const patient = cache.stamp({ + provider, + marginMs: 0, + nowMs: 0, + signal: undefined, + }); + assert.equal(invocations, 1); + + const givenUp = new Error('caller A gave up'); + controller.abort(givenUp); + // N1/XCUT-1: the SDK's own terminal type on Node's AbortController too, with the caller's + // reason kept as the cause -- not the raw reason the cache used to rethrow. + await assert.rejects(aborting, error => { + assert.ok(error instanceof CancellationError); + assert.equal(error.cause, givenUp); + return true; + }); + + release(createBearerToken('t1', 10_000)); + assert.equal((await patient).token, 't1'); + assert.equal(invocations, 1); + }); +}); diff --git a/tests/node-conformance/body-lifecycle.test.mjs b/tests/node-conformance/body-lifecycle.test.mjs new file mode 100644 index 0000000..931f5ec --- /dev/null +++ b/tests/node-conformance/body-lifecycle.test.mjs @@ -0,0 +1,413 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/body-lifecycle.test.mjs +// +// Phase 3b's public body surface, driven through the `@dexpace/core` specifier — the path a real consumer +// takes — on Node's Web Streams rather than Bun's. +// +// The reader-lock cases are the reason this file exists. `ReadableStream.cancel()` rejects with a +// TypeError on a locked stream, that check runs BEFORE the state check, and reading to `{done: true}` +// does NOT release the lock. Every one of those is spec text that two independent implementations can +// get subtly different, and getting it wrong turns every successful read into a rejection or silently +// holds a connection open. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + byteArrayBody, + Headers, + materialize, + multipartBody, + Protocol, + Request, + Response, + Status, + streamBody, + stringBody, + toHttpError, +} from '@dexpace/core'; +// The two logging taps are `@internal` -- `body/index.ts` holds them and the public barrel deliberately +// does not, so they are reached by direct `dist/` file path, exactly as `io-byte-stream.test.mjs` reaches +// `io/`. Still the BUILT artifact, never `src/`. +import {withRequestLogging} from '../../packages/core/dist/body/request-body-logging.js'; +import {withResponseLogging} from '../../packages/core/dist/body/response-body-logging.js'; + +/** `Response` above is the SDK's model class, which shadows the platform global this file also needs. */ +const PlatformResponse = globalThis.Response; + +function streamOf(bytes) { + return new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from(bytes)); + controller.close(); + }, + }); +} + +function responseWith(code, body, headers = Headers.newBuilder().build()) { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(code)) + .headers(headers) + .body(body) + .build(); +} + +async function collect(body) { + const chunks = []; + await body.writeTo( + new WritableStream({ + write: chunk => void chunks.push(Uint8Array.from(chunk)), + }), + ); + const total = chunks.reduce((sum, c) => sum + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; +} + +describe('Response reader-lock discipline on Node', () => { + it('bytes() succeeds and closes, rather than being replaced by a cancel-on-locked TypeError', async () => { + const response = responseWith( + 200, + streamOf([...new TextEncoder().encode('hello')]), + ); + assert.equal(new TextDecoder().decode(await response.bytes()), 'hello'); + // Idempotent, and already closed by bytes()' own finally. + await response.close(); + }); + + it('text() decodes with the declared charset and closes', async () => { + const headers = Headers.newBuilder() + .add('content-type', 'text/plain;charset=iso-8859-1') + .build(); + const response = responseWith(200, streamOf([0x68, 0xe9]), headers); + assert.equal(await response.text(), 'hé'); + }); + + it('close() releases the connection even when the body was never read', async () => { + let cancelled = false; + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + cancelled = true; + }, + }); + await responseWith(204, stream).close(); + assert.equal(cancelled, true); + }); + + it('close() tolerates a body an external consumer already locked', async () => { + const stream = streamOf([1, 2, 3]); + const response = responseWith(200, stream); + stream.getReader(); // an external consumer takes the lock; BODY-15 forbids assuming otherwise + // cancel() on a locked stream rejects with TypeError; close() must swallow exactly that one. + await response.close(); + }); + + it('cancels the body at most once however often close is called', async () => { + let cancels = 0; + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + }); + const delegate = stream.cancel.bind(stream); + stream.cancel = async reason => { + cancels += 1; + return delegate(reason); + }; + const response = responseWith(200, stream); + await response.close(); + await response.close(); + await response.close(); + assert.equal(cancels, 1); + }); +}); + +describe('Body.writeTo over Node Web Streams', () => { + it('writes a byte-array body repeatably, byte-for-byte', async () => { + const body = byteArrayBody(Uint8Array.from([9, 8, 7])); + assert.deepEqual([...(await collect(body))], [9, 8, 7]); + assert.deepEqual([...(await collect(body))], [9, 8, 7]); + }); + + it('does not cancel the caller stream when the sink fails', async () => { + // pipeTo's default preventCancel:false would cancel the SOURCE here. Whether a runtime honours + // preventCancel is exactly the kind of Streams-spec detail worth pinning on Node. + let cancelReason = 'NOT-CANCELLED'; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + }, + cancel(reason) { + cancelReason = reason; + }, + }); + const failing = new WritableStream({ + write() { + throw new Error('SOCKET GONE'); + }, + }); + + await assert.rejects(streamBody(source).writeTo(failing), /SOCKET GONE/); + assert.equal(cancelReason, 'NOT-CANCELLED'); + }); + + it('raises when a declared contentLength disagrees with the stream', async () => { + const body = streamBody(streamOf([1, 2]), undefined, 5); + await assert.rejects( + body.writeTo(new WritableStream({write: () => undefined})), + error => error.name === 'EndOfStreamError', + ); + }); + + it('refuses a second write of a single-use body', async () => { + const body = streamBody(streamOf([1])); + await body.writeTo(new WritableStream({write: () => undefined})); + await assert.rejects( + body.writeTo(new WritableStream({write: () => undefined})), + error => error.name === 'ConsumedBodyError', + ); + }); + + it('materializes a single-use stream body into a replayable one', async () => { + const replayed = await materialize(streamBody(streamOf([4, 5, 6]))); + assert.equal(replayed.replayable, true); + assert.deepEqual([...(await collect(replayed))], [4, 5, 6]); + assert.deepEqual([...(await collect(replayed))], [4, 5, 6]); + }); +}); + +describe('MultipartBody framing on Node', () => { + it('generates a boundary from Web Crypto and frames a part', async () => { + // crypto.getRandomValues is a global on the declared floor; if it were not, every multipart body + // this SDK produces would throw, and only running here would reveal it. + const generated = multipartBody([{name: 'a', body: stringBody('x')}]); + assert.match( + generated.mediaType, + /^multipart\/form-data; boundary=dexpace-[A-Za-z0-9]{32}$/, + ); + }); + + it('declares a length equal to the bytes it actually writes', async () => { + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + 'B', + ); + const written = await collect(body); + assert.equal(written.length, body.contentLength); + assert.equal( + new TextDecoder().decode(written), + '--B\r\n' + + 'Content-Disposition: form-data; name="field"\r\n' + + // stringBody declares text/plain; charset=utf-8 by default, so the part carries a Content-Type. + 'Content-Type: text/plain; charset=utf-8\r\n' + + '\r\n' + + 'value\r\n' + + '--B--\r\n', + ); + }); +}); + +describe('toHttpError buffering on Node', () => { + it('buffers a 4xx body and re-serves it replayably after the connection is released', async () => { + const payload = [...new TextEncoder().encode('not found')]; + const error = await toHttpError(responseWith(404, streamOf(payload))); + assert.ok(error); + assert.equal(error.status, 404); + assert.equal(error.preview(), 'not found'); + + const body = error.body(); + assert.equal(body.replayable, true); + assert.deepEqual([...(await collect(body))], payload); + assert.deepEqual([...(await collect(error.body()))], payload); + }); + + it('returns null for a non-error status and leaves the body intact', async () => { + const response = responseWith(200, streamOf([1, 2, 3])); + assert.equal(await toHttpError(response), null); + assert.deepEqual([...(await response.bytes())], [1, 2, 3]); + }); +}); + +describe("the multipart Content-Type parses in Node's own FormData reader (HTTP-51)", () => { + // The reproducer for the boundary-quoting fix, and the reason it belongs here rather than only in + // `bun test`: Bun's `Response.formData()` tolerates an unquoted `boundary=a,b`, Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`. Two independent parsers + // disagreeing about a header this SDK generates is precisely what this tree exists to catch. + for (const boundary of ['a,b', 'bound ary', 'a:b', 'a=b', 'a?b', '(a)/b']) { + it(`round-trips a body framed with ${JSON.stringify(boundary)}`, async () => { + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const parsed = await new PlatformResponse(await collect(body), { + headers: {'content-type': body.mediaType}, + }).formData(); + assert.equal(parsed.get('field'), 'value'); + }); + } + + it('leaves a boundary that is already a bare token unquoted', () => { + const body = multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1'); + assert.equal(body.mediaType, 'multipart/form-data; boundary=plain-1'); + }); +}); + +describe('an exact-length copy refuses a zero-length delivery on Node (HTTP-39/BODY-10)', () => { + it('raises rather than forwarding a chunked-encoding terminator to the sink', async () => { + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths = []; + + await assert.rejects( + streamBody(source, undefined, 3).writeTo( + new WritableStream({write: c => void chunkLengths.push(c.length)}), + ), + error => error.name === 'SourceContractViolationError', + ); + assert.deepEqual(chunkLengths, [2]); + }); + + it('still allows a declared length of 0 over a source that just closes', async () => { + let closed = false; + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo( + new WritableStream({ + write: c => void c, + close: () => void (closed = true), + }), + ); + assert.equal(closed, true); + }); +}); + +describe('withRequestLogging over Node Web Streams (BODY-17..21)', () => { + it('mirrors into the tap while the full untruncated payload reaches the primary', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2, 3, 4, 5])), + 2, + ); + assert.deepEqual([...(await collect(logged))], [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2]); + }); + + it('clears the tap between writes so a retry does not accumulate stale bytes (BODY-18)', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([9, 9])), + 8, + ); + await collect(logged); + await collect(logged); + assert.deepEqual([...logged.snapshot()], [9, 9]); + }); + + it('aborts the real sink when the delegate refuses before ever touching the adapter', async () => { + // A `ConsumedBodyError` on a second write reaches neither handler on the adapter stream, so without + // the wrapper's own catch the primary writer stays open and locked forever -- a held connection. + // Whether an abort dispatched on a writer reaches the underlying sink's algorithm, and does so + // instead of the close algorithm, is runtime plumbing rather than logic. + const logged = withRequestLogging( + streamBody(streamOf([1, 2, 3]), undefined, 3), + 8, + ); + await collect(logged); // consumes the single-use delegate + + let abortReason = 'NOT-ABORTED'; + let closed = false; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + await assert.rejects( + logged.writeTo(destination), + error => error.name === 'ConsumedBodyError', + ); + assert.equal(abortReason.name, 'ConsumedBodyError'); + // Never closed: a broken message must not be committed downstream as a well-formed short one. + assert.equal(closed, false); + }); +}); + +describe('withResponseLogging over Node Web Streams (BODY-22..28)', () => { + function chunked(...chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); + } + + async function readAll(stream) { + const out = []; + for await (const chunk of stream) out.push(...chunk); + return out; + } + + it('serves the prefix then the still-live tail, one pull at a time (BODY-24)', async () => { + const logged = withResponseLogging(chunked([1, 2], [3, 4, 5]), 3); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + }); + + it('cancelling the tail stream cancels the delegate exactly once (BODY-27)', async () => { + let cancels = 0; + const delegate = chunked([1, 2], [3, 4]); + const inner = delegate.cancel.bind(delegate); + delegate.cancel = async reason => { + cancels += 1; + return inner(reason); + }; + const logged = withResponseLogging(delegate, 1); + + await (await logged.read()).cancel(); + await logged.close(); + assert.equal(cancels, 1); + }); + + it('close() leaves the tap inert instead of poisoning it with a detached-reader TypeError', async () => { + // Node reports a released reader as `TypeError [ERR_INVALID_STATE]: Invalid state: The reader is not + // attached to a stream`. That message used to be cached as this wrapper's drain failure and reported + // by error() forever, over a capture that never failed. + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + await logged.close(); + + assert.deepEqual([...logged.snapshot()], []); + await new Promise(resolve => setTimeout(resolve, 0)); + assert.equal(logged.error(), null); + await assert.rejects( + logged.read(), + error => error.name === 'ClosedResourceError', + ); + }); + + it('a fits-cap capture stays repeatably readable after close (BODY-23, BODY-28)', async () => { + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + await logged.close(); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + assert.equal(logged.error(), null); + }); +}); diff --git a/tests/node-conformance/config-primitives.test.mjs b/tests/node-conformance/config-primitives.test.mjs new file mode 100644 index 0000000..f3086dd --- /dev/null +++ b/tests/node-conformance/config-primitives.test.mjs @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/config-primitives.test.mjs +// +// Phase 7a's runtime-divergent surfaces, driven through the `@dexpace/core` specifier on Node rather than +// Bun. Three things here are independent implementations, not shared code: +// +// * `Clock.sleep` races `setTimeout` against an `AbortSignal` and clears the timer on both paths. Bun's +// `AbortSignal` is not Node's — in particular the reason an `abort()` with no argument produces, which +// CFG-17 requires to surface to the caller unchanged. +// * `randomUuid` reads `globalThis.crypto`, which is absent from ESM on every Node 18 release and is the +// reason `engines.node` reads `>=20.3`. +// * `getBuildInfo().runtimeIdentity` feature-detects the host. On Node it must report the real +// `process.version`, which is precisely the claim NFR-15 makes and Bun cannot verify. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + CancellationError, + defaultClock, + getBuildInfo, + randomUuid, +} from '@dexpace/core'; +// `sleepInChunks` is @internal with no public subpath, so it is reached by direct `dist/` path, +// per this suite's import rule. +import {sleepInChunks} from '../../packages/core/dist/config/clock.js'; + +const UUID_V4 = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/u; + +async function rejectionOf(promise) { + try { + await promise; + } catch (reason) { + return reason; + } + throw new Error('expected the promise to reject, but it resolved'); +} + +describe('defaultClock.sleep on Node timers and AbortSignal (CFG-17)', () => { + it('resolves promptly at zero, on the next turn of the event loop', async () => { + // Prompt, but through a real timer rather than a resolved promise: a microtask-only zero starves + // Node's timer and I/O phases, which is exactly what a zero retry backoff would sit inside. + const start = defaultClock.monotonic(); + let timerRan = false; + setTimeout(() => { + timerRan = true; + }, 0); + + await defaultClock.sleep(0); + + assert.equal(timerRan, true); + assert.ok(defaultClock.monotonic() - start < 50); + }); + + it("does NOT let Node silently clamp a duration past one timer's reach (V13)", async () => { + // THE Node-specific behaviour this file exists for. Node clamps a `setTimeout` delay to a 32-bit + // signed integer and rewrites anything larger to 1, emitting only a `TimeoutOverflowWarning` on + // stderr -- so a naive `setTimeout(fn, 2 ** 31)` fires in about a millisecond, turning an + // overflowed retry backoff into a hot loop against the upstream. + // + // Asserted WITHOUT waiting: a real oversized sleep is 24.8 days. `sleepInChunks` takes the slice + // size, so a tiny chunk proves the slicing on real Node timers, and the control below proves + // Node really does clamp -- i.e. that the slicing is load-bearing and not decoration. + const slices = []; + await sleepInChunks(4, undefined, { + chunkMs: 1, + onChunk: sliceMs => slices.push(sliceMs), + }); + assert.deepEqual(slices, [1, 1, 1, 1]); + + // The control: an unsliced oversized delay resolves at once on Node. If this ever starts + // waiting, the platform changed and the chunking could be revisited. + const start = defaultClock.monotonic(); + await new Promise(resolve => { + setTimeout(resolve, 2 ** 31); + }); + assert.ok( + defaultClock.monotonic() - start < 1000, + 'expected Node to clamp an oversized setTimeout delay to ~1ms', + ); + }); + + it('rejects a negative duration with a RangeError', async () => { + assert.ok( + (await rejectionOf(defaultClock.sleep(-1))) instanceof RangeError, + ); + }); + + it("maps Node's own default abort reason to CancellationError, keeping it as cause", async () => { + // Node's default abort reason is a `DOMException` named `AbortError`, constructed by the + // platform -- an independent implementation of Bun's, and the reason this assertion lives here. + const controller = new AbortController(); + controller.abort(); + + const reason = await rejectionOf( + defaultClock.sleep(60_000, controller.signal), + ); + + assert.ok(reason instanceof CancellationError); + assert.equal(reason.cause, controller.signal.reason); + assert.equal(reason.cause.name, 'AbortError'); + }); + + it('keeps a caller-supplied abort reason as cause when cancelled mid-wait', async () => { + const controller = new AbortController(); + const supplied = new Error('cancelled'); + const start = defaultClock.monotonic(); + + const pending = defaultClock.sleep(60_000, controller.signal); + queueMicrotask(() => { + controller.abort(supplied); + }); + + const reason = await rejectionOf(pending); + assert.ok(reason instanceof CancellationError); + assert.equal(reason.cause, supplied); + assert.ok(defaultClock.monotonic() - start < 50); + }); + + it('aborts BETWEEN chunks on Node timers, not only at the end (V13)', async () => { + const controller = new AbortController(); + const supplied = new Error('gave up mid-wait'); + const slices = []; + + const pending = sleepInChunks(10, controller.signal, { + chunkMs: 1, + onChunk: sliceMs => { + slices.push(sliceMs); + if (slices.length === 3) controller.abort(supplied); + }, + }); + + const reason = await rejectionOf(pending); + assert.ok(reason instanceof CancellationError); + assert.equal(reason.cause, supplied); + assert.ok(slices.length < 10); + }); + + it('waits at least the requested duration on Node timers', async () => { + const start = defaultClock.monotonic(); + + await defaultClock.sleep(20); + + assert.ok(defaultClock.monotonic() - start >= 15); + }); + + it('leaves no timer behind that would keep the event loop alive after cancellation', async () => { + // Counted, not inferred from the process exiting. `assert.ok(true)` used to stand here on the + // argument that a leaked 1-hour timer would hang `node --test` -- true, but it reported as a + // passing assertion, and a runner started with `--test-force-exit` would have swallowed it. + // `process.getActiveResourcesInfo()` is Node >= 17, comfortably inside `engines.node`'s 20.3. + const pendingTimers = () => + process.getActiveResourcesInfo().filter(kind => kind === 'Timeout') + .length; + const before = pendingTimers(); + const controller = new AbortController(); + const pending = defaultClock.sleep(3_600_000, controller.signal); + + assert.equal(pendingTimers(), before + 1); + controller.abort(new Error('cancelled')); + await rejectionOf(pending); + + assert.equal(pendingTimers(), before); + }); +}); + +describe('randomUuid on Node WebCrypto (CFG-32)', () => { + it('produces the RFC 4122 version-4 layout from globalThis.crypto', () => { + assert.match(randomUuid(), UUID_V4); + }); + + it('produces no collisions across a batch', () => { + const seen = new Set(); + for (let i = 0; i < 1000; i += 1) seen.add(randomUuid()); + + assert.equal(seen.size, 1000); + }); +}); + +describe('getBuildInfo runtime detection on Node (CFG-36, NFR-15)', () => { + it('reports the real process.version rather than the unknown placeholder', () => { + const {runtimeIdentity} = getBuildInfo(); + + // Still an exact equality -- anything looser loses the ability to catch a hardcoded or mangled + // version -- but the expected value is derived independently: `slice(1)` off an asserted leading + // `v`, not the implementation's own `replace(/^v/u, '')`. Restating the implementation's + // expression and comparing to it is how this assertion previously agreed with itself. + assert.ok(process.version.startsWith('v')); + assert.equal(runtimeIdentity, `node/${process.version.slice(1)}`); + assert.match(runtimeIdentity, /^node\/\d+\.\d+\.\d+/u); + assert.notEqual(runtimeIdentity, 'unknown'); + }); + + it('reports a compiled-in SDK version, never the placeholder', () => { + assert.notEqual(getBuildInfo().sdkVersion, 'unknown'); + }); + + it('carries both identity tokens, none blank', () => { + const {identityTokens} = getBuildInfo(); + + assert.equal(identityTokens.length, 2); + for (const token of identityTokens) assert.notEqual(token.trim(), ''); + }); +}); diff --git a/tests/node-conformance/io-byte-stream.test.mjs b/tests/node-conformance/io-byte-stream.test.mjs new file mode 100644 index 0000000..c86f68c --- /dev/null +++ b/tests/node-conformance/io-byte-stream.test.mjs @@ -0,0 +1,386 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/io-byte-stream.test.mjs +// +// Phase 3a's byte-stream surface, on Node. §5.9:358 names this layer specifically: "chunk boundaries, +// backpressure timing, queueMicrotask ordering" are where Bun's and Node's independent Web Streams +// implementations diverge, and every one of its ~300 unit tests runs only on Bun. +// +// Imported by direct `dist/` file path rather than through the `@dexpace/core` specifier: `io/` is +// `@internal` by design and `exports` maps only `"."`, so there is deliberately no public subpath. This +// is still the BUILT artifact, never `src/` (§5.9:372). +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import {ByteQueue} from '../../packages/core/dist/io/byte-queue.js'; +import {BufferedSource} from '../../packages/core/dist/io/buffered-source.js'; +import {BufferedSink} from '../../packages/core/dist/io/buffered-sink.js'; +import {TeeSink} from '../../packages/core/dist/io/tee-sink.js'; +import {writeAll} from '../../packages/core/dist/io/pump.js'; +import {END_OF_STREAM} from '../../packages/core/dist/io/limits.js'; + +/** A stream that hands out exactly the chunk boundaries the caller asks for. */ +function streamOfChunks(chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); +} + +function collectingStream() { + const chunks = []; + const stream = new WritableStream({ + write: chunk => void chunks.push(Uint8Array.from(chunk)), + }); + return { + stream, + written: () => { + const total = chunks.reduce((sum, c) => sum + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.length; + } + return out; + }, + }; +} + +async function drain(source) { + const staging = new ByteQueue(); + while ((await source.read(staging, 8)) !== END_OF_STREAM) { + /* pull to exhaustion */ + } + return staging.snapshot(); +} + +describe('ByteQueue Uint8Array semantics on Node', () => { + it('keeps a snapshot independent of later mutation, in both directions', () => { + const queue = new ByteQueue(); + const input = Uint8Array.from([1, 2, 3]); + queue.writeBytes(input); + + // The copy is what makes zero-copy subarray transfers between queues safe. A runtime whose + // TypedArray slice/subarray semantics differed here would corrupt every body the SDK sends. + input[0] = 99; + const snapshot = queue.snapshot(); + assert.deepEqual([...snapshot], [1, 2, 3]); + + snapshot[1] = 88; + assert.deepEqual([...queue.snapshot()], [1, 2, 3]); + }); + + it('preserves byte order across arbitrary chunk splits and read increments', () => { + const queue = new ByteQueue(); + for (const chunk of [[1], [2, 3, 4], [], [5, 6], [7, 8, 9, 10]]) { + queue.writeBytes(Uint8Array.from(chunk)); + } + const dest = new ByteQueue(); + for (const take of [3, 1, 4, 2]) queue.read(dest, take); + + assert.equal(queue.size, 0); + assert.deepEqual([...dest.snapshot()], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]); + }); + + it('returns 0 for a zero-count read and END_OF_STREAM only when exhausted', () => { + const queue = new ByteQueue(); + const dest = new ByteQueue(); + assert.equal( + queue.read(dest, 0), + 0, + 'a zero-count read is 0, never end-of-stream', + ); + assert.equal(queue.read(dest, 4), END_OF_STREAM); + queue.writeBytes(Uint8Array.from([1, 2])); + assert.equal(queue.read(dest, 0), 0); + assert.equal(queue.read(dest, 4), 2); + }); +}); + +describe('BufferedSource over a Node ReadableStream', () => { + it('reads an exact count across chunk boundaries the stream chose, not the ones we asked for', async () => { + const source = BufferedSource.overStream( + streamOfChunks([[1, 2], [3], [4, 5, 6]]), + ); + assert.deepEqual([...(await source.readExactly(4))], [1, 2, 3, 4]); + assert.deepEqual([...(await source.readExactly(2))], [5, 6]); + assert.equal(await source.exhausted(), true); + await source.close(); + }); + + it('splits lines when the CRLF terminator straddles two stream chunks', async () => { + // The case hand-picked examples miss and the one most sensitive to how a runtime delivers chunks: + // "\r" ends one chunk and "\n" begins the next. + const source = BufferedSource.overStream( + streamOfChunks([ + [0x61, 0x0d], + [0x0a, 0x62, 0x0a], + ]), + ); + assert.equal(await source.readUtf8Line(), 'a'); + assert.equal(await source.readUtf8Line(), 'b'); + assert.equal(await source.readUtf8Line(), undefined); + await source.close(); + }); + + it('keeps a lone CR as line content rather than treating it as a terminator', async () => { + const source = BufferedSource.overStream( + streamOfChunks([[0x61, 0x0d, 0x62, 0x0a]]), + ); + assert.equal(await source.readUtf8Line(), 'a\rb'); + await source.close(); + }); + + it('serves a slice view without advancing the parent cursor', async () => { + const source = BufferedSource.overStream( + streamOfChunks([ + [0, 1, 2], + [3, 4], + [5, 6, 7, 8, 9], + ]), + ); + const view = source.slice(2, 5); + + assert.deepEqual([...(await drain(view))], [2, 3, 4, 5, 6]); + // Retention has to hold bytes the parent has not reached while the view races ahead — the + // RetentionWindow behavior that depends on when the underlying reader delivers. + assert.deepEqual( + [...(await drain(source))], + [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + ); + await source.close(); + }); + + it('releases the caller stream lock on close', async () => { + const stream = streamOfChunks([[1, 2, 3]]); + const source = BufferedSource.overStream(stream); + assert.equal(stream.locked, true); + await source.close(); + // cancel() cancels the stream but never releases the reader's lock; only releaseLock() does, and a + // leaked lock on a connection-backed source is a held socket. + assert.equal(stream.locked, false); + }); +}); + +describe('BufferedSink and TeeSink over a Node WritableStream', () => { + it('writes exactly the requested count and drains the source only after the write resolves', async () => { + const {stream, written} = collectingStream(); + const sink = BufferedSink.overStream(stream); + const source = new ByteQueue(); + source.writeBytes(Uint8Array.from([1, 2, 3, 4, 5])); + + await sink.write(source, 3); + assert.deepEqual([...written()], [1, 2, 3]); + assert.equal(source.size, 2, 'only the written bytes leave the source'); + await sink.close(); + }); + + it('mirrors into the tap while forwarding the full untruncated payload', async () => { + const {stream, written} = collectingStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 2); + const source = new ByteQueue(); + source.writeBytes(Uint8Array.from([1, 2, 3, 4, 5])); + + await tee.write(source, 5); + // The invariant logging exists for: the wire body is never reduced by the tap. + assert.deepEqual([...written()], [1, 2, 3, 4, 5]); + assert.deepEqual([...tee.snapshot()], [1, 2]); + await tee.close(); + }); + + it('pumps a source to exhaustion through a tee', async () => { + const {stream, written} = collectingStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 4); + const payload = Uint8Array.from( + Array.from({length: 200}, (_, i) => i % 256), + ); + + const total = await writeAll(BufferedSource.overBytes(payload), tee); + await tee.close(); + + assert.equal(total, payload.length); + assert.deepEqual([...written()], [...payload]); + assert.equal(tee.snapshot().length, 4); + }); + + it('surfaces a failed downstream write through flush rather than reporting success', async () => { + const failing = new WritableStream({ + write() { + throw new Error('WIRE DIED'); + }, + }); + const sink = BufferedSink.overStream(failing); + const source = new ByteQueue(); + source.writeBytes(Uint8Array.from([1])); + + await assert.rejects(sink.write(source, 1), /WIRE DIED/); + // Backpressure and error propagation timing is exactly the queueMicrotask-ordering surface §5.9 names. + await assert.rejects(sink.flush(), /WIRE DIED/); + assert.equal( + source.size, + 1, + 'a failed write leaves the payload for the caller to retry', + ); + }); +}); + +// The five Web Streams bridges had no case here at all until #77. Every one of them is a hand-written +// `ReadableStream`/`WritableStream` underlying-source or -sink object, so what they exercise is the +// runtime's own pull scheduling, cancel dispatch and reader-lock bookkeeping — the three things §5.9 +// names and the three that two independent Streams implementations are most likely to differ on. + +describe('BufferedSource.toReadableStream on Node (IO-16)', () => { + it('pulls one chunk at a time instead of draining the source eagerly', async () => { + let pulls = 0; + const stream = new ReadableStream({ + pull(controller) { + pulls += 1; + if (pulls > 4) { + controller.close(); + return; + } + controller.enqueue(Uint8Array.from([pulls])); + }, + }); + const bridge = BufferedSource.overStream(stream).toReadableStream(); + const reader = bridge.getReader(); + + const first = await reader.read(); + assert.deepEqual([...first.value], [1]); + // Node's default queuing strategy reads one chunk ahead, so at most one pull beyond the one just + // served. The assertion that matters is that the whole 4-chunk source has not been materialized. + assert.ok(pulls <= 2, `expected at most 2 pulls, saw ${pulls}`); + await reader.cancel(); + }); + + it('closes the bridge at natural EOF without tearing down the owning source', async () => { + // IO-19: closing the source here would invalidate every outstanding peek/slice view, defeating the + // bridge's most natural usage — take a preview, hand the bridge to the transport, read the preview + // afterwards. Only an explicit cancel closes the source (next case). + const source = BufferedSource.overStream(streamOfChunks([[1, 2], [3]])); + const preview = source.peek(); + const collected = []; + for await (const chunk of source.toReadableStream()) + collected.push(...chunk); + + assert.deepEqual(collected, [1, 2, 3]); + assert.deepEqual([...(await preview.readBytes())], [1, 2, 3]); + assert.equal(source.closed, false); + await source.close(); + }); + + it('cancelling the bridge closes the source AND releases the caller stream lock', async () => { + const stream = streamOfChunks([[1, 2, 3]]); + const source = BufferedSource.overStream(stream); + assert.equal(stream.locked, true); + + await source.toReadableStream().cancel(); + assert.equal(source.closed, true); + // cancel() cancels the stream but never releases the reader's lock; only releaseLock() does, and a + // leaked lock on a connection-backed source is a held socket. + assert.equal(stream.locked, false); + }); + + it('a mid-stream read failure closes the source rather than stranding the lock', async () => { + // The Streams spec does NOT invoke `cancel` on an errored stream, so the bridge has to close the + // source itself on this path. A runtime that dispatched cancel here would hide the bug. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + }, + pull() { + throw new Error('mid-stream read failure'); + }, + }); + const source = BufferedSource.overStream(stream); + const reader = source.toReadableStream().getReader(); + + assert.deepEqual([...(await reader.read()).value], [1, 2, 3]); + await assert.rejects(reader.read(), /mid-stream read failure/); + assert.equal(source.closed, true); + assert.equal(stream.locked, false); + }); +}); + +describe('BufferedSink.toWritableStream on Node (IO-16)', () => { + it('carries a pipeTo through to the destination and closes it', async () => { + // `pipeTo` closes its destination on natural EOF, and IO-16 says closing the bridge closes the + // sink, which closes the caller's stream. Three closes chained through two runtimes' plumbing. + const written = []; + let closed = false; + const destination = new WritableStream({ + write: chunk => void written.push(...chunk), + close: () => void (closed = true), + }); + const sink = BufferedSink.overStream(destination); + + await streamOfChunks([[1, 2], [3]]).pipeTo(sink.toWritableStream()); + assert.deepEqual(written, [1, 2, 3]); + assert.equal(sink.closed, true); + assert.equal(closed, true); + }); + + it('aborting the bridge aborts the sink and carries the reason, rather than closing it', async () => { + // Collapsing an abort into a graceful close commits a cancelled request body downstream as a + // well-formed complete one, so the peer cannot tell an aborted upload from a successful short one. + let closed = false; + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + const sink = BufferedSink.overStream(destination); + const writer = sink.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('user cancelled'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.equal(closed, false); + assert.equal(sink.closed, true); + }); + + it('drops a zero-length chunk rather than forwarding a chunked-encoding terminator', async () => { + const {stream, written} = collectingStream(); + const sink = BufferedSink.overStream(stream); + const writer = sink.toWritableStream().getWriter(); + await writer.write(new Uint8Array(0)); + await writer.write(Uint8Array.from([7])); + await writer.close(); + assert.deepEqual([...written()], [7]); + }); +}); + +describe('TeeSink.toWritableStream on Node (IO-16, IO-26)', () => { + it('routes through the tee, so bytes written to the bridge still reach the tap', async () => { + // Handing callers the PRIMARY's bridge instead would let every byte written through it bypass the + // tap, silently producing an empty capture. + const {stream, written} = collectingStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 2); + + await streamOfChunks([ + [1, 2], + [3, 4, 5], + ]).pipeTo(tee.toWritableStream()); + assert.deepEqual([...written()], [1, 2, 3, 4, 5]); + assert.deepEqual([...tee.snapshot()], [1, 2]); + }); + + it('forwards an abort to the primary while the tap survives to record what was attempted', async () => { + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + abort: reason => void (abortReason = reason), + }); + const tee = new TeeSink(BufferedSink.overStream(destination), 4); + const writer = tee.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('deadline exceeded'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.deepEqual([...tee.snapshot()], [1, 2, 3]); + }); +}); diff --git a/tests/node-conformance/observability.test.mjs b/tests/node-conformance/observability.test.mjs new file mode 100644 index 0000000..dcac9d3 --- /dev/null +++ b/tests/node-conformance/observability.test.mjs @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/observability.test.mjs +// +// Phase 7b's runtime-divergent surfaces, driven through the `@dexpace/core` specifier on Node.js: +// * AsyncLocalStorage store propagation across native Node promises, microtasks, and macrotask timers (OBS-10, OBS-24). +// * activateSpan / activateSpanForCorrelation scope restoration and MDC push on Node (OBS-22, OBS-23). +// * W3C trace/span identifier randomness via globalThis.crypto.getRandomValues on Node (OBS-26, OBS-27). +// * What a caller's async context holds AFTER `await runtime.send()` resolves (OBS-22, OBS-23, OBS-29, +// audit #67 / #80). Node's AsyncLocalStorage is the mechanism under test, not merely the host: the +// leak this pins was `enterWith` installing on the caller's async resource with the restore closure +// running on a later one, and Bun's suite passed over it for nine phases. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + NOOP_SPAN, + PipelineBuilder, + Protocol, + Request, + Response, + Status, + activateSpan, + activateSpanForCorrelation, + createInstrumentationBundle, + createLogger, + getActiveSpan, + loggingStep, +} from '@dexpace/core'; + +describe('observability on Node.js native runtime floor', () => { + it('generates valid W3C trace and span IDs via WebCrypto (OBS-26, OBS-27)', () => { + const bundle = createInstrumentationBundle(); + assert.equal(bundle.isValid, true); + assert.match(bundle.traceId, /^[0-9a-f]{32}$/u); + assert.notEqual(bundle.traceId, '0'.repeat(32)); + assert.match(bundle.spanId, /^[0-9a-f]{16}$/u); + assert.notEqual(bundle.spanId, '0'.repeat(16)); + }); + + it('preserves and restores active span across async execution turns on Node (OBS-22)', async () => { + const mockSpan = { + isRecording: true, + setAttribute() { + return this; + }, + recordException() { + return this; + }, + end() { + return; + }, + }; + + assert.equal(getActiveSpan(), NOOP_SPAN); + const scope = activateSpan(mockSpan); + try { + assert.equal(getActiveSpan(), mockSpan); + await new Promise(resolve => setTimeout(resolve, 5)); + assert.equal(getActiveSpan(), mockSpan); + } finally { + scope.close(); + } + assert.equal(getActiveSpan(), NOOP_SPAN); + }); + + it('folds context and single-emits correctly on Node (OBS-3, OBS-8)', () => { + const emitted = []; + const logger = createLogger((level, fields) => { + emitted.push({level, fields: Object.fromEntries(fields)}); + }); + + const event = logger.atLevel('info'); + event + .event('node.conformance') + .field('runtime', 'node') + .field('null_val', null); + event.emit(); + event.emit(); // second emit must be no-op (OBS-8) + + assert.equal(emitted.length, 1); + assert.equal(emitted[0].level, 'info'); + assert.equal(emitted[0].fields.event, 'node.conformance'); + assert.equal(emitted[0].fields.runtime, 'node'); + assert.equal(emitted[0].fields.null_val, 'null'); + }); + + it('correlates spanContext into logger diagnostic fields (OBS-23)', () => { + const traceId = '4bf92f3577b34da6a3ce929d0e0e4736'; + const spanId = '00f067aa0ba902b7'; + const recordingSpan = { + isRecording: true, + setAttribute() { + return this; + }, + recordException() { + return this; + }, + end() { + return; + }, + spanContext() { + return {traceId, spanId}; + }, + }; + + const emitted = []; + const logger = createLogger((level, fields) => { + emitted.push(Object.fromEntries(fields)); + }); + + const scope = activateSpanForCorrelation(recordingSpan); + try { + logger.atLevel('info').event('correlated.event').emit(); + } finally { + scope.close(); + } + + assert.equal(emitted.length, 1); + assert.equal(emitted[0]['trace.id'], traceId); + assert.equal(emitted[0]['span.id'], spanId); + }); +}); + +/** A transport literal: two methods, no socket. */ +const okTransport = { + send: async request => + Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(null) + .build(), + close: async () => undefined, +}; + +/** A recording tracer whose spans carry a spanContext, so OBS-23's correlation push actually fires. */ +function recordingTracer() { + const started = []; + return { + started, + tracer: { + startSpan(name) { + const record = {name, ended: 0}; + started.push(record); + const span = { + isRecording: true, + setAttribute: () => span, + recordException: () => span, + end: () => { + record.ended += 1; + }, + spanContext: () => ({ + traceId: '4bf92f3577b34da6a3ce929d0e0e4736', + spanId: '00f067aa0ba902b7', + }), + }; + return span; + }, + }, + }; +} + +describe('async context after Runtime.send on Node.js (OBS-22, OBS-23, OBS-29)', () => { + it('leaves the caller the active span and diagnostic fields it had before the call', async () => { + const {tracer, started} = recordingTracer(); + const emitted = []; + // A pipeline built the public way, with the LOGGING pillar inside it: that step is what pushes + // OBS-23's trace.id/span.id, and before 2026-09-05 the push outlived the call. + const runtime = new PipelineBuilder(okTransport, { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append( + loggingStep({ + granularity: 'headers', + logger: createLogger(() => undefined), + }), + ) + .build(); + const request = Request.newBuilder().url('https://example.com/one').build(); + + assert.equal(getActiveSpan(), NOOP_SPAN); + const response = await runtime.send(request); + assert.equal(response.status.code, 200); + + assert.equal(getActiveSpan(), NOOP_SPAN); + createLogger((level, fields) => { + emitted.push(Object.fromEntries(fields)); + }) + .atLevel('info') + .event('application.event.after.send') + .emit(); + assert.equal(emitted.length, 1); + assert.equal(emitted[0]['trace.id'], undefined); + assert.equal(emitted[0]['span.id'], undefined); + + // OBS-29's 1:1 binding, which the leak also broke: the second call opens its own operation span. + await runtime.send(request); + const operationSpans = started.filter( + span => span.name === 'http.client.operation', + ); + assert.equal(operationSpans.length, 2); + assert.deepEqual( + operationSpans.map(span => span.ended), + [1, 1], + ); + }); +}); diff --git a/tests/node-conformance/pagination.test.mjs b/tests/node-conformance/pagination.test.mjs new file mode 100644 index 0000000..1bf5ca9 --- /dev/null +++ b/tests/node-conformance/pagination.test.mjs @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/pagination.test.mjs +// +// Phase 6c's runtime-divergent surface, run against the BUILT artifact on real Node. +// +// Three things in this phase are runtime-divergent across Bun and Node: +// 1. Explicit Resource Management: `Page` installs `[Symbol.asyncDispose]` delegating to `close()`, +// guarded on the symbol existing — it postdates `engines.node`'s >=20.3 floor (it arrived in 20.4), +// so this suite's 20.3.0 matrix leg must assert its ABSENCE, not skip the check. +// 2. `AbortSignal` integration: threading signal into every request exchange and halting pagination walks at boundaries. +// 3. `Response.close()` cancelling active `ReadableStream` bodies upon advance, early break, or completion. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import {Page, pageInfo, Paginator, Request} from '@dexpace/core'; +import { + FakeTransport, + countingResponse, +} from '../../packages/core/dist/testing/fake-transport.js'; + +describe('Page explicit resource management on Node (PAGE-3, PAGE-12)', () => { + // Never index with a bare `Symbol.asyncDispose`. On the >=20.3 floor it is `undefined`, so + // `page[Symbol.asyncDispose]` reads `page['undefined']` — which used to resolve to a junk prototype + // entry left by an unguarded `async [Symbol.asyncDispose]()` class member and made this very + // assertion pass on a Page that could not be disposed at all. Branch on the symbol instead, and + // check the junk key is gone on both legs. + it('leaves no "undefined" prototype key on any Node version (guarded install)', () => { + const {response} = countingResponse(200); + const page = new Page(response, ['item-1']); + assert.ok( + !Object.getOwnPropertyNames(Object.getPrototypeOf(page)).includes( + 'undefined', + ), + 'Page.prototype carries an "undefined" key: [Symbol.asyncDispose] was declared as a plain class member ahead of the floor bump', + ); + assert.equal(typeof page.close, 'function'); + }); + + it('disposes the page via Symbol.asyncDispose, releasing the response body', async t => { + if (typeof Symbol.asyncDispose !== 'symbol') { + t.skip( + `Symbol.asyncDispose is absent on ${process.version} (arrives in 20.4); the guarded install is correctly a no-op here`, + ); + return; + } + const {response, cancelCount} = countingResponse(200); + const page = new Page(response, ['item-1', 'item-2']); + + assert.equal(typeof page[Symbol.asyncDispose], 'function'); + assert.deepEqual(page.items, ['item-1', 'item-2']); + + await page[Symbol.asyncDispose](); + assert.equal(cancelCount(), 1); + // Metadata survives close (PAGE-2) + assert.deepEqual(page.items, ['item-1', 'item-2']); + assert.equal(page.status.code, 200); + }); +}); + +describe('Paginator iteration and cancellation on Node (PAGE-1, PAGE-25, PAGE-26)', () => { + it('walks pages and items, closing responses before yielding items (PAGE-11)', async () => { + const closed = []; + const r1 = countingResponse({body: '{}', onCancel: () => closed.push(0)}); + const r2 = countingResponse({body: '{}', onCancel: () => closed.push(1)}); + + const transport = new FakeTransport([r1, r2]); + + let parseCount = 0; + const strategy = { + parse: async () => { + parseCount += 1; + const nextReq = + parseCount === 1 + ? Request.newBuilder() + .method('GET') + .url('https://api.test/items?page=2') + .build() + : undefined; + return pageInfo([`item-${parseCount}`], nextReq); + }, + }; + + const initialRequest = Request.newBuilder() + .method('GET') + .url('https://api.test/items?page=1') + .build(); + + const paginator = new Paginator({ + transport, + strategy, + initialRequest, + }); + + const items = []; + for await (const item of paginator.items()) { + items.push(item); + } + + assert.deepEqual(items, ['item-1', 'item-2']); + assert.deepEqual(closed, [0, 1]); + assert.equal(transport.sendCount, 2); + }); + + it('stops walk at page boundary on abort, threading AbortSignal (PAGE-25, PAGE-26)', async () => { + const controller = new AbortController(); + const responses = Array.from( + {length: 10}, + () => countingResponse(200).response, + ); + const transport = new FakeTransport(responses); + + const strategy = { + parse: async (_res, template) => { + return pageInfo(['x'], template); + }, + }; + + const initialRequest = Request.newBuilder() + .method('GET') + .url('https://api.test/items') + .build(); + + const paginator = new Paginator({ + transport, + strategy, + initialRequest, + signal: controller.signal, + }); + + let delivered = 0; + for await (const page of paginator.pages()) { + void page; + delivered += 1; + if (delivered === 2) controller.abort(); + } + + assert.equal(delivered, 2); + assert.equal(transport.sendCount, 2); + assert.equal(transport.sentSignals.length, 2); + assert.equal( + transport.sentSignals.every(s => s === controller.signal), + true, + ); + }); +}); diff --git a/tests/node-conformance/recovery-chain.test.mjs b/tests/node-conformance/recovery-chain.test.mjs new file mode 100644 index 0000000..73d5dbf --- /dev/null +++ b/tests/node-conformance/recovery-chain.test.mjs @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/recovery-chain.test.mjs +// +// Phase 4b (`RECOV-12`) is a runtime-divergent surface for one specific reason: the `SuppressedError` +// global. Bun ships it and so does current Node, but it is a V8 global from the full Explicit Resource +// Management proposal and is absent on this package's declared floor (`engines.node ">=20.3"`). A +// `new SuppressedError(...)` written straight into `response-chain.ts` would pass `bun test` and then throw +// `ReferenceError: SuppressedError is not defined` at a consumer's call time — exactly the `NFR-10` trap +// `docs/knowledge/harvested/tooling-and-quality-gates.md:60-61` describes. `suppress()` guards on the global; this +// file is what proves the guarded path actually works on the runtime the SDK ships to, at both ends of the +// matrix. +// +// The close-on-throw half also exercises `RECOV-12`'s "released exactly once" over Node's own Web Streams +// implementation, whose `cancel()` and reader-lock timing are independent of Bun's. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import {Protocol, Request, Response, Status} from '@dexpace/core'; +import { + FallbackSuppressedError, + suppress, +} from '../../packages/core/dist/suppress.js'; +import {ResponseRecoveryChain} from '../../packages/core/dist/recovery/response-chain.js'; +import {success} from '../../packages/core/dist/recovery/outcome.js'; + +function aResponse(body = null) { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .body(body) + .build(); +} + +describe('suppress() on the declared Node floor', () => { + it('produces a shape-compatible error whether or not the runtime has SuppressedError', () => { + const primary = new Error('primary'); + const secondary = new Error('secondary'); + + const result = suppress(primary, secondary, 'teardown failed'); + + assert.ok(result instanceof Error, 'suppress() must return an Error'); + assert.equal(result.name, 'SuppressedError'); + assert.equal( + result.error, + primary, + 'the primary throwable must stay primary', + ); + assert.equal(result.suppressed, secondary); + assert.equal(result.message, 'teardown failed'); + }); + + it('takes the branch this runtime actually has, and both legs of the matrix are covered', () => { + // Not forced by deleting the global — that would not survive parallel execution + // (docs/knowledge/harvested/testing.md:50). The matrix is the forcing function: the pinned 20.3.0 leg has + // no native class and takes the fallback, `lts/*` has one and takes the native branch. Either + // way the result must be usable without the caller knowing which. + const native = globalThis.SuppressedError; + const result = suppress( + new Error('primary'), + new Error('secondary'), + 'teardown failed', + ); + + if (typeof native === 'function') { + assert.ok( + result instanceof native, + 'a runtime with SuppressedError must produce the native class', + ); + } else { + assert.ok( + result instanceof FallbackSuppressedError, + 'a runtime without SuppressedError must produce the stand-in', + ); + } + assert.equal(result.name, 'SuppressedError'); + assert.equal(result.message, 'teardown failed'); + }); + + it('builds the fallback stand-in with the same observable shape', () => { + const primary = new Error('primary'); + const secondary = new Error('secondary'); + + const result = new FallbackSuppressedError( + primary, + secondary, + 'teardown failed', + ); + + assert.ok(result instanceof Error); + assert.equal(result.name, 'SuppressedError'); + assert.equal(result.error, primary); + assert.equal(result.suppressed, secondary); + }); +}); + +describe('RECOV-12 close-on-throw over Node Web Streams', () => { + it('closes the in-hand response exactly once and keeps the step error primary', async () => { + let cancels = 0; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + cancels += 1; + }, + }); + const thrownError = new Error('step failed'); + const chain = new ResponseRecoveryChain( + [ + () => { + throw thrownError; + }, + ], + [], + ); + + const result = await chain.apply(success(aResponse(body))); + + assert.equal(cancels, 1, 'the response must be released exactly once'); + assert.equal(result.kind, 'failure'); + assert.equal( + result.error, + thrownError, + 'the step error must survive by identity', + ); + }); + + it('attaches a close failure as suppressed without displacing the step error', async () => { + const closeError = new Error('close failed'); + const body = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + cancel() { + throw closeError; + }, + }); + const originalError = new Error('step failed'); + const chain = new ResponseRecoveryChain( + [ + () => { + throw originalError; + }, + ], + [], + ); + + const result = await chain.apply(success(aResponse(body))); + + assert.equal(result.kind, 'failure'); + assert.equal(result.error.name, 'SuppressedError'); + assert.equal( + result.error.error, + originalError, + 'RECOV-12: the original stays primary', + ); + assert.equal(result.error.suppressed, closeError); + }); +}); diff --git a/tests/node-conformance/redirect.test.mjs b/tests/node-conformance/redirect.test.mjs new file mode 100644 index 0000000..e70c5ba --- /dev/null +++ b/tests/node-conformance/redirect.test.mjs @@ -0,0 +1,325 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/redirect.test.mjs +// +// Phase 5b is a runtime-divergent surface at two specific points, and both fail silently rather than +// loudly if the runtimes disagree: +// +// 1. `decide.ts` delegates ALL of REDIR-12/13/14/18 to the platform's WHATWG `URL` -- reference +// resolution, percent-encoding preservation, userinfo clearing, the bracketed-IPv6 form, and which +// malformed inputs throw versus resolve as a relative reference. Bun's URL parser is an independent +// implementation of Node's. A divergence here does not crash: `%2F` silently decoding to `/` would +// change the path structure of every followed redirect, and a Location that Node treats as a parse +// failure where Bun treats it as a relative reference would flip "return the 3xx unfollowed" into +// "dispatch a request nobody asked for" -- with `bun test` green throughout. +// 2. PIPE-40/REDIR-22's response-lifecycle discipline rides on Web Streams: each superseded hop is +// released by `Response.close()` (which cancels the body stream), and the final response must be +// left uncancelled. Node's `cancel()`/`pull()` timing is an independent implementation of Bun's, +// and the whole close-count assertion is observed through that hook. +// +// `redirect/` is `@internal` with no public subpath in `exports`, so it is reached by direct `dist/` +// file path, per this suite's import rule. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + CancellationError, + Headers, + Protocol, + Request, + Response, + Status, + TransportFailureError, +} from '@dexpace/core'; +import {createRequestContext} from '../../packages/core/dist/context/context.js'; +import {Cursor} from '../../packages/core/dist/pipeline/cursor.js'; +import {originOf} from '../../packages/core/dist/redirect/cross-origin.js'; +import {decide} from '../../packages/core/dist/redirect/decide.js'; +import {redirectStep} from '../../packages/core/dist/redirect/redirect-step.js'; +import {redirectSettings} from '../../packages/core/dist/redirect/settings.js'; +import { + FakeTransport, + countingResponse, +} from '../../packages/core/dist/testing/fake-transport.js'; + +const SEED_URL = 'https://example.com/start'; + +function aRequest(url = SEED_URL) { + return Request.newBuilder().url(url).build(); +} + +function aRedirect(location, status = 302) { + return Response.newBuilder() + .request(aRequest()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .headers(Headers.newBuilder().setInbound('Location', location).build()) + .body(null) + .build(); +} + +function contextFor(request) { + return { + currentRequest: request, + seedOrigin: originOf(request.url), + visited: new Set([request.url.href]), + redirectsFollowed: 0, + }; +} + +function followedTarget(location, from = SEED_URL) { + const decision = decide( + aRedirect(location), + contextFor(aRequest(from)), + redirectSettings(), + ); + assert.equal(decision.kind, 'follow'); + return decision.nextRequest.url; +} + +function withLocation(response, location) { + return response + .newBuilder() + .headers( + response.headers.newBuilder().setInbound('Location', location).build(), + ) + .build(); +} + +describe("Location resolution on Node's own URL parser", () => { + it('resolves a relative reference against the current hop (REDIR-14)', () => { + assert.equal( + followedTarget('/next', 'https://example.com/a/b').href, + 'https://example.com/next', + ); + }); + + it('never re-encodes an already-percent-encoded path or query (REDIR-13)', () => { + const target = followedTarget('https://example.com/a%2Fb?q=x%26y'); + assert.equal(target.pathname, '/a%2Fb'); + assert.equal(target.search, '?q=x%26y'); + }); + + it('preserves a bracketed IPv6 literal host and an explicit port (REDIR-13)', () => { + const target = followedTarget('https://[2001:db8::1]:8443/x'); + assert.equal(target.hostname, '[2001:db8::1]'); + assert.equal(target.port, '8443'); + }); + + it('drops userinfo without disturbing the rest of the URL (REDIR-12)', () => { + const target = followedTarget('https://user:pass@other.example/x?q=1'); + assert.equal(target.username, ''); + assert.equal(target.password, ''); + assert.equal(target.href, 'https://other.example/x?q=1'); + }); + + it('treats a non-URL string as a relative reference, not a parse failure (REDIR-14)', () => { + // The behavior the `catch` in `resolveLocation` is deliberately NOT relied on for. If Node ever + // threw here where Bun resolves, the step would silently stop following a redirect it should follow. + assert.equal( + followedTarget(' not a url').href, + 'https://example.com/not%20a%20url', + ); + }); + + it('resolves dot segments per RFC 3986 (REDIR-14)', () => { + for (const [location, expected] of [ + ['.', 'https://example.com/a/b/'], + ['..', 'https://example.com/a/'], + ['../../x', 'https://example.com/x'], + ]) { + assert.equal( + followedTarget(location, 'https://example.com/a/b/c').href, + expected, + location, + ); + } + }); + + it('inherits the scheme for a protocol-relative Location (REDIR-14)', () => { + assert.equal( + followedTarget('//other.example/x').href, + 'https://other.example/x', + ); + }); + + it('normalizes case and the default port, which is what makes loop detection hold (REDIR-16)', () => { + // `visited` keys on `href`. If Node normalized differently from Bun here, a loop a Bun-run test + // says is caught would be followable on the runtime this package actually ships to. + assert.equal( + followedTarget('HTTPS://EXAMPLE.COM/a').href, + 'https://example.com/a', + ); + assert.equal( + followedTarget('https://example.com:443/a').href, + 'https://example.com/a', + ); + }); + + it('returns a malformed absolute form unfollowed rather than throwing (REDIR-18)', () => { + const decision = decide( + aRedirect('http://['), + contextFor(aRequest()), + redirectSettings(), + ); + assert.deepEqual(decision, { + kind: 'return-current', + reason: 'malformed-location', + }); + }); + + it('returns an unsupported scheme unfollowed, never dispatching it (REDIR-18)', () => { + for (const raw of [ + 'javascript:alert(1)', + 'data:text/html,x', + 'file:///etc/passwd', + ]) { + const decision = decide( + aRedirect(raw), + contextFor(aRequest()), + redirectSettings(), + ); + assert.deepEqual( + decision, + {kind: 'return-current', reason: 'malformed-location'}, + raw, + ); + } + }); +}); + +describe('redirect response lifecycle over real Node Web Streams', () => { + it('closes every superseded hop and leaves the final response open (PIPE-40)', async () => { + const first = countingResponse(301); + const second = countingResponse(301); + const third = countingResponse(200); + const transport = new FakeTransport([ + withLocation(first.response, 'https://example.com/mid'), + withLocation(second.response, '/final'), + third.response, + ]); + const seed = aRequest(); + + const response = await new Cursor({ + steps: [redirectStep()], + transport, + request: seed, + context: createRequestContext(seed), + }).advance(); + + assert.equal(transport.sendCount, 3); + assert.equal(first.cancelCount(), 1); + assert.equal(second.cancelCount(), 1); + assert.equal(third.cancelCount(), 0); // close-responsibility passes outward to the caller + assert.equal(response, third.response); + }); + + it('returns a loop-detected response open, without throwing (REDIR-16/REDIR-22c)', async () => { + const loop = countingResponse(301); + const located = withLocation(loop.response, SEED_URL); + const transport = new FakeTransport([located]); + const seed = aRequest(); + + const response = await new Cursor({ + steps: [redirectStep()], + transport, + request: seed, + context: createRequestContext(seed), + }).advance(); + + assert.equal(transport.sendCount, 1); + assert.equal(response, located); + assert.equal(loop.cancelCount(), 0); + }); + + it("honors an abort raised DURING a hop, on Node's AbortSignal", async () => { + // The redirect step's own per-hop guard: it runs before the step forks again, so the cursor's + // step-boundary check never sees this abort and the hop response is handed back OPEN, which is + // what PIPE-40 requires on the abandon path. + const controller = new AbortController(); + const hop = countingResponse(301); + const located = withLocation(hop.response, 'https://example.com/next'); + const never = countingResponse(200); + const inner = new FakeTransport([located, never.response]); + const seed = aRequest(); + const aborting = { + send: async (request, options, signal) => { + const response = await inner.send(request, options, signal); + controller.abort(); + return response; + }, + close: () => Promise.resolve(), + }; + + const response = await new Cursor({ + steps: [redirectStep()], + transport: aborting, + request: seed, + context: createRequestContext(seed), + signal: controller.signal, + }).advance(); + + assert.equal(inner.sendCount, 1); + assert.equal(response, located); + assert.equal(hop.cancelCount(), 0); + }); + + it("refuses the walk for a signal already aborted at entry, on Node's AbortSignal", async () => { + // `Cursor` checks the signal at every step boundary, and maps the abort through the SDK's own + // mapper rather than `throwIfAborted()`'s bare DOMException (N1). Node's + // AbortSignal/AbortController is an independent implementation of Bun's, and `signal.reason` + // defaulting is one of the places the two have diverged before -- so the mapped `cause` is + // asserted here and not only under `bun test`. + const controller = new AbortController(); + const reason = new Error('caller went away'); + controller.abort(reason); + const never = countingResponse(200); + const transport = new FakeTransport([never.response]); + const seed = aRequest(); + + await assert.rejects( + new Cursor({ + steps: [redirectStep()], + transport, + request: seed, + context: createRequestContext(seed), + signal: controller.signal, + }).advance(), + error => { + assert.ok(error instanceof CancellationError); + assert.equal(error.cause, reason); + return true; + }, + ); + + assert.equal(transport.sendCount, 0); + assert.equal(never.cancelCount(), 0); + }); + + it('maps a TIMEOUT abort to TransportFailureError, not CancellationError (XCUT-3)', async () => { + // The other half of the mapper: a cancellation must stay distinguishable from a timeout, and + // `AbortSignal.timeout()`'s reason (`TimeoutError`) is runtime-provided. + const signal = AbortSignal.timeout(1); + await new Promise(resolve => { + signal.addEventListener('abort', resolve, {once: true}); + }); + const never = countingResponse(200); + const transport = new FakeTransport([never.response]); + const seed = aRequest(); + + await assert.rejects( + new Cursor({ + steps: [redirectStep()], + transport, + request: seed, + context: createRequestContext(seed), + signal, + }).advance(), + error => { + assert.ok(error instanceof TransportFailureError); + assert.ok(!(error instanceof CancellationError)); + return true; + }, + ); + + assert.equal(transport.sendCount, 0); + }); +}); diff --git a/tests/node-conformance/retry.test.mjs b/tests/node-conformance/retry.test.mjs new file mode 100644 index 0000000..c686241 --- /dev/null +++ b/tests/node-conformance/retry.test.mjs @@ -0,0 +1,239 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/retry.test.mjs +// +// Phase 5a is a runtime-divergent surface at three specific points, and each one fails silently rather +// than loudly if the runtimes disagree: +// +// 1. `classify.ts` draws RETRY-23-vs-RETRY-24 (caller abort never retryable, read timeout still +// retryable) off the abort reason's `name`. That reason is a `DOMException` produced by +// `AbortSignal.timeout()`, whose class and `name` are the runtime's, not this package's -- if +// Node named it anything but `TimeoutError`, every timed-out request would silently stop being +// retried and `bun test` would still be green. +// 2. RETRY-34's trail used to go through `suppress()`, which picks the native `SuppressedError` or +// the shape-compatible fallback depending on the runtime -- Bun has the global, the declared +// floor (`engines.node >=20.3`) does not, and this suite's matrix runs both legs. #72 took that +// branch off the retry path entirely: the final attempt's own error is surfaced and the trail +// rides in a side table. So the assertion moved with it, from "the wrapper has the same shape on +// either runtime" to "neither runtime produces a wrapper", which is the stronger claim and the +// one a reintroduced `suppress()` would break differently on Node 20 than on Node 24. +// 3. RETRY-35/RECOV-16's "release the discarded response" rides on Web Streams: a retired response is +// drained to EOF by `toHttpError()`, an abandoned one is cancelled by `Response.close()`. Node's +// `cancel()`/`pull()` timing is an independent implementation of Bun's. +// 4. The inter-attempt wait itself is `defaultClock.sleep` (CFG-17), the one place the retry path +// touches a real `setTimeout` and a real `AbortSignal` listener. The unit suite injects a fake +// clock -- deliberately, so it stays deterministic -- which means the real timer/abort race is +// covered HERE and nowhere else. +// +// The engine itself is `@internal` with no public subpath in `exports`, so it is reached by direct +// `dist/` file path, per this suite's import rule. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + CancellationError, + Protocol, + Request, + Response, + retryAttempts, + Status, +} from '@dexpace/core'; +import { + isRetryableFailure, + RETRYABLE_STATUSES, +} from '../../packages/core/dist/retry/classify.js'; +import {runWithRetry} from '../../packages/core/dist/retry/engine.js'; +import {retrySettings} from '../../packages/core/dist/retry/settings.js'; +import {failure, success} from '../../packages/core/dist/recovery/outcome.js'; +import {defaultClock} from '../../packages/core/dist/config/clock.js'; + +const GET = Request.newBuilder().url('https://example.com').build(); + +const zeroClock = { + now: () => 0, + monotonic: () => 0, + sleep: () => Promise.resolve(), +}; + +function configOf(overrides) { + return { + settings: retrySettings(overrides), + clock: zeroClock, + random: () => 0.5, + }; +} + +/** Mirrors `testing/fake-transport.ts`'s helper: release is observable only through the body stream. */ +function countingResponse(status) { + let releases = 0; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1])); + }, + pull(controller) { + releases += 1; + controller.close(); + }, + cancel() { + releases += 1; + }, + }); + const response = Response.newBuilder() + .request(GET) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(status)) + .body(body) + .build(); + return {response, cancelCount: () => releases}; +} + +function scriptedDispatch(script) { + const calls = []; + const dispatch = request => { + calls.push(request); + return Promise.resolve( + script[Math.min(calls.length - 1, script.length - 1)], + ); + }; + dispatch.calls = calls; + return dispatch; +} + +describe('retry classification on the declared Node floor', () => { + it("names AbortSignal.timeout()'s reason TimeoutError, which RETRY-24 keys off", async () => { + const signal = AbortSignal.timeout(1); + // A ref'd deadline holds the loop open and fails the case if the abort never arrives -- awaiting + // the unref'd timer alone is what this suite's README warns against. + const aborted = await new Promise(resolve => { + const deadline = setTimeout(() => { + resolve(false); + }, 1000); + signal.addEventListener( + 'abort', + () => { + clearTimeout(deadline); + resolve(true); + }, + {once: true}, + ); + }); + + assert.equal(aborted, true); + assert.equal(signal.reason.name, 'TimeoutError'); + assert.equal(isRetryableFailure(signal.reason, RETRYABLE_STATUSES), true); + }); + + it('treats a caller abort as never retryable (RETRY-23)', () => { + const controller = new AbortController(); + controller.abort(); + + assert.equal(controller.signal.reason.name, 'AbortError'); + assert.equal( + isRetryableFailure(controller.signal.reason, RETRYABLE_STATUSES), + false, + ); + }); +}); + +describe('the retry engine on the declared Node floor', () => { + it('surfaces the final attempt error itself, on a runtime that may lack SuppressedError', async () => { + // Timeout aborts, because they are the retryable throwable this suite can build without reaching + // into another `dist/` module -- two of them exhaust the budget and produce a one-entry trail. + const first = new DOMException('timed out', 'TimeoutError'); + const last = new DOMException('timed out again', 'TimeoutError'); + const dispatch = scriptedDispatch([failure(first), failure(last)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 2, fixedDelayMs: 0}), + ); + + assert.equal(dispatch.calls.length, 2); + assert.equal(outcome.kind, 'failure'); + assert.equal(outcome.error, last); + assert.notEqual(outcome.error.name, 'SuppressedError'); + // RETRY-34's trail, read through the accessor as a CONSUMER reaches it -- the `@dexpace/core` + // specifier and the built `dist/`, not the engine's own module path. + const priors = retryAttempts(outcome.error); + assert.equal(priors.length, 1); + assert.equal(priors[0], first); + }); + + it('releases a discarded response through the drain route, over Node Web Streams (RETRY-35)', async () => { + const discarded = countingResponse(503); + const kept = countingResponse(200); + const dispatch = scriptedDispatch([ + success(discarded.response), + success(kept.response), + ]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({fixedDelayMs: 0}), + ); + + assert.equal(outcome.kind, 'success'); + assert.equal(discarded.cancelCount(), 1); + assert.equal(kept.cancelCount(), 0); + }); + + it('returns a response that survives the gates live and unread (RETRY-36)', async () => { + const only = countingResponse(503); + const dispatch = scriptedDispatch([success(only.response)]); + + const outcome = await runWithRetry( + GET, + dispatch, + configOf({maxAttempts: 1}), + ); + + assert.equal(outcome.kind, 'success'); + assert.equal(only.cancelCount(), 0); + }); + + it('waits on a REAL timer between attempts and resumes the loop (RETRY-26/31)', async () => { + const dispatch = scriptedDispatch([ + failure(new DOMException('timed out', 'TimeoutError')), + success(countingResponse(200).response), + ]); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({fixedDelayMs: 1, maxAttempts: 2}), + clock: defaultClock, + random: () => 0.5, + }); + + assert.equal(outcome.kind, 'success'); + assert.equal(dispatch.calls.length, 2); + }); + + it('cuts a REAL pending wait short when the caller aborts (RETRY-26/32)', async () => { + const controller = new AbortController(); + const dispatch = () => { + // Aborts from a macrotask, so the loop is already inside `defaultClock.sleep`'s timer when it + // fires -- the abort LISTENER settles the race, not the already-aborted short-circuit. + setTimeout(() => { + controller.abort(); + }, 1); + return Promise.resolve( + failure(new DOMException('timed out', 'TimeoutError')), + ); + }; + const startedAt = defaultClock.monotonic(); + + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({fixedDelayMs: 60_000, maxAttempts: 5}), + clock: defaultClock, + random: () => 0.5, + signal: controller.signal, + }); + + assert.equal(outcome.kind, 'failure'); + // The point of the case: it returned instead of sleeping out the full 60s backoff. + assert.ok(defaultClock.monotonic() - startedAt < 5_000); + // XCUT-1 over a REAL AbortSignal and a REAL timer, which is the half the unit suite's injected + // clock cannot reach. The trail is non-empty here by construction, so before #72 this was a + // `SuppressedError` and the assertion below was false. + assert.ok(outcome.error instanceof CancellationError); + }); +}); diff --git a/tests/node-conformance/rx-bridge.test.mjs b/tests/node-conformance/rx-bridge.test.mjs new file mode 100644 index 0000000..ae7a9dc --- /dev/null +++ b/tests/node-conformance/rx-bridge.test.mjs @@ -0,0 +1,366 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/rx-bridge.test.mjs +// +// Phase 8b's runtime-divergent surface, run against the BUILT artifact on real Node. +// +// `@dexpace/rx` is four one-line wrappers over `fromAsyncIterable`, and the whole reason that module is +// hand-written rather than `rxjs`'s own `from()` is a cancellation path whose behavior is decided by the +// runtime, not by this package: +// 1. Unsubscribing while a pull is suspended must reach the source. Whether that release lands depends on +// Node's Web Streams `cancel()` and on Node's async-generator `return()` queueing behind an in-flight +// `next()` -- both independent implementations of Bun's, and the SSE idle case is exactly the state a +// long-lived event stream sits in almost all the time. +// 2. The release ordering the bridge relies on (close the source, THEN return the iterator) only settles a +// suspended pull if the runtime's `ReadableStream` cancellation rejects/resolves the pending read. +// 3. `pages$` unsubscribed mid-walk must close the in-hand page's response body (PAGE-11/PAGE-26) through +// the same generator-return path. +// 4. The adapter's ownership transfer converges THREE release paths on one resource -- `sseEvents$`'s +// `release`, the iterator's `return()`, and `SseStream`'s own quiet release inside it. Whether they +// collapse to a single resource close is decided by Node's `ReadableStream` cancel semantics and by when +// Node resumes a generator parked in `return()`, not by this package. Bun agreeing proves nothing here. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import {firstValueFrom, toArray} from 'rxjs'; +import { + Paginator, + Protocol, + Request, + Response, + sseStreamFrom, + Status, +} from '@dexpace/core'; +import {pageItems$, pages$, sseEvents$, typedSse$} from '@dexpace/rx'; + +/** + * An SSE response whose body stays open after the given text: the reader is left suspended on the next pull, + * which is the idle state the cancellation cases below need. `cancel()` firing is the only sanctioned way to + * observe the release (`Response` instances are frozen, so a spy assignment throws). + */ +function openSseStream(text, onCancel) { + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + }, + cancel() { + onCancel(); + }, + }); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(body) + .build(); + return sseStreamFrom(response); +} + +function closedSseStream(text) { + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + controller.close(); + }, + }); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(body) + .build(); + return sseStreamFrom(response); +} + +const settle = () => new Promise(resolve => setTimeout(resolve, 20)); + +describe('sseEvents$ over Node Web Streams (SSE-41, ASYNC-21)', () => { + it('emits every parsed event in order and completes at end-of-stream', async () => { + const stream = closedSseStream('data: one\n\ndata: two\n\n'); + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + assert.deepEqual( + events.map(event => event.data), + [['one'], ['two']], + ); + }); + + it('releases the response body when unsubscribed while idle (ASYNC-6)', async () => { + let cancelled = 0; + const stream = openSseStream('data: one\n\n', () => { + cancelled += 1; + }); + + const received = []; + const subscription = sseEvents$(stream).subscribe({ + next: event => received.push(event.data[0]), + }); + + await settle(); + assert.deepEqual(received, ['one'], 'the first event should have arrived'); + assert.equal(cancelled, 0, 'an idle stream must stay open until cancelled'); + + subscription.unsubscribe(); + await settle(); + assert.equal(cancelled, 1, 'unsubscribing must release the response body'); + }); + + it('releases the response body when unsubscribed from inside next() (ASYNC-6)', async () => { + let cancelled = 0; + const stream = openSseStream('data: one\n\ndata: two\n\n', () => { + cancelled += 1; + }); + + const subscription = sseEvents$(stream).subscribe({ + next: () => { + subscription.unsubscribe(); + }, + }); + + await settle(); + assert.equal(cancelled, 1); + }); + + it('fails loudly on a second subscription (SSE-26, inherited)', async () => { + const stream = closedSseStream('data: one\n\n'); + const events$ = sseEvents$(stream); + await firstValueFrom(events$.pipe(toArray())); + await assert.rejects(() => firstValueFrom(events$.pipe(toArray()))); + }); +}); + +describe('typedSse$ over Node Web Streams (SSE-33..SSE-36)', () => { + it('decodes events and terminates on the mapper done sentinel', async () => { + const stream = closedSseStream( + 'event: delta\ndata: 1\n\nevent: delta\ndata: 2\n\nevent: end\ndata: x\n\n', + ); + const values = await firstValueFrom( + typedSse$(stream, (eventName, data) => + eventName === 'end' + ? {kind: 'done'} + : {kind: 'value', value: Number(data)}, + ).pipe(toArray()), + ); + assert.deepEqual(values, [1, 2]); + }); + + it('releases the response body when unsubscribed while idle (ASYNC-6)', async () => { + let cancelled = 0; + const stream = openSseStream('data: 100\n\n', () => { + cancelled += 1; + }); + + const subscription = typedSse$(stream, (_eventName, data) => ({ + kind: 'value', + value: Number(data), + })).subscribe({next: () => undefined}); + + await settle(); + assert.equal(cancelled, 0); + + subscription.unsubscribe(); + await settle(); + assert.equal(cancelled, 1); + }); +}); + +describe('pageItems$/pages$ over Node (PAGE-8, ASYNC-6)', () => { + const twoPages = () => { + const closed = []; + let sendCount = 0; + const transport = { + send(request) { + sendCount += 1; + const page = Number(request.url.searchParams.get('page') ?? '1'); + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{}')); + }, + cancel() { + closed.push(page); + }, + }); + return Promise.resolve( + Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(body) + .build(), + ); + }, + close: () => Promise.resolve(), + }; + const strategy = { + parse(_response, template) { + const page = Number(template.url.searchParams.get('page') ?? '1'); + const items = [`item_${page}_1`, `item_${page}_2`]; + if (page >= 2) { + return Promise.resolve({items, nextRequest: undefined}); + } + const nextUrl = new URL(template.url); + nextUrl.searchParams.set('page', String(page + 1)); + return Promise.resolve({ + items, + nextRequest: Request.newBuilder() + .method(template.method) + .url(nextUrl) + .build(), + }); + }, + }; + const paginator = new Paginator({ + transport, + initialRequest: Request.newBuilder() + .method('GET') + .url('https://api.example.com/items?page=1') + .build(), + strategy, + }); + return {paginator, closed, sendCount: () => sendCount}; + }; + + it('walks every page and is cold: a second subscription re-fetches', async () => { + const {paginator, sendCount} = twoPages(); + const items$ = pageItems$(paginator); + + assert.deepEqual(await firstValueFrom(items$.pipe(toArray())), [ + 'item_1_1', + 'item_1_2', + 'item_2_1', + 'item_2_2', + ]); + assert.equal(sendCount(), 2); + + await firstValueFrom(items$.pipe(toArray())); + assert.equal( + sendCount(), + 4, + 'PAGE-8: each subscription drives a fresh walk', + ); + }); + + it('closes the in-hand page body and stops fetching on unsubscribe (PAGE-11, ASYNC-6)', async () => { + const {paginator, closed, sendCount} = twoPages(); + + await new Promise(resolve => { + const subscription = pages$(paginator).subscribe({ + next: () => { + subscription.unsubscribe(); + resolve(); + }, + }); + }); + await settle(); + + assert.equal(sendCount(), 1, 'page 2 must never be requested'); + assert.deepEqual(closed, [1], 'page 1 body must be released'); + }); +}); + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both the levels + * `sseStreamFrom` uses: the reader `BufferedSource` takes, and the stream `Response.close()` cancels. + * + * A structural double, because `ResponseBuilder.body()` stores what it is handed. The platform stream's own + * `cancel` hook cannot do this job: Node invokes it at most once per stream and never after the producer has + * closed the controller, so a second release would collapse into the first and read as clean. + */ +function countingBody(bytes, counts) { + return { + get locked() { + return bytes.locked; + }, + getReader() { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => real.releaseLock(), + cancel: reason => { + counts.source += 1; + return real.cancel(reason); + }, + }; + }, + cancel: reason => { + counts.response += 1; + return bytes.cancel(reason); + }, + }; +} + +/** As {@link openSseStream}/{@link closedSseStream}, but reporting every release the owned resource sees. */ +function countingSseStream(text, ended) { + const counts = {source: 0, response: 0, socket: 0}; + const bytes = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (ended) controller.close(); + }, + cancel() { + counts.socket += 1; + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, counts)) + .build(); + return {stream: sseStreamFrom(response), counts}; +} + +describe('SSE ownership transfer releases once on Node (ASYNC-21 departure, SSE-28)', () => { + it('end-of-source reaches each half of the owned resource exactly once', async () => { + const {stream, counts} = countingSseStream( + 'data: one\n\ndata: two\n\n', + true, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + assert.equal(events.length, 2); + await settle(); + // `socket: 0` is not a miss -- the producer ended the byte stream, so Node has nothing left to tear down. + assert.deepEqual(counts, {source: 1, response: 1, socket: 0}); + }); + + it('unsubscribing while a pull is suspended reaches each half exactly once', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + const subscription = sseEvents$(stream).subscribe({next: () => undefined}); + + await settle(); + assert.deepEqual(counts, {source: 0, response: 0, socket: 0}); + + subscription.unsubscribe(); + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); + + it('a throwing mapper reaches each half exactly once, across three release paths (SSE-36)', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + + // `runMapper`'s explicit close, the adapter's `release`, and the mapping generator's `return()` unwinding + // into the facade's quiet release all run. Exactly one of them may reach the resource. + await assert.rejects(() => + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); +}); diff --git a/tests/node-conformance/seams.test.mjs b/tests/node-conformance/seams.test.mjs new file mode 100644 index 0000000..1ddc013 --- /dev/null +++ b/tests/node-conformance/seams.test.mjs @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/seams.test.mjs +// +// Folded in from the retired `scripts/verify-node-floor.mjs`, whose two assertions were the entirety of +// this repo's Node coverage before this suite existed (checkpoint §5.9). Keeping a second parallel Node +// entry point alongside `test:node` is what §5.9:375 tells us not to do. +// +// This file is the assertion that the declared floor is real rather than aspirational. `engines.node` says +// `">=20.3"`, and two separate built-ins put it there: `globalThis.crypto` — which `MultipartBody` needs to +// generate a boundary — is exposed unflagged only from Node 19.0.0 and is absent from ESM on every Node 18 +// release including 18.20.x; and `AbortSignal.any()`, backported to 18.17.0, reached the 20.x line only in +// 20.3.0. 20.3.0 is the first release carrying both. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import {composeSignal, isTimeoutSignal, RequestOptions} from '@dexpace/core'; + +describe('composeSignal on the declared Node floor', () => { + it('returns a distinct AbortSignal.any() result when both a signal and a timeout are supplied', () => { + const controller = new AbortController(); + const combined = composeSignal(controller.signal, 50); + + assert.ok( + combined instanceof AbortSignal, + 'composeSignal() must return an AbortSignal when both a user signal and a timeout are supplied', + ); + assert.notEqual( + combined, + controller.signal, + 'the combined signal must be a distinct AbortSignal.any() result, not the raw user signal', + ); + }); + + it('propagates a user abort through the composed signal, and does not call it a timeout', () => { + const controller = new AbortController(); + const combined = composeSignal(controller.signal, 60_000); + assert.equal(combined.aborted, false); + + controller.abort(new Error('caller cancelled')); + // Abort propagation through AbortSignal.any() is synchronous per spec, but the two runtimes reach it + // by different implementations — asserting it here rather than only on Bun is the point. + assert.equal(combined.aborted, true); + assert.equal( + isTimeoutSignal(combined), + false, + 'a caller abort must not be misreported as a timeout', + ); + }); + + it('reports a fired timeout by its structured reason, not by instanceof', async () => { + const timeoutOnly = composeSignal(undefined, 5); + assert.ok(timeoutOnly instanceof AbortSignal); + // Not yet fired: `reason` is undefined, so the predicate is false until the timer runs. + assert.equal(isTimeoutSignal(timeoutOnly), false); + + // `AbortSignal.timeout()`'s timer is unref'd on every Node version — deliberately, so a pending + // timeout never keeps a process alive on its own. Awaiting the `abort` event with nothing else + // scheduled therefore lets the loop drain before the 5ms timer runs, and Node 18.17's test runner + // reports that as `Promise resolution is still pending but the event loop has already resolved` + // and cancels the rest of the file. Newer runners hold the loop open through handles of their + // own, which is the whole reason this passed on current LTS and failed on the declared floor. + // Hold it open here rather than depending on the runner: the ref'd deadline keeps the loop alive + // and fails loudly if the timeout never arrives, instead of hanging until the job times out. + await new Promise((resolve, reject) => { + const deadline = setTimeout(() => { + reject(new Error('AbortSignal.timeout(5) did not fire within 5s')); + }, 5_000); + + timeoutOnly.addEventListener( + 'abort', + () => { + clearTimeout(deadline); + resolve(undefined); + }, + {once: true}, + ); + }); + + // The shape of what AbortSignal.timeout() stores in `reason` is the runtime-divergent part: + // isTimeoutSignal reads `reason.name === 'TimeoutError'` precisely because `instanceof DOMException` + // is realm-bound and would fail across a worker or node:vm boundary. + assert.equal(timeoutOnly.aborted, true); + assert.equal(isTimeoutSignal(timeoutOnly), true); + }); + + it('returns undefined when neither a signal nor a timeout is supplied', () => { + assert.equal(composeSignal(undefined, undefined), undefined); + }); +}); + +describe('Web Crypto on the declared Node floor', () => { + it('exposes globalThis.crypto.getRandomValues to an ES module', () => { + // The floor-defining global. `MultipartBody` reads it synchronously at construction, so there is no + // asynchronous fallback available to it, and no `node:crypto` import either — the package is documented + // as runnable on browsers, Deno, Bun and Workers, all of which supply the global. Asserted here in an + // `.mjs` file on purpose: Node 18 exposes `crypto` to CommonJS while leaving it undefined in ESM, so a + // CJS probe would have reported this floor as satisfied when it was not. + assert.equal( + typeof globalThis.crypto?.getRandomValues, + 'function', + 'the declared engines.node floor must expose globalThis.crypto.getRandomValues to ES modules', + ); + }); +}); + +describe('composeSignal timeout range on Node (HTTP-35)', () => { + // Runtime-divergent by measurement, 2026-09-05: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are accepted on Bun, and a + // negative delay raises `RangeError` on Node against `TypeError` on Bun. `bun test` therefore + // cannot assert either half of this, which is what puts the case here rather than only in + // `packages/core/src/seams/transport.test.ts`. Added by audit #67 / #76, which moved the range + // check onto `RequestOptionsBuilder.timeoutMs` for this reason. + it('accepts every timeout RequestOptionsBuilder accepts, at both ends of the range', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + assert.equal(accepted, value); + assert.ok( + composeSignal(undefined, accepted) instanceof AbortSignal, + `composeSignal must accept the timeout ${value}, which the model admits`, + ); + } + }); + + it('would raise RangeError on the values the model now rejects', () => { + for (const value of [1.5, 2 ** 32]) { + assert.throws( + () => composeSignal(undefined, value), + RangeError, + `Node's AbortSignal.timeout() must still reject ${value}; the model is what keeps it unreachable`, + ); + assert.throws(() => RequestOptions.newBuilder().timeoutMs(value), { + name: 'RequestOptionsValidationError', + }); + } + }); +}); diff --git a/tests/node-conformance/serde.test.mjs b/tests/node-conformance/serde.test.mjs new file mode 100644 index 0000000..a10141b --- /dev/null +++ b/tests/node-conformance/serde.test.mjs @@ -0,0 +1,382 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/serde.test.mjs +// +// Phase 6a's runtime-divergent surface, run against the BUILT artifact on real Node. +// +// Three things in this phase are independent implementations on Bun and on Node, and `bun test` only +// ever exercises Bun's: +// +// 1. Web Streams. `serializeTo` takes a caller-owned `WritableStream` and must release its writer +// lock without closing it; `deserializeFrom` takes a caller-owned `ReadableStream` and must read +// to EOF without cancelling it. Lock and close semantics are exactly where the two runtimes' +// stream implementations have diverged before. +// 2. `TextDecoder`'s streaming mode, which is what keeps a multi-byte character intact when it is +// split across two chunks. +// 3. `suppress()`'s runtime guard on `decodeResponse`'s close-failure path. The `20.3.0` leg of the +// CI matrix has no native `SuppressedError` and takes the fallback branch; the `lts/*` leg takes +// the native one. Asserted on SHAPE, never `instanceof`, for exactly that reason. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + absent, + decodeResponse, + DeserializationError, + nullValue, + present, + Protocol, + Request, + Response, + serdeBody, + Status, +} from '@dexpace/core'; +import {jsonSerde, tristateObject} from '@dexpace/codec-json'; + +const identity = {parse: input => input}; + +function streamOf(...chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(chunk); + controller.close(); + }, + }); +} + +describe("serializeTo against Node's WritableStream (SERDE-3)", () => { + it('writes fully, leaves the sink open, and releases the writer lock', async () => { + const written = []; + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write(chunk) { + written.push(Buffer.from(chunk).toString('utf8')); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + + const {serializer} = jsonSerde(); + await serializer.serializeTo({a: 1}, sink); + // A second write proves the lock really came back — a still-locked sink throws from getWriter(). + await serializer.serializeTo({b: 2}, sink); + + assert.equal(written.join(''), '{"a":1}{"b":2}'); + assert.equal( + closed, + false, + 'the caller owns the sink; the codec must not close it', + ); + assert.equal(aborted, false); + assert.equal(sink.locked, false); + }); +}); + +describe("deserializeFrom against Node's ReadableStream (SERDE-3, SERDE-12)", () => { + it("reads to EOF without cancelling the caller's source, and releases the reader lock", async () => { + let cancelled = false; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('{"id"')); + controller.enqueue(Buffer.from(':42}')); + controller.close(); + }, + cancel() { + cancelled = true; + }, + }); + + const value = await jsonSerde().deserializer.deserializeFrom(source, { + schema: identity, + typeName: 'Dto', + }); + + assert.deepEqual(value, {id: 42}); + assert.equal(cancelled, false); + assert.equal(source.locked, false); + }); + + it('propagates a stream failure unwrapped and still releases the lock', async () => { + const failure = new Error('socket reset'); + const source = new ReadableStream({ + start(controller) { + controller.error(failure); + }, + }); + + await assert.rejects( + jsonSerde().deserializer.deserializeFrom(source, { + schema: identity, + typeName: 'Dto', + }), + caught => { + assert.equal( + caught, + failure, + 'a stream failure must reach the caller untouched', + ); + return true; + }, + ); + assert.equal(source.locked, false); + }); + + it('keeps a multi-byte character intact when it is split across two chunks', async () => { + const full = Buffer.from('{"id":1,"n":"ü"}', 'utf8'); + const split = full.indexOf(0xc3); // the first byte of "ü" + + const value = await jsonSerde().deserializer.deserializeFrom( + streamOf(full.subarray(0, split + 1), full.subarray(split + 1)), + {schema: identity}, + ); + + assert.deepEqual(value, {id: 1, n: 'ü'}); + }); +}); + +describe('the Tristate wire contract on Node (SERDE-15, SERDE-17, SERDE-20)', () => { + it('omits Absent, emits Null, and unwraps Present', () => { + const encoded = Buffer.from( + jsonSerde().serializer.serialize({ + keep: absent(), + clear: nullValue(), + set: present('v'), + }), + ).toString('utf8'); + + assert.equal(encoded, '{"clear":null,"set":"v"}'); + }); + + it('degrades to a wire null where a key cannot be dropped', () => { + const {serializer} = jsonSerde(); + const encode = value => + Buffer.from(serializer.serialize(value)).toString('utf8'); + + assert.equal(encode(absent()), 'null'); + assert.equal(encode([present(1), absent()]), '[1,null]'); + }); + + it('resolves a missing key to Absent on the decode side', () => { + const schema = tristateObject({age: identity}); + + assert.equal(schema.parse({}).age.kind, 'absent'); + assert.equal(schema.parse({age: null}).age.kind, 'null'); + assert.equal(schema.parse({age: 30}).age.kind, 'present'); + }); +}); + +describe('serdeBody across the package boundary (SERDE-2)', () => { + it("stamps the codec's declared media type and produces a replayable body", async () => { + const body = serdeBody({name: 'ada', nickname: absent()}, jsonSerde()); + + assert.equal(body.mediaType, 'application/json'); + assert.equal(body.replayable, true); + + const chunks = []; + await body.writeTo( + new WritableStream({ + write(chunk) { + chunks.push(Buffer.from(chunk).toString('utf8')); + }, + }), + ); + assert.equal(chunks.join(''), '{"name":"ada"}'); + assert.equal(body.contentLength, '{"name":"ada"}'.length); + }); +}); + +function aResponse(body = null) { + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .body(body) + .build(); +} + +describe("decodeResponse's close-failure path on the declared Node floor (SERDE-27)", () => { + it('decodes a real Response and releases it exactly once', async () => { + const response = aResponse(streamOf(Buffer.from('{"id":7}'))); + + const value = await decodeResponse(response, jsonSerde().deserializer, { + schema: identity, + typeName: 'Dto', + }); + + assert.deepEqual(value, {id: 7}); + // `close()` memoizes, so a second call is a no-op rather than a second cancel; what matters is + // that the body was consumed and the response is no longer holding the stream open. + await response.close(); + assert.equal(response.body?.locked ?? false, false); + }); + + it('keeps the decode failure primary when releasing the response ALSO fails', async () => { + // The whole reason this file exists at this line: the pairing is built by `suppress()`, whose + // branch depends on whether the runtime has a native `SuppressedError`. The `20.3.0` leg of the + // matrix does not and takes the fallback; `lts/*` does. Asserted on SHAPE for that reason — an + // `instanceof SuppressedError` check would silently assert nothing on the floor. + // + // The deserializer rejects WITHOUT draining, which is what leaves the body live enough for + // `close()` to reach the underlying `cancel()` at all: a stream already read to EOF cancels + // trivially and never raises, so the real codec cannot reach this path. + const closeFailure = new Error('cancel exploded'); + const response = aResponse( + new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('{"id":1}')); + }, + cancel() { + throw closeFailure; + }, + }), + ); + const decodeFailure = new DeserializationError('malformed payload'); + const failingDeserializer = { + deserialize() { + throw decodeFailure; + }, + deserializeFrom() { + return Promise.reject(decodeFailure); + }, + }; + + await assert.rejects( + decodeResponse(response, failingDeserializer, { + schema: identity, + typeName: 'Dto', + }), + caught => { + assert.equal(caught.name, 'SuppressedError'); + assert.equal( + caught.error, + decodeFailure, + 'the decode failure must stay primary, not be replaced by the release failure', + ); + assert.equal(caught.suppressed, closeFailure); + return true; + }, + ); + }); +}); + +/** + * Fails the case instead of hanging it. `node --test` has no default per-test timeout, so a + * regression in the abort race would park the runner for as long as CI allows rather than reporting + * anything. The timer is ref'd, which also holds the loop open while the abort is in flight. + */ +async function settleWithin(promise, ms) { + let timer; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout( + () => reject(new Error(`did not settle within ${ms}ms`)), + ms, + ); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** Whatever `promise` rejected with, or the string marker when it resolved. */ +async function rejection(promise) { + try { + await promise; + return 'RESOLVED'; + } catch (e) { + return e; + } +} + +describe('an abort landing on a PENDING read or write (SERDE-3, audit #67 / #79)', () => { + // Runtime-divergent twice over. `AbortSignal` and Web Streams are independent implementations + // here, and the two disagree on what a reader release does to an outstanding read: measured + // 2026-09-05, Bun 1.3.14 rejects it with an `AbortError` and Node 20.3/26 with + // `TypeError: Invalid state: Releasing reader`. Neither may reach the caller in place of its own + // abort reason, and neither may escape as an unhandled rejection — which `node --test` would + // report as a failure of this file even if every assertion below passed. + it('settles deserializeFrom with the caller reason and unlocks the source', async () => { + let cancelled = false; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('{"id"')); + }, + // Parks the drain inside its second `read()`: the state a between-chunks check cannot see. + pull() { + return new Promise(() => {}); + }, + cancel() { + cancelled = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + const settled = rejection( + jsonSerde().deserializer.deserializeFrom( + source, + {schema: identity, typeName: 'Dto'}, + {signal: controller.signal}, + ), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(source.locked, false, 'the caller must get its source back'); + assert.equal(cancelled, false, 'the source is caller-owned (SERDE-3)'); + }); + + it('settles serializeTo with the caller reason and unlocks the sink', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write() { + return new Promise(() => {}); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + const settled = rejection( + jsonSerde().serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(sink.locked, false, 'the caller must get its sink back'); + assert.equal(closed, false, 'the sink is caller-owned (SERDE-3)'); + assert.equal(aborted, false); + }); + + it('leaves a completed drain untouched when the signal never fires', async () => { + const controller = new AbortController(); + const value = await jsonSerde().deserializer.deserializeFrom( + streamOf(Buffer.from('{"id"'), Buffer.from(':42}')), + {schema: identity}, + {signal: controller.signal}, + ); + + assert.deepEqual(value, {id: 42}); + // The listener is removed on the way out, so a later abort reaches nothing at all — an + // unremoved one would reject a promise nobody is waiting on any more. + controller.abort(new Error('too late')); + }); +}); diff --git a/tests/node-conformance/sse.test.mjs b/tests/node-conformance/sse.test.mjs new file mode 100644 index 0000000..9dd056c --- /dev/null +++ b/tests/node-conformance/sse.test.mjs @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/sse.test.mjs +// +// Phase 6b's runtime-divergent SSE surface, run against the BUILT artifact on real Node. +// +// Key runtime-divergent points asserted on real Node Web Streams: +// 1. Web Streams reader-lock discipline: releaseLock() on Node's ReadableStream while a read is in +// flight rejects with TypeError, which SseStream maps to IoError (SSE-31). +// 2. Response body cancellation and double release in closingBoth(). +// 3. TextDecoder ignoreBOM behavior across line boundaries on Node. +// 4. Async generator teardown (.return()) and resource release on early break. +// 5. AbortSignal listener lifecycle on Node. +import assert from 'node:assert/strict'; +import {describe, it} from 'node:test'; +import { + MAPPER_DONE, + MAPPER_SKIP, + mapperValue, + Protocol, + Request, + Response, + sseStreamFrom, + SseStreamError, + Status, + typedSseStream, +} from '@dexpace/core'; + +function streamOf(...chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) { + controller.enqueue( + typeof chunk === 'string' ? new TextEncoder().encode(chunk) : chunk, + ); + } + controller.close(); + }, + }); +} + +function responseOver(body) { + const req = Request.newBuilder() + .url('https://example.com/events') + .method('GET') + .build(); + return Response.newBuilder() + .request(req) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .body(body) + .build(); +} + +describe('SSE stream over Node Web Streams', () => { + it('parses events and preserves pull discipline on Node (SSE-1..8, SSE-39)', async () => { + let pulls = 0; + const body = new ReadableStream( + { + pull(controller) { + pulls++; + if (pulls > 2) { + controller.close(); + return; + } + controller.enqueue( + new TextEncoder().encode(`event: msg\ndata: item-${pulls}\n\n`), + ); + }, + }, + {highWaterMark: 0}, + ); + + const stream = sseStreamFrom(responseOver(body)); + const events = []; + for await (const event of stream) { + events.push(event); + if (events.length === 1) { + // Assert only one pull occurred to get the first event + assert.equal(pulls, 1); + } + } + assert.equal(events.length, 2); + assert.equal(events[0].event, 'msg'); + assert.deepEqual(events[0].data, ['item-1']); + assert.equal(events[1].event, 'msg'); + assert.deepEqual(events[1].data, ['item-2']); + }); + + it('strips leading BOM once and preserves subsequent BOM on Node (SSE-12)', async () => { + const bomPrefix = new Uint8Array([0xef, 0xbb, 0xbf]); + const payload = new TextEncoder().encode( + 'data: first\n\n\uFEFFdata: second\n\n', + ); + const combined = new Uint8Array(bomPrefix.length + payload.length); + combined.set(bomPrefix, 0); + combined.set(payload, bomPrefix.length); + + const stream = sseStreamFrom(responseOver(streamOf(combined))); + const events = []; + for await (const event of stream) { + events.push(event); + } + // First event has leading BOM stripped by line reader + assert.equal(events.length, 1); + assert.deepEqual(events[0].data, ['first']); + }); + + it('maps in-flight reader teardown to IoError on Node (SSE-31)', async () => { + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: initial\n\n')); + }, + }); + + const stream = sseStreamFrom(responseOver(body)); + const iterator = stream[Symbol.asyncIterator](); + const first = await iterator.next(); + assert.deepEqual(first.value.data, ['initial']); + + // Next pull blocks in Node Web Streams read + const pendingPull = iterator.next(); + await stream.close(); + + await assert.rejects( + async () => { + await pendingPull; + }, + err => { + assert.equal(err.name, 'IoError'); + return true; + }, + ); + }); + + it('releases response and reader locks on early break (SSE-25)', async () => { + let cancelled = false; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: 1\n\ndata: 2\n\n')); + }, + cancel() { + cancelled = true; + }, + }); + + const stream = sseStreamFrom(responseOver(body)); + for await (const event of stream) { + assert.deepEqual(event.data, ['1']); + break; + } + assert.equal(cancelled, true); + }); + + it('removes abort listener and prevents memory leaks on normal completion', async () => { + const controller = new AbortController(); + const body = streamOf('data: done\n\n'); + const stream = sseStreamFrom(responseOver(body), { + signal: controller.signal, + }); + + for await (const event of stream) { + assert.deepEqual(event.data, ['done']); + } + // Stream completed and closed cleanly + }); + + it('guards against re-iteration and post-close iteration on Node (SSE-26, SSE-27)', async () => { + const stream = sseStreamFrom(responseOver(streamOf('data: a\n\n'))); + const it1 = stream[Symbol.asyncIterator](); + assert.throws(() => stream[Symbol.asyncIterator](), SseStreamError); + + await stream.close(); + assert.throws(() => stream[Symbol.asyncIterator](), SseStreamError); + void it1; + }); +}); + +describe('typed SSE adapter on Node', () => { + it('lazily transforms events and terminates on mapper done (SSE-33..35)', async () => { + const body = streamOf('data: 1\n\ndata: 2\n\ndata: 3\n\n'); + const stream = sseStreamFrom(responseOver(body)); + const typed = typedSseStream(stream, (_name, data) => { + const num = Number(data); + if (num === 2) return MAPPER_SKIP; + if (num === 3) return MAPPER_DONE; + return mapperValue(num * 10); + }); + + const values = []; + for await (const val of typed) { + values.push(val); + } + assert.deepEqual(values, [10]); + }); + + it('releases stream resource before propagating mapper error on Node (SSE-36)', async () => { + let cancelled = false; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: boom\n\n')); + }, + cancel() { + cancelled = true; + }, + }); + + const stream = sseStreamFrom(responseOver(body)); + const mapperError = new Error('mapper failed'); + const typed = typedSseStream(stream, () => { + throw mapperError; + }); + + await assert.rejects(async () => { + for await (const val of typed) { + void val; + } + }, mapperError); + + assert.equal(cancelled, true); + }); +}); diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs new file mode 100644 index 0000000..92541c0 --- /dev/null +++ b/tests/node-conformance/transport.test.mjs @@ -0,0 +1,581 @@ +// SPDX-License-Identifier: MIT +// tests/node-conformance/transport.test.mjs +// +// Phase 8a's Node layer. This is the file the suite's membership rule was written for: `bun test` runs both +// transports against *Bun's* `fetch`, `AbortSignal`, and Web Streams, and the shipping runtime is Node's — +// two independent implementations of exactly the surfaces a transport is made of. Bun's `undici` shim alone +// already diverges enough that `undici-transport.ts` has to bypass it by module path. +// +// It is also the only layer that can join BODY-11 to TRANSPORT-28: `@dexpace/body-file` is a Node-only +// package and neither transport depends on it (they narrow structurally on `body.kind === 'file'`), so a real +// `fileBody()` crossing a real transport has no home inside either package's own suite. +// +// Exercises: TRANSPORT-1 (redirects not followed), TRANSPORT-4/20 (timeout and no-response classification), +// TRANSPORT-20 with RETRY-2 (an unsupported URL scheme is a permanent misconfiguration outside the IoError +// tree -- Node and Bun report it with entirely different error shapes), +// TRANSPORT-17 (a single-use body written once, its bytes on the wire), TRANSPORT-24 (vendor status codes), +// TRANSPORT-11/12 (a header the native layer refuses is dropped, not a failed send -- Node's undici-backed +// `fetch` rejects three names Bun's forwards), +// TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), BODY-13 (a truncate-after-stat +// short write fails the send on the streamed path, which only this runtime can assert), +// TRANSPORT-24/25 (a 204 and a HEAD carry a null body on this runtime as well -- Node's `fetch` +// returns null where Bun's returns a stream, and undici's dispatcher always returns a readable), +// HTTP-35 (a defaultTimeoutMs AbortSignal.timeout() cannot take is refused at the factory -- Node +// throws RangeError for two of the values Bun accepts), +// TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 +// (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). +import assert from 'node:assert/strict'; +import {createServer} from 'node:http'; +import {after, before, describe, it} from 'node:test'; +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {createHash} from 'node:crypto'; +import {Headers, isIoError, Request, RequestOptions} from '@dexpace/core'; +import {fileBody} from '@dexpace/body-file'; +import {fetchTransport} from '@dexpace/transport-fetch'; +import {undiciTransport} from '@dexpace/transport-undici'; + +/** Long enough that no timeout under test wins the race by luck. */ +const SLOW_RESPONSE_MS = 5_000; + +let server; +let origin; + +/** A genuinely single-use body: `replayable: false` forces the streaming request-body path on both transports. */ +function countingBody(counter) { + const payload = new TextEncoder().encode('payload'); + return { + kind: 'stream', + mediaType: 'text/plain', + contentLength: payload.byteLength, + replayable: false, + async writeTo(sink) { + counter.writes += 1; + const writer = sink.getWriter(); + await writer.write(payload); + await writer.close(); + }, + }; +} + +/** + * Distinguishable bytes, so a truncated or misaligned send fails the digest and not merely the + * length. Printable ASCII rather than the full byte range: the shared `/echo` fixture echoes the + * request body back as a UTF-8 string, which would mangle arbitrary bytes before any assertion here + * could see them. + */ +function fixtureBytes(size) { + const buf = Buffer.alloc(size); + for (let index = 0; index < size; index += 1) { + buf[index] = 33 + ((index * 7) % 94); + } + return buf; +} + +const sha = bytes => createHash('sha256').update(bytes).digest('hex'); + +/** `/fixed-length`'s payload; its length is what the HEAD response advertises and never delivers. */ +const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + +// Every hook and test lives inside this suite rather than at the file root, and that is +// load-bearing on the declared floor. Under Node 20.3.0 -- `engines.node`, and the floor leg of +// CI's node-conformance matrix -- an async ROOT-level `before` does not finish before subtests +// inside a `describe` start, in a file whose only root children are suites. This file is exactly +// that shape: the loop below contributes two `describe`s and no top-level `it`, so every test read +// `origin` as `undefined` and failed with `malformed or non-absolute URL: undefined/redirect`, +// while the matching root `after` never closed the server and the run hung. Node 22 fixed the +// ordering. Owning the hooks from a suite is correct on every version, and neither `bun test` nor +// a newer local Node can see the difference -- only the matrix floor leg can. +describe('the transport adapters on the Node runtime', () => { + before(async () => { + server = createServer((req, res) => { + const {pathname} = new URL(req.url ?? '/', 'http://localhost'); + if (pathname === '/slow') { + setTimeout(() => { + res.writeHead(200); + res.end('done'); + }, SLOW_RESPONSE_MS).unref(); + return; + } + if (pathname === '/redirect') { + res.writeHead(302, {location: '/echo'}); + res.end(); + return; + } + if (pathname === '/vendor') { + res.writeHead(520, {'content-type': 'text/plain'}); + res.end('vendor status body'); + return; + } + if (pathname === '/no-content') { + res.writeHead(204); + res.end(); + return; + } + if (pathname === '/fixed-length') { + // `node:http` suppresses the body for a HEAD request by itself and keeps the declared + // length, which is the trap: the header promises bytes no response will deliver. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return; + } + const chunks = []; + req.on('data', chunk => chunks.push(chunk)); + req.on('end', () => { + res.writeHead(200, {'content-type': 'application/json'}); + res.end( + JSON.stringify({ + headers: req.headers, + body: Buffer.concat(chunks).toString('utf8'), + }), + ); + }); + }); + await new Promise(resolve => { + server.listen(0, '127.0.0.1', resolve); + }); + origin = `http://127.0.0.1:${server.address().port}`; + }); + + after(async () => { + server.closeAllConnections(); + await new Promise(resolve => { + server.close(resolve); + }); + }); + + // `makeTransport` takes the transport-wide default timeout so the HTTP-35 case can build a + // misconfigured transport; every other case passes nothing and gets today's shape. + for (const [name, makeTransport] of [ + ['transport-fetch', defaultTimeoutMs => fetchTransport({defaultTimeoutMs})], + [ + 'transport-undici', + defaultTimeoutMs => undiciTransport({defaultTimeoutMs}), + ], + ]) { + describe(`${name} on the Node runtime`, () => { + it('returns a 302 raw and never follows it (TRANSPORT-1)', async () => { + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/redirect`).build(), + ); + assert.equal(response.status.code, 302); + assert.equal(response.headers.get('location'), '/echo'); + await response.close(); + } finally { + await transport.close(); + } + }); + + it('drops the headers the native layer refuses rather than failing the send (TRANSPORT-11/12)', async () => { + // The one case in this file where Bun and Node disagree about the *outcome*, not merely the + // implementation. `Expect`, `Keep-Alive` and `Upgrade` are undici's three unconditional + // rejections, and Node's global `fetch` is undici-backed, so on this runtime an undropped + // one rejects the send: `TypeError: fetch failed` through transport-fetch, which can only + // be classified as the RETRYABLE TransportFailureError, and a terminal TypeError through + // transport-undici. Bun's `fetch` instead forwards `Expect`/`Keep-Alive` to the wire and + // hangs on `Upgrade`, so the Bun conformance rows for these names prove a weaker claim than + // this one does (audit #67 / #81, measured 2026-09-05). + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder() + .url(`${origin}/echo`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(), + RequestOptions.newBuilder().timeoutMs(2_000).build(), + ); + const echoed = JSON.parse(await response.text()); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + assert.equal(echoed.headers[name], undefined, name); + } + assert.equal(echoed.headers['x-pass-through'], 'survives'); + } finally { + await transport.close(); + } + }); + + it('surfaces a vendor status with a readable body (TRANSPORT-24)', async () => { + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/vendor`).build(), + ); + assert.equal(response.status.code, 520); + assert.equal(await response.text(), 'vendor status body'); + } finally { + await transport.close(); + } + }); + + it('writes a single-use streaming body exactly once, bytes intact (TRANSPORT-17)', async () => { + // Node streams a request body through `duplex: 'half'` (fetch) or a `Readable` (undici); Bun's + // handling of both is its own implementation, which is the whole reason this case is here. + const transport = makeTransport(); + const counter = {writes: 0}; + try { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(countingBody(counter)) + .build(), + ); + const echoed = JSON.parse(await response.text()); + assert.equal(echoed.body, 'payload'); + assert.equal(counter.writes, 1); + } finally { + await transport.close(); + } + }); + + it('reports a null body for a 204 and a HEAD, on this runtime too (TRANSPORT-24/25)', async () => { + // The one place the WHATWG null-body rule can be checked against the runtime the SDK ships + // to. Node's `fetch` returns `null` for 204/304/HEAD by itself, Bun 1.3.14's returns a live + // `ReadableStream` for all three, and undici's dispatcher always hands back a + // `BodyReadable` -- so the Bun conformance rows prove the adapters normalise Bun's answers + // and this proves they did not normalise into Bun's shape (audit #67 / #82). + const transport = makeTransport(); + try { + const empty = await transport.send( + Request.newBuilder().url(`${origin}/no-content`).build(), + ); + assert.equal(empty.status.code, 204); + assert.equal(empty.body, null); + await empty.close(); + + const head = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(`${origin}/fixed-length`) + .build(), + ); + assert.equal(head.status.code, 200); + assert.equal(head.body, null); + // The advertised length survives; only the body a GET would have returned is absent. + assert.equal( + head.headers.get('content-length'), + String(FIXED_LENGTH_BODY.length), + ); + await head.close(); + + // A body-less decision that also nulled an ordinary response would pass every assertion + // above, so the same route is read once more over GET. + const full = await transport.send( + Request.newBuilder().url(`${origin}/fixed-length`).build(), + ); + assert.equal(await full.text(), FIXED_LENGTH_BODY); + } finally { + await transport.close(); + } + }); + + it('exposes the response body as a stream that close() releases (TRANSPORT-25)', async () => { + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/echo`).build(), + ); + assert.ok(response.body instanceof ReadableStream); + await response.close(); + await response.close(); // idempotent (BODY-15) + } finally { + await transport.close(); + } + }); + + it('refuses an unhonourable defaultTimeoutMs at the factory (HTTP-35)', async () => { + // Runtime-divergent, and the reason the check exists at all: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2**32)` throw `RangeError` on Node and are accepted by Bun 1.3.14, + // so before audit #67 / #82 the same misconfigured transport failed every send here and + // silently used a different deadline there. The factory now answers identically on both, + // which is what this pins on the runtime that used to be the strict one. + for (const value of [0, -1, 1.5, 2 ** 32, Number.NaN]) { + assert.throws( + () => makeTransport(value), + error => { + assert.ok( + error instanceof TypeError, + `expected a TypeError for ${value}, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + assert.ok(error.message.includes(String(value)), error.message); + return true; + }, + ); + } + // And a legitimate default still builds something that sends. + const transport = makeTransport(30_000); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/echo`).build(), + ); + assert.equal(response.status.code, 200); + await response.close(); + } finally { + await transport.close(); + } + }); + + it('classifies a per-call timeout as retryable, not cancellation (TRANSPORT-4)', async () => { + const transport = makeTransport(); + try { + await assert.rejects( + transport.send( + Request.newBuilder().url(`${origin}/slow`).build(), + RequestOptions.newBuilder().timeoutMs(50).build(), + ), + error => { + assert.equal(error.name, 'TransportFailureError'); + return true; + }, + ); + } finally { + await transport.close(); + } + }); + + it('classifies an unsupported URL scheme as permanent, not retryable (TRANSPORT-20, RETRY-2)', async () => { + // Runtime-divergent in the strongest sense: the two runtimes do not merely word this + // differently, they use different error shapes. Node's undici-backed `fetch` rejects + // `ftp://` with `TypeError: fetch failed` carrying `Error: unknown scheme` as its cause -- + // byte-identical, at the top level, to a DNS or connect failure -- while Bun 1.3.14 rejects + // with `TypeError [ERR_INVALID_ARG_VALUE]: protocol must be http:, https: or s3:` and no + // cause at all. The Bun conformance row therefore proves nothing about this runtime, which + // is the runtime the SDK ships to. undici's dispatcher agrees with itself on both + // (`UND_ERR_INVALID_ARG`) and is here for the pairing (audit #67 / #82). + const transport = makeTransport(); + try { + await assert.rejects( + transport.send( + Request.newBuilder().url('ftp://example.com/anything').build(), + ), + error => { + // `classify.ts` is an allow-list over `IoError`, so the class IS the retry verdict: + // a `TransportFailureError` here would spend the caller's whole budget re-proving a + // URL no retry can fix. + assert.ok( + error instanceof TypeError, + `expected a TypeError, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + return true; + }, + ); + } finally { + await transport.close(); + } + }); + + it('classifies a dead port as a retryable transport failure (TRANSPORT-20)', async () => { + const transport = makeTransport(); + try { + await assert.rejects( + transport.send( + Request.newBuilder().url('http://127.0.0.1:1').build(), + ), + error => { + assert.equal(error.name, 'TransportFailureError'); + return true; + }, + ); + } finally { + await transport.close(); + } + }); + + it('maps a caller abort to a terminal cancellation (TRANSPORT-3)', async () => { + const transport = makeTransport(); + const controller = new AbortController(); + try { + const pending = transport.send( + Request.newBuilder().url(`${origin}/slow`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 20).unref(); + await assert.rejects(pending, error => { + assert.equal(error.name, 'CancellationError'); + return true; + }); + } finally { + await transport.close(); + } + }); + + it('does not close a delivered body when the signal fires afterwards (SEAM-16)', async () => { + // Both native clients tie the response body's lifetime to the signal they were handed, so this + // only holds because the transport dispatches over a fork it detaches at delivery. + const transport = makeTransport(); + const controller = new AbortController(); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/vendor`).build(), + undefined, + controller.signal, + ); + controller.abort(); + assert.equal(await response.text(), 'vendor status body'); + } finally { + await transport.close(); + } + }); + + it('keeps concurrent sends independent of one another (TRANSPORT-29, SEAM-12)', async () => { + const transport = makeTransport(); + try { + const responses = await Promise.all( + Array.from({length: 10}, (_unused, index) => + transport.send( + Request.newBuilder() + .url(`${origin}/echo`) + .headers( + Headers.newBuilder().set('X-Call', String(index)).build(), + ) + .build(), + ), + ), + ); + const seen = await Promise.all( + responses.map(async response => { + const echoed = JSON.parse(await response.text()); + return echoed.headers['x-call']; + }), + ); + assert.equal(new Set(seen).size, 10); + } finally { + await transport.close(); + } + }); + + // The two halves of TRANSPORT-28 are tested apart everywhere else: body-file drives `writeTo` + // against a local sink, and transport-undici narrows on a hand-built `{kind: 'file'}` literal. + // Only here do a real factory and a real transport meet -- which matters most for undici, whose + // file path bypasses `writeTo` entirely for its own `createReadStream`. + describe('a real fileBody() over the wire (TRANSPORT-28, BODY-11)', () => { + let dir; + let path; + const source = fixtureBytes(300 * 1024); + + before(async () => { + dir = await mkdtemp(join(tmpdir(), 'dexpace-filebody-')); + path = join(dir, 'payload.bin'); + await writeFile(path, source); + }); + + after(async () => { + await rm(dir, {recursive: true, force: true}); + }); + + it('sends the whole file byte-exactly', async () => { + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(fileBody(path)) + .build(), + ); + const echoed = JSON.parse(await response.text()); + assert.equal(echoed.body.length, source.byteLength); + assert.equal(sha(Buffer.from(echoed.body, 'utf8')), sha(source)); + } finally { + await transport.close(); + } + }); + + it('honors start and count', async () => { + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(fileBody(path, {start: 10, count: 20})) + .build(), + ); + const echoed = JSON.parse(await response.text()); + assert.equal( + sha(Buffer.from(echoed.body, 'utf8')), + sha(source.subarray(10, 30)), + ); + } finally { + await transport.close(); + } + }); + + // BODY-13's short-write clause on the STREAMED request-body path, which is this tree's to + // hold for two independent reasons. The shared conformance row + // (`run-suite.ts`'s "a file body truncated after its length was captured") drives the + // buffered path only: above the adapters' 1,000,000-byte materialize bound the producer + // failure aborts a `TransformStream` mid-pull, and Bun 1.3.14's `Readable.fromWeb` leaks + // that abort reason as unhandled rejections, which `bun:test` scores against an unrelated + // row. Node's bridge does not. And this is the only layer where a real `fileBody()` — whose + // `transferred === count` invariant is the thing under test — meets a real transport. + // + // Until audit #67 / #81 the undici transport handed `createReadStream(path, …)` to undici + // and never called `writeTo` at all, so this case reported 200 with ten bytes on the wire + // while transport-fetch raised. `content-length` is dropped outbound, so the framing cannot + // catch it either. + it('fails a send whose file was truncated after its length was captured (BODY-13)', async () => { + const declared = 1_100_000; + const shortDir = await mkdtemp( + join(tmpdir(), 'dexpace-filebody-short-'), + ); + const shortPath = join(shortDir, 'payload.bin'); + const transport = makeTransport(); + try { + await writeFile(shortPath, fixtureBytes(declared)); + const body = fileBody(shortPath); + assert.equal(body.contentLength, declared); + await truncate(shortPath, 10); + await assert.rejects( + transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(body) + .build(), + ), + error => { + assert.equal(error.name, 'TransportFailureError'); + // BODY-13 names transferred-of-total. The streamed path rethrows the producer's own + // message; the buffered one carries it as a cause, so both are searched. + const chain = []; + for (let at = error; at instanceof Error; at = at.cause) { + chain.push(at.message); + } + assert.ok( + chain.some(message => + message.includes(`transferred 10 of ${declared} bytes`), + ), + `no transferred-of-total in ${JSON.stringify(chain)}`, + ); + return true; + }, + ); + } finally { + await transport.close(); + await rm(shortDir, {recursive: true, force: true}); + } + }); + }); + }); + } +}); diff --git a/tests/tsconfig.json b/tests/tsconfig.json new file mode 100644 index 0000000..b51866d --- /dev/null +++ b/tests/tsconfig.json @@ -0,0 +1,31 @@ +{ + // Covers `tests/conformance/` ONLY. `tests/node-conformance/` is `.mjs`, and this project sets + // neither `allowJs` nor `checkJs`, so `tsc` never opens that subtree — deliberately. Those files + // run on `node --test` against the built `dist/`, and typing them would mean either checking JS + // against declarations they reach by raw `dist/` file path, or converting a suite whose whole + // point is to run on Node with no build step in front of it. + // + // The consequence worth stating: `types: ["bun"]` below does not apply to `node-conformance/`, + // and neither does the `strictTypeChecked` tier in `eslint.config.js`. That subtree gets the + // gts/format baseline and `globals.node` (`eslint.config.js`'s `.mjs` override) and nothing more. + // It is the tree that tests the shipped artifact and the tree with the fewest static checks over + // it; the compensating control is that CI runs it on two Node versions. + // + // See CLAUDE.md, "HARD RULE — the `tests/` partition". + "extends": "../tsconfig.base.json", + "compilerOptions": { + "rootDir": ".", + "target": "ES2023", + "lib": [ + "ES2023", + "DOM", + "DOM.AsyncIterable" + ], + "types": [ + "bun" + ] + }, + "include": [ + "**/*.ts" + ] +} diff --git a/tsconfig.base.json b/tsconfig.base.json index 0c1d0df..679be34 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -2,11 +2,14 @@ "extends": "./node_modules/gts/tsconfig-google.json", "compilerOptions": { "composite": true, + "target": "ES2023", "module": "nodenext", "moduleResolution": "nodenext", "lib": [ - "ES2022", - "DOM" + "ES2023", + "ESNext.Disposable", + "DOM", + "DOM.AsyncIterable" ], "noUncheckedIndexedAccess": true, "exactOptionalPropertyTypes": true,