Skip to content

Retry engine and error classification: IoError rule, non-finite delayOverride, backoff NaN (#78) - #93

Merged
Wahbeh-Mohammad merged 6 commits into
audit/remediation-67from
audit/67/78-retry-classify-backoff
Sep 5, 2026
Merged

Retry engine and error classification: IoError rule, non-finite delayOverride, backoff NaN (#78)#93
Wahbeh-Mohammad merged 6 commits into
audit/remediation-67from
audit/67/78-retry-classify-backoff

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Closes #78. Part of the audit remediation umbrella #67, milestone 4, wave 5. Base audit/remediation-67.

Implements decision D16 (docs/audit-67-decisions.md) in full: three findings, two behaviour fixes and one
rule written down in the three places that carry it.

What changed

1. The classifier keeps instanceof IoError, and item 17 now says what that means

D16 option (b). retry/classify.ts's cause-walk matches two of the six classes in io/errors.tsIoError
and TransportFailureError — and not the four leaves that isIoError groups. That was true before this PR and
asserted nowhere; #68 recorded the gap and left the decision here.

The branch means "the wire failed". TRANSPORT-20 makes TransportFailureError an IoError for exactly
that reason, and RETRY-2's "an I/O error" is read as that boundary. The four flat leaves are this package's
own contract and lifecycle failures and are deterministic on re-send: a closed resource (IO-42) and a source
returning zero bytes for a positive read (IO-17) are caller programming errors, an allocation cap (IO-9) is
a limit the same request hits again, and EndOfStreamError is the exact-length-copy contract inside io/ — a
wire truncation is the transport's to surface, as a TransportFailureError.

No code change to classify.ts beyond the TSDoc that states the rule and a one-line comment at the branch.

2. fix: a non-finite delayOverride falls back like a throwing one (retry/engine.ts)

RETRY-40 makes a bad override non-fatal, and callerOverride implemented that for a throw only. A non-finite
return was the more damaging case: NaN and the infinities pass every guard downstream — overshootsBudget
and budgetExhausted compare false, waitFor's delayMs <= 0 short-circuit compares false — and reach
Clock.sleep, which rejects a non-finite duration with a RangeError. RETRY-33's catch-all folds that into
the terminal failure, so delayOverride: () => NaN with maxAttempts: 3 gave one send and surfaced
RangeError: Clock.sleep: durationMs must be a non-negative finite number, got NaN, with the real
TransportFailureError demoted to the attempt trail. Reproduced before the fix, exactly as the issue describes.

Now: screen the result for finiteness at the source, report http.retry.delayOverrideFailed at warning level
through the same emit path a throw uses, return undefined, and let RETRY-39's precedence fall through to the
computed schedule. Finiteness alone is the screen — a finite negative keeps continuing inline (RETRY-31), and
a fractional or very large delay is one RETRY-39 gives the caller precedence for. Both pinned.

3. fix: a zero initial delay never yields a NaN backoff (retry/backoff.ts)

computeDelay relied on Math.min to absorb an overflowing power into the cap. That holds for a positive base
and not for a zero one: 0 * Infinity is NaN, and Math.min propagates it. retrySettings() accepts
initialDelayMs: 0 with any finite multiplier >= 1, so {initialDelayMs: 0, multiplier: 1e200} gave NaN at
attempt 3 and {initialDelayMs: 0, multiplier: 2} at attempt 1100 — against RETRY-11. Short-circuit the zero
base before the power is taken; exact rather than a repair, since the schedule is 0 at every attempt there and
jitter around 0 is 0 for any sample.

Test rows added

packages/core/src/retry/classify.test.ts — new describe the I/O boundary the cause-walk tests (RETRY-2/RETRY-4, TRANSPORT-20):

Case Pins
IoError itself is retryable the branch's positive side
TransportFailureError is retryable (TRANSPORT-20) instanceof IoError holds, so the requirement routes with no retry-layer edit
EndOfStreamError is NOT retryable, buried in a cause chain either the leaf, and the walk one hop up
SourceContractViolationError is NOT retryable leaf
ClosedResourceError is NOT retryable leaf
AllocationLimitError is NOT retryable leaf

Each leaf case asserts both that isIoError accepts the value and what the classifier answers, so the
disagreement reads as a decision. Counterfactual measured: switching the branch to isIoError(current) turns
exactly those four red.

packages/core/src/retry/backoff.test.ts — new describe a zero initial delay (RETRY-11): stays zero where the power overflows (both overflow attempts), stays zero under jitter too (RETRY-10) (both sample extremes),
and property: every accepted schedule is finite and non-negative (RETRY-11) over the ranges retrySettings()
admits. All three red before the fix.

packages/core/src/retry/engine.test.ts — new describe a non-finite delayOverride is the throwing case (RETRY-40): every non-finite value falls back to the computed schedule (asserts the durations the clock was
asked for, [200, 400], under NaN/+Infinity/-Infinity) and it never reaches Clock.sleep as a duration, so the real failure survives (the reported symptom, against a clock that guards its input the way
defaultClock does: three sends, the final TransportFailureError surfaced, two entries in the trail). Plus
a finite override is honored unchanged, fractional and huge alike (RETRY-39) and
emits delayOverrideFailed when delayOverride returns a non-finite delay. Three of the four red before the fix.

A fifth commit keeps the new cases inside the lint caps (max-lines-per-function counts a describe callback;
max-params counts a fast-check property's arrow). recordingClock, guardingClock, oneFailureThenSuccess
and captureLogEvents are hoisted to module scope in engine.test.ts; the logger harness was copied verbatim
in two existing tests and would have been a third. No assertion weakened.

No tests/node-conformance/ case: nothing here touches a runtime-divergent surface. The engine's contact with
the runtime is Clock.sleep, whose non-finite guard is this package's own code and is already covered by
config-primitives.test.mjs; after this change no non-finite value reaches it.

Deviation rows added

None — and that is D16's instruction. This is a section edit to item 17 of docs/deviations.md, not a
row in the "Deviations recorded outside a phase" table. The row already existed; #68 wrote it and left the rule
undecided. What changed is item 17's rationale paragraph (the branch is a boundary, not a category), a second
paragraph on why the four leaves stay outside it, and the anchor-correction block, which now records both dates
and both anchors. packages/core/src/io/index.ts's comment says the same thing at the source. Re-anchored:
classify.ts:73:90; packages/core/src/index.ts:39-47:39-48.

node .claude/skills/housekeeping/probe.mjs --only=citations: no drift found.

Gate

node .claude/skills/ci-preflight/run-ci.mjs --clean (no --node-floor; other agents were running
concurrently), pinned Bun 1.3.14 from .bun-version:

clean: removed 29 build artifact(s) — starting from CI's state
bun: pinning every step to 1.3.14 from .bun-version (PATH has 1.4.0)
CI preflight — 20 step(s) from .github/workflows/ci.yml, in /home/mohammad/Projects/dexpace/wt-78
...
CI preflight: all 20 steps passed.

bun run api clean against all nine committed reports: no @public TSDoc changed, so no api:local was
needed. Every symbol touched (isRetryableFailure, computeDelay, callerOverride, RetryConfig) is
@internal.

Open question for the maintainer

RetryStepOptions.delayOverride (packages/core/src/retry/retry-step.ts:41-49) is the public entry point
for this behaviour, and its TSDoc documents only the undefined fall-through. It is not in this task's
partition, so per contract item 4 it was not edited. Recommendation: add one sentence — "A throw, or a
non-finite result, is ignored: the configured schedule is used for that attempt and
http.retry.delayOverrideFailed is logged (RETRY-40)." That changes a @public TSDoc, so it needs api:local
on core and a regenerated core.api.md.

Deferred — release machinery

Suspended for this run under D1; recover at the release pass.

  • Patch changeset for @dexpace/core, covering two caller-visible behaviour changes:
    • a delayOverride returning NaN, +Infinity or -Infinity is now ignored with a warning log
      (http.retry.delayOverrideFailed) and the computed schedule is used, instead of the retry loop terminating
      after one send with a RangeError from Clock.sleep;
    • computeDelay returns 0 rather than NaN for initialDelayMs: 0 at the attempt where the power
      overflows, so a retry configured that way no longer fails on its first backoff.
  • No docs/first-release.md edit, no version bump.

`classify.ts`'s cause-walk tests `current instanceof IoError`. After the 2026-09-04 tier
flattening only two of `io/errors.ts`'s six classes satisfy that: `IoError` and
`TransportFailureError`. `EndOfStreamError`, `SourceContractViolationError`,
`ClosedResourceError` and `AllocationLimitError` extend `DexpaceError` directly and are grouped
only by `isIoError`, so the walk does not match them -- undecided behaviour, asserted nowhere.

Audit #67 / #78 (decision D16) keeps `instanceof IoError` and reads it as the boundary it already
draws: the branch means "the wire failed". `TRANSPORT-20` makes `TransportFailureError` an
`IoError` for exactly that reason; the four flat leaves are this package's own contract and
lifecycle failures and are deterministic on re-send.

Six cases, one per class, each asserting both what `isIoError` says and what the classifier says,
so the disagreement reads as a decision rather than an oversight. Counterfactual measured:
switching the branch to `isIoError(current)` turns exactly the four leaf cases red.

Refs #78, #67
`computeDelay` took `initialDelayMs * multiplier ** (attempt - 1)` and relied on `Math.min` to
absorb an overflowing power into the cap. That holds for a positive base and not for a zero one:
`0 * Infinity` is `NaN`, and `Math.min` propagates `NaN` instead of clamping it. `retrySettings()`
accepts `initialDelayMs: 0` with any finite `multiplier >= 1`, so `{initialDelayMs: 0,
multiplier: 1e200}` produced `NaN` at attempt 3 and `{initialDelayMs: 0, multiplier: 2}` at
attempt 1100 -- against RETRY-11's "overflow-safe, saturating rather than throwing".

`NaN` is worse downstream than a large delay, because it fails every comparison: the budget check,
the overshoot check and the engine's `delayMs <= 0` short-circuit all read false and it reaches
`Clock.sleep`, which rejects with a `RangeError` that replaces the failure being retried.

Short-circuit the zero base before the power is taken. Exact, not a repair: the schedule is `0` at
every attempt there, and jitter around `0` is `0` for any sample. Three tests -- the two overflow
attempts, the jittered case at both sample extremes, and a property over the ranges
`retrySettings()` admits (its `initialDelayMs` arm draws an explicit `0`, because the failing
region needs a zero base and an overflowing power together and 100 unbiased runs never paired
them).

Refs #78, #67
RETRY-40 makes a bad caller delay-override non-fatal, and `callerOverride` implemented that for a
throw only. A non-finite RETURN was the more damaging case: `NaN` and the infinities pass every
guard downstream -- `overshootsBudget` and `budgetExhausted` compare false, `waitFor`'s
`delayMs <= 0` short-circuit compares false -- and reach `Clock.sleep`, which rejects a non-finite
duration with a `RangeError`. RETRY-33's catch-all folds that rejection into the terminal failure,
so `delayOverride: () => NaN` with `maxAttempts: 3` gave ONE send and surfaced
`RangeError: Clock.sleep: durationMs must be a non-negative finite number, got NaN`, with the real
`TransportFailureError` demoted to the attempt trail.

Screen the result for finiteness at the source and treat a non-finite one exactly as a throw:
report `http.retry.delayOverrideFailed` at warning level through the same emit path, return
`undefined`, let RETRY-39's precedence fall through to the computed schedule. The check sits
outside the `try` so a logger that throws while reporting it is not re-reported as an override
that threw. Finiteness alone is the screen -- a finite negative keeps continuing inline (RETRY-31),
and a fractional or very large delay is one RETRY-39 gives the caller precedence for; both pinned.

Four engine cases: the schedule the clock is actually asked to sleep for under all three non-finite
values, the reported symptom against a clock that guards its input the way `defaultClock` does,
the finite passthrough, and the log event with the cause naming the rejected value.

Refs #78, #67
Audit #67 / #68 re-anchored `deviations.md` item 17 and recorded that its rationale said something
false — "the cause-walk returns retryable for any `IoError`" reads as covering all five classes
`isIoError` names, and the branch matches two. It left the decision to #78.

#78 keeps `instanceof IoError` and writes the rule down in the three places that carry it:

- `deviations.md` item 17: the rationale now says the branch is a boundary, not a category, and a
  second paragraph says what the four flat leaves are and why re-sending them is pointless.
  `classify.ts:73` re-anchored to `:90`, and the anchor-correction block records both dates.
- `packages/core/src/io/index.ts`: "load-bearing on it" is qualified with what it actually matches,
  and with the instruction not to re-parent a leaf under `IoError` to tidy the tree.
  `packages/core/src/index.ts:39-47` re-anchored to `:39-48`.

No behaviour change. `probe.mjs --only=citations` clean.

Refs #78, #67
`max-lines-per-function` and `max-params` count a `describe` callback and a fast-check property's
arrow like any other function, and the previous three commits pushed three of them over.

- `backoff.test.ts`: the two zero-base cases and the totality property move to their own
  `a zero initial delay (RETRY-11)` describe; the property's four arbitraries become one
  `fc.record`, which is also closer to what `retrySettings()` actually validates.
- `engine.test.ts`: the two non-finite-override cases move to their own describe; `recordingClock`,
  `guardingClock`, `oneFailureThenSuccess` and `captureLogEvents` are hoisted to module scope. The
  logger harness was copied verbatim in two existing tests and would have been a third -- extracting
  it shortens all three, and the throwing-override case gains the `cause` assertion its non-finite
  sibling has.

No assertion weakened; `bun test packages/core/src/retry` is 173 pass, 0 fail.

Refs #78, #67
`delayOverride` is the public entry point for RETRY-40's non-fatal-override behaviour, and its
TSDoc documented only the `undefined` fall-through. A caller had no way to learn from the shipped
`.d.ts` that a throw is ignored, and after this branch's engine fix, that a non-finite result is
ignored the same way.

Backticked prose, no `{@link}` -- `http.retry.delayOverrideFailed` is an event name, not a symbol,
and a link out of a `@public` block is what `api:ci` fails on where `api:local` warns.

`core.api.md` regenerated: byte-identical. The report records the declaration and its release tag,
not the prose body, and no signature moved. The sentence does reach the shipped artifact --
verified in `packages/core/dist/retry/retry-step.d.ts`.

Round 2 of #78; the maintainer added this file to the task partition.

Refs #78, #67
@Wahbeh-Mohammad
Wahbeh-Mohammad merged commit 6c7fb65 into audit/remediation-67 Sep 5, 2026
3 checks passed
@Wahbeh-Mohammad
Wahbeh-Mohammad deleted the audit/67/78-retry-classify-backoff branch September 7, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant