[Bug]: the translated-path SSE inspector is unbounded and parses 3x — 64 MiB of delimiterless upstream reaches 13.5 GB RSS

[lidge-jun]

Client or integration

Claude Code

Area

Streaming

Summary

trackSseForRequestLog (src/server/relay.ts:354) — the SSE inspector used by every translated (non-passthrough) stream — still has the two retention defects that createSseInspector had before #314/#864: an uncapped incomplete-frame buffer and 3 JSON.parse calls per event. The passthrough inspector was bounded and folded into a single shared parse; this second inspector was never given the same treatment, so the fix is asymmetric.

This surfaced from an external Windows investigation of Bun memory growth (2026-08-01, OpenCodex 2.7.30 / Bun 1.3.14). Most of that report's proposal is already on dev — relaySseEagerBounded with an 8 MiB queue, single shared parse in createSseInspector, inspectResponseLogSsePayloadParsed, 15 s / 32 MiB post-cancel drain. The one thing it points at that we do not have is this second inspector.

Measured on dev bf063b57b, Bun 1.3.14, macOS arm64:

Path	64 MiB well-formed deltas	parses/event
createSseInspector (passthrough)	+27.9 MB peak RSS	1.00
trackSseForRequestLog (translated)	+24.2 MB peak RSS	3.00

The parse count is a steady 3× CPU/allocation tax on the translated path, but the buffer is the sharp edge. With a stream that never emits a blank-line SSE delimiter, buffer += decoder.decode(value, { stream: true }) (src/server/relay.ts:382) grows without any bound:

Input	Peak RSS delta
64 MiB, no delimiter, createSseInspector (4 MiB frame cap)	+4.6 MB
64 MiB, no delimiter, trackSseForRequestLog	+13,543.8 MB
16 MiB NDJSON mislabeled text/event-stream, trackSseForRequestLog	+441.6 MB

13.5 GB of RSS from 64 MiB of upstream bytes. The same input against the bounded inspector costs 4.6 MB — a ~2900× difference that is purely which inspector the request happened to route through.

This is not a Windows-only or eager-relay-only path. trackSseForRequestLog is reached through responseWithDeferredRequestLog on every platform and regardless of streamMode, for:

• /v1/chat/completions translated streams (src/server/chat-completions.ts:279, :284)
• Anthropic Messages inbound / Claude Code (src/server/claude-messages.ts:739)
• the src/server/index.ts:916 translated branch

An OpenAI-compatible provider that emits single-\n NDJSON under a text/event-stream content type — or any adapter whose upstream stalls mid-frame — is enough to trigger it. No malicious upstream is required.

Note on the reporter's framing: they attribute the Windows growth to Bun 1.3.14 fetch backpressure combined with tee(). The tee() half is real and already addressed on dev. Bun PR #32120 is merged upstream (2026-06-21) but still unreleased — 1.3.14 (2026-05-13) is the newest release — so MIN_FIXED_BUN_VERSION = null in src/lib/bun-stream-caps.ts:25 is still correct and auto correctly stays on tee. That part needs no change.

Two other pieces of their proposal I deliberately do not recommend adopting:

1. The post-first-output delta skip (a regex that skips JSON.parse for repeated standard deltas). Their own numbers show it saving 88.8 MB / 18.3%, but it trades correctness for allocation: a truncated delta that still matches the prefix loses its error record, which their own cross-validation flagged as an unresolved gap. Bounding the buffer is the larger win and carries no such trade.
2. Their focused numbers were reported alongside a full suite at 6384 pass / 9 fail / 2 errors with 4 unattributed failures, so their candidate as a whole is not a mergeable state.

Expected behavior: both SSE inspectors enforce the same frame bound and parse the payload once.

Reproduction

1. Check out dev at bf063b57b.
2. Save as .tmp/frame-probe.ts:

import { trackSseForRequestLog, createSseInspector, MAX_INSPECTION_SSE_FRAME_BYTES } from "../src/server/relay";
const enc = new TextEncoder();
const CHUNK = 1024 * 1024;
const junk = enc.encode("data: " + "x".repeat(CHUNK - 6) + "\n"); // single \n: never a blank-line delimiter
const TOTAL = 64;

{
  const insp = createSseInspector({ onTerminal: () => {}, logCtx: {} as any, onCompletedResponse: () => {}, onFirstOutput: () => {} });
  const before = process.memoryUsage().rss;
  for (let i = 0; i < TOTAL; i++) insp.feed(junk);
  console.log(`inspector cap=${MAX_INSPECTION_SSE_FRAME_BYTES / 1048576}MiB rss_delta=${((process.memoryUsage().rss - before) / 1048576).toFixed(1)}MB`);
  insp.dispose();
}
{
  Bun.gc(true);
  const before = process.memoryUsage().rss;
  let sent = 0;
  const src = new ReadableStream<Uint8Array>({ pull(c) { if (sent++ >= TOTAL) { c.close(); return; } c.enqueue(junk); } });
  const reader = trackSseForRequestLog(src, () => {}, () => {}, {} as any, () => {}).getReader();
  let peak = 0;
  for (;;) { const { done } = await reader.read(); peak = Math.max(peak, process.memoryUsage().rss); if (done) break; }
  console.log(`trackSseForRequestLog fed=${TOTAL}MiB peak_rss_delta=${((peak - before) / 1048576).toFixed(1)}MB`);
}

3. bun .tmp/frame-probe.ts

Observed:

inspector cap=4MiB  rss_delta=4.6MB
trackSseForRequestLog fed=64MiB  peak_rss_delta=13543.8MB

For the parse count, wrap JSON.parse with a counter and feed well-formed response.output_text.delta blocks to each inspector: createSseInspector reports 1.00 parses/event, trackSseForRequestLog reports 3.00 (inspectResponseLogSsePayload → src/server/request-log.ts:596, isFirstOutputSsePayload → relay.ts:286, terminalStatusFromSsePayload → relay.ts:276).

Suggested fix, both mechanical and local to relay.ts:

• Apply MAX_INSPECTION_SSE_FRAME_BYTES (already exported, relay.ts:19) to the buffer in trackSseForRequestLog, reusing the existing discard-and-resynchronize semantics rather than inventing a second policy. relaySseWithHeartbeat (relay.ts:510) has the same unbounded buffer at relay.ts:540 and should be covered in the same change.
• Parse once per payload and pass the result to inspectResponseLogSsePayloadParsed, firstOutputFromParsed, and terminalStatusFromParsed — all three already exist and are exported, so this is wiring, not new logic.
• Regression tests next to tests/sse-inspector-bounds.test.ts, driven red first: an over-cap delimiterless stream must stay bounded, and byte-for-byte client output must be unchanged.

Version

2.10.2 (dev bf063b5)

Operating system

macOS 15.5 (arm64); reporter's original observation was Windows 11

Provider and model

Any OpenAI-compatible provider on the translated path; reporter used OpenCodex 2.7.30 / Bun 1.3.14 on Windows

Logs or error output

$ bun .tmp/frame-probe.ts
inspector cap=4MiB  rss_delta=4.6MB
trackSseForRequestLog fed=64MiB  peak_rss_delta=13543.8MB

$ bun .tmp/realistic-probe.ts
passthrough inspector: events=8105 parses=8105 (1.00/event) peak_rss_delta=27.9MB ms=103
translated tracker:    events=8105 parses=24315 (3.00/event) peak_rss_delta=24.2MB ms=38

$ bun .tmp/ndjson-probe.ts
NDJSON-mislabeled 16MiB -> peak_rss_delta=441.6MB

Screenshots and supporting files

Source report (external, Korean): http://josungmiweb.dothome.co.kr/nextcocoCodex_Bun_memory_full_resolution_guide_20260801.html

Redacted configuration

{
  "streamMode": "auto"
}

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[github-actions]

_{Automated translation bookkeeping — detected language: unknown.}

[Ingwannu]

Confirmed on current dev at ef131787.

Both remaining string-based scanners still append decoded bytes before establishing any frame bound:

• trackSseForRequestLog retains an unterminated translated-path frame indefinitely and reparses each complete payload through the request-log, first-output, and terminal helpers.
• relaySseWithHeartbeat has the same unbounded partial-frame state, although it only needs terminal inspection.

The existing createSseInspector already defines the correct inspection-only behavior: cap one candidate frame, discard only the oversized frame, resynchronize at the next SSE delimiter, and leave downstream bytes untouched. I will keep the fix local to relay.ts, make the two wrappers use the same bounded scanning semantics, share one parsed payload across the translated-path hooks, and add focused regression coverage for over-cap delimiterless input, recovery on the next valid frame, parse count, and byte-for-byte output preservation. I will not change the Bun transport-selection policy or adopt the proposed regex fast path.

[Ingwannu]

The bounded fix is now in #1114.

It replaces both remaining string-retaining scanners with the existing byte-bounded createSseInspector, so an over-cap frame is dropped only from inspection, the parser resynchronizes at the next SSE delimiter, and downstream bytes remain exact. The translated request-log path now performs one JSON.parse per complete payload for all observers.

Verified locally:

• focused SSE/relay tests: 37 passed, 0 failed
• full test suite: passed
• TypeScript typecheck: passed
• privacy scan: passed

The issue should remain open until #1114 is reviewed and merged into dev.

[Wibias]

Closing as fixed on dev.

#1133 replaced the unbounded translated-SSE inspection path with the existing byte-bounded SSE inspector. Oversized/delimiterless frames can no longer grow the inspection buffer indefinitely, and downstream bytes remain unchanged.

The fix includes regression coverage for oversized frames, resynchronization, completion handling, and byte-for-byte relay preservation.