fix(chat): don't abort healthy streams during prefill (#6231) - #6232
Open
touzenesmy wants to merge 1 commit into
Open
fix(chat): don't abort healthy streams during prefill (#6231)#6232touzenesmy wants to merge 1 commit into
touzenesmy wants to merge 1 commit into
Conversation
The visibilitychange tab-recovery handler treated 'no bytes received' as a frozen connection. A slow local / multimodal model can spend minutes pre-filling (image + large agent prompt) before its first token, and the main prefill path sends no SSE heartbeat, so lastActivity stays frozen. Returning to the tab after 20s+ aborted a perfectly healthy stream with 'interrupted after the tab went inactive'. Track firstOutputAt on the stream entry (set on the first data event) and skip tab-recovery until the stream has produced output: - no first token yet -> normal prefill silence, don't abort - had output then froze -> recovery works exactly as before A genuinely dead prefill is still handled by the response timeout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a stream has not yet emitted its first token, the
visibilitychangetab-recovery handler treats "no bytes received since the tab was hidden" as evidence of a frozen connection — but a slow local / multimodal model can legitimately spend minutes pre-filling (image + large agent prompt) before its first chunk, and the main prefill path sends no SSE heartbeat at all, solastActivitystays frozen. Returning to the tab after 20s+ therefore aborted a perfectly healthy stream with "interrupted after the tab went inactive".Fix: track
firstOutputAton the stream entry (set on the first data event) and skip tab-recovery until the stream has produced output:A genuinely dead prefill is still caught by the response timeout, not tab-recovery.
Target branch
dev, notmain. All PRs land indev;mainis curated by the maintainer at each release. If your PR is onmainby accident, click "Edit" on this PR and change the base.Linked Issue
Fixes #6231 — reported there before filing this PR (no existing report existed: searched open and closed issues for the exact error string, 0 results). Repro steps in How to Test below mirror the issue's Steps to Reproduce.
Type of Change
Checklist
tab went inactive, andvisibilitychangePRs: perf(ui): pause background polling when tab is hidden #3821 pauses background polling, fix(ui): stop the whirlpool spinner animating when it is never attached #5989/fix(ui): stop the whirlpool spinner animating when it is never attached #5990 spinner — unrelated.)devdocker compose uporuvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough. (Verified on the running instance: image + large prompt, tab switched away 25s+ mid-prefill, stream survives; frozen-stream regression path unchanged.)How to Test
firstOutputAtguard only skips streams that never produced output.node --check static/js/chat.jspasses. (No unit test — this is frontend stream-state logic; the two behavioural paths above cover it.)Visual / UI changes — REQUIRED if you touched anything that renders
Anything that changes what the UI looks like — buttons, icons, padding, colors, fonts, spacing, layout, CSS, HTML, SVG, or any
static/js/module that draws to the DOM — needs all of the following. PRs that change rendering without these WILL be closed.static/jsclause a clip is attached below showing the stream surviving a tab-away during prefill.--red,--fg,--bg,--card,--border, etc.) — do not introduce new color values, font sizes, or spacing units.static/index.html) or plain text.Fira Code) for primary UI text. Don't override.Screenshots / clips
I had a lot of trouble to say below 10Mb
brave_PHihh1TWja.mp4
(clip of the tab-away-during-prefill repro attached before publishing)