Report an empty model response as itself, not as invalid JSON - #272
Open
mostafaseyedan wants to merge 1 commit into
Open
mostafaseyedan wants to merge 1 commit into
mostafaseyedan wants to merge 1 commit into
Conversation
When the language model returns an empty stream, parseLlmJson throws
"No JSON structure found in LLM response". That message matches the
/JSON|parse/i test used to classify retry failures, so the attempt is
counted as a parse failure and the user is told:
LLM returned invalid JSON after 3 attempts. Please try again.
Nothing was wrong with the model's JSON — there was no response at all.
The suggested remedy ("try again") is also wrong, since the usual cause
is an exhausted request quota or a model declining the request, neither
of which a retry fixes. Observed with three attempts failing in 11ms
with textLen=0, which no amount of retrying would have resolved.
Detect the empty stream directly and report it with the model id, so
the message names the real cause. An empty response is tracked
separately from parse failures rather than being folded into them.
Also log the available model catalog when no preferred family matches,
so the debug log records which models the selected fallback came from.
github-actions
Bot
requested review from
TamasBoncz,
Aymen (aymenfurter) and
Sanjay Singh (san360)
September 21, 2026 02:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the language model returns an empty stream,
callLlmJsonreports a JSON parsing problem that did not happen.parseLlmJson('')throwsNo JSON structure found in LLM response. That message matches the/JSON|parse/itest used to classify retry failures, so the attempt incrementsparseFailures, and the user sees:Nothing was wrong with the model's JSON — there was no response at all. The suggested remedy is wrong too: the usual causes are an exhausted request quota or a model declining the request, and neither is fixed by retrying.
This is actively misleading in practice. From a real
runtime.log, with the Copilotautomodel:All three attempts failed in 11 ms total with
textLen=0. The surfaced error sends the reader looking for a prompt or schema problem, when the model never produced a token. The same failure hitcontext_file_reviewa minute later, so it affects every AI feature, not one page.Fix
Detect the empty stream directly and report it with the model id:
Empty responses are tracked separately from parse failures rather than folded into them, so a genuine malformed-JSON reply still reports as before.
Also adds a
runtimeDebugline recording the available model catalog when no preferred family matches, so the log shows which models the fallback selection had to choose from — themodel=autoabove gave no indication of what else was on offer.Testing
npm run typecheckandnpm run lintclean (0 errors); 107 webview tests pass.