Conversation
Summarizing a turn range larger than an OpenAI model's context window now fails with the API's rejection instead of succeeding with a summary that only covers part of the range. `jp conversation compact --turn=2..150 --summary` against a GPT model previously returned a normal completed response built from whatever OpenAI kept, and that summary was stored as standing for all 149 turns; every later request projected it in place of turns the model never read. The cause is `truncation: auto`, which JP sent on every OpenAI request. It lets the API drop input items from the middle of a conversation and answer anyway, so the response is indistinguishable from one that read everything. `summarize_events` rejects a response the *model* cut short, but a request the *provider* cut short arrives as an ordinary `Completed` and passed straight through. `ChatQuery` gains a `truncation` field carrying whether the provider may drop input. Summarization asks for `Forbidden` and gets a 400 it can report; the main query loop and tool inquiries keep `Allowed`, since nothing fits the stream to the window on the query path and a long conversation stays answerable only if the provider may drop what it cannot hold. Only OpenAI offers the choice — the other providers either always reject an oversized request or truncate server-side, out of JP's reach. Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summarizing a turn range larger than an OpenAI model's context window now fails with the API's rejection instead of succeeding with a summary that only covers part of the range.
jp conversation compact --turn=2..150 --summaryagainst a GPT model previously returned a normal completed response built from whatever OpenAI kept, and that summary was stored as standing for all 149 turns; every later request projected it in place of turns the model never read.The cause is
truncation: auto, which JP sent on every OpenAI request. It lets the API drop input items from the middle of a conversation and answer anyway, so the response is indistinguishable from one that read everything.summarize_eventsrejects a response the model cut short, but a request the provider cut short arrives as an ordinaryCompletedand passed straight through.ChatQuerygains atruncationfield carrying whether the provider may drop input. Summarization asks forForbiddenand gets a 400 it can report; the main query loop and tool inquiries keepAllowed, since nothing fits the stream to the window on the query path and a long conversation stays answerable only if the provider may drop what it cannot hold. Only OpenAI offers the choice — the other providers either always reject an oversized request or truncate server-side, out of JP's reach.