Repository navigation
test(agent): drop_block's calibration skip is one stream's, and an always-on model's display goes out only with it #770
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
nissa-seru
merged 5 commits into
anima-research:main
from
nissa-seru:test/thinking-binding-drop-block-follow
Oct 23, 2026
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
19c87e4
feat(agent): thinkingBinding.dropBlock asks the provider to drop thin…
e6a5922
fix(agent): no calibration sample from a stream once a call dropped t…
24c9272
test(agent): drop_block's skip is one stream's, and an always-on mode…
46a045e
docs(agent): dropBlock's fold case holds where the strategy still sen…
86300c9
docs(changelog): drop_block's fragment names the fold case
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| - `thinkingBinding.dropBlock` in an agent's config asks the provider to drop | ||
| a signed thinking block whose earlier conversation has changed, as a fold | ||
| changes it, rather than refuse the request. An account that enforces | ||
| thinking-signature binding answers such a request with a 400; with the | ||
| setting, it drops the blocks that no longer match and answers. Where an | ||
| account doesn't enforce it, the provider decides for itself what to do with | ||
| such a block, and the setting has it drop them too. The setting is off by | ||
| default. | ||
| - **What's sent:** `thinking.block_binding.prefix_mismatch_behavior: | ||
| "drop_block"` with the `thinking-binding-controls-2026-08-01` beta, which | ||
| membrane's Anthropic and Bedrock adapters carry. It has no effect through | ||
| other adapters, or where the host sends a thinking config of its own, | ||
| through `request.extra` or `providerParams`, which replaces membrane's. | ||
| Any other model carries it only while thinking is on. A model whose | ||
| thinking is always on (Fable, Mythos) gets an adaptive thinking config | ||
| to carry it, with the agent's `thinking.display` when that's set, so on | ||
| such a model `display` takes effect only with this setting. membrane | ||
| notes that the Mythos family has refused thinking configs with a 400 | ||
| before, so on a Mythos model the setting may fail every request. | ||
| - **What's reported:** on a framework turn, each call's dropped blocks, and | ||
| any shown to the model though they failed the check, appear in its | ||
| `inference:usage` trace as `thinking`. A call that dropped blocks, and | ||
| every later call of its stream, isn't used to calibrate the context | ||
| strategy's token estimates, since the provider doesn't bill a block it | ||
| dropped and the strategy still counts it. Where the strategy still sends | ||
| the blocks a fold left bound to a conversation the request no longer | ||
| holds, the provider drops them on every call until a later fold takes | ||
| them out of the context, and while they're sent, no stream calibrates and | ||
| the estimates keep the value they had. The direct API carries the setting | ||
| too, but reports nothing of it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Medium severity. After a fold with dropBlock on, calibration can stay frozen for a long time, not just for one stream.
thinkingDroppedInStreamholds back every sample of a stream once one of its calls reports dropped blocks. The new doc in src/types/agent.ts (the fold case) says that after a fold, every block after the rewritten stretch is dropped on every call until a later fold removes it. So every stream after a fold drops blocks on its first call and never gives a sample. Calibration then keeps its old value across many turns, possibly until the next fold. The new two-turn test says that keeping the skip on the agent 'would stop calibration for good', but with stream scope the practical effect after a fold is nearly the same. The context strategy's token estimates go stale exactly when folds are happening, and this is when estimate accuracy matters. A fix could correct the sample for the dropped blocks, or count only the blocks that were not dropped, instead of throwing the sample away.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fold case is now in
thinkingBinding.dropBlock's doc and, for the low finding, in the changelog fragment (86300c9b). Both say the calibration keeps its value while the strategy still sends the blocks a fold left bound.— Effie-2756