Skip to content

fix(chat): harden the wait handover against model slips - #75

Merged
albanm merged 6 commits into
mainfrom
chore-agents-sim-wait
Oct 1, 2026
Merged

albanm merged 6 commits into
mainfrom
chore-agents-sim-wait

Conversation

@albanm

@albanm albanm commented Oct 1, 2026

Copy link
Copy Markdown
Member

Hardens the wait_for_user_action handover against slips seen in judged simulations, and makes the simulation harness more faithful.

  • Chat: wait arguments a model serialized into the message (a Haiku run sent the JSON tail as text) are repaired for display and for the wait itself.
  • Chat: a wait declared without a message is refused instead of waited on, so the model retries; the schema required it but the SDK does not enforce it.
  • Chat: after the person interrupts a wait, the reminder to declare it again now closes the new turn's hidden context instead of sitting in the interrupted wait's tool result, ahead of the person's message in history.
  • lib-sim (0.9.0): a look after a click or typing reads until the page settles (max 3 s), so a persona no longer reports a link "did nothing" before the route changed.
  • Runner: a persona message composed before the reply its own action caused is dropped and the persona asked again.

Why: judged runs showed silent handovers, a garbled handover message, and assistants answering the person's question then never waiting again, so what they handed over went unseen.

Heads-up:

  • A wait without message is now refused; hosts and prompts should rely on the message.
  • With the reminder, Sonnet re-declared the wait after 5/5 interruptions, Haiku after 1/5: for small models it is not enough on its own.
  • The mock model now sends a message with its waits.

albanm and others added 6 commits September 30, 2026 20:30
A Haiku run sent wait_for_user_action a message ending in its own other
arguments as escaped JSON, with literal \n sequences, and the person read
the JSON tail as the handover. The message is cut where the swallowed
arguments begin, they are recovered for the wait itself, and literal \n
become line breaks, both where the message is shown and where it runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A persona that clicked a chat link and looked straight away read the page
before the route changed, told the assistant the link did nothing, and the
assistant apologised for a link the application had reported as followed.
A look following a click or typing now reads until two consecutive
snapshots agree, within 3 seconds; other looks read once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…y its action caused

Acting on a declared wait and writing happen in one persona pass, so the
message was composed before the assistant reacted. Judged runs had the
persona ask what had just been created under the reply that said so, and
the stale message interrupted the next wait. When the action changed the
conversation, the message is dropped and the persona asked again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ssage starts

The instruction to declare wait_for_user_action again after an interrupted
wait lived in that wait's tool result, which sits in history ahead of the
person's new message. Judged runs, Sonnet and Haiku alike, answered the
question and never waited again, so the action they had handed over went
unseen. The reminder now closes the new turn's hidden context.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The schema requires the message but the SDK does not enforce it: a Haiku
run called wait_for_user_action with no arguments, the turn went silent,
and the person, told nothing, had to ask what to do. The wait is refused
with an error the model retries from. Tests and the mock model now send
a message, as real calls do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the fix label Oct 1, 2026
@albanm
albanm merged commit a96415c into main Oct 1, 2026
4 checks passed
@albanm
albanm deleted the chore-agents-sim-wait branch October 1, 2026 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant