Skip to content

Native harness audit: accepted reliability and instruction fixes - #585

Merged
itsdestin merged 15 commits into
masterfrom
session/native-harness-audit-20260926
Sep 28, 2026
Merged

itsdestin merged 15 commits into
masterfrom
session/native-harness-audit-20260926

Conversation

@itsdestin

@itsdestin itsdestin commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Review only — do not merge yet

Destin requested commits, a branch push and a PR for a separate final review. This draft must not be merged or released automatically. Updated 2026-09-28 with the review follow-ups below.

Summary

Implements the accepted native harness audit scope:

  • Recover from permission callback failures without dangling tool calls or broken subsequent turns.
  • Drain queued input reliably and deliver ordinary busy messages inside the running turn (after the current batch of actions — see follow-ups; a Send now control was added at Destin's request).
  • Retract abandoned automatic retry output consistently across display, effective saved replay and accepted history.
  • Capture the full ancestor instruction-file chain; show accurate multi-file startup facts; inherit owner-relative project rules and correct matching/deduplication.
  • Defer the first governed Write/Edit until a fresh model request sees new guidance (guidance that cannot fit is shown shortened once — see follow-ups).
  • Align specialist Bash companion tools with their permission caps; settle failed background handoffs accurately.
  • Use generation-owned MCP connections, covering config/credential changes and overlapping teardown.
  • Bound DNS waits by caller cancellation and the existing total deadline; verify Read bytes before repeat notices.
  • Keep duplicate-worded native questions independent while preserving legacy Claude Code behavior.

Includes desired-behavior regressions and dev-only fixtures. Remaining defect-observation probes are not included in the app test commit; their historical copies are in the companion workspace record.

Approved exclusions

  • C2 / R17 is explicitly deferred and not fixed: stronger process-group shutdown needs a broader ownership/supervisor design. Existing cleanup is best-effort; this PR makes no stronger descendant-termination claim.
  • Q10 / F07 denied: no stale credential-bearing Claude Code MCP projection cleanup.
  • Q22 native-only: Claude Code duplicate-question wording remains limited by its legacy answer format. No new disabled-Submit warning ships.
  • Other unapproved audit findings are outside scope.

Verification and acceptance

  • Fresh bash scripts/verify.sh --full <app-worktree> before commit: all desktop gates passed — full tests, source/test types, knip, lint, design lint, invariants, screens and journeys.
  • Independent contract grading: 17 pass, R17 deferred/not passed. Destin submitted acceptance on 2026-09-28.
  • Per-task red/green tests and independent reviews are retained in the companion workspace PR.
  • Common credential/private-key pattern scan over staged text found no matches; this is not an exhaustive security certification.
  • Not verified: Android, marketplace Worker, native Windows/macOS execution, real-provider/paid-model behavior. Test typecheck retains the project's existing exclusions.

Integration status (updated 2026-09-28)

Current master has been merged in (merge commit 9b8d4bdaf); conflicts in native-session-host.ts, tools/bash.ts and shell-registry.test.ts were resolved (Bash keeps master's admin-password onCallExit bookkeeping plus this branch's failed-hand-off handling). GitHub reports the PR mergeable. bash scripts/verify.sh --full passes on the final commit (types, full tests, knip, lint, design lint, ast-grep, screens, journeys). Android not built here.

Follow-up changes from Destin's PR review (2026-09-28)

Decisions recorded in the workspace PR (native-harness.pr-review.questions.answers.json, native-harness.send-now*.review.answers.json).

  • Mid-task messages are read after the assistant's current batch of actions finishes, instead of cancelling unstarted calls; a message that joins the turn resets the "keep going?" step count (Q-1, Q-2).
  • Instruction files: the session folder's own file gets first claim on the budget; broader ancestors get the rest and are still named if squeezed out (Q-4). The context panel shows plain-words notes, folder labels, a named "Shortened" summary, and "CLAUDE.md here not used" when AGENTS.md wins in the same folder.
  • Small-model rule fit: rules that cannot fit are shown shortened once and the model re-plans; the silent refusal is removed (Q-5; reverses the earlier accepted "refuse when guidance cannot fit").
  • Rule path patterns: {a,b} alternatives and [abc] classes now match; a leading / or ./ anchors at the rule owner; unreadable patterns are logged.
  • MCP: an excluded server's real error is logged again with every credential value it was given redacted.
  • Read: an unchanged repeat is verified by a piecewise fingerprint before loading the file (longest main-process stall at 50 MB: ~140 ms → ~1 ms); changed files are read as before.
  • Send now on a waiting message (new native:queue-send-now on every surface; Android replies not-implemented like queue-remove): stops the current task like Stop and sends that message next. UI: accent send-style button with an up arrow that reveals "Interrupt and Send Now" on hover, trash icon (matches doc-comments), plain pencil via a new shared EditPencilButton (QuickChips reuses it, unchanged look).
  • Tests: fixed two load-sensitive tests (LocalModelsSection late read, markdown streaming sweeps given a measured budget); removed item-list-authority.test.ts, whose only exemption no longer exists. Line budgets raised for the Send now plumbing; design-lint ratchet lowered 542 → 537.

Companion workspace PR: itsdestin/youcoded-dev#221.

Review entry point: audit, decisions and evidence.

Implement the accepted native harness audit scope with regression coverage. Preserve one queued-send flow, restore applicable guidance, isolate connection generations, and keep native question answers independent.

Process-group escalation remains deferred; Claude Code credential projection and duplicate-question behavior remain unchanged.

Submitted via YouCoded Assistant
itsdestin and others added 14 commits September 28, 2026 05:31
Resolves conflicts with the admin-password feature: toolWiring keeps both
the captured instruction chain and the RunningCalls options; Bash keeps its
named close/error listeners (so a failed hand-off still settles) and now
also runs onCallExit from them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… logs

- Rule path patterns: {a,b} alternatives and [abc] character classes now
  match instead of silently never firing; a leading / or ./ anchors at the
  rule's folder; unreadable patterns are logged.
- Instructions panel: plain-words notes instead of the model-facing notice,
  folder names instead of long paths, a named "Shortened" summary, and a
  note when a same-folder CLAUDE.md was not used because AGENTS.md was.
- MCP: an excluded server's real error is logged again, with every
  credential value it was given blanked out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… first

Per Destin's PR-review decisions (2026-09-28):
- A message sent while the assistant works is read once the whole batch of
  actions it already chose has run, instead of cancelling unstarted ones.
- A message that joins the running turn starts the "keep going?" step
  count over.
- When instruction files don't all fit, the session folder's own file takes
  the room it needs first; broader parent files get what is left and are
  still named if squeezed out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A repeat Read of an unchanged file loaded, decoded and line-split the whole
file on the main process before comparing it (~140 ms stall at 50 MB,
affecting every window). It is now checked in 256 KB pieces before loading
anything; measured longest stall at 50 MB drops to ~1 ms. Changed files are
still read normally and the content comparison stays exact.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…wn check

Under load the 50 ms test poll could land before the check and fail a
correct dialog; the read is now released explicitly with reveal().

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…inue

Per Destin's PR-review decision (Q-5, 2026-09-28): when project rules that
apply to a file about to change cannot fit a small model, the model is shown
the shortened rules (each naming the file to read for the rest) and asked
again, exactly as when they fit. The earlier refusal ended the turn silently
and left small models unable to edit files in rule-heavy projects. Rules are
shown once, so the re-issued change runs; there is no loop.

This reverses the accepted requirement "refuse changes when guidance cannot
fit" at Destin's direction.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Per Destin (2026-09-28): a message waiting behind a busy task gets a Send now
button that stops the current task, like Stop, and sends that message next,
ahead of other waiting messages. New native:queue-send-now request on every
surface (desktop IPC, remote WebSocket, remote shim; Android replies
not-implemented like queue-remove). The practice app can now queue messages
(?queueSends=1) so the strip can be reviewed.

Line budgets raised by the plumbing each surface needs: native-session-host
+15, App +11, remote-server +6, ipc-handlers +3, types +3, preload +2,
remote-shim +1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…sweeps

They take 1.5-7 s alone and passed 30 s under verify.sh's all-at-once full
run, timing out while correct. They count work, not clock time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ds on hover

Per Destin's review (2026-09-28, send-now deck S-1): a round dark button with
an up arrow, rightmost in the waiting-message row; on hover or keyboard focus
"Interrupt and Send Now" rolls out to the left of the arrow inside the button,
using the session pills' sanctioned max-width reveal and motion tokens. The ✕
becomes the doc-comments trash icon and sits to its left, sliding with it.

The item-list glyph test guarded only the strip's one ✕ exemption; with the ✕
gone it has nothing left to check, so it is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Per Destin's round-2 review (2026-09-28): the primary accent <Button> with
the composer's own arrow glyph instead of a hand-styled dark button, so its
colour, arrow, size and alignment match the app's other send buttons in
every theme. The hover label reveal is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Per Destin's round-3 review (2026-09-28):
- Send now's arrow points up.
- The waiting message's Edit reuses the quick chips' pencil button, now a
  shared EditPencilButton so the two cannot drift (QuickChips unchanged in
  look; its float-chrome hook class is passed in).
- The rolled-out label is trimmed to its letters (text-box) so it centres in
  every theme font; Meadow Mist's sat ~1px high. It clips sideways only so
  descenders survive the trim.

Design-lint ratchet lowered 542 -> 537: the strip's hand-restyled pencil
Button is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Per Destin's round-4 review (2026-09-28): EditPencilButton gains a plain
option (same pencil and size, no box) used by the waiting-message strip; the
quick chips keep their boxed button.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Per Destin's round-5 note ("bigger"): the plain pencil is 14px like the trash
beside it; the quick chips' boxed pencil is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…iled Send now

The MCP "excluded server" log line now also redacts credentials written into
a server's launch arguments (--api-key x, --token=x) or its address (user:pass@,
query values), not just its env/header values.

A Send now request that fails (e.g. the remote connection drops) now shows a
general "didn't go through — try again" toast instead of doing nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@itsdestin
itsdestin marked this pull request as ready for review September 28, 2026 23:26
@itsdestin
itsdestin merged commit a09f804 into master Sep 28, 2026
4 checks passed
@itsdestin
itsdestin deleted the session/native-harness-audit-20260926 branch September 28, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant