Skip to content

fix(voice): honest, bounded voice-input lifecycle with interim text (5.4.2) - #104

Merged
linxule merged 5 commits into
mainfrom
fix/device-voice-feedback
Oct 10, 2026
Merged

linxule merged 5 commits into
mainfrom
fix/device-voice-feedback

Conversation

@linxule

@linxule linxule commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Why

Owner hand test (Chrome 155, on-device voice): English works; Chinese shows "Listening…", the owner speaks, nothing appears, no error. Root-cause investigation (Codex reading Chromium source, Kimi reading our code, lead verified):

  • Ruled out: our code dropping text (language is fixed at consent; no re-render stops a session); the language tag (Chromium maps zh-CN → the cmn-Hans-CN pack in the recognition path, components/soda/constants.cc + speech_recognition_recognizer_impl.cc ResetSoda(), checked at main/157, not the exact 155 tag).
  • Still open, inside Chrome: the Chinese model either decodes nothing or emits only interim (never final) text. Console tests could not tell, so this PR makes the app itself show which.

What changes

  • ready after a speech-pack install (mic still never opens on its own); starting until audiostart; stopping while finals arrive.
  • interimResults on: a live, local-only preview (not stored, not sent, not an aria-live region); finals appended once; unfinalized speech goes to the editable answer box on end / stop timeout / non-terminal error (never auto-sent).
  • Empty end, nomatch, no-speech → "No text was captured"; a foreign aborted → "stopped unexpectedly" (mic stays available).
  • Start-up deadline (8 s) runs only once navigator.permissions reports the microphone granted, so a slow "Allow" click is not an error; 10 s stop grace; no overlapping sessions; timers cleared on every terminal path.
  • zh/ja dictated text joined without spaces (also fixes appending successive finals).
  • Four new strings in all six languages.
  • processLocally fail-closed checks unchanged; no cloud fallback.
  • Release 5.4.2 (bundles undeployed 5.4.1).

Review

  • Kimi review: no blockers; 6 minor findings accepted and fixed (CJK joining, stopping label/aria-pressed, preview not aria-live, keep draft on terminal device error, zh ellipsis, dead handler).
  • Side-agent note: the start-up deadline would fire while Chrome's permission prompt is open → fixed (permission-aware deadline).

Verification

  • npm run check:cloudflare 20/20 on b41bc02 (Node 24.19).
  • Unit: 3,745 passed, including new lifecycle tests with explicitly driven recognition events.
  • Not automatable (CLAUDE.local/AGENTS): real-speech recognition in Chrome; owner hand test on staging after deploy.

🤖 Generated with Claude Code

linxule and others added 5 commits October 10, 2026 17:25
Owner hand test (Chrome 155, on-device): English works, Chinese shows
"Listening..." and nothing appears, with no error. The hook made every
failure silent: it claimed to listen before audio capture, ignored interim
results, went quietly idle on an empty end and had no deadline.

- ready after pack install (no auto-open); starting until audiostart;
  stopping while awaiting finals.
- interimResults on: an ephemeral per-index preview; finals appended once;
  unfinalized speech goes to the answer box on end, stop timeout or a
  non-terminal error (never sent automatically).
- empty end, nomatch or no-speech -> noText; a foreign aborted ->
  interrupted (mic stays available); self-aborts are suppressed.
- 8 s startup deadline, 10 s stop grace, no overlapping sessions, timers
  cleared on every terminal path.
- processLocally fail-closed checks unchanged; no cloud fallback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…x languages

New statuses (ready, starting, noText, interrupted) in all six dictionaries;
a muted live preview tagged with the recognition language; Send and
Cmd/Ctrl+Enter held while starting/listening/stopping and released on every
terminal path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Kimi review of the lifecycle change (no blockers):
- zh/ja dictated text is joined without spaces, both interim fragments and
  successive finals appended to the answer (the latter predates this branch).
- While stopping, the mic button keeps "Stop" and aria-pressed, disabled.
- The live preview is no longer an aria-live region (it re-announced the
  whole growing text on every result).
- A terminal device error keeps already-recognized interim text, like every
  other error path.
- zh "preparing" uses the CJK ellipsis like the other zh voice strings;
  unused onspeechstart removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Chrome captures no audio while its "Allow microphone" prompt is open, so a
participant who read it for more than 8 s got a false "could not be turned
into text" on first use (side-agent review). The deadline now waits for
navigator.permissions to report microphone 'granted' (immediately, or via
onchange); without the Permissions API there is no deadline and Stop still
releases the session.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@linxule
linxule merged commit 043e86a into main Oct 10, 2026
6 checks passed
@linxule
linxule deleted the fix/device-voice-feedback branch October 10, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant