Skip to content

fix: deflake watchFile and invoke backspace tests - #2204

Merged
jariy17 merged 3 commits into
refactorfrom
fix/flaky-watchfile-and-invoke-tests
Sep 3, 2026
Merged

fix: deflake watchFile and invoke backspace tests#2204
jariy17 merged 3 commits into
refactorfrom
fix/flaky-watchfile-and-invoke-tests

Conversation

@tejaskash

@tejaskash tejaskash commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Closes #2140
Closes #2176

watchFile.test.ts (#2140)

Bun registers the macOS kqueue watch off the main thread. Edits written synchronously right after watch() are sometimes never reported (5 of 60 runs locally with no delay, 0 of 60 with a 20 ms settle). The test now settles 100 ms before writing and waits for the callback instead of sleeping a fixed 250 ms. 100 of 100 local runs pass; the previous version failed about 1 in 8 locally.

Test-only: project dev starts watching long before the user edits, so production is unaffected.

invoke.screen.test.tsx (#2176)

The chat frame renders session: <crypto.randomUUID()>. The backspace test asserted that "abcd" disappears from the whole frame, which can never happen when the random UUID contains that hex substring (about 1 in 2500 runs). Reproduced deterministically by seeding a session id containing abcd via the /:sessionId route. The test now types non-hex letters.

Verification

  • bun test full suite with --coverage passes locally
  • lint and typecheck clean

watchFile: Bun registers the macOS kqueue watch off-thread, so edits written
immediately after watch() were sometimes never reported. Settle before writing
and wait for the callback instead of a fixed sleep.

invoke backspace: the chat frame shows the random session UUID, so asserting
that "abcd" disappears could never pass when the UUID contained that hex
substring. Type non-hex letters instead.

Fixes #2140
Fixes #2176
@github-actions github-actions Bot added the size/xs PR size: XS label Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Sep 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Both flake fixes are well-reasoned and test-only:

  • src/io/watchFile.test.ts: The 100 ms settle before writing addresses Bun's off-thread kqueue registration on macOS, and swapping the fixed 250 ms sleep for waitFor(() => calls === 1) asserts on outcome rather than timing. Keeping the fixed 250 ms sleep after controller.abort() is correct — that assertion is that no additional callback fires, which a polling helper can't express.
  • src/handlers/harness/invoke/invoke.screen.test.tsx: Switching the typed characters from abcd (hex) to wxyz (non-hex) is the right fix — crypto.randomUUID() output can only contain [0-9a-f-], so collision with the session id in the rendered frame is now impossible rather than ~1/2500. The comment explains why clearly.

No production code is touched, so telemetry doesn't apply. Nothing else to flag.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 3, 2026
@codecov-commenter

codecov-commenter commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.08%. Comparing base (c796be3) to head (0be41d9).
⚠️ Report is 7 commits behind head on refactor.

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2204   +/-   ##
=========================================
  Coverage     97.08%   97.08%           
=========================================
  Files           544      544           
  Lines         37768    37768           
=========================================
  Hits          36666    36666           
  Misses         1102     1102           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added size/xs PR size: XS and removed size/xs PR size: XS labels Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@github-actions github-actions Bot added size/xs PR size: XS and removed size/xs PR size: XS labels Sep 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
Comment thread src/io/watchFile.test.ts Outdated
const controller = new AbortController();
let calls = 0;
watchFile(path, () => calls++, controller.signal);
// Bun registers the macOS kqueue watch off-thread, so earlier edits are never reported.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anyway to poll until this is registered somehow? I worry a fixed wait time could be flaky.

@jariy17
jariy17 merged commit 76bae61 into refactor Sep 3, 2026
20 of 21 checks passed
@jariy17
jariy17 deleted the fix/flaky-watchfile-and-invoke-tests branch September 3, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants