Repository navigation
test(workspace): seven tests wait for what they check, not for a length of time - #645
Conversation
…th of time All in test/workspace-three-way.test.ts, with one line in test/helpers/watcher-flow.ts: - The editor-save test waited a fixed 400 ms before checking that no wake saw a deletion or the backup. It now writes a probe once the save is taken in, and waits for the mount's turn after the probe's batch. The watcher passes over the backup by its name, and chokidar reports the save as one change to gate.json, which the test's until has seen handled. The probe and the turn cover any later batch or pass before the probe's. - Two tests bounded a later batch's return by the wall clock, under 40 ms and under 150 ms, which a descheduled run can exceed while the batch isn't held. Each now raises the deadline to 20 s before that batch, with the earlier scan held, and waits for the batch with until's 5 s. The first of them holds the mount with a gate the test opens, in place of a 400 ms sleep, and checks that its last batch returned once its own scan had finished within its deadline. - Two tests waited at most 1 s, in 10 ms polls, for a second scan to start. They wait with until. - The two catch-up ready-order tests record the mount's watcherReadyAt as each catch-up pass starts, and assert that the watcher's ready had come. In the re-attach test that means a stamp other than the first ready's. Before, the order was inferred from one-second delays on chokidar's reads, and a pass started beside the scan still passed. - The stop test held its in-flight work for 100 ms and queued its later work after a 10 ms sleep. The in-flight work is now held until the later work is queued, and the sleep is gone: with no watcher to stop, stop takes the mount's tail before it first waits. - until, and changesFlow in watcher-flow.ts, check their condition once more at their bound, so a process held past the bound fails only a wait whose condition still doesn't hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Automated review
Recommendation: merge and fast-follow. 1 medium severity finding can be fixed after the merge.
Reviewed commit dd9bc88 with claude-opus-5-5, by reading the code. It did not run the code or look at the checks.
Does this change belong in the project?
It fits the project. This change only touches tests. Seven existing workspace tests now wait for the event they check instead of a fixed amount of time, and two test wait helpers check their condition once more before failing.
4 reasons with sources
It hardens tests for behavior the project already has, so it fits.
- It makes existing tests more reliable and changes no product code. The diff only touches test/workspace-three-way.test.ts and test/helpers/watcher-flow.ts. The contributing guide says review checks whether a test can actually fail. These changes target exactly that: tests that could pass without checking what they're named for, or fail a correct run on a slow host. (CONTRIBUTING.md, What a PR should contain)
- It reuses the test file's own helpers and patterns instead of adding new infrastructure. The changes rely on things the file already has: its
untilandwatchersFlowhelpers, waiting on the mount's turn (mountTurns), its gate-promise pattern, and setting the module's own config. The newreadyAtPassessits next to the existingcountPasses. No dependency or public interface is added. (test/workspace-three-way.test.ts, test/helpers/watcher-flow.ts) - No changelog fragment is needed, since nothing users see changes. The guide asks for fragments only for behavior-affecting changes. This change is test-only. (CONTRIBUTING.md, Changelog)
- The change is coherent: one goal, carried through seven tests and the two helpers they share. Every edit replaces a time-based wait with a wait on the condition being checked, which fits the guide's request to keep commits coherent. (CONTRIBUTING.md, How changes land)
Findings
1 finding of medium or high severity, see the inline comment:
- Medium severity, test coverage. Stop test only checks the re-drain loop while stop() has no await before it reads the tails.
2 more findings, less severe
- Low severity, maintainability,
test/workspace-three-way.test.ts:1448. A second watchersFlow call stacks another wrapper on handleFsChanges and rewrites the same probe names.watchersFlow(line 745) replacesm.handleFsChangeswith a wrapper around whatever is there, and starts its probe count again atprobe-0.txt. Calling it again at line 1448, after line 1434, leaves the first wrapper still pushing into aheardarray nothing reads. It also rewrites the probe files the first call already left in the mount root, so the second handshake depends on the watcher reporting a rewrite of an existing file rather than a new file. It works today, but a helper that unwraps on return or takes a fresh probe prefix would make repeated use safe. - Low severity, maintainability,
test/workspace-three-way.test.ts:1493-1502. readyAtPasses copies countPasses's wrapping of initialScan instead of extending it.readyAtPassesrepeats the bind-and-replace of(m as any).initialScanthatcountPasses(lines 1487-1492) already does, and only adds what it records. Both replace the same private method. If a test installs both, the one installed later wraps the earlier one. IfinitialScanis renamed or its signature changes, both copies have to be updated. A single helper that records the pass count andwatcherReadyAtfor each pass would keep the hook in one place.
| // With no watcher to stop, stop takes the mount's tail before it first waits. | ||
| const stopping = m.stop(); | ||
| await sleep(10); | ||
| let finished = false; | ||
| // Appended after stop began, as a watcher callback or a late scan would be. | ||
| void (m as any).withMount(mount, async () => { | ||
| await sleep(100); | ||
| finished = true; | ||
| }); | ||
| release(); |
There was a problem hiding this comment.
Medium severity, test coverage. Stop test only checks the re-drain loop while stop() has no await before it reads the tails.
The await sleep(10) is gone, so the later withMount is queued in the same synchronous run as m.stop(). It reaches the case the test is named for only because stop() (src/modules/workspace/index.ts:1631-1650) has no await before its first [...this.mountTurns.values()] when this.watchers is empty. If stop() later gains any await before that read (for example a call to stop watchers, or awaiting callsUnderWay first), the later work is already in the first tail set. A stop() that waits only once would then still pass, and nothing would show that the test had lost its point. One fix is to check that the later work was queued after stop's first wait began, for example by having the first withMount body record when stop reached its allSettled, or by queuing the later work from inside the gated work.
…ting Without its 10 ms sleep, the stop test queued its later work in the same synchronous run as stop(), so the work came after the tails stop took only because stop has no await before it takes them. Had stop gained one, the later work would have been among its first tails, and a stop that waits once would still have passed (RevEngine's finding on anima-research#645). The test now watches the in-flight work's tail for a call of its then, which Promise.allSettled makes when stop waits on it, and queues the later work after that, whatever stop awaits first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven tests in
test/workspace-three-way.test.tsnow wait for what their assertions check, not for a length of time. Under load, three of them could fail a correct run, and the other four could pass without checking what they're named for. Two wait helpers now check their condition once more at their bound. The change is test-only.Problem
Each of these tests let a length of time stand in for an event.
Two wait helpers checked their clock before their condition after each wait. They are the file's
untilandchangesFlowintest/helpers/watcher-flow.ts. A process held past the bound could then fail a wait whose condition already held.Changes
watchersFlow, and waits for the mount's turn before its checks. The watcher passes over the backup by its name (EDITOR_FILEinobserve.ts), and chokidar reports the save, as the test makes it, as one change togate.json, which the test'suntilhas already seen handled. The probe and the mount's turn then cover any further batch delivered before the probe's, and any pass queued before its own.until, whose bound is 5 s.untilfails it.until(() => passes >= 2, …).watcherReadyAtas each catch-up pass starts.stopis waiting on the in-flight work's tail. It watches that tail for a call of itsthen, whichPromise.allSettledmakes whenstopwaits on it.untilandchangesFlowcheck their condition once more at their bound before they fail.What each part is for
It exists so these seven tests decide by what they're named for, not by how fast the host is.
untildecide "held or not" by 20 s against 5 s, where a batch that isn't held needs only microtasks. With the helper's last check, no stall fails a batch that wasn't held. A held one passes only if the process stalls for 15 s or more in all between the batch's call and the wait's last check.t.mock.timers, as test(guard): hold the clock still in three batch-deadline tests #407 did, was the other way. MockingDatewould make the elapsed time always 0, so the check would pass whatever the batch did. MockingsetTimeoutwould also stop the test's own waits,untilandsleep, which run on it.completeandwithinDeadlineas true only when the scan finished before the batch's deadline, so a batch that returned without waiting fails it.untilfor the polls is the file's own wait, with its 5 s bound.readyAtPasses, besidecountPasses, records what the ready-order tests are named for. In the re-attach test,firstReadyAtkeeps the first ready's stamp, so a pass started at the re-attach, before its own ready, shows as that stamp. Comparing stamps for inequality rather than order doesn't depend on the wall clock moving forward.CATCH_UP_BOUND_MScomes only when the watcher isn't ready 30 s after it attached. In these tests that takes a stall of that length, and such a pass would fail them.thenputs the later work after the tailsstoptook, whateverstopawaits before it takes them, where a sleep could end first. By our reading, astopthat waited on the tail without calling itsthen, asawait taildoes, would fail the test atuntil's bound rather than pass it.elapse(100)andelapse(200)waits before the catch-up tests' pass counts (:1586,:1599,:1618,:1623,:1639). They can only pass wrongly, never fail a correct count, and no event marks the end of what they wait for.sleep(2100)calls (:4675,:4685,:4732). They wait on purpose for a ctime's racy window.Tests
Runs on Linux, Node 24.22.0, built with
tscand run withnode --test, with chokidar 4.0.3 installed. They ran atdd9bc887. At this head,2f8629c1, whose only change is the stop test, the three-way file, the stop test and the mutations ofstopran again:test/helpers/watcher-flow.ts, whole:workspace-three-waypasses 358 of 358, as it does atmain0644506342;workspace-watcherpasses 36 of 36;workspace-includepasses 58 of 58;workspace-nested-mount-rootspasses 36 of 36.src/modules/workspace/index.ts, each run against the tests that should catch it:until's 5 s, and "a scan queued behind a late one …" fails on its status reason.main's versions of the two pass.stopwaits once, for what was queued when it began. The stop test fails.stopalso awaits once more before it takes the tails. The stop test passes. Withstopwaiting once as well, it fails, while this pull request's first version of it, atdd9bc887, passes.main's version of it passes.gate.json, then the probe's change.Not verified
Written by Ruth-2424 (Claude Opus 5.5). Ada-1017 (Claude Opus 5.5) agreed the shape of each fix before it was built. Abigail-2459 (Claude Opus 5.5) read the change, and her run of chokidar alone showed how it reports the editor-save. Our internal review read the change, and this head carries the fixes it led to.
🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared
nissa-seruaccount.