Repository navigation
ci: give CI's jobs time limits and npm test a ten-minute --test-timeout - #486
Merged
nissa-seru merged 1 commit intoOct 18, 2026
Merged
Conversation
… npm test Two macOS Node 20 Test steps hung on October 17 and 18 (runs 39082421393 and 39080940576). Each printed its last result about five minutes in, then nothing, and with no timeout-minutes the check stayed pending toward GitHub's six-hour default. Both stopped after tune-out-e2e's suite. Node's runner reports files in order, so the file that never finished is tune-out-e2e or the next one, tune-out-gate-composition; the logs can't say which. - npm test passes --test-timeout=600000. Under Node 20 (20.20.2 in CI) the bound applies to each test file as a whole, from the file's own start: a file still running at the bound is cancelled and fails by its path, whether it's stuck in a test or while loading. Under Node 24 (24.21.0 in CI) it applies to each test and hook, and a file stuck outside its tests and hooks, such as while loading, runs on to the job's limit. Ten minutes is loose on purpose: the slowest file, mcpl-runtime-lifecycle, takes about three on its own. - The unforced step's node --test gets the same flag. - timeout-minutes: 20 on Build & Test and 10 on the pack check. In the 59 most recent runs, passing Build & Test jobs took 3.6 to 6.5 minutes and pack checks 16 to 30 seconds. - test/npm-test-timeout pins npm test's flag and the unforced step's, and keeps Build & Test's limit at least a passing job's time above the bound, so that under Node 20 a file that sticks within that time fails by name before GitHub cancels the job. - CONTRIBUTING's npm test line names the flag, with a paragraph on what it bounds under each Node. - mcpl-coalescing's comment that npm test sets no timeout of its own now names the bound. publish.yml runs npm test under Node 20, so its Test step gets the file bound through package.json. Its jobs get no limits of their own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Required label not found on this PR. |
1 task done
nissa-seru
pushed a commit
to nissa-seru/agent-framework
that referenced
this pull request
Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI's Build & Test jobs now stop at 20 minutes and its pack check at 10, and
npm testpasses--test-timeout=600000. Before, a Test step that hung kept its check pending until someone cancelled it or GitHub's default limit of six hours ended the job, and nothing in the log said which test file had stopped. Under Node 20 the new bound cancels a test file still running at ten minutes and fails it by its path. Under Node 24 it fails a test still running at ten minutes, by its name.Problem
ci.ymlsets notimeout-minutes, so each job may run for GitHub's default of 360 minutes (workflow syntax).npm testsets no--test-timeout, so a test file that never finishes holds the Test step for as long as the job runs.Two macOS Node 20 jobs hung this way on October 17 and 18, 2026:
5e6448aa7(job): its Test step started at 00:21:28Z and printed its last result at 00:26:22Z,ok 1208 - tune-out end to end. Nothing followed until the job was cancelled at 02:14Z.82735148(job): its Test step started at 23:57:41Z and printed its last result at 00:02:39Z,ok 1207 - tune-out end to end. It was cancelled at 02:23Z, and a rerun on the same head passed.Node's test runner reports test files in order. It holds a file's results until every earlier file has finished (
runner.js, lines 254 to 262, andtest.js, lines 941 to 946). Both logs stop at the end oftest/tune-out-e2e.test.ts's results, so the runner was waiting ontune-out-e2eor on the file after it,tune-out-gate-composition. A test's lines print only once it ends, so the logs can't say which.Changes
npm testpasses--test-timeout=600000.ci.yml:timeout-minutes: 20on Build & Test andtimeout-minutes: 10on the pack check. The unforced test step'snode --testpasses--test-timeout=600000too.test/npm-test-timeout.test.tschecks thatnpm testpasses the flag, thatci.yml's unforced step passes the same, and that Build & Test's limit stays at least a passing job's time above it.CONTRIBUTING.md: itsnpm testline names the flag, and a paragraph says what the flag bounds under each Node and thatci.yml's jobs have limits.test/mcpl-coalescing.test.ts: a comment that saidnpm testsets no timeout of its own now names the bound.What each part is for
mcpl-runtime-lifecycle, takes about three minutes on its own, and the longest single test about 30 seconds.--test-force-exitso that work a test leaves running atstop()shows up. Work that keeps a file's process alive keeps the file from finishing. Under Node 20 the bound turns that into a failure that names the file, and under Node 24 the job limit ends it.npm test, asforce-exit-blocking-stdout's test pins that helper, and holdci.ymlto it: the unforced step passes the same value, and Build & Test's limit stays at least a passing job's time above it. Without the last, raising the flag past the job limit would let GitHub cancel a stuck Node 20 file's job before the bound named the file.publish.yml's jobs. Its Test step runsnpm testunder Node 20, so it gets the file bound, and no hang has been seen in a release run. Also left out: a step after a failed Test that lists the runner'snodeprocesses with their command lines, which could name a stuck file under Node 24 too. No hang has been seen under Node 24.What a caller notices
Nothing outside this repository. In CI and in a local
npm test, a test file still running at ten minutes under Node 20, or a test still running at ten minutes under Node 24, fails withtest timed out after 600000ms. GitHub ends a Build & Test job at 20 minutes and a pack check at 10.publish.ymlrunsnpm testunder Node 20, so its Test step gets the same bound.Tests
New, in
test/npm-test-timeout.test.ts:npm testpasses--test-timeout=600000to the runner, before the test files.ci.yml's unforcednode --testpassesnpm test's--test-timeout.Checks
At
009280cb4, on Linux with Node 24.22.0 and 20.20.2 unless another is named, withmain(e6703634) as the base. CI runs Node 20.20.2 and 24.21.0.CI's install, type-check, build and test steps (
npm install,npx tsc --noEmit,npm run build,npm test):tscand the build are clean.npm testpassed all 2863 tests in 193 suites under Node 24.22.0, in 193 s, and under Node 20.20.2, in 197 s.The unforced step's command, with the flag: all 111 tests passed under each Node.
Each test file alone, under Node 20.20.2: 176 files, 2863 tests, none failed. The slowest file was
mcpl-runtime-lifecycle, at 182 s, and the longest single test took 30.1 s.The new tests pass under both Nodes. Each removal below was confirmed to have changed its file before the tests ran:
npm test's line, all three fail;The bound on CI's two Node versions: a probe ran small test files with
--test-timeout=2000in place of ten minutes, under--test-force-exitasnpm testruns them, and one again without it, as the unforced step runs its files:test timed out after 2000ms); the quick test passesawaitthat never settles--test-force-exit--test-concurrency=1), taking 1.5 s and then 1 sdescribe()of three 1.2 s teststest timed out after 2000ms; the first test passesJob times, from GitHub's record of CI's 59 most recent runs as of October 18, 2026: 224 passing Build & Test jobs took 3.6 to 6.5 minutes, the slowest on macOS, and 59 passing pack checks took 16 to 30 seconds.
Not verified
Out of scope
What hung. The logs narrow it to
tune-out-e2eortune-out-gate-composition. If it happens again, the bound fails the stuck file by name under Node 20, where both hangs ran.Written by Langston-2479 (Claude Opus 5.5). Duncan-2410 and Phoebe-2419 found the two hangs and kept their logs, and Ada-1017 set the limits. After a round of our internal review, Greta-2425 (Claude Opus 5.5) reviewed it at
009280cb4.