Skip to content

ci: give CI's jobs time limits and npm test a ten-minute --test-timeout - #486

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:langston/f158-ci-timeouts
Oct 18, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:langston/f158-ci-timeouts

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

CI's Build & Test jobs now stop at 20 minutes and its pack check at 10, and npm test passes --test-timeout=600000. Before, a Test step that hung kept its check pending until someone cancelled it or GitHub's default limit of six hours ended the job, and nothing in the log said which test file had stopped. Under Node 20 the new bound cancels a test file still running at ten minutes and fails it by its path. Under Node 24 it fails a test still running at ten minutes, by its name.

Problem

ci.yml sets no timeout-minutes, so each job may run for GitHub's default of 360 minutes (workflow syntax). npm test sets no --test-timeout, so a test file that never finishes holds the Test step for as long as the job runs.

Two macOS Node 20 jobs hung this way on October 17 and 18, 2026:

Node's test runner reports test files in order. It holds a file's results until every earlier file has finished (runner.js, lines 254 to 262, and test.js, lines 941 to 946). Both logs stop at the end of test/tune-out-e2e.test.ts's results, so the runner was waiting on tune-out-e2e or on the file after it, tune-out-gate-composition. A test's lines print only once it ends, so the logs can't say which.

Changes

  • npm test passes --test-timeout=600000.
  • ci.yml: timeout-minutes: 20 on Build & Test and timeout-minutes: 10 on the pack check. The unforced test step's node --test passes --test-timeout=600000 too.
  • test/npm-test-timeout.test.ts checks that npm test passes the flag, that ci.yml's unforced step passes the same, and that Build & Test's limit stays at least a passing job's time above it.
  • CONTRIBUTING.md: its npm test line names the flag, and a paragraph says what the flag bounds under each Node and that ci.yml's jobs have limits.
  • test/mcpl-coalescing.test.ts: a comment that said npm test sets no timeout of its own now names the bound.

What each part is for

  • The test timeout makes a hung test fail with a name. Under Node 20, where both hangs ran, it applies to each test file as a whole, so a file stuck in a test or while loading is cancelled at ten minutes. Under Node 24 it applies to each test and hook: one still running at ten minutes fails, named by its test or suite, and a file stuck outside its tests and hooks, such as while loading, runs on.
  • Ten minutes leaves room for a loaded runner. The slowest file, mcpl-runtime-lifecycle, takes about three minutes on its own, and the longest single test about 30 seconds.
  • The unforced step's flag: its three files run without --test-force-exit so that work a test leaves running at stop() shows up. Work that keeps a file's process alive keeps the file from finishing. Under Node 20 the bound turns that into a failure that names the file, and under Node 24 the job limit ends it.
  • The job limits end a hang the test timeout doesn't reach, such as a Node 24 file stuck while loading, or a stalled install. Passing Build & Test jobs took 3.6 to 6.5 minutes in recent runs, and pack checks 16 to 30 seconds (Checks). Twenty minutes is at least a passing job's time above the test timeout, so under Node 20 a file that sticks within that time fails by name before the job limit ends the job.
  • The tests pin the flag in npm test, as force-exit-blocking-stdout's test pins that helper, and hold ci.yml to it: the unforced step passes the same value, and Build & Test's limit stays at least a passing job's time above it. Without the last, raising the flag past the job limit would let GitHub cancel a stuck Node 20 file's job before the bound named the file.
  • Left out: time limits on publish.yml's jobs. Its Test step runs npm test under Node 20, so it gets the file bound, and no hang has been seen in a release run. Also left out: a step after a failed Test that lists the runner's node processes with their command lines, which could name a stuck file under Node 24 too. No hang has been seen under Node 24.

What a caller notices

Nothing outside this repository. In CI and in a local npm test, a test file still running at ten minutes under Node 20, or a test still running at ten minutes under Node 24, fails with test timed out after 600000ms. GitHub ends a Build & Test job at 20 minutes and a pack check at 10. publish.yml runs npm test under Node 20, so its Test step gets the same bound.

Tests

New, in test/npm-test-timeout.test.ts:

  • npm test passes --test-timeout=600000 to the runner, before the test files.
  • ci.yml's unforced node --test passes npm test's --test-timeout.
  • Build & Test and the pack check have time limits, and Build & Test's leaves a passing job's time above the test timeout: at least seven minutes, the slowest passing job rounded up.

Checks

At 009280cb4, on Linux with Node 24.22.0 and 20.20.2 unless another is named, with main (e6703634) as the base. CI runs Node 20.20.2 and 24.21.0.

  • CI's install, type-check, build and test steps (npm install, npx tsc --noEmit, npm run build, npm test): tsc and the build are clean. npm test passed all 2863 tests in 193 suites under Node 24.22.0, in 193 s, and under Node 20.20.2, in 197 s.

  • The unforced step's command, with the flag: all 111 tests passed under each Node.

  • Each test file alone, under Node 20.20.2: 176 files, 2863 tests, none failed. The slowest file was mcpl-runtime-lifecycle, at 182 s, and the longest single test took 30.1 s.

  • The new tests pass under both Nodes. Each removal below was confirmed to have changed its file before the tests ran:

    • with the flag taken out of npm test's line, all three fail;
    • with its value changed to 300000, the first two fail;
    • with the flag taken out of the unforced step, or its value there changed to 300000, the second fails;
    • with Build & Test's limit taken out or lowered to 16 minutes, or the pack check's taken out, the third fails.
  • The bound on CI's two Node versions: a probe ran small test files with --test-timeout=2000 in place of ten minutes, under --test-force-exit as npm test runs them, and one again without it, as the unforced step runs its files:

    Test file Node 20.20.2 Node 24.21.0
    a quick test, then one that never ends the run fails after 2.3 s, naming the file by its path (test timed out after 2000ms); the quick test passes the run fails after 2.4 s, naming the test; the quick test passes
    a top-level await that never settles the run fails after 2.3 s, naming the file still running when the probe stopped it at 25 s
    one test, then a handle left open passes in 0.4 s passes in 0.4 s
    the same, without --test-force-exit the run fails after 2.3 s, naming the file; its test passes still running when the probe stopped it at 25 s
    two files one at a time (--test-concurrency=1), taking 1.5 s and then 1 s both pass: the bound counts from each file's own start both pass
    a describe() of three 1.2 s tests the file fails by its path, test timed out after 2000ms; the first test passes all three pass, and so does the suite
  • Job times, from GitHub's record of CI's 59 most recent runs as of October 18, 2026: 224 passing Build & Test jobs took 3.6 to 6.5 minutes, the slowest on macOS, and 59 passing pack checks took 16 to 30 seconds.

Not verified

  • macOS. Everything above ran on Linux, and CI runs macOS too.
  • A real hang ending at the bound. The hang hasn't come back. The probe stands in for it.

Out of scope

What hung. The logs narrow it to tune-out-e2e or tune-out-gate-composition. If it happens again, the bound fails the stuck file by name under Node 20, where both hangs ran.


Written by Langston-2479 (Claude Opus 5.5). Duncan-2410 and Phoebe-2419 found the two hangs and kept their logs, and Ada-1017 set the limits. After a round of our internal review, Greta-2425 (Claude Opus 5.5) reviewed it at 009280cb4.

… npm test

Two macOS Node 20 Test steps hung on October 17 and 18 (runs
39082421393 and 39080940576). Each printed its last result about five
minutes in, then nothing, and with no timeout-minutes the check stayed
pending toward GitHub's six-hour default. Both stopped after
tune-out-e2e's suite. Node's runner reports files in order, so the file
that never finished is tune-out-e2e or the next one,
tune-out-gate-composition; the logs can't say which.

- npm test passes --test-timeout=600000. Under Node 20 (20.20.2 in CI)
  the bound applies to each test file as a whole, from the file's own
  start: a file still running at the bound is cancelled and fails by its
  path, whether it's stuck in a test or while loading. Under Node 24
  (24.21.0 in CI) it applies to each test and hook, and a file stuck
  outside its tests and hooks, such as while loading, runs on to the
  job's limit. Ten minutes is loose on purpose: the slowest file,
  mcpl-runtime-lifecycle, takes about three on its own.
- The unforced step's node --test gets the same flag.
- timeout-minutes: 20 on Build & Test and 10 on the pack check. In the
  59 most recent runs, passing Build & Test jobs took 3.6 to 6.5 minutes
  and pack checks 16 to 30 seconds.
- test/npm-test-timeout pins npm test's flag and the unforced step's,
  and keeps Build & Test's limit at least a passing job's time above the
  bound, so that under Node 20 a file that sticks within that time fails
  by name before GitHub cancels the job.
- CONTRIBUTING's npm test line names the flag, with a paragraph on what
  it bounds under each Node.
- mcpl-coalescing's comment that npm test sets no timeout of its own now
  names the bound.

publish.yml runs npm test under Node 20, so its Test step gets the file
bound through package.json. Its jobs get no limits of their own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 18, 2026

Copy link
Copy Markdown

Required label not found on this PR.

nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
@nissa-seru
nissa-seru merged commit 14806f1 into anima-research:main Oct 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant