Skip to content

CI: run the whole suite once at Node 22.12.0, and check install and start at 22.0.0 and 24 - #19

Merged
nissa-seru merged 2 commits into
anima-research:mainfrom
nissa-seru:ci/testing-setup
Oct 28, 2026
Merged

nissa-seru merged 2 commits into
anima-research:mainfrom
nissa-seru:ci/testing-setup

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

This changes github-mcpl's CI to run the typecheck and the whole suite once, on Linux at Node 22.12.0, and adds a check that the server installs, builds and starts at Node 22.0.0, the floor engines declares, and at Node 24. CI at this head passed all three jobs. Nothing a user, operator or host of the server meets changes.

Problem

At main (7eaea03):

  • CI runs one job, "Typecheck & Test (node 22)", on Linux with node-version: 22 (.github/workflows/ci.yml:14, :26), which takes a 22.x release: main's last CI run used v22.23.3, from the runner's cache. No job runs on another Node version.
  • Nothing checks Node 22.0.0, the floor package.json's engines declares (>=22.0.0), and npm test can't run there. Its --experimental-test-isolation=none flag (package.json:13) is refused with "bad option" and exit 9 by every Node 22 release from 22.0.0 to 22.7.0, and taken by Node 22.8.0.

Changes

  • "Typecheck & Test (node 22.12.0)": the same three steps as before (npm ci, npm run typecheck, npm test), on Linux at Node 22.12.0 exactly.
  • "Install & Start (node 22.0.0)" and "Install & Start (node 24)": npm ci, npm run build and node scripts/ci-start-check.mjs, on Linux. Node 24 is given as 24, which takes a 24.x release, from the runner's cache when it has one, as 22 did. The matrix doesn't fail fast, so each Node version gets its own verdict. Each job has a 10-minute timeout.
  • scripts/ci-start-check.mjs (new): it runs the built server, dist/src/index.js --stdio, with a throwaway configuration: an RSA key it generates in a new temporary directory, one example repository, polling off and no webhook port. It sends one MCPL initialize, and passes only on a JSON-RPC 2.0 result whose capabilities.experimental.mcpl is an object, followed by exit 0 once it closes the server's stdin. It fails by name, with the server's stderr, on no answer within 30 s, a line that isn't a JSON object, an answer without that result, an exit before answering, an exit other than 0 after it, or no exit within 10 s of stdin closing. Every ending after the server starts kills the server if it's still running and removes the temporary directory.
  • README.md: the development section's "Node 22" now says npm test runs on 22.12 or newer.
  • b899b4b, this head, moved the suite job from Node 22.8.0 to 22.12.0, and changed the README's "needs 22.8 or newer" to "runs on 22.12 or newer". This pull request's first CI run, at 57820ab, had the suite job at 22.8.0, where none of the 18 test files loaded (Tests, below).

Tests

On Linux, after npm ci from the lockfile:

  • CI at this head (run 40770218205): "Typecheck & Test (node 22.12.0)" passed, with # tests 617, # pass 599, # fail 0 and # skipped 18 on Node v22.12.0. Both start legs passed, on Node v22.0.0 and v24.22.1.
  • CI's first run, at 57820ab (run 40769830290): both start legs passed. The suite job, then at Node 22.8.0, failed: # tests 18, # fail 18, each test file not ok with a TypeError, Unknown file extension ".ts" for that file. Node 22.8.0 took the flag, but no test file loaded.
  • The start check, on a machine of ours: it passed against the built server on Node 22.0.0 and on Node 24.22.1, each after npm ci and the build under that Node. No other Node 24 release ran. These runs were at 57820ab, whose scripts/ci-start-check.mjs is this head's.
  • The start check fails when the server doesn't start right. Against twelve stand-ins written over the built entry point, each failure named its cause and exited 1: no answer (after 31 s), an exit 1 at once (with its stderr), an answer and then no exit (after 10 s), a JSON-RPC error, plain MCP without the mcpl capability, a line that isn't JSON, an exit 3 after answering, a JSON null line, an answer without "jsonrpc": "2.0", "mcpl": true, a result with an error beside it, and a 500-character line, shown cut at 200 characters. With dist/src/server.js deleted, it failed with the server's ERR_MODULE_NOT_FOUND stderr. No temporary directory was left after any of these runs.
  • The Node floor: at main, npm test exited 9 on Node 22.0.0 with "node: bad option: --experimental-test-isolation=none". On a one-test file, node --test --experimental-test-isolation=none exited 9 with "bad option" on each of the ten Node releases from 22.0.0 to 22.7.0, and passed on 22.8.0.

Not verified

  • Node 22.9 to 22.11. The suite hasn't run on those releases, so whether it loads there isn't shown.
  • macOS and Windows. Neither the old workflow nor this one runs there. In package-lock.json, the only entries that set os or cpu are fsevents and 26 @esbuild/* packages, all development dependencies.

With other open pull requests

By git merge-tree against main 7eaea03 and each open pull request's head:


No changelog fragment: this changes CI and one README line.

Ogden-2688 made this change, and Agnes-984 made the calls on it. Crispin-2406 read it. Fiona-2757 wrote this description.

Ogden-2688 added 2 commits October 28, 2026 16:49
…0.0 and 24

The workflow ran npm test on Linux at whatever Node 22 release setup-node
picked. It now runs:

- the typecheck and the whole suite once, on Linux at Node 22.8.0 exactly:
  npm test's --experimental-test-isolation flag needs 22.8, and 22.0.0
  refuses it ("bad option").
- an install and start check at Node 22.0.0, engines' floor, and at the
  newest Node 24: npm ci, the build, and scripts/ci-start-check.mjs. The
  matrix doesn't fail fast, so each Node gets its own verdict.

The start check runs the built server with --stdio under a throwaway
configuration (a generated App key, one example repository, polling off,
no webhook), so it needs no credentials and reaches nothing. It sends one
MCPL initialize and requires a JSON-RPC 2.0 result carrying the mcpl
capability, then closes stdin and requires exit 0. Anything else fails it
by name within 30 s and 10 s, with the server's stderr: no answer, a line
that isn't a JSON object, an answer without that result, an exit before
answering, an exit other than 0 after it, or no exit. Every ending kills
the server if it still runs and removes the key's temporary directory.
A write to a server that has already exited fails with EPIPE; the check
ignores that error, since the exit itself is what it reports.

README's development section said Node 22; npm test needs 22.8 or newer
since the test files began to share one process.
The workflow's first run had the whole-suite job at Node 22.8.0, the first
release that takes npm test's --experimental-test-isolation=none. Node took
the flag, but all 18 test files failed to load with 'TypeError: Unknown
file extension ".ts"', so no test ran. The job moves to Node 22.12.0, where
the whole suite has run in one process with the same verdicts as on Node
24. The install and start legs stay at 22.0.0, engines' floor, and 24.

README's development line said npm test needs 22.8 or newer; it now says
it runs on 22.12 or newer.
@nissa-seru nissa-seru changed the title CI: run the whole suite once at Node 22.8.0, and check install and start at 22.0.0 and 24 CI: run the whole suite once at Node 22.12.0, and check install and start at 22.0.0 and 24 Oct 28, 2026
@nissa-seru
nissa-seru merged commit 4b5244d into anima-research:main Oct 28, 2026
3 checks passed
nissa-seru pushed a commit to nissa-seru/github-mcpl that referenced this pull request Oct 28, 2026
main now runs CI the testing setup's way (anima-research#19, anima-research#20): the whole suite once
at a pinned Node, and an install and start check on engines' floor, Node 24
and the newest Node release. With engines at >=22.12.0 here, the
resolution keeps that shape at this branch's floor:

- The suite job runs at Node 22.12.0, engines' floor, in place of this
  branch's Node 22 and 24 matrix of whole suites.
- The install and start check runs on Node 24 and on the newest release.
  main's 22.0.0 leg checked the old floor; the floor is now 22.12.0, where
  the suite job runs.
- README's development line keeps this branch's "Node 22.12 or newer",
  which covers npm test as well as the server.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant