[donfear]

Adds a Benchmark workflow that compares inup with the tools people use for the same job, and publishes the results to the README the same way the demo GIF is published: as a pull request to review.

What it measures

Every tool is asked the same question: which dependencies, in every package.json of the repository, are behind their latest version? Each one runs headless with flags that check every dependency and write nothing. hyperfine times it from launch to exit.

Tool	Command
inup	inup --json
npm-check-updates	ncu --jsonUpgraded [--workspaces]
taze	taze major [--recursive] --include-locked --no-github-actions --no-node-version --json
npm-check	npm-check --skip-unused (single package only: no workspace support)
package manager	npm outdated --json / pnpm outdated --recursive --format json

Repositories, pinned to a commit: expressjs/express (npm, 44 deps), vuejs/core (pnpm, 22 packages), TanStack/query (pnpm, 105 packages).

• First run: before each run, the tool gets a fresh HOME, fresh XDG folders and a fresh TMPDIR, so every cache it keeps is empty.
• Repeat run: caches kept, filled by one untimed warmup run.
• Median of 10 runs, with standard deviation and min–max on the docs page.
• Before timing, each tool runs once and must exit successfully and report updates, so a tool that silently checked nothing fails the job. The counts appear on the docs page.

How it runs

• .github/workflows/benchmark.yml has one job per repository on ubuntu-24.04. It installs hyperfine 1.20.0 (pinned, sha256-checked), inup from the commit being tested, and the other tools at their latest npm versions. Each tool's version is recorded.

• After a manual dispatch or a successful Publish to npm, the report job renders the results and opens or updates an automated-benchmark PR. That PR changes:

  • the README "Speed" section: a light and dark chart plus first-run and repeat-run tables
  • the results on docs/guide/benchmarks.md
  • docs/benchmark/speed-{light,dark}.svg

  It uses the same GH_PAT / GH_EMAIL / GH_USERNAME secrets as the demo workflow.

• A pull request that changes the benchmark gets a 3-run check, with the tables in the job summary only. This PR runs it.

• Locally: pnpm bench:compare --fixture vue-core (needs hyperfine), then node scripts/bench-report.mjs bench-results/*.json.

The README markers are invisible until the first workflow run fills them, so no numbers reach the README except from CI.

Fairness notes (also on the docs page)

• inup's times include its vulnerability check, which the other tools don't run.
• taze uses its own metadata service (npm.antfu.dev) by default. It also answers repeat runs within 30 minutes from its cache, without contacting the registry. inup revalidates with the registry on every run.
• npm-check-updates keeps no cache by default, so its repeat runs take as long as its first runs.

Local sample (MacBook, 2 runs, not published)

First run	express	vuejs/core	TanStack/query
inup	0.39 s	1.13 s	3.37 s
npm-check-updates	0.73 s	5.28 s	16.1 s
taze	1.03 s	2.61 s	11.2 s
npm-check	1.00 s	—	—
npm / pnpm outdated	0.91 s	1.88 s	13.1 s

On repeat runs taze (express, 0.14 s) and pnpm outdated (vuejs/core, 0.38 s) beat inup. The published tables will show that as measured.

Also

• Regenerated the README test badge (1945 → 1986, counted without the native addon, as CI counts it). The 32 new script tests took it past the drift limit.
• Tests: test/unit/scripts/bench-compare.test.ts covers command building, sandboxing, exit codes and output parsing. test/unit/scripts/bench-report.test.ts covers the tables, docs page, chart and README splicing.