ci(benchmark): add syncpack, Bun and Yarn repositories, and interactive pickers

.github/workflows/benchmark.yml

@@ -1,10 +1,12 @@
 name: Benchmark
 
-# Times inup next to npm-check-updates, taze, npm-check and npm/pnpm outdated on
-# public repositories pinned to a commit (scripts/bench-compare.mjs), then opens a
-# pull request that updates the README's speed section, docs/guide/benchmarks.md
-# and the chart (scripts/bench-report.mjs). Pull requests that change the
-# benchmark get a short run, with the tables in the job summary only.
+# Times inup next to npm-check-updates, taze, syncpack, npm-check and npm/pnpm/bun
+# outdated, and their interactive pickers next to yarn upgrade-interactive and
+# pnpm/bun update -i, on public repositories pinned to a commit
+# (scripts/bench-compare.mjs). Then it opens a pull request that updates the
+# README's speed section, docs/guide/benchmarks.md and the chart
+# (scripts/bench-report.mjs). Pull requests that change the benchmark get a
+# short run, with the tables in the job summary only.
 
 on:
   workflow_dispatch:
@@ -19,6 +21,7 @@ on:
   pull_request:
     paths:
       - scripts/bench-compare.mjs
+      - scripts/bench-pty.py
       - scripts/bench-report.mjs
       - .github/workflows/benchmark.yml
 
@@ -36,10 +39,10 @@ jobs:
     if: github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success'
     # A fixed image, so successive results come from the same software.
     runs-on: ubuntu-24.04
-    timeout-minutes: 60
+    timeout-minutes: 120
     strategy:
       matrix:
-        fixture: [express, vue-core, tanstack-query]
+        fixture: [express, vue-core, tanstack-query, vendure, backstage]
 
     steps:
       - name: Checkout repository

docs/guide/benchmarks.md

@@ -1,15 +1,27 @@
 # Benchmarks
 
-How long inup takes to find outdated dependencies, next to the tools people use for the same job: [npm-check-updates](https://github.com/raineorshine/npm-check-updates), [taze](https://github.com/antfu-collective/taze), [npm-check](https://github.com/dylang/npm-check), and the package manager's own `npm outdated` or `pnpm outdated`. The [Benchmark workflow](../../.github/workflows/benchmark.yml) produces the results below and the chart in the README.
+How long inup takes to find outdated dependencies, next to the tools people use for the same job: [npm-check-updates](https://github.com/raineorshine/npm-check-updates), [taze](https://github.com/antfu-collective/taze), [syncpack](https://github.com/JamieMason/syncpack), [npm-check](https://github.com/dylang/npm-check), and the package manager's own `npm outdated`, `pnpm outdated` or `bun outdated`. The [Benchmark workflow](../../.github/workflows/benchmark.yml) produces the results below and the chart in the README.
 
 ## What is measured
 
+Two things: how long each tool takes to list every outdated dependency, and how long its interactive picker takes to load every update.
+
+### Listing outdated dependencies
+
 Every tool answers the same question: which dependencies, in every `package.json` of the repository, have a newer version than the one declared? Each one runs headless with the flags that make it check every dependency against its latest version and change nothing (the exact commands are listed with each result). The time runs from launch to exit, with the output discarded.
 
 - **First run**: every cache the tool keeps is emptied before each run. Each run gets a fresh home folder, fresh XDG cache, config and data folders, and a fresh temp folder, so nothing is left from the run before. It matches a first run on a new machine, or a CI job without a cache, with the tool itself already installed.
 - **Repeat run**: the tool keeps its caches, filled by an untimed warmup run. This is the second run on your own machine.
 
-The interactive pickers and installing the upgrades are not measured. inup's picker loads versions with the same lookups as the headless run.
+### Opening the picker
+
+Each tool that has an interactive list of updates opens it in a terminal of 160 columns by 50 rows: inup, `yarn upgrade-interactive`, `pnpm update --interactive --latest`, `bun update --interactive`, taze, syncpack, npm-check and npm-check-updates. The time runs from launch until the picker has every update loaded, and nothing is selected or installed.
+
+"Loaded" is the last thing the picker draws on screen or writes to its cache before it stays quiet for 5 seconds. A picker can finish drawing the rows on screen and keep fetching the rest. `yarn upgrade-interactive` on backstage, for one, stops drawing after about 2.5 seconds and keeps fetching for several more. Counting cache writes includes that work. The 5 quiet seconds are not part of the time.
+
+First and repeat runs work as for listing. The pickers run in rounds, each tool once per round, so a slow spell on the network affects them all alike.
+
+Installing the upgrades is not measured.
 
 ## The repositories
 
@@ -18,33 +30,40 @@ Public repositories, each pinned to a commit so every run checks the same manife
 - [expressjs/express](https://github.com/expressjs/express): one package, npm
 - [vuejs/core](https://github.com/vuejs/core): a pnpm workspace with about 20 packages
 - [TanStack/query](https://github.com/TanStack/query): a pnpm workspace with about 100 packages, examples included
+- [vendurehq/vendure](https://github.com/vendurehq/vendure): a Bun workspace with about 50 packages
+- [backstage/backstage](https://github.com/backstage/backstage): a Yarn 4 workspace with about 270 packages and 590 dependencies
 
-Each repository is installed (`npm install` or `pnpm install --frozen-lockfile`, scripts off) before any timing, because `npm outdated`, `pnpm outdated` and npm-check read `node_modules`. pnpm runs at the version in the repository's `packageManager` field.
+Each repository is installed with its own package manager, with install scripts off, before any timing, because `npm outdated`, `pnpm outdated`, `bun outdated` and npm-check read `node_modules`. pnpm runs at the version in the repository's `packageManager` field, Yarn as the release the repository commits, and Bun at its latest version.
 
 ## How it runs
 
-- [hyperfine](https://github.com/sharkdp/hyperfine) times each tool: one warmup run, then the timed runs. The tables show the median, and the details below them show the standard deviation and the fastest and slowest runs.
+- [hyperfine](https://github.com/sharkdp/hyperfine) times each tool's listing, and the benchmark script times the pickers: one warmup run, then the timed runs. The tables show the median, and the details below them show the standard deviation and the fastest and slowest runs.
 - Each repository runs as its own job on a GitHub-hosted Ubuntu runner, and the tools run one after another on that runner. Compare tools within a column. Two columns, or two workflow runs, may come from different machines and network conditions.
 - inup is built from the commit being tested. The other tools are the latest versions on npm when the workflow runs, and the results list each version.
-- Before any timing, each tool runs once and must exit successfully and find updates. The **Updates found** column shows what each tool reported, as a check that they all did the full job. Counts differ because tools group results differently: `pnpm outdated` lists each package once for the whole workspace, while the others list it once for each `package.json` that declares it. npm-check prints only a table, so it has no count.
+- Before any timing, each tool runs once and must exit successfully and find updates. The **Updates found** column shows what each tool reported, as a check that they all did the full job. Counts differ because tools group results differently: `pnpm outdated` lists each package once for the whole workspace, syncpack lists each version of a dependency once however many `package.json` files share it, and the others list a dependency once for each `package.json` that declares it. npm-check prints only a table, so it has no count.
 
 ## Where tools differ
 
 - inup also checks every dependency for known vulnerabilities during the run, and its times include that check.
+- inup's first interactive run downloads its native core (about 1.5 MB) and checks it, so its first-run picker times include that download. Its listing runs never download it.
 - taze looks versions up through its own metadata service (`npm.antfu.dev`) by default. On a repeat run it reuses what it fetched in the last 30 minutes without contacting the registry. inup revalidates each package with the registry on every run, so a version published a minute ago is already in the report.
 - npm-check-updates keeps no cache by default, so its repeat runs take as long as its first runs.
+- syncpack shows how old each version is, so it downloads each package's full registry document, publish dates included, which is several times larger than the version list the other tools ask for. On a repeat run it reuses what it fetched in the last 30 minutes.
+- `bun outdated` reuses registry responses for as long as the registry's cache headers allow (5 minutes on registry.npmjs.org), so its repeat runs make no requests.
 - npm-check doesn't support workspaces, so it only runs on the single-package repository.
+- With `--workspaces`, npm-check-updates' picker shows one `package.json` at a time and fetches the next after you answer, so its picker only runs on the single-package repository.
+- Yarn 2 and later has no command that lists updates outside `yarn upgrade-interactive`, so the Yarn repository has no package-manager row in the listing tables. npm has no interactive update.
 
 ## Run it yourself
 
-You need [hyperfine](https://github.com/sharkdp/hyperfine#installation) and network access:
+You need [hyperfine](https://github.com/sharkdp/hyperfine#installation), `python3` (for the pickers' terminal) and network access, on macOS or Linux:
 
 ```bash
 pnpm bench:compare --fixture vue-core --runs 10   # writes bench-results/vue-core.json
 node scripts/bench-report.mjs bench-results/*.json  # rewrites the README section, this page and the chart
 ```
 
-Fixtures: `express`, `vue-core`, `tanstack-query`. Times from your machine and network will differ from the runner's.
+Fixtures: `express`, `vue-core`, `tanstack-query`, `vendure`, `backstage`. Times from your machine and network will differ from the runner's.
 
 ## Results