ci(static): run the unit suite once, under coverage - #236
Merged
Conversation
The `static` job ran `pnpm test` and then `pnpm test:coverage` back to back — the same files, the same tests, twice. `--coverage` changes what is measured, never what is collected, and V8 coverage is gathered through the inspector rather than by rewriting sources, so the second run was the first plus the thresholds and could not decide a result differently. Measured on two green pull-request runs: the plain run took 4m 56s and 4m 46s of an 11m 52s / 11m 37s job, against about 80s for setup and every other gate in it combined. Dropping it leaves one run of the suite and the coverage gate intact. No threshold moved and no gate was removed. `pnpm test` stays a script — developers run it, `pnpm verify` chains it, and the `migrations` job runs it against a real Postgres. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WBLHKvRatuUufybYpgER1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes MEI-127.
The
staticjob ranpnpm testand thenpnpm test:coverage— the same ~7,746 tests across 444 files, twice, back to back.--coveragechanges what is measured, never what is collected, so the first run established nothing the second did not.Measured
Per-step timings from
list_workflow_jobson two green runs:pnpm testpnpm test:coverageStatic checkstotalThe coverage step is ~35s dearer than the plain one — report generation and threshold checking, not slower tests.
Projected after: ~6m 55s. The job is strictly sequential, so that is job-minus-step arithmetic on measured values — but it is a projection, not an observation. This PR's own run is the first real figure.
Static checksis the last job to finish in both runs, so the whole-run wall clock moves with it.Why this is safe
--coverageselects a provider; it does not touchtest.include, so the collected set is identical. The V8 provider reads the inspector's coverage rather than rewriting sources, so there is no transformed code under test. A 224-test subset weighted toward timing-sensitive files ran 3.15s plain vs 3.05s instrumented — nothing for a marginal timing assertion to trip over.Static checkscontext is unchanged. Neither release workflow invokes either script.pnpm testremains a script.pnpm verifychains it, and themigrationsjob still runs it withTEST_DATABASE_URLset for the seams that need a real Postgres.No threshold changed, no gate removed.
What was considered and rejected
Splitting the gates into their own job. Its stated benefit does not exist: the gates already run before the tests and a job aborts at its first failure, so a broken lint rule or doc anchor already fails at ~1m 20s. Eleven minutes was only ever the cost of a green run. A split would run the gates in parallel with the tests — ~50s off this job, and zero off the run, because the browser shards are 5m 55s.
Sharding the unit suite. It would work, and buy nothing, for the same reason. Against that sits a real cost: each shard sees coverage only for the files its own tests touched, so thresholds must be disabled per shard, partial reports merged, and thresholds re-judged on the merge — a step that would pass silently if it produced an empty report. That is a hollow gate taken on for no wall-clock gain.
vitest.config.ts'smaxWorkers: 4matches the 4-vCPUubuntu-latestrunner and is left alone.Not fixed here
Six gates chained in
pnpm verifyrun in no CI workflow:workspace:check,i18n:check,board:gen:check,marketplace:gen:check,board-installer:gen:check,vercel-template:gen:check. A pull request breaking any of them merges green. Tracked as MEI-128 rather than widened into this change.pnpm verifypasses: 7746 tests.Generated by Claude Code