diff --git a/.agents/docs/architecture/testing-strategy.md b/.agents/docs/architecture/testing-strategy.md index e1b4519f4..735955abf 100644 --- a/.agents/docs/architecture/testing-strategy.md +++ b/.agents/docs/architecture/testing-strategy.md @@ -17,11 +17,11 @@ of operations. ## Tiers -| Tier | Runner | Where | Command | -| ---------- | ---------- | -------------------------------------------------------- | --------------- | -| Unit | vitest | `src/__tests__/` (or `test/`) in a participating package | `pnpm test` | -| End-to-end | Playwright | `testing/src/specs/` | `pnpm test:e2e` | -| Type-check | `tsc` | — | `pnpm lint` | +| Tier | Runner | Where | Command | +| ---------- | ---------- | -------------------------------------------------------- | -------------------- | +| Unit | vitest | `src/__tests__/` (or `test/`) in a participating package | `pnpm test:coverage` | +| End-to-end | Playwright | `testing/src/specs/` | `pnpm test:e2e` | +| Type-check | `tsc` | — | `pnpm lint` | There is no integration tier. `apps/api` unit tests mock the Prisma layer entirely; the only code path exercised against a real database is Playwright. @@ -115,6 +115,25 @@ no database. Only `testing` owns a `test:e2e` script; only `apps/gateway` owns a `globalDependencies` are `.env`, `eslint.config.js`, `tsconfig.json` and `prettier.config.js`. Touching any of them invalidates the cache for every task in the repo. +## Coverage + +`pnpm test:coverage` is the unit gate, locally and in CI. The root `vitest.config.ts` sets +`coverage.thresholds: { 100: true }`, so the run fails when statements, branches, functions or lines +drop below 100%. The scope is its `coverage.include` — `apps/{api,gateway,web}/src` and +`packages/*/src`; `runtime/**` and `apps/playground` are outside it — minus `coverage.exclude`, +which lists only non-code: test fixtures and helpers (`__tests__`, `__mocks__`, +`apps/web/src/testing`), the generated `route-tree.ts`, and the bare entrypoints. + +- **A line no test can reach is dead code.** Prove it (the types, every caller, the library's + contract) and delete it, or restructure so the impossible state cannot be written — e.g. ts-pattern + `.exhaustive()` instead of `.otherwise(() => null)`. Never add an ignore comment or an exclude entry + to pass. +- **A scoped run fails the threshold by construction**: `--project`, a file filter or a narrowed + `--coverage.include` measures a fraction of the scope. Add `--coverage.thresholds.100=false` to it + and read the per-file report. +- `apps/api` confines its SWC transform to its own files (`apps/api/vitest.config.ts`); letting SWC + transform the workspace packages it imports makes the merged report count their functions twice. + ## What CI gates `.github/workflows/ci.yaml`, one job (`lint-and-test`, ubuntu-latest), triggered on @@ -125,7 +144,7 @@ Touching any of them invalidates the cache for every task in the repo. | 1 | Generate Environment | `./scripts/generate-env.sh` | | 2 | Install Dependencies | `pnpm install --frozen-lockfile` | | 3 | Lint | `pnpm lint` | -| 4 | Unit Tests | `pnpm test` | +| 4 | Unit Tests | `pnpm test:coverage` | | 5 | Install Playwright | chromium + firefox `--with-deps`, capped at 10 minutes | | 6 | End-to-End Tests | `pnpm test:e2e` | @@ -135,8 +154,6 @@ does not hide their results. Not gated, despite existing in the repo: - **No separate type-check step** — it is folded into step 3. -- **No coverage.** Nothing in CI runs `pnpm test:coverage`, and the root config declares no - `thresholds`. Its `include` is `apps/**/*` and `packages/**/*`, so `runtime/**` is outside it. - **No knip step**, although `knip.ts` and a `pnpm knip` script exist. - **No format check.** Prettier runs only in the `.husky/pre-commit` hook (`prettier-pre-commit`), which formats rather than verifies; the hook does not run lint or tests. diff --git a/.agents/docs/playbooks/add-env-var.md b/.agents/docs/playbooks/add-env-var.md index 1c82306a2..57dac99d7 100644 --- a/.agents/docs/playbooks/add-env-var.md +++ b/.agents/docs/playbooks/add-env-var.md @@ -78,6 +78,6 @@ touched exactly two of these: `.env.template` and `$Env`, replacing a direct grep -n YOUR_KEY .env # present after regeneration pnpm dev # $Env is validated at boot; an invalid value crashes here. Ctrl-C once up pnpm lint # tsc + the no-process.env rule -pnpm test +pnpm test:coverage pnpm build # only this catches a missing turbo.json `env` entry ``` diff --git a/.agents/docs/playbooks/add-instrument.md b/.agents/docs/playbooks/add-instrument.md index 0674a8281..37d366f48 100644 --- a/.agents/docs/playbooks/add-instrument.md +++ b/.agents/docs/playbooks/add-instrument.md @@ -82,7 +82,7 @@ advice does not apply here; everything it says about the definition object does. pnpm exec turbo run build --filter=@opendatacapture/instrument-library # builds runtime/v1 first pnpm --filter @opendatacapture/instrument-library run available # reads dist, not src pnpm lint -pnpm test +pnpm test:coverage pnpm test:e2e ``` diff --git a/.agents/docs/playbooks/add-vendor-package.md b/.agents/docs/playbooks/add-vendor-package.md index 62cf7e9a7..3f2f2c49d 100644 --- a/.agents/docs/playbooks/add-vendor-package.md +++ b/.agents/docs/playbooks/add-vendor-package.md @@ -73,5 +73,5 @@ pnpm --filter @opendatacapture/runtime-v1 build # READ THE OUTPUT — it logs ls runtime/v1/dist/@ # step 8 was silent if this is missing pnpm exec vitest --project runtime-v1 # peer pairing pnpm exec vitest --project runtime-bundler -pnpm lint && pnpm test +pnpm lint && pnpm test:coverage ``` diff --git a/.agents/docs/playbooks/add-web-data-hook.md b/.agents/docs/playbooks/add-web-data-hook.md index 562c42508..70ebfa868 100644 --- a/.agents/docs/playbooks/add-web-data-hook.md +++ b/.agents/docs/playbooks/add-web-data-hook.md @@ -73,5 +73,5 @@ step type-checks and compiles, and fails at runtime or serves the wrong cached d ```sh pnpm exec vitest apps/web/src/hooks/__tests__/.test.ts # the new test alone pnpm exec vitest --project web # every web test -pnpm lint && pnpm test # required before you are done +pnpm lint && pnpm test:coverage # required before you are done ``` diff --git a/.agents/docs/playbooks/promote-to-react-core.md b/.agents/docs/playbooks/promote-to-react-core.md index 1325185a4..386c8f8fc 100644 --- a/.agents/docs/playbooks/promote-to-react-core.md +++ b/.agents/docs/playbooks/promote-to-react-core.md @@ -104,7 +104,7 @@ pnpm --filter @opendatacapture/react-core lint # no app aliases in this pro pnpm --filter @opendatacapture/web lint pnpm --filter @opendatacapture/gateway lint # meaningful only once step 9 wired the consumer pnpm exec vitest --project web -pnpm lint && pnpm test +pnpm lint && pnpm test:coverage pnpm test:e2e pnpm --filter @opendatacapture/storybook storybook # port 6006 — listed under React Core, not Web ``` diff --git a/.agents/skills/odc-done/SKILL.md b/.agents/skills/odc-done/SKILL.md index 782ce8282..541226bc8 100644 --- a/.agents/skills/odc-done/SKILL.md +++ b/.agents/skills/odc-done/SKILL.md @@ -37,14 +37,15 @@ exception rather than editing the tree yourself. ## 2. Test, and make the run name your file ```sh -pnpm test +pnpm test:coverage # fails below 100% on any metric — see testing-strategy.md §Coverage pnpm exec vitest list --filesOnly # a green run prints counts only; this prints the collected files ``` A test file the listing never names was collected by nothing — `.agents/skills/odc-testing/SKILL.md` opens on that failure and owns both the diagnosis and the fix. -**Done when** the run is green _and_ `list --filesOnly` names every test file this change added. +**Done when** the run is green at 100% coverage _and_ `list --filesOnly` names every test file this +change added. ## 3. Run the end-to-end suite, or raise the exception diff --git a/.agents/skills/odc-testing/SKILL.md b/.agents/skills/odc-testing/SKILL.md index 9bcadfe34..4c19cfd60 100644 --- a/.agents/skills/odc-testing/SKILL.md +++ b/.agents/skills/odc-testing/SKILL.md @@ -27,11 +27,11 @@ and the e2e test carries the change, or this change adds its `vitest.config.ts`. ## Choose the tier before you choose the file -| What you changed | The tier that can prove it | Runner | -| ------------------------------------------------------- | ------------------------------------------ | --------------- | -| A pure function, a schema, a hook, a rendered component | Unit, in that package's own project | `pnpm test` | -| Who can see or do what; anything reading real rows | End-to-end only | `pnpm test:e2e` | -| A type contract — inference, a discriminated union | The type-check, which is `tsc` inside lint | `pnpm lint` | +| What you changed | The tier that can prove it | Runner | +| ------------------------------------------------------- | ------------------------------------------ | -------------------- | +| A pure function, a schema, a hook, a rendered component | Unit, in that package's own project | `pnpm test:coverage` | +| Who can see or do what; anything reading real rows | End-to-end only | `pnpm test:e2e` | +| A type contract — inference, a discriminated union | The type-check, which is `tsc` inside lint | `pnpm lint` | **There is no integration tier, and the middle row is why.** `apps/api` unit tests mock the Prisma layer entirely (`apps/api/src/groups/__tests__/groups.service.spec.ts`), so the only code path diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 380b751c6..95e8b6fff 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -31,7 +31,7 @@ jobs: - name: Lint run: pnpm lint - name: Unit Tests - run: pnpm test + run: pnpm test:coverage - name: Install Playwright # Cap the step so a hang fails fast instead of hitting the 6h job limit, skip the interactive # apt prompt, and install only the browsers the e2e suite uses. diff --git a/AGENTS.md b/AGENTS.md index 6605d779f..bd1d66e07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,17 +21,20 @@ pnpm build # turbo build (all packages/apps) pnpm lint # turbo lint — `tsc && eslint --fix src` per package, plus lint:root for root-level scripts and configs pnpm format # turbo format (prettier per package) pnpm test # vitest across the workspace +pnpm test:coverage # vitest with coverage — fails below 100%, the gate CI and "Before you are done" run pnpm test:e2e # playwright, in testing/ ``` Scope a run to one package with `pnpm --filter @opendatacapture/