Skip to content

chore: Lower CI test times - #8999

Open
andypalmi wants to merge 1 commit into
mainfrom
chore/lower-ci-test-times
Open

andypalmi wants to merge 1 commit into
mainfrom
chore/lower-ci-test-times

Conversation

@andypalmi

Copy link
Copy Markdown
Contributor

Description

The Backend and Postgres test jobs take about 20 minutes each. Most of that time goes into the hooks that boot the app before each test (setupApp runs 423 times), not into the tests themselves. Each boot spends most of its time hashing seed passwords at bcrypt cost 10 and recompiling every route schema.

  1. Parallel mocha: test:unit:forge:ci runs with --parallel --jobs 4, which matches the 4 vCPUs of the runner. Under Postgres, each worker gets its own database (<name>_w<MOCHA_WORKER_ID>).
  2. bcrypt cost 4 under test: test/lib/fast-bcrypt.js, loaded with --require. Production code is unchanged.
  3. Shared schema compilation under test: test/lib/fastify-schema-cache.js passes fastify a compilersFactory that reuses compiled validators and serializers across app instances in the same process. Boot time drops from about 530ms to about 210ms.
  4. Test jobs no longer wait for the TypeScript drift check: the summary job still requires it, so drift still fails the run.
  5. Flaky tests the faster runs exposed:
    • The auth spec now takes the latest audit entry by id instead of createdAt, which ties within the same millisecond.
    • The device group spec now compares members regardless of order, because DeviceGroup.byId doesn't order them.
  6. MCP refresh token race: two simultaneous refreshes could both lose the compare-and-swap, so neither got a token. A refresh now claims the rotation by inserting the AccessTokenRefreshRotation record first; the unique tokenHash index lets only one insert succeed, and the other refresh resolves through that record. If the row was revoked between the read and the update, the claim is removed and the refresh returns null.

Local timings for the unit suite, 4087 tests:

SQLite Postgres
main, serial 462s 571s
this branch 56-68s 112-125s

nyc coverage is still collected from the parallel workers (475 files).

Related Issue(s)

#8478

Checklist

  • I have read the contribution guidelines
  • Suitable unit/system level tests have been added and they pass
  • Documentation has been updated
  • Changes flagged

- Run the forge unit tests with mocha --parallel (4 workers), giving each
  worker its own Postgres database
- Hash passwords with bcrypt cost 4 under test
- Share compiled route schemas between app instances in a test process
- Stop the test jobs waiting on the TypeScript drift check; the summary
  job still requires it
- Fix tests that the faster runs made flaky: order audit entries by id,
  compare device group members regardless of order
- Claim an MCP refresh token rotation before rotating the token, so two
  simultaneous refreshes cannot both lose the compare-and-swap
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.43%. Comparing base (44f3ce0) to head (8201ad3).

Files with missing lines Patch % Lines
forge/db/controllers/AccessToken.js 72.72% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8999      +/-   ##
==========================================
- Coverage   79.44%   79.43%   -0.02%     
==========================================
  Files         475      475              
  Lines       26017    26024       +7     
  Branches     6934     6936       +2     
==========================================
+ Hits        20670    20672       +2     
- Misses       5347     5352       +5     
Flag Coverage Δ
backend 79.43% <72.72%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch was successfully deployed

1 active deployment
staging — 8201ad3a Deployed Oct 20, 2026 by andypalmi via Deploy application #12627
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant