Skip to content

Cache repeated npm module loader filesystem work - #155

Merged
noise64 merged 9 commits into
bench/npm-metadata-baselinefrom
feat/npm-loader-caches
Sep 21, 2026
Merged

noise64 merged 9 commits into
bench/npm-metadata-baselinefrom
feat/npm-loader-caches

Conversation

@noise64

@noise64 noise64 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor
  • related to GOL-348
  • related to GOL-350
  • caches missing package.json reads within one outer CommonJS resolution graph, clearing the negative set when the graph ends and after guest filesystem mutations
  • caches successful module-loader realpaths for one QuickJS runtime while keeping CommonJS and ESM cache domains separate
  • preserves symlink flags, retries failures, and leaves public node:fs realpath uncached
  • retains aggregate prototype results in Markdown and only the final reviewed raw P2/P3 report pair

Results

The final candidate was measured with three serial, fresh-state, cold local-registry samples of npm --version, npm view, and npm ci on both P2 and P3. The physical call counts were identical on both targets.

Command Missing package.json reads Physical loader realpaths
npm --version 204 → 96 (−52.9%) 426 → 78 (−81.7%)
npm view 1,768 → 596 (−66.3%) 3,545 → 477 (−86.5%)
npm ci 2,645 → 847 (−68.0%) 5,007 → 616 (−87.7%)

Final cold local-registry timing medians:

Target / command Wall Process CPU
P2 npm --version 0.582s 0.570s
P2 npm view 3.963s 3.923s
P2 npm ci 6.974s 6.944s
P3 npm --version 0.592s 0.590s
P3 npm view 4.376s 4.233s
P3 npm ci 7.965s 7.533s

Timings are observational and can vary between runs. The reconciled physical filesystem call counts are the comparison invariant. All 60 final samples succeeded, every ci installed both pinned packages, local HTTP request counts matched expectations, and no trace overflow occurred.

Full methodology, aggregate prototype tables, and final raw-report links are in 2026-09-21-cache-experiments.md.

Correctness boundaries

  • failed realpaths remain uncached and retryable
  • public node:fs realpath APIs remain uncached and observe filesystem changes
  • guest filesystem mutations invalidate relevant missing-metadata entries
  • --preserve-symlinks bypasses both loader caches; --preserve-symlinks-main bypasses CommonJS main canonicalization
  • CommonJS and ESM realpath caches use separate domains
  • Wizer initialization cannot populate loader filesystem caches
  • relative CommonJS inputs resolve against process.cwd() before native cache lookup

Validation

  • complete GitHub P2/P3 runtime and node-compat matrix
  • all-target build and Clippy
  • ten-lane P2/P3 skeleton Clippy matrix
  • focused P2/P3 loader-cache runtime coverage and npm install/isolation cases
  • DTS goldenfile regeneration and verification
  • module-loader architecture checks and manual benchmark guards
  • Prettier, ESLint, and TypeScript compilation
  • post-push Claude exact-SHA review: clear

@noise64
noise64 marked this pull request as ready for review September 21, 2026 14:11
@noise64
noise64 merged commit 8bbdc5f into bench/npm-metadata-baseline Sep 21, 2026
80 of 81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant