Reduce npm and TypeScript module-loading overhead

[noise64]

• resolves GOL-348
• follows up GOL-350 and GOL-347
• reduces repeated npm loader filesystem work and TypeScript/CommonJS source scanning
• adds reproducible production-release P2/P3 measurement and report-currentness coverage

Why

Real npm and TypeScript workloads exposed avoidable work in package metadata
lookups, loader realpaths, whitespace-heavy ESM scanning, large CommonJS source
preparation, source-map extraction, and syntax detection for already-known
module formats. After those owners were reduced, matched production release
measurements were added to show the remaining end-to-end gap and screen further
ideas against correctness, memory, and reproducibility gates.

What changed

• adds a pinned P2/P3 npm benchmark with a deterministic local registry,
  bounded tracing, counter reconciliation, matched host runs, and retained
  baseline/final reports
• caches missing package.json reads only within one outer CommonJS resolution
  graph and invalidates them on guest filesystem mutation
• caches successful loader realpaths for one QuickJS runtime, then extends the
  cache with known non-symlink directory prefixes; CommonJS/ESM domains remain
  separate, failures retry, Wizer starts clean, and preserve-symlink paths
  bypass canonicalization
• adds ESM phase attribution and bulk-skips contiguous ASCII whitespace in the
  CJS-global ESM scanners; also bypasses import.meta.main rewriting when no
  import token exists
• moves TypeScript-feature CommonJS sourceMappingURL extraction to the
  existing native SWC lexer while retaining the JavaScript fallback for
  non-TypeScript and VM builds
• dispatches CommonJS export parsers only at accepted leading bytes and
  advances the direct-eval, import-attribute, and template-expression
  scanners between relevant sentinel bytes
• classifies fixed CommonJS/ESM extensions and package policies before source
  inspection, preserving cached-TypeScript and force_module precedence
• adds production release profiles to the measured local workflow, matched
  five-sample npm/TypeScript runners, memory and cache-isolation evidence, and
  CI currentness selection that handles direct, squash, and two-parent merge
  trees without rewriting historical reports
• adds focused runtime and architecture coverage for cache scope, invalidation,
  Wizer initialization, source maps, source preparation, format precedence,
  ESM phases, and import attributes, including the regenerated DTS goldenfile
• retains Markdown experiment summaries and only the final reviewed raw P2/P3
  pairs; temporary instrumentation and rejected sample-level artifacts are
  removed

Results

npm loader caches

The original three-sample fixed-fixture comparison reduced physical work
identically on P2 and P3:

Command	Missing package.json reads	Physical loader realpaths
npm --version	204 → 96 (-52.9%)	426 → 78 (-81.7%)
npm view	1,768 → 596 (-66.3%)	3,545 → 477 (-86.5%)
npm ci	2,645 → 847 (-68.0%)	5,007 → 616 (-87.7%)

The later directory-prefix cache was measured against the production release
pair with five samples:

Target / workload	Before	Prefix cache	Change
P2 cold metadata	1,332.059 ms	1,069.307 ms	-19.7%
P3 cold metadata	1,277.538 ms	1,052.270 ms	-17.6%
P2 warm-tarball npm ci	2,718.971 ms	2,272.384 ms	-16.4%
P3 warm-tarball npm ci	2,503.777 ms	2,360.249 ms	-5.7%

The original and prefix-cache rows were collected in separate measurement
sessions, not as an interleaved A/B, so the wall-time deltas are directional
rather than a causal speedup estimate. A contemporaneous TypeScript pair also
drifted upward after host time was subtracted: 146/306 ms cold, 68/68 ms
repeated, and 28/154 ms incremental on P2/P3. The deterministic path-work
reductions and regression coverage remain valid independently of wall time.

Peak observed linear memory was 55.25 MiB on P2 and 48.25 MiB on P3, +4.7%
and +0.3% against the anchors and within the 10% gate. All 60 retained
host/Wasm samples succeeded; HTTP totals, npm logs, outputs, installed package
identities, lockfiles, and counters reconciled. The original cache comparison
retains measured revision 8d030cf7; the release prefix-cache reports retain
831632c6.

stripped ESM loading

The five-sample 64 KiB stripped-ESM phase experiment fell from
10,657.94 → 192.28 ms (P2) and 11,058.54 → 197.36 ms (P3), a 98.2%
reduction for that deliberately pathological workload. The reports retain
measured revision 7bed8b04; the refreshed uninstrumented matrix records
prepared-ESM medians of 190.68 ms (P2) and 188.89 ms (P3).

TypeScript module loading

Native source-map extraction reduced five-sample cold tsc --noEmit from
19.17 → 16.72 s (P2, -12.7%) and 19.22 → 16.90 s (P3, -12.1%).
Repeated unchanged medians improved 11.2%/11.0%, warm incremental medians
improved 19.6%/17.4%, and TypeScript API import improved 30.9%/29.3%.

On top of that candidate, CommonJS source preparation reduced TypeScript API
import from 8.33 → 4.22 s (P2, -49.3%) and 8.49 → 4.22 s
(P3, -50.3%). The known-format step then reduced it from
4.22 → 3.47 s (P2, -17.9%) and 4.22 → 3.44 s (P3, -18.4%). Final
clean reports retain measured revision dd689c8c.

production release baselines

The final matched TypeScript pair records:

Workload	Host (P2/P3 run)	P2	P3
cold tsc --noEmit	0.530/0.528 s	5.773 s	5.879 s
repeated unchanged	0.445/0.459 s	5.617 s	5.621 s
warm incremental	0.196/0.200 s	2.729 s	2.830 s

Against the practical 8 × host + 1 s target, P2/P3 miss by 0.534/0.658 s
cold, 1.059/0.951 s repeated, and 0.159/0.231 s incremental. Linear-memory
high water remains 145.06 MiB.

The final npm pair records 1.069/1.052 s P2/P3 cold metadata and 2.272/2.360 s
warm-tarball npm ci. Metadata misses 3 × host + 0.5 s by about 99/104 ms;
ci misses 2 × host + 1 s by about 776/898 ms.

Each timed invocation uses a fresh QuickJS execution job. Cold rows also use a
fresh component instance, workspace, and cache. A timed warm-tarball npm job
reuses the component instance and workspace/cache populated by its untimed
seed; its memory high-water observation therefore includes the seed peak.

Remaining-cost attribution

• repeated P2 tsc --extendedDiagnostics attributed 4.830 s of a 5.865 s
  instrumented median to TypeScript: 2.850 s checking, 1.520 s program
  construction, and 0.450 s binding; the inner CLI/reporting residual was
  0.780 s and the outer execution/export envelope 0.246 s
• warm P2 npm attribution put reify:unpack at 930 ms versus 93 ms on the
  host and observed a 1.388 s CommonJS graph envelope across 1,727 requires;
  ownership was broad, with no package reaching the 194 ms screening value
• the npm trace ranks packages directionally but does not separate startup
  from command-time lazy loads; npm timers do not separate archive processing
  from filesystem work

Temporary attribution code and raw reports were removed after their results
were summarized.

Rejected and deferred experiments

• missing CommonJS path classifications reduced probes but were reverted:
  invalidation local to one runtime cannot observe host or sibling-runtime
  mutations on a shared mount
• generic whitespace skipping in all source walkers improved profiled import
  by only about 1% and had no production end-to-end win; reverted
• skeleton/QuickJS opt-level changes, direct bytecode dispatch, SIMD, and
  assertion/dump-code variants either regressed npm/TypeScript, widened tails,
  or increased component size without a durable cross-workload win; reverted
• eager/lazy built-in bytecode variants either exceeded the memory gate or
  moved work into the measured path without an end-to-end win
• a source-stripped TypeScript bytecode cache was fast but violated observable
  source semantics; the source-preserving version exceeded the memory gate;
  the compact-span follow-up had no durable npm/cold-TypeScript benefit and
  GOL-663 was canceled without retaining a fork
• direct async-lstat error delivery worsened host-adjusted warm-ci median
  overhead by about 139 ms; reverted
• Binaryen wasm-opt -O3 produced real but modest improvements and smaller
  components, but remains deferred to GOL-661 because a supported
  cross-platform integration cannot depend on an undeclared system binary

Correctness boundaries

• cache scope never crosses a QuickJS runtime; failed realpaths remain
  retryable; public node:fs realpath is uncached; symlink-preserving modes
  bypass loader caches; Wizer initialization cannot populate them
• guest filesystem mutations invalidate graph-scoped missing-package metadata;
  no broad negative path cache remains
• source scanning still skips strings, templates, comments, and regex literals;
  batched cursor targets are ASCII UTF-8 boundaries and export-parser gates are
  accepted prefixes or safe supersets
• fixed-format classification preserves cached CommonJS TypeScript, .cts,
  .cjs, .mts, explicit package type, default-type node_modules, and
  force_module precedence; ambiguous sources still use syntax detection
• the TypeScript profiler imports typescript.js, not CLI _tsc.js; its
  import phases are attribution and not direct cold-CLI timings
• no public runtime API or Node compatibility inventory/runnable-case count
  changes

Evidence and validation

• retained npm and TypeScript reports pass schema, provenance, pair, success,
  recovery, memory, counter, and exact-currentness validation
• current-report manifests pin the production TypeScript pair to measured source
  968657ac and the production npm pair to measured source 831632c6; CI
  validates each pair from a pristine worktree at that exact revision while
  retaining older reports as historical evidence
• TypeScript release-report validation recomputes median, p95, throughput,
  incremental-seed values, and reused-instance memory high water from the raw
  samples; mutation tests cover both timing summaries and aggregate memory
• final runtime source 831632c6 passes focused loader cache regression
  coverage; final TypeScript loading source dd689c8c passed focused units,
  P2/P3 module-resolution and TypeScript-runtime slices, all ten skeleton
  Clippy lanes, formatting, and diff hygiene
• exact PR head a79a0763ed6c20ac40a3e190605200d24cd9a3ce
  passes exact-source TypeScript/npm report validation, selector regression
  tests, focused Clippy, formatting, diff hygiene, JSON invariant checks, and
  independent read-only review; complete GitHub CI run 36177926653 passed all
  40 jobs, and the tree contains no diagnostic runtime changes or project-owned
  Python files

The npm syscall reductions, stripped-ESM microbenchmark, TypeScript CLI rows,
and TypeScript API import attribution are separate claims. This PR does not
claim that arbitrary npm installs or TypeScript programs became 17–98% faster.