Skip to content

perf(tests): premerge-findings fixture is ~96% interpreter startup #784

Open

Description

@TheFJK

Surfaced by the Efficiency lens on #768 and deliberately not applied in #781, because it is not a drop-in swap.

One run of tests/premerge-findings.test.sh starts 374 python3 processes (counted with a PATH shim):

  • 225 execute the module under test. That is the price of driving the shipped CLI with real argv and real exit codes, and it is not waste.
  • 149 are scaffolding: building a small input document, writing a blocked-on artifact, reading one field out of a counts object, rendering a ledger. write_input alone is 81 call sites, write_blocked 11, counts_field 12.

Measured on an M-series host: the fixture takes 12.55 s wall. A bare interpreter start is ~19 ms (250 starts = 4.79 s) and a start that also imports the module is ~41 ms (20 starts = 0.83 s), so 225 × 41 ms + 149 × 19 ms ≈ 12.1 s — the fixture is roughly 96% interpreter startup, and the scaffolding half is ~2.8 s, or 22%.

#768 amplified an existing cost rather than creating it: its additions contribute 28 new write_input calls and 73 new driver calls, and per-section timing puts the added B32/B33 blocks at 3.89 s of the 12.55 s.

jq is already a hard dependency of this file — it exits FATAL at the top when jq is absent — and starts in ~3.4 ms (250 invocations = 0.85 s). Routing the scaffolding through it would take that 2.8 s to roughly 0.5 s.

Why this is not a mechanical swap

Each helper needs its own equivalence check:

  • b32_growths asserts against Python repr spellings that jq renders differently.
  • counts_field's missing-key sentinel needs a jq form that still separates a null value from an absent key.

Do this helper by helper, verifying every assertion string each one feeds. A blanket swap will turn assertions green against different text.

Do not extend it to the driver calls

The 225 module executions are the fixture's whole value — they drive the shipped CLI rather than a transcription of it. Replacing any of those with an in-process import would re-introduce exactly the replica-test class this suite exists to refuse.

Related: PR #768, PR #781

Activity

  1. added
    premerge-findingAuto-filed by /uberdev:premerge from deferred findings (RFC 0002)
    on Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    premerge-findingAuto-filed by /uberdev:premerge from deferred findings (RFC 0002)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions