Repository navigation
Consolidate redundant verifier fixture preflight - #21
Closed
david-hoffman wants to merge 1 commit into
Closed
david-hoffman wants to merge 1 commit into
david-hoffman wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidate the invariant Black stand-in preflight from 75 launches to one per verifier test module. All 75 cases keep their own private
VerificationCommandfixture. No cases, assertions, numerical tolerances, deadlines, coverage settings, dependencies, workflows or verification gates change. The original 48-case rationale documents the same setup purpose; 27 cases were inherited from the newer base.The complete-suite 300-second goal remains unmet. Current warm median is 1,293.802 s, 993.802 s over the goal; cold is 1,311.454 s, 1,011.454 s over. The two warm pairs do not establish a consistent whole-suite speedup (candidate median 8.566 s / 0.667% slower). This PR removes demonstrated duplicate setup; it does not claim the overall performance objective is achieved.
Validation on exact unchanged candidate
445432eeb15298168cbadc5f26c8951677e84023, frozen integration baselinea835a490373835fa01e4460ce2159365150d1946:All six complete local runs exited 0: 1,831 passed, zero failures/errors/skips, 185 warnings. All 24 canonical stages passed, including quality, audit, fresh builds/installations, tests, subprocess coverage combination and final report validation. Receipt/input/log seals and retained artifact hashes were checked. Both trees and dependency inventories stayed unchanged.
All 19 owned Python runtime files individually have exact 100% statement/branch coverage and zero exclusions. Relevant subprocess/copied-helper data is combined and never-imported owned files remain in the gate. Shell-hook statement/branch coverage is unsupported by coverage.py; real local bare-Git hook tests remain included. Evidence is local macOS only. No remote runner or hosted verification evidence is used.
Measured command, launched in each isolated arm:
Hardware: Mac16,12, macOS 27.0 arm64, 10 physical/logical cores, 16 GiB RAM. Python 3.13.12. One pytest worker, no xdist workers;
BLACK_NUM_WORKERS=1,PYTEST_XDIST_AUTO_NUM_WORKERS=1, andOMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS,VECLIB_MAXIMUM_THREADS,NUMEXPR_NUM_THREADS,BLIS_NUM_THREADSall 1.PYTEST_ADDOPTS=--durations=30,PYTHONHASHSEED=0,MPLBACKEND=Agg. No CPU/worker/thread allocation increased. Runs were sequential in order candidate cold, baseline cold, baseline warm1, candidate warm1, baseline warm2, candidate warm2.External monotonic timing includes declared cold preparation before command launch and every canonical step through final exit. Cold clears separate arm XDG/pip/Matplotlib and checkout mypy/pytest/build/egg-info/bytecode caches inside the timer. Warm retains each arm caches. OS/shared platform caches are not claimed cold. The same locked outer runtime is a command precondition for both arms; required nested environments, installation checks and report validation stay inside the command. Independent post-run evidence archival is outside the canonical timer. Audit/install operations access network data; execution stays local.
Interpreter executable SHA256
4e999a1436fc1ff0c1d6a6ecc8e93566ecfc6706157de6232096238c420cf357; unchangedrequirements-dev.lockSHA2564ad780fe68daa3e413bf7156cd158768450c26c7c243013ceac304c33fd47eca. All locked installed versions matched before/after and across arms (except the freshly rebuilt distribution under test). Key versions: NumPy 2.5.3, SciPy 1.18.1, pandas 3.0.6, Matplotlib 3.11.2, scikit-image 0.26.0, pytest 9.1.1, coverage 7.16.1, Black 26.5.1, pip 26.2.1, mypy 2.3.1.Measured bottlenecks: candidate cold tests 1,280.289 s (97.62% of total); release workflow module 483.593 s, release smoke 212.428 s, reuse 148.419 s, verifier 109.319 s. Five installation-retargeting cases (
PYTHONPATH,PYTHONHOME,PIP_TARGET,PIP_PREFIX,PIP_USER) alone total 399.890 s. Their distinct behaviors and real clean installations are retained. Verifier module times were 111.816/112.067/112.442 s baseline versus 109.319/107.301/106.758 s candidate; this smaller metric does not substitute for the unmet full-command target.All earlier full attempts are disclosed:
Historical baseline
e557c7d5822be5d8413d039454782eb7772c9074, candidate/reviewed checkpointd66ea14e0e402f137aba6dba7aeb71232396d43c, command/private/tmp/dphtools-verification-env/bin/python tools/verification.py full: all nine old passes had 1,232 tests, 2,094/2,094 statements and 544/544 branches. Their vanished temporary raw evidence cannot be independently reread and is not reused for current verification. Exact executable paths for the first two historical failed integration attempts are unavailable in retained raw data; both ran canonicaltools/verification.py full. A historical independent reviewer also failed ENOSPC and supplied no acceptance.Environment failures were independently classified as tooling. Primary Miniconda remains intact; a private base copies identical Python/stdlib/native-library bytes and installs the unchanged lock directly so nested environments inherit locked tools. Fresh preflight and all 59 affected reuse tests passed without candidate/test changes. The external archival helper initially failed on missing reports after the restricted audit attempt; it was corrected to retain partial failed receipts before the matched plan froze. No verifier gate changed.
Current full measurements: 7,825.023 s; all 19 full attempts: 11,015.148 s, 15 pass/4 fail. The original two-scenario specialist slice had B acceptance at round 1 and zero C repairs; all 35 nonfixture syntax trees on the current base are preserved. Fresh independent root reviewer accepted this exact candidate with no blocking defects; no edits or repairs. Session
01a11583-fffb-7ca0-997d-c6899f3e0fea, external review wall time 385.481 s of the 900 s budget. It directly validated all six seals, reports, raw coverage and all 890 runtime-copy file hashes. Performance benefit remains unproven. No merge/release is requested.