Repository navigation
feat(harness): a build-test-fix cycle for every component - #51
Merged
Merged
Conversation
Every component now owns setup/build/lint/test/fix/cycle in its own Makefile and the root Makefile is the umbrella. The interpreter is pinned to 3.12 and ruff to 0.17.1 inside every venv, so local and CI finally agree. - corvus-v2: the harness moves into corvus-v2/Makefile and gains the "fix" target the old "Build > Test > Fix" comment promised but never had. "build" is now a real proof (health + authenticated read + the 401 control) in scripts/dev_check.py. Deps floored at the tested versions, capped at the next major. - corvus-sdk: new harness; 54 tests (was 35), client.py and models.py at 100%. - corvus-cli: new harness; 50 tests from zero, main.py and config.py at 100%. - corvus-server / corvus-hooks: lint + import guards only. v1 is archived at v1-archive and not deployed; no tests are resurrected for it. - corvus-splunk: config validation (conf/xml/csv/py). - CI: a matrix over all six components, plus a phase-gates job for the mutation and conformance targets that BUILD-PLAN.md requires of CI but that CI had never run. The metrics drift guard now gates the test count. It previously compared only coverage percent (tolerance 7.0) and line totals, so the committed 108-tests/79.1% file passed against a 90-test/73.0% run. The tolerance is recalibrated to 1.0 now that the interpreter is pinned on both sides. Also: ruff --fix repaired 26 pre-existing violations in the archived corvus-hooks scripts. They are mechanical and behavior-preserving (IOError->OSError is the same object; Optional/Dict -> X | None under "from __future__ import annotations"), but they are a real diff on archived code, called out here so a reviewer sees them coming. docs/DEV-READINESS-REVIEW.md: the plan and vision against what the harness can prove today, with every claim instrumented.
…, not the stale one
…ed its own shell Two measured defects in the v2 conformance target, both found by running it in CI: 1. It cleaned up with pkill -f "engine_shim --port 8933". The recipe's own shell carries that string in its argv, so pkill matched the shell and killed it: the run passed 2/0 and make then reported "Terminated", exit 2 (CI 40058979560). It now kills the shim by PID. 2. It bound a fixed 8933. This host runs a launchd agent (com.themillertribe.corvus-v2-engine) already bound to 127.0.0.1:8933 from a different worktree, so a local "pass" was evidence about that process, not this code. The port is now ephemeral and a shim that fails to start fails loudly. Recorded both in docs/DEV-READINESS-REVIEW.md as blind spots.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Every Corvus component now has a real build > test > fix harness, and CI runs all of them.
The interpreter is pinned to 3.12 and ruff to 0.17.1 inside every venv, so local and CI
finally agree on what a green run means.
make cycleat the repo root is the one command. Per component:make -C <component> cycle.Per component
corvus-v2corvus-sdkcorvus-clicorvus-servercorvus-hookscorvus-splunkThe
fixtarget is proven, not just declared: inject a ruff-fixable violation,make fix,make lintpasses, and the file is byte-identical to HEAD afterwards.make fixon a cleantree is a no-op.
Two real bugs the harness surfaced
dev-metrics.jsonclaimed 108 tests / 79.1% / 1274 lines; a fresh 3.12 run is 90 tests /73.0% / 1277. The old guard compared only coverage percent (tolerance 7.0) and line totals,
so the 108-vs-90 test drift passed silently. It now gates the test count exactly, and the
tolerance is 1.0 because the interpreter is pinned on both sides.
BUILD-PLAN.mdrequires "the conformance suiteruns green in CI with the mutation evidence". At HEAD,
ci-v2.yamlran neither. There is nowa
phase-gatesjob that runs both.Please read before merging
ruff --fixrepaired 26 pre-existing violations in the archivedcorvus-hooksscripts(7 files, +16/−19). They are mechanical and behavior-preserving —
IOErrorisOSError(the same object), and the
Optional/Dict→X | Nonechanges are underfrom __future__ import annotationsso they never evaluate at runtime. But they are a diff onarchived v1 code, and they are in this PR because otherwise the new lint guard would ship red.
What the harness does not prove
docs/DEV-READINESS-REVIEW.mdmaps the plan and vision against the current state, with everyclaim instrumented. The sharpest findings:
corvus.engine_shim(Dockerfile:21),which serves one route. The SDK's 100% coverage and the CLI's 100% coverage are against
mocked HTTP: 27 of the SDK's 30
/opspaths and 12 of the CLI's 15 are not served by v2.Green here proves the client's own logic, not parity.
TrustLedgerhas no importer and nothing writesops_trust,so every live conflict answer carries the default
ESCALATE. Capability 3 (graded autonomy)is not on the deployed path.
emptyDir— decision records do not survive a pod restart.None of these are fixed here. They are named so the next goal starts from evidence.
Rollback
Revert the merge commit; the tree returns to the pre-PR state. No data is migrated or deleted.
Closes nothing — the follow-ups above are deliberately out of scope.