Skip to content

docs: the Phase 4 handoff - #187

Merged
craig-dt merged 1 commit into
mainfrom
docs/handoff-phase-4
Sep 1, 2026
Merged

craig-dt merged 1 commit into
mainfrom
docs/handoff-phase-4

Conversation

@craig-dt

@craig-dt craig-dt commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Docs only — a handoff for the next session, plus the tracker.

docs/RESUME-phase-4.md records where Phase 4 stands: one step from complete, with bare blfile
emitting 880 flows across 24 captures where it emitted zero on 2026-08-27. Only P4-6 remains, and
it is deliberately off the critical path.

It points a new session at the two files to read in order — the RESUME for state, and
docs/corpus-2026-08-28.md before the corpus is used as ground truth, because the numbers alone
invite over-trust: one capture is 41% of it, 52% of the flows are port 22, 20% carry only
indicator-reference sources, and 462 source entries are CC-BY-SA-4.0 share-alike while blfile
writes no NOTICE.

What it says about the review process

Craig asked "has this had a fresh-eyes eng review?" three times this phase — #177, #180, #182 — and
all three times the answer was no. Same shape each time: a review ran, produced substantial new
content, and that content shipped unreviewed. The gap is not the first review; it is everything the
first review causes you to write.

And rounds 2 and 3 repeatedly found the code sound and the justifications false — a fabricated
measurement, "build already refuses elsewhere", "coverage gains fields as loss conditions are
added", "a digest so the next added field cannot repeat this". Each a confident sentence that one
command contradicted.

Also

Closes #144 on real rows: 8 flows composed from two different runs, with the join hazard excluded (0
cases of one 5-tuple producing two flow_keys). Noted on closing that 8 of 880 is thin.

Named as the first things to do next: P4-6; #183, the scheduled false-positive review that
failed on 2026-08-30 and nobody has looked at; and PRD Q11 — whether these 24 captures are
publicly-published or internal traffic, which is a gate the PRD names and nobody has answered.

Verified: 1,823 tests pass (the one failure is the decided Zeek pin). YAML parses; log/blocked_on/
known_gaps at 336/2/26.

🤖 Generated with Claude Code

Docs only, on `RESUME-ls-9.md`'s precedent. Records where Phase 4 stands — one step from complete,
with the store finally doing what it was built for — and points a new session at the two files it
must read in order: this one for state, and `corpus-2026-08-28.md` before treating the corpus as
ground truth.

The transferable half is the review process. Craig asked "has this had a fresh-eyes eng review?"
three times this phase, on #177, #180 and #182, and all three times the honest answer was no. The gap
had the same shape every time: a review ran, produced substantial new content, and that content
shipped unreviewed. `CLAUDE.md` already says re-cut and re-review after acting; Phase 4 is the
evidence for why, and the lesson is that the gap is not the first review — it is everything the first
review causes you to write.

Sharper still: rounds 2 and 3 repeatedly found the code sound and the *justifications* false. A
fabricated measurement pasted into a block captioned "one real X"; "`build` already refuses
elsewhere"; "`coverage` gains fields as loss conditions are added"; "a digest so the next added field
cannot repeat this". Each was a confident sentence that one command contradicted. So: write the
sentence, then run the thing that would falsify it.

Also closes #144 on real rows — 8 flows composed from two different runs, with the join hazard
excluded — and records that 8 of 880 is thin enough that a richer overlap wants its own issue.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@craig-dt
craig-dt merged commit 8cca5af into main Sep 1, 2026
8 checks passed
@craig-dt
craig-dt deleted the docs/handoff-phase-4 branch September 1, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-detection flows have never occurred in real data, so best_tier and threat-name selection are tested but unexercised

1 participant