Repository navigation
feat: integrate Matt Pocock engineering skills into worker brief contract - #1
Merged
Merged
Conversation
…serial 3': tests/fm-pi-branch-extension.test.sh case 'Pi outcomes rendering consumers must preserve stock behavior' (Calm-off ToolExecutionComponent rendering differs from Pi stock). Root cause: upstream version drift, not this PR's diff — CI does npm install -g @earendil-works/pi-coding-agent (latest), which now resolves to 1.0.2, and since Pi 0.99.0 the stock call-fallback renders the tool's args beside the bold title (collapsed muted preview, e.g. 'fm_branch_outcomes recent=2') or as an indented per-arg block when expanded. The tracked extension renderers (fm_branch_outcomes, fm_branch_processed) in .pi/extensions/fm-branch-supervision.ts still emitted only the bare bold title, so calm-off rendering diverged from stock and the stock-comparison case failed. Reproduced locally with an isolated npm prefix of pi-coding-agent@1.0.2 (FM_PI_PACKAGE_DIR): the case failed before the fix with the exact CI error. Fix (smallest root-cause change, matching the file's existing version-floor precedent): (1) added a stockToolCallComponent helper in the extension that byte-mirrors Pi 0.99.0+'s formatToolCallWithArgs fallback and made both tools' renderCall use it; (2) raised the test's PI_STOCK_RENDER_FLOOR from 0.84.4 to 0.99.0 so the comparison skips (named version skip) on hosts with a Pi that predates the args-in-call contract, per the existing version-skew pattern. Verified: bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh now exits 0 against Pi 1.0.2 with the previously failing case passing; bin/fm-test-run.sh tests/fm-pi-primary-types.test.sh passes strict no-emit typecheck against Pi 1.0.2; bin/fm-lint.sh passes with the pinned ShellCheck 0.11.0 and actionlint 1.7.12. Intentionally not attempted: pinning the Pi version in CI — the repo's established contract is that the extension tracks Pi's current stock rendering (runtime probes + named version floors), so updating the renderers and the floor is the root-cause fix consistent with existing behavior
…table serial 1': tests/fm-calm-pi-extension.test.sh 'Pi calm renderer and lifecycle contract' ('grep disappeared from /export calm.html HTML while calm mode was on'). Root cause: upstream Pi version drift, not this PR's diff — CI installs @earendil-works/pi-coding-agent latest (1.0.2), and Pi 1.0.1 renamed the createToolHtmlRenderer lookup dependency from getToolDefinition to getToolRenderers (agent-session's exportToHtml now resolves renderers via the extension runner). The fixtures still passed getToolDefinition, so on Pi >= 1.0.1 the renderer resolved no definition and renderCall returned undefined. Reproduced locally with an isolated npm prefix of pi-coding-agent@1.0.2 (FM_PI_PACKAGE_DIR): the case failed before the fix with the exact CI error. Fix (smallest root-cause change): pass both lookup spellings in the affected fixtures — older Pi destructures getToolDefinition and ignores the extra key, newer Pi uses getToolRenderers, with identical lookup semantics, so no version check is needed. Two files: tests/fm-calm-pi-extension.test.sh (new exportToolHtmlDeps helper used by both createToolHtmlRenderer sites in the renderer/lifecycle case) and tests/fm-pi-branch-extension.test.sh (the structured-fallback export assertion, which under Pi >= 1.0.1 was passing only vacuously from the unrecognized key — adding getToolRenderers restores real exercise of the tool's throw-to-fallback contract). Verified: bin/fm-test-run.sh tests/fm-calm-pi-extension.test.sh exits 0 on both Pi 1.0.2 and the locally installed Pi 0.85.1; bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh exits 0 on Pi 1.0.2; bin/fm-lint.sh passes with pinned ShellCheck 0.11.0 and actionlint 1.7.12. Only test fixtures changed; no production code consumes createToolHtmlRenderer
…isible Since Pi 1.0's fullscreen TUI, restored transcript rows live in Pi's virtualized scrollback, so a 44-row tmux capture no longer sees the E2E-appended rows (CI failure: missing CALM_E2E_OUTPUT on Pi 1.0.2). Raise the session/resize height to 400 at the three affected sites so the viewport holds the whole restored transcript plus appended rows.
…hook rows CI Pi 1.0.3 failed the 'rendered export DOM violated the Calm conversation boundary' assertion because newer Pi exports display:false custom messages as provenance rows with class 'hook-message hook-message-hidden', kept out of the rendered conversation by the export's stock CSS unless the body carries show-hidden-messages; Pi <= 1.0.2 omits those rows entirely. The old flat includes() assertions matched the hidden provenance row's label text in the messages region and failed. Scope the assertions to the visible conversation: every hook row must carry hook-message-hidden, the body must not activate show-hidden-messages, and the [firstmate-synthetic-input] label must not survive once hidden hook rows are stripped. Genuine user/assistant entries, the five current operational user-role inputs, and tree provenance assertions are unchanged, so the Calm boundary semantics are preserved and older Pi (no hidden rows emitted) still passes. Verified headlessly against the prior worker's existing /tmp/fm-dom-repro Pi 1.0.3 rendered DOM (now passes; it previously failed exactly on the synthetic-label predicate), plus fixtures: old-Pi markup without hidden rows passes, a visible hook row fails, and an active show-hidden-messages body fails. bash -n and bin/fm-lint.sh (pinned ShellCheck 0.11.0, actionlint 1.7.12) pass. The tmux/Chrome E2E verdict belongs to GitHub's exact check.
…rdr calls and daemon join tests/fm-remote-job.test.sh's 'worker shutdown terminates the active command tree' case polled worker exit with a fixed 100x0.05 fork-count loop; on a contended CI serial shard the supervisor-mediated shutdown (quarantine, KILL escalation, owner reap) outlasted that budget and the case failed with 'the worker did not finish its TERM shutdown'. Poll on a wall-clock 30s deadline instead - the same bound this file already uses for the repeatedly signalled worker - while keeping the exact assertion: a TERM'd worker must finish its shutdown. The Behavior tests (Herdr) lane timed out blind after 20 minutes inside tests/fm-afk-inject-herdr-e2e.test.sh (CI run 37309436481). The failed assertion was never printed: every herdr CLI call made by the test, its lab provisioning/teardown, and the supervise daemon was unbounded, so a wedged herdr server socket (or the daemon blocked inside such a call, outliving TERM) hung the script with no per-command evidence. Fix within the affected test: (1) the existing herdr shim now wraps every call in bin/fm-timeout-lib.sh's fm_run_timed at a 30s bound, naming the exceeded bound on stderr and returning the shared 124 contract; the long-lived `server` launch is exempt since bounding it would kill the lab server under test; the shim is created up front and put first on PATH, so lab provisioning, backend adapter operations, the daemon's env, the supervisor pane's own herdr calls, and teardown all sit under the same bound; (2) the daemon join in stop_daemon and cleanup escalates TERM -> 30s -> KILL and stop_daemon fails with a named timeout instead of waiting without bound. The previously proposed ci.yml per-script flag is replaced by this in-test bounded behavior (the workflow credential cannot push workflow edits). Verified: bin/fm-test-run.sh tests/fm-afk-inject-herdr-e2e.test.sh exits 0 against real herdr 0.8.0 with all four scenarios passing (68s); the rendered shim against a never-answering fake herdr exits 124 in 2s with the named bound message; bin/fm-test-run.sh tests/fm-remote-job.test.sh and tests/fm-ci-workflow.test.sh exit 0; bash -n passes; bin/fm-lint.sh passes with pinned ShellCheck 0.11.0 and actionlint 1.7.12.
mattleus-cohere
approved these changes
Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Integrate Matt Pocock's installed skill pack into FirstMate so planning and workers use stronger engineering methods while FirstMate retains orchestration, isolation, supervision, communication, delivery, and merge authority. Add an internal integration skill and routing rules, generate and validate task-specific required engineering skills and agreed public testing seams for ship and scout workers, preserve older saved instructions for recovery, carry the contract through scout promotion, document installation and architecture, and add behavioral tests. Prevent workers from launching nested orchestration or duplicate delivery and review flows: no-mistakes remains the sole final review flow when selected, while project issues hold planning artifacts and FirstMate's backlog tracks execution. Do not configure individual projects yet. Follow FirstMate's standing validation policy: keep the Test step intent-targeted, use bin/fm-test-run.sh for selected suites, and do not configure or run an unrelated full-suite test command.
What Changed
matt-pocock-integrationskill that routes upstream Matt Pocock workflows (grill-me, wayfinder, research, implement, code-review, etc.) into FirstMate scouts, ships, and planning artifacts while keeping orchestration, isolation, supervision, delivery, and merge authority with FirstMate and blocking nested orchestration or duplicate final-review paths.bin/fm-brief.shnow scaffolds a# Engineering methodsection with task-specific## Required skillsand## Agreed testing seamssubsections in ship and scout briefs; newbin/fm-dod-lib.shhelpers letbin/fm-spawn.shandbin/fm-promote.shreject leftover{REQUIRED_SKILLS}/{TESTING_SEAMS}placeholders or empty subsections, while briefs predating the section remain launchable for recovery and promotion carries the validated method to the live worker and replacement instructions.docs/matt-pocock-skills.mdplusdocs/architecture.md,AGENTS.md, andREADME.mdupdates, and added behavioral test coverage for the new brief contract, spawn/promotion validation, and fixtures.Risk Assessment
✅ Low: The change adds a well-bounded scaffold section and validation gate that reuse the existing exact-match heading-parser contract, refuse before any state mutation, preserve legacy-brief launchability, and are covered by behavioral spawn/promote tests, with documentation and intent conformance all verified against source.
Testing
Ran the intent-targeted suites through bin/fm-test-run.sh (fm-brief, fm-task-delivery, fm-documentation-audiences — all exit 0, zero gate skips) and drove five manual scenarios against the real fm-brief.sh/fm-spawn.sh/fm-promote.sh CLIs in isolated FM_HOME fixtures: 19/19 checks passed covering the new Engineering method contract in scaffolds, adversarial refusal of unfilled and emptied skill/seam subsections at both spawn and promotion with no state written, carry-through of the filled contract into promoted worker instructions, and recovery compatibility for pre-contract briefs. The only untested aspect is live-LLM compliance with the prompt-level orchestration guard, which the repo's own test policy assigns to development-only evaluation.
bin/fm-test-run.sh tests/fm-brief.test.sh(test_ship_modes_generate_clean_briefs, test_scout_and_secondmate_scaffold), exit 0bin/fm-test-run.sh tests/fm-documentation-audiences.test.sh, exit 0 (real consumer of docs/documentation-audiences.json validates the new entries and links)Evidence: Manual drive script and its 19-check pass transcript source
Source: Manual drive script and its 19-check pass transcript source
Evidence: Generated ship brief carrying the Engineering method contract (real fm-brief.sh output)
Source: Generated ship brief carrying the Engineering method contract (real fm-brief.sh output)
Evidence: Generated scout brief carrying the Engineering method contract (real fm-brief.sh output)
Source: Generated scout brief carrying the Engineering method contract (real fm-brief.sh output)
Evidence: Legacy-brief spawn output proving validation passed with no engineering refusal
Source: Legacy-brief spawn output proving validation passed with no engineering refusal
Evidence: fm-promote refusal transcript naming the unfilled engineering placeholders and subsections
Source: fm-promote refusal transcript naming the unfilled engineering placeholders and subsections
Evidence: fm-spawn refusal transcript for emptied engineering subsection bodies
Source: fm-spawn refusal transcript for emptied engineering subsection bodies
Evidence: Ship instructions published by promoting a legacy (pre-contract) scout brief
Source: Ship instructions published by promoting a legacy (pre-contract) scout brief
Evidence: Legacy scout promotion transcript
Source: Legacy scout promotion transcript
Evidence: Targeted suite result via bin/fm-test-run.sh
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
bin/fm-test-run.sh tests/fm-brief.test.sh(test_ship_modes_generate_clean_briefs, test_scout_and_secondmate_scaffold), exit 0bin/fm-test-run.sh tests/fm-documentation-audiences.test.sh, exit 0 (real consumer of docs/documentation-audiences.json validates the new entries and links)bash bin/fm-test-run.sh tests/fm-brief.test.sh tests/fm-task-delivery.test.sh tests/fm-documentation-audiences.test.sh(selected suites per the intent's validation policy; FM_TEST_SUMMARY total=3 failed=0)Manual drivebash manual-drive.shS1: real fm-brief.sh scaffolded ship, scout, and secondmate-charter briefs in an isolated FM_HOME and the generated files were asserted for the Engineering method contract and guard wordingManual drive S2: real fm-spawn.sh against a legacy brief with no # Engineering method section confirmed recovery compatibility (reached post-validation flow, no error, no refusal)Manual drive S3 (adversarial): real fm-promote.sh against a filled scout brief with leftover {REQUIRED_SKILLS}/{TESTING_SEAMS} placeholders confirmed refusal, named subsections, and no published ship instructionsManual drive S4 (adversarial): real fm-spawn.sh against a brief with emptied engineering subsection bodies confirmed refusal with the nonempty-subsection error and no task metadata writtenManual drive S5: real fm-promote.sh against a legacy scout brief with no Engineering method section confirmed ship instructions are published with the selected delivery contract✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.