Skip to content

[P3] Feed repo lessons from pr-fix/tombstone outcomes back into AGENTS.md automatically #754

Description

@joryirving

Idea (stolen from agent-flywheel.com's "CM" memory concept)

The loop already generates repo-specific lessons; it just forgets them. Today they get preserved only when a human notices and hand-writes them somewhere:

  • windowstead's assert_eq arity trap (a 2-arg call is a parse error that silently drops the whole test file) — hand-written into coder-godot's systemPrompt after fix(deps): update dependency effect (3.21.2 → 3.21.3) #321
  • pinchflat's sqlean-in-test-env and /app path gotchas — hand-written into its AGENTS.md

Every repo already has (or can have) an AGENTS.md that coders and reviewers read. The flywheel version closes the loop automatically:

When a pr-fix item resolves or a workload exhausts its attempts, extract the repo-specific lesson and propose it as an AGENTS.md addition — via a normal PR, not a direct write.

Sketch

  1. Trigger: a PrFixQueueItem reaching FIXED after ≥2 attempts, or a workload ending BLOCKED/tombstoned. These are exactly the cases where something non-obvious about the repo burned attempts.
  2. Input: the item's accumulated feedback[] + the final fix diff — both already stored.
  3. A small model call (the groomer's json_schema pattern) produces either no_lesson or a 1-3 sentence, checkable gotcha ("X takes 3 args; omitting the name is a parse error that drops the file").
  4. Open a docs PR appending it to the repo's AGENTS.md under a ## Learned by the loop section. Human merges — the human-accountability gate stays.
  5. Dedupe against the existing file (embed-or-grep) so the same lesson isn't proposed twice.

Why it compounds

A lesson costs the fleet 2-3 burned attempts to learn and is then re-learned per repo, per quarter, forever. Written into AGENTS.md, every future coder AND the AI reviewer see it on every task in that repo — the compounding is the point. The two hand-written examples above have already prevented recurrences (windowstead#321's exact failure shape is now in the coder-godot prompt).

Guardrails

  • PRs only, never direct pushes; Refs, never Fixes.
  • Cap the section length; oldest lessons rotate out or get folded into the repo's standards proper.
  • no_lesson must be the common case — a lesson-per-fix would drown the signal (same bar as the groomer's binary ready/backlog: don't make a small model perform insight, let it decline).

Files: src/lib/pr-fix-queue.ts (where an item reaches FIXED and the trigger fires), src/lib/groomer/llm.ts (the json_schema pattern this reuses).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions