Skip to content

feat: add pr iteration loop workflow - #1821

Merged
groupthinking merged 5 commits into
mainfrom
copilot/add-agentic-workflow
Sep 9, 2026
Merged

feat: add pr iteration loop workflow#1821
groupthinking merged 5 commits into
mainfrom
copilot/add-agentic-workflow

Conversation

Copilot AI commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Canonical issue

Link to the canonical issue in the PR metadata.

Outcome

Adds a repository-specific pr-iteration-loop agentic workflow for Copilot: it selects one bounded repo problem per run, iterates on one long-running draft PR, persists cross-run memory, and grades whether the run delivered operational value for EventRelay.

Scope

  • Included:
    • New workflow.github/workflows/pr-iteration-loop.md with issue/PR/main/scheduled triggers, repo-scoped prompt, cache-memory, Playwright CLI, agentic-workflows tooling, LSP config, chart uploads, and bounded loop rules.
    • Evaluations — operational-value grader plus binary checks for single-checkpoint selection, verification evidence, memory updates, and visualization output.
    • Deterministic selection — pre-step seed that prioritizes recent failing workflows, then week-stale PRs/issues.
    • gh-aw alignment — compiled pr-iteration-loop.lock.yml, added the workflow to validation, and updated the repo’s gh-aw pin/lock references to v0.88.7.
  • Explicitly excluded:
    • Product/runtime behavior changes outside workflow automation.
    • New deployment, auth, or infrastructure policy.
evals:
  - id: operational_value
    question: Does the agent output show that this run delivered an evidence-backed recommendation or accepted iteration proving which of Chopin, Continuous AI, Autoloop, or Agentic Workflows most helps one high-value EventRelay problem on a single long-running draft pull request?

Risk

  • Risk level: medium
  • Failure mode:
    • gh-aw schema/runtime drift can change compiled lock output or workflow behavior.
    • lsp and merge-pull-request remain experimental surfaces.
  • Rollback:
    • Revert pr-iteration-loop files and the gh-aw pin/validation updates.

Verification

List exact automated and manual checks, tied to the current head SHA.

  • Focused tests
    • gh aw version
    • gh aw compile canonical-pr-remediator focused-coverage-controller pr-iteration-loop repo-assist --validate --approve
    • gh aw compile canonical-pr-remediator focused-coverage-controller pr-iteration-loop repo-assist --actionlint --zizmor --poutine --approve
    • git diff --check
    • changed-file secret scan on the final diff
  • Required CI
    • gh-aw compile + actionlint succeeded locally on the generated lock files
  • Review threads resolved

Production evidence

Not applicable; this PR only adds repository automation workflow definitions and compiled lockfiles.

Agent handoff

  • One canonical issue is linked
  • No competing PR implements the same issue
  • Acceptance criteria are satisfied
  • Required checks pass on the current head
  • Human decision is requested only for product, security, irreversible infrastructure, or production approval

Copilot AI and others added 3 commits September 9, 2026 09:14
Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>
Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>
Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
v0-uvai Ready Ready Preview, v0 Sep 9, 2026 4:26pm UTC

@github-actions github-actions Bot added ci/cd documentation Improvements or additions to documentation labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

🔍 PR Validation

⚠️ Large PR detected (5981 lines changed)

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 41aa783.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

OpenSSF Scorecard

PackageVersionScoreDetails
actions/actions/checkout 3d3c42e5aac5ba805825da76410c181273ba90b1 🟢 7
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Maintained🟢 1022 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Binary-Artifacts🟢 10no binaries found in the repo
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Packaging⚠️ -1packaging workflow not detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
SAST🟢 10SAST tool is run on all commits
Branch-Protection🟢 6branch protection is not maximal on development and all release branches
actions/github/gh-aw-actions/setup 5e508589e03a7757a7e05b26e834292f5445bfb6 UnknownUnknown

Scanned Files

  • .github/workflows/repo-assist.lock.yml

@groupthinking
groupthinking marked this pull request as ready for review September 9, 2026 15:49
Copilot AI balanced review requested due to automatic review settings September 9, 2026 15:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The selection logic violates its documented priority, the version bump breaks an existing governance test, and Poutine-only changes bypass validation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds an automated PR iteration workflow that selects repository work, maintains cross-run memory, verifies outcomes, and updates gh-aw tooling.

Changes:

  • Adds the PR iteration workflow and compiled lockfile.
  • Upgrades gh-aw to v0.88.7 and regenerates workflow output.
  • Updates scanner configuration and workflow documentation.
File summaries
File Description
.github/workflows/pr-iteration-loop.md Defines triggers, selection, tools, safe outputs, and reporting rules.
.github/workflows/pr-iteration-loop.lock.yml Compiled executable workflow.
.github/workflows/focused-coverage-controller.lock.yml Regenerates the controller with gh-aw v0.88.7.
.github/workflows/gh-aw-validation.yml Pins and validates gh-aw v0.88.7 workflows.
.github/aw/actions-lock.json Pins matching gh-aw action commits.
.poutine.yml Allows GitHub’s ubuntu-slim runner.
.github/workflows/README.md Documents the new workflow and version.
.github/workflows/AUDIT.md Updates the validation audit entry.
docs/AGENT_CAPABILITIES_CHECKLIST.md Updates the documented gh-aw version.
Review details
  • Files reviewed: 7/11 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +8 to +11
"github/gh-aw-actions/setup@v0.88.7": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.82.14",
"sha": "b6d1443e05b8716267fa19425b99aa4f12006b4a"
"version": "v0.88.7",
"sha": "5e508589e03a7757a7e05b26e834292f5445bfb6"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41aa783. Updated governance expectations to gh-aw v0.88.7 (keys, SHAs, install pin, and workflow list) so the test now matches the lock/runtime configuration.

generated_at: new Date().toISOString(),
};

if (context.eventName === "issues") {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41aa783. Selection now always computes ranked candidates and picks by the documented priority order; issue/PR trigger payloads are retained as context only.

Comment thread .poutine.yml
@@ -0,0 +1,8 @@
# Configure poutine security scanner

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41aa783. Added .poutine.yml to both push and pull_request paths filters in gh-aw-validation.yml, with a regression test.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

CI/Coverage failure diagnosed and fix ready (blocked by branch history, not this change)

Root cause: a958a97c1 ("chore: update gh-aw pin for copilot workflow support") bumped the gh-aw runtime pin to v0.88.7 in .github/workflows/gh-aw-validation.yml and .github/aw/actions-lock.json, but left tests/unit/test_gh_aw_workflow_governance.py::test_gh_aw_validation_pins_runtime_version asserting the old v0.82.14 entry/sha/pin string. This fails both CI (run 34334090490) and Coverage (run 34334090588) with:

KeyError: 'github/gh-aw-actions/setup@v0.82.14'

Verified fix (passes locally, PYTHONPATH=src pytest tests/unit/test_gh_aw_workflow_governance.py -v --no-cov → 9 passed, ruff check clean):

diff --git a/tests/unit/test_gh_aw_workflow_governance.py b/tests/unit/test_gh_aw_workflow_governance.py
index a548147d6..fa58ef6a3 100644
--- a/tests/unit/test_gh_aw_workflow_governance.py
+++ b/tests/unit/test_gh_aw_workflow_governance.py
@@ -164,11 +164,11 @@ def test_gh_aw_validation_pins_runtime_version() -> None:
     actions_lock = json.loads((ROOT / ".github/aw/actions-lock.json").read_text())
 
     assert workflow["name"] == "gh-aw Validation"
-    entry = actions_lock["entries"]["github/gh-aw-actions/setup@v0.82.14"]
-    assert entry["sha"] == "b6d1443e05b8716267fa19425b99aa4f12006b4a"
+    entry = actions_lock["entries"]["github/gh-aw-actions/setup@v0.88.7"]
+    assert entry["sha"] == "5e508589e03a7757a7e05b26e834292f5445bfb6"
     step_scripts = [step.get("run", "") for step in workflow["jobs"]["validate-gh-aw"]["steps"]]
     combined = "\n".join(step_scripts)
-    assert "gh extension install github/gh-aw --pin v0.82.14" in combined
+    assert "gh extension install github/gh-aw --pin v0.88.7" in combined
     assert "eventrelay-ci-investigator" not in combined
     assert "canonical-pr-remediator" in combined
     assert "focused-coverage-controller" in combined

Why I couldn't push it directly: push_to_pull_request_branch reported this branch's history contains .poutine.yml (added in a958a97c1, already merged into this branch before my session started), which falls outside this workflow's allowed-files list. That block is pre-existing and unrelated to this fix — it will affect any future stacked commit until resolved by a maintainer (e.g. widening the allowed-files config or landing .poutine.yml through a normal PR path).

Please apply the patch above directly (e.g. git apply or a manual edit) to unblock CI and Coverage on this PR.

Generated by pr-iteration-loop for #1821 · copilot · auto · 95.2 AIC · ⌖ 8.03 AIC · ⊞ 12.1K ·

@vercel vercel Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

.poutine.yml is missing from the push/pull_request paths: filters, so poutine-policy-only changes never trigger the validation workflow that runs the poutine check

Fix on Vercel

Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>

Copilot AI commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

🟡 Changes recommended

The selection logic violates its documented priority, the version bump breaks an existing governance test, and Poutine-only changes bypass validation....

Addressed in 41aa783. All requested fixes from this review thread are now applied and validated with targeted governance tests plus parallel validation.

@groupthinking
groupthinking merged commit 82f2612 into main Sep 9, 2026
27 checks passed
@groupthinking
groupthinking deleted the copilot/add-agentic-workflow branch September 9, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants