Skip to content

Add PR Agent review, describe, and improve workflow on CI #367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
PeterDaveHello wants to merge 1 commit into
base: master
Choose a base branch
from
+93 −0
Open

Add PR Agent review, describe, and improve workflow on CI #367

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .github/workflows/pr-agent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: PR Agent
Comment thread
PeterDaveHello marked this conversation as resolved.

on:
pull_request:
branches: ["master"]
types: [opened, reopened, ready_for_review, review_requested, synchronize]
Comment thread
PeterDaveHello marked this conversation as resolved.
issue_comment:
types: [created]

concurrency:
# PR events group by PR so newer pushes replace stale auto runs.
# issue_comment events use run_id here; manual commands are grouped below.
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
Comment thread
PeterDaveHello marked this conversation as resolved.

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}

@AndrewSouthpaw AndrewSouthpaw Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is the OpenAI key coming from? Whose account? This seems like a blocker.

@PeterDaveHello PeterDaveHello Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it's mine, that's the reason why I want to keep it using a more affordable model.

config.model: "gpt-5.4-mini"
config.reasoning_effort: "xhigh"
config.ai_timeout: "600"
pr_reviewer.final_update_message: "false"
pr_code_suggestions.commitable_code_suggestions: "true"
github_action_config.pr_actions: '["opened", "reopened", "ready_for_review", "review_requested", "synchronize"]'

jobs:
auto:
if: >-
github.event_name == 'pull_request' &&
github.event.sender.type != 'Bot' &&
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository
Comment on lines +29 to +32

@github-actions github-actions Bot Jun 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Avoid blanket-excluding all bot-authored PR events here. As written, Dependabot, Renovate, and other automation will never get auto review, so the CI workflow silently misses a big class of PRs. If the goal is only to avoid self-triggering, narrow this check to the PR Agent bot account instead of filtering every bot. [general, importance: 4]

Suggested change
github.event_name == 'pull_request' &&
github.event.sender.type != 'Bot' &&
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository
github.event_name == 'pull_request' &&
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository

runs-on: ubuntu-latest
permissions: &pr-agent-permissions
contents: read
issues: write
pull-requests: write
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- tool: describe
auto_describe: "true"
auto_review: "false"
auto_improve: "false"
- tool: review
auto_describe: "false"
auto_review: "true"
auto_improve: "false"
- tool: improve
auto_describe: "false"
auto_review: "false"
auto_improve: "true"

Comment on lines +41 to +55

@AndrewSouthpaw AndrewSouthpaw Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this matrix seems like unnecessary overhead, just run them all together in one?

@PeterDaveHello PeterDaveHello Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of doing it this way is to trigger different tasks in parallel, effectively speeding up the entire workflow. This ensures that both contributors and reviewers don't have to spend too much time waiting for automated feedback.

While the implementation will certainly be slightly more complex, I believe it's worth considering given the value of the time saved waiting.

@AndrewSouthpaw AndrewSouthpaw Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear that, and also my experience is that I can give AI a prompt like "look over the PR description, code, and ways to improve the code" and it'll give an answer equivalently fast. I don't think we're saving a worthwhile amount of time. Do you have numbers on the amount of time saved?

steps:
- name: Run PR Agent ${{ matrix.tool }}
uses: &pr-agent-image docker://pragent/pr-agent:0.38.0-github_action@sha256:ea2ea90f072fd97708755e59827a317272f66097a1ef349eca23d39160bb0baf
env:
github_action_config.auto_review: ${{ matrix.auto_review }}
github_action_config.auto_describe: ${{ matrix.auto_describe }}
github_action_config.auto_improve: ${{ matrix.auto_improve }}

manual:
if: >-
github.event_name == 'issue_comment' &&
github.event.sender.type != 'Bot' &&
github.event.issue.pull_request != null &&
github.event.issue.state == 'open' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR"]'), github.event.comment.author_association) &&
Comment thread
PeterDaveHello marked this conversation as resolved.

@github-actions github-actions Bot Jun 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Restrict manual slash-command execution to trusted repository roles only. CONTRIBUTOR includes external users who have merely made a prior contribution, so this job can be invoked by people who should not have access to a workflow that carries OPENAI_KEY and write-scoped tokens. If you need broader access, add a separate explicit trust check instead of treating all contributors as trusted. [security, importance: 7]

Suggested change
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR"]'), github.event.comment.author_association) &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&

(
contains(fromJSON('["/review", "/describe", "/improve", "/ask"]'), github.event.comment.body) ||
startsWith(github.event.comment.body, '/review ') ||
startsWith(github.event.comment.body, fromJSON('"/review\n"')) ||
startsWith(github.event.comment.body, '/describe ') ||
startsWith(github.event.comment.body, fromJSON('"/describe\n"')) ||
startsWith(github.event.comment.body, '/improve ') ||
startsWith(github.event.comment.body, fromJSON('"/improve\n"')) ||
startsWith(github.event.comment.body, '/ask ') ||
startsWith(github.event.comment.body, fromJSON('"/ask\n"'))
)
Comment on lines +71 to +81

@AndrewSouthpaw AndrewSouthpaw Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems overkill, let's just support one command like /review

@PeterDaveHello PeterDaveHello Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"/improve" should be its core function, otherwise it won't provide code suggestions.

@AndrewSouthpaw AndrewSouthpaw Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That suggestion is assuming we collapse into a single function; otherwise supporting all the slash commands makes sense.

runs-on: ubuntu-latest
permissions: *pr-agent-permissions
timeout-minutes: 60
concurrency:
# The job if gate validates exact command forms; keep these prefixes aligned.
# Regenerating command reruns supersede older same-command runs; /ask is per-comment.
group: ${{ github.workflow }}-manual-${{ github.event.issue.number }}-${{ startsWith(github.event.comment.body, '/ask') && github.event.comment.id || startsWith(github.event.comment.body, '/review') && 'review' || startsWith(github.event.comment.body, '/describe') && 'describe' || startsWith(github.event.comment.body, '/improve') && 'improve' || github.event.comment.id }}
cancel-in-progress: true

steps:
- name: Run PR Agent command
uses: *pr-agent-image