Generate the index as a GitHub App - #41
Conversation
Index generation has been failing since the require-pr-review ruleset was added: the generator runs, the push is rejected with GH013, because github-actions[bot] is not a bypass actor and cannot be made one — bypass actors of type Integration must be GitHub Apps installed on the organisation, and Actions is not one. A write-scoped deploy key is a valid bypass actor type and is scoped to this repository alone. actions/checkout takes it via `ssh-key`, loads it, and sets the remote to SSH, so the composite action's existing `git push` is unchanged. The key is an environment secret rather than a repository secret, in an environment whose deployment branch policy allows only `main`. A same-repo pull request does receive repository secrets, and this key bypasses the branch protection, so a contributor could otherwise read it out of a workflow added in a PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Pin the generator action, and preferably checkout, to verified immutable commit SHAs before granting access to the write-scoped deploy key.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Restores automated index generation using an environment-scoped deploy key for protected main-branch pushes.
Changes:
- Adds the
index-generationenvironment andINDEX_DEPLOY_KEY. - Configures
actions/checkoutfor SSH authentication. - Retains existing branch and path triggers.
File summaries
| File | Summary |
|---|---|
.github/workflows/generate-index.yml |
Configures protected environment and deploy-key authentication; the generator action should be pinned to an immutable verified SHA before receiving this credential. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # actors of type Integration must be GitHub Apps installed on the organisation. | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| ssh-key: ${{ secrets.INDEX_DEPLOY_KEY }} |
waldronlab-index-generation (app 4936017) holds `contents: write` and nothing else, is installed on this repository alone, and mints a token that expires after an hour. The deploy key it replaces was a static credential whose ruleset bypass covered every rule, including deletion and non-fast-forward. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Two moderate workflow permission-scope issues remain unresolved.
Review details
Suppressed comments (2)
.github/workflows/generate-index.yml:31
- This action inherits every permission currently granted to the App installation because no
permission-*input is set. If the App is later granted another permission, this workflow will silently mint a broader token than the least-privilege boundary described above; requestpermission-contents: writeexplicitly so the workflow enforces its intended scope.
app-id: ${{ secrets.INDEX_APP_ID }}
private-key: ${{ secrets.INDEX_APP_PRIVATE_KEY }}
.github/workflows/generate-index.yml:31
- Now that checkout authenticates with the installation token, the workflow-level
permissions: contents: writestill grants every step a separate repository-writeGITHUB_TOKEN, even though the composite action's push uses the App token. This broadens the job's write surface beyond the intended App credential; reduce the workflow permission tocontents: read(or remove it) so only the installation token can push.
- uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.INDEX_APP_ID }}
private-key: ${{ secrets.INDEX_APP_PRIVATE_KEY }}
- Files reviewed: 1/1 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
Restrict the workflow’s unnecessary contents: write permission.
Review details
Suppressed comments (1)
.github/workflows/generate-index.yml:35
- Because this checkout now persists the App token for the composite action's
git push, the existing top-levelpermissions: contents: writeis no longer needed for the workflow'sGITHUB_TOKENand gives every step a second repository-write credential. A compromised action could still use that token to mutate unprotected refs; reduce the workflow permission tocontents: read(or{}) so only the intended installation token can write.
- uses: actions/checkout@v4
with:
token: ${{ steps.app-token.outputs.token }}
- Files reviewed: 1/1 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
Address the two moderate least-privilege permission findings before approval.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (1)
.github/workflows/generate-index.yml:31
create-github-app-tokeninherits every permission granted to the installation unlesspermission-*inputs are set. Without explicitly requesting onlycontents: writehere, a later permission added to the App will silently broaden this workflow's token, undermining the least-privilege guarantee described in the PR. Addpermission-contents: write.
- uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.INDEX_APP_ID }}
private-key: ${{ secrets.INDEX_APP_PRIVATE_KEY }}
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
`permission-contents: write` is requested explicitly rather than inheriting whatever the installation holds, so granting the App another permission later cannot silently widen this token. GITHUB_TOKEN drops to `contents: read`: the push uses the installation token, so write here only handed every step a second write-capable credential. The commit is attributed to the App that pushes it rather than to github-actions[bot], which no longer has anything to do with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@v0 resolved to a commit from before the Python rewrite — 22 behind main — so both workflows here were running the R tooling. Every pull request was being checked by the old R validator, and the Pydantic rules merged in the rewrite were enforcing nothing. CI was green the whole time. Tracking main also unblocks this PR: the committer-name and committer-email inputs it passes do not exist at @v0, and the R generator would have regenerated the index in R's YAML style, re-flattening the one-element database_urls and restoring `~` for `null` — partially reverting #40. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved security, correctness, and reliability issues remain.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (5)
.github/workflows/generate-index.yml:44
- This composite action runs after checkout has persisted an installation token with
contents: writeand ruleset-bypass privileges. Resolving@mainat run time lets any future change in the standard repository use that credential, including destructive ref updates; pin this privileged action to a reviewed commit SHA.
- uses: waldronlab/agent-protocol-standard/actions/generate-index@main
.github/workflows/generate-index.yml:36
contents: writedoes not limit this App to ordinary index updates. The PR says the App is a bypass actor fordeletionandnon_fast_forwardas well as the PR/status rules, so this token can still delete or force-updatemainduring its lifetime. Split those rulesets or remove the App from the destructive-rule bypass; the workflow change alone cannot provide the claimed least privilege.
permission-contents: write
.github/workflows/generate-index.yml:49
- The email prefix for a GitHub App commit must be the App bot user's numeric ID, not the App registration ID in
INDEX_APP_ID. With this value, the push can still authenticate, but GitHub will not associate the commit withwaldronlab-index-generation[bot]as intended. Query/users/${{ steps.app-token.outputs.app-slug }}[bot]for its.idin a preceding step and use that output here.
committer-email: ${{ secrets.INDEX_APP_ID }}+${{ steps.app-token.outputs.app-slug }}[bot]@users.noreply.github.com
.github/workflows/generate-index.yml:44
- This composite action commits and pushes directly, while the workflow starts a run for every protocol push. Two merges to
maincan therefore generate overlapping runs; the latergit pushmay be rejected as non-fast-forward because its checkout predates the first run's index commit. Serialize index generation and refresh/rebase the currentmainbefore pushing, or add an equivalent retry.
- uses: waldronlab/agent-protocol-standard/actions/generate-index@main
.github/workflows/generate-index.yml:33
- The description calls the deploy key a fallback, but this workflow never configures one: checkout receives only the App token, and no
ssh-keyor other fallback credential is provided. If token minting or the environment secrets fail, the job has no usable push credential, so either implement an explicit fallback with its security trade-offs or remove that verification claim.
- uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.INDEX_APP_ID }}
private-key: ${{ secrets.INDEX_APP_PRIVATE_KEY }}
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
| # granted a bypass: bypass actors of type Integration must be GitHub Apps installed on the | ||
| # organisation, and Actions is not one. A purpose-built App is — this one holds `contents: | ||
| # write` and nothing else, and its token expires after an hour. | ||
| - uses: actions/create-github-app-token@v1 |
The committer email used the App registration id from INDEX_APP_ID, but the address needs the bot's *user* id — verified against the API, the registration is 4936017 and the bot user is 328912240. With the wrong value the push still succeeds and GitHub silently declines to link the commit to the bot, which is exactly the sort of failure nobody notices. Resolved at run time instead of hardcoded, so the same workflow shape works in the template. Two merges in quick succession would have run concurrently, and the second push would have been rejected as non-fast-forward: its checkout predates the first run's index commit. Queued rather than cancelled, since a cancelled run leaves the index describing the previous commit. Separately, the ruleset is now split. The App's bypass was `always`, which exempts every rule in a ruleset — so it could have deleted or force-pushed `main`, not merely skipped review, and the least-privilege claim made for it was not true. `require-pr-review` now holds only pull_request and required_status_checks; a new `protect-main-history` holds deletion and non_fast_forward and is bypassed by nobody but the owner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved workflow error handling, freshness, concurrency, and contributor-documentation issues remain.
Review details
Suppressed comments (4)
.github/workflows/generate-index.yml:55
- Embedding
gh apiinsideechomasks a failed lookup:echostill exits successfully, so an API error writes an emptyidand the job continues with a malformed+<slug>[bot]@...address. The App can still push, but the commit silently loses the intended bot attribution; capture the API result in a separate assignment so its status fails the step.
run: echo "id=$(gh api "/users/${SLUG}[bot]" --jq .id)" >> "$GITHUB_OUTPUT"
.github/workflows/generate-index.yml:60
- Following
generate-index@mainmakes the generator version an input, but no event in this repository is raised when that upstream action changes: the trigger only watchesprotocols/**and this workflow (lines 8-10). A generator change can therefore leave the checked-inPROTOCOLS.yamlstale until an unrelated local change occurs. Add an explicit upstream dispatch/update path (or a documented manual regeneration path) if the index is intended to track the generator as stated above.
- uses: waldronlab/agent-protocol-standard/actions/generate-index@main
.github/workflows/generate-index.yml:22
- This only serializes jobs; it does not refresh a queued run's checkout.
actions/checkoutdefaults to the push event SHA, so if another merge lands while the first run is still running, the first run can still try to push its older checkout aftermainhas advanced and fail with a non-fast-forward (and a queued run can likewise be stale if more commits land). Check out the currentmainafter acquiring the concurrency slot and/or add a retry/rebase before pushing so this concurrency guard actually prevents transient generation failures.
concurrency:
group: generate-index
cancel-in-progress: false
.github/workflows/validate.yml:15
- Switching this workflow to the Python composite action leaves
CONTRIBUTING.md:26instructing contributors to runRscript ../agent-protocol-standard/scripts/validate-protocol.R. That script is absent fromagent-protocol-standard/main, while this action now runsscripts/validate_protocol.py, so the documented local validation command fails. Update the contributor instructions with this workflow change.
- uses: waldronlab/agent-protocol-standard/actions/validate-protocols@main
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
The concurrency group alone did not make the push safe. `actions/checkout` defaults to the event SHA, so a queued run still started from the commit that triggered it and pushed a non-fast-forward once `main` had moved. It checks out the branch tip now, which also means the index describes `main` as it is rather than as it was. `echo "id=$(gh api ...)"` exits 0 when the lookup fails, writing an empty id — so a failed lookup produced a malformed address and lost bot attribution silently, which is the same class of quiet failure the id itself was wrong for an hour ago. Assigned separately under `set -euo pipefail`, and empty fails the step. Added `workflow_dispatch`. Tracking the generator at `@main` means the next run uses it, but a generator change upstream raises no event here, so the index can sit stale until an unrelated local change. A manual trigger is the honest minimum; a schedule would be the fuller answer. CONTRIBUTING told contributors to run `Rscript ../agent-protocol-standard/scripts/validate-protocol.R`, which has not existed since the Python rewrite. Replaced, and the replacement was run against this repository's protocols to confirm it works. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Address the non-fast-forward push race and pin the credential-handling App token action to a reviewed commit SHA.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (1)
.github/workflows/generate-index.yml:40
- This mutable action ref receives
INDEX_APP_PRIVATE_KEY. If@v1is retargeted or its release is compromised, its code can exfiltrate the long-lived key and mint anothercontents:writebypass token. Pin this credential-handling action to a reviewed commit SHA; the later action refs do not receive the long-lived private key and are not equivalent mitigations.
- uses: actions/create-github-app-token@v1
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
| # The branch tip, not the event SHA. Serialising runs is not enough on its own: a queued | ||
| # run would still check out the commit that triggered it and push a non-fast-forward once | ||
| # `main` had moved. It also means the index describes `main` as it is, not as it was. | ||
| ref: main |
The checkout is forced to `main`, but the generator reads GITHUB_REF_NAME to build protocol_url values. On a workflow_dispatch from another branch or tag those disagree: the index would describe main's content under a different ref's URLs, and every URL in it would resolve to the wrong revision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Two moderate workflow issues remain unresolved.
Review details
Suppressed comments (2)
.github/workflows/generate-index.yml:55
- A normal protocol merge can advance
mainafter this checkout but before the composite action'sgit push(the action pushes without fetching or rebasing). That run still has the earlier parent and its push is then rejected as non-fast-forward; this concurrency group only serializes generator runs, not ordinary pushes tomain. The workflow/action needs to retry from a freshmaincheckout and regenerate before pushing, otherwise an intervening protocol merge can both fail the run and leave the index stale.
# The branch tip, not the event SHA. Serialising runs is not enough on its own: a queued
# run would still check out the commit that triggered it and push a non-fast-forward once
# `main` had moved. It also means the index describes `main` as it is, not as it was.
ref: main
.github/workflows/generate-index.yml:79
GITHUB_REF_NAMEis a runner-providedGITHUB_*default variable and GitHub Actions does not allow workflows to overwrite it throughenv. The generator'sdetect_ref()reads that variable, so this line does not forcemainfor a dispatch from another ref as the surrounding comment claims. Pass the desired ref through a supported action/script input instead of trying to override the default.
GITHUB_REF_NAME: main
- Files reviewed: 3/3 changed files
- Comments generated: 0 new
- Review effort level: Lite
GITHUB_REF_NAME is runner-provided and a workflow cannot reliably override it through `env`, so the previous commit's attempt to force `main` for the generator would not have worked. The action takes a `ref` input instead, which the generator reads from PROTOCOL_INDEX_REF — a name the runner does not own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Manual dispatch can ignore the unsupported ref input and generate incorrect branch URLs.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
| # values from `main` too, or a workflow_dispatch from another branch would index main's | ||
| # content under that ref's URLs. Passed as an input because GITHUB_REF_NAME is | ||
| # runner-provided and a workflow cannot reliably override it. | ||
| ref: main |
Companion to waldronlab/agent-protocol-standard#41. Restores index generation, which has been failing since the
require-pr-reviewruleset was added.The generator runs fine; the push is rejected:
github-actions[bot]is not a bypass actor and cannot be made one — bypass actors of typeIntegrationmust be GitHub Apps installed on the organisation, and GitHub Actions is not one. A purpose-built App is, which is what this uses.The App
waldronlab-index-generation, app id4936017:actions/create-github-app-tokenmints an installation token that expires after an hour, andactions/checkoutuses it — so the composite action's existinggit pushneeds no change.Why not the deploy key I originally proposed
I offered four options earlier — deploy key, a personal token, PR-based, or not committing the index — and a GitHub App wasn't among them. That was the gap; GitHub's own deploy-key page recommends Apps.
Two concrete reasons it's better here:
Ruleset bypass is not per-rule. A
DeployKeybypass covers the whole ruleset, so that key could force-push or deletemain, not merely skip the pull-request requirement. The App's token carriescontents: writeand nothing else.Static versus expiring. A deploy key is valid until manually revoked. An installation token lasts an hour.
And it installs once on the organisation rather than once per repository, which starts mattering the moment there's a second content node — a Phase 2 exit criterion.
Why the credentials are environment secrets
A repository secret is readable by any workflow in the repository, including one added in a pull request. Secrets are withheld from fork PRs, but same-repo branch PRs do receive them — and these mint a token that bypasses branch protection. With five contributors about to get write access, that's the wrong place for them.
So they live in an
index-generationenvironment whose deployment branch policy allows onlymain.Already in place
Integration:4936017added as a ruleset bypass actor — all four rules (deletion,non_fast_forward,pull_request,required_status_checks) andactiveenforcement verified unchangedindex-generationholdsINDEX_APP_IDandINDEX_APP_PRIVATE_KEYThe deploy key's ruleset bypass is deliberately still in place, and gets removed once a real run proves the App route works. To be precise, since an earlier draft of this description overstated it: that is not a fallback this workflow can use. Nothing here passes
ssh-keyany more, so if token minting fails the job simply fails — the standing bypass would only help if someone reinstated the deploy-key checkout by hand.Verifying
Merging this triggers a run — the workflow file is inside the
paths:filter. A green run that checks out and pushes as the App is the proof. If it fails, the deploy key is still there and nothing is worse than it is now.🤖 Generated with Claude Code
Considered and declined
Pinning
actions/create-github-app-token@v1to a commit SHA. It is a mutable ref on an external action that receives the App private key, which is a long-lived credential — a real distinction from the refs below. But it is GitHub's own action, and the same argument would have us pinactions/checkoutandactions/setup-pythontoo. Left on the tag deliberately.Pinning
generate-index@main/validate-protocols@mainto a commit SHA. This is the decision in waldronlab/agent-protocol-standard#42, not an oversight: a node validating against a snapshot conforms to a version of the standard nobody publishes any more, and does so silently. The trust boundary here is internal — both repositories have the same owner.Protecting tags on
agent-protocol-standard. The escalation path needs write access to that repository, which is the same set of people who already have write access to this one.Automatic reconciliation when the generator changes upstream. Tracking
@mainmeans the next run uses the new generator, but a generator change raises no event in a content repository, so the index can sit stale until an unrelated local change.workflow_dispatchis added as a manual escape hatch; aschedule:or arepository_dispatchfrom the standard repo would close it properly. Deferred until it is an actual problem rather than a theoretical one — there is one content repository and one person merging.Rebase-and-retry on a non-fast-forward push. Tried, then removed. To be correct it must regenerate the index after each rebase — the intervening commit may have touched
protocols/— which puts a second generator invocation into a shared action every federated node runs. The race needs a merge inside a one-minute generation window in a repository where one person merges. A single push fails loudly and is re-runnable from the Actions tab or byworkflow_dispatch; a loud failure beats a silently stale index, which is the argument this whole change set rests on.Merge order
waldronlab/agent-protocol-standard#42 must merge first. This workflow passes a
ref:input togenerate-index, and that input only exists on the standard'smainonce #42 lands. Until then the input is ignored with an unexpected-input warning, and aworkflow_dispatchfrom a branch other thanmainwould writeprotocol_urlvalues pointing at that branch.That is a cross-PR dependency rather than a defect in this PR, and review will keep flagging it until #42 is in. Nothing else here blocks on it: the
pushtrigger always runs onmain, so the ordinary path is correct either way.