Skip to content

ci(verify): give each CodeBuild job a unique runner label - #2229

Merged
tejaskash merged 2 commits into
aws:refactorfrom
aidandaly24:ci/unique-codebuild-labels
Sep 4, 2026
Merged

ci(verify): give each CodeBuild job a unique runner label#2229
tejaskash merged 2 commits into
aws:refactorfrom
aidandaly24:ci/unique-codebuild-labels

Conversation

@aidandaly24

Copy link
Copy Markdown
Contributor

Description

Gives the Linux and Windows verify matrix jobs a unique runner label (verify-linux, verify-windows) so GitHub cannot route one job to the runner CodeBuild created for the other.

Every CodeBuild job in a workflow run shares the codebuild-agentcore-e2e-<run_id>-<attempt> label, and CodeBuild registers each ephemeral runner with every label its job requested. The Windows runner therefore carries {base, image:windows-1.0}, a superset of the plain Linux job's {base}. When the Windows runner registers while the Linux job is still queued, GitHub hands it the Linux job. The Linux runner that arrives next has only {base}, so the Windows job never matches and sits queued until someone cancels and reruns.

Attempt 1 of the three runs reported in Slack shows exactly this pattern. In each, the Windows job was queued a few seconds before the Linux job, was cancelled without ever getting a runner, and a plain-label Linux job failed on a CodeBuild runner:

Run Windows job with no runner Linux job that ran on the Windows runner
33810416086 unit-test / Test (Windows) build / Build (Linux)
33816761295 build / Build (Windows) check / check
33878953551 verify / Build (Windows) verify / check

AWS documents this failure mode and recommends a unique custom label per job: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-github-action-runners-update-labels.html. CodeBuild ignores labels it does not recognize for build configuration but still registers them on the runner, so verify-linux and verify-windows only affect routing.

Any future CodeBuild job added to a run that also calls verify.yml needs its own label too, otherwise a plain-label job can still be picked up by the verify-linux runner and strand the Linux verify job.

Originally opened as #2222 into the #2206 branch. Rebased onto refactor after #2206 merged, so CI on this PR exercises the new labels directly.

Related Issue

N/A. CI routing fix for the runner mis-assignment reported in Slack. Follows #2206, which introduced verify.yml.

Documentation PR

N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • I ran npm run test:unit and npm run test:integ
  • I ran npm run typecheck
  • I ran npm run lint
  • If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
  • Parsed the workflow as YAML and parsed each runner value as JSON with the expressions substituted
  • Confirmed the fix against attempt-1 job labels and runner assignments of the three failing runs above
  • CI on this PR runs verify.yml with the new labels; each CodeBuild job should land on its own runner

Source tests were not run because this change only touches workflow runner labels.

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

Every CodeBuild job in a run shares the codebuild-agentcore-e2e label and
CodeBuild registers each runner with all labels its job requested, so a
runner created for the Windows job also satisfies the plain Linux job.
GitHub then runs the Linux job on Windows and the Windows job never gets
a runner. A per-job label makes each job match only its own runner.

https://docs.aws.amazon.com/codebuild/latest/userguide/sample-github-action-runners-update-labels.html
@github-actions github-actions Bot added the size/xs PR size: XS label Sep 4, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Sep 4, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Nice diagnosis. The runner-label superset problem is a well-documented CodeBuild/GitHub Actions failure mode, and the fix — a unique per-job label appended to each entry in the verify.yml matrix — is the standard remediation AWS recommends. Both runner JSON arrays are valid, the Windows entry preserves image:windows-1.0 for image selection, and the two new labels (verify-linux, verify-windows) will be ignored by CodeBuild for build configuration while forcing GitHub's routing to be unambiguous.

One thing worth keeping in mind (already called out in the PR description, so no change needed here): any future job that joins the same workflow run and requests only the base codebuild-agentcore-e2e-<run>-<attempt> label can still be picked up by the verify-linux runner. If more CodeBuild jobs get added later, they'll each need their own unique label too.

No source changes, no test/telemetry surface — safe to merge.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 4, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 4, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.97%. Comparing base (1445639) to head (a253ed4).

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2229   +/-   ##
=========================================
  Coverage     96.97%   96.97%           
=========================================
  Files           559      559           
  Lines         38581    38581           
=========================================
  Hits          37412    37412           
  Misses         1169     1169           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tejaskash
tejaskash merged commit 9be9780 into aws:refactor Sep 4, 2026
18 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants