Repository navigation
ci: a release-check runs wide legs that finish in minutes, and the release owner can re-check main's tip at once (AST-030 FR23, DEC-18) - #9221
Conversation
…e legs, and a dispatch can name its commit (AST-030 FR21)
…lease owner can re-check main's tip at once (AST-030 FR23, DEC-18)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
meta-cla
Bot
added
the
CLA Signed
|
AI review status for this pull request.
|
![astracat-bot[bot]](https://avatars.githubusercontent.com/in/4796231?s=60&v=4){kind=small}
There was a problem hiding this comment.
This needs an engineering owner's spec decision before it can be approved. The decision covers the new spec:AST-030 FR23 and DEC-18, and the FR12 change that lets a release re-check cancel main's running release-check, which DEC-7 currently says never happens. An engineering owner records it by commenting /approve-spec 4c550486837e57af662194c99018597142747311, and the author may do so as an engineering owner. Whatever that decision, these still need fixing:
.github/scripts/pr-concurrency.test.mjsfails against the newcancel-in-progressexpression inci.yml, so thetestcheck will go red until its allowed list includes the new expression.- Please attach the measured release-check dispatch, because every leg now fails at 15 minutes instead of 360 (30 for RTL).
- DEC-18 and
release-recheck.ymlcite DEC-16, which exists only in #9006, and the cost note saysmainruns this check on a cadence, whilemainstill dispatches it on every push. listedTestskeeps anonymoustest.describe(() => …)titles that Playwright's test list drops, so a long file that gains one would silently lose tests from the release-check.
[Automated review]
| concurrency: | ||
| group: "ci-${{ github.event.pull_request.number || github.event.merge_group.head_sha || (github.event_name == 'workflow_dispatch' && github.triggering_actor == 'github-actions[bot]' && 'main-heavy-lanes') || github.run_id }}" | ||
| cancel-in-progress: ${{ github.event_name != 'workflow_dispatch' }} | ||
| cancel-in-progress: ${{ github.event_name != 'workflow_dispatch' || (inputs.recheck && github.triggering_actor == 'github-actions[bot]') }} |
There was a problem hiding this comment.
pr-concurrency.test.mjs:57-64 allows only true, ${{ github.event_name == 'pull_request' }} and ${{ github.event_name != 'workflow_dispatch' }}, so it fails on this expression (it passes on the base). Add this expression to that list, or assert that it still evaluates to true for pull_request.
| column: spec.column ?? 0, | ||
| }); | ||
| for (const child of suite.suites ?? []) | ||
| visit(child, [...titles, child.title], child.file ?? file); |
There was a problem hiding this comment.
An anonymous test.describe(() => …) arrives here with title: '', but Playwright's titlePath(), which --test-list matches against, drops it. A cut file containing one gets lines like [chromium] › file › › first that Playwright never selects, while lineSelects still reports each test as selected once. On Playwright 1.61.1, a three-test reproduction selects 1 of the 3. Use child.title ? [...titles, child.title] : titles here, or have canCut refuse an empty title.
| # Pull requests never hold it back: CI's ceiling leaves main what it needs | ||
| # (DEC-16). |
| const {data} = await github.rest.actions.listWorkflowRuns({ | ||
| ...context.repo, workflow_id, event: 'workflow_dispatch', branch: 'main', head_sha: sha, per_page: 20, | ||
| }); | ||
| const run = data.workflow_runs.find(candidate => candidate.triggering_actor?.login === 'github-actions[bot]'); |
There was a problem hiding this comment.
Any workflow-token dispatch matches here, including main-heavy-lanes.yml's per-push release-check. Because the loop re-runs ci.yml before it looks at deploy.yml, a failed-only run with no earlier re-check re-runs the push check's failed jobs and then throws "No re-check … in deploy.yml". Resolve both runs before re-running either.
|
/approve-spec 4c55048 |
# Conflicts: # docs/specs/AST-030/spec.md
# Conflicts: # .github/scripts/ci-heavy-lanes.test.mjs # docs/specs/AST-030/spec.md
There was a problem hiding this comment.
This needs an engineering owner's spec decision at this head before it can be approved: the new spec:AST-030 FR23 and DEC-18, and the FR12 change that lets a release re-check cancel main's running release-check, which DEC-7 says never happens. The earlier approval named the previous commit, so an engineering owner records it by commenting /approve-spec 2057354e851502937ce0f2a1211034c335ba3f05, and the author may do so as an engineering owner. Whatever that decision, these still need fixing:
failed-onlyinrelease-recheck.ymlalso matchesmain-heavy-lanes.yml's hourly check, and it re-runsci.ymlbefore confirming adeploy.ymlre-check exists, so it can re-run jobs and then fail; resolve both runs before re-running either.- DEC-18 and
release-recheck.ymlcite DEC-16, which only #9006 adds, so either stack on it or drop those sentences. listedTestskeeps anonymoustest.describe(() => …)titles that Playwright's test list drops, so a cut file that gains one would silently lose tests, and DEC-7 should name DEC-18 under "Superseded in part by".
[Automated review]
| checks `main`'s tip in full, so the cancelled run's commits stay covered, and | ||
| the release owner waits only for the re-check itself. Re-running only failed | ||
| jobs is allowed at the same commit, because that run's passing jobs already | ||
| checked it. This narrows DEC-7's never-cancel rule to every check other than a |
There was a problem hiding this comment.
DEC-7 still reads "a running check is never cancelled" and lists only DEC-11 under "Superseded in part by". Add spec:AST-030/DEC-18 there as well, so a reader of DEC-7 finds this narrowing.
|
/approve-spec b67abf2 |
There was a problem hiding this comment.
An engineering owner has approved the spec direction (FR23, DEC-18 and the FR12 re-check amendment) at this commit, but a few fixes are needed before this can be approved.
failed-onlyinrelease-recheck.ymlalso matchesmain-heavy-lanes.yml's hourly check and re-runsci.yml's failed jobs before confirming adeploy.ymlre-check exists; resolve both runs before re-running either.- DEC-18 and
release-recheck.ymlcite DEC-16, which only #9006 adds; stack on it or drop those sentences. listedTestskeeps anonymoustest.describe(() => …)titles that Playwright's test list drops, and DEC-7 should name DEC-18 under "Superseded in part by".
[Automated review]
…commit before the package-size measurement, and checks sync-exports out at it (AST-030 FR22)
# Conflicts: # docs/specs/AST-030/spec.md
…p before re-running either, and re-checks Main at its tip (AST-030 FR23)
|
/approve-spec bf51e08 |
There was a problem hiding this comment.
The spec direction is owner-approved at this commit, but two fixes are needed before this can be approved.
failed-onlyinrelease-recheck.ymlreturns before the main-tip check, so re-running an older commit's re-check cancelsmain's running release-check and Main validation of the newer tip; move the tip check above that branch and add an older-commit test case.- The description's Cost line says total spend falls, but
mainalready runs this check about hourly (DEC-11), so each check now costs about 14% more, plus each re-check; please reword it.
[Automated review]
| throw new Error(`sha must be a full 40-character commit SHA, got "${sha}".`); | ||
| } | ||
|
|
||
| if (failedOnly) { |
There was a problem hiding this comment.
failed-only returns at line 105, before the tip check at lines 110-113, so it accepts a commit main has moved past. Its re-run keeps recheck: true and runs as the workflow token, so it takes main's lanes with cancel-in-progress (ci.yml:92, deploy.yml:32) and cancels the running release-check and Main validation of the newer tip, which DEC-18 allows only for a re-check of the tip. Move the tip check above this branch and add a failed-only case with an older commit to release-check-shape.test.mjs.
…cords and only-spaces input, and is named for that commit (AST-030 FR22)
…obs too, so it never cancels the tip's checks for an older commit (AST-030 FR23)
# Conflicts: # docs/specs/AST-030/spec.md
|
/approve-spec f5c6157 |
There was a problem hiding this comment.
The spec direction (FR23, DEC-18 and the FR12 re-check amendment) is owner-approved for this commit, and the workflow and contract-sharding changes check out; one description fix remains before this can be approved. The CI impact: line gives the release-check's peak job count but not the average concurrent jobs that spec:AST-030 FR19 asks for: about +0.35, from +500 runner-minutes a day ÷ 1,440. Please add it to that line.
[Automated review]
|
/approve-spec fc289f2 |
There was a problem hiding this comment.
The spec direction (FR23, DEC-18 and the FR12 re-check amendment) is owner-approved for this commit, the CI impact: line now gives the average concurrent jobs, and the workflow and contract-sharding changes check out, so this looks ready to merge into #9207's branch. Two optional follow-ups:
canCutin.github/scripts/a11y-contract-shard.mjscould also refuse a title containing a line break, since Playwright's test-list reader would split that line and the test would run in no leg.- The comment above
pr-a11y-shardinci.ymlstill says three shard jobs; it is now six.
[Automated review]
| const paths = tests.map(test => test.titles); | ||
| if ( | ||
| paths.some(titles => | ||
| titles.some(title => title.includes('\u203A') || title !== title.trim()), |
There was a problem hiding this comment.
Optional hardening: Playwright's --test-list reader splits the file on newlines before it matches, but lineSelects checks the unsplit line, so a title containing a line break in a cut file would pass the exactly-once check and then be selected by no leg. No title has one today; adding || /[\r\n]/.test(title) here (with a test case) would keep the proof honest.
# Conflicts: # docs/specs/AST-030/spec.md
|
/approve-spec 85fa84b |
There was a problem hiding this comment.
The spec direction (FR23, DEC-18 and the FR12 re-check amendment) is owner-approved for this commit, and the wide release-check legs and the Release re-check check out: Playwright's own test-list reader runs each of the 2,215 contract tests in exactly one leg, so this looks ready to merge. Three optional follow-ups:
canCutin.github/scripts/a11y-contract-shard.mjscould also refuse a title containing a line break, since Playwright's test-list reader would split that line and the test would run in no leg.- The comment above
pr-a11y-shardinci.ymlstill says three shard jobs; it is now six. - The comment in
main-heavy-lanes.ymlthat a cancelled check "was replaced while waiting and never ran" now also covers a running check a re-check cancelled.
[Automated review]
PR Analysis Report🔎 Preview ·
|
| Package | Where | Signature | Usage | |
|---|---|---|---|---|
| + | core |
. · Types · interface FieldStatusProps |
message: string |
import type {FieldStatusProps} from '@astryxdesign/core'; const x: FieldStatusProps = {message: …} |
| + | core |
. · Types · interface FieldStatusProps |
ref?: React.Ref<HTMLDivElement> |
import type {FieldStatusProps} from '@astryxdesign/core'; const x: FieldStatusProps = {ref: …} |
| + | core |
. · Types · interface FieldStatusProps |
type: InputStatusType |
import type {FieldStatusProps} from '@astryxdesign/core'; const x: FieldStatusProps = {type: …} |
| + | core |
. · Types · interface FieldStatusProps |
variant?: FieldStatusVariant = 'attached' |
import type {FieldStatusProps} from '@astryxdesign/core'; const x: FieldStatusProps = {variant: …} |
| + | core |
. · Types · interface FieldStatusProps |
xstyle?: StyleXStyles |
import type {FieldStatusProps} from '@astryxdesign/core'; const x: FieldStatusProps = {xstyle: …} |
| + | core |
. · Types · interface FieldStatusProps |
inherits AriaAttributes, DOMAttributes, HTMLAttributes |
import type {FieldStatusProps} from '@astryxdesign/core' |
| + | core |
. · Types · interface SearchContext |
interface SearchContext<T extends SearchableItem = SearchableItem> |
import type {SearchContext} from '@astryxdesign/core' |
| + | core |
./Typeahead · Also exported from an earlier entry point · SearchContext |
SearchContext (.) |
import {SearchContext} from '@astryxdesign/core/Typeahead' |
| ~ | core |
. · Components · component ComplexSelector |
prop statusVariant?: FieldStatusVariant = 'attached' for input selectors; 'detached' for secondary and ghost selectors → prop statusVariant?: FieldStatusPlacement = 'attached' for input selectors; 'detached' for secondary and ghost selectors |
<ComplexSelector statusVariant="attached" /> |
| ~ | core |
. · Components · component DateInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<DateInput statusVariant="attached" /> |
| ~ | core |
. · Components · component DateRangeInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<DateRangeInput statusVariant="attached" /> |
| ~ | core |
. · Components · component Field |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<Field statusVariant="attached" /> |
| ~ | core |
. · Components · component FieldStatus |
component FieldStatus(props: FieldStatusProps) → component FieldStatus @deprecated |
<FieldStatus /> |
| ~ | core |
. · Components · component FieldStatus |
prop variant?: FieldStatusVariant = 'attached' → prop variant: FieldStatusVariant = 'attached' |
<FieldStatus variant="attached" /> |
| ~ | core |
. · Components · component FileInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<FileInput statusVariant="attached" /> |
| ~ | core |
. · Components · component MultiSelector |
prop statusVariant?: FieldStatusVariant = 'attached' for input selectors; 'detached' for secondary and ghost selectors → prop statusVariant?: FieldStatusPlacement = 'attached' for input selectors; 'detached' for secondary and ghost selectors |
<MultiSelector statusVariant="attached" /> |
| ~ | core |
. · Components · component NumberInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<NumberInput statusVariant="attached" /> |
| ~ | core |
. · Components · component PowerSearch |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<PowerSearch statusVariant="attached" /> |
| ~ | core |
. · Components · component Selector |
prop statusVariant?: FieldStatusVariant = 'attached' for input selectors; 'detached' for secondary/ghost selectors → prop statusVariant?: FieldStatusPlacement = 'attached' for input selectors; 'detached' for secondary/ghost selectors |
<Selector statusVariant="attached" /> |
| ~ | core |
. · Components · component TextArea |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<TextArea statusVariant="attached" /> |
| ~ | core |
. · Components · component TextInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<TextInput statusVariant="attached" /> |
| ~ | core |
. · Components · component TimeInput |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<TimeInput statusVariant="attached" /> |
| ~ | core |
. · Components · component Tokenizer |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<Tokenizer statusVariant="attached" /> |
| ~ | core |
. · Components · component Typeahead |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusPlacement = 'attached' |
<Typeahead statusVariant="attached" /> |
| ~ | core |
. · Types · interface FieldStatusProps |
interface FieldStatusProps extends BaseProps<HTMLDivElement> (props of FieldStatus) → interface FieldStatusProps extends BaseProps<HTMLDivElement> |
import type {FieldStatusProps} from '@astryxdesign/core' |
| ~ | core |
. · Types · interface FieldStatusVariantMap |
tooltip: true → tooltip: true @deprecated |
import type {FieldStatusVariantMap} from '@astryxdesign/core'; const x: FieldStatusVariantMap = {tooltip: …} |
| ~ | core |
. · Types · interface SearchSource |
search: (query: string) => Promise<T[]> | T[] → search: (query: string, context?: SearchContext<T>) => Promise<T[]> | T[] |
import type {SearchSource} from '@astryxdesign/core'; const x: SearchSource = {search: …} |
| ~ | core |
. · Types · interface UseInputStatusIconOptions |
statusVariant?: FieldStatusVariant → statusVariant?: FieldStatusPlacement |
import type {UseInputStatusIconOptions} from '@astryxdesign/core'; const x: UseInputStatusIconOptions = {statusVariant: …} |
| ~ | richtext |
. · Components · component RichTextEditor |
prop statusVariant?: FieldStatusVariant = 'attached' → prop statusVariant?: FieldStatusVariant | 'tooltip' = 'attached' |
<RichTextEditor statusVariant="tooltip" /> |
Not run on this PR: add ci:run-visual, ci:run-a11y or ci:run-rtl to run them.
CI impact: 0 job-min per update · pool 0% · some runners chosen by event or unknown · 3 other workflows changed (not measured) · 11 CI action or script files changed (not measured) ⚠️
| Before | After | Change | |
|---|---|---|---|
| Job-minutes per PR update | 78.7 | 78.7 | 0 |
| Runner pool, runner-minutes a day | 36,690 | 36,690 | 0 (0%) |
| Average concurrent jobs | 25.5 | 25.5 | 0 |
| Time to green (median → this run) | — | 13.8 min | — |
| Runs a day | 466 | 466 | 0 |
Changed jobs: Accessibility spec-test contracts changed: standard or non-standard by event (4-core-ubuntu) · admission changed: standard · pr-a11y changed: standard or non-standard by event (4-core-ubuntu) · pr-rtl changed: standard or non-standard by event (2-core-ubuntu-arm, 4-core-ubuntu) · release-authority changed: standard · Stable visual scout changed: standard or non-standard by event (4-core-ubuntu)
Other workflows: deploy.yml changed: runs on push, workflow_dispatch; standard runners; its minutes are not measured here · main-heavy-lanes.yml changed: runs on push, schedule (7,22,37,52 * * * *); standard runners; its minutes are not measured here · release-recheck.yml added: runs on workflow_dispatch; standard runners; its minutes are not measured here
Not measured (the jobs that use these keep their recent minutes above): .github/actions/ci-admission/action.yml, .github/scripts/a11y-contract-shard.mjs, .github/scripts/a11y-contract-shard.test.mjs, .github/scripts/ci-admission.test.mjs, .github/scripts/ci-heavy-lanes.test.mjs, .github/scripts/ci-matrix.mjs, .github/scripts/ci-test-routing.test.mjs, .github/scripts/pr-concurrency.test.mjs, .github/scripts/release-check-shape.test.mjs, .github/scripts/release-check-workflow.test.mjs, and 1 more.
Flags: the workflow triggers changed, so runs a day may change too.
Assumptions: before is each job's average over the last 20 pull-request runs; after keeps that for unchanged jobs and uses this run's minutes for added or changed ones, weighted by how often each changed job ran in those runs; runs a day averages the seven whole days before today; time to green compares this run with the median of those runs that ran the same jobs.
Generated by PR Enrichment workflow | View full report
…ase-check holds 37 jobs at once, so its admission peak is 37
github-actions
Bot
added
the
needs:spec-owner-review
|
/approve-spec 3a426a5 |
github-actions
Bot
removed
the
needs:spec-owner-review
There was a problem hiding this comment.
The spec direction (FR23, DEC-18 and the FR12 re-check amendment) is owner-approved for this commit, and the wide release-check legs and the Release re-check check out: Playwright's own test-list reader runs each of the 2,225 contract tests in exactly one leg. One fix is needed before this can be approved: legs() in .github/scripts/ci-admission.test.mjs (lines 40-48) keeps its own parser of the event-keyed matrix expression that the new matrixFor in .github/scripts/ci-matrix.mjs reads, and it counts an expression it does not recognise as one leg where matrixFor throws, so have legs() call matrixFor and drop its copy. Optional: the comment above pr-a11y-shard in ci.yml still says three shard jobs; it is now six.
[Automated review]
| * @param {string} event e.g. `workflow_dispatch` or `pull_request` | ||
| * @returns {unknown[]} | ||
| */ | ||
| export function matrixFor(value, event) { |
There was a problem hiding this comment.
legs() in .github/scripts/ci-admission.test.mjs (lines 40-48) parses this same expression with its own unanchored regex and counts anything it does not recognise as one leg, where this throws. Have it map each matrix value through matrixFor(value, event) and drop its copy, so the admission peak test reads the matrices the same way and fails loudly on a new shape.
…s matrices through matrixFor
… and two comments name six audit shards and the release re-check's cancel (AST-030 FR23)
github-actions
Bot
added
the
needs:spec-owner-review
|
/approve-spec 9bcd5a8 |
github-actions
Bot
removed
the
needs:spec-owner-review
There was a problem hiding this comment.
Recommending approval: the spec:AST-030 changes (FR23, DEC-18 and the FR12 re-check amendment) carry an engineering owner's approval for this commit, and the duplicate matrix parser from the last review is gone now that legs() calls matrixFor. Playwright's own reader selects each of the 2,225 contract tests in exactly one of the six legs, and the CI-script suites pass. One optional follow-up: the description's evidence still quotes 2,212 tests and the old per-leg counts; at this commit the split is 271 + 290 + 429 + 413 + 402 + 420 = 2,225.
[Automated review]
…nly named unit runs; the release re-check names its commit
…se needs one at its cut (AST-030 FR21) (#9213) * ci: main's release-check runs about once an hour, covering every merge (AST-030 FR12, DEC-10) * spec(AST-030): number the cadence decision DEC-11 * ci(release): publish only from a cut commit whose release-check passed * Merge remote-tracking branch 'origin/main' into ci/main-release-check-hourly * ci: count re-run and only passing checks; DEC-7 superseded in part; the publish guard points to a re-cut * wip: main's full unit runs about hourly (AST-030 FR21) * wip: FR21 tests * ci: a release-check names the commit it was asked to check; a failed check of the tip counts * Merge #9181's head into the unit-run branch * test(ci-test-routing): Main tests its event's own commit * spec(AST-030): FR21's unit run keeps its own hourly schedule, with every release-check too * ci: tell a mismatched release-check by its refused release-authority job, not a run name * Merge #9181's head into the unit-run branch * ci: pair a unit run with a person's release-check of the tip; re-cut advice needs both receipts; release docs name both * Merge main (with #9181) into the unit-run branch * Merge remote-tracking branch 'origin/main' into ci/main-unit-tests-hourly * ci: FR21 pairs a person's check while its commit is the tip; captures bring no unit run; dispatch before reporting * Merge main (#9207's Main shape) into the unit-run branch; unit runs name their commit; a person's newer run replaces an incomplete one * Merge main (#9006); Main's push peak is its fast checks; a unit run counts only for the commit it names; newest by group * Merge main (#9221) into the unit-run branch; the dispatcher credits only named unit runs; the release re-check names its commit * Merge branch 'main' into ci/main-unit-tests-hourly * ci: a skipped, timed-out or unstarted unit run of the tip covers nothing, so it is re-run and reported * spec(AST-030): FR22's Verification row names a person's sha run, beside the re-check that names main's tip * spec(AST-030): DEC-15 and FR21 name the release re-check as the one thing that replaces a unit run
Maintainer impact
main's visual, accessibility and RTL release-check took a median of 26 minutes and 39 at the 90th percentile over the last two days. Its slowest leg, an accessibility-contract leg, took about 20 minutes because one spec file alone took over 9. Re-checking a fix during a release meant waiting that long, often behind a check already running.main's tip. It starts the release-check and the Main workflow's full unit run (ci: main's full unit run starts every leg at once, six UI and six Node legs, and a dispatch can name its commit (AST-030 FR22) #9207) on that commit.main's tip in full, so nothing goes unchecked.failed-only, it re-runs only the failed jobs of that commit's latest re-check. That is sound because it is the same commit, so a passing re-run is a full passing check for the release publish. A fix is a new commit and gets a full re-check.mainalready runs this check about hourly (DEC-11), so that is up to about +500 runner-minutes a day, plus 223 for each release re-check (the release-check and the full unit run).CI impact: runner classes unchanged (a workflow-token release-check stays on standard GitHub-hosted runners; a person's keeps the existing larger runners), and the new re-check workflow is one
ubuntu-slimjob run on demand; pull-request updates unchanged (their legs keep today's shape); a release-check goes from 148 to 169 runner-minutes and from 23 to 35 jobs at its peak, up to about +500 runner-minutes a day at the hourly cadence (about +0.35 average concurrent jobs), plus 223 runner-minutes for each release re-check; time to green for a release-check goes from a 26-minute median to 11.8 minutes.release-check's join and receipt.Authority
Asked for by the Astryx TL: "we need to speed up these runs when they happen with more parallelization. It can be more expensive per run with runs less often or on demand during release." And: "Ideally it should take hardly any time so the release train can make refinements/fixes and recheck quickly." This PR adds
spec:AST-030FR23 and DEC-18, amends FR12's never-cancel rule for the re-check, and needs an owner's/approve-specat this head.Stacked on #9207, which adds the
shaandrecheckinputs to the Main workflow.Changes
.github/workflows/ci.yml:timeout-minutes: 15on the four leg jobs;--test-list;recheckinput that only a workflow-token dispatch can use to cancel..github/workflows/release-recheck.yml(new): the maintainer-only trigger..github/scripts/a11y-contract-shard.mjs: test-level partitioning with consecutive file parts. The partition proves each test is selected by exactly one leg, using Playwright's own test-list rules. A file is never cut when its titles cannot be named exactly or when it runs serially..github/a11y-contract-durations.json: per-file and per-test seconds from today's release-check contract legs..github/scripts/ci-matrix.mjs: reads an event-dependent matrix for contract tests.Tests:
release-check-shape.test.mjs(new) covers legs per event, the budget, the join count, the cancel rule, and the trigger's start, refusals and failed-only cases;a11y-contract-shard.test.mjsis rewritten;ci-test-routing,release-check-workflow,ci-heavy-lanesandvisual-maintenance-workfloware updated.spec:AST-030: FR12 amended, FR23, a Verification row, and DEC-18.After ci: CI holds itself to about 90 concurrent jobs; every run waits for room and main goes first (AST-030 FR16, DEC-16) #9006 landed:
ci-admission.test.mjsreads each job's legs throughmatrixFor(one parser of the event-keyed matrix), and a release-check's admission peak is 37, the jobs its wider legs hold at once, derived from the job graph.Review follow-ups: a contract file whose test title holds a line break is never cut, because Playwright's test-list reader would split that line; the audit-shard comment in
ci.ymlsays six; andmain-heavy-lanes.ymlsays a cancelled check may also have been stopped by a release re-check.Evidence
--listover the six generated test lists selects 237 + 279 + 436 + 391 + 435 + 434 = 2,212 tests, the whole suite, with no test in two legs.release-checkjoin, against a median of 26 and a 90th percentile of 39 before.release-authority, and with it the join, failed, as expected for a check dispatched from a branch instead ofmain.Scope
Writing rubric results