CC-39844 Track the Robot test migrations with generated matrices - #1092
Draft
stereomon wants to merge 12 commits into
Draft
CC-39844 Track the Robot test migrations with generated matrices#1092stereomon wants to merge 12 commits into
stereomon wants to merge 12 commits into
Conversation
Deterministic extract + thin AI canonical-helper map + enrich pipeline. Schema v2: type/subsystems/auth/variant, dup_of for variant clones, mutation for negative API tests, cross-framework migration-helper-map.
Generated per-scenario matrices with verdicts, per-domain PR checklists, a progress rollup and the drift gates that keep them honest. - 1721 API scenarios: 1285 MIGRATE, 418 REVIEW (contract already asserted in a Glue Cest, needs confirmation), 18 undecided (no parseable contract). - 105 UI scenarios: 74 MIGRATE, 23 REVIEW, 4 DROP, 3 RESHAPE, 1 OBSOLETE. - Canonical 35-domain taxonomy shared with the suite and cypress-tests matrices.
The rollup led with a verdict total that never moves, so a finished matrix still read as work outstanding. Lead with the remaining count instead, and keep Prettier off the matrix tree: it is shared verbatim across three repos whose configs disagree.
The matrices were built against a tree 76 commits behind master, so the classifier scored the UI rows against a Cypress checkout 41 specs stale. Re-running extract.py and enrich.py brings the inventory back in step: 5587 records, 653 Robot UI cases across the same 105 canonical scenarios. Scanned against cypress-tests master rather than the matrix branch, so target state reflects the specs that actually exist today.
The AI pass read a stale Cypress tree and produced MIGRATE 74 / REVIEW 23 / DROP 4 / RESHAPE 3 / OBSOLETE 1 — effectively "nothing can be ignored". Three corrections move work off the port list: - 41 specs landed on cypress-tests master since the classifier ran, so eight rows shrink from "author a spec" to "extend a spec". - Nine REVIEW rows overlap a sibling Robot leader rather than a Cypress spec, so the sibling's port retires them; they carry covered_by. - Lane reality decides what a deletion costs: nothing runs tests/ui/**, and five scenarios are reachable only by manual dispatch. Settles all 105 rows as decided_by human: 66 need a new spec, 22 are a delta into an existing one, 9 fold into a sibling, 5 delete outright and 3 reshape with their API half moving to Codeception. customer/Reorder stays REVIEW on purpose — the gated Cypress spec early-returns on the repositories the Robot copies run in, so the deletion needs a look first. Delta rows whose target spec already exists carry a planned test name. Without one the scanner proves only that the file exists and the row reads as already ported.
G2 was the one gate with no implementation: it needs a view of the diff, so it could not live in gate.py, and until now it was a rule honoured by hand. Both matrices retire Robot tests, so without it the tree grows faster than the batches shrink it. The check compares the test-case names in every changed .robot file against the same file at the merge base and fails on any name that is new, reusing extract.py's heading rule so the gate and the inventory agree on what counts as a test. A pull request that genuinely needs one carries the allow-new-source-test label, and the job then passes without running the check rather than being skipped, so the status stays reportable. This is the repository's first pull_request workflow -- every existing one is workflow_dispatch only.
This was the last unsettled row and it blocked the customer batch. The concern was that yves/reorder/reorder-product-offers.cy.ts early-returns on repository b2c and b2b while the Robot copies run in b2c-mp and the plain b2c shop. Reading both sides dissolves it. The mp_b2c copy asserts merchant preservation, and that spec skips only b2c and b2b, so it still runs in b2c-mp. The b2c copy asserts no merchant at all despite its documentation line claiming otherwise -- it swaps that assertion for a plain product-presence check -- and that is covered by yves/reorder/reorder-concrete-products.cy.ts, which carries no repository gate and whose isB2c() helper explicitly handles b2c and b2c-mp. Both halves are covered in the repositories the copies run in, so the row needs no port and no ungating. The UI matrix now has no unresolved verdict: 90 in scope, 15 needing no port.
This reverts commit 80a7d7f. Robot is being retired, not maintained, so policing new Robot tests with a new CI workflow is ceremony on a framework that is going away. The lane is being removed from suite CI instead, which stops the tests running at all. G2 stays documented in the migrations README as a rule honoured by hand.
stereomon
force-pushed
the
feature/cc-39844-migration-matrices
branch
from
August 21, 2026 17:17
a36e3ed to
1270425
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
test-inventory/migrations/— the tracking layer for retiring Robot Framework. Test-only, noproduction or CI code touched.
Two matrices, one row per canonical scenario (variant clones collapse onto their leader):
robot-api-to-codeceptionspryker/suiterobot-ui-to-cypressspryker/cypress-testsdecisions.jsonlanddomains.yamlare the only hand-edited files; the matrices, the per-domainchecklists and
PROGRESS.mdregenerate. Migration status is observed byscan.pyfrom bothworking trees — no one ticks a box by hand.
Read
test-inventory/migrations/README.mdfor how to execute a batch, andPROGRESS.mdfor thecurrent state.
Jira: CC-39844 (sub-task of
CC-39273) — design rationale and the canonical
domain taxonomy are documented there.
Test plan
gate.pyprintsgates cleanon this branch. G1 (every inventory leader appears in exactly onematrix), G3 (a recorded CI run implies a present target) and G4 (a skipped target never counts as
ported) all pass. Wiring the gates into CI is follow-up work, not part of this PR.