test(rtl): close applicability gaps with fail-closed evidence

[freddymeta]

Cause

The RTL applicability contract requires every Core/Lab surface to be measured, verified N-A, or a coverage gap. The registry originally had almost no reviewed N/A declarations, so the soft-gated audit was permanently red and hid new RTL findings.

The remaining correctness hole was that a verified-N/A row survived later source or story changes. A declaration could therefore keep masking a newly introduced relationship outside the automatic detectors.

What changed

Fail-closed N/A evidence

Every verified-N/A declaration now stores SHA-256 evidence for:

• the canonical component source;
• directly imported helpers/styles in the same top-level component directory;
• owned Storybook story files.

A changed, added, removed, or renamed evidence file returns the component to coverage-gap until that one declaration is manually re-reviewed and refreshed. Missing, empty, malformed, duplicate, or undiscoverable declarations also fail closed.

pnpm rtl:audit:verify-na -- --component core/Text --write refreshes exactly one reviewed declaration. Bulk refresh is refused.

Storybook's existing required build command runs the evidence check after producing dist/index.json, so component-, story-, and registry-only PRs cannot bypass it even though pr-rtl remains soft-gated.

Current-main coverage delta

Against current main, this PR adds 96 reviewed N/A declarations and four curated targets:

target	measured
Drawer D4 lab-drawer--showcase	overlay side
Tokenizer D2 core-tokenizer--logical-order	token/clear order
TransferList D2 lab-transferlist--default	selected/available order
Typeahead D2 core-typeahead--logical-order	token/clear order

The Stepper D2 target moved to #5495, which landed and now owns the component and regression on main.

Regression coverage

The focused suite proves:

• a current declaration is verified-na;
• changing its component source returns it to coverage-gap;
• refreshing that evidence restores verified-na;
• direct helper/style changes invalidate it;
• owned story edits and additions invalidate it;
• grouped and story-only surfaces map to the right evidence;
• empty evidence, invalid paths/digests, empty reasons, and duplicate IDs fail closed;
• Storybook's required build retains the evidence check.

Real mutation controls also changed Text.tsx and Text.stories.tsx independently: both failed with the exact changed path, and restoring the file made the checker pass again.

Current validation

On pushed head 540b8cc694abecd78136500b1a9c5ba3a69594c4:

• all 101 evidence declarations are current;
• the merged real-audit scope (DropdownMenu, Layer, Stepper, Drawer, Tokenizer, TransferList, Typeahead, InternationalizationProvider, Theme, Timestamp) reports 7 measured / 3 verified N-A / 0 gap / 0 stale;
• every declaration invalidated by landed main changes was manually re-reviewed and individually refreshed;
• every required GitHub check is green on this head; only code-review approval remains.

Gates

• pnpm rtl:audit:verify-na — pass: 101 verified-N/A evidence declarations are current
• pnpm lint:strict — 0 errors, 85 existing warnings
• Storybook typecheck — pass
• focused RTL coverage suite — 21/21 pass
• merged target audit — 7 measured / 3 verified N-A / 0 gap / 0 stale, exit 0
• latest Field/FieldLabel/FieldStatus/CircularProgress audit — 0 measured / 4 verified N-A / 0 gap / 0 stale, exit 0

No changeset: this is internal audit infrastructure and checked-in audit data; no public package API changes.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
astryx	Ready	Preview	Sep 3, 2026 1:48am UTC

[freddymeta]

Scoped runs, i.e. what pr-rtl actually executes on a PR (--filter from analysis.json):

--filter Badge                 exit=0   COV : 0 measured / 1 verified N-A / 0 gap / 0 stale
--filter Button,Text,Divider   exit=0   COV : 0 measured / 3 verified N-A / 0 gap / 0 stale
--filter TabList               exit=0   COV : 1 measured / 0 verified N-A / 0 gap / 0 stale
--filter Drawer                exit=0   COV : 1 measured / 0 verified N-A / 0 gap / 0 stale
--filter Stepper,TransferList  exit=0   COV : 2 measured / 0 verified N-A / 0 gap / 0 stale
--filter Table                 exit=1   COV : 0 measured / 0 verified N-A / 1 gap / 0 stale
--filter SideNav               exit=1   COV : 0 measured / 0 verified N-A / 1 gap / 0 stale

Table and SideNav stay red on purpose — they are two of the components this PR refuses to stamp.

pr-rtl will not run on this PR itself: check-components only sets has_components from packages/(core|lab)/src/, and this changes neither.

[github-actions]

PR Analysis Report

Preview availability: Storybook and Sandbox were not published for this CI run.

No new or modified components detected.

Bundle Size Summary

No component packages changed.

Accessibility Audit

Status: No accessibility violations detected.

---

_{Generated by PR Enrichment workflow | View full report}

[cixzhang]

Thanks for doing the full applicability pass. Main now measures CheckboxList’s start/end relationship in core-checkboxlist--rich-descriptions, but this head still declares core/CheckboxList verified N/A. I replayed this registry against current main: D2 passes, then coverage exits with 1 stale. Please merge main, keep its CheckboxList target, and remove this verified-N/A row so the RTL signal goes green for the right reason.

[Reviewed by Robohands]

[freddymeta]

Addressed in 16c8eafb6:

• merged current main
• retained its measured core-checkboxlist--rich-descriptions D2 target
• removed the stale core/CheckboxList verified-N/A entry

Verification:

• pnpm build
• pnpm -F @astryxdesign/storybook build
• pnpm -F @astryxdesign/storybook rtl-audit -- --filter CheckboxList → COV : 1 measured / 0 verified N-A / 0 gap / 0 stale
• pnpm lint:strict

pnpm test hit the existing ContextMenu adaptive-presentation concurrency flake; the isolated file passes 48/48 on this head and 48/48 on current main. CI is rerunning on the pushed head.

— Puck, on behalf of Freddy

[freddymeta]

Follow-up: main advanced while the first CI run was completing, so I merged it again in 798f046b7 and resolved the RTL registry conflict by retaining both this PR's targets and main's new Typeahead/Tokenizer targets.

Reverified on the merged tree:

• RTL audit: 3 measured / 0 verified N-A / 0 gap / 0 stale
• pnpm build
• Storybook build
• strict lint
• Storybook typecheck
• RTL coverage unit tests: 10/10

— Puck, on behalf of Freddy

[cixzhang]

Thanks, the CheckboxList stale row is fixed, and this head’s full audit confirms every added target plus 0 stale.

Main moved again after your merge: #5648 added the Chat D2 target in the same targets.json block, so GitHub now reports this head as conflicting. Please merge main once more and retain both Chat and this PR’s Drawer/Stepper/TabList/TransferList targets; dropping either side would hide an RTL relationship from future PR checks.

[Reviewed by Robohands]

[cixzhang]

Thanks, the two earlier asks—removing CheckboxList's stale declaration and retaining Chat's target—are fixed. The registry at apps/storybook/rtl-audit/verified-not-applicable.json:3 keeps every verified N-A row valid after its component or stories change, so a later RTL relationship outside the automatic detectors can stay hidden while PR CI reports it verified. Please tie each declaration to the source/story evidence it reviewed and fail closed when that evidence changes. Can we add a regression showing a changed declared component returns to a coverage gap until it is re-verified?
[Reviewed by Robohands]

[freddymeta]

Addressed the evidence-invalidation request in fbaf65934224926ee8d6957027cd7aa9dd590beb.

• Every verified-N/A row is now tied to hashes of its component entry, direct local helpers/styles, and owned stories.
• Source changes, helper changes, story edits/additions, missing evidence, malformed rows, and duplicate IDs fail closed.
• The requested regression proves a changed declared component returns to coverage-gap and only returns to verified-na after its evidence is refreshed.
• Real source and story mutation controls both failed with the exact changed path.
• Storybook's required build now runs the freshness check; the soft pr-rtl job is not the enforcement path.
• Bulk refresh is refused; only one explicitly re-reviewed component can be updated at a time.

I also merged current main, removed landed duplicate coverage, reviewed and refreshed TypeaheadItem after #5555 changed its story, and moved Stepper ownership fully to #5495.

Validation on the pushed head: build pass, Storybook build + 101-row evidence check pass, strict lint pass, Storybook typecheck pass, 21/21 focused regressions pass, and scoped real audit 4 measured / 0 gap / 0 stale.