diff --git a/docs/DATABASE.md b/docs/DATABASE.md index b5401174..9677e1a1 100644 --- a/docs/DATABASE.md +++ b/docs/DATABASE.md @@ -118,9 +118,10 @@ constraints**, and *deliberately not* native Postgres `ENUM` types. enum ordering is definition-order, not semantic. For a platform that will add variants and refine terminations over years, this rigidity is a liability. - **Lookup tables** (a `code TEXT PRIMARY KEY` catalog + FK) for vocabularies that - **evolve or carry metadata**: `variants` (add a variant → one seed row, referential - integrity everywhere it's used) and `terminations`. This gives FK enforcement, - a natural place for display names/flags, and trivial extension. + carry metadata: `variants` and `terminations`. The variants catalog also has a + canonical-domain `CHECK`, so adding a variant is a coordinated application + + constraint + seed migration, not an arbitrary catalog insert. This keeps FK + enforcement and display metadata without allowing database/application drift. - **`CHECK` constraints** for **small, fixed, security- or protocol-defined** sets where a whole table is overkort and the set changes only with a code+migration change anyway: `speed`, `result`, `role`, credential `kind`. A `CHECK` is easy to @@ -236,10 +237,14 @@ game was recorded, and keeps upgrades backward-compatible and reversible. ### 4.1 Lookup / catalog tables ```sql -CREATE TABLE variants ( -- evolving vocabulary (add a variant = 1 seed row) +CREATE TABLE variants ( -- closed application vocabulary, migration-evolved code TEXT PRIMARY KEY, -- 'standard','chess960','kingofthehill',... name TEXT NOT NULL, - enabled BOOLEAN NOT NULL DEFAULT true + enabled BOOLEAN NOT NULL DEFAULT true, + CONSTRAINT variants_code_check CHECK (code IN ( + 'standard', 'chess960', 'kingofthehill', 'atomic', + 'crazyhouse', 'threecheck', 'horde', 'racingkings' + )) ); CREATE TABLE terminations ( -- evolving/annotated vocabulary code TEXT PRIMARY KEY, -- 'checkmate','resignation','timeout',... diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md index ec95d5ad..5f6de8dd 100644 --- a/docs/PROJECT_STATE.md +++ b/docs/PROJECT_STATE.md @@ -4,7 +4,34 @@ > to read **only this file** and continue immediately. Updated after every > milestone and every significant architectural step. -_Last updated: 2026-08-29 — M15 Increment 41: Chess960 production integration (ADR-0137)._ +_Last updated: 2026-09-02 — M15 Increment 42: Closed variant catalog and studies FK integrity._ + +## M15 Increment 42 — Closed variant catalog and studies FK integrity + +`variants(code)` is now a closed database domain matching the application's eight canonical +variants, and `studies.variant` derives from that catalog via `studies_variant_fk`. Migration `0028` +idempotently restores missing canonical catalog rows and adds `variants_code_check NOT VALID`; +`0029` validates the catalog before `0030` installs the studies FK and removes the duplicated inline +CHECK from migration `0022`; `0031` validates the FK. + +This completes the database integrity conversion candidate originally deferred in M15 Increment 10 +("Decided and not done: studies.variant stays a CHECK, for now"). Historical entries in earlier +increment logs record the pre-migration state when the column was governed by a CHECK constraint. + +All database variant columns (`games.variant`, `ratings.variant`, `seeks.variant`, and +`studies.variant`) now share identical relational integrity semantics: +- Inserting a noncanonical `variants.code` is rejected with SQLSTATE `23514`; a lookup insert cannot + silently broaden the application's variant domain. +- Inserting an unsupported variant code into `studies.variant` is rejected by PostgreSQL with SQLSTATE + `23503` (`foreign_key_violation`) referencing `studies_variant_fk`. +- Missing canonical lookup rows are reconstructed without overwriting existing metadata. Existing + noncanonical rows fail catalog validation and are neither deleted nor rewritten. +- Existing study rows retain `NOT NULL DEFAULT 'standard'`. +- Foreign key semantics use default `NO ACTION` to protect `variants(code)` against accidental deletions + while referenced by active studies. +- `scripts/check-variant-parity.mjs` verifies the lookup seed and catalog CHECK against `Variant`, + requires the catalog CHECK and studies FK to be validated in later migrations, rejects unsafe + ordering, and forbids a surviving studies-specific CHECK mirror. ## M15 Increment 41 — Chess960 production integration (ADR-0137) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index d6846cb5..edbd130a 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -28,7 +28,7 @@ The correctness-critical foundation everything else depends on. - **Perft suites for each variant (RESOLVED in Increment 32 / ADR-0098 and completed in Increment 42).** All eight variants now have perft verification against published reference vectors or equality/divergence invariants. `horde` and `racingkings` coverage added from official `lichess-org/scalachess` perft resources (ADR-0098), resolving variant-rule defects in Horde rank-1 pawn double pushes and Racing Kings 8th-rank goal turn semantics. - **Chess960 was a label with nothing behind it (RESOLVED in M15 Increment 41 / ADR-0137; rules in ADR-0136).** Bigger than the "castling-by-file" wording suggested, and verified by running the code. (1) `Position.initial('chess960')` returns the standard array on every call, and `packages/game/src/game.ts:92` uses it for any seek without an explicit FEN — so a Chess960 game was ordinary chess. (2) `generateCastles` in `packages/chess-core/src/movegen.ts` pins the king to e1/e8 and looks for rooks at fixed offsets, so castling generates for exactly one of the 960 start positions, and that one is standard chess: king on b1 with rooks a1/h1 produces 0 castling moves, as does king g1 with rooks f1/h1. (3) `packages/chess-core/src/fen.ts` discards file-letter castling rights, so `HAha` on kiwipete gives `perft(1) = 46`, identical to no rights, against 48 for `KQkq`. **Withheld in Increment 33 (ADR-0099):** removed from the lobby's offered variants so nobody receives a mislabelled standard game; still accepted by the API and still a rule set in `chess-core`. **Open:** implementing it — 960-position generation, castling from arbitrary king and rook squares, Shredder/X-FEN in and out, the UCI king-takes-rook encoding, SAN, and perft against published values. **Server contract closed in M15 Increment 14 (ADR-0123):** withholding it in the lobby only protected browser users — `OFFERED_VARIANTS` is a list in the web bundle, and every other client (script, bot, mobile, curl) still reached `Game.create`, which wrote `variant: 'chess960'` beside a standard `initialFen` into an append-only event store. That is a durable falsehood, not a UI wart: afterwards nothing can tell such a row from a real Chess960 game. `Game.create` now refuses the variant outright — the one place every game is born, so seek acceptance, the bot route and the tournament launcher all inherit it — and `CREATABLE_VARIANTS` carries the same rule at the three creation routes so the refusal arrives as the API’s ordinary 422 rather than the 500 an unmapped `GameError` would produce. Seek acceptance re-checks the stored variant (409) because that value comes from a row, not a request. `chess960` remains valid everywhere that reads — the enum, the `variants` table, and every View schema — and only the three Request schemas narrowed. **Rules implemented in ADR-0136:** all 960 arrangements from the Scharnagl numbering, castling from arbitrary king and rook squares, Shredder-FEN in and canonical X-FEN out, the UCI king-takes-rook encoding, SAN unchanged, and perft against all 960 published reference positions — with the refusal deliberately kept, because the engine could now play any arrangement but nothing could yet *tell* it which one. **RESOLVED in M15 Increment 41 (ADR-0137):** `GameCreated` carries an optional `chess960StartId`, and the server draws it — `crypto.randomInt` at seek acceptance and on the bot route, derived from the launch identity for tournaments, so racing replicas agree on the arrangement instead of each drawing their own. Replay validates the stored id against the stored FEN rather than trusting either alone, and a legacy `chess960` event with no id replays from its FEN and reports its start as unknown, never as 518 — the guess that would look plausible. `Game.create` requires the id for the variant and refuses it for every other; `CREATABLE_VARIANTS` and `OFFERED_VARIANTS` admit `chess960`; `openapi.json` is regenerated. The seek-accept 409 is *kept* rather than removed as the checklist said, because `seek.variant` is read from a database column and the type system does not span the SQL (ADR-0137 §6). Move *input* needed no Chess960 logic in the browser — `BoardInteraction` is oracle-driven and the server's legal-move map already spells castling king-takes-rook — but move *projection* did: `applyMove` advances the client's own board between snapshots and recognised castling only at exactly two files, projecting `d1a1` as a king on a1 with the rook deleted. It now treats a king landing on a friendly rook as a castle (ADR-0137 §8). **Nothing left open on the variant.** - **`Position.snapshot()` lost three-check state (RESOLVED in M15 Increment 8).** `snapshot()` in `packages/chess-core/src/position.ts` round-tripped through `parseFen(this.fen(), variant)`, and `toFen` does not serialise `checkCount`, so both counters reset to zero. Found during the Increment 33 audit (ADR-0099 §4) and recorded there as "latent, not live" on the grounds that a repetition key uses only the first four FEN fields. **That assessment was wrong.** `packages/chess-core/src/repetition.ts` had appended the delivered-check counters to the key for `threecheck` since 2026-07-13 — three weeks before the audit — and `packages/game/src/game.ts` builds that key from the lossy snapshot on both the live and replay paths. Every three-check position therefore reported `0+0`, and a board that repeated while the check counts climbed was treated as a repetition: `Re1+ Kf8 Rd1 Ke8 Re1+ Kf8 Rd1 Ke8` was declared a threefold draw with White one check from winning. **Resolved in M15 Increment 8:** `snapshot()` returns `cloneState(this.state)`, the existing authoritative deep copy, so no `PositionState` field is dropped; the line above now continues and White wins `1-0` on the third check. Serialising three-check counters into FEN was deferred to M15 Increment 9 and is **RESOLVED** there (ADR-0120): `toFen` emits the canonical Fairy-Stockfish field — `N+M` remaining, in field five — and `parseFen` accepts that, the trailing `+N+M` delivered form, and the legacy six-field form. The engine defect it was hiding is closed with it: Fairy-Stockfish 14 reads a missing counter field as `1+1`, so every three-check analysis had been scored as though one check won the game. -- **The supported-variant list is written out seven times (GUARDED in M15 Increment 10).** `Variant` in `chess-core`, `VARIANTS` in the API, `StudyVariant` in studies, `SUPPORTED_VARIANTS` in ai-features, `VARIANTS` in the web client, the `variants` lookup table, and the `CHECK` on `studies.variant` added by Increment 9 — seven hand-maintained copies, none derived from another. The type system does not span the SQL: with a ninth variant added to every TypeScript site and to the `variants` lookup table but not to the `CHECK`, `npm run build` exits 0 and `npm run lint` is clean; the test suite then reports exactly one failure, and it is the wrong one — a stale `openapi.json`, which says "regenerate me" rather than "the database will reject this". After the regeneration a developer obviously runs, the suite passes with nothing red, and the variant fails as a constraint violation in production on the first study created with it. `scripts/check-variant-parity.mjs` compares the six mirrors to `chess-core`'s `Variant` and runs in CI beside the other static guards. It replays the migration directory rather than reading 0001 and 0022, because applied migrations are checksummed and immutable and a new variant arrives in a new file; it strips comments before matching, so a commented-out entry cannot pass as live. Both properties are pinned by `scripts/test/check-variant-parity.test.mjs`. **Open:** converting `studies.variant` to `REFERENCES variants(code)` like every other variant column, which would remove one copy — deferred deliberately, since it does not change the failure mode the guard closes. +- **The supported-variant list is written out across mirrors (GUARDED in M15 Increment 10; RESOLVED in M15 Increment 42).** `Variant` in `chess-core`, `VARIANTS` in the API, `StudyVariant` in studies, `SUPPORTED_VARIANTS` in ai-features, `VARIANTS` in the web client, and the `variants` lookup table — originally seven hand-maintained copies (including an inline `CHECK` on `studies.variant`), none derived from another. The type system does not span the SQL, so `scripts/check-variant-parity.mjs` replays the immutable migration history and compares every active mirror to `Variant`. **Resolved in M15 Increment 42:** migrations `0028`–`0031` restore missing canonical catalog rows, install and validate a closed eight-value CHECK on `variants(code)`, then replace the duplicated studies CHECK with a separately validated FK. Unsupported legacy catalog rows stop validation without being rewritten or deleted; a catalog insert alone can no longer broaden any FK consumer. The guard now verifies the seed, catalog CHECK, safe validation order, final studies FK, and absence of the obsolete studies CHECK. - **CI depends on the Ubuntu package mirror for Stockfish (RESOLVED in M15 Increment 11 / ADR-0121).** The `analysis smoke` job apt-installs Stockfish, and that step has now stalled indefinitely three times: once on PR #140 (cancelled and re-run successfully) and twice post-merge on `cbe6bce` (jobs `96156044656` and `96200357632`, the rerun cancelled after ~34 minutes). Each stall was in the mirror step, before Fairy-Stockfish was installed and before any smoke test ran, so it proves nothing about the code and costs the full job timeout. Fairy-Stockfish in the same job is already a pinned, checksummed release download and has never stalled. **Resolved in M15 Increment 11 (ADR-0121):** Stockfish now comes from release `sf_16`, asset `stockfish-ubuntu-x86-64.tar`, pinned by SHA-256 `efca1c60ec11fd9628425f3ee40644ad1618535ddf881c16385a86f7fc9e0983`, extracted one member by exact path, `chmod`ed only after verification, and asserted to report `id name Stockfish 16` before the suite runs. `sf_16` is the version apt was already serving, so the engine under test is unchanged. `apt-get` no longer appears in any executable line of any workflow, and the job now carries `timeout-minutes: 15` so a future stall is capped rather than inheriting the six-hour default. Production Docker images used apt until **M15 Increment 12**, which closed the last of it. Before that, `release.yml` built `Dockerfile.api` and `Dockerfile.gateway` on a `v*` tag push and those builds ran the apt layers — meaning production shipped Debian bookworm's `stockfish 15.1-4` while CI proved the engine boundary against 16, and a base-image move to trixie would have made it 17 with no commit of ours. Both images now take the binary from a pinned `stockfish` artefact stage using the same release, asset and digest as CI, with the licence and corresponding source copied beside it for GHCR redistribution, and a `docker-images` CI job builds both before merge instead of first exercising them at release time. `scripts/check-engine-pin-parity.mjs` fails if the four copies of the pin ever disagree. - **The web image was published but never built before the tag (RESOLVED in M15 Increment 12).** `release.yml` pushes three images to GHCR on a `v*` tag — `Dockerfile.api`, `Dockerfile.gateway` and `Dockerfile.web` — but the `docker-images` CI job as first written built only the two that carry the pinned engine. `Dockerfile.web` copies `docker/web/nginx.conf.template` into `/etc/nginx/templates/`, where the image entrypoint runs `envsubst` over it at container start, so a broken template is not a build error at all: the image builds clean and the container dies on boot, and nothing before the release tag rendered it. Raised in the Qodo review of PR #143. **Resolved in the same increment:** the job builds all three images and checks the web one for what can actually break in it — the entrypoint renders the template with representative loopback upstreams (`nginx -t` resolves a literal `proxy_pass` host at config-load time, so the compose defaults would fail on a runner for the wrong reason), `nginx -t` must accept the result, both upstreams must appear substituted, and `$http_host` and `$uri` must survive, which is what `NGINX_ENVSUBST_FILTER` exists to guarantee and nothing tested. `docker/` joins the `images` path filter so the filter and the job cover the same set. - **A study movetext walker that never asked which variant it was reading (RESOLVED in M15 Increment 13 / ADR-0122).** `importGame` in `packages/studies/src/import.ts` resolved every SAN through `resolveSan(reader, fen, san)` and `reader.play(fen, san)` with no variant, and `resolveSan` defaults to standard rather than failing — so a Crazyhouse or Three-Check game imported through it would have been validated against standard chess, rejecting legal moves and accepting illegal ones with no error to say why. Latent, not live: a whole-repository search found it referenced only by its own definition and its own test file, and both real import paths (`InMemoryStudiesRepository.buildTreeFromMovetext` via `appendNode`, and `PgStudiesRepository.buildTreeFromMovetextInternal` via a required parameter) already thread the study variant correctly. It was also a third implementation of a descent the two adapters already have, and ADR-0091 §10 records what happened the last time two copies of this walk diverged. **Resolved in M15 Increment 13 (ADR-0122):** deleted, together with the types and the `START_FEN` alias that existed only to serve it; `chapterNameFor` stays, because both adapters import it. `resolveSan`’s standard default is kept and now pinned by a test that states why — `@chess-platform/learning` relies on it and lessons carry no variant of their own. The coverage that was only reachable through the dead function moved onto `resolveSan` itself, and variant propagation through side variations — previously unverified, since the existing three-check test had no variations — is now a mutation-checked regression test. diff --git a/packages/persistence/migrations/0028_studies_variant_fk.sql b/packages/persistence/migrations/0028_studies_variant_fk.sql new file mode 100644 index 00000000..c346e554 --- /dev/null +++ b/packages/persistence/migrations/0028_studies_variant_fk.sql @@ -0,0 +1,26 @@ +-- Migration 0028: Restore the canonical variant catalog and install its closed domain constraint. + +INSERT INTO variants (code, name) VALUES + ('standard', 'Standard'), + ('chess960', 'Chess960'), + ('kingofthehill', 'King of the Hill'), + ('atomic', 'Atomic'), + ('crazyhouse', 'Crazyhouse'), + ('threecheck', 'Three-check'), + ('horde', 'Horde'), + ('racingkings', 'Racing Kings') +ON CONFLICT (code) DO NOTHING; + +-- NOT VALID closes the domain for new writes immediately while deferring the legacy-row scan. +ALTER TABLE variants + ADD CONSTRAINT variants_code_check + CHECK (code IN ( + 'standard', + 'chess960', + 'kingofthehill', + 'atomic', + 'crazyhouse', + 'threecheck', + 'horde', + 'racingkings' + )) NOT VALID; diff --git a/packages/persistence/migrations/0029_validate_variants_code_check.sql b/packages/persistence/migrations/0029_validate_variants_code_check.sql new file mode 100644 index 00000000..7770955d --- /dev/null +++ b/packages/persistence/migrations/0029_validate_variants_code_check.sql @@ -0,0 +1,4 @@ +-- Migration 0029: Reject unsupported legacy variant catalog rows before changing studies. + +ALTER TABLE variants + VALIDATE CONSTRAINT variants_code_check; diff --git a/packages/persistence/migrations/0030_studies_variant_fk.sql b/packages/persistence/migrations/0030_studies_variant_fk.sql new file mode 100644 index 00000000..0456c677 --- /dev/null +++ b/packages/persistence/migrations/0030_studies_variant_fk.sql @@ -0,0 +1,8 @@ +-- Migration 0030: Replace the duplicated studies.variant CHECK with the canonical catalog FK. + +ALTER TABLE studies + ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID; + +ALTER TABLE studies + DROP CONSTRAINT studies_variant_check; diff --git a/packages/persistence/migrations/0031_validate_studies_variant_fk.sql b/packages/persistence/migrations/0031_validate_studies_variant_fk.sql new file mode 100644 index 00000000..6ff99dee --- /dev/null +++ b/packages/persistence/migrations/0031_validate_studies_variant_fk.sql @@ -0,0 +1,4 @@ +-- Migration 0031: Validate the studies.variant foreign key after its non-blocking installation. + +ALTER TABLE studies + VALIDATE CONSTRAINT studies_variant_fk; diff --git a/packages/persistence/test/studies.integration.test.ts b/packages/persistence/test/studies.integration.test.ts index c1f45d47..4c40cf6f 100644 --- a/packages/persistence/test/studies.integration.test.ts +++ b/packages/persistence/test/studies.integration.test.ts @@ -13,10 +13,12 @@ const DATABASE_URL = process.env['DATABASE_URL']; const skip = DATABASE_URL ? false : 'DATABASE_URL not set'; class CorePositionReader implements PositionReader { + /** Returns legal SAN moves for the supplied position and variant. */ legalSans(fen: string, variant: StudyVariant = 'standard'): readonly string[] { const pos = Position.fromFen(fen, variant); return pos.legalMoves().map((m) => pos.toSan(m)); } + /** Applies one legal SAN move and returns the resulting FEN. */ play(fen: string, san: string, variant: StudyVariant = 'standard'): string { const pos = Position.fromFen(fen, variant); const moves = pos.legalMoves(); @@ -28,6 +30,20 @@ class CorePositionReader implements PositionReader { } } +interface PgErrorShape { + code?: string; + constraint?: string; +} + +/** Identifies a PostgreSQL constraint violation without assuming every thrown value is an Error. */ +function isPgConstraintViolation(err: unknown, code: string, constraint?: string): boolean { + if (typeof err !== 'object' || err === null) return false; + const pgErr = err as PgErrorShape; + if (pgErr.code !== code) return false; + if (constraint !== undefined && pgErr.constraint !== constraint) return false; + return true; +} + test('pg studies repository integration tests', { skip }, async () => { const pool = createPool(); await migrate(pool, join(process.cwd(), 'migrations')); @@ -36,6 +52,7 @@ test('pg studies repository integration tests', { skip }, async () => { const reader = new CorePositionReader(); const createdUserIds: string[] = []; + /** Inserts and tracks a disposable integration-test user. */ const createUser = async (name: string): Promise => { const id = uuidv7(); const handle = `usr-${name.toLowerCase()}-${id.slice(0, 8)}`; @@ -70,7 +87,7 @@ test('pg studies repository integration tests', { skip }, async () => { `INSERT INTO study_collaborators (study_id, player_id, role) VALUES ($1, $2, 'owner')`, [studyId1, bob] ), - (err: any) => err && err.code === '23505' + (err: unknown) => isPgConstraintViolation(err, '23505') ); // 2. Collaborators and Ownership Transfer (demotes before promoting) @@ -81,7 +98,7 @@ test('pg studies repository integration tests', { skip }, async () => { // New owner must already be a collaborator await assert.rejects( async () => repo.transferOwnership(studyId1, alice, charlie, t2), - (err: any) => err instanceof StudyRuleError && err.code === 'not_found' + (err: unknown) => err instanceof StudyRuleError && err.code === 'not_found' ); // Transfer ownership from Alice to Bob @@ -97,7 +114,7 @@ test('pg studies repository integration tests', { skip }, async () => { // Non-collaborator gets not_found await assert.rejects( async () => repo.getStudy(privStudyId, charlie), - (err: any) => err instanceof StudyRuleError && err.code === 'not_found' + (err: unknown) => err instanceof StudyRuleError && err.code === 'not_found' ); // Unlisted study @@ -126,7 +143,7 @@ test('pg studies repository integration tests', { skip }, async () => { `INSERT INTO study_chapters (id, study_id, name, order_index, starting_fen) VALUES ($1, $2, 'Bad Order', 0, $3)`, [uuidv7(), studyId1, ch1.startingFen] ), - (err: any) => err && err.code === '23505' + (err: unknown) => isPgConstraintViolation(err, '23505') ); // Reorder chapters @@ -142,7 +159,7 @@ test('pg studies repository integration tests', { skip }, async () => { await assert.rejects( async () => repo.deleteChapter(chId2, bob, t2), - (err: any) => err instanceof StudyRuleError && err.code === 'invalid_transition' + (err: unknown) => err instanceof StudyRuleError && err.code === 'invalid_transition' ); // 5. Tree Node Management (append, resolve SAN, return existing child, delete node cascade) @@ -217,13 +234,13 @@ test('pg studies repository integration tests', { skip }, async () => { await assert.rejects( async () => repo.importPgn(studyId1, bob, `[Event "Bad"]\n\n1. e4 invalidmove *`, reader, t2), - (err: any) => err instanceof StudyRuleError && err.code === 'invalid_input' + (err: unknown) => err instanceof StudyRuleError && err.code === 'invalid_input' ); // `Nf6` is perfectly good SAN — there is simply no white knight that can reach f6 on move 2. await assert.rejects( async () => repo.importPgn(studyId1, bob, `[Event "Bad"]\n\n1. e4 e5 2. Nf6 *`, reader, t2), - (err: any) => + (err: unknown) => err instanceof StudyRuleError && err.code === 'invalid_move' && /Nf6/.test(err.message) ); @@ -270,7 +287,7 @@ test('pg studies repository integration tests', { skip }, async () => { for (const badId of ['not-a-uuid', 'bad-id-123']) { await assert.rejects( async () => repo.getStudy(badId, alice), - (err: any) => err instanceof StudyRuleError && err.code === 'not_found', + (err: unknown) => err instanceof StudyRuleError && err.code === 'not_found', `getStudy('${badId}') must yield not_found` ); } @@ -284,8 +301,97 @@ test('pg studies repository integration tests', { skip }, async () => { await assert.rejects( async () => repo.getStudy(cascadeStudyId, alice), - (err: any) => err instanceof StudyRuleError && err.code === 'not_found' + (err: unknown) => err instanceof StudyRuleError && err.code === 'not_found' + ); + + // 10. Closed variant catalog and studies FK integrity (migrations 0028-0031) + const domainRes = await pool.query<{ convalidated: boolean }>(` + SELECT convalidated + FROM pg_constraint + WHERE conrelid = 'variants'::regclass + AND contype = 'c' + AND conname = 'variants_code_check' + `); + assert.deepEqual(domainRes.rows, [{ convalidated: true }]); + + // 10.1 Metadata verification: FK constraint exists and points to variants(code) + const fkRes = await pool.query<{ + constraint_name: string; + table_name: string; + column_name: string; + foreign_table_name: string; + foreign_column_name: string; + }>(` + SELECT + tc.constraint_name, + tc.table_name, + kcu.column_name, + ccu.table_name AS foreign_table_name, + ccu.column_name AS foreign_column_name + FROM information_schema.table_constraints tc + JOIN information_schema.key_column_usage kcu + ON tc.constraint_name = kcu.constraint_name AND tc.table_schema = kcu.table_schema + JOIN information_schema.constraint_column_usage ccu + ON ccu.constraint_name = tc.constraint_name AND ccu.table_schema = tc.table_schema + WHERE tc.table_name = 'studies' + AND tc.constraint_type = 'FOREIGN KEY' + AND kcu.column_name = 'variant' + `); + assert.equal(fkRes.rows.length, 1, 'expected exactly one FK constraint on studies.variant'); + assert.equal(fkRes.rows[0]?.constraint_name, 'studies_variant_fk'); + assert.equal(fkRes.rows[0]?.foreign_table_name, 'variants'); + assert.equal(fkRes.rows[0]?.foreign_column_name, 'code'); + + // 10.2 Absence of old CHECK constraint + const oldCheckRes = await pool.query<{ conname: string }>(` + SELECT conname + FROM pg_constraint + WHERE conrelid = 'studies'::regclass + AND contype = 'c' + AND conname = 'studies_variant_check' + `); + assert.equal(oldCheckRes.rows.length, 0, 'old CHECK constraint studies_variant_check must no longer exist'); + + // 10.3 The catalog rejects noncanonical codes before they can broaden FK consumers. + await assert.rejects( + pool.query(`INSERT INTO variants (code, name) VALUES ('noncanonical_variant', 'Invalid')`), + (err: unknown) => isPgConstraintViolation(err, '23514', 'variants_code_check'), + 'inserting a noncanonical catalog code must raise check_violation (23514)' + ); + + // An unsupported study remains rejected by the FK. + const badStudyId = uuidv7(); + await assert.rejects( + async () => + pool.query( + `INSERT INTO studies (id, owner_id, name, description, visibility, variant, created_at, updated_at) + VALUES ($1, $2, 'Bad Variant Study', '', 'public', 'nonexistent_variant', NOW(), NOW())`, + [badStudyId, alice] + ), + (err: unknown) => isPgConstraintViolation(err, '23503', 'studies_variant_fk'), + 'inserting an invalid variant must raise foreign_key_violation (23503) referencing studies_variant_fk' ); + + // 10.4 All 8 canonical StudyVariants can be created and queried through repository + const ALL_VARIANTS: readonly StudyVariant[] = [ + 'standard', + 'chess960', + 'kingofthehill', + 'atomic', + 'crazyhouse', + 'threecheck', + 'horde', + 'racingkings', + ]; + for (const v of ALL_VARIANTS) { + const vStudyId = uuidv7(); + const vStudy = await repo.createStudy(vStudyId, alice, `Study ${v}`, '', 'public', t0, { variant: v }); + assert.equal(vStudy.id, vStudyId); + assert.equal(vStudy.variant, v); + const fetched = await repo.getStudy(vStudyId, alice); + assert.equal(fetched.variant, v); + } + } finally { if (createdUserIds.length > 0) { await pool.query(`DELETE FROM users WHERE id = ANY($1)`, [createdUserIds]); diff --git a/packages/persistence/test/variant-migrations.integration.test.ts b/packages/persistence/test/variant-migrations.integration.test.ts new file mode 100644 index 00000000..daae0865 --- /dev/null +++ b/packages/persistence/test/variant-migrations.integration.test.ts @@ -0,0 +1,273 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import { copyFileSync, mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { Pool } from 'pg'; +import { createPool } from '../src/pg/pool'; +import { migrate, migrationFiles, migrationsDir } from '../src/pg/migrate'; + +const DATABASE_URL = process.env['DATABASE_URL']; +const skip = DATABASE_URL ? false : 'DATABASE_URL not set'; +const MIGRATIONS_DIR = migrationsDir(); +const CANONICAL_VARIANTS = [ + 'standard', + 'chess960', + 'kingofthehill', + 'atomic', + 'crazyhouse', + 'threecheck', + 'horde', + 'racingkings', +] as const; + +interface PgErrorShape { + readonly code?: string; + readonly constraint?: string; +} + +interface ConstraintRow { + readonly conname: string; + readonly convalidated: boolean; + readonly definition: string; +} + +/** Returns whether an unknown thrown value is the expected PostgreSQL constraint violation. */ +function isConstraintViolation(error: unknown, code: string, constraint: string): boolean { + if (typeof error !== 'object' || error === null) return false; + const pgError = error as PgErrorShape; + return pgError.code === code && pgError.constraint === constraint; +} + +/** Replaces the database path in the configured PostgreSQL connection URL. */ +function databaseUrlFor(database: string): string { + const url = new URL(DATABASE_URL!); + url.pathname = `/${database}`; + return url.toString(); +} + +/** Copies migrations through the requested version into a disposable directory. */ +function migrationsThrough(version: number): { readonly dir: string; cleanup(): void } { + const dir = mkdtempSync(join(tmpdir(), `variant-migrations-${version}-`)); + for (const migration of migrationFiles(MIGRATIONS_DIR)) { + if (migration.version > version) continue; + copyFileSync(join(MIGRATIONS_DIR, migration.file), join(dir, migration.file)); + } + return { dir, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +/** Runs a callback in an isolated database and always releases its pools and database. */ +async function withDatabase(run: (pool: Pool) => Promise): Promise { + const admin = createPool({ connectionString: DATABASE_URL, max: 2 }); + const database = `variant_migration_${randomUUID().replaceAll('-', '')}`; + let databaseCreated = false; + try { + await admin.query(`CREATE DATABASE "${database}"`); + databaseCreated = true; + const pool = createPool({ connectionString: databaseUrlFor(database), max: 4 }); + try { + await run(pool); + } finally { + await pool.end(); + } + } finally { + try { + if (databaseCreated) { + await admin.query(`DROP DATABASE IF EXISTS "${database}" WITH (FORCE)`); + } + } finally { + await admin.end(); + } + } +} + +/** Reads one named constraint from PostgreSQL's catalog. */ +async function constraint( + pool: Pool, + table: 'variants' | 'studies', + name: string, +): Promise { + const found = await pool.query( + `SELECT conname, convalidated, pg_get_constraintdef(oid) AS definition + FROM pg_constraint + WHERE conrelid = $1::regclass AND conname = $2`, + [table, name], + ); + return found.rows[0]; +} + +/** Inserts the minimum user row needed to own a study and returns its id. */ +async function insertUser(pool: Pool): Promise { + const id = randomUUID(); + await pool.query( + `INSERT INTO users (id, handle, email_hash) VALUES ($1, $2, $3)`, + [id, `variant_${id.replaceAll('-', '')}`, Buffer.from(id)], + ); + return id; +} + +/** Inserts a study with the requested variant and returns its id. */ +async function insertStudy(pool: Pool, ownerId: string, variant: string): Promise { + const id = randomUUID(); + await pool.query( + `INSERT INTO studies + (id, owner_id, name, description, visibility, variant, created_at, updated_at) + VALUES ($1, $2, $3, '', 'public', $4, NOW(), NOW())`, + [id, ownerId, `Study ${variant}`, variant], + ); + return id; +} + +test('fresh install enforces the closed variants domain and validated studies FK', { skip }, async () => { + await withDatabase(async (pool) => { + await migrate(pool, MIGRATIONS_DIR); + + const variants = await pool.query<{ code: string }>('SELECT code FROM variants ORDER BY code'); + assert.deepEqual( + variants.rows.map((row) => row.code), + [...CANONICAL_VARIANTS].sort(), + ); + + await assert.rejects( + pool.query(`INSERT INTO variants (code, name) VALUES ('unsupported', 'Unsupported')`), + (error: unknown) => isConstraintViolation(error, '23514', 'variants_code_check'), + ); + + const domain = await constraint(pool, 'variants', 'variants_code_check'); + assert.equal(domain?.convalidated, true); + assert.match(domain?.definition ?? '', /^CHECK \(\(code = ANY \(ARRAY\[/); + assert.equal(await constraint(pool, 'studies', 'studies_variant_check'), undefined); + + const foreignKey = await constraint(pool, 'studies', 'studies_variant_fk'); + assert.equal(foreignKey?.convalidated, true); + assert.equal(foreignKey?.definition, 'FOREIGN KEY (variant) REFERENCES variants(code)'); + + const ownerId = await insertUser(pool); + for (const variant of CANONICAL_VARIANTS) await insertStudy(pool, ownerId, variant); + + await assert.rejects( + insertStudy(pool, ownerId, 'unsupported'), + (error: unknown) => isConstraintViolation(error, '23503', 'studies_variant_fk'), + ); + await assert.rejects( + pool.query(`DELETE FROM variants WHERE code = 'racingkings'`), + (error: unknown) => isConstraintViolation(error, '23503', 'studies_variant_fk'), + ); + }); +}); + +test('upgrade restores missing canonical rows before replacing the studies CHECK', { skip }, async () => { + const through27 = migrationsThrough(27); + const through28 = migrationsThrough(28); + const through29 = migrationsThrough(29); + const through30 = migrationsThrough(30); + try { + await withDatabase(async (pool) => { + await migrate(pool, through27.dir); + const ownerId = await insertUser(pool); + const atomicStudy = await insertStudy(pool, ownerId, 'atomic'); + const hordeStudy = await insertStudy(pool, ownerId, 'horde'); + await pool.query(`DELETE FROM variants WHERE code IN ('atomic', 'horde')`); + await pool.query( + `UPDATE variants SET name = 'Operator Standard', enabled = false WHERE code = 'standard'`, + ); + + await migrate(pool, through28.dir); + assert.equal((await constraint(pool, 'variants', 'variants_code_check'))?.convalidated, false); + assert.notEqual(await constraint(pool, 'studies', 'studies_variant_check'), undefined); + assert.equal(await constraint(pool, 'studies', 'studies_variant_fk'), undefined); + await assert.rejects( + pool.query(`INSERT INTO variants (code, name) VALUES ('new_rogue', 'New Rogue')`), + (error: unknown) => isConstraintViolation(error, '23514', 'variants_code_check'), + ); + + await migrate(pool, through29.dir); + assert.equal((await constraint(pool, 'variants', 'variants_code_check'))?.convalidated, true); + assert.notEqual(await constraint(pool, 'studies', 'studies_variant_check'), undefined); + + await migrate(pool, through30.dir); + assert.equal(await constraint(pool, 'studies', 'studies_variant_check'), undefined); + assert.equal((await constraint(pool, 'studies', 'studies_variant_fk'))?.convalidated, false); + await assert.rejects( + insertStudy(pool, ownerId, 'new_rogue'), + (error: unknown) => isConstraintViolation(error, '23503', 'studies_variant_fk'), + ); + + await migrate(pool, MIGRATIONS_DIR); + assert.equal((await constraint(pool, 'studies', 'studies_variant_fk'))?.convalidated, true); + const studies = await pool.query<{ id: string; variant: string }>( + 'SELECT id, variant FROM studies WHERE id = ANY($1) ORDER BY variant', + [[atomicStudy, hordeStudy]], + ); + assert.deepEqual(studies.rows.map((row) => row.variant), ['atomic', 'horde']); + const standard = await pool.query<{ name: string; enabled: boolean }>( + `SELECT name, enabled FROM variants WHERE code = 'standard'`, + ); + assert.deepEqual(standard.rows[0], { name: 'Operator Standard', enabled: false }); + }); + } finally { + through27.cleanup(); + through28.cleanup(); + through29.cleanup(); + through30.cleanup(); + } +}); + +test('unsupported legacy catalog data fails loudly, survives rollback, and retries deterministically', { skip }, async () => { + const through27 = migrationsThrough(27); + const through28 = migrationsThrough(28); + const through29 = migrationsThrough(29); + try { + await withDatabase(async (pool) => { + await migrate(pool, through27.dir); + const ownerId = await insertUser(pool); + await pool.query( + `INSERT INTO variants (code, name, enabled) VALUES ('legacy_rogue', 'Legacy Rogue', false)`, + ); + await pool.query( + `INSERT INTO ratings (user_id, variant, rating, rd, vol) + VALUES ($1, 'legacy_rogue', 1400, 100, 0.06)`, + [ownerId], + ); + await pool.query( + `INSERT INTO seeks (id, creator_id, variant, time_control, rated) + VALUES ($1, $2, 'legacy_rogue', '{}', false)`, + [randomUUID(), ownerId], + ); + await pool.query( + `INSERT INTO games (id, variant, rated, speed, result, ply_count, last_seq, started_at) + VALUES ($1, 'legacy_rogue', false, 'rapid', '*', 0, 0, NOW())`, + [randomUUID()], + ); + + await migrate(pool, through28.dir); + assert.equal((await constraint(pool, 'variants', 'variants_code_check'))?.convalidated, false); + + await assert.rejects(migrate(pool, through29.dir), /variants_code_check/); + await assert.rejects(migrate(pool, through29.dir), /variants_code_check/); + const preserved = await pool.query<{ source: string }>(` + SELECT 'variant' AS source FROM variants WHERE code = 'legacy_rogue' + UNION ALL SELECT 'rating' FROM ratings WHERE variant = 'legacy_rogue' + UNION ALL SELECT 'seek' FROM seeks WHERE variant = 'legacy_rogue' + UNION ALL SELECT 'game' FROM games WHERE variant = 'legacy_rogue' + ORDER BY source + `); + assert.deepEqual(preserved.rows.map((row) => row.source), ['game', 'rating', 'seek', 'variant']); + assert.notEqual(await constraint(pool, 'studies', 'studies_variant_check'), undefined); + assert.equal(await constraint(pool, 'studies', 'studies_variant_fk'), undefined); + + await pool.query(`DELETE FROM games WHERE variant = 'legacy_rogue'`); + await pool.query(`DELETE FROM seeks WHERE variant = 'legacy_rogue'`); + await pool.query(`DELETE FROM ratings WHERE variant = 'legacy_rogue'`); + await pool.query(`DELETE FROM variants WHERE code = 'legacy_rogue'`); + await migrate(pool, MIGRATIONS_DIR); + assert.equal((await constraint(pool, 'variants', 'variants_code_check'))?.convalidated, true); + assert.equal((await constraint(pool, 'studies', 'studies_variant_fk'))?.convalidated, true); + }); + } finally { + through27.cleanup(); + through28.cleanup(); + through29.cleanup(); + } +}); diff --git a/scripts/check-variant-parity.mjs b/scripts/check-variant-parity.mjs index fcb840b2..5b846f42 100644 --- a/scripts/check-variant-parity.mjs +++ b/scripts/check-variant-parity.mjs @@ -2,18 +2,13 @@ /** * Fails when the hand-maintained copies of the supported-variant list stop agreeing. * - * The set of rule sets this platform supports is written out in six places, in two languages, and - * nothing derives from anything else. That is survivable only while they match, and there was no - * check that they do. - * - * The sharp edge is the database. Every other variant column is - * `variant TEXT NOT NULL REFERENCES variants(code)`, so once a row exists in the `variants` lookup - * table the database accepts that value in the games and ratings columns. `studies.variant` alone - * (migration 0022, M15 Increment 9) is governed by an inline `CHECK (variant IN (...))`, so the - * same row does nothing for studies: the type system says the variant is fine, the API accepts it, - * and Postgres rejects the insert at runtime as a constraint violation. The application-level - * declarations below still need their own updates in either case — the lookup row settles only - * what the *database* will store. + * The set of rule sets this platform supports is written out in six active mirrors, in two + * languages. That is survivable only while they match. + * + * The database variant columns reference `variants(code)`. Migrations 0028/0029 close and validate + * that catalog's canonical domain; migrations 0030/0031 then replace the historical + * `studies.variant` CHECK with a validated FK. The guard replays both the catalog seed and its + * database CHECK, so a lookup row alone cannot widen the domain. * * `chess-core`'s `Variant` is treated as the root: it is the type the engine actually branches on, * so a variant that is not there is not a variant at all. Every other list is compared to it. @@ -39,12 +34,60 @@ * * Run: node scripts/check-variant-parity.mjs */ +import { createHash } from 'node:crypto'; import { readFileSync, readdirSync } from 'node:fs'; import { join } from 'node:path'; import { pathToFileURL } from 'node:url'; +/** + * Pinned allowlist of reviewed, immutable historical migrations containing top-level PostgreSQL DO blocks. + * + * Deterministic variant-parity replay cannot evaluate arbitrary procedural PL/pgSQL code. + * Any migration file containing a top-level DO block that is not explicitly pinned by both + * its relative filename, its canonical UTF-8 SHA-256 digest, and its expected DO statement count + * will fail closed immediately. + * + * @type {ReadonlyMap} + */ +export const KNOWN_HISTORICAL_PROCEDURAL_MIGRATIONS = new Map([ + [ + '0021_engine_bots.sql', + { + sha256: 'a59b1e4e9cefed19bca7de45cbd5f7d533a85f13fa197300d6bad9226c255508', + expectedDoCount: 1, + }, + ], +]); + +/** + * Directory holding the database migration SQL scripts. + * @type {string} + */ export const MIGRATIONS_DIR = 'packages/persistence/migrations'; +const DOLLAR_QUOTE_REGEX = /^\$(?:[_\p{L}\p{Nl}][_\p{L}\p{Nl}\p{Nd}\p{Mn}\p{Mc}]*)?\$/u; + +/** + * Scans a valid PostgreSQL dollar-quote delimiter at the given source offset. + * + * PostgreSQL dollar tags follow unquoted identifier rules except that `$` is forbidden + * within the tag. Delimiters can be untagged `$$` or tagged `$tag$`. + * + * @param {string} source Source text. + * @param {number} offset Starting character offset in source text. + * @returns {{ delimiter: string, end: number } | null} Delimiter metadata or null if invalid. + */ +export function readDollarQuoteDelimiter(source, offset) { + if (source[offset] !== '$') return null; + const match = DOLLAR_QUOTE_REGEX.exec(source.slice(offset)); + if (match === null) return null; + const delimiter = match[0]; + return { + delimiter, + end: offset + delimiter.length, + }; +} + /** * Blanks out comments, leaving everything else at its original offset. * @@ -54,8 +97,9 @@ export const MIGRATIONS_DIR = 'packages/persistence/migrations'; * the first because the escape is consumed, the second because the closing quote immediately reopens * a new string. * - * @param {string} text - * @param {'ts' | 'sql'} dialect + * @param {string} text The raw source code to strip. + * @param {'ts' | 'sql'} dialect The language dialect determining comment syntax. + * @returns {string} Comment-stripped source code with preserved offsets. */ export function stripComments(text, dialect) { const lineMarker = dialect === 'sql' ? '--' : '//'; @@ -85,6 +129,20 @@ export function stripComments(text, dialect) { continue; } + if (dialect === 'sql' && ch === '$') { + const delim = readDollarQuoteDelimiter(text, i); + if (delim !== null) { + const tag = delim.delimiter; + const endStr = text.indexOf(tag, delim.end); + if (endStr !== -1) { + const fullDollar = text.slice(i, endStr + tag.length); + out += fullDollar; + i += fullDollar.length; + continue; + } + } + } + if (ch === lineMarker[0] && next === lineMarker[1]) { while (i < text.length && text[i] !== '\n') { out += ' '; @@ -109,6 +167,222 @@ export function stripComments(text, dialect) { return out; } +/** + * Token represents a lexical unit extracted from SQL source text. + * @typedef {Object} SqlToken + * @property {'word' | 'ident' | 'string' | 'punct'} type The syntactic category of the token. + * @property {string} value The normalized token value (lowercase for identifiers/keywords). + * @property {string} raw The verbatim token text from source. + * @property {number} pos The starting character offset in the source. + */ + +/** + * Tokenizes SQL source into a flat array of lexical tokens. + * + * Correctly distinguishes single-quoted strings (with doubled quote escaping `''`), + * double-quoted identifiers (with doubled quote escaping `""`), keywords/unquoted words, + * and punctuation tokens. + * + * @param {string} sql Comment-stripped SQL text. + * @returns {SqlToken[]} Array of SQL tokens. + */ +export function tokenizeSql(sql) { + /** @type {SqlToken[]} */ + const tokens = []; + let i = 0; + while (i < sql.length) { + const ch = sql[i]; + + if (/\s/.test(ch)) { + i++; + continue; + } + + if (ch === '$') { + const delim = readDollarQuoteDelimiter(sql, i); + if (delim !== null) { + const tag = delim.delimiter; + const start = i; + const endStr = sql.indexOf(tag, delim.end); + if (endStr === -1) { + throw new Error(`unterminated dollar-quoted string at position ${start}`); + } + const raw = sql.slice(start, endStr + tag.length); + const body = sql.slice(start + tag.length, endStr); + tokens.push({ + type: 'string', + value: body, + raw: raw, + pos: start, + }); + i = endStr + tag.length; + continue; + } + } + + if (ch === "'") { + const start = i; + let val = ''; + i++; + while (i < sql.length) { + if (sql[i] === "'") { + if (sql[i + 1] === "'") { + val += "'"; + i += 2; + continue; + } + i++; + break; + } + val += sql[i]; + i++; + } + tokens.push({ + type: 'string', + value: val, + raw: sql.slice(start, i), + pos: start, + }); + continue; + } + + if (ch === '"') { + const start = i; + let val = ''; + i++; + while (i < sql.length) { + if (sql[i] === '"') { + if (sql[i + 1] === '"') { + val += '"'; + i += 2; + continue; + } + i++; + break; + } + val += sql[i]; + i++; + } + tokens.push({ + type: 'ident', + value: val, + raw: sql.slice(start, i), + pos: start, + }); + continue; + } + + if (/[a-zA-Z_]/.test(ch)) { + const start = i; + while (i < sql.length && /[a-zA-Z0-9_$]/.test(sql[i])) { + i++; + } + const word = sql.slice(start, i); + tokens.push({ + type: 'word', + value: word.toLowerCase(), + raw: word, + pos: start, + }); + continue; + } + + // Punctuation and operators (;, ,, (, ), ., *, |, =, <, >, :, +, -, etc.) + tokens.push({ + type: 'punct', + value: ch, + raw: ch, + pos: i, + }); + i++; + } + return tokens; +} + +/** + * Splits a stream of SQL tokens into individual statements delimited by top-level semicolons. + * + * @param {SqlToken[]} tokens Array of SQL tokens. + * @returns {SqlToken[][]} Array of statement token arrays. + */ +export function splitSqlStatements(tokens) { + const statements = []; + let current = []; + for (const token of tokens) { + if (token.type === 'punct' && token.value === ';') { + if (current.length > 0) { + statements.push(current); + current = []; + } + } else { + current.push(token); + } + } + if (current.length > 0) { + statements.push(current); + } + return statements; +} + +/** + * Parses a table reference from a token stream starting at `startIndex`. + * Handles optional `ONLY` and optional `schema.` qualifiers. + * + * @param {SqlToken[]} tokens Array of SQL tokens. + * @param {number} startIndex Position in token stream to begin parsing table reference. + * @returns {{ schema: string, table: string, nextIndex: number } | null} Parsed table reference or null if invalid. + */ +export function parseQualifiedTableTarget(tokens, startIndex) { + let idx = startIndex; + if (tokens[idx]?.value === 'only') idx++; + + if (!tokens[idx] || (tokens[idx].type !== 'word' && tokens[idx].type !== 'ident')) { + return null; + } + + const firstIdent = tokens[idx].value; + idx++; + + if (tokens[idx]?.type === 'punct' && tokens[idx].value === '.') { + idx++; + if (!tokens[idx] || (tokens[idx].type !== 'word' && tokens[idx].type !== 'ident')) { + return null; + } + const secondIdent = tokens[idx].value; + idx++; + return { schema: firstIdent, table: secondIdent, nextIndex: idx }; + } + + return { schema: 'public', table: firstIdent, nextIndex: idx }; +} + +/** + * Computes a collision-free structured tuple key for a table reference. + * + * @param {{ schema?: string, table: string } | null} ref Parsed table reference. + * @returns {string} Structured key encoding [schema, table]. + */ +export function tableKey(ref) { + if (ref === null) return ''; + return JSON.stringify([ref.schema || 'public', ref.table]); +} + +export const STUDIES_TABLE_KEY = tableKey({ schema: 'public', table: 'studies' }); +export const VARIANTS_TABLE_KEY = tableKey({ schema: 'public', table: 'variants' }); + +/** + * Determines if a parsed table reference matches a specified table and default schema. + * + * @param {{ schema?: string, table: string } | null} ref Parsed table reference. + * @param {string} targetTable Expected table name. + * @param {string} [targetSchema='public'] Expected schema name (defaults to 'public'). + * @returns {boolean} True if the table reference matches the target. + */ +function isTableTarget(ref, targetTable, targetSchema = 'public') { + if (ref === null) return false; + return tableKey(ref) === tableKey({ schema: targetSchema, table: targetTable }); +} + /** * Pulls the quoted variant codes out of one region of a source file. * @@ -119,7 +393,8 @@ export function stripComments(text, dialect) { * A region that does not match is a hard failure, never an empty list. A guard that silently starts * checking nothing after a rename is worse than no guard, because the green tick still gets trusted. * - * @param {{label: string, file: string, open: RegExp, close: RegExp, dialect?: 'ts' | 'sql', text?: string}} spec + * @param {{label: string, file: string, open: RegExp, close: RegExp, dialect?: 'ts' | 'sql', text?: string}} spec Target region specification. + * @returns {{ label: string, file: string, variants: string[] }} Extracted variant list. */ export function extractRegion({ label, file, open, close, dialect = 'ts', text }) { const source = stripComments(text ?? readFileSync(file, 'utf8'), dialect); @@ -151,6 +426,9 @@ export function extractRegion({ label, file, open, close, dialect = 'ts', text } * names this repository uses the two orders coincide, but `9_x.sql` and `10_y.sql` would apply * as `10` then `9`, and a guard that sorted numerically would disagree with the schema on disk. * Fidelity to the runner is the invariant, not numeric intuition. + * + * @param {string} [dir=MIGRATIONS_DIR] Directory containing migration SQL files. + * @returns {string[]} Sorted migration file names. */ export function migrationFiles(dir = MIGRATIONS_DIR) { return readdirSync(dir) @@ -162,6 +440,9 @@ export function migrationFiles(dir = MIGRATIONS_DIR) { * Splits SQL into statements, respecting string literals so a `;` inside one does not end one. * * Comment stripping runs first, so only quotes are left to worry about. + * + * @param {string} sql Comment-stripped SQL text. + * @returns {string[]} Individual SQL statements. */ export function splitStatements(sql) { const statements = []; @@ -196,9 +477,14 @@ export function splitStatements(sql) { * migration and 0001 can never change. Any statement that mutates the table in a way this does not * model is a hard failure: quietly returning a set that ignores a `DELETE` would be a guard * confidently reporting the wrong schema. + * + * @param {string} [dir=MIGRATIONS_DIR] Path to the directory containing migration files. + * @returns {string[]} Array of variant codes present in the lookup table. + * @throws {Error} If unmodelled mutations (DELETE/UPDATE) or no INSERT statements are found. */ export function effectiveLookupVariants(dir = MIGRATIONS_DIR) { const codes = []; + const presentCodes = new Set(); for (const file of migrationFiles(dir)) { const sql = stripComments(readFileSync(join(dir, file), 'utf8'), 'sql'); @@ -211,10 +497,16 @@ export function effectiveLookupVariants(dir = MIGRATIONS_DIR) { ); } - for (const insert of sql.matchAll(/INSERT\s+INTO\s+variants\s*\([^)]*\)\s*VALUES([\s\S]*?);/gi)) { + for (const insert of sql.matchAll(/INSERT\s+INTO\s+variants\s*\([^)]*\)\s*VALUES[\s\S]*?;/gi)) { + const idempotentReseed = /\bON\s+CONFLICT\s*\(\s*code\s*\)\s+DO\s+NOTHING\b/i.test(insert[0]); // The first column of each tuple is `code`; the second is a display name that is capitalised // or hyphenated, so taking the leading element of each `(...)` keeps them apart reliably. - for (const tuple of insert[1].matchAll(/\(\s*'([^']+)'/g)) codes.push(tuple[1]); + for (const tuple of insert[0].matchAll(/\(\s*'([^']+)'/g)) { + const code = tuple[1]; + if (idempotentReseed && presentCodes.has(code)) continue; + codes.push(code); + presentCodes.add(code); + } } } if (codes.length === 0) throw new Error(`no INSERT INTO variants found under ${dir}`); @@ -222,91 +514,1113 @@ export function effectiveLookupVariants(dir = MIGRATIONS_DIR) { } /** - * How `studies.variant` is constrained after every migration has run. + * Allocates the next implicit constraint name following PostgreSQL's naming convention. + * + * If the candidate base name is free on the table, it is chosen. If already occupied anywhere + * in the table's constraint namespace, the lowest available positive integer suffix is appended. + * + * @param {Set} constraintNamespace The set of all constraint names currently active on the table. + * @param {string} baseName The base constraint name (e.g. 'studies_variant_check' or 'studies_variant_fkey'). + * @returns {string} The allocated unique constraint name. + */ +function nextImplicitConstraintName(constraintNamespace, baseName) { + if (!constraintNamespace.has(baseName)) { + return baseName; + } + let suffix = 1; + while (constraintNamespace.has(`${baseName}${suffix}`)) { + suffix++; + } + return `${baseName}${suffix}`; +} + +/** + * Splits action clauses of an ALTER TABLE statement by comma at parenthesis nesting depth 0. * - * The last migration to define the constraint wins, so replacing it (`DROP CONSTRAINT ... , - * ADD CONSTRAINT ... CHECK (...)`) is a supported forward change rather than a reason to edit 0022. + * @param {SqlToken[]} tokens Action tokens following the table target. + * @returns {SqlToken[][]} Array of token arrays, one per action clause. + */ +function splitAlterActions(tokens) { + const actions = []; + let current = []; + let depth = 0; + for (const token of tokens) { + if (token.type === 'punct') { + if (token.value === '(') depth++; + else if (token.value === ')') depth--; + else if (token.value === ',' && depth === 0) { + if (current.length > 0) { + actions.push(current); + current = []; + } + continue; + } + } + current.push(token); + } + if (current.length > 0) { + actions.push(current); + } + return actions; +} + +/** + * Parses a strict IN-list of string literals: `('literal1', 'literal2', ...)`. * - * A migration that swaps the `CHECK` for `REFERENCES variants(code)` — the conversion recorded as a - * candidate in PROJECT_STATE — makes the column derive from the lookup table, at which point there - * is no separate list left to drift. That returns `null`, and the caller skips the mirror. + * @param {SqlToken[]} tokens Array of tokens. + * @param {number} startIndex Index of first token inside the `IN (` list. + * @returns {{ variants: string[], nextIndex: number } | null} Parsed variants and next token index, or null if malformed. */ +function parseStrictVariantInList(tokens, startIndex) { + let idx = startIndex; + const variants = []; + let expectLiteral = true; + + while (idx < tokens.length) { + const t = tokens[idx]; + if (expectLiteral) { + if (t.type === 'string') { + variants.push(t.value); + expectLiteral = false; + idx++; + } else { + return null; + } + } else { + if (t.type === 'punct' && t.value === ',') { + expectLiteral = true; + idx++; + } else if (t.type === 'punct' && t.value === ')') { + return { variants, nextIndex: idx + 1 }; + } else { + return null; + } + } + } + + return null; +} + /** - * `CREATE TABLE studies` / `ALTER TABLE studies`, and nothing else. + * Parses the ALTER TABLE suffix allowed after a table-level CHECK constraint. * - * `IF EXISTS` and `IF NOT EXISTS` are both accepted because both are valid PostgreSQL and a - * migration is exactly where the defensive form gets written. Skipping `ALTER TABLE IF EXISTS - * studies` would leave the guard comparing against a constraint that statement had just replaced. - * Raised in the CodeRabbit review of PR #141. + * `NO INHERIT` belongs to the CHECK definition, while `NOT VALID` belongs to the ALTER TABLE + * action. `NOT ENFORCED` is deliberately rejected because it would not protect new writes. + * + * @param {SqlToken[]} tokens Tokens following the CHECK expression. + * @param {string} file Current migration filename. + * @param {string} target Human-readable constrained column. + * @returns {boolean} Whether the constraint was added NOT VALID. */ -const TARGETS_STUDIES = - /^\s*(?:CREATE\s+TABLE(?:\s+IF\s+NOT\s+EXISTS)?|ALTER\s+TABLE(?:\s+IF\s+EXISTS)?)\s+(?:ONLY\s+)?"?studies"?[\s(]/i; +function parseCheckAddSuffix(tokens, file, target) { + let idx = 0; + if (tokens[idx]?.value === 'no' && tokens[idx + 1]?.value === 'inherit') idx += 2; + if (tokens[idx]?.value === 'enforced') idx++; + if (tokens[idx]?.value === 'not' && tokens[idx + 1]?.value === 'enforced') { + throw new Error(`${file} adds a NOT ENFORCED constraint on \`${target}\`, which cannot protect writes.`); + } + const addedNotValid = tokens[idx]?.value === 'not' && tokens[idx + 1]?.value === 'valid'; + if (addedNotValid) idx += 2; + if (idx !== tokens.length) { + throw new Error(`${file} adds an unsupported suffix to the \`${target}\` CHECK constraint.`); + } + return addedNotValid; +} + +const INLINE_CONSTRAINT_STARTERS = new Set([ + 'check', + 'constraint', + 'generated', + 'primary', + 'references', + 'unique', +]); /** - * The name PostgreSQL gives a `CHECK` on `studies.variant` that was written without one. + * Rejects an inline constraint that PostgreSQL marks NOT ENFORCED without consuming later constraints. * - * `__check` is the server's own convention, so this is what a later migration has to - * name in its `DROP CONSTRAINT` — which makes it the right default to track against. + * @param {SqlToken[]} tokens Full column-definition tokens. + * @param {number} startIndex First token after the inline constraint. + * @param {string} file Current migration filename. + * @param {string} target Human-readable constrained column. */ -const IMPLICIT_CONSTRAINT_NAME = 'studies_variant_check'; +function assertInlineConstraintEnforced(tokens, startIndex, file, target) { + for (let idx = startIndex; idx < tokens.length; idx++) { + const token = tokens[idx]?.value; + const next = tokens[idx + 1]?.value; + const previous = tokens[idx - 1]?.value; + if (token === 'not' && next === 'enforced') { + throw new Error(`${file} adds a NOT ENFORCED constraint on \`${target}\`, which cannot protect writes.`); + } + if ( + INLINE_CONSTRAINT_STARTERS.has(token) || + (token === 'not' && next === 'null') || + ((token === 'default' || token === 'null') && previous !== 'set') + ) return; + } +} -/** A `CHECK (variant IN (...))`, with the constraint name when the statement gives one. */ -const VARIANT_CHECK = /(?:CONSTRAINT\s+"?(\w+)"?\s+)?CHECK\s*\(\s*variant\s+IN\s*\(([\s\S]*?)\)\s*\)/gi; +/** + * Parses PostgreSQL's optional table-level foreign-key attributes and trailing NOT VALID marker. + * + * The parser stays fail-closed: every token must belong to a supported MATCH, referential-action, + * deferrability, timing, enforcement, or validation clause. `NOT ENFORCED` is rejected because the + * parity invariant requires the FK to protect new writes. + * + * @param {SqlToken[]} tokens Tokens following the referenced column list. + * @param {string} file Current migration filename. + * @returns {boolean} Whether the foreign key was added NOT VALID. + */ +function parseForeignKeyAddSuffix(tokens, file) { + let idx = 0; + let addedNotValid = false; + const seen = new Set(); -const normalise = (name) => name.replace(/"/g, '').toLowerCase(); + while (idx < tokens.length) { + const token = tokens[idx]?.value; + let clause = null; + + if (token === 'match') { + clause = 'match'; + if (!['full', 'partial', 'simple'].includes(tokens[idx + 1]?.value)) break; + idx += 2; + } else if (token === 'on' && ['delete', 'update'].includes(tokens[idx + 1]?.value)) { + clause = `on ${tokens[idx + 1].value}`; + idx += 2; + const action = tokens[idx]?.value; + if (action === 'no' && tokens[idx + 1]?.value === 'action') { + idx += 2; + } else if (action === 'restrict' || action === 'cascade') { + idx++; + } else if (action === 'set' && ['null', 'default'].includes(tokens[idx + 1]?.value)) { + idx += 2; + if (tokens[idx]?.value === '(') { + idx++; + let expectColumn = true; + while (idx < tokens.length && tokens[idx]?.value !== ')') { + const current = tokens[idx]; + if (expectColumn) { + if (current?.type !== 'word' && current?.type !== 'ident') break; + } else if (current?.value !== ',') { + break; + } + expectColumn = !expectColumn; + idx++; + } + if (expectColumn || tokens[idx]?.value !== ')') break; + idx++; + } + } else { + break; + } + } else if (token === 'deferrable') { + clause = 'deferrable'; + idx++; + } else if (token === 'not' && tokens[idx + 1]?.value === 'deferrable') { + clause = 'deferrable'; + idx += 2; + } else if (token === 'initially' && ['deferred', 'immediate'].includes(tokens[idx + 1]?.value)) { + clause = 'initially'; + idx += 2; + } else if (token === 'enforced') { + clause = 'enforced'; + idx++; + } else if (token === 'not' && tokens[idx + 1]?.value === 'enforced') { + throw new Error( + `${file} adds a NOT ENFORCED \`studies.variant\` foreign key, which cannot protect writes.`, + ); + } else if ( + token === 'not' && + tokens[idx + 1]?.value === 'valid' && + idx + 2 === tokens.length + ) { + clause = 'not valid'; + addedNotValid = true; + idx += 2; + } else { + break; + } + + if (seen.has(clause)) break; + seen.add(clause); + } + + if (idx !== tokens.length) { + throw new Error(`${file} adds an unsupported suffix to the \`studies.variant\` foreign key.`); + } + return addedNotValid; +} + +/** + * Scans a column definition clause for CHECK and REFERENCES constraints on variant. + * + * @param {SqlToken[]} clause Tokens making up the column definition. + * @param {string} file Current migration filename. + * @param {Set} constraintNamespace Set of active table constraint names. + * @param {Set} variantConstraints Set of constraint names dependent on the variant column. + * @param {Map} activeChecks Active CHECK map. + * @param {Set} activeFks Active foreign key set. + */ +function scanColumnConstraints(clause, file, constraintNamespace, variantConstraints, activeChecks, activeFks) { + for (let i = 0; i < clause.length; i++) { + // Check for inline CHECK (variant ...) + if (clause[i].value === 'check' && clause[i + 1]?.value === '(') { + let depth = 1; + let endIdx = i + 2; + while (endIdx < clause.length && depth > 0) { + if (clause[endIdx].value === '(') depth++; + else if (clause[endIdx].value === ')') depth--; + endIdx++; + } + const checkTokens = clause.slice(i + 2, endIdx - 1); + const referencesVariant = checkTokens.some((t) => (t.type === 'word' || t.type === 'ident') && t.value === 'variant'); + if (referencesVariant) { + assertInlineConstraintEnforced(clause, endIdx, file, 'studies.variant'); + const pIdx = i + 2; + if (clause[pIdx]?.value === 'variant' && clause[pIdx + 1]?.value === 'in' && clause[pIdx + 2]?.value === '(') { + let inlineName = null; + if (i >= 2 && clause[i - 2]?.value === 'constraint') { + inlineName = clause[i - 1]?.value; + } + const parsedIn = parseStrictVariantInList(clause, pIdx + 3); + if (parsedIn === null) { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + if (clause[parsedIn.nextIndex]?.value !== ')') { + throw new Error( + `${file} defines a compound or non-standard CHECK predicate on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard compound CHECK predicates ` + + `rather than ignoring suffix expressions.`, + ); + } + const name = inlineName ?? nextImplicitConstraintName(constraintNamespace, 'studies_variant_check'); + constraintNamespace.add(name); + variantConstraints.add(name); + activeChecks.set(name, { file, name, variants: parsedIn.variants }); + } else { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + } + } + + // Check for inline REFERENCES variants(code) + if (clause[i].value === 'references') { + const inlineRef = parseQualifiedTableTarget(clause, i + 1); + if (inlineRef && isTableTarget(inlineRef, 'variants')) { + const afterRef = inlineRef.nextIndex; + if (clause[afterRef]?.value === '(' && clause[afterRef + 1]?.value === 'code' && clause[afterRef + 2]?.value === ')') { + assertInlineConstraintEnforced(clause, afterRef + 3, file, 'studies.variant'); + let inlineName = null; + if (i >= 2 && clause[i - 2]?.value === 'constraint') { + inlineName = clause[i - 1]?.value; + } + const name = inlineName ?? nextImplicitConstraintName(constraintNamespace, 'studies_variant_fkey'); + constraintNamespace.add(name); + variantConstraints.add(name); + activeFks.add(name); + } + } + } + } +} + +/** + * Replays all migrations through a deterministic schema state machine to evaluate constraints on `studies.variant`. + * + * Tracks table drops (including multi-table and cascaded variants drops), table renames, column drops, + * column renames, explicit constraint additions/drops/renames, and implicit PostgreSQL constraint name allocation. + * + * @param {string} [dir=MIGRATIONS_DIR] Migrations directory path. + * @returns {{ + * check: { file: string, name: string, variants: string[] } | null, + * checks: Array<{ file: string, name: string, variants: string[] }>, + * hasForeignKey: boolean, + * foreignKeys: Array<{ name: string, file: string, validated: boolean, validatedFile: string | null }>, + * variantDomainChecks: Array<{ file: string, name: string, variants: string[], validated: boolean, addedNotValid: boolean, validatedFile: string | null }>, + * unsafeStudyConstraintTransition: boolean + * }} Effective variant-domain and studies constraint state. + */ +export function replayStudiesSchema(dir = MIGRATIONS_DIR) { + const canonicalVariants = extractRegion(ROOT).variants; + const canonicalVariantSet = new Set(canonicalVariants); + /** + * @type {Map, + * variantConstraints: Set, + * activeChecks: Map, + * activeCodeChecks: Map, + * activeFks: Set, + * validatedFks: Set, + * fkAddedFiles: Map, + * fkValidatedFiles: Map + * }>} + */ + const tables = new Map(); + let unsafeStudyConstraintTransition = false; + + /** Creates empty replay state for one PostgreSQL table. */ + function newTableState() { + return { + hasVariantColumn: false, + hasCodeColumn: false, + constraintNamespace: new Set(), + variantConstraints: new Set(), + activeChecks: new Map(), + activeCodeChecks: new Map(), + activeFks: new Set(), + validatedFks: new Set(), + fkAddedFiles: new Map(), + fkValidatedFiles: new Map(), + }; + } + + /** Whether the effective catalog has a validated CHECK equal to the canonical Variant domain. */ + function hasValidatedVariantDomain() { + const variantsTable = tables.get(VARIANTS_TABLE_KEY); + return variantsTable !== undefined && + Array.from(variantsTable.activeCodeChecks.values()).some( + (check) => check.validated && disagreements(canonicalVariants, check.variants).length === 0, + ); + } + + /** Whether an active studies CHECK excludes every non-canonical variant. */ + function hasSafeStudyCheck(table) { + return Array.from(table.activeChecks.values()).some( + (check) => + check.variants.length > 0 && check.variants.every((variant) => canonicalVariantSet.has(variant)), + ); + } + + /** Returns existing replay state for a table or creates it on first use. */ + function getOrCreateTable(key) { + let t = tables.get(key); + if (!t) { + t = newTableState(); + tables.set(key, t); + } + return t; + } -export function effectiveStudyVariantConstraint(dir = MIGRATIONS_DIR) { - let current = null; for (const file of migrationFiles(dir)) { - const sql = stripComments(readFileSync(join(dir, file), 'utf8'), 'sql'); + const rawSql = readFileSync(join(dir, file), 'utf8'); + const stripped = stripComments(rawSql, 'sql'); + const tokens = tokenizeSql(stripped); + const statements = splitSqlStatements(tokens); - // Per statement, and only statements that name `studies` as their table. Testing the file as a - // whole let any other table move the answer: one migration that touches `studies` and also gives - // some other table its own `variant` CHECK would have overwritten this, and a `REFERENCES - // variants(code)` elsewhere in the same file — which is how games and ratings are already - // declared — would have cleared it, silently skipping the mirror the guard exists to compare. - // Raised in the CodeRabbit review of PR #141. - for (const statement of splitStatements(sql)) { - if (!TARGETS_STUDIES.test(statement)) continue; - - // A rename would leave every name tracked below pointing at something that no longer answers - // to it, and the drop that follows would look like an unrelated constraint. There is no - // half-right answer available, so say so rather than report a schema that is not there. - const renamed = /RENAME\s+CONSTRAINT\s+"?(\w+)"?/i.exec(statement); - if (renamed !== null && current !== null && normalise(renamed[1]) === current.name) { + const doStatements = statements.filter((stmt) => stmt[0]?.value === 'do'); + if (doStatements.length > 0) { + const known = KNOWN_HISTORICAL_PROCEDURAL_MIGRATIONS.get(file); + const canonicalSql = rawSql.replace(/\r\n/g, '\n'); + const fileHash = createHash('sha256').update(canonicalSql, 'utf8').digest('hex'); + if ( + !known || + fileHash !== known.sha256 || + doStatements.length !== known.expectedDoCount + ) { throw new Error( - `${file} renames the constraint governing \`studies.variant\` ` + - `(\`${renamed[1]}\`). Teach this guard \`RENAME CONSTRAINT\` rather than leaving it ` + - `tracking a name nothing answers to.`, + `${file} contains an unsupported top-level PostgreSQL DO block. ` + + `The deterministic variant-parity replay cannot prove schema state across procedural execution. ` + + `Use declarative DDL or explicitly review and pin the immutable historical migration.`, ); } + } + + for (const stmt of statements) { + if (stmt.length === 0) continue; + if (stmt[0]?.value === 'do') continue; + if (stmt.length < 2) continue; + + // ----------------------------------------------------------------------- + // 1. DROP TABLE [IF EXISTS] [ONLY] table1 [, table2 ...] [CASCADE | RESTRICT] + // ----------------------------------------------------------------------- + if (stmt[0].value === 'drop' && stmt[1].value === 'table') { + let idx = 2; + if (stmt[idx]?.value === 'if' && stmt[idx + 1]?.value === 'exists') { + idx += 2; + } + + const hasCascade = stmt.some((t) => t.value === 'cascade'); + + while (idx < stmt.length) { + if (stmt[idx]?.value === 'cascade' || stmt[idx]?.value === 'restrict') { + break; + } + const ref = parseQualifiedTableTarget(stmt, idx); + if (ref === null) break; + idx = ref.nextIndex; + + const key = tableKey(ref); + + if (key === VARIANTS_TABLE_KEY && hasCascade) { + for (const tbl of tables.values()) { + for (const fkName of tbl.activeFks) { + tbl.constraintNamespace.delete(fkName); + tbl.variantConstraints.delete(fkName); + } + tbl.activeFks.clear(); + tbl.validatedFks.clear(); + tbl.fkAddedFiles.clear(); + tbl.fkValidatedFiles.clear(); + } + } - // Order matters: `DROP CONSTRAINT` then `ADD CONSTRAINT ... CHECK` is how a constraint is - // replaced without editing the migration that first defined it. The name is compared exactly - // against the one being tracked — a substring test for "variant" both missed a legitimately - // named constraint (`DROP CONSTRAINT allowed_codes`) and would have fired on an unrelated one - // that happened to contain the word. Raised in the CodeRabbit review of PR #141. - const dropped = /DROP\s+CONSTRAINT\s+(?:IF\s+EXISTS\s+)?"?(\w+)"?/i.exec(statement); - if (dropped !== null && current !== null && normalise(dropped[1]) === current.name) { - current = null; + if (key === STUDIES_TABLE_KEY || tables.has(key)) { + tables.delete(key); + } + + if (stmt[idx]?.type === 'punct' && stmt[idx].value === ',') { + idx++; + } else { + break; + } + } + continue; } - if (/DROP\s+COLUMN\s+(?:IF\s+EXISTS\s+)?"?variant"?/i.test(statement)) current = null; - - for (const m of statement.matchAll(VARIANT_CHECK)) { - current = { - file, - name: normalise(m[1] ?? IMPLICIT_CONSTRAINT_NAME), - variants: [...m[2].matchAll(/'([a-z0-9]+)'/g)].map((t) => t[1]), - }; + + // ----------------------------------------------------------------------- + // 2. ALTER TABLE [IF EXISTS] [ONLY] target ... + // ----------------------------------------------------------------------- + if (stmt[0].value === 'alter' && stmt[1].value === 'table') { + let idx = 2; + let isTableIfExists = false; + if (stmt[idx]?.value === 'if' && stmt[idx + 1]?.value === 'exists') { + isTableIfExists = true; + idx += 2; + } + + const ref = parseQualifiedTableTarget(stmt, idx); + if (ref === null) { + continue; + } + + const key = tableKey(ref); + if (key !== STUDIES_TABLE_KEY && key !== VARIANTS_TABLE_KEY && !tables.has(key)) { + continue; + } + + if (isTableIfExists && !tables.has(key)) { + continue; + } + + const currentTable = getOrCreateTable(key); + idx = ref.nextIndex; + + // Table rename: ALTER TABLE target RENAME TO new_name + if (stmt[idx]?.value === 'rename' && stmt[idx + 1]?.value === 'to') { + const newName = stmt[idx + 2]?.value; + if (newName) { + const newKey = tableKey({ schema: ref.schema || 'public', table: newName }); + tables.delete(key); + tables.set(newKey, currentTable); + } + continue; + } + + // Table schema move: ALTER TABLE target SET SCHEMA new_schema + if (stmt[idx]?.value === 'set' && stmt[idx + 1]?.value === 'schema') { + const newSchema = stmt[idx + 2]?.value; + if (newSchema) { + const newKey = tableKey({ schema: newSchema, table: ref.table }); + tables.delete(key); + tables.set(newKey, currentTable); + } + continue; + } + + const actionTokens = stmt.slice(idx); + const actionClauses = splitAlterActions(actionTokens); + + for (const action of actionClauses) { + if (action.length === 0) continue; + + // Action A: DROP CONSTRAINT [IF EXISTS] + if (action[0].value === 'drop' && action[1]?.value === 'constraint') { + let cIdx = 2; + if (action[cIdx]?.value === 'if' && action[cIdx + 1]?.value === 'exists') { + cIdx += 2; + } + if (action[cIdx]?.type === 'word' || action[cIdx]?.type === 'ident') { + const name = action[cIdx].value; + if ( + key === STUDIES_TABLE_KEY && + currentTable.activeChecks.has(name) && + !hasValidatedVariantDomain() + ) { + unsafeStudyConstraintTransition = true; + } + currentTable.constraintNamespace.delete(name); + currentTable.variantConstraints.delete(name); + currentTable.activeChecks.delete(name); + currentTable.activeCodeChecks.delete(name); + currentTable.activeFks.delete(name); + currentTable.validatedFks.delete(name); + currentTable.fkAddedFiles.delete(name); + currentTable.fkValidatedFiles.delete(name); + } + continue; + } + + // Action B: VALIDATE CONSTRAINT + if (action[0].value === 'validate' && action[1]?.value === 'constraint') { + const name = action[2]?.value; + const codeCheck = name === undefined ? undefined : currentTable.activeCodeChecks.get(name); + if (codeCheck !== undefined) { + codeCheck.validated = true; + codeCheck.validatedFile = file; + } + if (name !== undefined && currentTable.activeFks.has(name)) { + currentTable.validatedFks.add(name); + currentTable.fkValidatedFiles.set(name, file); + } + continue; + } + + // Action C: RENAME CONSTRAINT TO + if (action[0].value === 'rename' && action[1]?.value === 'constraint') { + const oldName = action[2]?.value; + if ( + oldName && + (currentTable.activeChecks.has(oldName) || currentTable.activeCodeChecks.has(oldName)) + ) { + throw new Error( + `${file} renames a constraint governing the variant domain ` + + `(\`${oldName}\`). Teach this guard \`RENAME CONSTRAINT\` rather than leaving it ` + + `tracking a name nothing answers to.`, + ); + } + if (oldName && action[3]?.value === 'to' && action[4]) { + const newName = action[4].value; + if (currentTable.constraintNamespace.has(oldName)) { + currentTable.constraintNamespace.delete(oldName); + currentTable.constraintNamespace.add(newName); + } + if (currentTable.variantConstraints.has(oldName)) { + currentTable.variantConstraints.delete(oldName); + currentTable.variantConstraints.add(newName); + } + if (currentTable.activeFks.has(oldName)) { + currentTable.activeFks.delete(oldName); + currentTable.activeFks.add(newName); + } + if (currentTable.validatedFks.has(oldName)) { + currentTable.validatedFks.delete(oldName); + currentTable.validatedFks.add(newName); + } + if (currentTable.fkAddedFiles.has(oldName)) { + currentTable.fkAddedFiles.set(newName, currentTable.fkAddedFiles.get(oldName)); + currentTable.fkAddedFiles.delete(oldName); + } + if (currentTable.fkValidatedFiles.has(oldName)) { + currentTable.fkValidatedFiles.set(newName, currentTable.fkValidatedFiles.get(oldName)); + currentTable.fkValidatedFiles.delete(oldName); + } + } + continue; + } + + // Action D: DROP [COLUMN] [IF EXISTS] + if (action[0].value === 'drop') { + let cIdx = 1; + if (action[cIdx]?.value === 'column') cIdx++; + if (action[cIdx]?.value === 'if' && action[cIdx + 1]?.value === 'exists') cIdx += 2; + if (action[cIdx]?.value === 'variant') { + currentTable.hasVariantColumn = false; + for (const name of currentTable.variantConstraints) { + currentTable.constraintNamespace.delete(name); + } + currentTable.variantConstraints.clear(); + currentTable.activeChecks.clear(); + currentTable.activeFks.clear(); + currentTable.validatedFks.clear(); + currentTable.fkAddedFiles.clear(); + currentTable.fkValidatedFiles.clear(); + } + if (action[cIdx]?.value === 'code') { + currentTable.hasCodeColumn = false; + for (const name of currentTable.activeCodeChecks.keys()) { + currentTable.constraintNamespace.delete(name); + } + currentTable.activeCodeChecks.clear(); + } + continue; + } + + // Action E: RENAME [COLUMN] TO + if (action[0].value === 'rename') { + let cIdx = 1; + if (action[cIdx]?.value === 'column') cIdx++; + if (action[cIdx]?.value === 'variant' && action[cIdx + 1]?.value === 'to') { + currentTable.hasVariantColumn = false; + currentTable.variantConstraints.clear(); + currentTable.activeChecks.clear(); + currentTable.activeFks.clear(); + currentTable.validatedFks.clear(); + currentTable.fkAddedFiles.clear(); + currentTable.fkValidatedFiles.clear(); + } + if (action[cIdx]?.value === 'code' && action[cIdx + 1]?.value === 'to') { + currentTable.hasCodeColumn = false; + currentTable.activeCodeChecks.clear(); + } + continue; + } + + // Action F: ADD [COLUMN] [IF NOT EXISTS] variant/code ... + let colIdx = 0; + if (action[colIdx]?.value === 'add') colIdx++; + if (action[colIdx]?.value === 'column') colIdx++; + let isColIfNotExists = false; + if (action[colIdx]?.value === 'if' && action[colIdx + 1]?.value === 'not' && action[colIdx + 2]?.value === 'exists') { + isColIfNotExists = true; + colIdx += 3; + } + if (action[colIdx]?.value === 'variant') { + if (currentTable.hasVariantColumn && isColIfNotExists) { + continue; + } + currentTable.hasVariantColumn = true; + const previousFks = new Set(currentTable.activeFks); + scanColumnConstraints(action.slice(colIdx), file, currentTable.constraintNamespace, currentTable.variantConstraints, currentTable.activeChecks, currentTable.activeFks); + for (const name of currentTable.activeFks) { + if (previousFks.has(name)) continue; + currentTable.validatedFks.add(name); + currentTable.fkAddedFiles.set(name, file); + currentTable.fkValidatedFiles.set(name, file); + if ( + key === STUDIES_TABLE_KEY && + !hasValidatedVariantDomain() && + !hasSafeStudyCheck(currentTable) + ) { + unsafeStudyConstraintTransition = true; + } + } + continue; + } + if (action[colIdx]?.value === 'code') { + if (currentTable.hasCodeColumn && isColIfNotExists) continue; + currentTable.hasCodeColumn = true; + continue; + } + + // Action G: Table-level ADD [CONSTRAINT ] CHECK or FOREIGN KEY + let explicitName = null; + let aIdx = 0; + if (action[aIdx]?.value === 'add') aIdx++; + if (action[aIdx]?.value === 'constraint') { + explicitName = action[aIdx + 1]?.value ?? null; + aIdx += 2; + } + + let handled = false; + + // Search for the table-level variants(code) domain CHECK. + if (key === VARIANTS_TABLE_KEY) { + for (let i = 0; i < action.length; i++) { + if (action[i].value !== 'check' || action[i + 1]?.value !== '(') continue; + let depth = 1; + let endIdx = i + 2; + while (endIdx < action.length && depth > 0) { + if (action[endIdx].value === '(') depth++; + else if (action[endIdx].value === ')') depth--; + endIdx++; + } + const checkTokens = action.slice(i + 2, endIdx - 1); + const referencesCode = checkTokens.some( + (token) => + (token.type === 'word' || token.type === 'ident') && token.value === 'code', + ); + if (!referencesCode) continue; + const predicateIndex = i + 2; + if ( + action[predicateIndex]?.value !== 'code' || + action[predicateIndex + 1]?.value !== 'in' || + action[predicateIndex + 2]?.value !== '(' + ) { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`variants.code\`. ` + + `Teach this guard that predicate rather than ignoring the domain constraint.`, + ); + } + const parsedIn = parseStrictVariantInList(action, predicateIndex + 3); + if (parsedIn === null || action[parsedIn.nextIndex]?.value !== ')') { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`variants.code\`. ` + + `Teach this guard that predicate rather than extracting a partial domain.`, + ); + } + const suffix = action.slice(parsedIn.nextIndex + 1); + const addedNotValid = parseCheckAddSuffix(suffix, file, 'variants.code'); + const name = + explicitName ?? nextImplicitConstraintName(currentTable.constraintNamespace, 'variants_code_check'); + currentTable.constraintNamespace.add(name); + currentTable.activeCodeChecks.set(name, { + file, + name, + variants: parsedIn.variants, + validated: !addedNotValid, + addedNotValid, + validatedFile: addedNotValid ? null : file, + }); + handled = true; + } + } + + // Search for table-level CHECK (variant ...) + for (let i = 0; i < action.length; i++) { + if (action[i].value === 'check' && action[i + 1]?.value === '(') { + let depth = 1; + let endIdx = i + 2; + while (endIdx < action.length && depth > 0) { + if (action[endIdx].value === '(') depth++; + else if (action[endIdx].value === ')') depth--; + endIdx++; + } + const checkTokens = action.slice(i + 2, endIdx - 1); + const referencesVariant = checkTokens.some((t) => (t.type === 'word' || t.type === 'ident') && t.value === 'variant'); + if (referencesVariant) { + const pIdx = i + 2; + if (action[pIdx]?.value === 'variant' && action[pIdx + 1]?.value === 'in' && action[pIdx + 2]?.value === '(') { + let name = explicitName; + if (name === null && i >= 2 && action[i - 2]?.value === 'constraint') { + name = action[i - 1]?.value ?? null; + } + const parsedIn = parseStrictVariantInList(action, pIdx + 3); + if (parsedIn === null) { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${action.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + if (action[parsedIn.nextIndex]?.value !== ')') { + throw new Error( + `${file} defines a compound or non-standard CHECK predicate on \`studies.variant\` ` + + `(\`${action.map((t) => t.raw).join(' ')}\`). Teach this guard compound CHECK predicates ` + + `rather than ignoring suffix expressions.`, + ); + } + parseCheckAddSuffix( + action.slice(parsedIn.nextIndex + 1), + file, + 'studies.variant', + ); + const assignedName = name ?? nextImplicitConstraintName(currentTable.constraintNamespace, 'studies_variant_check'); + currentTable.constraintNamespace.add(assignedName); + currentTable.variantConstraints.add(assignedName); + currentTable.activeChecks.set(assignedName, { file, name: assignedName, variants: parsedIn.variants }); + handled = true; + } else { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${action.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + } + } + } + + // Search for table-level FOREIGN KEY (variant) REFERENCES [public.]variants(code) + const fkIdx = action.findIndex((t) => t.value === 'foreign'); + if ( + fkIdx !== -1 && + action[fkIdx + 1]?.value === 'key' && + action[fkIdx + 2]?.value === '(' && + action[fkIdx + 3]?.value === 'variant' && + action[fkIdx + 4]?.value === ')' + ) { + let rIdx = fkIdx + 5; + if (action[rIdx]?.value === 'references') { + const refTarget = parseQualifiedTableTarget(action, rIdx + 1); + if (refTarget && isTableTarget(refTarget, 'variants')) { + const afterRef = refTarget.nextIndex; + if (action[afterRef]?.value === '(' && action[afterRef + 1]?.value === 'code' && action[afterRef + 2]?.value === ')') { + const name = explicitName ?? nextImplicitConstraintName(currentTable.constraintNamespace, 'studies_variant_fkey'); + currentTable.constraintNamespace.add(name); + currentTable.variantConstraints.add(name); + currentTable.activeFks.add(name); + currentTable.fkAddedFiles.set(name, file); + const suffix = action.slice(afterRef + 3); + const addedNotValid = parseForeignKeyAddSuffix(suffix, file); + if (!addedNotValid) { + currentTable.validatedFks.add(name); + currentTable.fkValidatedFiles.set(name, file); + } + if ( + key === STUDIES_TABLE_KEY && + !hasValidatedVariantDomain() && + !hasSafeStudyCheck(currentTable) + ) { + unsafeStudyConstraintTransition = true; + } + handled = true; + } + } + } + } + + if (handled) continue; + + // Register any other explicitly named constraint + if (explicitName !== null) { + currentTable.constraintNamespace.add(explicitName); + } + } + continue; } - // Once the column derives from the lookup table there is no second list left to drift. - if (/\bvariant\b[\s\S]*?REFERENCES\s+variants\s*\(\s*code\s*\)/i.test(statement)) { - current = null; + // ----------------------------------------------------------------------- + // 3. CREATE TABLE [IF NOT EXISTS] target (...) + // ----------------------------------------------------------------------- + if (stmt[0].value === 'create' && stmt[1].value === 'table') { + let idx = 2; + let isTableIfNotExists = false; + if (stmt[idx]?.value === 'if' && stmt[idx + 1]?.value === 'not' && stmt[idx + 2]?.value === 'exists') { + isTableIfNotExists = true; + idx += 3; + } + const ref = parseQualifiedTableTarget(stmt, idx); + if (ref === null) { + continue; + } + + const key = tableKey(ref); + if (key !== STUDIES_TABLE_KEY && key !== VARIANTS_TABLE_KEY && !tables.has(key)) { + continue; + } + + if (tables.has(key) && isTableIfNotExists) { + continue; + } + + const currentTable = newTableState(); + tables.set(key, currentTable); + + const openParen = stmt.findIndex((t) => t.type === 'punct' && t.value === '('); + if (openParen === -1) continue; + + let closeParen = -1; + let bodyDepth = 0; + for (let i = openParen; i < stmt.length; i++) { + if (stmt[i].type !== 'punct') continue; + if (stmt[i].value === '(') bodyDepth++; + else if (stmt[i].value === ')') { + bodyDepth--; + if (bodyDepth === 0) { + closeParen = i; + break; + } + } + } + if (closeParen === -1) { + throw new Error(`${file} has an unterminated CREATE TABLE column list.`); + } + + const bodyTokens = stmt.slice(openParen + 1, closeParen); + const clauses = splitAlterActions(bodyTokens); + + for (const clause of clauses) { + if (clause.length === 0) continue; + + // Column-level: variant ... + if (clause[0]?.value === 'variant') { + currentTable.hasVariantColumn = true; + scanColumnConstraints(clause, file, currentTable.constraintNamespace, currentTable.variantConstraints, currentTable.activeChecks, currentTable.activeFks); + continue; + } + if (clause[0]?.value === 'code') { + currentTable.hasCodeColumn = true; + continue; + } + + let explicitName = null; + let cIdx = 0; + if (clause[cIdx]?.value === 'constraint') { + explicitName = clause[cIdx + 1]?.value ?? null; + cIdx += 2; + } + + let handled = false; + + // Table-level CHECK (variant ...) + for (let i = 0; i < clause.length; i++) { + if (clause[i].value === 'check' && clause[i + 1]?.value === '(') { + let depth = 1; + let endIdx = i + 2; + while (endIdx < clause.length && depth > 0) { + if (clause[endIdx].value === '(') depth++; + else if (clause[endIdx].value === ')') depth--; + endIdx++; + } + const checkTokens = clause.slice(i + 2, endIdx - 1); + const referencesVariant = checkTokens.some((t) => (t.type === 'word' || t.type === 'ident') && t.value === 'variant'); + if (referencesVariant) { + const pIdx = i + 2; + if (clause[pIdx]?.value === 'variant' && clause[pIdx + 1]?.value === 'in' && clause[pIdx + 2]?.value === '(') { + let name = explicitName; + if (name === null && i >= 2 && clause[i - 2]?.value === 'constraint') { + name = clause[i - 1]?.value ?? null; + } + const parsedIn = parseStrictVariantInList(clause, pIdx + 3); + if (parsedIn === null) { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + if (clause[parsedIn.nextIndex]?.value !== ')') { + throw new Error( + `${file} defines a compound or non-standard CHECK predicate on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard compound CHECK predicates ` + + `rather than ignoring suffix expressions.`, + ); + } + const addedNotValid = parseCheckAddSuffix( + clause.slice(parsedIn.nextIndex + 1), + file, + 'studies.variant', + ); + if (addedNotValid) { + throw new Error( + `${file} adds NOT VALID to a \`studies.variant\` CHECK during CREATE TABLE.`, + ); + } + const assignedName = name ?? nextImplicitConstraintName(currentTable.constraintNamespace, 'studies_variant_check'); + currentTable.constraintNamespace.add(assignedName); + currentTable.variantConstraints.add(assignedName); + currentTable.activeChecks.set(assignedName, { file, name: assignedName, variants: parsedIn.variants }); + handled = true; + } else { + throw new Error( + `${file} defines an unsupported CHECK predicate shape on \`studies.variant\` ` + + `(\`${clause.map((t) => t.raw).join(' ')}\`). Teach this guard non-standard CHECK predicates ` + + `rather than ignoring the constraint.`, + ); + } + } + } + } + + // Table-level FOREIGN KEY (variant) REFERENCES [public.]variants(code) + const fkIdx = clause.findIndex((t) => t.value === 'foreign'); + if ( + fkIdx !== -1 && + clause[fkIdx + 1]?.value === 'key' && + clause[fkIdx + 2]?.value === '(' && + clause[fkIdx + 3]?.value === 'variant' && + clause[fkIdx + 4]?.value === ')' + ) { + let rIdx = fkIdx + 5; + if (clause[rIdx]?.value === 'references') { + const refTarget = parseQualifiedTableTarget(clause, rIdx + 1); + if (refTarget && isTableTarget(refTarget, 'variants')) { + const afterRef = refTarget.nextIndex; + if (clause[afterRef]?.value === '(' && clause[afterRef + 1]?.value === 'code' && clause[afterRef + 2]?.value === ')') { + const name = explicitName ?? nextImplicitConstraintName(currentTable.constraintNamespace, 'studies_variant_fkey'); + currentTable.constraintNamespace.add(name); + currentTable.variantConstraints.add(name); + currentTable.activeFks.add(name); + handled = true; + } + } + } + } + + if (handled) continue; + + if (explicitName !== null) { + currentTable.constraintNamespace.add(explicitName); + } + } + + if (key === STUDIES_TABLE_KEY) { + for (const name of currentTable.activeFks) { + currentTable.validatedFks.add(name); + currentTable.fkAddedFiles.set(name, file); + currentTable.fkValidatedFiles.set(name, file); + } + if ( + currentTable.activeFks.size > 0 && + !hasValidatedVariantDomain() && + !hasSafeStudyCheck(currentTable) + ) { + unsafeStudyConstraintTransition = true; + } + } } } } - return current; + + const studiesTable = tables.get(STUDIES_TABLE_KEY); + const variantsTable = tables.get(VARIANTS_TABLE_KEY); + const variantDomainChecks = + variantsTable === undefined ? [] : Array.from(variantsTable.activeCodeChecks.values()); + const foreignKeys = + studiesTable === undefined + ? [] + : Array.from(studiesTable.activeFks).map((name) => ({ + name, + file: studiesTable.fkAddedFiles.get(name) ?? '', + validated: studiesTable.validatedFks.has(name), + validatedFile: studiesTable.fkValidatedFiles.get(name) ?? null, + })); + + if (!studiesTable) { + return { + check: null, + checks: [], + hasForeignKey: false, + foreignKeys, + variantDomainChecks, + unsafeStudyConstraintTransition, + }; + } + + let latestCheck = null; + for (const item of studiesTable.activeChecks.values()) { + latestCheck = item; + } + + return { + check: latestCheck, + checks: Array.from(studiesTable.activeChecks.values()), + hasForeignKey: studiesTable.activeFks.size > 0, + foreignKeys, + variantDomainChecks, + unsafeStudyConstraintTransition, + }; +} + +/** + * Replays all migrations to compute the effective CHECK constraint governing `studies.variant`. + * + * @param {string} [dir=MIGRATIONS_DIR] Path to the migrations directory. + * @returns {{ file: string, name: string, variants: string[] } | null} The active CHECK constraint or null if none exists. + */ +export function effectiveStudyVariantConstraint(dir = MIGRATIONS_DIR) { + return replayStudiesSchema(dir).check; +} + +/** + * Replays all migrations to determine whether `studies.variant` has an active foreign key + * referencing `variants(code)`. + * + * @param {string} dir The migrations directory to replay. + * @returns {boolean} Whether studies.variant has an active foreign key referencing variants(code). + */ +export function effectiveStudyVariantForeignKey(dir = MIGRATIONS_DIR) { + return replayStudiesSchema(dir).hasForeignKey; } /** The root. Everything else is measured against this one. */ @@ -368,22 +1682,110 @@ export function collectMirrors(dir = MIGRATIONS_DIR) { file: dir, variants: effectiveLookupVariants(dir), }); - const study = effectiveStudyVariantConstraint(dir); - if (study !== null) { + const replayed = replayStudiesSchema(dir); + for (const checkItem of replayed.variantDomainChecks) { mirrors.push({ - label: '`studies.variant` CHECK constraint, after all migrations', - file: join(dir, study.file), - variants: study.variants, + label: `\`variants.code\` CHECK constraint (${checkItem.name}), after all migrations`, + file: join(dir, checkItem.file), + variants: checkItem.variants, }); } - return { mirrors, studyConstraint: study }; + return { + mirrors, + studyConstraint: replayed.check, + studyChecks: replayed.checks, + hasStudyVariantFk: replayed.hasForeignKey, + studyForeignKeys: replayed.foreignKeys, + variantDomainConstraint: + replayed.variantDomainChecks.length === 1 ? replayed.variantDomainChecks[0] : null, + variantDomainConstraints: replayed.variantDomainChecks, + unsafeStudyConstraintTransition: replayed.unsafeStudyConstraintTransition, + }; } -function main() { +/** + * Evaluates variant parity across TypeScript mirrors and SQL migrations. + * + * @param {string} [dir=MIGRATIONS_DIR] Path to the migrations directory. + * @returns {{ + * failures: string[], + * mirrors: Array<{ label: string, file: string, variants: string[] }>, + * studyConstraint: { file: string, name: string, variants: string[] } | null, + * hasStudyVariantFk: boolean + * }} Parity evaluation results and any failure descriptions. + */ +export function evaluateParity(dir = MIGRATIONS_DIR) { const root = extractRegion(ROOT); - const { mirrors, studyConstraint } = collectMirrors(); + const collected = collectMirrors(dir); + const { + mirrors, + studyConstraint, + studyChecks, + hasStudyVariantFk, + studyForeignKeys, + variantDomainConstraint, + variantDomainConstraints, + unsafeStudyConstraintTransition, + } = collected; const failures = []; + for (const mirror of mirrors) { + const problems = disagreements(root.variants, mirror.variants); + if (problems.length > 0) { + failures.push(`${mirror.label} (${mirror.file}): ${problems.join('; ')}`); + } + } + + if (variantDomainConstraints.length === 0) { + failures.push('`variants.code` has no CHECK constraint enforcing the canonical variant domain'); + } else if (variantDomainConstraints.length > 1) { + failures.push('`variants.code` has multiple active domain CHECK constraints'); + } + if (variantDomainConstraint !== null) { + if (!variantDomainConstraint.addedNotValid) { + failures.push('`variants.code` domain CHECK was not added with `NOT VALID`'); + } + if (!variantDomainConstraint.validated) { + failures.push('`variants.code` domain CHECK is not validated'); + } else if (variantDomainConstraint.validatedFile === variantDomainConstraint.file) { + failures.push('`variants.code` domain CHECK must be validated in a later migration'); + } + } + + if (studyChecks.length > 0) { + failures.push( + `\`studies.variant\` retains obsolete CHECK constraint(s): ${studyChecks.map((check) => check.name).join(', ')}`, + ); + } + if (!hasStudyVariantFk) { + failures.push('`studies.variant` has no foreign key referencing `variants(code)`'); + } else if (studyForeignKeys.length > 1) { + failures.push('`studies.variant` has multiple active foreign keys referencing `variants(code)`'); + } else if (studyForeignKeys[0]?.validated !== true) { + failures.push('`studies.variant` foreign key referencing `variants(code)` is not validated'); + } else if (studyForeignKeys[0]?.validatedFile === studyForeignKeys[0]?.file) { + failures.push('`studies.variant` foreign key must be validated in a later migration'); + } + if (unsafeStudyConstraintTransition) { + failures.push( + '`studies.variant` protection changed before the variants domain CHECK was validated', + ); + } + + return { failures, ...collected }; +} + +/** Prints the parity report and exits unsuccessfully when any invariant disagrees. */ +function main() { + const root = extractRegion(ROOT); + const { + failures, + mirrors, + studyConstraint, + hasStudyVariantFk, + variantDomainConstraint, + } = evaluateParity(); + console.log(`root: ${root.label} (${root.file})`); console.log(` ${root.variants.join(', ')}\n`); @@ -393,16 +1795,26 @@ function main() { console.log(` ok ${mirror.label}`); } else { console.log(` FAIL ${mirror.label} (${mirror.file}): ${problems.join('; ')}`); - failures.push(mirror.label); } } + if (variantDomainConstraint !== null && variantDomainConstraint.validated) { + console.log(' ok `variants.code` domain CHECK is validated'); + } + if (studyConstraint === null) { - console.log( - ' -- `studies.variant` has no CHECK left; it derives from `variants(code)`, nothing to compare', - ); + if (hasStudyVariantFk) { + console.log( + ' -- `studies.variant` has no CHECK left; it derives from `variants(code)`, nothing to compare', + ); + } } + const invariantFailures = failures.filter( + (failure) => !mirrors.some((mirror) => failure.startsWith(`${mirror.label} (`)), + ); + for (const failure of invariantFailures) console.log(` FAIL ${failure}`); + if (failures.length > 0) { console.log( `\nThe supported-variant list disagrees with ${root.label} in ${failures.length} place(s).\n` + @@ -427,3 +1839,4 @@ if (process.argv[1] !== undefined && import.meta.url === pathToFileURL(process.a process.exit(1); } } + diff --git a/scripts/test/check-variant-parity.test.mjs b/scripts/test/check-variant-parity.test.mjs index 80573815..4449ac5e 100644 --- a/scripts/test/check-variant-parity.test.mjs +++ b/scripts/test/check-variant-parity.test.mjs @@ -12,16 +12,30 @@ import { mkdtempSync, writeFileSync, readFileSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; +import { createHash } from 'node:crypto'; import { stripComments, splitStatements, + tokenizeSql, + splitSqlStatements, + parseQualifiedTableTarget, + readDollarQuoteDelimiter, + replayStudiesSchema, extractRegion, migrationFiles, effectiveLookupVariants, effectiveStudyVariantConstraint, + effectiveStudyVariantForeignKey, + collectMirrors, + evaluateParity, + tableKey, + STUDIES_TABLE_KEY, + VARIANTS_TABLE_KEY, + KNOWN_HISTORICAL_PROCEDURAL_MIGRATIONS, disagreements, ROOT, TS_MIRRORS, + MIGRATIONS_DIR, } from '../check-variant-parity.mjs'; /** A throwaway migration directory. Files are named so the runner's ordering applies. */ @@ -35,6 +49,7 @@ test('a commented-out variant does not count as present', () => { // The defect this replaced: quoted tokens were matched in raw source, so commenting an entry out // left the guard green while the executable array no longer held it. Raised in the Qodo review of // PR #141. + /** Extracts variants from a synthetic TypeScript declaration body. */ const region = (body) => extractRegion({ label: 'test', @@ -77,6 +92,24 @@ INSERT INTO variants (code, name) VALUES } }); +test('an idempotent canonical re-seed preserves lookup set semantics', () => { + const dir = migrations({ + '0001_init.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY, name TEXT NOT NULL); +INSERT INTO variants (code, name) VALUES + ('standard', 'Standard'), + ('atomic', 'Atomic');`, + '0028_reseed.sql': `INSERT INTO variants (code, name) VALUES + ('standard', 'Standard'), + ('atomic', 'Atomic') +ON CONFLICT (code) DO NOTHING;`, + }); + try { + assert.deepEqual(effectiveLookupVariants(dir), ['standard', 'atomic']); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + test('migrations replay in the order the runner applies them, which is lexicographic', () => { // `pg/migrate.ts` sorts with a plain `.sort()`, so that is the order the database ends up in and // the order this must model. Zero-padded names make lexicographic and numeric order coincide, so @@ -307,6 +340,533 @@ ALTER TABLE studies ADD CONSTRAINT studies_variant_fk } }); +test('the committed migrations directory leaves studies.variant derived from foreign key with no CHECK', () => { + assert.equal(effectiveStudyVariantConstraint(MIGRATIONS_DIR), null); + assert.equal(effectiveStudyVariantForeignKey(MIGRATIONS_DIR), true); +}); + +test('dropping the CHECK without adding a foreign key leaves effectiveStudyVariantForeignKey false', () => { + const dir = migrations({ + '0022_study_variant.sql': `ALTER TABLE studies + ADD COLUMN variant TEXT NOT NULL DEFAULT 'standard' + CHECK (variant IN ('standard', 'atomic'));`, + '0025_drop_only.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check;`, + }); + try { + assert.equal(effectiveStudyVariantConstraint(dir), null); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks named foreign keys through drop and rename', () => { + const dir = migrations({ + '0001_fk.sql': `ALTER TABLE studies ADD CONSTRAINT custom_fk FOREIGN KEY (variant) REFERENCES variants(code);`, + '0002_rename.sql': `ALTER TABLE studies RENAME CONSTRAINT custom_fk TO renamed_fk;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping under the old name does nothing because it was renamed + writeFileSync(join(dir, '0003_drop_old.sql'), `ALTER TABLE studies DROP CONSTRAINT custom_fk;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping under the new name clears active FK + writeFileSync(join(dir, '0004_drop_new.sql'), `ALTER TABLE studies DROP CONSTRAINT renamed_fk;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks multiple foreign keys independently when one is dropped', () => { + const dir = migrations({ + '0001_fk1.sql': `ALTER TABLE studies ADD CONSTRAINT fk_one FOREIGN KEY (variant) REFERENCES variants(code);`, + '0002_fk2.sql': `ALTER TABLE studies ADD CONSTRAINT fk_two FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping fk_one leaves fk_two active + writeFileSync(join(dir, '0003_drop_one.sql'), `ALTER TABLE studies DROP CONSTRAINT fk_one;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping fk_two clears all + writeFileSync(join(dir, '0004_drop_two.sql'), `ALTER TABLE studies DROP CONSTRAINT fk_two;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks multiple foreign keys added in a single comma-separated statement', () => { + const dir = migrations({ + '0001_multi_add.sql': `ALTER TABLE studies + ADD CONSTRAINT fk_alpha FOREIGN KEY (variant) REFERENCES variants(code), + ADD CONSTRAINT fk_beta FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + writeFileSync(join(dir, '0002_drop_alpha.sql'), `ALTER TABLE studies DROP CONSTRAINT fk_alpha;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + writeFileSync(join(dir, '0003_drop_beta.sql'), `ALTER TABLE studies DROP CONSTRAINT fk_beta;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey clears active foreign keys when variant column is renamed (with or without COLUMN keyword)', () => { + const dirWithColumn = migrations({ + '0001_inline.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_col.sql': `ALTER TABLE studies RENAME COLUMN variant TO old_variant;`, + '0003_readd_unconstrained.sql': `ALTER TABLE studies ADD COLUMN variant TEXT NOT NULL DEFAULT 'standard';`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dirWithColumn), false); + } finally { + rmSync(dirWithColumn, { recursive: true, force: true }); + } + + const dirShorthand = migrations({ + '0001_inline.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_shorthand.sql': `ALTER TABLE studies RENAME variant TO old_variant;`, + '0003_readd_unconstrained.sql': `ALTER TABLE studies ADD COLUMN variant TEXT NOT NULL DEFAULT 'standard';`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dirShorthand), false); + } finally { + rmSync(dirShorthand, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey recognizes double-quoted identifiers in table-level and inline foreign keys', () => { + const dirTable = migrations({ + '0001_table_quoted.sql': `ALTER TABLE "studies" ADD CONSTRAINT "fk_quoted" FOREIGN KEY ("variant") REFERENCES "variants"("code");`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dirTable), true); + } finally { + rmSync(dirTable, { recursive: true, force: true }); + } + + const dirInline = migrations({ + '0001_inline_quoted.sql': `CREATE TABLE "studies" ( + "id" UUID PRIMARY KEY, + "variant" TEXT NOT NULL REFERENCES "variants"("code") + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dirInline), true); + } finally { + rmSync(dirInline, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks multiple unnamed foreign keys with non-colliding names', () => { + const dir = migrations({ + '0001_two_unnamed.sql': `ALTER TABLE studies + ADD FOREIGN KEY (variant) REFERENCES variants(code), + ADD FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping the first generated name studies_variant_fkey leaves the second active + writeFileSync(join(dir, '0002_drop_first.sql'), `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // Dropping the second generated name studies_variant_fkey1 clears all + writeFileSync(join(dir, '0003_drop_second.sql'), `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey1;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey reuses base unnamed constraint name in add-drop-add-drop sequence', () => { + const dir = migrations({ + '0001_add_first.sql': `ALTER TABLE studies ADD FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // 0002 drops the first generated name studies_variant_fkey + writeFileSync(join(dir, '0002_drop_first.sql'), `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + // 0003 adds another unnamed FK which reuses the available base name studies_variant_fkey + writeFileSync(join(dir, '0003_add_second.sql'), `ALTER TABLE studies ADD FOREIGN KEY (variant) REFERENCES variants(code);`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + // 0004 drops studies_variant_fkey again + writeFileSync(join(dir, '0004_drop_second.sql'), `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, 'utf8'); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE studies clears constraint and foreign key state', () => { + const dropDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_table.sql': `DROP TABLE studies;`, + '0003_recreate_unconstrained.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropDir), false); + assert.equal(effectiveStudyVariantConstraint(dropDir), null); + } finally { + rmSync(dropDir, { recursive: true, force: true }); + } +}); + +test('ALTER TABLE studies RENAME TO clears constraint and foreign key state', () => { + const renameDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_table.sql': `ALTER TABLE studies RENAME TO old_studies;`, + '0003_recreate_unconstrained.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(renameDir), false); + assert.equal(effectiveStudyVariantConstraint(renameDir), null); + } finally { + rmSync(renameDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE variants CASCADE clears active studies foreign key', () => { + const dropVariantsCascadeDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_variants.sql': `DROP TABLE variants CASCADE;`, + '0003_recreate_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropVariantsCascadeDir), false); + } finally { + rmSync(dropVariantsCascadeDir, { recursive: true, force: true }); + } + + const cascadeRecreateDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_variants.sql': `DROP TABLE variants CASCADE;`, + '0003_recreate_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY);`, + '0004_readd_fk.sql': `ALTER TABLE studies ADD FOREIGN KEY (variant) REFERENCES variants(code);`, + '0005_drop_fk.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, + }); + try { + // Releasing the cascaded FK name allows the re-added FK to use base name studies_variant_fkey, + // so dropping studies_variant_fkey properly clears it. + assert.equal(effectiveStudyVariantForeignKey(cascadeRecreateDir), false); + } finally { + rmSync(cascadeRecreateDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE multi-table containing variants with CASCADE clears active studies foreign key', () => { + const dropMultiVariantsCascadeDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_multi_cascade.sql': `DROP TABLE archive, variants CASCADE;`, + '0003_recreate_unconstrained.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropMultiVariantsCascadeDir), false); + } finally { + rmSync(dropMultiVariantsCascadeDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE public.studies with schema qualifier clears state', () => { + const dropPublicStudiesDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_public_studies.sql': `DROP TABLE public.studies;`, + '0003_recreate_unconstrained.sql': `CREATE TABLE studies (id UUID PRIMARY KEY, variant TEXT NOT NULL);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropPublicStudiesDir), false); + assert.equal(effectiveStudyVariantConstraint(dropPublicStudiesDir), null); + } finally { + rmSync(dropPublicStudiesDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE multi-table containing studies clears state', () => { + const dropMultiStudiesDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_multi_studies.sql': `DROP TABLE studies, studies_backup;`, + '0003_recreate_unconstrained.sql': `CREATE TABLE studies (id UUID PRIMARY KEY, variant TEXT NOT NULL);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropMultiStudiesDir), false); + assert.equal(effectiveStudyVariantConstraint(dropMultiStudiesDir), null); + } finally { + rmSync(dropMultiStudiesDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE variants.archive CASCADE in another schema does not clear public.variants foreign keys', () => { + const dropVariantsSchemaArchiveDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_other_schema.sql': `DROP TABLE variants.archive CASCADE;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropVariantsSchemaArchiveDir), true); + } finally { + rmSync(dropVariantsSchemaArchiveDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE "archived variants" CASCADE with quoted name does not clear public.variants foreign keys', () => { + const dropQuotedVariantsNameDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_other_table.sql': `DROP TABLE "archived variants" CASCADE;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropQuotedVariantsNameDir), true); + } finally { + rmSync(dropQuotedVariantsNameDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE public.variants CASCADE with schema qualification clears active studies foreign key', () => { + const dropPublicVariantsCascadeDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_public_variants.sql': `DROP TABLE public.variants CASCADE;`, + '0003_recreate_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropPublicVariantsCascadeDir), false); + } finally { + rmSync(dropPublicVariantsCascadeDir, { recursive: true, force: true }); + } +}); + +test('DROP TABLE variants RESTRICT does not cascade to clear active studies foreign key', () => { + const dropVariantsRestrictDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_variants_restrict.sql': `DROP TABLE variants RESTRICT;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dropVariantsRestrictDir), true); + } finally { + rmSync(dropVariantsRestrictDir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey ignores string literals containing RENAME TO keyword', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_add_column_with_default.sql': `ALTER TABLE studies ADD COLUMN note TEXT DEFAULT 'RENAME TO archive';`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey distinguishes escaped quoted identifier from variant column', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + "archived""variant" TEXT NOT NULL REFERENCES variants(code), + variant TEXT NOT NULL + );`, + }); + try { + // "archived""variant" is a separate column from "variant", so studies.variant has no FK + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks occupied constraint namespace across constraint types', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + );`, + // Unrelated constraint occupies the base name studies_variant_fkey + '0002_occupy_name.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fkey CHECK (id IS NOT NULL);`, + // Unnamed FK receives next free name studies_variant_fkey1 + '0003_add_unnamed_fk.sql': `ALTER TABLE studies ADD FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + + // Dropping studies_variant_fkey drops the unrelated CHECK, leaving the FK studies_variant_fkey1 active + writeFileSync( + join(dir, '0004_drop_unrelated.sql'), + `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, + 'utf8', + ); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + + // Dropping studies_variant_fkey1 drops the FK + writeFileSync( + join(dir, '0005_drop_fk.sql'), + `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey1;`, + 'utf8', + ); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey recognizes inline column references on studies', () => { + const dir = migrations({ + '0001_inline.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey tracks explicit inline constraint names and clears on drop', () => { + const dir = migrations({ + '0001_inline_named.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CONSTRAINT custom_inline_fk REFERENCES variants(code) + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + writeFileSync( + join(dir, '0002_drop_inline.sql'), + `ALTER TABLE studies DROP CONSTRAINT custom_inline_fk;`, + 'utf8', + ); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey handles multiple DROP CONSTRAINT clauses in a single statement', () => { + const dir = migrations({ + '0001_fk.sql': `ALTER TABLE studies ADD CONSTRAINT custom_fk FOREIGN KEY (variant) REFERENCES variants(code);`, + '0002_multi_drop.sql': `ALTER TABLE studies DROP CONSTRAINT unrelated_constraint, DROP CONSTRAINT custom_fk;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey does not match subsequent column referencing variants', () => { + const dir = migrations({ + '0001_distinct_columns.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL, + source TEXT REFERENCES variants(code) + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantForeignKey does not match prefix column like archived_variant referencing variants', () => { + const dir = migrations({ + '0001_archived_variant.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL, + archived_variant TEXT REFERENCES variants(code) + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('effectiveStudyVariantConstraint preserves active CHECK constraint when FK is added without dropping CHECK', () => { + const dir = migrations({ + '0001_check.sql': `ALTER TABLE studies ADD COLUMN variant TEXT NOT NULL DEFAULT 'standard' CHECK (variant IN ('standard', 'atomic'));`, + '0002_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk FOREIGN KEY (variant) REFERENCES variants(code);`, + }); + try { + const check = effectiveStudyVariantConstraint(dir); + assert.notEqual(check, null); + assert.deepEqual(check.variants, ['standard', 'atomic']); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('collectMirrors exposes whether effective studies.variant foreign key is present', () => { + const committed = collectMirrors(MIGRATIONS_DIR); + assert.equal(committed.studyConstraint, null); + assert.equal(committed.hasStudyVariantFk, true); + + const dropOnlyDir = migrations({ + '0001_variants.sql': `INSERT INTO variants (code) VALUES ('standard');`, + '0022_study_variant.sql': `ALTER TABLE studies + ADD COLUMN variant TEXT NOT NULL DEFAULT 'standard' + CHECK (variant IN ('standard', 'atomic'));`, + '0025_drop_only.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check;`, + }); + try { + const dropOnly = collectMirrors(dropOnlyDir); + assert.equal(dropOnly.studyConstraint, null); + assert.equal(dropOnly.hasStudyVariantFk, false); + } finally { + rmSync(dropOnlyDir, { recursive: true, force: true }); + } +}); + test('a renamed declaration fails loudly instead of checking nothing', () => { // The failure mode that makes a guard worse than no guard: it keeps passing having stopped // looking at anything. @@ -331,3 +891,1264 @@ test('every mirror the guard claims to read is really there', () => { assert.ok(found.variants.includes('standard'), `${spec.label} is missing 'standard'`); } }); + +test('tokenizeSql correctly distinguishes string literals, escaped quotes, and punctuation', () => { + const sql = `ALTER TABLE "public"."studies" ADD COLUMN note TEXT DEFAULT 'It''s a ''quoted'' string; not a stmt';`; + const tokens = tokenizeSql(sql); + + assert.equal(tokens[0].value, 'alter'); + assert.equal(tokens[1].value, 'table'); + assert.equal(tokens[2].type, 'ident'); + assert.equal(tokens[2].value, 'public'); + assert.equal(tokens[3].value, '.'); + assert.equal(tokens[4].type, 'ident'); + assert.equal(tokens[4].value, 'studies'); + + const stringToken = tokens.find((t) => t.type === 'string'); + assert.notEqual(stringToken, undefined); + assert.equal(stringToken.value, "It's a 'quoted' string; not a stmt"); + + const stmts = splitSqlStatements(tokens); + assert.equal(stmts.length, 1, 'semicolon inside string literal must not split statement'); +}); + +test('parseQualifiedTableTarget handles schema qualification and ONLY keyword', () => { + const t1 = tokenizeSql('ONLY "public"."studies"'); + const ref1 = parseQualifiedTableTarget(t1, 0); + assert.deepEqual(ref1, { schema: 'public', table: 'studies', nextIndex: 4 }); + + const t2 = tokenizeSql('studies'); + const ref2 = parseQualifiedTableTarget(t2, 0); + assert.deepEqual(ref2, { schema: 'public', table: 'studies', nextIndex: 1 }); + + const t3 = tokenizeSql('variants.archive'); + const ref3 = parseQualifiedTableTarget(t3, 0); + assert.deepEqual(ref3, { schema: 'variants', table: 'archive', nextIndex: 3 }); +}); + +test('inline column definition containing both CHECK and REFERENCES records both constraints', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CONSTRAINT allowed_check CHECK (variant IN ('standard', 'atomic')) REFERENCES variants(code) + );`, + '0002_drop_check.sql': `ALTER TABLE studies DROP CONSTRAINT allowed_check;`, + }); + try { + assert.equal(effectiveStudyVariantConstraint(dir), null); + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('column definition containing multiple inline CHECK constraints records all of them', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT check1 CHECK (variant IN ('standard', 'atomic')) + CONSTRAINT check2 CHECK (variant IN ('standard', 'atomic', 'chess960')) + );`, + }); + try { + const replayed = replayStudiesSchema(dir); + assert.equal(replayed.checks.length, 2); + assert.equal(replayed.checks[0].name, 'check1'); + assert.deepEqual(replayed.checks[0].variants, ['standard', 'atomic']); + assert.equal(replayed.checks[1].name, 'check2'); + assert.deepEqual(replayed.checks[1].variants, ['standard', 'atomic', 'chess960']); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('renaming column variant preserves existing constraint names in namespace for new unnamed FKs', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_col.sql': `ALTER TABLE studies RENAME COLUMN variant TO old_variant;`, + '0003_add_new_variant.sql': `ALTER TABLE studies ADD COLUMN variant TEXT NOT NULL REFERENCES variants(code);`, + '0004_drop_new_fk.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey1;`, + }); + try { + // The renamed column kept studies_variant_fkey, so the new FK was assigned studies_variant_fkey1. + // Dropping studies_variant_fkey1 clears the active FK on the new variant column. + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('dropping column variant releases dependent constraint names from namespace', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_drop_col.sql': `ALTER TABLE studies DROP COLUMN variant;`, + '0003_readd_col.sql': `ALTER TABLE studies ADD COLUMN variant TEXT NOT NULL REFERENCES variants(code);`, + '0004_drop_readded_fk.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_fkey;`, + }); + try { + // Dropping the variant column released studies_variant_fkey, so re-adding allows reusing studies_variant_fkey. + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('CREATE TABLE IF NOT EXISTS studies skips constraints when table already exists', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + title TEXT NOT NULL + );`, + '0002_conditional_recreate.sql': `CREATE TABLE IF NOT EXISTS studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + // The second CREATE TABLE IF NOT EXISTS is a no-op in PostgreSQL because studies already exists. + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('ALTER TABLE ADD COLUMN IF NOT EXISTS variant skips constraints when variant already exists', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + );`, + '0002_conditional_add.sql': `ALTER TABLE studies ADD COLUMN IF NOT EXISTS variant TEXT NOT NULL REFERENCES variants(code);`, + }); + try { + // The conditional column add is a no-op in PostgreSQL because variant already exists. + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('compound CHECK predicate with suffix fails loudly rather than ignoring predicate', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'atomic') AND variant <> 'atomic') + );`, + }); + try { + assert.throws( + () => replayStudiesSchema(dir), + /defines a compound or non-standard CHECK predicate on `studies.variant`/, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('ALTER TABLE IF EXISTS studies skips actions when table does not exist', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + title TEXT NOT NULL + );`, + '0002_drop.sql': `DROP TABLE studies;`, + '0003_conditional_alter.sql': `ALTER TABLE IF EXISTS studies ADD COLUMN variant TEXT REFERENCES variants(code);`, + }); + try { + // ALTER TABLE IF EXISTS is a no-op in PostgreSQL because studies was dropped. + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('unsupported inline CHECK predicate shape fails loudly rather than being ignored', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant = ANY (ARRAY['standard', 'atomic'])) + );`, + }); + try { + assert.throws( + () => replayStudiesSchema(dir), + /defines an unsupported CHECK predicate shape on `studies.variant`/, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('unsupported table-level CHECK predicate shape fails loudly rather than being ignored', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL, + CONSTRAINT chk_custom CHECK (studies.variant IN ('standard', 'atomic')) + );`, + }); + try { + assert.throws( + () => replayStudiesSchema(dir), + /defines an unsupported CHECK predicate shape on `studies.variant`/, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('operators or expressions inside IN-list fail loudly rather than extracting partial literals', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard' || 'chess960', 'atomic')) + );`, + }); + try { + assert.throws( + () => replayStudiesSchema(dir), + /defines an unsupported CHECK predicate shape on `studies.variant`/, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('table rename round-trip preserves foreign key constraint when renamed back to studies', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_away.sql': `ALTER TABLE studies RENAME TO studies_temp;`, + '0003_rename_back.sql': `ALTER TABLE studies_temp RENAME TO studies;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('table rename round-trip preserves CHECK constraint when renamed back to studies', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'atomic')) + );`, + '0002_rename_away.sql': `ALTER TABLE studies RENAME TO studies_temp;`, + '0003_rename_back.sql': `ALTER TABLE studies_temp RENAME TO studies;`, + }); + try { + const found = effectiveStudyVariantConstraint(dir); + assert.equal(found?.file, '0001_initial.sql'); + assert.deepEqual(found?.variants, ['standard', 'atomic']); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('inline column definition with REFERENCES and CHECK in reverse order records both', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL DEFAULT 'standard' + REFERENCES variants(code) + CHECK (variant IN ('standard', 'atomic')) + );`, + }); + try { + const replayed = replayStudiesSchema(dir); + assert.equal(replayed.checks.length, 1); + assert.equal(replayed.checks[0].name, 'studies_variant_check'); + assert.deepEqual(replayed.checks[0].variants, ['standard', 'atomic']); + assert.equal(replayed.hasForeignKey, true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('inline column definition with explicit CONSTRAINT names on both CHECK and REFERENCES', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT chk_study_variant CHECK (variant IN ('standard', 'atomic')) + CONSTRAINT fk_study_variant REFERENCES variants(code) + );`, + }); + try { + const replayed = replayStudiesSchema(dir); + assert.equal(replayed.checks.length, 1); + assert.equal(replayed.checks[0].name, 'chk_study_variant'); + assert.deepEqual(replayed.checks[0].variants, ['standard', 'atomic']); + assert.equal(replayed.hasForeignKey, true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('dropping only FK constraint leaves CHECK active when both defined on same column', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT chk_variant CHECK (variant IN ('standard', 'atomic')) + CONSTRAINT fk_variant REFERENCES variants(code) + );`, + '0002_drop_fk.sql': `ALTER TABLE studies DROP CONSTRAINT fk_variant;`, + }); + try { + const replayed = replayStudiesSchema(dir); + assert.notEqual(replayed.check, null); + assert.equal(replayed.check?.name, 'chk_variant'); + assert.deepEqual(replayed.check?.variants, ['standard', 'atomic']); + assert.equal(replayed.hasForeignKey, false); + assert.equal(effectiveStudyVariantForeignKey(dir), false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation succeeds when surviving FK provides integrity after CHECK drop', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT chk_v CHECK (variant IN ('standard', 'atomic')) + );`, + '0005_add_fk.sql': `ALTER TABLE studies ADD CONSTRAINT fk_v + FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID;`, + '0006_drop_chk.sql': `ALTER TABLE studies DROP CONSTRAINT chk_v;`, + '0007_validate_fk.sql': `ALTER TABLE studies VALIDATE CONSTRAINT fk_v;`, + }); + try { + const { failures, mirrors, studyConstraint, hasStudyVariantFk } = evaluateParity(dir); + assert.deepEqual(failures, []); + assert.equal(studyConstraint, null); + assert.equal(hasStudyVariantFk, true); + const lookupMirror = mirrors.find((m) => m.label.includes('variants')); + assert.notEqual(lookupMirror, undefined); + assert.deepEqual(disagreements(extractRegion(ROOT).variants, lookupMirror.variants), []); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation rejects a surviving studies CHECK even when its values match Variant', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings'); +ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID; +ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0002_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT chk_v CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + CONSTRAINT fk_v REFERENCES variants(code) + );`, + '0003_drop_fk.sql': `ALTER TABLE studies DROP CONSTRAINT fk_v;`, + }); + try { + const { failures, studyConstraint, hasStudyVariantFk } = evaluateParity(dir); + assert.notEqual(studyConstraint, null); + assert.equal(hasStudyVariantFk, false); + assert.ok( + failures.some((failure) => /retains obsolete CHECK constraint/.test(failure)), + failures.join('\n'), + ); + assert.ok( + failures.some((failure) => /has no foreign key referencing `variants\(code\)`/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation detects seed and domain disagreement independently', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + const { failures } = evaluateParity(dir); + assert.ok( + failures.some((failure) => /variants.*lookup table.*missing `horde`/.test(failure)), + failures.join('\n'), + ); + assert.ok( + !failures.some((failure) => /variants\.code.*missing `horde`/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation requires a validated variants domain CHECK matching Variant', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + const { failures, variantDomainConstraint } = evaluateParity(dir); + assert.equal(variantDomainConstraint?.name, 'variants_code_check'); + assert.equal(variantDomainConstraint?.validated, false); + assert.ok( + failures.some((failure) => /variants\.code.*not validated/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation detects missing and unknown values in variants domain CHECK', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'antichess', 'racingkings')) NOT VALID; +ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0003_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + const { failures } = evaluateParity(dir); + assert.ok( + failures.some((failure) => /variants\.code.*missing `horde`.*unknown `antichess`/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation detects studies CHECK removal before variants domain validation', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + );`, + '0003_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0004_unsafe.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check; +ALTER TABLE studies ADD CONSTRAINT studies_variant_fk FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID;`, + '0005_too_late.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check; +ALTER TABLE studies VALIDATE CONSTRAINT studies_variant_fk;`, + }); + try { + const { failures, unsafeStudyConstraintTransition } = evaluateParity(dir); + assert.equal(unsafeStudyConstraintTransition, true); + assert.ok( + failures.some((failure) => /before the variants domain CHECK was validated/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation permits staging the studies FK while its canonical CHECK remains active', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + );`, + '0003_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0004_stage_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID;`, + '0005_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0006_replace_check.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check;`, + '0007_validate_fk.sql': `ALTER TABLE studies VALIDATE CONSTRAINT studies_variant_fk;`, + }); + try { + const { failures, unsafeStudyConstraintTransition } = evaluateParity(dir); + assert.equal(unsafeStudyConstraintTransition, false); + assert.deepEqual(failures, []); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation recognizes PostgreSQL FK options before trailing NOT VALID', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + );`, + '0005_stage_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) + MATCH FULL ON DELETE RESTRICT ON UPDATE NO ACTION + DEFERRABLE INITIALLY DEFERRED NOT VALID;`, + '0006_replace_check.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check;`, + '0007_validate_fk.sql': `ALTER TABLE studies VALIDATE CONSTRAINT studies_variant_fk;`, + }); + try { + const { failures, studyForeignKeys } = evaluateParity(dir); + assert.deepEqual(failures, []); + assert.equal(studyForeignKeys[0]?.file, '0005_stage_fk.sql'); + assert.equal(studyForeignKeys[0]?.validatedFile, '0007_validate_fk.sql'); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation recognizes CHECK NO INHERIT before trailing NOT VALID', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NO INHERIT NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + );`, + '0005_stage_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID;`, + '0006_replace_check.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check;`, + '0007_validate_fk.sql': `ALTER TABLE studies VALIDATE CONSTRAINT studies_variant_fk;`, + }); + try { + const { failures, variantDomainConstraint } = evaluateParity(dir); + assert.deepEqual(failures, []); + assert.equal(variantDomainConstraint?.addedNotValid, true); + assert.equal(variantDomainConstraint?.validatedFile, '0003_validate_domain.sql'); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity replay fails closed on an unknown foreign-key suffix', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +CREATE TABLE studies (id UUID PRIMARY KEY, variant TEXT NOT NULL);`, + '0002_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) MATCH UNKNOWN NOT VALID;`, + }); + try { + assert.throws( + () => replayStudiesSchema(dir), + /unsupported suffix.*studies\.variant.*foreign key/i, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('CREATE TABLE replay accepts a final table-level studies variant CHECK', () => { + const dir = migrations({ + '0001_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL, + CONSTRAINT studies_variant_check CHECK (variant IN ('standard')) + );`, + }); + try { + assert.deepEqual(replayStudiesSchema(dir).check?.variants, ['standard']); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity replay rejects NOT ENFORCED variant integrity constraints', () => { + const checkDir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY);`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard')) NOT ENFORCED NOT VALID;`, + }); + const fkDir = migrations({ + '0001_initial.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +CREATE TABLE studies (id UUID PRIMARY KEY, variant TEXT NOT NULL);`, + '0002_fk.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_fk + FOREIGN KEY (variant) REFERENCES variants(code) NOT ENFORCED NOT VALID;`, + }); + const alteredStudyCheckDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies (id UUID PRIMARY KEY, variant TEXT NOT NULL);`, + '0002_check.sql': `ALTER TABLE studies ADD CONSTRAINT studies_variant_check + CHECK (variant IN ('standard')) NOT ENFORCED NOT VALID;`, + }); + const tableStudyCheckDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL, + CONSTRAINT studies_variant_check CHECK (variant IN ('standard')) NOT ENFORCED +);`, + }); + const inlineStudyCheckDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard')) NOT ENFORCED +);`, + }); + const inlineStudyFkDir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + ON DELETE SET NULL DEFERRABLE INITIALLY DEFERRED NOT ENFORCED +);`, + }); + try { + assert.throws(() => replayStudiesSchema(checkDir), /NOT ENFORCED.*variants\.code.*protect writes/i); + assert.throws(() => replayStudiesSchema(fkDir), /NOT ENFORCED.*studies\.variant.*protect writes/i); + assert.throws( + () => replayStudiesSchema(alteredStudyCheckDir), + /NOT ENFORCED.*studies\.variant.*protect writes/i, + ); + assert.throws( + () => replayStudiesSchema(tableStudyCheckDir), + /NOT ENFORCED.*studies\.variant.*protect writes/i, + ); + assert.throws( + () => replayStudiesSchema(inlineStudyCheckDir), + /NOT ENFORCED.*studies\.variant.*protect writes/i, + ); + assert.throws( + () => replayStudiesSchema(inlineStudyFkDir), + /NOT ENFORCED.*studies\.variant.*protect writes/i, + ); + } finally { + rmSync(checkDir, { recursive: true, force: true }); + rmSync(fkDir, { recursive: true, force: true }); + rmSync(alteredStudyCheckDir, { recursive: true, force: true }); + rmSync(tableStudyCheckDir, { recursive: true, force: true }); + rmSync(inlineStudyCheckDir, { recursive: true, force: true }); + rmSync(inlineStudyFkDir, { recursive: true, force: true }); + } +}); + +test('parity evaluation rejects a broad validated domain CHECK as transition authorization', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_broad_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_legacy_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings', 'antichess')) NOT VALID; +ALTER TABLE variants VALIDATE CONSTRAINT variants_code_legacy_check;`, + '0003_canonical_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + );`, + '0005_unsafe.sql': `ALTER TABLE studies DROP CONSTRAINT studies_variant_check; +ALTER TABLE studies ADD CONSTRAINT studies_variant_fk FOREIGN KEY (variant) REFERENCES variants(code) NOT VALID;`, + '0006_too_late.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check; +ALTER TABLE variants DROP CONSTRAINT variants_code_legacy_check; +ALTER TABLE studies VALIDATE CONSTRAINT studies_variant_fk;`, + }); + try { + const { failures, unsafeStudyConstraintTransition } = evaluateParity(dir); + assert.equal(unsafeStudyConstraintTransition, true); + assert.ok( + failures.some((failure) => /before the variants domain CHECK was validated/.test(failure)), + failures.join('\n'), + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('parity evaluation flags failure when both CHECK and FK are removed from studies.variant', () => { + const dir = migrations({ + '0001_variants.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + CONSTRAINT chk_v CHECK (variant IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) + CONSTRAINT fk_v REFERENCES variants(code) + );`, + '0005_drop_both.sql': `ALTER TABLE studies DROP CONSTRAINT chk_v, DROP CONSTRAINT fk_v;`, + }); + try { + const { failures, studyConstraint, hasStudyVariantFk } = evaluateParity(dir); + assert.equal(studyConstraint, null); + assert.equal(hasStudyVariantFk, false); + assert.equal(failures.length, 1); + assert.match(failures[0], /`studies\.variant` has no foreign key referencing `variants\(code\)`/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('quoted identifier case is preserved so renaming to "Studies" leaves effectiveStudyVariantForeignKey false', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_case.sql': `ALTER TABLE studies RENAME TO "Studies";`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), false); + assert.equal(effectiveStudyVariantConstraint(dir), null); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('shadow table recreation and drop preserves original renamed table constraints on rename back', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_rename_away.sql': `ALTER TABLE studies RENAME TO studies_backup;`, + '0003_create_shadow.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + note TEXT + );`, + '0004_drop_shadow.sql': `DROP TABLE studies;`, + '0005_rename_back.sql': `ALTER TABLE studies_backup RENAME TO studies;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('creating or altering table in another schema does not overwrite public.studies constraints', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_archive_schema.sql': `CREATE TABLE archive.studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL CHECK (variant IN ('standard', 'atomic')) + );`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + assert.equal(effectiveStudyVariantConstraint(dir), null); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('ALTER TABLE SET SCHEMA moving studies out of public leaves effectiveStudyVariantForeignKey false', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_move_schema.sql': `ALTER TABLE studies SET SCHEMA archive;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), false); + assert.equal(effectiveStudyVariantConstraint(dir), null); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('ALTER TABLE SET SCHEMA moving table into public restores effectiveStudyVariantForeignKey true', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_move_to_archive.sql': `ALTER TABLE studies SET SCHEMA archive;`, + '0003_move_back_to_public.sql': `ALTER TABLE archive.studies SET SCHEMA public;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('tokenizeSql and splitSqlStatements treat untagged dollar-quoted body with semicolons as one statement', () => { + const sql = `DO $$ + BEGIN + PERFORM 1; + PERFORM 2; + END + $$;`; + const tokens = tokenizeSql(sql); + const statements = splitSqlStatements(tokens); + assert.equal(statements.length, 1); + assert.equal(tokens[1].type, 'string'); +}); + +test('tokenizeSql and splitSqlStatements treat tagged dollar-quoted body with semicolons as one statement', () => { + const sql = `DO $migration$ + BEGIN + PERFORM 1; + PERFORM 2; + END + $migration$;`; + const tokens = tokenizeSql(sql); + const statements = splitSqlStatements(tokens); + assert.equal(statements.length, 1); + assert.equal(tokens[1].type, 'string'); +}); + +test('unknown harmless DO statement fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_harmless_do.sql': `DO $$ + BEGIN + RAISE NOTICE 'hello'; + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('stored-function invocation inside DO statement fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_stored_proc.sql': `DO $$ + BEGIN + PERFORM remove_variant_fk(); + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('top-level DO statement with direct DDL fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_direct_ddl.sql': `DO $$ + BEGIN + ALTER TABLE studies DROP CONSTRAINT studies_variant_fk; + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('top-level DO statement with DROP TABLE studies fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_procedural_drop.sql': `DO $$ + BEGIN + DROP TABLE studies; + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('top-level DO statement with DROP TABLE variants CASCADE fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_procedural_cascade.sql': `DO $$ + BEGIN + DROP TABLE variants CASCADE; + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('top-level DO statement with dynamic SQL EXECUTE fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_dynamic_ddl.sql': `DO $$ + BEGIN + EXECUTE 'ALTER TABLE studies DROP CONSTRAINT studies_variant_fk'; + END + $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('tagged top-level DO statement fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_tagged_do.sql': `DO $migration$ + BEGIN + PERFORM 1; + END + $migration$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('Unicode-tagged top-level DO statement fails closed', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_unicode_do.sql': `DO $函数$ + BEGIN + PERFORM 1; + END + $函数$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('canonical 0021_engine_bots.sql migration matches allowlist fingerprint and passes replay', () => { + const realFile = join(MIGRATIONS_DIR, '0021_engine_bots.sql'); + const rawSql = readFileSync(realFile, 'utf8'); + const canonicalSql = rawSql.replace(/\r\n/g, '\n'); + const hash = createHash('sha256').update(canonicalSql, 'utf8').digest('hex'); + const entry = KNOWN_HISTORICAL_PROCEDURAL_MIGRATIONS.get('0021_engine_bots.sql'); + assert.ok(entry); + assert.equal(hash, entry.sha256); + assert.equal(entry.expectedDoCount, 1); + + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0021_engine_bots.sql': rawSql, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('0021_engine_bots.sql with ONE modified byte is rejected', () => { + const realFile = join(MIGRATIONS_DIR, '0021_engine_bots.sql'); + const rawSql = readFileSync(realFile, 'utf8'); + const modifiedSql = rawSql + ' '; // one changed byte + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0021_engine_bots.sql': modifiedSql, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('0021_engine_bots.sql filename with different file content is rejected', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0021_engine_bots.sql': `DO $$ BEGIN PERFORM 1; END $$;`, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('same safe DO content copied into a different migration filename is rejected', () => { + const realFile = join(MIGRATIONS_DIR, '0021_engine_bots.sql'); + const rawSql = readFileSync(realFile, 'utf8'); + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0030_copied_engine_bots.sql': rawSql, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('new second DO added to allowlisted migration causes rejection via fingerprint and count mismatch', () => { + const realFile = join(MIGRATIONS_DIR, '0021_engine_bots.sql'); + const rawSql = readFileSync(realFile, 'utf8'); + const twoDoSql = rawSql + '\nDO $$ BEGIN PERFORM 1; END $$;'; + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0021_engine_bots.sql': twoDoSql, + }); + try { + assert.throws(() => replayStudiesSchema(dir), /contains an unsupported top-level PostgreSQL DO block/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('non-DO dollar-quoted function body remains lexically atomic and is ignored by replay', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + '0002_func.sql': `CREATE OR REPLACE FUNCTION log_change() RETURNS trigger AS $$ + BEGIN + -- Semicolons inside function do not split top-level statements + PERFORM 1; + RETURN NEW; + END; + $$ LANGUAGE plpgsql;`, + }); + try { + assert.equal(effectiveStudyVariantForeignKey(dir), true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('dollar-quoted string requires exact matching tag and does not stop on mismatched inner tag', () => { + const sql = `DO $outer$ body with $inner$ inner text $inner$ more body $outer$;`; + const tokens = tokenizeSql(sql); + assert.equal(tokens.length, 3); // DO, string, ; + assert.equal(tokens[1].type, 'string'); + assert.equal(tokens[1].value, ' body with $inner$ inner text $inner$ more body '); +}); + +test('Unicode dollar tag with CJK characters is tokenized as one atomic string', () => { + const sql = `$函数$\nSELECT 1;\nSELECT 2;\n$函数$;`; + const tokens = tokenizeSql(sql); + const statements = splitSqlStatements(tokens); + assert.equal(statements.length, 1); + assert.equal(tokens[0].type, 'string'); + assert.equal(tokens[0].value, '\nSELECT 1;\nSELECT 2;\n'); +}); + +test('Unicode dollar tag with accented Latin characters is tokenized as one atomic string', () => { + const sql = `$étiquette$\nSELECT 1;\n$étiquette$;`; + const tokens = tokenizeSql(sql); + const statements = splitSqlStatements(tokens); + assert.equal(statements.length, 1); + assert.equal(tokens[0].type, 'string'); + assert.equal(tokens[0].value, '\nSELECT 1;\n'); +}); + +test('dollar tag starting with underscore and containing alphanumeric characters works', () => { + const sql = `$_tag123$\nSELECT 1;\n$_tag123$;`; + const tokens = tokenizeSql(sql); + const statements = splitSqlStatements(tokens); + assert.equal(statements.length, 1); + assert.equal(tokens[0].type, 'string'); +}); + +test('Unicode dollar tag requires exact matching tag and does not stop on different tag', () => { + const sql = `$函数$\nSELECT 1;\n$different$\nSELECT 2;\n$函数$;`; + const tokens = tokenizeSql(sql); + assert.equal(tokens.length, 2); // string, ; + assert.equal(tokens[0].type, 'string'); + assert.equal(tokens[0].value, '\nSELECT 1;\n$different$\nSELECT 2;\n'); +}); + +test('unterminated Unicode-tagged dollar quote throws explicit error', () => { + assert.throws(() => tokenizeSql('$函数$ SELECT 1;'), /unterminated dollar-quoted string/); + assert.throws(() => tokenizeSql('$étiquette$ SELECT 1; $different$'), /unterminated dollar-quoted string/); +}); + +test('dollar tag starting with digit like $9bad$ is rejected by readDollarQuoteDelimiter', () => { + assert.equal(readDollarQuoteDelimiter('$9bad$', 0), null); + const tokens = tokenizeSql('$9bad$'); + assert.equal(tokens[0].type, 'punct'); + assert.equal(tokens[0].value, '$'); + assert.equal(tokens[1].type, 'punct'); + assert.equal(tokens[1].value, '9'); +}); + +test('comment stripping and tokenizer recognize the exact same dollar delimiter set', () => { + const codeWithComment = `$étiquette$\n-- this is not a comment line\nSELECT 1;\n$étiquette$;`; + const stripped = stripComments(codeWithComment, 'sql'); + const tokens = tokenizeSql(stripped); + assert.equal(tokens[0].type, 'string'); + assert.match(tokens[0].value, /-- this is not a comment line/); +}); + +test('unterminated dollar-quoted string throws explicit error', () => { + assert.throws(() => tokenizeSql('DO $$ BEGIN PERFORM 1;'), /unterminated dollar-quoted string/); + assert.throws(() => tokenizeSql('DO $tag$ BEGIN PERFORM 1; $different$'), /unterminated dollar-quoted string/); +}); + +test('ordinary single-quoted strings and positional parameters are not confused with dollar quotes', () => { + const sql = `SELECT 'hello $world$', $1, $2 FROM t;`; + const tokens = tokenizeSql(sql); + assert.equal(tokens[0].value, 'select'); + assert.equal(tokens[1].type, 'string'); + assert.equal(tokens[1].value, 'hello $world$'); + assert.equal(tokens[3].type, 'punct'); + assert.equal(tokens[3].value, '$'); + assert.equal(tokens[4].type, 'punct'); + assert.equal(tokens[4].value, '1'); +}); + +test('structured tableKey distinguishes quoted identifiers containing periods', () => { + const key1 = tableKey({ schema: 'archive.x', table: 'studies' }); + const key2 = tableKey({ schema: 'archive', table: 'x.studies' }); + assert.notEqual(key1, key2); + assert.equal(key1, '["archive.x","studies"]'); + assert.equal(key2, '["archive","x.studies"]'); +}); + +test('structured tableKey distinguishes public.studies from a table named "public.studies" in public schema', () => { + const canonical = tableKey({ schema: 'public', table: 'studies' }); + const literalDotted = tableKey({ schema: 'public', table: 'public.studies' }); + assert.notEqual(canonical, literalDotted); + assert.equal(canonical, STUDIES_TABLE_KEY); + assert.equal(literalDotted, '["public","public.studies"]'); +}); + +test('end-to-end parity failure when colliding-under-old-model table has variant constraints but public.studies is unconstrained', () => { + const dir = migrations({ + '0001_initial.sql': `CREATE TABLE variants (code TEXT PRIMARY KEY); +INSERT INTO variants (code) VALUES + ('standard'), ('chess960'), ('kingofthehill'), ('atomic'), + ('crazyhouse'), ('threecheck'), ('horde'), ('racingkings');`, + '0002_domain.sql': `ALTER TABLE variants ADD CONSTRAINT variants_code_check + CHECK (code IN ('standard', 'chess960', 'kingofthehill', 'atomic', 'crazyhouse', 'threecheck', 'horde', 'racingkings')) NOT VALID;`, + '0003_validate_domain.sql': `ALTER TABLE variants VALIDATE CONSTRAINT variants_code_check;`, + '0004_unconstrained_studies.sql': `CREATE TABLE studies ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL + );`, + '0005_colliding_archive.sql': `CREATE TABLE "public.studies" ( + id UUID PRIMARY KEY, + variant TEXT NOT NULL REFERENCES variants(code) + );`, + }); + try { + const { failures, studyConstraint, hasStudyVariantFk } = evaluateParity(dir); + assert.equal(studyConstraint, null); + assert.equal(hasStudyVariantFk, false); + assert.equal(failures.length, 1); + assert.match(failures[0], /`studies\.variant` has no foreign key referencing `variants\(code\)`/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + + + + + + + + + + + +