Skip to content

feat: Add maintain command (catalog discovery + per-table extend + CDC-readiness guard)#29

Closed
codybswaney wants to merge 14 commits into
masterfrom
feature/pgslice-maintain
Closed

feat: Add maintain command (catalog discovery + per-table extend + CDC-readiness guard)#29
codybswaney wants to merge 14 commits into
masterfrom
feature/pgslice-maintain

Conversation

@codybswaney

Copy link
Copy Markdown

Summary

Adds a maintain command: discover every managed partitioned table from the catalog and extend each in one run, so a scheduler runs a single command and newly-partitioned tables are picked up automatically with no per-table configuration.

  • Discovery — finds top-level partitioned parents (relkind = 'p', not themselves partitions) that carry a valid pgslice settings comment; optional --schema filter.
  • Per-table extend — runs add_partitions for each discovered table; native (parent-owned PK) and classic-pgslice (per-child PK) tables are handled automatically.
  • Per-table isolation — one table's failure is recorded and does not stop the rest of the fleet; the command exits non-zero if any table failed.
  • CDC-readiness guard — after extending, verifies every leaf has a replica identity usable for logical replication (a PK under the default identity, or FULL / USING INDEX); surfaces a CDC-unsafe table loudly. Read-only — no replica-identity DDL.

Test plan

  • src/maintain.test.ts — e2e against real Postgres (13.20): discovery (exactly the settings-commented parents; unmanaged/plain tables excluded), native-vs-classic classification, bounds-anchored extension with expected names, idempotent re-run, --schema restriction, PK + grant inheritance on new partitions, REPLICA IDENTITY USING INDEX propagation, a CDC-unsafe leaf flagged, and per-table failure isolation (a DEFAULT-blocked table fails alone while the rest still extend).
  • src/commands/maintain.test.ts — command-level: exit 0 + per-table summary on a healthy fleet, exit 1 when a leaf has no usable replica identity, and the "no managed tables" message.
  • Full suite green on PG 13.20; build + typecheck + prettier clean.

Notes

Targets feature/pgslice-partition-shapes so the diff is maintain-only — review/merge after that branch lands.

🤖 Generated with Claude Code

codybswaney and others added 13 commits June 26, 2026 14:32
Add tests for the partition-shape capabilities — weekly/ISO-week period,
bounds-anchored extension of existing tables, UTC/timestamptz-correct bounds,
composite/parent-owned primary keys, and grant inheritance — plus shared
fixtures. These fail against the current implementation; the functionality
follows to turn them green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement the partition-shape capabilities to turn the tests green:

- Weekly (ISO-week, Monday-aligned) period: date math, IYYY"w"IW suffix, and
  parser in date-ranges.ts, alongside day/month/year.
- Bounds-anchored extension: when a table already has partitions, anchor on the
  max/min existing bound and chain contiguous period-sized ranges to the
  horizon, by bounds rather than names — extending legacy-named tables in place
  with no rename, gap, or overlap; idempotent. DEFAULT/MINVALUE/MAXVALUE are
  recognized and ignored when anchoring.
- UTC-pinned, type-correct bounds: pin the transaction to UTC and pick the cast
  by key type, so timestamptz tables extend deterministically.
- Composite / parent-owned primary keys: skip the per-child ADD PRIMARY KEY
  when the parent owns the key; emit a composite child key in the classic model.
- Grant inheritance: re-issue the parent's grants on each new partition
  (default on, --no-inherit-grants to disable), and summarize created
  partitions in the add_partitions command.

The maintain fleet command and CDC-readiness guard are intentionally out of
scope here (follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the weekly period, managing existing (already-partitioned) tables via
bounds-anchored extension, composite/parent-owned PK handling, and grant
inheritance (--no-inherit-grants). Add a CHANGELOG entry. Enable a vitest v8
coverage block + `coverage` script (measured, not gated). The maintain command
is documented with its own follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From our own review + best-practices research before external review:

- Genericize two internal references in a test comment for the public fork (an
  incident id and a CDC-vendor name).
- Add coverage: forward extension is skipped past a MAXVALUE catch-all (nothing
  to add, no error); and a classic (parent has no PK) table retrofits with each
  new partition inheriting the existing 3-column composite key.
- Clarify docs: formatDateForSql emits UTC-midnight (calendar-aligned) bounds,
  so non-midnight legacy bounds are unsupported; fix the
  #partitionPrimaryKeyColumns fallback wording.
- README: weekly cron + monitoring examples to match the new period.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ents

Address review feedback:

- parseBoundValue accepts only a single quoted literal, so a multi-column RANGE
  bound (e.g. a (timestamp, int) compound key) is treated as unmanaged (null)
  rather than misread as a single date bound with the extra columns dropped.
- addPartitions computes the parent primary key first and only queries existing
  partitions in the classic (parent-has-no-PK) path, removing a wasted catalog
  round-trip on every run for native parent-owned-PK tables.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address review feedback:

- addPartitions forward extension no longer requires future > 0. With future=0
  the horizon is today, so a retrofit table's current period is still created
  when uncovered — matching the fresh-table path (which always includes today)
  and avoiding silently leaving today's partition missing on a back-fill run.
- parseBoundValue's date regex is anchored (^...$ with an optional UTC offset),
  so unexpected trailing content yields null instead of a silently-truncated
  parse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dation

Address review feedback:

- Grant inheritance reads grants from the original table, so the prep
  (--intermediate) flow propagates the original's grants onto new partitions
  instead of silently reading the freshly-prepped intermediate (which has
  none). The retrofit path is unchanged (original == target there).
- parsePartitionDate's week-suffix guard rejects out-of-range ISO week numbers
  (e.g. w00, w54), not just non-numeric/legacy-prefixed suffixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ediates

No behavior change (the ISO-week tests cover year boundaries and week 53):
name the week's Monday and Thursday explicitly and avoid mutating a value
before its name is accurate, so the trickiest date math is easier to audit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…antics

Comment-only (no behavior change): note the intentional inequality asymmetry
between extendRanges and extendRangesBackward, and document that
assertContiguous checks only finite bounds (MINVALUE/MAXVALUE/DEFAULT are
ignored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an optional per-table `format` template (stored in the settings comment,
e.g. `p{YYYY}w{WW}` or `y{YYYY}m{MM}`) so a retrofitted table can keep its
existing partition naming convention. Placeholders resolve to the same date
components already used — ISO week-year/week for weekly tables, calendar
year/month/day otherwise — so a custom format changes only the rendered name,
never which dates a partition covers. A single shared spec drives both
formatDateSuffix and parsePartitionDate, keeping them symmetric; literals are
restricted to [a-z0-9] since the suffix is recovered by splitting on `_`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bounds-anchored extension emits new partition boundaries at UTC midnight,
so a table whose existing partitions sit on non-midnight boundaries cannot
be extended contiguously. This previously surfaced either as a silent
no-op (the midnight-rounded horizon falls short of the non-midnight anchor,
so zero candidates are generated) or, with a larger --future, a
partition-overlap error at CREATE time. Detect a non-midnight anchor (the
max upper / min lower bound) up front and throw a clear error instead of
stalling silently.

Also parse the settings comment by splitting on the first ":" only, so a
value containing a colon (e.g. a mistyped format template) is preserved
and validated downstream rather than truncated at the second colon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
formatDateSuffix recompiled the partition-name template (a regex scan plus
allocation) and recomputed the ISO week twice on every render — once per
{YYYY}/{WW} placeholder — so a large back-fill repeated that work for each
partition. Cache compileFormat by (period, template) and share a single
isoWeekInfo result between the two weekly placeholders within a render.
Also drop an unused zod import from pgslice.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…guard)

Discovers every partitioned parent carrying a valid pgslice settings
comment and extends each with add_partitions in one run, with per-table
failure isolation and a read-only CDC-readiness guard (every leaf must
have a replica identity usable for logical replication). Exits non-zero
if any table failed or is CDC-unsafe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jul 1, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a fleet-wide maintain command for managed partitioned tables. The main changes are:

  • Catalog discovery for top-level partitioned parents with valid pgslice settings comments.
  • Per-table partition extension with period-specific future horizons.
  • Per-table result reporting with failure isolation across the fleet.
  • CDC readiness checks for leaf partitions after maintenance.
  • Command and end-to-end tests for discovery, extension, idempotency, grants, schema filtering, and replica identity handling.

Confidence Score: 5/5

Safe to merge with minimal risk.

The change is well-scoped to the new maintain command and supporting catalog checks. Tests cover discovery, per-period extension, idempotency, grants, CDC readiness, and per-table failure isolation. No blocking correctness or security issues were found in the changed paths.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • Checked the base state for the maintain discovery extension; the before log shows the base build succeeds, pgslice --help lacks maintain, pgslice maintain --help exits 1 as an unknown command, and the test path src/commands/maintain.test.ts did not exist.
  • Observed the after state for the maintain discovery extension; the after log shows the head build succeeds, pgslice --help includes maintain, pgslice maintain --help documents automatic catalog discovery plus --schema and per-period future options, and maintain tests start but fail because PGSLICE_URL cannot resolve a usable database.
  • Identified a blocker: no local psql, postgres, initdb, pg_ctl, Docker, or usable database URL was available to create and query the fleet.
  • Viewed the before state for maintain-isolation-exit; the base checkout shows a cc38013 and npm test -- src/commands/maintain.test.ts --run reported No test files found.
  • Viewed the after state for maintain-isolation-exit; the head checkout fc71924 is used with a local PostgreSQL startup, and the Vitest run across the maintain tests passes all 15 tests.
  • Noted that the after state includes a PostgreSQL server log error for blocked_202602 about partition constraints, while the Vitest case passes for the rest of the fleet.
  • Checked the before state for the maintain CLI guard; the base artifact shows there is no maintain command guard in place yet.
  • Reviewed the after state for the maintain CLI guard; MAINTAIN_EXIT=1 outputs appear with unsafe_default_p2026w01 and unsafe_nothing_p2026w01 only, while stdout reports safe FULL, USING INDEX, and default-identity states, and no replica-identity DDL is issued.
  • Examined the before state for maintain-models; the base checkout shows src/maintain.test.ts is absent and Vitest reports no test files.
  • Reviewed the after state for maintain-models; head checkout with local Postgres shows MAINTAIN_FIRST creating trex_classic_2026w04/w05 and trex_native creating trex_native_202602/202603, MAINTAIN_SECOND creates no partitions, CATALOG_PK shows primary-key constraints on all children, and the full maintain suite passes 12/12.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
src/commands/maintain.ts Implements the maintain CLI options, per-table reporting, and non-zero exit on failures.
src/pgslice.ts Adds catalog discovery and per-table maintain orchestration using addPartitions plus replica-identity validation.
src/table.ts Adds catalog inspection for direct child partitions without usable replica identity.
src/types.ts Defines maintain option/result types and partition model metadata.
src/maintain.test.ts Adds end-to-end fleet coverage for discovery, partition extension, idempotency, CDC readiness, grants, and failure isolation.
src/commands/maintain.test.ts Adds command-level tests for success, CDC guard failure, and empty discovery output.
src/cli.ts Registers the new maintain command with the CLI.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
actor Scheduler
participant CLI as MaintainCommand
participant P as Pgslice.maintain
participant Catalog as Postgres catalog
participant Add as addPartitions
participant Table as Table helpers

Scheduler->>CLI: Run pgslice maintain
CLI->>P: Call maintain with horizons and schema
P->>Catalog: Discover managed partitioned parents
Catalog-->>P: Return tables and periods
loop each managed table
    P->>Add: Extend table for period horizon
    Add-->>P: Return created partition names
    P->>Table: Inspect primary keys partitions and replica identity
    Table-->>P: Return model counts and unsafe leaves
    P-->>CLI: Return MaintainResult
end
CLI-->>Scheduler: Print summaries and exit status
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
actor Scheduler
participant CLI as MaintainCommand
participant P as Pgslice.maintain
participant Catalog as Postgres catalog
participant Add as addPartitions
participant Table as Table helpers

Scheduler->>CLI: Run pgslice maintain
CLI->>P: Call maintain with horizons and schema
P->>Catalog: Discover managed partitioned parents
Catalog-->>P: Return tables and periods
loop each managed table
    P->>Add: Extend table for period horizon
    Add-->>P: Return created partition names
    P->>Table: Inspect primary keys partitions and replica identity
    Table-->>P: Return model counts and unsafe leaves
    P-->>CLI: Return MaintainResult
end
CLI-->>Scheduler: Print summaries and exit status
Loading

Reviews (2): Last reviewed commit: "feat: Give maintain per-period future ho..." | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Jul 1, 2026

Copy link
Copy Markdown

T-Rex pricing update — T-Rex was free through June 2026. Effective July 1, 2026, T-Rex adds 2 credits on top of the standard 1-credit review (3 total). T-Rex settings

A single --future N means very different runway for weekly vs monthly vs
yearly tables (3 weeks vs 3 months vs 3 years). Replace it with per-period
--future-daily / --future-weekly / --future-monthly / --future-yearly
(defaults 90 / 26 / 6 / 1); discovery returns each table's period, and each
table is extended by the horizon for its own period, so the fleet gets
comparable forward coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Base automatically changed from feature/pgslice-partition-shapes to master July 1, 2026 15:05
@codybswaney codybswaney closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant