Skip to content

feature: Support additional table shapes for partition provisioning#28

Merged
codybswaney merged 12 commits into
masterfrom
feature/pgslice-partition-shapes
Jul 1, 2026
Merged

feature: Support additional table shapes for partition provisioning#28
codybswaney merged 12 commits into
masterfrom
feature/pgslice-partition-shapes

Conversation

@codybswaney

@codybswaney codybswaney commented Jun 30, 2026

Copy link
Copy Markdown

Overview

This PR adds support for other table shapes for partition preovisioning, specifically the following functionality:

  1. Partitioning based on real partition bounds
  2. Expanded partition naming schemas (such as y{YYYY}m{MM}, y{YYYY}w{WW}, p{YYYY}w{WW}), configurable via settings
  3. Weekly partitioning
  4. UTC-pinned boundary rendering and a loud rejection of unsupported non-midnight-aligned tables
  5. Inheriting parent grants on new leaves and handling parent-owned keys (composite or single)

The net effect is that we can support future partition provisioning for new shapes, and all of those internally at WorkOS.

Validation

Aside from existing and expanded test coverage, there's a write up on the specific internal validation procedure here.

For INFRA-5626

codybswaney and others added 12 commits June 26, 2026 14:32
Add tests for the partition-shape capabilities — weekly/ISO-week period,
bounds-anchored extension of existing tables, UTC/timestamptz-correct bounds,
composite/parent-owned primary keys, and grant inheritance — plus shared
fixtures. These fail against the current implementation; the functionality
follows to turn them green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement the partition-shape capabilities to turn the tests green:

- Weekly (ISO-week, Monday-aligned) period: date math, IYYY"w"IW suffix, and
  parser in date-ranges.ts, alongside day/month/year.
- Bounds-anchored extension: when a table already has partitions, anchor on the
  max/min existing bound and chain contiguous period-sized ranges to the
  horizon, by bounds rather than names — extending legacy-named tables in place
  with no rename, gap, or overlap; idempotent. DEFAULT/MINVALUE/MAXVALUE are
  recognized and ignored when anchoring.
- UTC-pinned, type-correct bounds: pin the transaction to UTC and pick the cast
  by key type, so timestamptz tables extend deterministically.
- Composite / parent-owned primary keys: skip the per-child ADD PRIMARY KEY
  when the parent owns the key; emit a composite child key in the classic model.
- Grant inheritance: re-issue the parent's grants on each new partition
  (default on, --no-inherit-grants to disable), and summarize created
  partitions in the add_partitions command.

The maintain fleet command and CDC-readiness guard are intentionally out of
scope here (follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the weekly period, managing existing (already-partitioned) tables via
bounds-anchored extension, composite/parent-owned PK handling, and grant
inheritance (--no-inherit-grants). Add a CHANGELOG entry. Enable a vitest v8
coverage block + `coverage` script (measured, not gated). The maintain command
is documented with its own follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From our own review + best-practices research before external review:

- Genericize two internal references in a test comment for the public fork (an
  incident id and a CDC-vendor name).
- Add coverage: forward extension is skipped past a MAXVALUE catch-all (nothing
  to add, no error); and a classic (parent has no PK) table retrofits with each
  new partition inheriting the existing 3-column composite key.
- Clarify docs: formatDateForSql emits UTC-midnight (calendar-aligned) bounds,
  so non-midnight legacy bounds are unsupported; fix the
  #partitionPrimaryKeyColumns fallback wording.
- README: weekly cron + monitoring examples to match the new period.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ents

Address review feedback:

- parseBoundValue accepts only a single quoted literal, so a multi-column RANGE
  bound (e.g. a (timestamp, int) compound key) is treated as unmanaged (null)
  rather than misread as a single date bound with the extra columns dropped.
- addPartitions computes the parent primary key first and only queries existing
  partitions in the classic (parent-has-no-PK) path, removing a wasted catalog
  round-trip on every run for native parent-owned-PK tables.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address review feedback:

- addPartitions forward extension no longer requires future > 0. With future=0
  the horizon is today, so a retrofit table's current period is still created
  when uncovered — matching the fresh-table path (which always includes today)
  and avoiding silently leaving today's partition missing on a back-fill run.
- parseBoundValue's date regex is anchored (^...$ with an optional UTC offset),
  so unexpected trailing content yields null instead of a silently-truncated
  parse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dation

Address review feedback:

- Grant inheritance reads grants from the original table, so the prep
  (--intermediate) flow propagates the original's grants onto new partitions
  instead of silently reading the freshly-prepped intermediate (which has
  none). The retrofit path is unchanged (original == target there).
- parsePartitionDate's week-suffix guard rejects out-of-range ISO week numbers
  (e.g. w00, w54), not just non-numeric/legacy-prefixed suffixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ediates

No behavior change (the ISO-week tests cover year boundaries and week 53):
name the week's Monday and Thursday explicitly and avoid mutating a value
before its name is accurate, so the trickiest date math is easier to audit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…antics

Comment-only (no behavior change): note the intentional inequality asymmetry
between extendRanges and extendRangesBackward, and document that
assertContiguous checks only finite bounds (MINVALUE/MAXVALUE/DEFAULT are
ignored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an optional per-table `format` template (stored in the settings comment,
e.g. `p{YYYY}w{WW}` or `y{YYYY}m{MM}`) so a retrofitted table can keep its
existing partition naming convention. Placeholders resolve to the same date
components already used — ISO week-year/week for weekly tables, calendar
year/month/day otherwise — so a custom format changes only the rendered name,
never which dates a partition covers. A single shared spec drives both
formatDateSuffix and parsePartitionDate, keeping them symmetric; literals are
restricted to [a-z0-9] since the suffix is recovered by splitting on `_`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bounds-anchored extension emits new partition boundaries at UTC midnight,
so a table whose existing partitions sit on non-midnight boundaries cannot
be extended contiguously. This previously surfaced either as a silent
no-op (the midnight-rounded horizon falls short of the non-midnight anchor,
so zero candidates are generated) or, with a larger --future, a
partition-overlap error at CREATE time. Detect a non-midnight anchor (the
max upper / min lower bound) up front and throw a clear error instead of
stalling silently.

Also parse the settings comment by splitting on the first ":" only, so a
value containing a colon (e.g. a mistyped format template) is preserved
and validated downstream rather than truncated at the second colon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
formatDateSuffix recompiled the partition-name template (a regex scan plus
allocation) and recomputed the ISO week twice on every render — once per
{YYYY}/{WW} placeholder — so a large back-fill repeated that work for each
partition. Cache compileFormat by (period, template) and share a single
isoWeekInfo result between the two weekly placeholders within a render.
Also drop an unused zod import from pgslice.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codybswaney codybswaney requested a review from ryanmcilmoyl June 30, 2026 17:22
@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown

Greptile Summary

This PR expands partition provisioning support for existing and new partitioned tables. The main changes are:

  • Bound-based extension for already-partitioned tables without renaming existing leaves.
  • Weekly partition support with ISO-week naming and custom suffix formats.
  • UTC-pinned partition boundary rendering with rejection for unsupported non-midnight bounds.
  • Parent-owned and composite primary key handling for new leaves.
  • Optional grant inheritance for newly created partitions.
  • Tests and reusable fixtures for legacy table shapes, custom formats, and coverage reporting.

Confidence Score: 5/5

The changes are well-scoped to partition planning, naming, boundary rendering, and provisioning behavior, with broad test coverage for the new supported table shapes.

The implementation is backed by expanded unit and command coverage across daily, weekly, legacy-shape, custom-format, grant, and key-inheritance cases, and no code issues were identified in the reviewed diff.

T-Rex T-Rex Logs

What T-Rex did

  • Observed the pre-state of the real-bounds test; the baseline showed No settings found: public.evt with evt_default and evt_legacy_week available.
  • Observed the post-state of the real-bounds test; the run created partitions ["evt_2026w25", "evt_2026w26"] with contiguous relpartbound bounds after the legacy partition and DEFAULT unchanged.
  • Captured artifacts documenting the real-bounds runs, including the commands executed, working directory, revisions/test output, and exit codes.
  • Grants-keys tests could not proceed due to environment checks failing (PGSLICE_URL must be set and Docker not found), preventing local Postgres provisioning.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "perf: Cache compiled formats and memoize..." | Re-trigger Greptile

@codybswaney codybswaney requested a review from a team July 1, 2026 14:19
@codybswaney codybswaney merged commit abac09a into master Jul 1, 2026
7 checks passed
@codybswaney codybswaney deleted the feature/pgslice-partition-shapes branch July 1, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants