Conversation
github-actions
Bot
force-pushed
the
master
branch
4 times, most recently
from
6850c08 to
3b6f413
Compare
github-actions
Bot
force-pushed
the
master
branch
10 times, most recently
from
c3ee91f to
c7e7712
Compare
github-actions
Bot
force-pushed
the
master
branch
13 times, most recently
from
c2a5026 to
d8fed9e
Compare
- Hourly upstream sync from postgres/postgres (24x daily) - AI-powered PR reviews using AWS Bedrock Claude Sonnet 4.5 - Multi-platform CI via existing Cirrus CI configuration - Cost tracking and comprehensive documentation Features: - Automatic issue creation on sync conflicts - PostgreSQL-specific code review prompts (C, SQL, docs, build) - Cost limits: $15/PR, $200/month - Inline PR comments with security/performance labels - Skip draft PRs to save costs Documentation: - .github/SETUP_SUMMARY.md - Quick setup overview - .github/QUICKSTART.md - 15-minute setup guide - .github/PRE_COMMIT_CHECKLIST.md - Verification checklist - .github/docs/ - Detailed guides for sync, AI review, Bedrock See .github/README.md for complete overview Complete Phase 3: Windows builds + fix sync for CI/CD commits Phase 3: Windows Dependency Build System - Implement full build workflow (OpenSSL, zlib, libxml2) - Smart caching by version hash (80% cost reduction) - Dependency bundling with manifest generation - Weekly auto-refresh + manual triggers - PowerShell download helper script - Comprehensive usage documentation Sync Workflow Fix: - Allow .github/ commits (CI/CD config) on master - Detect and reject code commits outside .github/ - Merge upstream while preserving .github/ changes - Create issues only for actual pristine violations Documentation: - Complete Windows build usage guide - Update all status docs to 100% complete - Phase 3 completion summary All three CI/CD phases complete (100%): ✅ Hourly upstream sync with .github/ preservation ✅ AI-powered PR reviews via Bedrock Claude 4.5 ✅ Windows dependency builds with smart caching Cost: $40-60/month total See .github/PHASE3_COMPLETE.md for details Fix sync to allow 'dev setup' commits on master The sync workflow was failing because the 'dev setup v19' commit modifies files outside .github/. Updated workflows to recognize commits with messages starting with 'dev setup' as allowed on master. Changes: - Detect 'dev setup' commits by message pattern (case-insensitive) - Allow merge if commits are .github/ OR dev setup OR both - Update merge messages to reflect preserved changes - Document pristine master policy with examples This allows personal development environment commits (IDE configs, debugging tools, shell aliases, Nix configs, etc.) on master without violating the pristine mirror policy. Future dev environment updates should start with 'dev setup' in the commit message to be automatically recognized and preserved. See .github/docs/pristine-master-policy.md for complete policy See .github/DEV_SETUP_FIX.md for fix summary Optimize CI/CD costs by skipping builds for pristine commits Add cost optimization to Windows dependency builds to avoid expensive builds when only pristine commits are pushed (dev setup commits or .github/ configuration changes). Changes: - Add check-changes job to detect pristine-only pushes - Skip Windows builds when all commits are dev setup or .github/ only - Add comprehensive cost optimization documentation - Update README with cost savings (~40% reduction) Expected savings: ~$3-5/month on Windows builds, ~$40-47/month total through combined optimizations. Manual dispatch and scheduled builds always run regardless.
github-actions
Bot
force-pushed
the
master
branch
6 times, most recently
from
14db4ac to
bad9884
Compare
A seqlock protects read-mostly, rarely-written shared data with a single sequence counter (even = stable, odd = write in progress). Readers take no lock and pay no atomic read-modify-write and no StoreLoad fence on the common path: they read the counter, copy the data into local variables, re-read the counter, and retry if it changed. A writer -- serialized by the caller's own mutex -- bumps the counter odd, mutates the single copy in place, and bumps it even. This is cheaper than a left-right read (which needs a per-read SeqCst fence) and far cheaper than an LWLock shared acquire (a CAS on a shared counter), at the cost that readers may retry under a concurrent writer, so it suits read-mostly data with short, infrequent writes. Documented in the lmgr README and covered by a test_seqlock module that verifies the counter transitions, the read/retry handshake, and torn-read rejection.
Introduce two TableAmRoutine booleans and the begin_bulk_insert callback that the UNDO subsystem builds on, plus the RelationAmSupportsUndo() accessor index AMs use to gate UNDO record generation on the parent table. am_supports_undo marks an AM that registers an UNDO resource manager and emits UNDO records tagged with its own rmid; the UNDO core stays AM-agnostic and interprets the payload only through that RM's callbacks. am_inplace_update_keeps_tid marks an AM that updates in place and keeps the row's TID, so the executor can skip redundant index re-inserts for unchanged keys. The heap AM leaves both false. This commit only adds the routine fields and the accessor; no AM sets the flags yet.
github-actions
Bot
force-pushed
the
master
branch
8 times, most recently
from
44349f6 to
7e77e41
Compare
pgstat_relation.c hardcodes heap semantics for every table AM: every UPDATE is assumed to leave behind one dead tuple (delta_dead_tuples += tuples_updated + tuples_deleted), which is true for heap (a new tuple version plus an old one to reclaim) but wrong for any in-place-update access method that overwrites the committed tuple bytes and leaves nothing behind. Add a new TableAmRoutine capability flag, am_inplace_update_no_dead_tuple, alongside the existing am_inplace_update_keeps_tid. pgstat_count_heap_update() consults it per relation and tracks the subset of updates that created no dead tuple in a new PgStat_TableXactStatus counter (tuples_updated_no_dead); AtEOXact_PgStat_Relations() subtracts that count back out of delta_dead_tuples on both the commit and abort paths. TRUNCATE/DROP counter save/restore is updated to keep the new counter in sync with tuples_updated. Without this fix, an in-place-update AM correctly reports zero real dead tuples but pgstat inflates n_dead_tup by one per UPDATE anyway -- an accounting artifact, not real bloat, but one that also feeds autovacuums dead-tuple-threshold decision with phantom data. The heap AM leaves the new flag false (unchanged behavior, verified: heap n_dead_tup accounting is bit-for-bit identical before and after this change).
github-actions
Bot
force-pushed
the
master
branch
from
7e77e41 to
f9645af
Compare
github-actions
Bot
force-pushed
the
master
branch
from
f9645af to
7ca55a1
Compare
Add the AM-agnostic UNDO engine: the in-WAL UNDO record format and insertion path, the per-relation RELUNDO fork with its own resource manager, the shared sLog tuple-state map, the rollback apply driver and compensation-log generation, the discard horizon, and the background revert/undo workers. Register the UNDO, ATM, and RELUNDO resource managers and wire the subsystem into transaction start/commit/abort, two-phase commit, recovery, and process startup. The engine interprets UNDO payloads only through per-RM callbacks and the RelUndo*_hook function pointers (defined here, left NULL), so the core has no compile-time knowledge of any specific access method. Heap, vacuum, pruning, reloptions, and executor integration consume only the AM-agnostic interfaces. No UNDO-producing AM is registered yet: RegisterUndoRmgrs() initializes the dispatch table but registers no per-AM handlers. The index-AM apply handlers and the AM that sets am_supports_undo arrive in later commits.
Add the UNDO resource-manager handlers for the nbtree and hash index AMs and register them from RegisterUndoRmgrs(). On rollback of an aborting transaction, the nbtree handler re-descends to the leaf entry by key and heap TID before marking it dead, so a committed entry that shifted onto the recorded slot under concurrent inserts or leaf splits is never killed; entries inside posting-list tuples are left for VACUUM. The hash handler reverses its own inserts analogously. Both are gated by RelationAmSupportsUndo() on the parent table, so they are inert until an UNDO-supporting table AM exists.
Add pg_setxattr/pg_getxattr/pg_removexattr/pg_listxattr wrappers over the platform extended-attribute syscalls (Linux/*BSD/macOS, no-op stubs where unsupported) and build them into libpgport. The transactional file-ops resource manager added next uses these to record and reverse xattr mutations.
Filesystem mutations the server performs outside the buffer manager -- creating and removing directories, copying trees, writing version files, managing symlinks, setting extended attributes -- historically had no WAL coverage and relied on best-effort cleanup callbacks that do not survive a crash. Add FILEOPS: a resource manager (RM_FILEOPS_ID) that records each mutation as a deferred pending operation, executes it at commit, and WAL-logs it so redo reproduces it during crash recovery and standby replay. This commit adds the operation engine, the public FileOps* API, the WAL record formats, the redo handler, and the rmgrdesc descriptors. A README under src/backend/storage/file explains why FILEOPS exists and how to use the API. The rollback (UNDO) side and the rewiring of existing callers follow in subsequent commits.
PostgreSQL's heap appends a new tuple version on every UPDATE and abandons the old one to VACUUM. For update-heavy workloads on narrow hot rows -- counters, queue heads, running aggregates, time-series tail writes -- the resulting version churn dominates buffer traffic, bloats relations, and keeps autovacuum permanently behind. RECNO is a table access method (amname "recno", RM_RECNO_ID) that updates tuples in place. An UPDATE overwrites the committed bytes on the main fork; the prior image is preserved in the per-relation UNDO fork (RELUNDO_FORKNUM) so a ROLLBACK restores it and a concurrent snapshot reader can still reconstruct the version it is entitled to see. Heap is untouched; a relation opts in with USING recno. Visibility is driven by a hybrid logical clock rather than per-tuple xmin/xmax scans. Each tuple header carries an HLC commit stamp; a snapshot captures an HLC horizon and a tuple is visible when its stamp precedes the horizon and its writer has committed. Transient state (uncommitted / deleted / updated) lives in header flag bits that UNDO clears on rollback through the RelUndoClearTransientFlags hook, and in-place delta reversal runs through RelUndoReverseDelta; RECNO installs both at init via RecnoRelUndoInstallHooks() so the UNDO core never sees a RECNO tuple layout. Same-page updates rewrite the slot directly; updates that no longer fit spill to an overflow chain with optional per-attribute dictionary compression (LZ4/ZSTD, ANALYZE-refreshed). A sparsemap-backed dirty map and a partitioned secondary log (sLog) serve before-images to old readers without a separate version-store cache; free space and visibility are tracked in RECNO-private FSM and VM forks. WAL coverage is complete: insert, in-place and out-of-place update, delete, tuple-lock, VACUUM, dict writes, and overflow all log redo records under RM_RECNO_ID with matching recovery routines, and crash recovery cooperates with the UNDO driver to roll back in-place loser transactions. Index AMs over a RECNO table force a heap recheck on index-only scans and tolerate stale index entries left by in-place updates until before-image reclamation retires them. The commit also wires RECNO into the supporting infrastructure: pg_am / pg_proc catalog entries, the recno rmgr description routine, pageinspect coverage (pageinspect 1.13 -> 1.14), the in-tree documentation chapter, and isolation, regression, and crash-recovery test suites exercising MVCC correctness, overflow, dictionary compression, and dual-mode UNDO rollback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.