Overview
This issue tracks the implementation of ReplacePartitions in C++, assigned in #637.
ReplacePartitions performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses operation="overwrite" and summary property "replace-partitions"="true".
Implementation Plan
PR 1 — Core implementation
New files:
src/iceberg/update/replace_partitions.h
src/iceberg/update/replace_partitions.cc
Modified files:
src/iceberg/snapshot.h — add kReplacePartitions constant to SnapshotSummaryFields
src/iceberg/type_fwd.h — add class ReplacePartitions;
src/iceberg/CMakeLists.txt — register new source
PR 2 — Table/Transaction API surface + tests
Modified files:
src/iceberg/table.h / table.cc — add NewReplacePartitions()
src/iceberg/transaction.h / transaction.cc — add NewReplacePartitions()
New files:
src/iceberg/test/replace_partitions_test.cc
src/iceberg/test/CMakeLists.txt — register test
Design
ReplacePartitions extends SnapshotUpdate (same pattern as FastAppend) and uses ManifestFilterManager to mark existing partition files as DELETED:
AddFile(file): calls filter_manager_.DropPartition(spec_id, partition), accumulates new file in new_data_files_by_spec_
ValidateAppendOnly(): calls filter_manager_.FailAnyDelete()
Apply(): for unpartitioned tables calls filter_manager_.DeleteByRowFilter(AlwaysTrue); calls FilterManifests() on existing snapshot, then prepends new manifests
operation(): returns DataOperation::kOverwrite
Summary(): sets "replace-partitions"="true"
Relates to: #637
Overview
This issue tracks the implementation of
ReplacePartitionsin C++, assigned in #637.ReplacePartitionsperforms dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot usesoperation="overwrite"and summary property"replace-partitions"="true".Implementation Plan
PR 1 — Core implementation
New files:
src/iceberg/update/replace_partitions.hsrc/iceberg/update/replace_partitions.ccModified files:
src/iceberg/snapshot.h— addkReplacePartitionsconstant toSnapshotSummaryFieldssrc/iceberg/type_fwd.h— addclass ReplacePartitions;src/iceberg/CMakeLists.txt— register new sourcePR 2 — Table/Transaction API surface + tests
Modified files:
src/iceberg/table.h/table.cc— addNewReplacePartitions()src/iceberg/transaction.h/transaction.cc— addNewReplacePartitions()New files:
src/iceberg/test/replace_partitions_test.ccsrc/iceberg/test/CMakeLists.txt— register testDesign
ReplacePartitionsextendsSnapshotUpdate(same pattern asFastAppend) and usesManifestFilterManagerto mark existing partition files as DELETED:AddFile(file): callsfilter_manager_.DropPartition(spec_id, partition), accumulates new file innew_data_files_by_spec_ValidateAppendOnly(): callsfilter_manager_.FailAnyDelete()Apply(): for unpartitioned tables callsfilter_manager_.DeleteByRowFilter(AlwaysTrue); callsFilterManifests()on existing snapshot, then prepends new manifestsoperation(): returnsDataOperation::kOverwriteSummary(): sets"replace-partitions"="true"Relates to: #637