Skip to content

feat: implement ReplacePartitions (two-PR plan) #775

Description

@shangxinli

Overview

This issue tracks the implementation of ReplacePartitions in C++, assigned in #637.

ReplacePartitions performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses operation="overwrite" and summary property "replace-partitions"="true".

Implementation Plan

PR 1 — Core implementation

New files:

  • src/iceberg/update/replace_partitions.h
  • src/iceberg/update/replace_partitions.cc

Modified files:

  • src/iceberg/snapshot.h — add kReplacePartitions constant to SnapshotSummaryFields
  • src/iceberg/type_fwd.h — add class ReplacePartitions;
  • src/iceberg/CMakeLists.txt — register new source

PR 2 — Table/Transaction API surface + tests

Modified files:

  • src/iceberg/table.h / table.cc — add NewReplacePartitions()
  • src/iceberg/transaction.h / transaction.cc — add NewReplacePartitions()

New files:

  • src/iceberg/test/replace_partitions_test.cc
  • src/iceberg/test/CMakeLists.txt — register test

Design

ReplacePartitions extends SnapshotUpdate (same pattern as FastAppend) and uses ManifestFilterManager to mark existing partition files as DELETED:

  • AddFile(file): calls filter_manager_.DropPartition(spec_id, partition), accumulates new file in new_data_files_by_spec_
  • ValidateAppendOnly(): calls filter_manager_.FailAnyDelete()
  • Apply(): for unpartitioned tables calls filter_manager_.DeleteByRowFilter(AlwaysTrue); calls FilterManifests() on existing snapshot, then prepends new manifests
  • operation(): returns DataOperation::kOverwrite
  • Summary(): sets "replace-partitions"="true"

Relates to: #637

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions