Skip to content

CASSANDRA-21550 Phase1 BIG splitter only - #5063

Open
clohfink wants to merge 3 commits into
apache:trunkfrom
clohfink:phase1-big-splitter-only
Open

CASSANDRA-21550 Phase1 BIG splitter only#5063
clohfink wants to merge 3 commits into
apache:trunkfrom
clohfink:phase1-big-splitter-only

Conversation

@clohfink

Copy link
Copy Markdown
Contributor

Summary

This adds an opt-in --zero-copy path to sstablesplit for compressed BIG-format SSTables.

Instead of deserializing and recompressing partitions, the splitter retains existing compression chunks and rebuilds the child SSTables’ indexes and metadata. On supported Linux filesystems it uses byte-range reflinks to share data extents; otherwise it copies the compressed bytes without rewriting rows.

The implementation:

  • Supports compressed BIG pa-or-later SSTables and writes pb children.
  • Requires storage_compatibility_mode: NONE.
  • Rejects uncompressed SSTables, SSTable-attached indexes, and unsupported components.
  • Persists the first indexed partition position so readers, streaming, scrub, verification, loading, and upgrades correctly handle retained chunk prefixes.
  • Treats the requested size as a maximum unless a single partition’s compression-chunk span exceeds it.
  • Falls back safely to copying when reflinks are unavailable.

Commits

  1. ce328da5550a129d1d5d8377afd16da458129514Add zero-copy splitting for compressed BIG SSTables

    Contains the production implementation and sstablesplit documentation, including the splitter, index rebuilding, reflink support, metadata changes, and updates to affected SSTable consumers.

  2. c07e4931783181e87054484b7b51626b21f6300bAdd tests for zero-copy SSTable splitting

    Contains only tests covering splitting and sizing, reflink behavior and fallback, metadata and version compatibility, corruption handling, streaming, scrub, verification, upgrades, memory-mapped regions, and the command-line tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant