test(reader): end-to-end delete-application tests (position, equality, mixed) - #3087
Open
JanKaul wants to merge 1 commit into
Open
test(reader): end-to-end delete-application tests (position, equality, mixed)#3087JanKaul wants to merge 1 commit into
JanKaul wants to merge 1 commit into
Conversation
…, mixed) Add a `delete_application` test module that exercises `ArrowReader::read()` over a data file plus one or more delete files and asserts the surviving rows. The reader already loads and applies position and equality deletes, but the existing tests cover the loader / predicate / row-group pieces in isolation; there was no end-to-end coverage for scenarios that combine or stack delete files. New tests: - a single equality delete removes exactly the matching rows, - multiple equality delete files targeting one data file (keys unioned), - multiple position delete files targeting one data file (positions unioned), - mixed position + equality deletes on the same data file, - a data file whose every row is deleted yields no rows. The module uses only the public reader API (ArrowReaderBuilder / FileScanTask / FileScanTaskDeleteFile / FileIO) and a local filesystem FileIO, so it needs no external services. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jan Kaul <jankaul@mailbox.org>
JanKaul
force-pushed
the
test/reader-delete-application
branch
from
August 28, 2026 07:05
77e3d43 to
f73e7d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None — this is additional test coverage.
What changes are included in this PR?
ArrowReaderalready loads and applies both position and equality deletes, but the existing tests cover the loader / predicate / row-group pieces in isolation. There's no end-to-end coverage (read()→ assert surviving rows) for scenarios that combine or stack delete files.This adds an
arrow::reader::delete_applicationtest module with five end-to-end tests:Each test writes a small data file and its delete file(s) to a
TempDir, reads them throughArrowReader::read()with a local-filesystemFileIO, and asserts the survivingids.Are these changes tested?
Yes — the five new tests pass (
cargo test -p iceberg). They use only the public reader API (ArrowReaderBuilder,FileScanTask,FileScanTaskDeleteFile,FileIO::new_with_fs) and need no external services.