CASSANDRA-21607: Fix reads and compaction of retained complex cells - #5065
Open
1fanwang wants to merge 7 commits into
Open
CASSANDRA-21607: Fix reads and compaction of retained complex cells#50651fanwang wants to merge 7 commits into
1fanwang wants to merge 7 commits into
Conversation
frankgh
requested changes
Aug 26, 2026
frankgh
left a comment
frankgh
left a comment
Contributor
There was a problem hiding this comment.
This is not sufficient. You still won't be able to read the table after the compaction occurred. In your test after line 113, if you select * from the table, it will also produce an error.
frankgh
reviewed
Aug 26, 2026
1fanwang
force-pushed
the
fix-cassandra-21607
branch
from
August 26, 2026 17:55
7abfb6e to
51a909f
Compare
Contributor
I added a minimal shell script to the issue which demonstrates this. |
1fanwang
changed the title
Contributor
|
This is what I'm seeing with your patch, after running the steps mentioned in the JIRA: and the stacktrace of the error is: |
Iterator compaction can encounter complex cells newer than a column drop even when the current schema has no complex columns. Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
1fanwang
force-pushed
the
fix-cassandra-21607
branch
from
August 27, 2026 05:37
cac75a1 to
40e5a0c
Compare
Contributor
|
I have some suggestions so we don't have to filter, but I will only get to this tomorrow. |
fetches() answered true for every column, including ones dropped after the filter was built, while fetchedColumns() returned the fixed set it was built with. A read crossing a schema change then deserialized cells it could not serialize: IllegalStateException: [m] is not a subset of [] Skipping them here lets ReadCommand stop filtering. Signed-off-by: 1fanwang <1fannnw@gmail.com>
1fanwang
force-pushed
the
fix-cassandra-21607
branch
from
August 31, 2026 06:35
194a872 to
9b96953
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading or compacting a table fails when a dropped non-frozen collection still has cells newer than the drop timestamp. Both paths reach the same rows and throw:
Cannot invoke "org.apache.cassandra.db.rows.ComplexColumnData$Builder.newColumn(...)" because "this.complexBuilder" is nullRead response serialization and compaction share a row merger that sizes its complex-column state from the current schema. After the collection is dropped, that schema reports no complex columns, but older SSTables can still return newer cells that must survive. The merger now creates complex state only when those cells arrive. Separately, a wildcard column filter answered its fetch check with an unconditional yes while reporting a fixed set of fetched columns. A read whose filter was built before the drop therefore deserialized cells it could not then serialize. The filter now answers from the set it was built with, so those cells are skipped as rows are read rather than filtered out afterwards.
The regression creates real SSTables through CQL and flushes, then drops the map column. Native-protocol reads succeed before and after compaction, a command built before the drop no longer fetches the dropped column, and dropped-only rows do not consume forward, reverse, or paged read limits. Re-adding the map exposes both retained cells.
Jira: https://issues.apache.org/jira/browse/CASSANDRA-21607
Testing
Raw logs