Stop AnsiSkippingString treating literal 0xff bytes as its own sentinel - #3199
Open
afonsojanu wants to merge 1 commit into
Open
Stop AnsiSkippingString treating literal 0xff bytes as its own sentinel#3199afonsojanu wants to merge 1 commit into
afonsojanu wants to merge 1 commit into
Conversation
AnsiSkippingString marks the end of a recognized ansi escape sequence by overwriting its terminating 'm' with 0xff internally, then flips it back when producing a substring. The problem is that any literal 0xff byte already present in the original text (say, from stringifying raw bytes) looks exactly the same, so the code can't tell the two apart. substring() ends up turning real data bytes into 'm', and the reverse iterator can walk clean off the start of the string hunting for an escape sequence that was never there, hitting an assert in debug builds or worse in release. This tracks the byte offsets where a genuine sentinel was inserted and checks against those instead of comparing raw byte values, so a plain 0xff in the text is left alone in both places. Added a test that reproduces the crash on an unmodified build and passes with the fix. Closes catchorg#2960
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## devel #3199 +/- ##
==========================================
+ Coverage 91.17% 91.19% +0.02%
==========================================
Files 206 206
Lines 9031 9048 +17
==========================================
+ Hits 8234 8251 +17
Misses 797 797 🚀 New features to boost your workflow:
|
Member
|
In a funny coincidence, I did some work on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2960
AnsiSkippingStringmarks the terminatingmof a recognized ansi escape sequence by overwriting it with0xffinternally, and flips it back when a substring is produced. The trouble is a literal0xffbyte that was already part of the original text (for instance from stringifying raw byte data, as in the linked issue) looks exactly the same as one of these markers, and the existing code has no way to tell them apart.Two consequences of that, both reachable with plain text that has nothing to do with ansi escapes at all:
substring()blindly turns every0xffbyte in the output back into'm', silently corrupting real data.0xffbyte, assumes it must have found one of its own markers and starts scanning backwards for the\033that "must" precede it. If the byte was never one of ours, that scan walks straight off the beginning of the string, tripping an assert in debug builds (and undefined behavior in release, matching the exception/crash reports in the issue).The fix records the byte offsets where a real sentinel was inserted (
m_sentinelPositions) and checks membership there instead of comparing raw byte values, in bothsubstring()and the iterator'sunadvance()/tryParseAnsiEscapes(). A plain0xffbyte in the text is now left completely alone.Added a test (
TextFlow::Column handles literal 0xff bytes that are not ansi escapes) that:Ran the full self-test suite before and after; identical pass/fail counts either way (the one pre-existing
[!shouldfail]TextFlow test is unrelated to this).