Repository navigation
fix(translate): re-escape single-quoted literals for SQLite (mysqldump fidelity) - #62
Merged
Merged
Conversation
…p fidelity) sqlparser's Display leaves a single quote bare when a backslash precedes it (MySQL backslash-escaping), which SQLite and Turso misparse since they never interpret backslash escapes (NO_BACKSLASH_ESCAPES is effectively always on). A mysqldump replay silently dropped the rows whose text had a backslash immediately before a quote -- 45 Sendy campaigns rows in the wun load. Double every embedded quote in a SingleQuotedString before handing the AST to Display, so the emitted literal is the SQLite-correct form. Adds a mysqldump-fidelity regression test over the wire on both backends covering the escape shapes (incl. 'a\\'b'), and locking in that USING BTREE/HASH index hints parse-and-drop in CREATE TABLE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
mysqldumpreplay over the wire silently dropped rows whose text contained a backslash immediately before a single quote (the SQL source shape'a\\\'b'). In the wun Sendy load this lost exactly 45campaignsrows, which had to be backfilled via bound params.Root cause (emit-side, not the lexer)
sqlparser stores the fully-decoded value in the AST (the source
'a\\\'b'becomes the four bytesa\'b) — that part was already correct. But sqlparser'sDisplayforValue::SingleQuotedStringre-serializes for a backslash-escaping dialect: when a'is immediately preceded by a\, it emits the quote bare. SQLite/Turso never interpret backslash escapes, so they read the emitted literal as a string ending early followed by stray tokens — a syntax error standalone, or a silently dropped/garbled row inside a multi-row INSERT.Fix
A
VisitorMut(SqliteStringEscaper) doubles every embedded'→''in eachSingleQuotedStringbeforeDisplay. Once every quote sits in a''pair, Display has no lone quote to mishandle and emits the SQLite-correct literal verbatim; backslashes stay literal bytes. Applied inemit_statement(cold path, behindTranslateCache).Round-trip for the data-loss shape: source
'a\\\'b'now emits a literal SQLite decodes back to exactlya\'b.USING BTREE / HASH
These already parse-and-drop at
main(ef896fb) in every position tested (inline KEY/UNIQUE/PRIMARY, standalone CREATE INDEX, ALTER ADD KEY) — no code change needed; added a regression test to lock it in.Tests
New
crates/litewire/tests/mysql_dump_fidelity.rs(over the wire, rusqlite + turso): asserts exact stored bytes and no dropped row for 7 escape shapes including the data-loss shape and a multi-row INSERT; and that a CREATE TABLE withUSING BTREE/USING HASHloads and aUNIQUE KEY ... USING HASHenforces uniqueness. clippy--all-targets -D warningsand nightly fmt are clean.