fix: number bare ? placeholders at parse time, in text order (found by SQE) - #712
Open
dpsiderius wants to merge 2 commits into
Open
dpsiderius wants to merge 2 commits into
dpsiderius wants to merge 2 commits into
Conversation
`UPDATE t SET v = ? WHERE k = ?` bound its two values backwards, matched nothing, and returned `Ok(0)`. That is also the rows-affected value an optimistic-concurrency check reads as "someone else won the race", so a compare-and-swap built on it failed every time while looking like ordinary contention. On a table with a usable index, a projection of index columns only collapsed both placeholders onto index 1 and reported wanting one parameter when it had two. Both were the same cause. The index for a bare `?` was assigned during code generation, from a counter on `RegAlloc`, which makes it a property of compilation rather than of the SQL text. Compilation breaks that two ways: `compile_update` compiles the `WHERE` operand before the `SET` assignments, so the numbering came out reversed; and eight sites call `RegAlloc::new()`, each starting the counter at zero, so a plan that compiles part of a statement through a second allocator restarts numbering mid-statement. Assign the index in the parser instead, in text order, and carry it on `ParamKind::Anonymous(u32)` — which is where SQLite assigns it (`sqlite3ExprAssignVarNumber`) and why its numbering cannot depend on the plan. `Parser` holds one high-water mark, per-statement by construction because every parse entry point builds its own `Parser`. `?NNN` raises the mark and a following bare `?` continues past it. `RegAlloc`'s two parameter methods and the field they mutated are deleted, being dead afterwards. Explicit `?NNN` was never affected, which is why this survived: the existing parameter tests are written with `?1`/`?2`, the form this repository's own code writes. Drivers emit bare `?`. `tests/unit/param_numbering_test.rs` asserts the property on the AST rather than on a compiled program, so no future plan can reintroduce it; all 8 of its tests fail if the numbering is collapsed. Found by SQE driving the embedding API with sqlx-shaped SQL. ADR-0044 records why the alternative — a numbering pass between parse and codegen — was rejected: it needs a visitor reaching every expression position in every statement type, and a position it misses is this same bug, silently. Spend: small, matched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Introduces an `[Unreleased]` heading, which this file did not have. The versioning policy above it is about which *minor* version a completed plan phase ships as; it says nothing about where a fix lands before its version is cut, and Keep a Changelog — which the file's own header says it follows — answers that with `[Unreleased]`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
UPDATE t SET v = ? WHERE k = ?bound its two values in the wrong order. Itmatched no row and returned
Ok(0)— which is also the rows-affected value anoptimistic-concurrency check reads as "someone else won the race", so a
compare-and-swap built on it failed 100% of the time while looking like
ordinary contention. Nothing errored, and
param_countcorrectly said 2.Separately,
SELECT <index columns only> FROM t WHERE a = ? AND b = ?on atable with a usable index reported
statement wants 1 parameter(s) but 2 were bound.These were the same bug. The index for a bare
?was assigned during codegeneration, from a
next_paramcounter onRegAlloc. That makes the index aproperty of compilation rather than of the SQL text, and compilation breaks
that two ways:
compile_updatecompiles the
WHEREoperand before theSETassignments, because the scanmust be positioned before the row body is emitted
(
src/codegen/stmt/update.rs:186). So theWHEREplaceholder took index 1and the
SETplaceholder took 2.RegAllocper statement. Eight sites callRegAlloc::new(), each starting the counter at zero. The covering-index seekpath compiles through a second one, so numbering restarted mid-statement and
both placeholders became index 1 — hence
param_count= max = 1.So the general statement is broader than either symptom: bare
?numberingdepended on visit order and plan shape, for every statement type. These two
are the shapes that happened to surface.
The fix
Assign the index in the parser, in text order, and carry it on
ParamKind::Anonymous(u32). That is where SQLite assigns it(
sqlite3ExprAssignVarNumber) and why its numbering cannot depend on the plan.Parserholds one high-water mark, per-statement by construction because everyparse entry point builds its own
Parserfor one statement's tokens.?NNNraises the mark and a following bare
?continues past it.RegAlloc::anonymous_paramandRegAlloc::numbered_paramare deleted alongwith the field they mutated, being dead afterwards.
ADR-0044 records why the alternative — a numbering pass between parse and
codegen — was rejected: it needs a visitor reaching every expression position
in every statement type, and a position it misses is this same bug, silently,
in a shape nobody has tested. Assigning at parse makes "was this numbered?"
unrepresentable rather than merely tested.
Why this survived
Explicit
?NNNwas never affected, and every existing parameter test iswritten with
?1/?2— the form this repository's own code writes. Driversemit bare
?.tests/corpus/api_oracle_test.rs::parameterised_writes_match_the_oracleon #705 used
?1/?2throughout, so it was testing our dialect rather than aconsumer's.
Found by SQE driving the embedding API with sqlx-shaped SQL, not by our own
suite. That is the part worth remembering.
Tests
tests/unit/param_numbering_test.rs, 8 tests, asserting on the AST ratherthan on a compiled program — deliberately, so the property is pinned where it
is now decided and no future plan can reintroduce the divergence. Covers
UPDATE/SELECT/INSERT/DELETEleft-to-right order, the?NNNhigh-waterinteraction, a repeated
?NNNcounting once, and per-statement reset.All 8 fail under a mutant that collapses every bare
?to index 1.Verified end to end against the reporter's exact statements, with this commit
cherry-picked onto #705's facade branch:
Gates
make testall green,make test-corpus387/387 (unchanged frommain'sbaseline),
make lintexit 0 on both clippy passes,make check-mod-filesclean.
make check-mvl-limitnot run locally (cargo-mvl-limitnot installed);no lifetimes,
dynorunsafeadded, so CI should confirm.Relationship to #705
Independent, and #705 does not need to wait. This is a pre-existing engine
bug, not something the facade introduced — but the facade is what made it
reachable, since there was no writable parameterised entry point before it.
One trivial conflict to expect: this adds ADR-0044 and #705 adds ADR-0043, so
.openspec/adr/index.mdconflicts on adjacent lines. Whichever merges secondtakes both entries.
Note also that #705's spec 013 Requirement 1 presents rows-affected as a
reliable conflict signal. That was not true for bare
?until this lands.spend: small, matched.
Refs: 013/Req-1, 013/Req-3, #705
🤖 Generated with Claude Code