Skip to content

Fix prepared handle ownership across streamed execution - #599

Merged
Saurabh Singh (saurabh500) merged 10 commits into
mainfrom
dev/saurabh/fix-streamed-parameter-rebind
Sep 26, 2026
Merged

Saurabh Singh (saurabh500) merged 10 commits into
mainfrom
dev/saurabh/fix-streamed-parameter-rebind

Conversation

@saurabh500

@saurabh500 Saurabh Singh (saurabh500) commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fix the third-rebind panic and prepared-handle leak after streamed execution is cancelled and retried.

Streamed sp_prepexec carries the old handle in its by-reference @handle, avoiding a separate cleanup round trip. The orphan and its cached encryption metadata remain owned until the complete message is sent. Only then is the new statement identity assigned, even if a post-send timeout or response failure occurs. Cancelling a parked request retains the old orphan without leaving an inert new identity that can overwrite it on the next rebind.

The new end_execute_prepared_param API closes prepared streams with the caller's statement and orphan slot. SQLParamData uses it for both immediate and deferred prepared DAE, restoring ownership before handling success or failure; ad-hoc streams continue using end_streamed_param. Reusing an already-live sp_execute still releases a separate cross-identity orphan through standalone sp_unprepare, because that RPC has no piggyback drop slot. Deviation entry 19 is narrowed to that case.

PacketWriter now distinguishes an attempted send from a successfully completed packet. A per-send incomplete flag survives suspension; shared retraction, streamed chunk failures, and withdrawal failures retire interrupted writes without IGNORE, attention, or TLS shutdown. This also covers interruption after earlier packets completed. Cancellation between complete packets remains retractable, and cancellation before any send remains locally discardable. Standalone unprepare uses the shared retirement path.

The shared timeout helper deducts whole elapsed seconds rather than rounding every sub-second step up, preserving one-second command budgets. Multi-step callers use the original budget and cumulative elapsed time; rounding leaves less than one second of slack, while exact/over-budget exhaustion still fails before sending.

Merged main (cc7bb82e) in 503fc43f; the parity registry now lives in mssql-odbc/docs/parity-deviations.md. Streamed piggyback landed in a8602cc9; the interrupted-send follow-up is 6297939e.

Merged main (50a315de) in 7bf8fc32, preserving its entry 18 and renumbering this PR's entry to 19. Post-merge workspace and Python formatting/Clippy pass; 4,066 workspace library tests pass. Live integration tests were not rerun locally; CI must validate the new merge commit.

Validation

Local validation for 6297939e:

  • cargo bfmt and separate mssql-py-core formatting check pass.
  • cargo bclippy, production mssql-tds --lib Clippy, and separate mssql-py-core --all-features --all-targets Clippy pass with -D warnings.
  • cargo btest -E 'kind(lib)': 4,022 workspace library tests passed under coverage. This is not a complete live integration-suite run.
  • Repeated one-second rebind/cancel/retry coverage asserts one streamed sp_prepexec with a non-NULL old handle and no standalone sp_unprepare. Separate tests cover cross-identity release, multiple parameters, cached encryption metadata, cancelled/failed final writes, and errors after a completed send.
  • The ODBC regression runs three cancel/rebind cycles and then a completed request with a failed response, checking that the pending slot is never overwritten.
  • The interrupted-send regression reproduced unsafe connection reuse before the fix. It now covers both chunk and final-packet sends, with and without a prior completed packet, asserting no cleanup bytes, attention, or transport shutdown. Repeated cancellation/chunk calls cannot write again. The healthy between-packets cancellation test confirms the connection remains reusable.
  • Mutation checks: reducing the partial-send payload below a packet fails the partial-send assertion; removing the injected write delay fails the timeout assertion; moving cancellation one send later fails the interrupted-send regression. All setups were restored. The pending-send double reproduces an interrupted write; the EOF response double tests ownership on an error, not absence of a network hang.
  • Live SQL Server/ODBC C++ end-to-end tests, native TLS interruption reproduction, and build-specific retail-driver wire/diagnostic comparisons were not rerun. Required CI must rerun on the pushed head; prior-head green checks do not validate this revision.

Historical results: a8602cc9 passed 4,020 library tests and 503fc43f passed 4,014; before the main merge, 2,211 TDS library tests and 3,911 workspace library tests passed. Seven initial missing-certificate failures cleared after running scripts/generate_mock_tds_server_certs.ps1. Earlier debug/release/reference-driver regressions and CI passed on 0b0bee43 (99% diff coverage), while an original full local run reported 3,925 passed / 405 failed / 14 skipped, mostly missing credentials/certificates. These historical results do not establish current-head live integration coverage; the earlier unqualified 2,009-test statement is superseded.

Related Issues

Fixes #598

The pre-existing materialized sp_prepexec piggyback ownership issue is tracked separately in #647.

Checklist

  • cargo bfmt passes
  • cargo bclippy passes
  • cargo btest passes (library-only selection passed; full live suite not rerun locally)
  • New/changed functionality has tests
  • Public API changes are documented

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An unsent cancelled sp_unprepare can discard the orphan ID and leak the live server handle.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates streamed prepared execution to release superseded handles before opening cancellable RPC streams.

Changes:

  • Adds standalone sp_unprepare handling with shared timeout accounting.
  • Adds unit and ODBC regressions for cancellation, rebinding, cleanup, and reuse.
  • Updates ownership documentation.
File summaries
File Description
mssql-tds/src/connection/tds_client.rs Implements orphan release and unit coverage.
mssql-odbc/src/api/execute.rs Updates streamed-execution ownership comments.
mssql-odbc/tests/e2e/tests/execute_test.cpp Adds immediate and deferred streaming regressions.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mssql-tds/src/connection/tds_client.rs Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Note

Copilot is running an experiment and ran this review at Balanced.

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

98%

🎯 Overall Coverage

94.3%

📦 Project: mssql-tds + mssql-odbc + mssql-py-core
ℹ️ Note: diff coverage is reported, not enforced.


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql-odbc/src/api/execute.rs (100%)
  • mssql-odbc/src/api/param_data.rs (42.9%): Missing lines 307-309,312-315,329-331,334-339
  • mssql-tds/src/connection/tds_client.rs (99.8%): Missing lines 2447,3721
  • mssql-tds/src/io/packet_writer.rs (100%)

Summary

  • Total: 1017 lines
  • Missing: 18 lines
  • Coverage: 98%

mssql-odbc/src/api/param_data.rs

  303     }
  304 
  305     let (mut prepared, mut orphaned) = {
  306         let Ok(mut stmt_state) = stmt.inner.lock() else {
! 307             error!("SQLParamData: stmt mutex poisoned taking the streamed plan");
! 308             return_client_idle(dbc, statement_handle, client);
! 309             return SQL_ERROR;
  310         };
  311         let Some(dae) = stmt_state.dae.as_mut() else {
! 312             error!("SQLParamData: DAE sequence vanished before closing the parameter");
! 313             drop(stmt_state);
! 314             return_client_idle(dbc, statement_handle, client);
! 315             return SQL_ERROR;
  316         };
  317         (dae.take_prepared(), dae.take_orphaned())
  318     };
  319     let end_result = match prepared.as_mut() {

  325     // The send may have completed even when its response failed. Restore the
  326     // updated ownership before either completion or error tears down DAE.
  327     {
  328         let Ok(mut stmt_state) = stmt.inner.lock() else {
! 329             error!("SQLParamData: stmt mutex poisoned restoring the streamed plan");
! 330             return_client_idle(dbc, statement_handle, client);
! 331             return SQL_ERROR;
  332         };
  333         let Some(dae) = stmt_state.dae.as_mut() else {
! 334             error!("SQLParamData: DAE sequence vanished after closing the parameter");
! 335             stmt_state.prepared = prepared;
! 336             stmt_state.pending_unprepare = orphaned;
! 337             drop(stmt_state);
! 338             return_client_idle(dbc, statement_handle, client);
! 339             return SQL_ERROR;
  340         };
  341         dae.restore_plan(prepared, orphaned);
  342     }

mssql-tds/src/connection/tds_client.rs

  2443                 drop(message);
  2444                 if send_incomplete {
  2445                     self.retire_without_writing();
  2446                 } else {
! 2447                     self.abort_streamed_write().await;
  2448                 }
  2449                 Err(e)
  2450             }
  2451         }

  3717         orphaned: &mut Option<StatementId>,
  3718         options: ExecuteOptions<'_>,
  3719     ) -> TdsResult<()> {
  3720         let Some(id) = *orphaned else {
! 3721             return Ok(());
  3722         };
  3723         let result = self.execute_sp_unprepare(id, true, options).await;
  3724         if !self.prepared_handles.contains_key(&id) {
  3725             *orphaned = None;


🔗 Quick Links

View Azure DevOps Build · Coverage Report

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@saurabh500
Saurabh Singh (saurabh500) marked this pull request as ready for review September 22, 2026 05:06
@saurabh500
Saurabh Singh (saurabh500) requested a review from a team as a code owner September 22, 2026 05:06
Copilot AI review requested due to automatic review settings September 22, 2026 05:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The cancellation-sensitive protocol and FFI ownership changes still lack a passing full post-merge validation run.

Review effort: Balanced
Findings: None

@Theekshna ttk (Theekshna) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unattended hourly review sweep. Checked out at 11eff55a (merge-base b758050e against main, 5 files, +657/-51). Read all prior inline comments, reviews, and issue comments first — Copilot's earlier bot review flagged an unsent-cancellation handle leak on execute_sp_unprepare, and the author's reply plus the current diff show it fixed and mutation-tested (a failing zero-byte-cancellation test that a bytes-removed control makes pass): message.send_attempted()/nothing_sent() on SuspendedMessage distinguish "definitely never touched the wire" from "began writing, fate unknown," and I independently verified the mechanism against tokio-util 0.7.19's RunUntilCancelledFuture::poll (biased to the future; an already-cancelled token short-circuits before the future is polled at all, so send_attempted staying false really does prove nothing was sent). That thread is resolved and I did not re-file it.

Local build note: cargo nextest run -p mssql-tds --lib fails to even compile in this sandbox (E9012/E9013 codegen errors in crc32fast's AVX-512/vpclmulqdq path against this toolchain, reproduced both with and without -C target-feature=-avx512f,...) — the same pre-existing environment problem noted on PR #620's review, unrelated to this diff. I relied on the Code Coverage Report bot comment (99% diff coverage, 1 missing line) plus static/call-site analysis instead of a local mutation pass. gh pr checks is still fully pending on this exact head (11eff55a, pushed ~2h ago after merging main); no failures yet, nothing to flag there, and I'm not triggering or waiting on it further.

msodbcsql parity (check 1): the reference tree's DropPrepHandle (sqlcfunc.cpp:730-763) and the piggyback sites in sqlccmd.cpp:1642-1660 and :8244-8262 piggyback a same-statement superseded handle onto the next sp_prepare/sp_prepexec's by-reference @handle parameter — including for DAE/cursor-open RPCs (sqlccmd.cpp:6722-6726). I initially read this PR's new standalone sp_unprepare-before-stream as a divergence from that (an extra round trip), but @handle is a single by-reference slot carrying one statement's own old→new handle; it has no second slot for an unrelated statement's orphaned handle. Grepping this crate, orphaned: &mut Option<StatementId> is exactly that cross-identity case (a handle superseded by a different PreparedStatement id), and it is never piggybacked anywhere in this codebase, streamed or not — execute_prepared_batch_inner's existing None => arm (tds_client.rs:3830-3832, unchanged by this PR) already released a same-shaped orphaned via a standalone call before this PR touched the streamed path. So this isn't a new or streamed-specific divergence; it's the pre-existing convention extended consistently. No finding.

Six-check accounting: (3) Divergences — N/A per check 1, nothing to add to .github/instructions/mssql-odbc.instructions.md. (4) Description — see Nit below; linked issue #598 matches the diff's intent, checklist's unchecked cargo btest is honest given CI is still pending. (5) Verbose slop — none; new doc comments explain an invariant or the ownership contract (e.g. send_attempted's doc, unprepare's retry-safety note), and the e2e tests' Benefits-from-mock-tds: framing matches an existing convention (call_routing_test.cpp, describe_param_test.cpp, etc.), not new filler. (6) Evidence — re-derived the diff-coverage claim directly against the Code Coverage Report comment: 99%, exactly one missing line (tds_client.rs:3644), which is the same line my Suggestion below is about. I could not re-run the description's local full-suite numbers (2,009 unit tests; 3,925/405/14) — that requires a live SQL Server and local fixtures this sandbox doesn't have — so I'm not disputing them, only noting they're unverified here; 309 #[test]/#[tokio::test] functions in tds_client.rs alone make the total plausible.

Severity Count
Blocking 0
Suggestion 1
Nit 1

[Nit] The description's checklist section ends with "Kept draft as requested," but this PR is not a draft (isDraft: false) and hasn't been since it was opened in this state. Leftover text like this reads as stale/contradictory to a reviewer checking currency — worth deleting.

This is an unattended automated review; findings may be incomplete or wrong, push back on anything that looks off.

Comment thread mssql-tds/src/connection/tds_client.rs

@Theekshna ttk (Theekshna) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 11eff55a (merge-base b758050e). Read the prior Copilot and ttk (@Theekshna) reviews first; not re-filing the :3644 guard or the draft-text nit. One blocking finding, reproduced locally, plus two suggestions. Mechanism itself checks out — I mutation-tested the send_attempted guard, the retire_without_writing call, and the set-point of send_attempted relative to the await; each mutation fails a test, so the new tests are not vacuous.

Comment thread mssql-tds/src/connection/tds_client.rs
Comment thread mssql-tds/src/connection/tds_client.rs Outdated
Comment thread mssql-tds/src/connection/tds_client.rs

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unattended automated review. Findings were not checked by a human first — push back on anything that looks wrong.

Summary

Reviewed at 11eff55a (merge base b758050e against current origin/main, 5 files, +657/−51). The change is coherent and the mechanism holds up: execute_sp_unprepare stops evicting prepared_handles / prepared_param_encryption up front and instead evicts them only once SuspendedMessage::send_attempted() proves bytes were handed to the writer, retiring the connection when a send started but no packet flushed. begin_execute_prepared then releases a superseded handle in a standalone sp_unprepare before parking the cancellable stream, so the orphan is no longer carried into DaeState and restored by take_dae into an already-occupied pending_unprepare — which is exactly the orphan_prepared_handle: a pending unprepare already exists panic in #598. I traced that path end to end (stmt.rs:1398 → :1430, execute.rs:392-440) and the fix matches the reported repro.

No blocking findings from me. One suggestion concerns the sibling release path the PR deliberately did not change, and I confirmed it with a probe rather than by reading.

I did not re-file the four findings already open from ttk (@Theekshna) (:3644 redundant guard, :3950 deduct_timeout re-charge, :3895 deviation-registry entry, :16070 test gap) or the stale "Kept draft as requested" line. On :3950 I independently confirmed the arithmetic is real — deduct_timeout (tds_client.rs:1446-1451) rounds elapsed up to whole seconds, and CommandTimeoutBudget::Exhausted maps to TimeoutError in into_timeout (:1449, :1155) — so with timeout=1 any nonzero sub-second elapsed exhausts the budget. Worth noting that the same re-charge already ships on the materialized path (execute_prepared_batch_inner, unchanged by this PR), so it is a pre-existing shape this PR extends rather than introduces.

Verification performed

  • Repos/policy. Fetched origin/main (120eab88) and refs/pull/599/head; merge base b758050e. Read .github/copilot-instructions.md, .github/instructions/pr-workflow.instructions.md, .github/instructions/mssql-odbc.instructions.md, and .github/skills/code-review/SKILL.md + posting.md from origin/main, not the checkout. Note: the brief for this run said the code-review skill file was absent — it is in fact present at origin/main:.github/skills/code-review/SKILL.md, so I applied it.
  • Prior discussion. Read all three endpoints (pulls/599/comments, pulls/599/reviews, issues/599/comments) plus every review thread and its resolution state before drafting. Read issue #598 in full.
  • Tests run locally (CARGO_TARGET_DIR redirected; main worktree cache untouched):
    cargo nextest run -p mssql-tds --lib -E 'test(orphan) + test(unprepare) + test(send_attempt) + test(retract_partial)' → 24 passed, 0 failed, including all six new ownership tests and io::packet_writer::tests::send_attempt_survives_suspend_and_resume.
  • Probe (the basis for the Suggestion below). I added a temporary test driving execute_sp_prepexec_for_test with cancel_on_writer_creation — the same "cancelled before the writer's first send" hook this PR adds — and printed the resulting ownership state:
    PROBE err=true bytes_sent=0 orphan=None map_has_handle=false dead=false
    Restored afterwards; git status --porcelain is empty and the worktree is clean.
  • Flush-ordering check. Confirmed any_packet_flushed / message_complete are set after send_data_fut.await? (packet_writer.rs:459-461) while send_attempted is set inside the cancellable future (:445-448), so the new send_attempted && nothing_sent pair genuinely distinguishes "never polled" from "polled, fate unknown". Verified the biased short-circuit semantics are what the cancel_on_writer_creation arm of begin_execute_prepared_release_cancellation_tracks_send_attempt relies on.
  • Not verified: the two new C++ e2e tests — no live SQL Server or ODBC driver registration in this environment. They rest on the ADO validation leg, which passed on this head. I also did not re-run the description's local full-suite numbers (2,009 unit tests; 3,925/405/14); I am not disputing them, only noting they are unverified here.

Findings

Blocking

None.

Suggestion

1. execute_sp_prepexec still discards the orphan before the send boundary — the same window this PR closes for the streamed path. (mssql-tds/src/connection/tds_client.rs:4233-4236)

This PR's own rationale on begin_execute_prepared says it best:

Piggybacking the drop would let cancellation discard it while the server still held the plan.

That is a precise description of what the piggyback path still does. orphan.take() and both map evictions happen at RPC-build time — before the packet writer even exists — so a cancellation landing before the first flush loses the orphan. Measured, not inferred:

PROBE err=true bytes_sent=0 orphan=None map_has_handle=false dead=false

Zero bytes reached the server, yet the caller is told there is nothing left to release and the client has forgotten handle 7. The server still holds that plan, so it leaks until disconnect. Contrast the new execute_sp_unprepare behaviour on the identical hook, which begin_execute_prepared_release_cancellation_tracks_send_attempt pins: orphaned == Some(id), both map entries retained, connection reusable, release retryable.

Two supporting details:

  • The doc contract at :4122-4128 says the orphan is "still Some if the call failed before serialization (the caller may retry the release)". The probe is a failure before anything was serialized to the wire, and it came back None — so the documented guarantee is narrower in practice than it reads.
  • The pre-existing test execute_sp_prepexec_consumes_orphan_after_send_begins passes for a weaker reason than its name suggests. Because the take happens at build time, it would pass even if no send ever began; nothing currently pins the pre-send arm. execute_sp_prepexec_preserves_orphan_on_pre_send_error only covers the has_open_batch usage error, which returns before the take.

This is pre-existing on main and this PR strictly narrows the gap, so I am not treating it as blocking. But send_attempted() now exists precisely to close it, and leaving two of three release paths on the new contract and one on the old is the kind of asymmetry a later parity pass gets wrong. Suggest a follow-up issue rather than expanding this PR.

Nit

1. The retire comment describes only one of the two conditions that reach it. (mssql-tds/src/connection/tds_client.rs:3621-3623)

if serialize_result.is_err() && message.nothing_sent() {
    // A cancelled write may have left a partial packet on the wire.
    self.retire_without_writing();
}

Since any_packet_flushed is set only after send_data_fut.await? succeeds, this branch is also taken when network_writer.send() returns a plain I/O error, not just on cancellation. Retiring is the right action in both cases, so this is wording only — but "a cancelled write" reads as an exhaustive description of when the connection gets condemned, and it is not.

Required CI

coverage-report is failing on this head (run 35689467030). Reading the log, it is not a test or coverage regression — the job polls for the Cobertura artifact and gave up after 12 attempts:

Attempt 12: checking for coverage artifact...
⏳ Coverage artifact not ready yet (attempt 12)...
Available artifacts: none
❌ Timeout: coverage artifact not found within 75 minutes

Every other required check is green, including the full mssql-rs Pull request validation ADO pipeline (all five build legs, Test MacOS, Kerberos, and both cross-repo mssql-python legs) and Merge Coverage. Since the ADO validation now passed on this exact head, the unchecked cargo btest box in the description looks like it can be checked.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 24, 2026 01:27
@saurabh500

Copy link
Copy Markdown
Contributor Author

execute_sp_prepexec still discards the orphan before the send boundary.

Confirmed the build-time orphan.take() and map eviction ordering and tracked the pre-existing materialized-path issue in #647, rather than expanding this streamed-cleanup fix. The interrupted-write wording is corrected in f30e7a7 as well.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Pending handle ownership must be preserved when cleanup fails before sending.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread mssql-tds/src/connection/tds_client.rs
@Theekshna ttk (Theekshna) added the ready for human review Automation flag indicating an item is ready for human review. label Sep 24, 2026

@David-Engel David Engel (David-Engel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review (unattended run)

Reviewed the full diff against merge-base cc7bb82e (10 files, +1438/-144), focused on the streamed sp_prepexec piggyback, the send-ownership boundary in PacketWriter, and the ODBC SQLParamData plan restoration. This came from an unattended run, so the findings below were not checked by a human first.

The design holds up: ownership settles on message_complete() rather than at park time, cancellation retains the orphan without minting an inert identity, and the ODBC layer restores the plan on both success and failure. Verified the new tests are not vacuous with two mutations (cargo nextest run -p mssql-tds --lib, 35 prepared/streamed tests):

  • forcing the streamed @handle back to SqlType::Int(None) fails begin_execute_prepared_piggybacks_each_orphan_with_one_second_timeout;
  • inverting the message.message_complete() gate fails 9 tests, including prepared_stream_partial_cancel_retains_orphan_and_metadata and prepared_stream_completed_send_consumes_orphan_on_error.

Also re-ran begin_execute_prepared_release_cancellation_tracks_send_attempt 40x to check the new send_attempted flag for a race when the handle is already cancelled: 40/40 deterministic, because CancellationToken::run_until_cancelled has an is_cancelled() fast path that never polls the inner future. No flakiness there.

No blocking findings. Two non-blocking items are left inline:

  1. Suggestion — withdraw_sent_message samples send_incomplete only after writing the IGNORE packet (tds_client.rs:2644).
  2. Nit — duplicated live_handle lookup in begin_execute_prepared (tds_client.rs:4034).

PR-wide (no attachable diff line): the two red checks are infrastructure, not this PR. The macOS job ran Run tests (mssql-tds and mssqlodbc packages) to success; it then failed on Publish Cobertura coverage artifact with ##[error]Artifact CoberturaCoverageRust_MacOS already exists for build 177888, which skipped the ODBC e2e steps and failed Publish ODBC e2e test results (macOS). coverage-report then failed only because it refuses to report against a failed ADO build. Worth a re-run rather than a code change — but the e2e C++ cases added here (RebindAfterCancelledStreamReleasesPriorHandle, RebindAfterDeferredStreamReleasesPriorHandle) have therefore not actually executed on macOS in this build.

Comment thread mssql-tds/src/connection/tds_client.rs
Comment thread mssql-tds/src/connection/tds_client.rs
@saurabh500

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unattended automated review. Findings were not checked by a human first — push back on anything that looks wrong.

Summary

Reviewed the full diff at 6297939e against merge base cc7bb82e (10 files, +1438/−144). The ownership redesign holds up under everything I could check here. The settle-on-message_complete() boundary in end_streamed_param_inner (tds_client.rs:2908-2923), the send_attempted / send_incomplete split in PacketWriter (packet_writer.rs:443-457), and the take/restore around end_execute_prepared_param in SQLParamData (param_data.rs:305-342) trace cleanly through the cancel, interrupted-send, completed-send-then-error, multi-parameter and cross-identity paths. I also confirmed the structural reason take_dae's unconditional self.pending_unprepare = dae.orphaned (stmt.rs:1444) is now safe: orphan_prepared_handle early-returns while a plan is parked in DAE because StmtState::prepared is None, so nothing can occupy the slot the panic in #598 tripped over.

No blocking findings. Two suggestions, both on the parity-registry entry this PR adds — the one ledger row no prior reviewer has audited.

Three reviews already cover this exact head (ttk (@Theekshna), David Engel (@David-Engel) with the :2644 suggestion and :4034 nit, and Copilot). I am not re-filing any of those, nor the resolved :3721, :4029, :4142 or :16591 threads. In particular I read David Engel (@David-Engel)'s :2644 thread before drafting, and I agree cancel_streamed_write → withdraw_sent_message is the path that relies on the invariant implicitly — nothing to add.

Verification

  • Live state. Confirmed repository microsoft/mssql-rs, PR #599 open, non-draft, author Saurabh Singh (@saurabh500), head 6297939e, base main, immediately before posting. mergeable: true.
  • Repos. Fetched origin/main (27707b21); merge base with this head is cc7bb82e. Read .github/copilot-instructions.md, .github/instructions/pr-workflow.instructions.md, .github/instructions/mssql-odbc.instructions.md and .github/skills/code-review/SKILL.md + posting.md from origin/main, not from the checkout.
  • Prior discussion. Read all three endpoints (pulls/599/comments, pulls/599/reviews, issues/599/comments) plus every review thread and its resolution state before drafting, and confirmed #598 and #647 are both open.
  • Tests run locally (detached worktree, CARGO_TARGET_DIR redirected so the main worktree cache is untouched):
    cargo nextest run -p mssql-tds --lib -E 'test(prepared_stream) + test(begin_execute_prepared) + test(deduct_timeout) + test(send_attempt) + test(retract_partial) + test(cancel_streamed_write)' → 37 passed, 0 failed.
  • Mutation (novel — checking the description's claim that the interrupted-send regression "now covers both chunk and final-packet sends"). In write_streamed_chunk's error arm (tds_client.rs:2445) I forced send_incomplete to false, so an interrupted chunk write would fall back to abort_streamed_write() (which closes the transport) instead of retire_without_writing(). cargo nextest run -p mssql-tds --lib -E 'test(prepared_stream) + test(begin_execute_prepared) + test(streamed)' → 50 tests, 49 passed, 1 failed: prepared_stream_interrupted_send_retires_without_writing_again. The chunk arm is genuinely guarded, not just the finalize arm. Restored afterwards; git status --porcelain shows only the untracked cargo-target/ this review created.
  • Branch coverage spot-checks. The new end_streamed_param_inner usage guard ("Prepared streams must be closed with end_execute_prepared_param", tds_client.rs:2850) is pinned — prepared_stream_partial_cancel_retains_orphan_and_metadata asserts it at :16263, and also that the guard leaves the stream usable afterwards. The attention-after-complete-send case is pinned by prepared_stream_completed_send_consumes_orphan_on_error (delayed_send = true, asserting attentions == 1 alongside orphaned.is_none()), which is what Suggestion 2 below is about.
  • Required CI. Every check is green on this head after the /azp run re-trigger: ADO mssql-rs Pull request validation build 178213 passes on all five build legs, Test MacOS (48m18s), Kerberos, both cross-repo mssql-python legs and Merge Coverage; GitHub coverage-report, CodeQL, Analyze and check all pass. The macOS Artifact CoberturaCoverageRust_MacOS already exists collision David Engel (@David-Engel) reported on the earlier build is not present in 178213.
  • Not verified. There is no msodbcsql checkout on this host (Test-Path on the three plausible roots returns False for all), so I could not re-derive the DropPrepHandle / BuildSPPrepExec / ProcessDAEParam citations in deviation entry 18 — Suggestion 2 is an internal-consistency argument against the entry's own stated reading, not a claim about msodbcsql. I also did not run the C++ e2e suite (no live SQL Server / driver registration here); RebindAfterCancelledStreamReleasesPriorHandle and RebindAfterDeferredStreamReleasesPriorHandle rest on ADO 178213. I did not re-run the description's 4,022-test figure.

Findings

Blocking

None.

Suggestion

Two, filed inline on mssql-odbc/docs/parity-deviations.md.

Nit

The cargo btest checklist box can be checked now. The description qualifies it as "library-only selection passed; full live suite not rerun locally", which was accurate when written — but ADO 178213 on this exact head runs Test MacOS, the Kerberos leg and both cross-repo mssql-python legs to green, so the full suite has run on this revision even though it did not run locally.

PR-wide note — the merge base has moved

origin/main is now 27707b21, nine commits ahead of this PR's base cc7bb82e, and three of those commits touch files this PR also edits:

File Change on main since cc7bb82e
mssql-odbc/src/api/exec_common.rs +294 (#643, #644)
mssql-odbc/docs/parity-deviations.md +21/−10 (#643)
mssql-odbc/src/handles/stmt.rs +24

I checked the two ways this normally bites and both are clear: GitHub reports mergeable: true, and main's registry still tops out at entry 17, so this PR's entry 18 does not collide. Nothing to do now — just worth a git fetch origin main and a re-diff right before merge rather than relying on 178213, which was computed against cc7bb82e.

Comment thread mssql-odbc/docs/parity-deviations.md
Comment thread mssql-odbc/docs/parity-deviations.md

@David-Engel David Engel (David-Engel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — generated by GitHub Copilot on behalf of David Engel (@David-Engel). This is not an approval and does not satisfy the human review requirement. Findings may be incomplete or wrong; push back on anything that looks off.

Summary

Reviewed the full diff at 6297939e against merge base cc7bb82e (10 files, +1438/−144). The ownership redesign holds up. Settling on message.message_complete() in end_streamed_param_inner rather than at park time, the send_attempted / send_incomplete split in PacketWriter, and the take/end_execute_prepared_param/restore_plan window in SQLParamData trace cleanly through cancel, interrupted-send, completed-send-then-error, multi-parameter and cross-identity paths. I re-derived the deduct_timeout round-up→truncate change against every call site in tds_client.rs and cursor_ops.rs: each one passes either a single reconnect_elapsed or a cumulative started.elapsed() measured before check_and_reconnect, so no caller double-charges or forgets recovery time. I also checked the streamed_params.is_empty() early return, which delegates to execute_prepared and therefore still assigns statement.id and consumes the orphan even though begin_execute_prepared no longer does so itself.

No blocking findings. One suggestion and one nit, both filed inline. Neither is about the Rust ownership mechanism.

Prior reviews from ttk (@Theekshna), David Engel (@David-Engel), Vahid Beiranvand (@Vahid-b) and Copilot already cover this exact head. I am not re-filing the :2644 withdraw_sent_message sampling suggestion, the :4034 duplicated live_handle nit, the two parity-deviations.md content suggestions, the checklist nit, or any resolved thread.

Verification

  • Live state. Confirmed author Saurabh Singh (@saurabh500), head 6297939e, base main, PR open and non-draft, immediately before posting.
  • Repos. Fetched origin/main (now 50a315de); merge base with this head is cc7bb82e. Read .github/skills/code-review/SKILL.md, .github/copilot-instructions.md and .github/instructions/mssql-odbc.instructions.md from the checkout.
  • Prior discussion. Read all reviews, issue comments and all 21 inline review comments before drafting.
  • Tests run locally (detached worktree, CARGO_TARGET_DIR redirected): cargo nextest run -p mssql-tds --lib -E 'test(prepared_stream) + test(begin_execute_prepared) + test(deduct_timeout) + test(send_attempt) + test(retract_partial) + test(unprepare)' → 36 passed, 0 failed, including send_attempt_survives_suspend_and_resume and all the new prepared-stream ownership tests.
  • Merge check. git merge-tree --write-tree HEAD origin/main reports exactly one conflict, in mssql-odbc/docs/parity-deviations.md. GitHub agrees: mergeable: CONFLICTING. Everything else auto-merges, including exec_common.rs, stmt.rs and parameters_plan.md.
  • Required CI. All 18 checks are green on this head, including ADO build 178213 (all five build legs, Test MacOS, Kerberos, both cross-repo mssql-python legs, Merge Coverage) and GitHub coverage-report, CodeQL, Analyze and check. The earlier macOS Cobertura artifact collision is gone.
  • Not verified. No live SQL Server or registered ODBC driver here, so RebindAfterCancelledStreamReleasesPriorHandle and RebindAfterDeferredStreamReleasesPriorHandle rest on ADO 178213. No msodbcsql checkout, so I could not re-derive the DropPrepHandle / BuildSPPrepExec / ProcessDAEParam citations in deviation entry 18. I did not re-run the description's 4,022-test figure.

Findings

Blocking

None.

Suggestion

  1. The new parity registry entry is numbered 18, but main has since landed its own entry 18 — this is the sole merge conflict on the PR right now (mssql-odbc/docs/parity-deviations.md, inline).

Nit

  1. The rewritten DaeState::orphaned doc omits the streamed sp_prepexec piggyback this PR is actually about (mssql-odbc/src/handles/stmt.rs, inline).

Comment thread mssql-odbc/docs/parity-deviations.md Outdated
Comment thread mssql-odbc/src/handles/stmt.rs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 25, 2026 20:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes cancellation-sensitive network and ODBC ownership behavior, while current-head platform and live validation remains in progress.

Review effort: Balanced
Findings: 1 Low severity

Open (1)

Comment thread mssql-tds/src/connection/tds_client.rs

@David-Engel David Engel (David-Engel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — generated by GitHub Copilot on behalf of David Engel (@David-Engel). This is not an approval and does not satisfy the human review requirement. Findings may be incomplete or wrong; push back on anything that looks off.

Summary

The ownership model holds up. sp_prepexec now carries the orphan in its by-reference @handle, end_execute_prepared_param settles both the new statement identity and the orphan eviction from message.message_complete() rather than from the Result, and PacketWriter's split of send_attempted / send_incomplete correctly distinguishes "never reached the wire" (locally discardable), "complete packets only" (retractable with EOM | IGNORE) and "interrupted mid-packet" (retire without writing again). The deduct_timeout truncation now matches deduct_query_timeout, and every multi-step caller I checked passes the original budget with cumulative elapsed time, so the sub-second slack stays bounded at under one second rather than accumulating. Parity entry renumbering to 19 is clean — the registry reads 1–19 with no gaps or duplicates. One suggestion below, on a new ODBC error path.

Verification

  • Merge base 50a315de; reviewed the full git diff $BASE..HEAD (10 files, +1438/-144) plus surrounding unchanged code in tds_client.rs, packet_writer.rs, exec_common.rs, stmt.rs and param_data.rs.
  • Read all 20 prior reviews, 15 review threads and the top-level comments first; nothing below restates an already-answered thread.
  • cargo nextest run -p mssql-tds --lib --no-fail-fast: 2234 run, 2227 passed, 7 failed — all 7 are the pre-existing certificate_validator / win_tls fixture failures that also fail on the merge base (missing generated tests/test_certificates/*.pem), not this PR.
  • Mutation check on the core claim. Replaced if message.message_complete() with if true in end_streamed_param_inner (i.e. settle ownership from the Result instead of the wire state) and re-ran the prepared-stream tests: 6 of 19 failed — prepared_stream_failed_final_send_retains_ownership, prepared_stream_multiple_params_assigns_id_only_after_last, prepared_stream_partial_cancel_retains_orphan_and_metadata, begin_execute_prepared_cancelled_final_send_keeps_orphan_for_retry, prepared_stream_interrupted_send_retires_without_writing_again, prepared_stream_cancellation_between_packets_keeps_connection_reusable. The new tests genuinely guard the settle boundary; the source was restored afterwards.
  • Traced every Self::deduct_timeout call site in tds_client.rs and cursor_ops.rs: all are either single-step (reconnect_elapsed) or cumulative from a started instant captured before recovery, so the truncation change cannot compound across steps.
  • Verified execute_sp_unprepare's new send_attempted() gate is evaluated after SqlRpc::serialize (message/rpc.rs:257 finalizes and flushes inside serialize), so a successful release still evicts the map entry and unprepare_orphan still clears the caller's slot.
  • Not run: live SQL Server / ODBC C++ e2e, TLS interruption reproduction, reference-driver wire comparison.

Findings

Blocking

None.

Suggestion

  1. mssql-odbc/src/api/param_data.rs:308 — the two new bail-outs that take the streamed plan return the client to the idle pool without cancelling the still-open streamed write. Details inline.

Nit

None.

CI

mssql-rs Pull request validation (Build Stage Build Linux ARM) is failing on this head (7bf8fc32, build 178265); several other legs are still pending. I could not read the ADO job log from this environment, so I cannot say whether the failure is this PR's or infrastructure. Worth confirming before merge — the description's checklist is honest that the full suite was not rerun locally for this merge commit.

Comment thread mssql-odbc/src/api/param_data.rs
@saurabh500

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unattended automated review. Findings were not checked by a human first — push back on anything that looks wrong.

Summary

Re-reviewed at 7bf8fc32 (merge base 50a315de, which is current origin/main; 10 files, +1438/−144). I previously reviewed this PR at 6297939e.

The PR's own content is byte-identical to what I reviewed at 6297939e. I diffed the two contribution diffs (cc7bb82e...6297939e vs 50a315de...7bf8fc32) against each other: both are 2165 lines, and the only content difference is 18. → 19. on the parity entry. Everything else that moved is hunk offsets and blob hashes. So the main merge in 7bf8fc32 changed nothing about the ownership redesign, and my earlier assessment of the mechanism carries over unchanged.

The merge itself is clean in both directions: main's entry 18 (Authentication=ActiveDirectoryPassword is refused) is preserved, this PR's entry is renumbered to 19, and the registry reads 1–19 with no gaps or duplicates. Since the merge base is origin/main, git diff origin/main..HEAD touching only the PR's 10 files also confirms the merge reverted nothing from main.

No findings. Both suggestions I filed at 6297939e were answered and resolved, and all 16 review threads on this PR are resolved. One required check is red — see below; I believe it is a re-trigger timing artifact rather than a code defect, but it needs a human to re-run it.

Verification performed

  • Live state. Confirmed microsoft/mssql-rs, PR #599 open, non-draft, author Saurabh Singh (@saurabh500), head 7bf8fc32, base main, labels empty — checked before reading the diff and again immediately before posting.
  • Prior discussion. Read all three endpoints (pulls/599/comments, pulls/599/reviews, issues/599/comments) plus every review thread and its resolution state before drafting. All 16 threads are resolved, including David Engel (@David-Engel)'s :2644 / :4034 / param_data.rs:308 and both of my parity-deviations.md suggestions. Nothing below re-files an answered thread, and I am not restating David Engel (@David-Engel)'s review at this same head.
  • Semantic merge check — the gap prior reviews left. main added +294 to exec_common.rs and +24 to stmt.rs since this PR's old base, and this PR edits both. A semantic conflict there would not show as a textual one. Prior reviewers ran mssql-tds only, so I built and ran the ODBC crate: cargo nextest run -p mssqlodbc --lib --no-fail-fast → 1796 run, 1796 passed, 0 failed. The merge is semantically clean for the crate main changed most.
  • PR core tests at the merged head. cargo nextest run -p mssql-tds --lib -E 'test(prepared_stream) + test(begin_execute_prepared) + test(deduct_timeout) + test(send_attempt) + test(retract_partial) + test(cancel_streamed_write) + test(streamed)' → 61 run, 61 passed.
  • The description's test count reproduces exactly. cargo nextest run --workspace --lib --no-fail-fast → 4066 run, 4059 passed, 7 failed — matching the description's "4,066 workspace library tests" precisely. All 7 failures are the documented pre-existing certificate_validator / win_tls::validate fixture failures (I did not run scripts/generate_mock_tds_server_certs.ps1), not this PR.
  • deduct_timeout truncation, re-derived. The new body uses elapsed.as_secs() with no round-up. The zero-inversion hazard is handled correctly: a computed remaining of 0 becomes Exhausted (via NonZeroU32::new returning None), while a caller-supplied Some(0) returns None/unlimited from the earlier guard — the two zeros stay distinct. I independently traced every Self::deduct_timeout call site in tds_client.rs and cursor_ops.rs: the multi-step ones (:3899, :3910, :3932, :4029) all pass the original budget with started.elapsed() from a single Instant captured at :3896, so there is no per-step re-seeding and the slack stays bounded under one second rather than compounding. exec_common.rs's rewritten doc comment now matches that behaviour.
  • Parity registry. Enumerated all top-level entries in parity-deviations.md: 1–19, no gaps, no duplicates; entry 19 is appended after main's entry 18 without disturbing it.
  • Deferral tracking. Confirmed #598, #647 and #659 all exist and are open. The param_data.rs:308 bail-outs David Engel (@David-Engel) raised were deferred to #659 — a filed issue, which satisfies the repo's "deferrals need a filed item, not a code comment" rule.
  • e2e harness (a ledger row no prior review covered). Read the +124 in execute_test.cpp. Both new tests carry the required Benefits-from-mock-tds: comments; RebindAfterCancelledStreamReleasesPriorHandle asserts payload.size() > packet_size so the "cancel must follow a packet flush" premise fails loudly rather than silently passing, and the SQLGetData loop has a forward-progress guard. The actual.append(chunk) NUL-stop is safe here because the generated payload is '!' + n % 90, so it contains no zero bytes — correct for a varchar(max) / SQL_C_CHAR case.
  • Not verified. No live SQL Server or registered driver on this host, so RebindAfterCancelledStreamReleasesPriorHandle / RebindAfterDeferredStreamReleasesPriorHandle rest on CI. No msodbcsql checkout here, so entry 19's DropPrepHandle / BuildSPPrepExec / ProcessDAEParam citations are unre-derived — ttk (@Theekshna) reported confirming them against sqlccmd.cpp independently. I did not run the C++ e2e suite, the TLS interruption reproduction, or a retail-driver wire comparison.

Findings

Blocking

None.

Suggestion

None. Both suggestions I raised at 6297939e were answered and resolved by the author, and I found nothing new in the merge delta.

Nit

None.

Withdrawing one of my own: at 6297939e I nitted that the cargo btest checklist box could be ticked because CI had gone green on that head. That no longer holds — 7bf8fc32 is a new head whose validation build is still in flight, so the description's qualifier ("library-only selection passed; full live suite not rerun locally") is the accurate statement for this revision. The description already says CI must revalidate the merge commit, which is right.

Relevant required-CI failures

coverage-report — failure (run 36188613280), completed 22:12:38Z after 1h17m. This finished after every prior review on this head was posted, so it is not reflected in them.

It is a poll timeout, not a coverage regression. The job waits for the merged artifact named exactly CoberturaCoverage (.github/workflows/pr-code-coverage.yml:81) with a 75-minute deadline (:179, :207). The log's own "Available artifacts" list shows the per-leg artifacts present at every attempt — CoberturaCoverageRust_Linux, CoberturaCoverageRust_Windows, CoberturaCoverageRust_MacOS, CoberturaCoverageOdbcE2E_Linux, CoberturaCoverageOdbcE2E_Windows, CoberturaCoveragePyCoreRust, CoberturaCoveragePython — but never the merged one, because the ADO Merge Coverage job that publishes it had not run. The job polled from 20:55:30Z and gave up at 22:12:36Z on attempt 12.

The likely cause is the /azp run at 21:57Z, which started ADO build 178279 roughly 62 minutes into this job's 75-minute window, so the artifact it was pinned to was never going to arrive in time. Suggest re-running coverage-report once 178279 finishes rather than treating it as a code signal.

Two related notes on CI state:

  • David Engel (@David-Engel)'s review at this head reported Build Stage Build Linux ARM failing on build 178265. That build is superseded — the same leg is now in_progress on 178279 (started 21:58:16Z), so the earlier red is stale and should be judged on 178279.
  • Everything else on this head is currently pass or pending: Kerberos Authentication Tests, Evaluate PR duplicate, Analyze (actions), CodeQL, check and license/cla pass; the remaining ADO legs are still running.

Because a required check is currently red, I have not applied the ready for human review label (the PR carries no labels today, and I made no label change).

@David-Engel David Engel (David-Engel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — generated by GitHub Copilot on behalf of David Engel (@David-Engel). This is not an approval and does not satisfy the human review requirement. Findings may be incomplete or wrong; push back on anything that looks off.

Summary

Re-reviewed the full diff at 7bf8fc32 against merge base 50a315de (10 files, +1438/−144). The ownership redesign holds up: end_streamed_param_inner settles the new statement identity and the piggybacked orphan from message.message_complete() rather than from the Result, so a post-send timeout or a failed response drain still transfers ownership while a cancellation before the send boundary retains it; PacketWriter's send_attempted / send_incomplete split correctly separates "never polled a send" (locally discardable), "only complete packets" (retractable with EOM | IGNORE) and "interrupted mid-packet" (retire without IGNORE, attention, or TLS shutdown); and SQLParamData's take/close/restore_plan window keeps the plan and orphan recoverable on every exit. No findings.

I read all three discussion endpoints first. Everything I independently arrived at was already filed and answered — the withdraw_sent_message sampling suggestion (tds_client.rs:2644), the duplicated live_handle lookup (tds_client.rs:4034), and the param_data.rs:308 bail-outs deferred to #659 — so nothing is re-filed here. A prior unattended review already covered this exact head; the two things this pass adds are a mutation check of the new regression tests and a refreshed CI reading.

Verification performed

  • Live state. Author Saurabh Singh (@saurabh500), PR open, non-draft, head 7bf8fc32, base main, no labels — checked before reading the diff and again immediately before posting.

  • Targeted tests (dedicated worktree, redirected CARGO_TARGET_DIR): cargo nextest run -p mssql-tds --lib -E 'test(/streamed|prepexec|deduct_timeout|send_attempt|unprepare|orphan/)' → 66 run, 66 passed.

  • Mutation — do the new tests actually guard the change? Two independent mutations, then restored:

    • Removed self.send_incomplete = true from the send future (packet_writer.rs:448), so an interrupted write would look clean. → prepared_stream_interrupted_send_retires_without_writing_again fails.
    • Disabled the message.message_complete() settle in end_streamed_param_inner (tds_client.rs:2918). → prepared_stream_completed_send_consumes_orphan_on_error, prepared_stream_multiple_params_assigns_id_only_after_last, begin_execute_prepared_piggybacks_each_orphan_with_one_second_timeout, begin_execute_prepared_cancelled_final_send_keeps_orphan_for_retry, begin_execute_prepared_release_cancellation_tracks_send_attempt and begin_execute_prepared_streams_into_sp_prepexec_when_unmaterialized all fail.

    Both boundaries are genuinely covered, not asserted around. git status --porcelain is clean apart from this review's own scratch files.

  • execute_sp_unprepare's new send_attempted() gate. Traced that SqlRpc::serialize finalizes internally (message/rpc.rs:257), so the flag is sampled after the whole message has gone out, not before finish_send — a successful unprepare still evicts both map entries, and only a pre-send failure retains them for retry. unprepare_orphan's !contains_key check derives orphan ownership from that same eviction.

  • deduct_timeout truncation. Re-derived every call site in tds_client.rs and cursor_ops.rs: each passes either a single reconnect_elapsed or cumulative started.elapsed() from an Instant taken before recovery, so the sub-second slack stays bounded under one second and cannot compound. begin_execute_prepared recomputes from the original opts.timeout, so the orphan release is not charged twice.

  • begin_execute_prepared routing. The streamed_params.is_empty() early return still delegates to execute_prepared, which assigns the id and consumes the orphan the old way; the sp_execute branch releases a cross-identity orphan through standalone sp_unprepare first and propagates its failure without opening a stream.

  • Not verified. No live SQL Server or registered driver on this host, so the C++ e2e cases rest on CI; no msodbcsql checkout, so entry 19's DropPrepHandle / BuildSPPrepExec / ProcessDAEParam citations are unre-derived here.

Findings

Blocking

None.

Suggestion

None.

Nit

None.

It looks ready for human review — but this is not an approval, and one required check is still red (below).

Relevant required-CI failures

coverage-report — failure (run 36188613280), a 75-minute poll timeout waiting for the merged CoberturaCoverage artifact rather than a coverage regression; this was already diagnosed on this head. Worth noting what has changed since that diagnosis: ADO build 178279 has now finished and every ADO leg on this head passes, including Build Stage Build Linux ARM (26m30s), which supersedes the earlier red reported against build 178265. Analyze (actions), CodeQL, check and license/cla also pass. A re-run of coverage-report now that 178279 is complete should clear the last red check.

Because a required check is red, no label was applied and no label change was made (the PR carries no labels).

@Theekshna ttk (Theekshna) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unattended hourly review sweep found no new findings. This pass was triggered by the merge-main commit (7bf8fc32) resolving the parity-deviations.md conflict David-Engel flagged; the PR's own net diff (vs current main) is unchanged from my prior two clean passes (10 files, +1438/-144).

Covered: verified the merge correctly renumbered/preserved the PR's divergence entry as #19 (diffed origin/main's copy against this head's copy of parity-deviations.md — pure addition, nothing dropped or duplicated) and that the PR description's own "entry 18/19" self-reference was updated to match; confirmed all 16 review threads (Copilot bot, David-Engel, Vahid-b, mine) are resolved, including the exact renumbering/wording concerns this merge commit addresses, and that the one deferred suggestion (param_data.rs:308, connection-stuck-after-internal-failure) was pushed to a real tracked follow-up (#659, open) rather than silently dropped; git diff origin/main:mssql-odbc/src/api/exec_common.rs and :mssql-odbc/src/handles/stmt.rs against this head show only the PR's own prior doc-comment wording, so the merge introduced no logic changes to already-reviewed code; cargo check -p mssql-tds --lib --tests --offline and cargo check -p mssqlodbc --lib --offline both compile clean post-merge (mutation-test build for cargo nextest still blocked in this sandbox by the pre-existing crc32fast AVX-512 codegen failure, unrelated to this PR). gh pr checks 599 is green on all 17 ADO/GitHub Actions stages except coverage-report, which times out after 75 minutes polling for a coverage artifact that the underlying ADO "Merge Coverage" stage itself reports as passed — an infra/polling issue, not a code regression; the PR's own "Code Coverage Report" bot comment is stale (2026-09-17, predates this session's pushes by over a week) and shouldn't be read as current-head coverage.

This is an unattended automated review; findings may be incomplete or wrong, push back on anything that looks off.

@Theekshna ttk (Theekshna) added the ready for human review Automation flag indicating an item is ready for human review. label Sep 25, 2026

@David-Engel David Engel (David-Engel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — generated by GitHub Copilot on behalf of David Engel (@David-Engel). This is not an approval and does not satisfy the human review requirement. Findings may be incomplete or wrong; push back on anything that looks off.

Summary

Re-reviewed the full diff at 7bf8fc32 against merge base 50a315de (10 files, +1438/−144). The ownership model holds up end to end: begin_execute_prepared piggybacks the orphan in sp_prepexec's by-reference @handle and defers both the new statement identity and the orphan eviction to message.message_complete() in end_streamed_param_inner, so a post-send timeout or failed response drain still transfers ownership while anything before the send boundary retains it; PacketWriter's send_attempted / send_incomplete split cleanly separates "no send polled" (locally discardable), "only complete packets" (retractable with EOM | IGNORE) and "interrupted mid-packet" (retire without IGNORE, attention, or TLS shutdown); and SQLParamData's take / close / restore_plan window keeps the plan and orphan recoverable on both success and error. No findings.

This is an unattended run at a head three prior passes already reviewed. Rather than restate their conclusions, this pass adds two mutation checks nobody had run and an independent trace of the state predicates. Nothing below re-files an answered thread — all 16 review threads are resolved, including tds_client.rs:2644, :4034, parity-deviations.md:515 / :531, stmt.rs:913 and param_data.rs:308 (deferred to #659).

Verification

  • Live state. Author Saurabh Singh (@saurabh500), PR open, non-draft, head 7bf8fc32, base main, label ready for human review present — checked before reading the diff and again immediately before posting. Read all three discussion endpoints plus every review thread and its resolution state first.
  • Predicate consumers, traced exhaustively. Every non-test use of nothing_sent() / send_incomplete() / send_attempted() / message_complete() in mssql-tds is accounted for: write_streamed_chunk:2442, cancel_streamed_write:2516, retract_partial_request:2556-2568, withdraw_sent_message:2644, end_streamed_param_inner:2918 and execute_sp_unprepare:3699. No call site was left reading the old any_packet_flushed semantics, and the cancel_streamed_write invariant the author defended on :2644 does hold: all three paths that re-park a message (Ok(Some(next)) after a successful write, and the two error paths that take the context out with mem::replace) leave send_incomplete == false or never re-park at all.
  • Mutation 1 — is the new send_attempted() gate in execute_sp_unprepare actually guarded? (Not previously checked.) Replaced if message.send_attempted() with if true, so an unsent release would evict the map entries as before this PR. → begin_execute_prepared_release_cancellation_tracks_send_attempt and prepared_batch_unsent_release_keeps_orphan_for_retry_with_one_second_timeout both fail (64/66 pass). Restored.
  • Mutation 2 — is the ODBC-side restore-on-error guarded? (Not previously checked; prior passes mutated only mssql-tds.) Made dae.restore_plan(prepared, orphaned) conditional on end_result.is_ok(), i.e. the pre-PR shape where a failed response loses the updated plan. → streamed_cancel_rebind_and_failed_response_preserve_ownership fails at execute.rs:2254. Restored.
  • Targeted suites (assigned worktree, dedicated CARGO_TARGET_DIR): cargo nextest run -p mssql-tds --lib -E 'test(/streamed|prepexec|deduct_timeout|send_attempt|unprepare|orphan/)' → 66 run, 66 passed; cargo nextest run -p mssqlodbc --lib -E 'test(/streamed_cancel_rebind|param_data|deferred/)' → 20 run, 20 passed, and streamed_cancel_rebind_and_failed_response_preserve_ownership on its own → 1 passed. git status --porcelain is clean after both mutations were reverted.
  • API-break surface. end_streamed_param now rejects a prepared stream with a usage error, so I checked every caller of begin_execute_prepared / end_streamed_param / end_execute_prepared_param outside tds_client.rs: only mssql-odbc's execute.rs and param_data.rs, and their routing key (dae.prepared.is_some()) matches the client's (ctx.prepared.is_some()) on all three open paths — immediate prepared execute, deferred open_deferred_rpc, and SQLExecDirect's ad-hoc sp_executesql. The JS, Python and CLI crates do not touch this surface.
  • deduct_timeout truncation. Independently re-derived: Some(0) still means unlimited via the early guard while a computed remaining of 0 becomes Exhausted, and the multi-step call sites (:3899, :3910, :3932, :4029) all pass the original opts.timeout with cumulative started.elapsed(), so the sub-second slack stays bounded under one second instead of compounding. The orphan release in begin_execute_prepared recomputes from the original budget rather than the already-deducted one.
  • Not verified. No live SQL Server or registered driver on this host, so the two new execute_test.cpp cases rest on CI; no msodbcsql checkout, so entry 19's DropPrepHandle / BuildSPPrepExec / ProcessDAEParam citations are unre-derived here. I did not rerun the C++ e2e suite, the TLS interruption reproduction, or a retail-driver wire comparison.

Findings

Blocking

None.

Suggestion

None.

Nit

None.

It looks ready for human review — but this is not an approval, and one check is still red (below).

CI

coverage-report — failure (run 36188613280). Confirmed from the log that this is the previously diagnosed 75-minute poll timeout waiting for the merged CoberturaCoverage artifact, not a coverage or test regression: the per-leg artifacts are listed at every attempt and the merged one never appears. Every ADO leg on this head now passes, including Build Stage Build Linux ARM, so a re-run of coverage-report should clear it. Because a check is red, no label was applied, removed, or otherwise changed by this run.

@saurabh500
Saurabh Singh (saurabh500) merged commit d001edf into main Sep 26, 2026
19 of 20 checks passed
@saurabh500
Saurabh Singh (saurabh500) deleted the dev/saurabh/fix-streamed-parameter-rebind branch September 26, 2026 00:51
ttk (Theekshna) pushed a commit that referenced this pull request Sep 26, 2026
main added its own entry 19 in #599, so the registry entry added here and the e2e comment citing it both move to 20.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ttk (Theekshna) added a commit that referenced this pull request Sep 28, 2026
* Bind CLR UDT and binary sql_variant parameters

A UDT parameter had no way in: the conversion matrix gave SQL_SS_UDT no
row, and nothing carried the three-part type name the TDS parameter
header requires. SQL_C_BINARY could not reach sql_variant either.

Character and binary buffers now reach both targets. A UDT's payload is
already its IBinarySerialize form, so it passes through untouched; only
the identity has to be supplied. That comes from the IPD's
SQL_CA_SS_UDT_* fields, or from the server's suggested_user_type_*
columns when the application describes the parameter before binding -
the same two sources msodbcsql uses (AutoFillIPD, sqlcdesc.cpp).

The identity lives on a new per-ordinal ParamSnapshot rather than on
BoundParam, which stays Copy: for_row runs once per parameter per row,
and an owned name there would re-allocate the same strings for every row
of a parameter array.

A UDT is declared to sp_executesql by its own quoted type name. The TDS
type name "udt" is not resolvable by the server, which answers "Cannot
find data type udt" (2715).

AB#48248

* Pin the mssql-python reset-then-describe UDT sequence

microsoft/mssql-python#818 resolves SQL_SS_UDT parameter identities by calling SQLFreeStmt(SQL_RESET_PARAMS) and then SQLDescribeParam before binding. That works only because the reset truncates the IPD: a record left from an earlier bind reads as explicitly bound, and refine_ipd leaves those alone. Cover the sequence end to end so the interop does not regress.

AB#48248

* Address Copilot review on the UDT parameter path

Fixes found by review, each with a regression test:

- fuzz_support.rs still called bound_param_to_rpc with the old arity, so the
  cfg(fuzzing) target no longer built.
- The cache-served SQLDescribeParam dropped the UDT identity. After one
  describe, SQL_RESET_PARAMS truncates the IPD but leaves the metadata cache,
  so the next describe rebuilt the record with no type name and the execute
  failed. That is the reset-then-describe cycle mssql-python#818 runs on every
  execution. Cached the identities beside the scalar descriptions.
- refine_ipd used udt_names.is_none(), which cannot tell an application's
  identity from one it auto-filled earlier. A re-SQLPrepare keeps IPD records,
  so a stale name survived onto a different statement. Tracked with
  udt_names_auto_filled, mirroring explicitly_bound.
- ParameterDefinition omitted the UDT names, so changing one after the first
  execute left a materialized plan in place and reused the old declaration.
  The assembly name stays excluded; it never reaches the wire.
- A catalog with no schema invented dbo, silently choosing a different type for
  a caller whose default schema is not dbo. msodbcsql quotes the absent schema
  to the empty string and emits [db]..[type]; match it.
- Three comments claimed SQLDescribeParam cannot report a UDT name, and one
  called an execute-time refusal a bind-time one.

Registered the sql_variant ceiling SQLSTATE divergence as parity deviation 18
with the measured SQL_DRIVER_VER, and corrected deviation 17, which still said
UDT parameter binding was unsupported.

Not taken: the extra payload clone in to_column_value_and_context. The binary,
varbinary and image arms beside it clone the same way, so this is a driver-wide
property of that function rather than something the UDT arm introduces; a
borrowing path belongs in its own change.

AB#48248

* Stop an auto-filled UDT name from outliving its SQL

A name refine_ipd auto-fills describes the text it was described from, not the
binding. A re-SQLPrepare keeps IPD records, so the stale name survived; a later
SQLBindParameter then marked the record explicitly bound, which stops the
self-healing describe path from ever correcting it, and execute sent the
previous statement's type identity.

DescHandle::clear_auto_filled_udt_names drops those on a successful prepare and
keeps application-supplied ones. Verified the regression: with the call
disabled, AnAutoFilledNameDoesNotSurviveARePrepare fails.

Not applied to SQLExecDirect, which the review also asked for. Measured
msodbcsql 18.6.2.1 (SQL_DRIVER_VER 18.06.0002) on SQL Server 2022, 2026-09-25:
it reuses the auto-filled name there and the execute succeeds. Matching it
rather than diverging unilaterally; SQLExecDirectReusesAnAutoFilledName pins
that on both legs.

EachUdtParameterCarriesItsOwnName now binds hierarchyid and geometry instead of
hierarchyid twice, so a driver that reused parameter 1's name for parameter 2
can no longer pass it.

AB#48248

* Reject UDT TVP columns and normalize empty UDT name parts

Two holes the new public SqlType::Udt variant opened.

write_type_info emits only the RPC parameter form of a UDT - three B_VARCHARs.
TvpValue::validate reaches it through sqltypes.rs:1197 on the serialization
path and accepted a UDT column, which would have written COLMETADATA without
the MAX_BYTE_SIZE and assembly-qualified name that form requires, i.e.
malformed TDS. TvpColumnDef::validate now rejects it, alongside the existing
text/ntext restriction.

write_b_varchar encodes an empty name part and an absent one identically, but
format_udt_sql_name matched on Some(_), so a UdtTypeName built with
Some(String::new()) - which the public constructor and public fields both
allow - declared [] against a header that named nothing. Empty parts are now
read as absent, so the declaration and the wire identity agree.

AB#48248

* Fix the mssql-py-core build and address review follow-ups

mssql-py-core is excluded from the workspace, so `cargo clippy --workspace`
never saw that SqlType::Udt made its sql_type_metadata match non-exhaustive.
CI runs scripts/bclippy.ps1, which lints the workspace and then mssql-py-core,
which is why all four legs failed while the workspace was green. Added the arm,
and validated with the CI script rather than a bare workspace clippy.

The UDT name is part of the declaration, so it belongs in the reuse identity.
Unlike the TVP arm beside it, absent parts stay absent: `Point` and `dbo.Point`
are different declarations and must not share a prepared statement.

Also pinned the accept side of the binary length ceiling, and recorded in
`unrelated_c_types_do_not_reach_decimal_xml_or_variant` that the sql_variant
row is narrower than msodbcsql's fValidConversion by deferral, not intent,
tracked in AB#48453.

AB#48248

* Scope UDT name provenance and drop per-row identity clones

Writing SQL_CA_SS_UDT_ASSEMBLY_TYPE_NAME marked the whole bundle as
application-supplied, even though that field never reaches the wire. After an
auto-filled name, setting only the assembly field stopped SQLPrepare clearing
the stale wire identity and stopped refine_ipd replacing it, so a later
statement could execute against the previous statement's type. Only the three
wire-relevant parts now claim the identity; the clear and the describe refresh
both preserve an application's assembly name, which no describe supplies.

clear_auto_filled_udt_names ignored a poisoned descriptor mutex and let
SQLPrepare report success with the stale identity still in place. It now
returns SQL_ERROR and SQLPrepare propagates it, per the repository's
mutex-poison rule.

Two identity clones removed. build_positional_params cloned the whole tail into
a temporary Vec once per stored-procedure call; it borrows the slice instead.
The parameter-array validation loop cloned each ParamSnapshot once per row per
parameter, reallocating the boxed name strings - exactly the cost the snapshot
split exists to avoid - and now copies only the Copy half.

AUdtParameterWithoutATypeNameFails asserted only SQL_ERROR, so the diagnostic
could regress to any SQLSTATE unnoticed. It now asserts HY000 as the
application observes it, and that passes on the msodbcsql leg too.

AB#48248

* Keep an assembly-named UDT record refreshable by describe

`clear_auto_filled_udt_names` dropped the auto-filled flag for every
record it visited, but only cleared the record itself when no assembly
name was set. A record kept for its echo-only assembly name was left
`Some(..)` and no longer auto-filled - exactly the state `refine_ipd`'s
gate refuses to touch. The server's `suggested_user_type_*` identity was
never re-applied, so the execute failed with ERR_MISSING_UDT_TYPE_NAME
through a plain describe / set-assembly-name / re-prepare / describe
sequence.

Keep the flag set on the branch that retains the record, and stop
`refine_ipd` from discarding a record whose only remaining content is an
assembly name no describe can restore.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Let the server name a UDT an assembly-only record never claimed

`refine_ipd` gated on `udt_names.is_some()`, which treats any record with
a `UdtNames` as an application claim. Writing only the echo-only
`SQL_CA_SS_UDT_ASSEMBLY_TYPE_NAME` on a never-described record produces
`Some(..)` with an empty `type_name` and the provenance bit still false,
so a later describe skipped it and the execute failed with
ERR_MISSING_UDT_TYPE_NAME - route (b) silently broken for an application
that knows its assembly but relies on the server for the type name.

Gate on the record's state instead: an empty `type_name` carries no wire
identity however it was reached, whether by that setter or by
`clear_auto_filled_udt_names` keeping a record for its assembly name
alone. The two histories are now indistinguishable, as the descriptor
contract says they should be, and an application-written `type_name`
still outranks the server.

Also post a diagnostic on `SQLPrepareW`'s poisoned-IPD exit, which
otherwise returned SQL_ERROR with nothing for SQLGetDiagRec to report,
and retarget a `mssql-py-core` TODO this change satisfies at the
remaining Python-side blocker.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Pin UDT PLP framing with byte-level serialization tests

The e2e comment claimed `datatypes::sql_udt::tests` pinned the TYPE_INFO
name block *and* the PLP body framing, but those tests only exercise
`write_udt_type_name` - nothing serialized a `SqlType::Udt`, so the type
byte, the chunk framing and the terminator were unpinned.

Add two tests over the real `serialize` path: one for a populated UDT
covering the type byte, the three B_VARCHARs, the PLP length marker,
chunk and terminator, and one for a NULL UDT, which must still name its
type and then declare the PLP null length. Correct the e2e comment to
cite what each test actually covers.

Recorded while pinning it: this driver declares the PLP body
unknown-length and relies on chunk framing, where msodbcsql's
WriteUDTHeader writes the actual byte count when it knows it. Both are
valid PLP; the note is in the test so the next reader does not take the
difference for a defect.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Record that msodbcsql hex-decodes a character buffer bound to a UDT

The character arm passes the buffer's bytes through verbatim, and the comment beside it read as though that were parity. It is not: rgbTRANSTYPE* maps SQL_UDT_MAPPED to a SQL_C_BINARY transfer type, so ConvertLongData takes its conversion path rather than its pass-through one and hex-decodes, two characters per byte. Same binding, different wire payload.

Source reading only, so this is a cited divergence note rather than a registry entry - the measurement that would settle it is named in the comment, along with the SQL_NTS caveat that makes a character UDT binding need an explicit length.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Refuse character buffers against a UDT until the decode is measured

The matrix admitted SQL_C_CHAR and SQL_C_WCHAR for SQL_SS_UDT and the conversion sent the buffer verbatim, but msodbcsql hex-decodes two characters to the byte. Advertising the rows while diverging on the payload ships a wrong-bytes path that no e2e case covers, since every UDT test binds SQL_C_BINARY.

Drop the two rows instead. The matrix documents itself as a progress list where a missing entry means unbuilt (HYC00), which is the gap category in parity-deviations.md - a code comment plus a work item, not a registry entry. Nothing regresses: this PR is what introduces the capability. The citation chain and the measurement that would close it are recorded at both sites.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Binary-search UDT identities instead of scanning per marker

refine_ipd replays the cached identity list on every cached SQLDescribeParam answer, and searched it linearly for each marker. A describe-all-before-bind pass over N UDT markers - the sequence microsoft/mssql-python#818 runs on every execution - therefore cost O(N^3) ordinal comparisons: 62,625,000 at 500 placeholders.

Sort the sparse list once where it is cached, then binary-search it. sp_describe_undeclared_parameters does not promise ordinal order, so the sort cannot be skipped by assuming push order; a debug_assert guards the invariant for future callers. Operation-count reasoning, not a latency measurement - the per-call name clones the review also noted are unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Share cached UDT identities and skip unchanged descriptor replays

The cache-served SQLDescribeParam path deep-cloned every cached identity out from under the STMT lock, then refine_ipd rebuilt an owned copy for each marker. A describe-all pass over N UDT markers - what microsoft/mssql-python#818 runs per execution - therefore made O(N^2) string allocations.

Hold the cached identities in Arc so releasing the STMT lock costs a refcount bump instead of N deep copies, and skip the descriptor write entirely when a replay finds the three wire parts already in place. Comparing them allocates nothing, so the steady state is now allocation-free; the descriptor still owns its copy and the lock ordering is unchanged.

Also narrow parity deviation 19: a zero-filled overflow past the 8000-byte ceiling is trimmed and sent by both drivers (trim_zero_overflow is CheckTrailingZeros, sqlccnvt.cpp:8690), so only a non-zero overflow diverges. The recorded measurement used 0xAB bytes and is labelled as such.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Stop an assembly-only UDT record from forcing a re-prepare

`parameter_definition()` projected `Some(("", "", ""))` for a record
carrying nothing but the echo-only assembly name, where a record with no
identity projects `None`. Writing `SQL_CA_SS_UDT_ASSEMBLY_TYPE_NAME` on
an otherwise-unnamed record therefore orphaned a materialized prepared
handle and forced a re-prepare for a field the declaration never
mentions - contradicting the invariant the adjacent test already stated.

Project the wire parts only when at least one is set, and cover the
assembly-only case. Mutation-checked both directions: over-filtering
breaks the rename-invalidates leg.

Also correct parity deviation 19. Narrowing it last commit replaced an
overstatement with an unmeasured parity claim - that both drivers trim a
zero-filled overflow - which is a source reading only. It is now
labelled as such, with the both-leg measurement that would close it
named, per the evidence rule in mssql-odbc.instructions.md.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Renumber the sql_variant deviation to 20 after rebase

main added its own entry 19 in #599, so the registry entry added here and the e2e comment citing it both move to 20.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Pin the zero-overflow variant case the registry cites

Deviation 20 cited a_binary_variant_payload_past_the_byte_ceiling_is_truncation for the zero-only overflow half, but that test only exercised 0xFF payloads at 8001 and 8000 bytes - the doc comment asserted the zero behaviour without covering it. an_all_zero_binary_overflow_is_trimmed_silently does cover zero padding, but for plain varbinary/binary/image targets rather than a variant.

Add the variant-specific leg so the cited test actually pins what the registry claims.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Scope the UDT projection to records that declare a UDT

The SQL_CA_SS_UDT_* fields are writable on any IPD record - classify_field gates on descriptor kind, not concise type - so writing a UDT name on an int marker changed ParameterDefinition and orphaned a materialized prepared handle, even though an int declaration never references that name.

Filter the projection on SQL_SS_UDT. Switching the record to a UDT later still invalidates through sql_type and brings the name with it, which the new test pins alongside the non-UDT case.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Align the UDT projection gate with refine_ipd's claimed test

The projection admitted a record with any of the three wire parts set, so a catalog-only or schema-only IPD record still orphaned a materialized prepared handle - the same spurious re-prepare the assembly-only fix removed, reachable through a different field. Such a record declares nothing and udt_type_name refuses to execute it at all.

Gate on a non-empty type name, which is exactly refine_ipd's claimed test, so the two definitions of having a wire identity are identical. Catalog and schema still travel with a named UDT, so changing either continues to invalidate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Take descriptor test handles from the TestHandles fixture

mssql-odbc.instructions.md:429-439 requires unit-test ODBC handles to come from crate::test_support::TestHandles. The two clear_auto_filled_udt_names tests constructed a standalone DescHandle instead; in production that method is only ever called on a statement's IPD, so the fixture is also the more faithful shape.

The two pre-existing DescHandle::new calls in this module stay: they test the constructor's own defaults, so they have to call it directly.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Propagate refine_ipd failures out of SQLDescribeParam

refine_ipd was best-effort, which was defensible while it only refined type/size/scale the call already answered from its in-memory descriptions. This PR made it load-bearing: on the describe-before-bind route its IPD write is the only source of SQL_CA_SS_UDT_TYPE_NAME. A poisoned IPD lock or a failed prepared-definition invalidation still returned SQL_SUCCESS, so the application saw the failure later as a misleading missing-name error at execute, or reused a stale declaration.

Return SqlReturn and propagate from both the cached and fresh paths, posting HY000 on the statement. Matches clear_auto_filled_udt_names, which already propagates its own poisoned-mutex failure, and the crate rule that a poisoned mutex returns SQL_ERROR.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Correct the assembly-name rationale and the refine_ipd doc block

sp_describe_undeclared_parameters does return suggested_assembly_qualified_type_name (0-based column 11), so three comments claiming the server never supplies one were wrong as written. msodbcsql even reads it (sqlcdesc.cpp:9155-9162), but only into a scratch FRS_Format field - unlike the catalog/schema/type parts it never reaches the IPD name pool, so the descriptor field reads back empty there too. The merge rule is unchanged; only its justification was wrong. Say instead that this driver chooses not to read the column, and why.

Also repair the refine_ipd doc block: propagating failures left the old best-effort paragraph in place, directly contradicting the new one, and duplicated the locking-order line.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Validate the whole UDT name before writing any of it

UdtTypeName::validate only checked that the type name was non-empty; the 255-unit B_VARCHAR bound was enforced inside write_b_varchar, per part, as each part was reached. The parts are written in sequence and PacketWriter sends on overflow (handle_overflow_if_needed -> populate_header_and_send), so a 255-unit catalog can fill and flush a small packet before an oversized schema or type name returns UsageError. Invalid local input then became a half-sent RPC needing cancel-and-drain instead of a clean local failure.

Check all three parts up front. The per-part check in write_b_varchar stays as the encoding-level guard; this one keeps the rejection local. Pinned by a test asserting nothing reaches the mock network, mutation-checked by disabling the bound.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Validate UDT identities before the RPC writes anything

The preflight added in 96955fb8 sat in write_udt_type_name, which is already too late: RpcParameter::serialize writes the parameter name and status flags first, write_type_info writes the UDT type byte, and in a multi-parameter RPC an earlier parameter can have flushed whole packets. An invalid local identity still became a half-sent request needing cancel-and-drain.

Hoist the check to the message: SqlRpc::validate_parameters runs before serialize_prefix and serialize_batch_command write a single byte.

Also fix both new tests. They passed MockNetworkWriter::new(4096) with Some(512) as PacketWriter::new's third argument, which is the timeout - packet size comes from the writer, so nothing ever overflowed and the no-bytes-sent assertions were vacuous. Found by mutation: removing the preflight left them green. With the packet size set on the mock, both now fail without their fix.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Route every fallible SqlType metadata check through the preflight

The preflight matched only SqlType::Udt, so a sql_variant carrying a UDT slipped past it. validate_variant_inner then rejected that pairing inside write_type_info - after serialize had written the parameter name and status flags, and after an earlier parameter could have flushed whole packets. Same half-sent RPC, reached through a different type.

Move the rule onto SqlType::validate_for_send, which now owns both fallible checks, and have RpcParameter delegate to it. The duplicates inside write_type_info stay: that function has to remain correct for callers reaching it another way. Extended the RPC test to cover a later Variant(Udt(..)) parameter; mutation-checked by dropping the variant arm.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Complete the send preflight with the parameter-name bound

The preflight covered SqlType metadata but not the parameter-name length check in serialize, so an overlong name on a later named parameter could still fail after an earlier one had flushed packets. An incomplete guarantee is worse than none, since readers rely on it.

Extract validate_name_length and share it with the write path so the two cannot drift, and split validate_named_before_send from validate_before_send: serialize only writes and length-checks the name on the named path, so validating names on positional parameters would newly reject an unused overlong name. Mutation-checked by routing named parameters through the value-only preflight.

Recorded while there: the bound is measured in UTF-8 bytes while the payload is written as UTF-16, which agree only for the ASCII names this driver generates. Pre-existing, left as-is and documented rather than changed silently.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Extend the send preflight to vectors and scope its doc honestly

The preflight's _ => Ok(()) arm skipped SqlType::Vector, whose three dimension/base-type checks then failed inside write_type_info after earlier parameters had flushed - the exact failure the preflight exists to remove, and reachable from mssql-py-core's InputSqlType::Vector. Factor them into validate_vector, shared with the write path, and cover a later mismatched vector in the regression.

Also correct the doc claim, which said every fallible check in write_type_info belonged here. TVP validation lives across serialize_table, write_tvp_type_name and write_tvp_column_metadata and is a larger hoist, so it is now named as a known exception rather than implied to be covered. Likewise scope validate_parameters: it guards one RPC message, while a batched prepared execution validates per command - a pre-existing property it shares with reject_data_at_exec and the ForceColumnEncryption check.

Restore validate_variant_inner's doc comment, which an earlier commit orphaned by inserting validate_for_send between it and its function.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Correct a stale count in the preflight test doc

The test's doc said it covered both fallible metadata checks, but 3500cc73 added a third case - a vector whose declaration disagrees with its value - without updating the sentence above the array.

Third instance of this shape in the series, after the describe_param.rs block and the orphaned validate_variant_inner comment: appending beside an existing comment without re-reading it. A sweep of all 30 changed files for count claims found no others.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Share the UDT identity through parameter_definition instead of copying it

The Arc cache removed the per-replay clone of the cached identity list, but parameter_definition still deep-copied catalog, schema and type into an owned tuple - and refine_ipd takes that snapshot twice per record, before and after refinement. A cache-served describe over N UDT markers therefore still made ~2N^2 string allocations, which is the cost the Arc was introduced to remove.

Carry Option<Arc<UdtNames>> through DescRecord, BoundParam and DaeParam so the snapshot is a refcount bump; writers use Arc::make_mut, so a record whose identity is also held by a live snapshot copies once on write rather than on every read.

PartialEq is now hand-written because the shared UdtNames carries assembly_type_name, which must not affect equality: it never reaches the declaration, so a derived impl would orphan a prepared handle on an echo-only write. Mutation-checked - comparing the whole struct fails the_udt_name_is_part_of_the_prepared_parameter_definition on exactly that assertion.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Track UDT name provenance per field, not per record

The three wire parts of a UDT identity are independently writable through
SQLSetDescField, so a single `udt_names_auto_filled` bit could not say which
of them the application owns. Two orders broke:

1. A describe fills the identity, the application overrides only the schema.
   The one flag marked the whole identity application-owned, so the
   prepare-time clear preserved the server's stale type name and the next
   describe refused to refresh it - the new statement executed against the
   previous one's type.
2. The application writes only the catalog. `refine_ipd`'s gate keyed off
   `type_name` alone, so the describe overwrote the catalog it had set.

Replace the bit with `UdtNameClaims`, one bool per wire part. `set_udt_name`
claims only the part it writes; `clear_auto_filled_udt_names` clears only
unclaimed parts; `refine_ipd` merges the server's values into unclaimed parts
and leaves claimed ones alone. The assembly name still claims nothing - it
never reaches the wire and no describe supplies it.

`refine_ipd` keeps the steady-state early-out and still shares the describe's
`Arc` when the application has claimed nothing, so the per-marker copy the
previous change removed does not come back.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Keep the UDT identity merge off the non-UDT path

`refine_ipd`'s per-part merge ran for every record, so an ordinary scalar
marker entered it with nothing stored and nothing described, allocated an
`Arc<UdtNames>` through `get_or_insert_with`, copied empty into empty, and had
the cleanup below free it again. `refine_ipd` is replayed for every
cache-served describe answer, so that is ~N^2 allocations across a
describe-all pass over N markers - on statements containing no UDT at all,
the same order of cost 53c6de48 removed one level down.

Guard the merge on `described.is_some() || record.udt_names.is_some()` and
move it into `merge_udt_identity`. Skipping is behaviour-preserving: both
merge branches are no-ops when the record and the describe agree there is no
identity, and a claim cannot exist without a record to live on - `set_udt_name`
creates the record before claiming, and both clear paths drop the record only
when nothing is claimed. A `debug_assert!` pins that invariant.

Also filter the IPD identity out of `ParamSnapshot` for non-`SQL_SS_UDT`
parameters. `classify_field` gates the `SQL_CA_SS_UDT_*` fields on
`DescKind::ImpParam` alone, so an application can leave an identity on a record
it later rebinds to a scalar - which contradicted the field's documented
contract and cost a refcount bump per execute. This is the same filter
e433dc25 applied to `parameter_definition`.

Correct one stale cross-reference in `parameter_definition`: it cited
`refine_ipd`'s `claimed` gate, which 32a2c35f replaced with per-field claims.
The test it names is `udt_type_name`'s.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Cover the output direction of a UDT parameter

Binding SQL_SS_UDT with SQL_PARAM_OUTPUT became reachable for the first time
in this PR - before it, SQLBindParameter rejected the type with HYC00 for want
of a conversion-matrix row, so no direction was bindable. `sql_bind_parameter_safe`
accepts every non-streamed direction and `is_output_only` routes straight to
`typed_null`'s SQL_SS_UDT arm, but nothing exercised the leg after that:
`write_back_output_params` had never run against a returned UDT, and all 17
e2e UDT cases bind SQL_PARAM_INPUT.

Add both legs. The unit test pins the write-back: a returned UDT decodes to
`ColumnValues::Bytes` (decoder.rs, the TdsDataType::Udt arm), so delivery goes
through the binary path. The e2e drives a procedure with a `hierarchyid OUTPUT`
parameter and compares the bytes against the same value fetched independently.

Accepting the binding is parity, not a divergence: msodbcsql applies no
direction gate to SQL_UDT_MAPPED - `CheckParamBindInfo` (sqlccmd.cpp:9977)
checks only that a type name is present - so refusing the non-input directions
would have been the departure needing a record.

Correct the `SqlType::Udt` doc, which still called the type "input-only". A
returned UDT never reaches that type; it arrives as plain bytes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Write the missing-UDT-name diagnostic in this driver's own words

The text of ERR_MISSING_UDT_TYPE_NAME was copied from a comment in the
reference driver's source rather than written here, and the same borrowed
wording was repeated in the `udt_type_name` doc comment and the e2e test
comment. That put proprietary source text into a public repository, and it
also misrepresented what it was: a maintainer's comment, not the message that
error ID actually renders.

Reword the constant independently and reduce all three citations to path and
line, which is how the rest of this PR cites the reference. The parity claim
that survives is the SQLSTATE - HY000, asserted on both legs by
`AUdtParameterWithoutATypeNameFails` - and the doc now says that is the claim.
No test asserted the literal text, so the behavior is unchanged.

One more instance of the same class, not flagged but the same defect: the
character-to-UDT note in `bound_param_to_value_with_outcome` quoted a comment
from `sqlccnvt.cpp`. Replaced with a description of the behavior, keeping the
path and line.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Correct two contracts this PR's edits outgrew

Neither is a behavior change; both are statements that stopped being true when
the code beneath them moved.

`sql_prepare_w_safe` said it holds the DBC and STMT locks "for the whole
body". It now releases both before calling `clear_auto_filled_udt_names`,
because the crate forbids holding a STMT lock while taking a DESC lock. The
invariant the sentence protected still holds - the state check and the store
are under one continuous STMT lock - but the release, and the rule that forces
it, went unmentioned at the point where it bites.

`udt_name_fields_are_writable_on_ipd_only` said every kind other than the IPD
"has no use for" the UDT identity. `SQLColAttribute` already answers all four
parts for a result column out of COLMETADATA's UDT_INFO
(`col_attribute.rs:286-309`), so the IRD does have a use for them and the test
was reading as if it had settled the `SQLGetDescField` route. Reworded to
scope the assertion to writes and leave the IRD read side visibly open.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Say what the IRD assertion decides, and record the deviation it pins

The comment added in 6a418798 said the IRD read route was "deliberately left
open rather than decided here", but the assertion ten lines below asserts
`classify_field(ImpRow, SQL_CA_SS_UDT_*)` is `None`. That gate is the whole
read path - `get_desc_field.rs:140` returns HY091 the moment it yields `None` -
so the route is closed, and the test is what holds it closed. Both readings of
"left open" were wrong.

Reword to say the route is closed and that this assertion pins it.

The parity half, which the previous comment left as an open question, is now
answered: msodbcsql serves all four through that route. SQLGetDescFieldW's
default arm dispatches SQL_HANDLE_IRD to GetIRDField with fSQLGETDESCFIELD
(`sqlcdesc.cpp:2390`), and GetIRDField answers the UDT name parts from the
column's name pool (`sqlcdesc.cpp:6844-6880`) with no gate separating that
caller from SQLColAttribute. So refusing the read is a divergence, recorded as
parity deviation 21 with its evidence limit: source reading only, no
comparison run.

The gap predates this work - these fields were absent from `classify_field`
for every kind before it - so no behavior changes here. Wiring up the IRD read
is result-column work.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Record the IRD getter divergence as a gap, not a deviation

Entry 21, added in ceb4e355, described itself as "a gap rather than a choice"
and then registered it as a deliberate deviation. The registry's own boundary
rules that out: entries there are decisions - "we know what msodbcsql does, we
could match it, and we chose not to" - and a gap is "a code comment plus a
work item, not an entry here". `.github/instructions/mssql-odbc.instructions.md:79-81`
says the same.

Remove the entry and move its full content into the code comment on
`udt_name_fields_are_writable_on_ipd_only`, which is the prescribed half I can
deliver. Nothing is lost: the comment keeps what msodbcsql does
(`sqlcdesc.cpp:2390` -> `GetIRDField`, answering at `:6844-6880`), what this
driver does instead (HY091 via `classify_field`), that the evidence is source
reading rather than a measured run, and that the write-side citation is not
evidence for the read side.

The other half - a tracked work item - still needs filing; the comment says so
rather than implying the gap is recorded somewhere it is not.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Fuzz the UDT arm, and fold an empty name part to absent in the py cache key

The conversion-matrix row this PR adds makes SQL_SS_UDT satisfy
`is_supported_conversion`, so `PARAM_SQL_TYPES` now draws it - but
`fuzz_bound_param` passed `udt_names: None`, and the conversion consults the
identity before it reads the value buffer. Every UDT draw returned
`MissingUdtTypeName` immediately, leaving the newly reachable
`SqlType::Udt` construction with no coverage at all.

Draw the three name parts from the same cursor. A literal would reach the
construction but not the code that parses hostile text, so the lengths are
drawn past the 255-UTF-16-unit bound to reach `UdtTypeName::validate`'s
rejection, and can be zero to reach the absent-part branches of
`format_udt_sql_name`, whose `]` doubling now also sees fuzzer bytes. Lossy
UTF-8 conversion rather than a validity check, so no draw is wasted.

Separately, `sql_type_metadata`'s Udt arm keyed `Some("")` and `None` apart
while `format_udt_sql_name` renders them identically - it filters empties
before choosing its branch. Two values with the same `sp_executesql` text
would have missed the prepared-statement cache. Unreachable today because
`InputSqlType::Udt` is still refused, so this is a latent bug fixed before it
can fire rather than an observable one.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Reach the UDT declaration formatter the fuzz harness claimed to cover

The oversized type-name draw added in 99cdc749 was shaped by a belief I did
not check: that `bound_param_to_rpc` evaluates `UdtTypeName::validate`'s
255-UTF-16-unit bound and `format_udt_sql_name`'s `]` doubling. Neither is on
that path. `udt_type_name` only checks the type name is non-empty; `validate`
runs from `write_udt_type_name` and the RPC send preflight, and the formatter
runs from `sql_declaration`. A probe binding 300-unit parts with a raw `]`
returns Ok and carries the `]` through unescaped.

Two consequences, both fixed here.

The draw cost entropy for nothing. It consumed up to 329 bytes ahead of
`cur.rest()` on *every* draw, including every non-UDT `sql_type`, taking that
budget straight out of the value buffer. The parts are now drawn only when
`sql_type` is SQL_SS_UDT, which is stricter than restoring the old 5 bytes:
non-UDT draws now pay nothing at all.

And the coverage was worth having, so take it rather than dropping the claim.
Rendering the declaration from the converted value reaches
`format_udt_sql_name`, whose `]` doubling is the injection-relevant escaping
and was fuzzed by nothing - mssql-tds's `fuzz_api_inputs` calls `get_sql_name`
but never generates `SqlType::Udt`. Both seams this needs
(`RpcParameter::get_sql_name`, `get_value`) are already `#[cfg(fuzzing)]`
exports.

`UdtTypeName::validate` stays out of reach: it is `pub(crate)` to mssql-tds
and runs at serialization time, so fuzzing it belongs in that crate's targets.
The comments now say that instead of claiming it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Track the character-to-UDT decode in its own work item

The four citations for that deferral pointed at AB#48248, this PR's own task,
which closes when the PR merges and would leave the deferral untracked. They
now point at AB#48815, which carries the msodbcsql source chain, the
measurement that would close it, and the note that AB#48249 needs this on its
explicit unimplemented-feature list so the rows keep HYC00 rather than being
flipped to 07006 by matrix absence.

Comment-only; no behaviour change.

AB#48248

* Bound a UDT parameter ColumnSize, and drop one more borrowed comment

`parameter_column_size_is_valid` had no SQL_SS_UDT arm, so it fell through to
`_ => return true` and accepted any size. msodbcsql rejects `> SQL_PREC_UDT`
(8000) with IDS_S1_104 in `CheckSqlPrecScale`'s `case SQL_UDT_MAPPED`
(sqlcdesc.cpp:11790), which SQLBindParameter reaches at sqlcdesc.cpp:3038;
`FixupColumnSizeDecimalDigits` has no UDT arm, so the application's value
arrives unmodified and that check is genuinely reachable. A probe confirmed
this driver accepted 8001, 65535 and 100000.

This PR is what made the arm reachable - before it the conversion matrix
refused the bind, so the table was never consulted for a UDT. Zero stays legal
because it is the `max` spelling, matching the reference's one-sided test.
Mutation-verified: deleting the arm fails the assertion.

Also removes a comment quoted verbatim from the reference driver's source in
`conversion_matrix.rs`. Three siblings went in eebd82d8; the sweep then used a
single-line pattern and this one spans three lines. The 2-chars-to-1-byte
behaviour it described is restated in this driver's own words with the citation
kept.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Add mssql-tds integration tests for the UDT parameter path

The new UDT code in mssql-tds had unit tests only. The ODBC e2e suite does
exercise it against a live server on both driver legs, but only through what
ODBC exposes: it cannot reach a caller that builds an RpcParameter itself -
which is how the JS and Python bindings use this crate - and it cannot reach
the send preflight's rejection arms at all, because SQLBindParameter refuses
those inputs before mssql-tds is called.

Five cases in tests/test_udt_parameters.rs, using hierarchyid so no
CREATE ASSEMBLY is needed: the payload round trip, a NULL UDT still carrying
its name, the two-part schema-qualified name, and both preflight rejections -
an empty type name and a 256-unit name part. The rejection cases assert the
connection still serves the next query, which is the half a unit test on
`validate` cannot show: the preflight exists so locally-invalid input fails
before any bytes reach the wire, rather than half-sending an RPC that needs a
cancel-and-drain.

Also corrects two comments on `TvpTableData::validate`. It has no caller
outside this module's tests - `serialize_table` validates the type name and
then writes the rows - so the UDT-column arm added here states what a TVP
should reject rather than preventing it, and the comment said otherwise. That
predates this branch: validate was already test-only at the merge base.
Wiring it in means a second pass over every row ahead of the writing pass, so
it is recorded as a tracked gap rather than changed in passing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Measure the UDT ColumnSize ceiling on both legs

The ceiling added in ae5e308f was a new client-side rejection with no test
driving it through SQLBindParameter, and every UDT case in the e2e suite binds
ColumnSize 0, so neither leg exercised the boundary. That matters because
mssql-python's documented setinputsizes([(SQL_SS_UDT, 8000, 0)]) sits exactly
on the inclusive edge.

Adds a both-leg e2e asserting HY104 at 8001 and a successful bind at 8000 and
at 0, so the parity claim is measured rather than source-read, plus a
bind-level unit test covering the same three points through SQLBindParameter
rather than through the predicate alone.

Both use literals instead of SQL_PREC_UDT. Written first in terms of the
constant, the mutation that motivated the test - shrinking SQL_PREC_UDT by one
- left both tests green, because the assertions moved with the constant they
were meant to pin. With literals the same mutation fails both.

Also names the zero after the public contract: it is SQL_SS_LENGTH_UNLIMITED
(msodbcsql.h:564), the only way to bind a UDT larger than the ceiling, not
merely a permissive zero by analogy with varchar(max).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Retract a wrong claim about TVP validation not being on the send path

8d9b12da relabelled `TvpTableData::validate` as dead code and rewrote two
comments around it to say a UDT column in a TVP still writes malformed bytes.
That is wrong. `serialize_table` calls `data.validate()?` in its `Some(data)`
arm (sqltypes.rs:1245), before any column metadata or row bytes are written,
so the guard does prevent the malformed output it describes.

Restore the original meaning and record the boundary that is actually true:
the check runs after the TVP type byte and three-part name, so a rejection
keeps the metadata and rows off the wire but leaves that prefix in the writer -
already sent if it overflowed a packet. That is the partial-request shape
`validate_parameters` exists to avoid for scalar parameters, and it is worth
stating rather than implying a full preflight.

Also corrects the new integration test's own doc, which said the JS and Python
bindings reach this path. They do not yet: `mssql-py-core` refuses a UDT
parameter (types.rs:441) and `mssql-js` has no UDT path. The justification that
survives is narrower and checked - `SqlType`, `UdtTypeName` and `RpcParameter`
are public mssql-tds surface, and the preflight's rejection arms are
unreachable from the ODBC suite because SQLBindParameter refuses an empty or
overlong type name before mssql-tds is called.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Bound a UDT payload by its declared ColumnSize

The binary-to-UDT arm forwarded the whole buffer regardless of the declaration,
so a bounded binding such as ColumnSize 3 accepted and sent a four-byte
payload. msodbcsql raises 22001 when cbData > min(cbColDef, SQL_PREC_UDT)
(sqlcfunc.cpp:2681-2696), and skips the check entirely when ColumnSize is
SQL_SS_LENGTH_UNLIMITED - which is what fIsVarMax means for a UDT
(sqlcfunc.cpp:2577-2584).

Reject rather than trim, which is the part worth reading the neighbouring arm
for: SQL_BINARY/VARBINARY/LONGVARBINARY call CheckTrailingZeros and trim a
zero-only overflow (sqlcfunc.cpp:2606-2616), the behaviour trim_zero_overflow
mirrors here. The UDT arm has no such call, so reusing that helper would have
accepted a zero-padded payload the reference refuses. Both halves are asserted.

Unit coverage over the declaration edge, one byte past it, the zero-padded
overflow and the unlimited case; mutation-verified by deleting the guard. Plus
a both-leg e2e asserting 22001 for a payload past a bounded ColumnSize and a
successful round trip for the same payload at SQL_SS_LENGTH_UNLIMITED, so the
reference side stays measured rather than source-read.

This is the neighbour of the ceiling added in f507f273: that bounded the
declaration, this bounds the payload against it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Measure a buffered UDT as its chunks arrive

The ColumnSize bound added in fa598754 caught a data-at-execution UDT only at
close. `dae_length_limit` returned None for SQL_C_BINARY -> SQL_SS_UDT because
`same_unit` recognised SqlFamily::Binary alone, so nothing bounded the value
per chunk: a bounded UDT could accumulate arbitrarily more than its
at-most-8000-byte declaration across SQLPutData calls, and the 22001 arrived on
SQLParamData rather than on the call that overflowed. The rest of the DAE path
deliberately reports on the overflowing call, mirroring msodbcsql's SQLPutData
arms, so this was inconsistent with its own neighbours as well as the
reference.

Treat a UDT as byte-measurable: a UDT payload is bytes passed through
untouched, which is the same correspondence SqlFamily::Binary has. The bound
follows the materialized arm - min(ColumnSize, SQL_PREC_UDT), unbounded at
SQL_SS_LENGTH_UNLIMITED.

Overflow policy is now explicit rather than implied by pad_unit. Every
character and binary declaration keeps trimming an all-padding overflow; a UDT
does not, because the reference's SQL_UDT_MAPPED arm has no CheckTrailingZeros
call (sqlcfunc.cpp:2681-2696 against the varbinary arm at :2606-2616). Reusing
the binary policy would have accepted a zero-padded payload msodbcsql refuses.

Regression covers the accumulated-total bound, the first overflowing chunk, the
zero-padded overflow and the unlimited case. Mutation-verified twice: dropping
the Udt arm from same_unit, and letting a UDT trim padding, each fail it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Use a multi-byte payload in the bounded-UDT e2e

AUdtPayloadPastABoundedColumnSizeIsRefused parsed a single-level hierarchyid,
which serializes to exactly one byte, so its guard assertion failed and took
the Windows and Linux ARM legs red before either half of the test ran.

The guard was right and the payload was wrong. With a one-byte payload
`payload.size() - 1` is zero, and zero is SQL_SS_LENGTH_UNLIMITED - so even
without the assertion the "bounded" case would have bound an unlimited
parameter and proved nothing. Every other hierarchyid in this suite is
single-level, so there was no precedent to borrow the size from.

Use a ten-level path, keep the assertion as the guard the reviewer asked for,
and give it a message so a future mismatch reports what is wrong rather than
`1 vs 1`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* State the evidence level on the two new UDT ColumnSize parity claims

Both new e2e cases asserted a reference-leg SQLSTATE off a source reading, and
one of them said "measured on both legs rather than read from source" - which
is the opposite of what backs it. Neither records a SQL_DRIVER_VER or tested
build, and the PR body's both-legs run is dated 2026-09-25, before either test
existed, so nothing in the PR measures them.

Section 2.1 of the ODBC instructions requires a source citation and a
measurement for a behavioral parity claim, and admits a source citation alone
only when the entry states its evidence level and names the measurement that
would close it. These now do that: each carries an EVIDENCE paragraph naming
the --compare-with-msodbcsql run that would close it, matching how
BinaryVariantPayloadPastTheCeilingIsRefused records its own.

The unconditional assertion stays rather than becoming an ODBC_TEST_TARGET
split, and the comment says why: PR validation runs this suite on both legs, so
a wrong expectation fails the reference leg instead of shipping. If that run
shows msodbcsql answering differently, it is a measured divergence needing the
split and a registry entry, not a changed expectation.

Swept the rest of the PR's added measurement claims. The dated ones belong to
tests that existed for the 2026-09-25 run; the remaining "measured on both
legs" in this file predates the branch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Let a buffered UDT trim zero padding, matching the reference's DAE arm

9b1bbb8d gave the DAE limit a hard-reject overflow policy for a UDT, reasoning
by symmetry with the materialized arm. msodbcsql is not symmetric here.
`ValidatePutDataLength`'s SQL_C_BINARY case tests IsSQLBinary, which includes
SQL_UDT_MAPPED (sqlcprot.h:1294), and on overflow calls CheckTrailingZeros and
shortens cbValue rather than failing (sqlccmd.cpp:11192-11215) - where
ParamToSQLType's SQL_UDT_MAPPED arm has no such call (sqlcfunc.cpp:2681-2696).
So a bounded DAE value with a zero-only overflow is accepted there and was
refused here.

Drop the trims_padding field rather than set it true everywhere: with the UDT
case corrected no caller wants the strict policy, and a field that is always
true is dead weight that invites the wrong conclusion about which paths differ.
The asymmetry is now recorded on both sides - on pad_unit, and on the
materialized arm - so neither gets "fixed" later to agree with the other.

The materialized hard reject from fa598754 is unchanged and still correct.

Evidence: source reading only on both arms; a retail measurement recording
SQL_DRIVER_VER would close it. Regression keeps the per-chunk bound, the
accumulated-total bound and the unlimited case, and now asserts the trim;
mutation-verified by dropping Udt from same_unit.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Assert the zero-padded UDT overflow on both legs, and pin the 8000 clamp

Two gaps, both of which the bar this PR sets should have caught.

The e2e comment said a zero-padded overflow is "asserted here" while the body
had only a non-zero overflow and the unlimited round trip. The zero case was
covered by a unit test that never runs on the msodbcsql leg - and it is
precisely the case read against the reference's *neighbouring* varbinary arm
rather than the UDT arm itself, so it is the least certain reading and the one
most worth measuring. Adds a third phase binding the payload plus a trailing
0x00 at ColumnSize = payload length, expecting 22001 on both legs.

The min(column_size, SQL_PREC_UDT) clamp on both the materialized and DAE arms
was unpinned: removing it from both left the whole crate green. It is not dead
code - SQLBindParameter caps ColumnSize at 8000, but SQL_DESC_LENGTH is
writable on an IPD (classify_field) and set_desc_field stores it without
calling parameter_column_size_is_valid, so bind at 0, set SQL_DESC_LENGTH to
20000, and the clamp is the only thing holding the 8000 ceiling. Covered now on
both arms; that mutation fails them.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Theekshna Kotian <tkotian@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for human review Automation flag indicating an item is ready for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streamed parameter rebind leaks prepared handles and panics after repeated cancellation

5 participants