setencoding(ctype=SQL_CHAR) is silently ignored when binding text parameters

[Theekshna]

Summary

Connection.setencoding(encoding=..., ctype=SQL_CHAR) is validated, stored, and returned by getencoding() — but it has no effect on parameter binding. Text parameters are always encoded UTF-16LE and bound as ODBC SQL_C_WCHAR, whatever encoding was requested.

Nothing fails, so callers have no way to discover the setting was ignored.

Repro

import mssql_python
from mssql_python import SQL_CHAR

conn = mssql_python.connect(CONN_STR)
cur = conn.cursor()
cur.execute("CREATE TABLE #t (v VARCHAR(10) COLLATE SQL_Latin1_General_CP1_CI_AS)")

conn.setencoding(encoding="cp1252", ctype=SQL_CHAR)
assert conn.getencoding() == {"encoding": "cp1252", "ctype": 1}   # stored as asked

cur.execute("INSERT INTO #t (v) VALUES (?)", "café")

• Expected: parameter encoded CP1252 and bound narrow — 63 61 66 E9, C type SQL_C_CHAR.
• Actual: parameter encoded UTF-16LE and bound wide — 63 00 61 00 66 00 E9 00, C type SQL_C_WCHAR. The cp1252 setting is never read.

The row round-trips correctly, so the no-op is invisible from the result.

Cause

SQLExecute_wrap honours the requested encoding only when the C type is a real ODBC SQL_C_CHAR:

std::string charEncoding = "utf-8";
if (encoding_settings.contains("ctype") && encoding_settings.contains("encoding")) {
    int ctype = encoding_settings["ctype"].cast<int>();
    if (ctype == SQL_C_CHAR /* real ODBC value: 1 */) {
        charEncoding = encoding_settings["encoding"].cast<std::string>();
    }
}

charEncoding is then consumed only inside case SQL_C_CHAR: in BindParameters / BindParameterArray. But no route sets paramCType to 1:

Route	C type source	Value
execute()	PARAM_C_TYPE_TEXT in param_detect.hpp (ODBC's real SQL_C_WCHAR)	-8
execute() + setinputsizes	_SQL_TO_C_TYPE → ConstantsDDBC.SQL_C_CHAR.value	-8
executemany()	ParamInfo built in cursor.py, same map	-8

(ConstantsDDBC.SQL_C_CHAR is -8, which is ODBC's SQL_C_WCHAR, not SQL_C_CHAR (1) — a long-standing alias, already noted in param_detect.hpp.)

So execution always reaches case SQL_C_WCHAR:, which does param.cast<std::u16string>() and never reads charEncoding. case SQL_C_CHAR: is unreachable for text parameters.

To be clear: binding text wide is a deliberate, well-reasoned choice, and param_detect.hpp:146 explains why (matching the legacy path, unixODBC requiring wide chars on Linux/macOS, and avoiding a Windows-only divergence). This issue is not asking to change that. It is asking that setencoding stop silently accepting a request it cannot honour.

The existing tests can't catch this

Worth flagging separately, since it means CI would not detect a regression here either:

• test_sql_c_char_encoding_failure — ends if not error_raised: pass
• test_encoding_error_propagation_in_bind_parameters — "If no error was raised, that's also acceptable behavior (data may be mangled)", then assert count >= 0
• test_shift_jis_encoding_japanese — calls fetchone() and never asserts the result
• test_cpp_bind_params_str_encoding — ASCII-only payload, passes either way

test_sql_c_char_encoding_failure sets ascii and inserts "Non-ASCII: 你好世界". If the narrow path actually ran, .encode("ascii", "strict") would raise deterministically and the test could assert unconditionally.

Request

Make the no-op non-silent. In order of preference:

1. Honour the requested encoding for parameters; or
2. Document setencoding as not affecting parameter binding; or
3. warnings.warn (or raise) when a requested combination cannot be honoured.

Accepting the value, storing it, echoing it from getencoding(), and ignoring it is the worst of the available options.

Two smaller notes

execute and executemany disagree on the gate. SQLExecuteMany_wrap reads encoding with no ctype check:

std::string charEncoding = "utf-8";  // default
if (encodingSettings.contains("encoding")) {
    charEncoding = encodingSettings["encoding"].cast<std::string>();
}

Inert today because paramCType is never 1, but the two paths should agree.

If ConstantsDDBC.SQL_C_CHAR is ever corrected to 1, please do it together with the above. On its own it would route executemany text through case SQL_C_CHAR: with charEncoding defaulting to utf-16le — i.e. UTF-16 bytes bound as narrow characters, with no user configuration involved.

Question

Is binding text parameters as SQL_C_WCHAR a stable contract across all three routes (execute, execute + setinputsizes, executemany), or is it intentional only for the native execute() path and incidental for the other two?

param_detect.hpp documents the intent clearly for execute(). It's the other two routes, which reach the same result only through the -8 alias, where the guarantee is unclear.

Environment

• mssql-python main, static source review; not reproduced against a packaged wheel
• Files: mssql_python/constants.py, connection.py, cursor.py, pybind/ddbc_bindings.cpp, pybind/param_detect.hpp
• Related config exercised by tests/test_017_varchar_cp1252_boundary.py

Tracking

• Draft PR: FIX: Warn when setencoding settings cannot be applied (#825) #828 (warning-based fix; preserves wide-character binding).
• ADO bug: AB#48882.

[github-actions]

Hi ttk (@Theekshna), thank you for opening this issue!

Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.

If you have additional information to share, please feel free to update the issue.

Thank you for your patience!

[jahnvi480]

I reproduced the behavior against SQL Server and confirmed that the requested encoding is stored but does not change text parameter binding.

Regarding your question: all three routes currently bind Python text as SQL_C_WCHAR—execute(), execute() with setinputsizes(), and executemany(). The native path makes this explicit; the other routes reach the same behavior through the legacy internal SQL_C_CHAR = -8 alias. Our proposed fix preserves that existing cross-platform behavior and does not change the alias independently.

The distinction is that wide-character binding is deliberate, but silently accepting a setting that is not applied is an API/documentation mismatch. The current wiki describes setencoding() as configuring outgoing encoding without documenting this limitation.

PR #828 addresses this by:

• Emitting a caller-visible UserWarning when valid requested settings cannot be applied, while leaving the supported defaults warning-free.
• Clarifying that getencoding() returns the requested settings, not the effective binding.
• Aligning the native encoding checks and replacing the permissive regression assertions identified in this report.

This follows the warning-based option in your request. It does not implement configurable narrow binding; existing UTF-16LE / SQL_C_WCHAR handling remains unchanged.