fix: stop claiming the registered search column is the only indexed one

[rohan-hotdata]

Closes the measurement half of #73 and ships the fix it points at.

The defect

sql_tool_description told the model, unconditionally:

Here the BM25-indexed column is '<column>' on <table>

It derived that from search_table/search_column, which the caller supplies to configure one tool — the search tool, which necessarily ranks over one corpus. The library republished that as a statement about the database. Those are different propositions, and on any database indexing more than one column the second is false.

This is the same class as the claims #59 fixed: an unverified exclusive assertion in the model-facing contract, invisible to tests, followed by the model in preference to what hotdata_describe_tables reports. The library already holds itself to the right standard next door — query_catalogs runs a live information_schema query rather than trusting the database record's default_catalog.

What the measurement says

84 runs, gpt-5.1, langchain_bm25_demo, one fresh thread each. The question is an aggregate whose numbers live on listings while the search tool is registered over listing_corpus, a 500-row sample of it. Runs are classified mechanically off the submitted SQL, never off the answer prose.

SQL description's worked example names	searched listings	composed	used ILIKE
the corpus (shipped 0.13.0)	0/12	7/12	4/12
nothing	0/12	5/12	7/12
the corpus, plus prose saying to check searchable_by	0/12	7/12	5/12
the corpus, with both columns named	2/12	8/12	4/12
one call each, corpus leading	3/12	8/12	4/12
this PR's generated wording	7/12	10/12	2/12

Three findings, in order of how much they shaped the fix:

The model writes the call it is shown. Prose is inert — three sentences telling it that indexed columns vary by table and to check searchable_by moved the route in 0 of 12. Naming the column without a call moved 2 of 12. Giving the column its own worked call moved 8 of 12.

searchable_by alone never moved anything: 0 of 24. Two runs had the annotation in context from an earlier describe turn and then wrote LOWER(description) LIKE '%quiet garden%' against that same column. #40 shipped true information that nothing consumes on its own.

Removing the concrete example makes things worse, not better — substring fallback went 4/12 to 7/12. So the fix could not be to soften the sentence into vagueness.

What changed

• The sentence now says the registered column has an index, which is what the caller actually told us.
• make_hotdata_tools(searchable_columns=[(table, column), ...]) names the others. Each pair is confirmed against list_search_indexes before being named, and one no ready index covers is dropped with a warning — BM25 has no brute-force fallback, so a column offered wrongly is a hard error at the point the model has committed to the route. One control-plane call per distinct table.
• Every confirmed column gets its own worked call. Order is the caller's, since the leading one is what the model reaches for.
• SearchableColumn and verify_searchable_columns exported.

Honest limits

Only the headline is significant. Right-table rate 0/12 → 7/12 is Fisher exact p = 0.005. Composition 7/12 → 10/12 (p = 0.37) and ILIKE 4/12 → 2/12 (p = 0.64) are direction, not effect, and the CHANGELOG labels them that way rather than quoting them as results.

One model, one dataset, one question, n=6 per cell. The behavioural claim also rests on the two-column case; the builder handles N but nothing measured N > 2.

Most of the gain needs a prior describe call — 5/6 with one in the thread, 2/6 without. The weaker condition is the likelier one for a single-question session.

This does not fix the answers. 0 of 84 runs across every wording produced the correct figure. Fixing the route relocates the error onto k: every run guessed 100 or 500 against a match pool of 1,426. Also recorded here, verified live — omitting bm25_search's limit caps at exactly 1,000 with nothing marking the truncation, so there is no way to ask for a whole pool without already knowing its size. That is #62 and it is untouched by this PR.

Nothing in CI can catch a regression in the effect. The new tests pin the description's text, not the routing.

Checks

571 tests pass (23 added), ruff clean, mypy 59 → 54 against the origin/main baseline with zero new errors. Docs audited over the final diff: the roadmap's claim that discovery made un-pinning the corpus "safe" is corrected, since the agent can read searchable_by and was measured never acting on it, and the matching README sentence with it. README documents the new parameter; its example was run against the live engine.

[claude]

nit: a confirmed column can be silently dropped from the description after the caller paid a control-plane call for it (not blocking).

Three filters decide what survives, and none of them tell anyone when they discard something:

• Here, kind == SEMANTIC and composable — a declared text column is dropped whenever the registered route is semantic.
• Line 233, kind == TEXT — a declared semantic column is dropped whenever the registered route is text.
• The if semantic and not search_route.composable branch at line 191 never calls _search_examples at all, so a plain-vector registered route discards every declared column, including BM25 ones on other tables that compose perfectly well.

verify_searchable_columns warns loudly for a pair no index covers, so a caller reasonably reads "no warning" as "named". These drops are the case where the index does exist and the column still doesn't appear.

The plain-vector branch is also where the PR's own thesis bites: it asserts "Ranking rows by meaning is not available in SQL here", which is a claim about the database derived from one tool's registration. If the caller declared a provider-backed semantic column on another table, that sentence is false in exactly the way the sentence this PR is fixing was.

A logger.debug/logger.warning on the discarded ones would be enough for the two kind filters; the plain-vector branch probably wants the composable paragraph emitted for whatever declared columns do compose.

[claude]

nit: this warning asserts something we don't know when the listing failed (not blocking).

list_search_indexes fails open — an unreachable control plane or a token that can't list indexes returns [] plus its own warning. Every declared column on that table then lands here and logs "no ready search index covers 'description' on d.public.listings", which is a statement about the database made on the strength of a call that didn't complete. The index may well be there.

Dropping is the right conservative behaviour either way; only the message is wrong. Distinguishing "listing returned nothing" from "listing failed" would need list_search_indexes to signal the difference, so the cheap version is softening the wording to what is actually known — e.g. "no ready search index was found covering …".

[claude]

nit: nothing tests this wiring (not blocking).

searchable_columns appears in no test — tests/test_indexes.py exercises verify_searchable_columns directly and tests/test_descriptions.py passes also_searchable= straight to sql_tool_description. The join between them (this call, plus also_searchable=confirmed at line 634) is the only part a caller actually touches, and it is the part that would silently stop threading through. descriptions() already builds tools off a MagicMock client, so one test patching hotdata_langchain.tools.verify_searchable_columns and asserting the worked call reaches hotdata_execute_sql's description would close it.

Also worth a line in the docstring above: searchable_columns is ignored entirely without database_id, since verify_searchable_columns returns [] with no database to confirm against, and a malformed pair raises ValueError from here.

[claude]

super nit: excluded names the wrong thing — the tuple holds kwargs sets, and the first element excludes nothing. kwarg_sets or variants reads truer for what is only a mypy annotation fix (not blocking).

[claude]

Reviewed the full diff plus the surrounding source in hotdata_langchain/tools.py, hotdata_langchain/indexes.py, tests/test_indexes.py and tests/test_descriptions.py.

The core change holds up: the exclusive claim is gone, verify_searchable_columns reads back before naming rather than trusting the caller, order is preserved through both the verifier and _search_examples, the registered pair is deduped against the declared ones, and a semantic column is never offered to bm25_search. The CHANGELOG and README separate the significant result from the directional ones, and the limits section says what the change does not fix.

Four non-blocking comments inline — the main one is that a confirmed column can be silently dropped from the description by the kind filters or by the plain-vector branch, with no signal to the caller.

CI (CI / Test (Python 3.12)) had not reported at review time, so I have not verified the 23 new tests pass.

[claude]

nit: the prefer sentence is now emitted twice whenever both paragraphs run (not blocking).

Previously exactly one branch produced composable, so prefer appeared once. Now text_paragraph() and semantic_paragraph() each end with it, and both are joined for the two cases this commit adds — a semantic route with declared text columns (test_a_declared_text_column_survives_a_semantic_registered_route) and a text route with declared composable semantic columns. The model then reads, a sentence apart:

… Prefer this whenever the answer aggregates over the matches rather than listing them — it keeps the whole cohort in the query instead of passing ids back as literals. To rank rows by how well their text matches a phrase, call bm25_search inside SQL: … Prefer this whenever the answer aggregates over the matches rather than listing them — it keeps the whole cohort in the query instead of passing ids back as literals.

It is not wrong — "this" resolves to the preceding call form each time — and nothing in the tests pins the count, so it passes. But this PR's own thesis is that the description text is the product, and a verbatim repeat is the kind of thing the measurement is sensitive to and CI is not. Hoisting it out of both paragraphs and appending it once to composable would read better; if the repetition is deliberate (each function getting its own "prefer"), a line saying so would keep the next reader from deduplicating it.

[claude]

super nit: this interpolates None when a caller passes a route without the pair (not blocking).

make_hotdata_tools never reaches it — search_route is None unless has_search — but sql_tool_description is exported, and a direct call with search_route= and no search_table/search_column now renders Ranking 'None' on None by meaning needs the query as a vector. The old wording was pair-independent, so this is new. Falling back to the generic "Ranking rows by meaning" form when either is missing would cover it.

[claude]

All four prior threads are addressed in e876806: text and semantic columns now survive whichever route is registered, the plain-vector branch scopes its claim to the registered column and still offers the columns that compose, the verify_searchable_columns warning no longer asserts what a failed listing cannot know, the wiring gets a test, and excluded is renamed. Two new non-blocking notes inline. CI / Test (Python 3.12) had not reported when this review started.