packaging: move the Ollama client behind a [semantic] extra

[paoValle]

Implements #75 for the ollama part. numpy stays a hard dependency, and the reason is below: moving both as the issue first proposed would break the default search path.

What changes

• ollama leaves dependencies and becomes the semantic extra in pyproject.toml (uv.lock regenerated).

• embed.py imports the client lazily and, when it is missing, raises an ImportError that names what to install:

  Semantic search needs the Ollama Python client, which is an optional extra:
    pip install "opensdmx[semantic]"
  It also needs a running Ollama server with the embedding model pulled:
    ollama pull nomic-embed-text-v2-moe
  

• Deprecation path: from opensdmx import semantic_search keeps working, because the client is only imported when the function is called. Only a caller that actually calls semantic_search/build_embeddings without the extra breaks, and it gets an error that says how to fix it (in the CLI this already goes through except Exception as e → Error: ..., so no traceback).

• README: the extra is named in the install section and in "Semantic search / Setup".

Why numpy cannot move

The issue states that only search --semantic, embed and guide use numpy. That is not the case: the keyword ranking (the default path) is BM25 implemented on numpy — ranking.py:75-128 uses np.array, np.fromiter, np.nonzero, and discovery.py:408-432 calls it on every opensdmx search. Moving it behind the extra would break default keyword search.

Making numpy optional as well needs a numpy-free scorer (or a pure-Python fallback with benchmarks), which is a change of its own; I left it out of this PR. Happy to open a separate issue describing it if you want.

Verification

• tests/test_semantic_extra.py: _require_ollama/semantic_search/build_embeddings all name the extra when the client is missing (client hidden through sys.modules), the keyword path does not need it, and a packaging regression test keeps ollama in the extra and out of the mandatory dependencies.
• uv run pytest tests/ -v → 409 passed (the dev environment now genuinely runs without ollama, so the ImportError path is exercised for real rather than through an importorskip).
• uv run ruff check src/ and uv run mypy clean.

I did not touch LOG.md: the release note is yours to write with the format and measurements you normally use — tell me how to title it if you would rather I add it.

[greptile-apps]

[Medium risk] Moves Ollama client to an optional dependency.

The PR appears safe to merge; no blocking issue remains.

Summary

Moves the Ollama Python client into the semantic extra while keeping numpy required for keyword search.

• The guide extra also includes Ollama, addressing the earlier installation issue.
• Missing-client errors show installation commands, and the CLI preserves [semantic] in those messages.
• The keyword regression test now calls search_dataset with Ollama hidden.
• The documentation change since the previous review matches the current setup. No new actionable issues were found.

_{Reviews (4) · Last reviewed commit: "docs: name the [semantic] extra in the s..."}

[paoValle]

Both findings addressed in c97c0ba.

1. [guide] lost its client (real regression, thank you). run_guide falls back to semantic_search when no dataset is supplied (guide.py:36,72), so [guide] now carries ollama>=0.6.1 as well, with a comment saying why. Evidence rather than assertion: the dev environment installs opensdmx[guide], and import ollama → ollama 0.6.1 is back in it; uv.lock regenerated. A packaging test now pins ollama in both semantic and guide.

2. The keyword test never searched. Right — it called ranking.tokenize. It now runs the real search_dataset over the stub catalog from tests/test_discovery.py with the client hidden, and asserts it returns UNEMP for "unemployment": the test only proves the keyword path is client-free if a search actually happens.

uv run pytest tests/ -q → 410 passed; ruff check src tests and mypy src clean.

[aborruso]

@paoValle thanks. I checked the branch at c97c0ba: 410 passed, ruff and mypy clean. I also built the wheel and installed it in two empty venvs: without the extra ollama is not installed, import opensdmx and keyword search work; with [semantic] the client is there. On numpy you are right: ranking.py builds BM25 on it, so it has to stay.

One bug and a couple of smaller points inline. Two more that have no line in the diff:

• docs/search.md:12 still says only that semantic search needs a reachable Ollama server. I think it should mention the extra too.
• LOG.md and the release note: I'll write them at release time, no need to add them here.

[aborruso]

A question about the name rather than a change request. The semantic backend is an open question (#77: a static retriever might replace Ollama). If that happens, [semantic] would mean something different from today. Would it be better to name the extra after the backend ([ollama]), or do we keep [semantic] as the stable name and change what it contains later?

[aborruso]

@paoValle agreed, let's keep [semantic]. One thing left: docs/search.md:12 still says only that semantic search needs a reachable Ollama server. Could you mention the extra there too?

[paoValle]

All three covered in 9f3a08f.

1. Rich was eating [semantic] (real bug, thank you). Both handlers now print rich.markup.escape(str(e)) — search --semantic (cli.py) and embed (cli.py). The hint also names the uv path now, mirroring the README:

$ opensdmx search --semantic unemployment      # on an install without the extra
Error: Semantic search needs the Ollama Python client, which is an optional 
extra:
  pip install "opensdmx[semantic]"
  uv tool install "opensdmx[semantic]"   # if installed as a CLI with uv
It also needs a running Ollama server with the embedding model pulled:
  ollama pull nomic-embed-text-v2-moe
Tip: use keyword search instead:  opensdmx search <keyword>

Two CliRunner tests (test_search_semantic_cli_prints_the_install_hint, test_embed_cli_prints_the_install_hint) assert opensdmx[semantic] and the uv line in the CLI output, and they fail without the escape — I verified the mutation: without it, the output loses the brackets and shows pip install "opensdmx" exactly as you measured. They patch _check_api_reachable out, following the repo's CLI-test convention, so nothing here touches the network.

2. Extra naming. I would keep [semantic]. The extra's contract is the capability (--semantic, semantic_search), and the backend is exactly what #77 leaves open: renaming to [ollama] would churn every install command, README and hint at the moment the backend lands, while keeping the name stable lets the contents change with no user-visible break. The one text that will need updating when #77 lands is the hint's first line ("needs the Ollama Python client") — a content change, not a naming one. Happy to go either way if you prefer [ollama] as the stable name.

3. Test duplication. The two packaging tests now share a session-scoped pyproject fixture instead of each loading the file.

Verification on the PR head: uv run pytest tests/ -q → 412 passed (410 + the two new), uv run ruff check clean on the touched files (the 4 repo-wide errors in scripts/monitor_latency.py are pre-existing, present on the base too), uv run mypy src/opensdmx clean.

[paoValle]

Done — docs/search.md now names the extra on the semantic-search bullet (1aa343b): the semantic commands need the [semantic] extra (pip install "opensdmx[semantic]" / uv tool install "opensdmx[semantic]") and a reachable Ollama server. And thank you for verifying the wheel both ways.