feat(search): make the embedding model configurable

[mrrobertkent]

The problem

Semantic search is a headline feature, but the model behind it is fixed:

// src/retriv/index.ts
const embeddings = await cachedEmbeddings(transformersJs())

With no argument, retriv falls back to bge-small-en-v1.5: 384 dimensions, 33M parameters, the smallest model it ships. There's no config key, no flag, and no env var to change it. The device is fixed too, and transformers.js defaults to CPU under Node.

That's a fine default for a handful of packages. It gets thinner as you scale, because search builds one sqlite-vec DB per package and pools results across all of them:

// src/commands/search.ts
allResults = await Promise.all(
  dbs.map(dbPath => searchSnippets(query, { dbPath }, { limit, filter })),
)

Scores from separately-built indexes get merged, sorted, and truncated together. Cross-corpus ranking is exactly where embedding quality shows up, so the more skills you install, the more the smallest-model default costs you.

What this adds

An Embedding model and Embedding device entry in skilld config:

◆  Settings
│  ○ Data sources
│  ○ OAuth providers
│  ○ Enhancement model
│  ● Embedding model (bge-small-en-v1.5 · local model powering skilld search)
│  ○ Embedding device (auto · where the embedding model runs)
│  ○ Target agent
│  ↑/↓ to navigate • Enter: confirm
└

◆  Embedding model — indexes and queries docs for skilld search
│  ● BGE small (English) (384d · fastest to index, smallest download)
│  ○ BGE base (English)
│  ○ BGE large (English)
│  ○ BGE m3 (multilingual)
│  ↑/↓ to navigate • Enter: confirm
└

Backed by embedModel and embedDevice config keys, plus SKILLD_EMBED_MODEL and SKILLD_EMBED_DEVICE for single runs. Precedence is env, then config, then default.

Every model runs offline through transformers.js, so no new dependency and no API key.

Device benchmarks

Apple M5 Max, 120 documents, best of 3 after warm-up (docs/sec, higher is better):

Model	cpu	coreml	webgpu
bge-small-en-v1.5	664	198	1713
bge-base-en-v1.5	198	68	580
Xenova/bge-large-en-v1.5	71	9	201

WebGPU is 2.6x to 2.9x faster than CPU at every size, and 4.4x end to end through the index pipeline (chunk, embed, write, query). The practical effect is bigger than the ratio suggests: bge-large on WebGPU indexes faster than bge-base does on CPU, so you can move up two model sizes and still finish sooner.

CoreML is consistently slower, by 3x to 8x, because it falls back to CPU for unsupported ops and pays for graph partitioning. It's still offered, because the ranking is hardware-specific and users on other machines should be able to try it. The picker leads with that caveat rather than burying it.

Backward compatibility

Both defaults are unchanged. The model stays bge-small-en-v1.5, and the device defaults to auto, which resolves to undefined so the option is omitted entirely and transformers.js keeps its own resolution. Nothing changes unless you pick something.

I deliberately did not change the default model. A larger one would strand every existing index, and that migration call belongs to you, not to a feature PR. It's a one-line change in DEFAULT_EMBED_MODEL if you want it.

Two correctness details

bge-large is pinned by full repo id. retriv's bare bge-large-en-v1.5 preset maps to onnx-community/bge-large-en-v1.5, whose weights return 401. resolveModelForPreset and getModelDimensions both succeed for it, so nothing surfaces the problem until the first index fails with an Unauthorized error. Xenova/bge-large-en-v1.5 carries the same weights. (Fixing the preset itself is in skilld-dev/retriv#17.)

The embedding cache needed a stronger identity. It keyed vectors by text hash and validated only the dimension count. That was safe while the model was fixed. Selecting one makes it reachable, since bge-large-en-v1.5 and bge-m3 are both 1024d, so switching kept every cached vector and served one model's embeddings against the other's queries.

No crash, just silently wrong ranking. On a 43-document corpus the correct answer for "how do I install this" dropped out of the top 3 entirely:

before:  f24, f35, f39
after:   d2   ("Install the package with npm and add it to your dependencies.")

Cache identity is now <model>@<device>, cleared on change. Device is included because the same model on a different backend can differ numerically. A cache with no stored model predates the key and has unknown provenance, so it's cleared too.

Testing

test/unit/embed-models.test.ts (17 tests) covers resolution precedence for model and device, blank-env handling, auto collapsing to undefined, and registry integrity. One test cross-checks every entry against retriv/embeddings/model-info:

for (const model of EMBED_MODELS) {
  const resolved = resolveModelForPreset(model.id, 'transformers.js')
  expect(resolved).toContain('/')
  expect(getModelDimensions(model.id)).toBe(model.dimensions)
}

Our declared width drives the rebuild warning, so drifting from retriv's registry would mean telling users the wrong thing. This fails loudly instead.

test/unit/embedding-cache-identity.test.ts (5 tests) pins each invalidation branch: unchanged identity keeps vectors, model change at equal dimensions clears, dimension change clears, device-only change clears, legacy cache clears.

Test Files  55 passed (56)
     Tests  883 passed (884)

pnpm typecheck clean.

Two pre-existing issues, flagged not worked around

Both reproduce on an unmodified checkout:

1. test/unit/git-skills.test.ts has one failing test (fetchGitSkills local path returns []). Verified identical on a clean tree via git stash.
2. pnpm lint cannot run at all. The repo pins typescript@7.0.2 and @typescript-eslint@8.66.0 refuses to load with "typescript-eslint does not support TS 7.0". I matched surrounding style by hand, so happy to reformat anything that's off.

CI

.github/workflows/test.yml triggers on push only, with no pull_request event. Pushes from a fork run in the fork, so PRs from forks never report a check here. Glad to add the trigger in a separate PR if useful, since it would unblock check reporting for every outside contribution.

---

Happy to adjust the model list, naming, or where these live in the config menu.