Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,34 +241,50 @@ The large default context can exceed memory for big models on constrained hardwa

### Embedding Model

`skilld search` uses a local embedding model. It runs offline through transformers.js. It needs no API key or network after the first download. Pick one under **Embedding model** in `skilld config`:
`skilld search` is powered by a local embedding model. It runs offline through transformers.js. It needs no API key or network traffic after the first download. Pick one under **Embedding model** in `skilld config`:

| Model | Dimensions | Notes |
|-------|-----------:|-------|
| `bge-small-en-v1.5` | 384 | Default. Fastest to index, smallest download. |
| `bge-base-en-v1.5` | 768 | Balanced accuracy and speed. |
| `Xenova/bge-large-en-v1.5` | 1024 | Most accurate English retrieval, slowest to index. |
| `bge-m3` | 1024 | Multilingual, 8192-token context. |

Larger models retrieve more accurately but cost more time and memory when indexing. Set `SKILLD_EMBED_MODEL` to override the saved setting:
Larger models retrieve more accurately but cost more time and memory when indexing. Locally-pulled [Ollama](#ollama-embedding-models) models can be used too. Export `SKILLD_EMBED_MODEL` to override the saved setting for every index and search command in the current shell:

```bash
export SKILLD_EMBED_MODEL=bge-m3
skilld update --force
skilld update
```

Each search index belongs to one model and device. Keep environment overrides set for both indexing and querying. Rebuild indexes after either setting changes:
Search indexes must use one embedding model and device. Rebuild them after switching either setting:

```bash
skilld update --force
skilld update
```

### Ollama Embedding Models

If [Ollama](https://ollama.com) is running, locally-pulled embedding models appear in the **Embedding model** picker alongside the built-in ones. They are addressed as `ollama:<name>`, matching the `-m ollama:<name>` syntax used for enhancement models:

```bash
ollama pull qwen3-embedding
SKILLD_EMBED_MODEL=ollama:qwen3-embedding skilld add npm:vue
```

Only models that advertise the `embedding` capability are listed, so chat models cannot be selected by mistake. Dimensions and context length are read from Ollama. skilld validates each vector before indexing.

This talks to Ollama's HTTP API directly. It needs no additional dependency or API key. Set `OLLAMA_HOST` to point at a non-default daemon. If Ollama is not running, the picker simply shows the built-in models.

Ollama manages its own execution device, so **Embedding device** does not apply to `ollama:` models.

### Embedding Device

The embedding model runs on the CPU by default. **Embedding device** in `skilld config` moves it onto a GPU backend, which can be substantially faster:

| Device | Notes |
|--------|-------|
| `auto` | Default. Lets transformers.js choose, CPU under Node. |
| `auto` | Default. Lets transformers.js choose; CPU under Node. |
| `cpu` | Always available, predictable. |
| `webgpu` | Fastest on Apple Silicon in testing. |
| `coreml` | Apple Neural Engine. Measured slower than CPU for these models. |
Expand All @@ -281,13 +297,13 @@ Measured on an Apple M5 Max, 120 documents, best of 3 after warm-up (docs/sec):
| `bge-base-en-v1.5` | 198 | 68 | **580** |
| `Xenova/bge-large-en-v1.5` | 71 | 9 | **201** |

WebGPU was 2.6 to 2.9 times faster than CPU at every size. `bge-large` on WebGPU indexed faster than `bge-base` on CPU. CoreML was consistently slower.
WebGPU was 2.6 to 2.9 times faster than CPU at every size. On WebGPU, `bge-large` indexed faster than `bge-base` on CPU and improved retrieval. CoreML was consistently slower.

The ranking is hardware-specific, so benchmark before trusting a device on other machines. Set `SKILLD_EMBED_DEVICE` to override the saved setting:
The ranking is hardware-specific, so benchmark before trusting a device on other machines. Export `SKILLD_EMBED_DEVICE` to override the saved setting in the current shell:

```bash
export SKILLD_EMBED_DEVICE=cpu
skilld update --force
skilld update
```

If a backend is unavailable, indexing fails to start. Switch back to `auto`.
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@
"devDependencies": {
"@antfu/eslint-config": "catalog:dev-lint",
"@types/node": "catalog:dev-build",
"@typescript/native": "catalog:",
"@vitest/coverage-v8": "catalog:dev-test",
"bumpp": "catalog:",
"eslint": "catalog:dev-lint",
Expand Down
1 change: 1 addition & 0 deletions packages/protocol/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
"@antfu/eslint-config": "catalog:dev-lint",
"@arethetypeswrong/cli": "catalog:",
"@types/node": "catalog:dev-build",
"@typescript/native": "catalog:",
"eslint": "catalog:dev-lint",
"obuild": "catalog:dev-build",
"publint": "catalog:",
Expand Down
Loading