Repository navigation
fix: drop voy-search and the WASM module it distributed (#228) - #231
Merged
Merged
Conversation
WP.org review R agentic-admin/schmitzoide/1Jun26/T4 asked us to ship
readable source for the distributed .module.wasm. Removing the binary
answers that more completely than vendoring its Rust source would.
voy-search was used for one thing: nearest-neighbour lookup over
embeddings from Xenova/all-MiniLM-L6-v2. Those come out L2-normalised
(`normalize: true`), so cosine similarity is a plain dot product, and at
384 dimensions an exhaustive scan over a few thousand chunks costs
single-digit milliseconds — far below the cost of embedding the query
itself. An ANN index buys nothing at this scale.
- vector-store.js: replace the Voy instance with a Float32Array[] and an
exhaustive cosine scan. search() now returns real similarity scores
instead of the inverse-rank placeholder Voy's lack of distances forced.
- IndexedDB v1 -> v2: vectors persist as Float32Array via structured
clone, in a new `embedding-index` store; the old `voy-index` store is
dropped on upgrade. Restore requires vectors and metadata to be present
and equal in length, so a stale or partial index resets instead of
scoring against mismatched metadata.
- index(): clear chunkMetadata alongside `embeddings`. It was rebuilt
from scratch while metadata accumulated, so a second index() run left
the two misaligned and returned the wrong chunk for a query. The
positional alignment this change relies on made the latent bug real.
- webpack: drop the asyncWebAssembly experiment, now unused.
The worker interchange shape ({ id, title, url, embeddings }) is
unchanged, so indexing-worker.js needs no edit.
Net effect on the distribution: build-extensions/ ships JS and CSS only.
No WebAssembly, no binaries, every file traceable to source in the repo.
Refs #228
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
8 of 14 tasks
ivdimova
approved these changes
Aug 25, 2026
ivdimova
left a comment
ivdimova
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed the voy → plain-JS cosine replacement in depth.
- Dot-product-as-cosine is correct: both embed paths (vector-store.js and indexing-worker.js) set
normalize: true, so the vectors are L2-normalised and the dot product is the cosine. - IndexedDB v1→v2 migration creates the new
embedding-indexstore and drops the legacyvoy-index; incomplete/legacy state resets cleanly to empty. - Good catch on the real bug: index() now clears chunkMetadata alongside embeddings, fixing the append-vs-rebuild misalignment that returned wrong chunks on re-index.
- Exhaustive scan is the right call at this scale — single-user, in-browser; an ANN index buys nothing and dropping it removes the only WASM binary we ship, permanently closing the WP.org T4 item.
CI green (build, JS lint, PHP lint, unit tests). LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What: removes the
voy-searchdependency, so the plugin no longer ships a.wasmbinary.Why
WP.org review T4 asked us to include readable source for the
.module.wasmwe distribute. We had already answered that at T1 by pointing at voy's upstream repo, and the reviewer raised the same file again at T4. Removing the binary settles it permanently instead of explaining it a third time.@ivdimova your drafted reply proposed keeping it and citing upstream, which is a fair call and exactly what we did in the earlier rounds. We went the other way only because this file keeps coming back and the 1 June email warned the next round could be the last. Your two fixes in #230 are merged and unaffected.
What replaced it
voy did one thing: find the embeddings closest to a query. Our embeddings come out of the model already L2-normalised, so similarity is just a dot product. The replacement is about 50 lines in
vector-store.js: score every vector, sort, take the top K.Also in here:
Float32Arrayin a newembedding-indexstore (DB v1 to v2). The old voy index is dropped, so existing users re-index once.search()now returns real similarity scores instead of thetopK - rankplaceholder that voy's missing distances forced.index()rebuilt the vectors from scratch but appended to the chunk metadata. Re-indexing left the two misaligned, so a query returned the wrong code chunk.It works
Tested in the browser on a real install, 11,241 indexed chunks.
Query: "find the code that verifies WordPress core file checksums"
Correct files, correct order, real scores. The index survived a hard refresh and restored from IndexedDB. Search is instant at 11k chunks, so no ANN index is needed at this scale.
Also verified: the built zip activates in a clean WP Playground install with
WP_DEBUGon, no PHP errors, 34 abilities register. 91 unit tests pass,composer lintandlint-jsclean.Result
build-extensions/ships JavaScript and CSS only. No WebAssembly, no binaries, every distributed file has readable source in this repo.Version stays at
0.11.0. Part of #228.