Conversation
| return None, "The installed embedding model did not match the collection index." | ||
| return embed_text, None | ||
|
|
||
| # Fallback: a legacy/imported collection whose stored key digest does not line |
Contributor
There was a problem hiding this comment.
This appears unrelated to typo correction. Could you clarify why it is included in this PR? If it is not required for this feature, could we move it to a separate PR?
| for material_id in material_ids: | ||
| vocab = None if VOCAB_FORCE_REBUILD else load_vocabulary(user_data_path, material_id) | ||
| if vocab is None: | ||
| texts = iter_chunk_texts(user_data_path, [material_id]) |
Contributor
There was a problem hiding this comment.
For an already-indexed large material, the first search after enabling this feature loads all stored chunks and builds its vocabulary before retrieval can continue. This could surprise users, who may interpret the delay as slower search rather than one-time setup.
Could we build the vocabulary when the feature is enabled, ideally with progress feedback, or at least tell users that the first search may have a one-time setup delay?
Indexing records a per-material list of high-frequency textbook words, and retrieval repairs likely typos in a query against it before searching, so a misspelled term still reaches the right passages. Off by default, controlled by Settings > Advanced > Typo Correction. Enabling it builds any missing vocabularies up front rather than leaving the first search to pay for the build; retrieval still builds one on demand for materials added later.
arjunj05
force-pushed
the
feature/typo-correction
branch
from
September 23, 2026 00:54
ca724c8 to
c98abae
Compare
arjunj05
marked this pull request as draft
September 23, 2026 16:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds a per-material vocabulary of high-frequency textbook words at index time and uses it to repair likely typos in retrieval queries. Off by default; controlled by Settings > Advanced > Typo Correction.