Skip to content

Expand ambiguous surfaces to every lemma they are a form of - #124

Merged
kdroidFilter merged 1 commit into
masterfrom
feat/multi-lemma-surfaces
Oct 11, 2026
Merged

kdroidFilter merged 1 commit into
masterfrom
feat/multi-lemma-surfaces

Conversation

@kdroidFilter

Copy link
Copy Markdown
Owner

Summary

Fixes kdroidFilter/SeforimMagicIndexer#3 on the reader side.

Some surfaces are forms of several lemmas, but lexical.db tied each surface to one base_id only. Since #116/#117, a token expands to the entries it is truly a surface of, so פנינו expanded to פנה only and never to פנים ("our face").

The v0.5.0 dictionary adds surface_extra_base(surface_id, base_id), with 26,670 extra lemmas judged in kdroidFilter/SeforimMagicIndexer#5. MagicDictionaryIndex now reads it:

  • A token matches each lemma it is a form of.
  • The surface counts among the forms (inflections) of each of those lemmas, so expansionFor treats them as its own entries.
  • The surface's variants stay with its primary lemma, since they hold associations of that reading.
  • Dictionaries without the table keep the previous LOOKUP_SQL, so older lexical.db files work unchanged.

Testing

  • :search:jvmTest --tests '*MagicDictionaryIndexTest*': 22/22, including a new test for the פנינו → פנה + פנים case.
  • Query run on the real v0.5.0 DB: פנינו → פנה, פני, פנים. Timing is unchanged (2–20 ms for frequent words like כל, אמר).

Note

DownloadLexicalDb only picks a release asset named exactly lexical.db. v0.5.0 uses that name, and the v0.4.1 asset was renamed to match.

lexical.db v0.5.0 lists the extra lemmas of ambiguous surfaces in
surface_extra_base (פנינו: base פנה, also פנים). MagicDictionaryIndex now
matches a token to each of its lemmas and counts it among the forms of
each, so a search for פנינו also expands to the paradigm of פנים. The
surface's variants stay with its primary lemma. Dictionaries without the
table keep the previous query.

Fixes kdroidFilter/SeforimMagicIndexer#3

Signed-off-by: Elie Gambache <elyahou.hadass@gmail.com>
@kdroidFilter
kdroidFilter merged commit de978b5 into master Oct 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ambiguous surface forms are tied to one lemma only (פנינו → פנה, not פנים)

1 participant