Create a search box in the documentation

[kildom]

The best solution so far is Orama: supports both lexical and vector (AI embedding) search and runs entirely in browser.

[kildom]

Full chat with AI

User

Is there a very small model that can generate embeddings for some sentence? I want to use it to search text in my big documentation. The model should be small enough to run it in web browser to provide fast search on my page on client side.

Assistant

Yes. There are several embedding models specifically small enough to run in the browser using WebGPU or even WASM. For client-side semantic search over documentation, you don't need a state-of-the-art 1B+ parameter model.

Here are the best options, roughly ordered by practicality.

Model	Size	Embedding Dim	Browser	Quality
model: all-MiniLM-L6-v2 - Sentence Transformer embedding model	~23 MB	384	✅ Excellent	Very good
model: bge-small-en-v1.5 - BAAI embedding model	~33 MB	384	✅ Good	Excellent
model: gte-small - Alibaba embedding model	~35 MB	384	✅ Good	Excellent
model: nomic-embed-text-v1 - Nomic embedding model	~130 MB	768	⚠️ Heavy	Outstanding
model: jina-embeddings-v2-small-en - Jina AI embedding model	~35–40 MB	512	✅ Good	Excellent

My recommendation

For browser search:

all-MiniLM-L6-v2

Pros:

• Only about 23 MB
• Fast (tens of milliseconds for a query)
• 384-dimensional vectors
• Works well for documentation search
• Widely used and tested
• Can run through Transformers.js

Searching a few thousand documentation chunks is nearly instantaneous.

---

Even better: BGE Small

bge-small-en-v1.5

This is one of the strongest small embedding models today.

Advantages:

• Better semantic understanding than MiniLM
• Still only ~33 MB
• 384-dimensional vectors
• Very popular for RAG systems

If you're starting a new project, I'd probably choose BGE Small.

---

Running in the browser

Using the Transformers.js library:

import { pipeline } from '@xenova/transformers';

const embedder = await pipeline(
    'feature-extraction',
    'Xenova/all-MiniLM-L6-v2'
);

const output = await embedder(
    "How do I install the package?",
    { pooling: 'mean', normalize: true }
);

const embedding = output.data;

Then:

1. Precompute embeddings for every documentation chunk.
2. Store them as JSON or binary.
3. Embed the user's query.
4. Compute cosine similarity against all chunks.
5. Return the top results.

For 5,000–20,000 chunks, brute-force cosine similarity in JavaScript is often fast enough.

---

If you want the smallest possible model

There are TinyBERT-based embedding models around 10–15 MB.

They're roughly 2× smaller but noticeably worse at semantic search.

Unless your documentation is huge or bandwidth is critical, I'd stick with MiniLM or BGE Small.

---

Browser performance

Approximate numbers on a modern laptop:

Model	Download	Query embedding
MiniLM	23 MB	10–20 ms
BGE Small	33 MB	20–40 ms
GTE Small	35 MB	20–40 ms
Nomic	130 MB	70–150 ms

The cosine search itself over 10,000 chunks typically takes only a few milliseconds.

---

A good architecture

A common approach is:

• Build time

  • Split documentation into chunks (e.g. 200–500 tokens each).
  • Generate embeddings once using the same model you'll use for queries.
  • Save the embeddings (binary formats like Float32Array or Float16Array are more compact than JSON).

• Client side

  • Load the embedding model once.
  • Load the precomputed embedding index.
  • Embed the user's search query.
  • Compute cosine similarities in JavaScript.
  • Show the best-matching chunks.

This keeps all search local to the browser with no server round trips, making it ideal for static documentation sites.

User

Let's assume I will go with BGE Small. Is it also good for single word search? For example, my documentation describies HTML-like document syntax and there are different align attributes, e.g. paragraph, table, image, e.t.c. Will it correctly search all those attributes? Or, should I also back my engine with some procedural search (e.g. using snowball for word stemming and some library to calculate word distance)?

Assistant

I'd definitely combine semantic search (embeddings) with a traditional lexical search. This is how most modern search engines work (GitHub, Notion, Elastic, Azure AI Search, etc.).

The reason is that embeddings and keyword search excel at different things.

Query	Embeddings	Lexical search
"center an image"	✅ Excellent	⚠️ Depends on wording
"how to align a table"	✅ Excellent	✅ Good
align	⚠️ Okay	✅ Excellent
align="center"	❌ Often poor	✅ Excellent
table-align	❌ Poor	✅ Excellent
img.align	❌ Poor	✅ Excellent
paragraph align	✅ Good	✅ Excellent

Why embeddings struggle with single words

Embedding models are trained to represent the meaning of text, not to index exact tokens.

For example, the query:

align

will produce an embedding roughly representing concepts like:

• positioning
• placement
• arrangement
• justification

The nearest chunks might be:

"Use the position attribute..."

instead of

"align attribute"

because semantically they're similar.

That's exactly what you don't want when someone is searching for an API, attribute, class, or function name.

---

The ideal approach

I'd score each document using both methods.

score =
    0.7 * semantic_similarity +
    0.3 * lexical_score

or even simpler:

if lexical match:
    give a bonus

rank by final score

This works remarkably well.

---

Even better: detect "code-like" queries

Many documentation searches aren't natural language.

These should use mostly lexical search:

align
rowspan
colspan
table
margin-left
<image>
border-radius

Whereas these should use embeddings:

how do I center a table

how can I align text vertically

images overflow outside the page

A surprisingly effective heuristic is:

contains spaces?

No spaces:

align

→ keyword search

Contains spaces:

how to align image

→ semantic search + keyword search

You can refine this by also checking for punctuation:

align=
<paragraph>
foo.bar
table.align

These are almost certainly identifier searches.

---

I would also use stemming

Definitely.

For documentation search, a lightweight pipeline like this works well:

• lowercase
• Unicode normalization
• remove punctuation (except code symbols you care about)
• Snowball/Porter stemming
• BM25 (or TF-IDF if your corpus is small)

BM25 remains one of the strongest lexical ranking algorithms, even today.

---

For your HTML-like language

Suppose your docs contain:

<table align="center">

A user might search:

align

Lexical search finds every occurrence.

Or:

alignment

Stemming finds align.

Or:

center table

Embeddings understand the relationship.

Or:

how do I center tables

Embeddings shine.

Or:

table align attribute

Both methods agree, producing excellent results.

---

What I'd build

For a documentation site, my search pipeline would look like:

1. Lexical index

   • Tokenization
   • Stemming
   • BM25 ranking
   • Exact identifier matching
   • Prefix matching (nice for autocomplete)

2. Embedding index (BGE Small)

   • One embedding per chunk (around 300–500 tokens)
   • Cosine similarity search

3. Hybrid ranking

   • Exact identifier match gets a large boost.
   • BM25 contributes to the score.
   • Embedding similarity contributes to the score.
   • Optionally boost matches in titles or headings.

For a documentation corpus of a few thousand pages or tens of thousands of chunks, this hybrid approach is still lightweight enough to run entirely in the browser. In practice, the lexical index often fits comfortably in memory, and a brute-force cosine search over a few thousand 384-dimensional embeddings is fast enough on modern hardware. The result is a search experience that handles both natural-language questions ("How do I center an image?") and exact API lookups (align, rowspan, margin-left) very effectively.

User

Is there already a library that I can use to do this hybrid search in a browser?

Assistant

Yes, and you don't necessarily need one monolithic library. In fact, the best browser-based solution is often a combination of two specialized libraries:

• a lexical search engine (BM25/full-text)
• an embedding search (vector similarity)
• a small amount of code (~50–100 lines) to merge the scores

This gives you more flexibility and is simpler than it sounds.

Option 1 (my recommendation): MiniSearch + BGE Small

MiniSearch is an excellent pure JavaScript full-text search library.

Pros:

• Tiny (~30 KB gzipped)
• BM25 ranking
• Prefix search
• Fuzzy search
• Field boosting (titles > body)
• Runs entirely in the browser

Example:

const results = miniSearch.search(query, {
    prefix: true,
    fuzzy: 0.2
});

Then compute BGE embeddings for the query:

const embedding = await embed(query);

Calculate cosine similarity against your precomputed vectors:

for (const chunk of chunks) {
    chunk.semanticScore = cosine(queryEmbedding, chunk.embedding);
}

Finally merge:

score =
    bm25Score * 0.4 +
    semanticScore * 0.6;

This is exactly what many production search systems do.

---

Option 2: FlexSearch

FlexSearch is even faster than MiniSearch.

Pros:

• Extremely fast
• Good fuzzy matching
• Memory efficient
• Mature

Cons:

• Ranking isn't as "information retrieval"-oriented as BM25.
• Less suited to documentation search than MiniSearch.

---

Option 3: Orama

This is probably the closest thing to what you're asking.

Orama includes:

• BM25
• fuzzy search
• filters
• facets
• vector search
• hybrid search

All in the browser.

It even has APIs like:

search({
    term: "align table",
    mode: "hybrid"
});

So you don't have to combine scores yourself.

Its main downside is that it's a larger dependency than MiniSearch, but it's still designed for client-side use.

---

Option 4: Fuse.js

Fuse.js is fantastic for fuzzy matching names:

align
aling
algin

But it isn't a true search engine.

I wouldn't recommend it for documentation.

---

Option 5: Lunr.js

This was the classic documentation search library.

Today I'd choose MiniSearch instead because:

• better maintained
• faster
• smaller
• more flexible

---

Vector search

For a few thousand chunks, you don't even need a dedicated vector index.

Suppose you have:

5,000 chunks
384 dimensions

That's:

5,000 × 384 = 1.9 million floats

A brute-force cosine search involves about 2 million multiply-add operations, which modern browsers can perform in just a few milliseconds. There's no need for approximate nearest neighbor (ANN) algorithms like HNSW until your corpus grows much larger (typically tens or hundreds of thousands of chunks).

---

If your documentation grows

Chunks	Recommendation
<10k	Brute-force cosine
10k–50k	Still fine in JS
50k–200k	Consider an ANN index (e.g. HNSW)
>200k	Server-side vector database is usually worthwhile

---

What I'd use

For a static documentation website, my stack would be:

• Lexical search: MiniSearch (BM25, prefix, fuzzy)
• Semantic search: BGE Small running via Transformers.js
• Vector search: Brute-force cosine over precomputed embeddings
• Ranking: A custom weighted combination of BM25 and cosine similarity, with boosts for title matches and exact identifier matches

That gives you a search experience very similar to high-end documentation sites while keeping everything client-side. The only "custom" part is the final ranking function, which is usually just a few dozen lines of straightforward code. It's easy to tune as you learn how your users search, and you avoid being locked into a framework's scoring heuristics.

User

What is BM25?

Assistant

BM25 (short for Best Matching 25) is the ranking algorithm used by many traditional search engines. It's the successor to TF-IDF and is still the default ranking algorithm in systems like software: Elasticsearch - search engine, software: Apache Solr - search platform, and software: Whoosh - Python search library.

The key idea is simple:

A document should rank higher if it contains the search terms frequently, but not just because it's long.

A simple example

Imagine you have three documentation pages:

Page A
------
The align attribute specifies horizontal alignment.

Page B
------
The align attribute can be used on table, image, paragraph, and cell elements.
The align attribute...
The align attribute...
The align attribute...

Page C
------
A complete language reference.
(100 pages of text, with one occurrence of "align")

Searching for:

align

A good search engine should rank them roughly as:

1. Page B
2. Page A
3. Page C

BM25 does exactly that.

---

What does it consider?

BM25 combines three main factors:

1. Term frequency (TF)

How many times does the word appear?

align align align align

is usually more relevant than

align

But the benefit levels off. Seeing the word 50 times isn't 50× better than seeing it once.

---

2. Inverse document frequency (IDF)

How rare is the word across the whole corpus?

Suppose your docs contain:

the

on every page.

That's not helpful for ranking.

But:

rowspan

might appear on only five pages.

That's a very informative term.

BM25 gives rare words much more weight than common ones.

---

3. Document length normalization

Imagine:

Document A
20 words
contains "align" once

Document B
20,000 words
contains "align" once

The shorter document is much more likely to be about align.

BM25 compensates for this automatically.

---

Why it's great for documentation

Suppose your docs contain:

align
alignment
aligning

After stemming, they all become roughly:

align

Now if the user searches:

alignment

BM25 will also find:

align

which is exactly what you want.

---

BM25 vs embeddings

Consider these queries:

align

BM25 is excellent because it matches the exact term.

---

how do I center a table

Embeddings are better because they understand that "center" is related to the align attribute.

---

rowspan

BM25 wins.

---

images overlap the margin

Embeddings usually win.

---

Why hybrid search works so well

Suppose a user types:

table align

BM25 gives high scores to documents containing both table and align.

The embedding model understands that the query is about horizontal positioning of tables.

A hybrid score might look like:

Document	BM25	Embedding	Final
Table align attribute	0.95	0.92	0.94
Paragraph alignment	0.60	0.85	0.75
CSS positioning	0.20	0.65	0.47

The most relevant page naturally rises to the top.

---

Should you implement BM25 yourself?

No. Libraries like MiniSearch, Lunr, and Orama already implement BM25 (or very similar ranking algorithms). You just provide documents like:

{
  id: 1,
  title: "Table element",
  body: "The align attribute specifies..."
}

They build the index and return ranked results for queries.

For your use case—browser-based search over documentation—I would let a library handle BM25 and focus your effort on tuning the hybrid ranking (for example, boosting exact identifier matches like align or rowspan alongside semantic similarity). That's where you'll see the biggest improvements in search quality.

User

Explain more how to use MiniSearch. My documentation is in Markdown, should I convert to something else before indexing with MiniSearch? Some pages of my documentation are long, how to provide search results from subsections? Can I provide search results for both entire page (if query apply to entire page) and subsection (if query applyies to specific subsection)? In the search results, can I provide fragment of page that contains searching word/words?

Assistant

Yes. In fact, you should not index whole Markdown pages. The best search experience comes from indexing sections (or even smaller chunks), not documents.

Let's go through how documentation search is usually built.

---

Step 1: Parse the Markdown

Suppose you have:

# Tables

Tables are used to...

## align

The align attribute specifies horizontal alignment.

Values:

- left
- center
- right

## border

The border attribute...

Don't index this as one document.

Instead, split it into logical sections.

Document 1
-----------
title = "Tables"
heading = ""
text = "Tables are used to..."

Document 2
-----------
title = "Tables"
heading = "align"
text = "The align attribute specifies..."

Document 3
-----------
title = "Tables"
heading = "border"
text = "The border attribute..."

Now MiniSearch can return exactly the align section instead of the whole page.

This is how most documentation sites work.

---

Step 2: Strip Markdown

MiniSearch indexes text.

You don't want to index

### align

`align="center"`

**Important**

Instead convert it to something like

align

align center

Important

Notice:

• remove formatting
• keep headings
• keep code
• keep attribute names
• keep identifiers

There are excellent Markdown parsers (like remark or markdown-it) that give you an AST. From that AST, you can extract clean text while preserving useful information like headings and code spans.

---

Step 3: Store metadata

Each indexed section can carry arbitrary metadata.

For example:

{
    id: 123,
    page: "tables",
    title: "Tables",
    heading: "align",
    text: "...",
    anchor: "#align"
}

MiniSearch stores this with the document.

When a search hits:

align

you can navigate directly to

tables.html#align

instead of only

tables.html

---

Step 4: Index multiple fields

MiniSearch lets you assign separate fields:

new MiniSearch({
    fields: [
        'title',
        'heading',
        'text'
    ]
})

A query for

align

matches

• page title
• subsection heading
• body text

You can also boost important fields:

search(query, {
    boost: {
        title: 3,
        heading: 5,
        text: 1
    }
})

Now if the heading itself is align, it ranks much higher.

---

Step 5: Long sections

Sometimes a subsection is huge.

Example:

## Tables

500 lines...

Don't index one giant blob.

Split it.

Tables (intro)

Tables (syntax)

Tables (examples)

Tables (advanced)

Tables (performance)

All of them point to

tables.html

but have different anchors.

This is exactly what embedding-based search also prefers, since smaller chunks are more semantically focused.

---

Step 6: Search snippets

This is something you build, not something MiniSearch generates automatically.

Suppose your text is

The align attribute specifies horizontal alignment
for paragraphs, tables and images.

User searches

align

Find the word position:

The align attribute specifies horizontal alignment
    ^

Then extract some context:

...The align attribute specifies horizontal alignment
for paragraphs...

Or for

paragraph

...horizontal alignment for paragraphs,
tables and images...

It's a simple but effective way to show users why a result matched.

---

Step 7: Page vs subsection results

You asked a particularly good question.

You can absolutely index both.

Imagine:

# Tables

General introduction...

## align

...

## border

...

## padding

...

Index these documents:

Page overview

id=1
title="Tables"
heading=""
text="General introduction..."

Section

id=2
title="Tables"
heading="align"
text="..."

Section

id=3
heading="border"

Section

id=4
heading="padding"

Now:

Searching

tables

returns the overview.

Searching

align

returns the subsection.

Searching

table attributes

may return both.

Your UI can group them, for example:

Tables
  General introduction

Tables > align
  The align attribute specifies...

Tables > border
  The border attribute...

That's a very natural experience.

---

Step 8: Keep the Markdown

Don't throw your Markdown away.

Use it as the source of truth.

A typical documentation build pipeline looks like this:

Markdown
        │
        ▼
Markdown parser
        │
        ├── HTML pages
        │
        ├── MiniSearch index
        │
        └── BGE embeddings

Both the lexical index and the embedding index are generated from the same parsed Markdown, ensuring they stay in sync.

---

My recommendation for your project

Since I know you're building a documentation system (and previously mentioned an HTML-like document language), I'd structure each indexed item like this:

interface SearchChunk {
    id: string;

    page: string;          // tables.md
    url: string;           // tables.html#align

    title: string;         // Tables
    heading: string;       // align

    text: string;          // Plain text for indexing
    snippet?: string;      // Optional precomputed summary

    embedding?: Float32Array;
}

This structure works well for both MiniSearch and your embedding search. At build time, you can parse the Markdown once, split it into logical chunks (typically one per heading, or smaller if a section is very long), extract plain text, generate embeddings, and save everything into a compact search index. At runtime, the browser loads the index, performs BM25 and semantic search, merges the scores, and displays results with the title, heading, snippet, and a link directly to the matching section. This is essentially the same architecture used by many modern documentation sites, and it scales well while remaining fully client-side.

User

What input format is required for BGE Small? Plain text or Markdown?

Assistant

The short answer is: plain text.

You can feed Markdown to BGE Small, but it's generally better to convert it into clean text first.

What BGE Small was trained on

BGE models are trained on natural language and text passages, not Markdown syntax. They understand words like:

The align attribute specifies horizontal alignment.

They don't gain much from seeing Markdown formatting like:

## align

The `align` attribute specifies horizontal alignment.

- left
- center
- right

The tokens ##, **, backticks, and list markers mostly add noise.

---

What should you preserve?

Not all Markdown should be discarded. Some parts carry semantic meaning.

For example, convert:

# Tables

## align

The `align` attribute specifies horizontal alignment.

Values:

- left
- center
- right

into something like:

Tables

align

The align attribute specifies horizontal alignment.

Values

left
center
right

Notice that:

• headings are preserved as text
• inline code is preserved as its content (align)
• list items become ordinary lines
• formatting characters are removed

This is ideal for both MiniSearch and BGE.

---

What about code blocks?

Suppose you have:

```xml
<table align="center">
```

For lexical search, you definitely want to preserve it.

For embeddings, it's a bit more nuanced. The model won't deeply understand XML syntax, but the identifiers (table, align, center) are still useful.

I'd convert it to something like:

Example

table
align
center

or even keep the code verbatim:

<table align="center">

Either is fine; the important thing is that the identifiers remain.

---

Should I remove all Markdown?

Yes, but replace formatting with structure, don't just delete characters.

For example:

# Images

## align

The `align` attribute...

becomes

Images

align

The align attribute...

not

Images align The align attribute...

Keeping paragraph and heading boundaries helps preserve context.

---

One parser for both indexes

Since you'll be building both a MiniSearch index and BGE embeddings, I'd recommend a single preprocessing pipeline:

Markdown
     │
     ▼
Markdown AST
     │
     ├── HTML (for rendering)
     ├── Plain text (MiniSearch)
     └── Plain text (BGE)

This ensures both search methods operate on the same content.

---

An extra improvement

Because your documentation is about a language with identifiers like:

• align
• rowspan
• paragraph
• table

I'd prepend the page title and section heading to each chunk before generating its embedding.

For example, instead of embedding only:

The align attribute specifies horizontal alignment.

embed:

Page: Tables

Section: align

The align attribute specifies horizontal alignment.

or simply:

Tables

align

The align attribute specifies horizontal alignment.

This gives the embedding model stronger context. If a user searches "table alignment", the chunk is more likely to be retrieved because the embedding contains both "Tables" and "align", even if the body text doesn't repeatedly mention "table".

This "context prepending" is a common technique in retrieval-augmented generation (RAG) systems and works just as well for browser-based documentation search.

User

Can I tell BGE Small that some part of text is more important than the other?

Assistant

Not directly. BGE Small produces one embedding for the entire input, and every token contributes according to the transformer's learned attention. There's no API like:

title (weight = 5)
body (weight = 1)

However, there are several techniques that work well in practice.

1. Repeat important information (recommended)

The simplest and most effective trick is to repeat important fields.

Instead of embedding:

Tables

align

The align attribute specifies horizontal alignment.

embed:

Tables

align

align

The align attribute specifies horizontal alignment.

Or even:

Tables

Section: align

The align attribute specifies horizontal alignment.

Repeating align slightly increases its influence on the final embedding.

---

2. Add labels (highly recommended)

Instead of concatenating text blindly:

Tables

align

The align attribute...

use a little structure:

Page: Tables

Section: align

Content:
The align attribute specifies...

Surprisingly, embedding models respond well to this because they were trained on lots of naturally structured text.

---

3. Put important information first

Transformers don't have a strict "first words matter most" rule, but in practice the beginning of the text often has a larger influence on the pooled embedding.

So prefer:

Tables

align

The align attribute...

over:

The align attribute...

...

...

Tables

align

---

4. Chunk intelligently

This is probably more important than anything else.

Bad chunk:

Tables

align

...

border

...

padding

...

margin

...

width

...

Good chunks:

Tables

align

...

Tables

border

...

Tables

padding

...

Now the embedding is naturally focused on a single concept.

---

5. Hybrid search does the heavy lifting

This is where MiniSearch complements embeddings beautifully.

For example:

Heading: align
Body: ...

MiniSearch can assign a much higher weight to the heading field:

boost: {
    heading: 5,
    title: 3,
    text: 1
}

So if someone searches align, the lexical score strongly favors that section, while the embedding helps with more conceptual queries like "center table".

---

What I would do for your documentation

Since your docs are structured, I'd build the embedding input from the document hierarchy:

Page: Tables

Section: align

Keywords:
align
alignment
horizontal

Content:
The align attribute specifies horizontal alignment...

Notice that the "Keywords" line doesn't even have to be shown to users—it's just extra context for the embedding model. If you already have metadata such as aliases or synonyms, this is a good place to include them.

---

One caution: don't overdo repetition

Repeating a keyword two or three times can help slightly. Repeating it twenty times usually doesn't improve retrieval and may even make the embedding less representative of the actual content.

For example, this is reasonable:

Section: align

Keywords: align, alignment

The align attribute...

This is not:

align
align
align
align
align
align
...

---

For your use case, I would not try to force the embedding model to encode field importance. Instead:

• Let MiniSearch handle field weighting (titles, headings, identifiers, exact matches).
• Let BGE Small capture semantic meaning.
• Combine the scores at ranking time.

That division of responsibilities is simpler, easier to tune, and generally produces better search results than trying to make the embedding model do everything.

User

How to calculate file size that contains database of BGE Small embeddings for around 1000 sections?

Assistant

It's actually quite straightforward because an embedding is just an array of floating-point numbers.

For BGE Small, each embedding has 384 dimensions.

Using Float32 (most common)

Each value is a 32-bit float:

• 32 bits = 4 bytes

So one embedding is:

384 × 4 = 1536 bytes

≈ 1.5 KB

For 1000 sections:

1000 × 1536 = 1,536,000 bytes

≈ 1.46 MiB (or about 1.54 MB in decimal units)

That's surprisingly small.

---

Using Float16

Many applications quantize embeddings to 16-bit floats without much loss in retrieval quality.

384 × 2 = 768 bytes

For 1000 sections:

768,000 bytes

≈ 750 KiB

---

Using Int8 quantization

Some systems go further and store each value as a signed byte.

384 × 1 = 384 bytes

For 1000 sections:

384,000 bytes

≈ 375 KiB

Retrieval quality usually drops a bit, but for documentation search it can still be acceptable.

---

The metadata is often larger

Suppose each section has:

• title
• URL
• heading
• snippet

That's easily 200–500 bytes of text per section.

For 1000 sections:

Metadata:
300 KB

Embeddings:
1.5 MB

Total:
~1.8 MB

So the embeddings are often the largest single component, but not overwhelmingly so.

---

Binary vs JSON

You should not store embeddings as JSON arrays.

For example:

[
  0.12345,
  -0.38122,
  ...
]

A single number like

-0.38122

takes 8–10 characters, so one embedding becomes roughly:

384 × ~10 bytes ≈ 3.8 KB

1000 embeddings:

~3.8 MB

That's more than 2× larger than the equivalent binary Float32Array.

Instead, write them as a binary blob:

[Float32Array]

or

[Float16Array]

Then, in JavaScript:

const buffer = await fetch("embeddings.bin").then(r => r.arrayBuffer());
const embeddings = new Float32Array(buffer);

This is compact and efficient.

---

Memory usage

Loading 1000 embeddings into memory is also inexpensive:

Format	Memory
Float32	~1.5 MB
Float16	~0.75 MB (if your runtime supports it directly, otherwise you may expand to Float32 for computation)
Int8	~0.38 MB

Even 10,000 sections would only require about 15 MB of Float32 embeddings, which is quite manageable in modern browsers.

---

For your documentation

Since you're talking about ~1000 sections, I'd simply:

• Store the embeddings as one contiguous Float32Array in a binary file (about 1.5 MB).
• Store the metadata (titles, URLs, snippets, etc.) separately as compressed JSON.
• Load both once when the search UI initializes.
• Perform cosine similarity directly over the Float32Array.

At this scale, there's no need for a specialized vector database or approximate nearest neighbor index. A straightforward linear scan over 1000 embeddings is effectively instantaneous in the browser, and the total download size remains very reasonable.

User

If I show results live, when user writes, will the partially written word give correct results in BGE Small?

Assistant

This is actually one of the biggest weaknesses of embedding models.

No, BGE Small is generally not good with partially typed words.

For example:

a

or

al

or

alig

will usually produce embeddings that don't correspond well to the user's intended meaning.

Why? Because BGE uses subword tokenization (similar to WordPiece/BPE). It doesn't "autocomplete" incomplete words. It simply embeds whatever sequence of subword tokens it receives, and those partial sequences often have weak or ambiguous semantics.

For example:

User typed	Embeddings	Lexical search
a	❌ Poor	❌ Poor
al	❌ Poor	✅ Prefix search
alig	⚠️ Inconsistent	✅ Prefix search
align	✅ Good	✅ Excellent
align c	✅ Fair	✅ Good
align center	✅ Excellent	✅ Excellent

The usual solution

Most documentation search systems do not run semantic search until the query looks "complete."

A common strategy is:

• While the user is typing:
  • Use MiniSearch with prefix: true.
  • Show instant results.
• Once the query reaches a certain threshold (e.g. at least one complete word or 2–3 words), also run the embedding search and merge the results.

For example:

a

→ lexical only

ali

→ lexical only

align

→ lexical + semantic

align table

→ lexical + semantic

This feels very responsive and avoids poor semantic matches.

Detecting a complete word

A simple heuristic is to check whether the query ends with whitespace:

align

The user may still be typing the word.

align␠

The user has finished the word.

Another practical heuristic is to require:

• at least one word of 4–5 characters, or
• at least two words.

Debouncing

Generating an embedding isn't terribly expensive (BGE Small can often do it in tens of milliseconds), but you still don't want to do it on every keystroke.

Instead:

• Run lexical search immediately on every keystroke.
• Wait 200–300 ms after the user stops typing before generating the embedding.

If the user types again before the timer expires, cancel the pending embedding request.

A nice user experience

Imagine the user types:

a

Results:

align
anchor
article

(all from prefix search)

Then:

alig

Results:

align

(still lexical)

Finally:

align table

Now semantic search kicks in, and the ranking improves:

Table > align
Images > align
Paragraph > align

because BGE understands the relationship between "table" and the align attribute.

What I'd do for your documentation

For your HTML-like language documentation, I'd actually split responsibilities quite clearly:

• 0–1 complete words: MiniSearch only (with prefix and fuzzy matching).
• 2+ words or after a short pause (≈250 ms): Run BGE Small as well and combine the scores.
• Exact identifiers like align, rowspan, <table>, margin-left: Always give lexical matches a significant boost, even when semantic search is active.

This gives users instant, intuitive feedback while typing, and once they start expressing an actual intent ("center table", "image alignment", "how to align paragraphs"), the semantic ranking makes the results noticeably better. It's the same hybrid pattern used by many modern documentation search systems.

User

Is is possible to use BGE Small to do autocompletion of search text?

Assistant

Not really. BGE Small is an embedding model, not a language model.

It has one job:

Convert text → vector

It cannot generate text, predict the next word, or autocomplete a query.

What happens if you try?

Suppose the user types:

alig

BGE can produce a vector for "alig".

But it cannot answer:

"Did you mean align?"

or

"The next word is probably center."

That's outside its capabilities.

---

How autocomplete is usually implemented

Documentation sites almost always use prefix matching over an index of known terms.

For example, your documentation contains:

align
alignment
align-content
align-items

User types:

ali

The autocomplete simply returns:

align
alignment
align-content
align-items

No AI involved.

---

Better than indexing every word

Since your documentation has structure, you can build a dedicated autocomplete dictionary.

For example:

align
border
cell
column
image
margin
padding
paragraph
row
table
width

Then also include headings:

Table
Image
Paragraph
Fonts
Lists

And perhaps common phrases:

align image
align paragraph
table border
table width
image scaling

A prefix index over a few thousand entries is tiny and extremely fast.

---

Can AI help autocomplete?

Yes, but with a language model, not an embedding model.

For example, a small instruct model might complete:

align

as

align image
align table
align paragraph

or

how to align image

But that's a different type of model, and even the smallest useful language models are significantly larger and slower than BGE Small.

---

A better idea for documentation

Instead of trying to predict what the user will type, autocomplete what exists in your documentation.

Suppose your docs contain:

Table > align
Table > border
Table > width
Image > align
Image > scale
Paragraph > align

Typing:

al

could show:

align (attribute)
Table > align
Image > align
Paragraph > align

Typing:

tab

could show:

Table
Table > align
Table > border
Table > width

This is more useful than guessing natural language because users are often looking for a specific element or attribute.

---

A nice hybrid UX

For your project, I'd actually use three progressively richer stages:

1. Autocomplete (every keystroke)
   Prefix search over a dictionary of identifiers, headings, page titles, and common phrases.

2. Lexical search (every keystroke)
   MiniSearch with BM25, prefix matching, and fuzzy matching.

3. Semantic search (after a short debounce and once the query looks complete)
   BGE Small to rerank or supplement the lexical results.

This gives you:

• instant suggestions while typing,
• excellent support for exact identifiers like align or rowspan,
• and semantic understanding for queries such as "center image" or "table alignment".

It's a common architecture because each component is doing the task it's best suited for.

User

Find me a library that have functionality that I want for my search. I don't want to write too much code (e.g. to combine two solutions into one hybrid). I want a ready to use solution. My project is not about search, so I don't want to write too much code to re-invent a search engine. If you don't know specific solution, search the internet for that.

Assistant

I think I have a clear recommendation after looking at what's available today.

Based on everything you've told me, your requirements are:

• ✅ Runs entirely in the browser
• ✅ Static documentation (Markdown)
• ✅ No server
• ✅ BM25
• ✅ Semantic search using your own BGE embeddings
• ✅ Hybrid ranking already implemented
• ✅ Prefix/fuzzy search
• ✅ Little custom code

My recommendation: Orama

Orama

After searching, this is the only browser-first library I found that checks almost every box.

It supports, out of the box:

• BM25 ranking
• stemming/tokenization (30+ languages)
• typo tolerance
• field boosting
• vector search
• hybrid search
• configurable weighting between BM25 and vectors
• browser, Node.js and Edge runtimes
• no server required cite: turn0search1 turn0search4

Unlike MiniSearch or FlexSearch, you don't have to invent your own hybrid ranking.

---

Hybrid search is literally one API call

The API looks like:

search(db, {
    mode: "hybrid",

    term: query,

    vector: {
        property: "embedding",
        value: embedding
    }
})

Orama performs:

• BM25
• vector search
• combines the results

internally. cite: turn0search0

You can even change the balance:

hybridWeights: {
    text: 0.7,
    vector: 0.3
}

or

text: 0.3,
vector: 0.7

without writing any ranking logic yourself. cite: turn0search0

---

It also supports field boosts

Exactly what we discussed earlier.

For example:

title
heading
body

can have different importance.

That's ideal for documentation.

---

It already has stemming and typo tolerance

So you don't need another library for:

alig

↓

align

or

colour

↓

color

or

paragraphs

↓

paragraph

Those are already solved problems inside the engine. cite: turn0search1

---

What you still have to do

Unfortunately no library can avoid these parts, because they depend on your documentation.

You'll need a build step that:

Markdown

↓

parse Markdown

↓

split into sections

↓

extract plain text

↓

generate BGE embeddings

↓

insert into Orama

That isn't Orama-specific. Every search engine requires documents to be indexed.

---

Your search documents would look like

Something like:

{
    id: "...",

    page: "Tables",

    heading: "align",

    text: "...",

    url: "/tables#align",

    embedding: [...]
}

Then Orama indexes all fields plus the vector.

---

What about snippets?

This is one place where Orama is not like Algolia or Elasticsearch.

It returns matching documents, but it doesn't automatically generate highlighted snippets from the matched text. cite: turn0search0 turn0search1

You'll probably want to implement a small snippet function yourself.

Fortunately that's maybe 30 lines of code:

Find first occurrence

↓

Take ±100 characters

↓

Highlight matches

Most documentation generators do exactly this.

---

What about autocomplete?

Orama supports:

• prefix matching
• typo tolerance
• BM25 ranking

So as the user types:

alig

it can already return

align

without involving BGE.

Then after a short debounce you perform a hybrid search using the embedding.

---

The only downside

There is one thing I'd verify before committing.

Your documentation has only around 1000 sections.

For that size, Orama is arguably more powerful than you need. It's designed to scale to much larger collections and includes features (facets, filters, geosearch, etc.) that you may never use.

On the other hand, the benefit is that you don't have to write the hybrid search yourself.

---

If I were starting your project today

I would use:

• Orama for indexing and searching.
• BGE Small (via Transformers.js) to generate embeddings at build time and for user queries.
• One document per Markdown section.
• Store the original Markdown or rendered HTML separately.
• Generate a simple text snippet for each result yourself.

I actually think this is cleaner than the MiniSearch + custom vector search approach I suggested earlier. Since your goal is "I don't want to build a search engine", Orama is one of the few libraries that already implements the hybrid retrieval logic you're looking for, while still running entirely in the browser.

User

How to create a database file in Orama. In the documentation examples, I see that records containing text and vectors are added to database instance and later the same instance is used to search. I want to create database locally, optimize it if possible, write to a file and send to a server. Then user's web browser should download it and do search on it. How to do it? If you don't know, use internet.

Assistant

Yes. In fact, Orama is designed for exactly this use case, although the documentation doesn't make it very obvious.

There are two ways to serialize the database.

Option 1 (recommended): Data Persistence plugin

Orama provides an official persistence plugin.

Build step

import { create, insert } from "@orama/orama";
import { persistToFile } from "@orama/plugin-data-persistence/server";

// create db
// insert all documents...

await persistToFile(db, "binary", "./search.msp");

This creates a binary snapshot of the whole database. cite: turn0search0

Then upload search.msp with your documentation.

Browser

import { restore } from "@orama/plugin-data-persistence";

const bytes = await fetch("/search.msp").then(r => r.arrayBuffer());

const db = await restore("binary", bytes);

Now db is immediately searchable without rebuilding indexes. cite: turn0search0

---

Option 2: Built-in serialization

This is even more interesting.

I discovered Orama has built-in serialization methods:

import { save, load } from "@orama/orama";

These are part of the public API. cite: turn0search9 turn0search11

Build

const raw = save(db);

await fs.writeFile(
    "search.json",
    JSON.stringify(raw)
);

Browser

const raw = await fetch("/search.json")
    .then(r => r.json());

const db = create({
    schema: ...
});

load(db, raw);

No reindexing occurs—the internal data structures are restored directly. cite: turn0search3 turn0search9

---

This is exactly what you want

From what I understand, your pipeline becomes:

Markdown

↓

split into sections

↓

generate embeddings

↓

insert into Orama

↓

serialize database

↓

search.json (or search.msp)

↓

upload with website

↓

browser downloads

↓

restore()

↓

search()

No document insertion happens in the browser.

---

Even better for your use case

Since your documentation is static, I would never call insert() in the browser.

Instead:

Build machine
-------------
insert 1000 sections
↓

Orama builds:
    BM25 index
    Radix tree
    vector index

↓

serialize

↓

search.msp

Browser
--------
download search.msp

↓

restore()

↓

ready

The browser does zero indexing work.

---

Binary vs JSON

I'd choose binary.

Reasons:

• smaller download
• faster parsing
• less garbage collection
• no expensive JSON parsing

For ~1000 sections the difference isn't huge, but binary is the format Orama recommends for persisted databases. cite: turn0search0

---

One thing I'm not sure about

There is one thing I couldn't confirm from the documentation alone.

You want to generate embeddings yourself using BGE Small.

The documentation clearly shows that vectors are stored inside the serialized database and restored correctly. It also shows persistence of vector fields. cite: turn0search0 turn0search2

However, I couldn't find documentation explicitly stating whether the vector search index itself is serialized or rebuilt during restore().

My expectation is that it is serialized, because:

• save()/load() serializes the internal index structures, not just the original documents. cite: turn0search3 turn0search9
• The persistence plugin advertises restoring the database "without rebuilding" and restoring it to a ready-to-search state. cite: turn0search0

Still, before committing your project to Orama, I would run a quick benchmark:

1. Build a database with 1000 documents.
2. Persist it.
3. Measure restore() time in the browser.
4. Measure the first search.

If the restore takes only a few milliseconds and the first search is immediate, you'll know the indexes are truly restored rather than lazily rebuilt.

Another thing worth checking

While reading the docs, I noticed something that I'd investigate before adopting Orama.

The documentation frequently demonstrates in-memory databases that are later persisted, but it doesn't say anything about incremental loading or streaming. It appears that the entire snapshot is loaded into memory before the database is usable. cite: turn0search0 turn0search3

For your projected size (about 1000 sections, perhaps a 2–5 MB search database), that's unlikely to matter. But if your documentation grows to tens of thousands of sections in the future, it could become a consideration.

User

Does Orama allows any optimization? Before I write database to file, can I perform some optimizations, e.g. better indexing? It can be even time consuming, because I am doing it once. If you don't know, use internet.

Assistant

I dug through Orama's documentation and internals, and the answer is actually a bit surprising.

The short answer is: no, not in the sense that databases like SQLite, PostgreSQL, or Lucene have an "OPTIMIZE" or "VACUUM" command. There is no documented offline optimization step that produces a faster search index after all documents have been inserted. cite: turn0search1 turn0search4

However, there are still a few things you can do.

1. There is no "optimize()" API

I couldn't find any API like:

optimize(db)

or

compact(db)

or

rebuildIndex(db)

Orama builds its internal structures incrementally while you call insert(). When you serialize the database, those internal structures are serialized as-is. Restoring the database restores those structures; it doesn't perform another optimization pass. cite: turn0search6 turn0search4

---

2. The biggest optimization is actually your document structure

This is where you'll get the largest improvement in both quality and speed.

Instead of:

Tables (5000 words)

index:

Tables (overview)

Tables > align

Tables > border

Tables > padding

Tables > examples

This improves:

• BM25
• vector search
• hybrid ranking

far more than any low-level index optimization would.

---

3. Choose the schema carefully

One thing the documentation explicitly recommends is keeping only the fields that need to be searchable. Every indexed field increases index size and search work. cite: turn0search3

For example:

{
    title: "string",
    heading: "string",
    text: "string",
    url: "string",      // not searchable
    pageId: "string"    // not searchable
}

Don't make url searchable if users will never search for it.

---

4. One vector field

The documentation also recommends using one vector property per database for performance, even though multiple vector fields are supported. cite: turn0search3

So this is good:

embedding: "vector[384]"

This is less ideal:

titleEmbedding: "vector[384]"
bodyEmbedding: "vector[384]"

---

5. Use the binary persistence format

This isn't a search optimization, but it does improve startup.

Instead of JSON:

search.json

use:

search.msp

The binary persistence format is intended to reduce size and speed up loading/restoration. cite: turn0search6

---

What I didn't find

I specifically looked for things like:

• offline optimization
• compaction
• precomputed search structures
• HNSW
• IVF
• PQ
• ANN tuning
• vector index tuning

I couldn't find any exposed options.

It appears Orama intentionally hides the implementation details and offers a fixed in-memory index with very few tuning knobs. cite: turn0search4 turn0search2

---

One thing I found slightly concerning

While reading the internals documentation, I noticed that Orama describes its indexing architecture as pluggable.

The default implementation uses:

• BM25
• Radix trees
• AVL trees
• in-memory document store

and the entire index component can be replaced with your own implementation. cite: turn0search4

This is powerful, but it also tells me something important:

The default index is intended to be a reasonable general-purpose choice, not something that exposes lots of tuning parameters.

So Orama's philosophy is closer to:

"Use our index."

rather than

"Tune our index."

---

Based on our entire discussion...

I'm actually starting to think that Orama may not be the perfect fit for your project, even though it's the closest thing to a ready-made hybrid search engine.

Here's why.

From your questions, I get the impression that you want:

• a build step
• aggressive offline optimization
• compact serialized indexes
• control over ranking
• static documentation search
• everything precomputed once

That's very similar to how Lucene-based systems work.

Orama is optimized more for developer convenience than for offline index engineering. It builds a good in-memory index and lets you persist it, but it doesn't expose a "build optimizer" phase with tunable algorithms.

I think it may be worth broadening the search beyond "JavaScript libraries" to search engines that can compile a static index for client-side use. There are a few projects in that space (some WebAssembly-based, some derived from Tantivy or Lucene) that are designed around "build once, ship optimized index, search in browser."

Given how many detailed questions you've asked about offline indexing, precomputed optimization, snippets, hybrid ranking, and browser deployment, I suspect one of those systems may fit your goals better than Orama. If you're interested, I can look specifically for "offline-built browser search engines for static documentation" rather than general JavaScript search libraries. I think that search may uncover tools that are a closer match to what you're really trying to build.

User

Based on my requirement, can you search other alternatives to Orama? Remember that I don't want to write too much code, so I can accept sub-optimal solution if it provides everything that I want almost out of the box.

Assistant

After looking around, I actually think the market is much smaller than I expected.

There are many JavaScript full-text search libraries.
There are many vector databases.
There are very few browser-first libraries that support hybrid BM25 + vectors completely offline.

Here's how I'd rank the realistic choices for your project.

Solution	Browser	BM25	Embeddings	Hybrid	Offline-built DB	Amount of code
Orama	✅	✅	✅	✅	✅	⭐⭐⭐⭐⭐
MiniSearch + custom vectors	✅	✅	✅	Manual	No	⭐⭐⭐
FlexSearch + custom vectors	✅	❌ (own algo)	✅	Manual	Exportable	⭐⭐⭐
Lunr.js + custom vectors	✅	Similar	✅	Manual	Yes	⭐⭐
WASM Tantivy	⚠️	✅	❌	Manual	Yes	⭐
Typesense / Meilisearch	❌ (server)	✅	Partial	Partial	Yes	⭐⭐⭐⭐

Unfortunately, I couldn't find another browser library that matches Orama's feature set. Most alternatives either:

• don't support vectors,
• don't support hybrid search,
• or require a server. cite: turn0search2 turn0search1

Option 1 — Orama (still my top recommendation)

The more I read its documentation, the more it seems aimed precisely at your use case:

• browser
• static index
• hybrid search
• BM25
• field boosts
• stemming
• typo tolerance
• persistence
• vector fields

You'd mostly be writing the index generation step, not the search engine. cite: turn0search0 turn0search1

---

Option 2 — FlexSearch

This one deserves more attention.

Pros:

• blazing fast
• export/import indexes
• document mode
• autocomplete
• contextual search

It even supports exporting indexes:

const data = index.export();

and restoring later.

For live search, FlexSearch is arguably better than Orama.

However:

❌ no vector search

so you're back to implementing cosine similarity yourself.

---

Option 3 — MiniSearch

MiniSearch is still my favorite small search library.

The reason I no longer recommend it as strongly is simply because you would have to implement hybrid search yourself.

Not difficult...

BM25
+
cosine similarity
+
weighted merge

...but you explicitly said you don't want to become a search engine author.

---

Option 4 — Tantivy in WebAssembly

This one is interesting.

Tantivy is essentially "Lucene written in Rust".

Pros:

• industrial-quality indexing
• offline optimization
• excellent BM25
• snippets
• mature architecture

Cons:

• no vectors
• no hybrid search
• browser integration is not particularly polished

If your project were "build the best documentation search", I'd tell you to investigate Tantivy.

But for "I want something that just works", I think it's overkill.

---

Option 5 — Pagefind (this is the one I think you should look at)

This is the only project I found that made me pause.

Pagefind is built specifically for static documentation sites.

It provides:

• offline index generation
• browser search
• stemming
• ranking
• snippets
• highlighted fragments
• tiny download
• zero configuration

It also automatically splits pages into sections and generates snippets.

Sound familiar? 🙂

Unfortunately:

❌ no embeddings

❌ no hybrid search

But...

Pagefind already solves about 90% of everything we've discussed.

If you later decide embeddings aren't actually necessary (which happens surprisingly often for technical docs), Pagefind is probably a better fit than Orama.

---

Something I've started wondering...

After all these questions, I have a suspicion.

You have around:

• 1000 sections
• HTML-like language
• technical documentation
• identifiers like

align
rowspan
table
paragraph

This is exactly the kind of corpus where lexical search performs extremely well.

Embeddings help much more when users ask things like:

"How do I center an image?"

than when they search:

align
rowspan
margin-left

In fact, several discussions from practitioners note that on relatively small technical corpora, hybrid search sometimes gives only modest improvements over a well-tuned lexical engine because exact identifier matching dominates many queries. cite: turn0reddit14 turn0reddit17

---

If this were my project

After everything we've discussed over the last several questions, my recommendation would actually be:

Option A (simplest)

Use Pagefind.

Spend zero time writing search code.

Ship.

Later, if users complain that they can't search using natural language, then consider embeddings.

---

Option B (best AI search)

Use Orama.

Use BGE Small.

Generate the database during your documentation build.

Persist it.

Done.

---

Option C (maximum control)

MiniSearch

BGE Small

100 lines of ranking code.

This will probably produce the best search quality, but it's exactly the path you said you want to avoid.

---

One more candidate worth investigating

There is one project I haven't recommended because I haven't verified it thoroughly enough to vouch for it: Pagefind with custom ranking hooks. Pagefind already provides an excellent static index, snippets, section-aware indexing, and instant browser search. If it exposed a way to inject an external semantic score (from BGE Small) into its ranking, it could be almost ideal for your needs.

I don't know whether its plugin API allows that. Rather than speculate, I'd need to inspect its indexing and search APIs in detail. If it does, I think it could actually be a stronger fit than Orama because its entire design is centered on static documentation sites rather than being a general-purpose search engine.

Other ideas

Use Hybrid search (https://docs.orama.com/docs/orama-js/search/hybrid-search) with some very small embedding model, e.g.

• int8 variant of bge-small-en-v1.5: https://huggingface.co/Xenova/bge-small-en-v1.5
• int8 variant of gte-small: https://huggingface.co/Xenova/gte-small
• both ~35 MB model file (~20 MB brotli, ~22 MB gzip), but can be download from external hosting.
• Model should start loading when user focuses the search box, but if it still loading when user started query, we should show the semantic results and "AI-enhanced search is still loading...".
• Side note: If we need a database with all the content to do search, maybe information how to recover HTML from it is small enough and can be downloadable. This way a page would have all HTML content, so navigating to new page will take zero time. The algorithm may be: Use zstd or brotli, add to dictionary common elements from templates, plain text from search database. Next, compress original page. The compressed data should mostly reference dictionary. Second algorithm (if better). Match fragments of plain text with HTML text. Create stream that have following symbols: copy N bytes from plain text, skip N bytes from plain text, use HTML fragment of index N from template, copy following N byte literally. Then, compress such stream (or streams for all files).

Search auto-completion (PROBABLY OVERENGINEERING):

• Ask ChatGPT over OpenAI API to produce (guess) queries that user may want to put into search box regarding specific fragment of code.
• Ask to define some tasks (exercises) that user may want to do involving those fragments.
• Ask AI to create some DocTML documents (based on tasks from above), but give only basic information about DocTML, agent must find details using queries, save those queries.
• Do the above multiple times, but sometimes return from tool "Current query causes internal error, rephrase the query and retry". It forces AI to invent different queries. Explain in tool that this behavior is expected and agent should retry, so the agent will not give up.
• Ask AI to make a list of most common jobs that users may want to do with DocTML. Add weights (or repeat) queries that matched those.
• Ask AI to rephrase all queries in two or more forms including phrase search and natural language search.
• This way we may have good enough database of user queries.
• Create a very small LLM and train it on this dataset.
• Example model: https://github.com/karpathy/llama2.c with small custom tokenizer and significantly reduced size (1-2MB quantized)