laghu

model2vec makes static embedding models. A static embedding model changes a text into a vector. It splits the text into tokens. It finds a stored vector for each token. Then it calculates the average of these vectors. No neural network runs, so these models are very fast. Each model has two main parts: an embedding table with one row for each token, and a tokenizer that splits text into tokens.

laghu (लघु, Sanskrit for “light”) loads the same models with much less memory. It gives the same vectors as model2vec, bit for bit. A model that laghu loads has the same encode, encode_as_sequence and tokenize methods as a model2vec model. The section What is not supported gives the one difference.

Where the memory goes

A process is one running program. Private memory is memory that only one process can use. model2vec copies two large parts of the model into the private memory of each process. laghu shares these parts between processes.

Part of potion-multilingual-128M	model2vec	laghu
Embedding table	512 MB, copied into each process	Mapped from model.safetensors and shared
Tokenizer search structure for 500,353 pieces	About 610 MB in each process	Mapped from a vocabulary file and shared
Private memory after encode	About 1.3 GB	About 60 MB, mostly Python and its imports

laghu maps the embedding table from the file. The operating system reads a page of the file only when laghu uses it. The operating system keeps the page in its cache. It shares the page with every process that maps the same file. When memory is low, the operating system can remove the page and read it again later.

For a Unigram model, laghu also replaces the search structure of the tokenizer. A Unigram tokenizer selects the most likely sequence of pieces from a long list. laghu builds a vocabulary file one time for each tokenizer.json. The file holds the pieces in a compact marisa trie, with their ids and scores. Every process maps this file and shares it. laghu then finds the best pieces with a Viterbi search. The search is written in Python. With the optional laghu-rs wheel, the search runs in Rust. The text normalization and the first split into words come from tokenizers. laghu builds them from the same tokenizer.json, without the vocabulary.

Models with a BPE or WordLevel tokenizer use tokenizers to tokenize. WordPiece models also use tokenizers when laghu-rs is not installed. laghu maps the table of these models too. The memory that you save depends on the size of the table compared with the size of the tokenizer.

Install

uv add laghu        # or: pip install laghu

model2vec is not a dependency. Install the fallback extra, uv add 'laghu[fallback]', in these two cases:

• You want load to use model2vec when laghu cannot read a model.
• You want to use same_as_model2vec.

The extra installs model2vec 0.9, which laghu matches. model2vec 0.10 gives different vectors, and its tokenize does not take max_length.

The optional Rust core is the wheel laghu-rs (module laghu_rs). It needs Python 3.11 or later. Install it with pip install laghu-rs, or with the extra: uv add 'laghu[rust]'.

The optional Rust core

With laghu-rs installed, laghu tokenizes these models in Rust, on all CPU cores:

• Unigram models. laghu_rs normalizes the text, finds the added tokens and splits the text into words with the tokenizers crate. Then it runs the Viterbi search of laghu over a byte trie in the shared vocabulary file.
• WordPiece models that have a BertNormalizer, a BertPreTokenizer and a WordPiece model. The English potion models are examples. laghu_rs.WordPiece is a port of FastWordPiece from model2vec-rs. It calculates only the token ids. It does not calculate the offsets that tokenizers keeps.

For a float32 table without weights or a token mapping, laghu_rs also calculates the average vectors. On every path, the ids and vectors are the same as the ids and vectors of model2vec, bit for bit.

This table gives the speed and the private memory for 500-character chunks of text. The computer was an M3 Max with 14 cores. The measurements are from 2026-10-24, with a load average of 6.5 to 9. Each speed is the best of three runs of laghu.bench.speed.

Model	Loader	Chunks per second	Private memory
potion-multilingual-128M (Unigram)	laghu with laghu-rs	42,385	72 MB
	laghu without laghu-rs	1,940	67 MB
	model2vec 0.9	21,105	1,369 MB
potion-base-8M (WordPiece)	laghu with laghu-rs	432,938	81 MB
	laghu without laghu-rs	55,514	93 MB
	model2vec 0.9	55,600	140 MB

laghu 0.3.0 gave 5,276 chunks per second for model2vec on potion-multilingual-128M. That number was measured with tokenizers on one core, probably because TOKENIZERS_PARALLELISM=false was set. On one core, model2vec gives 5,462 chunks per second on the same computer.

When laghu does not use Rust

For a Unigram model, laghu tokenizes in pure Python when one of these conditions is true:

• laghu cannot import laghu_rs.
• LAGHU_RUST=0 is set when the model loads.
• The model loads with shared=False.
• The tokenizer.json truncates or pads.

For a Unigram model, laghu splits the text in Python and does the search in Rust when one of these conditions is true:

• The Python tokenizers has a different major or minor version from laghu_rs.TOKENIZERS. A different patch version is accepted. laghu-rs 0.1.0 and 0.2.0 use tokenizers 0.23.2. They accept all Python tokenizers 0.23.x versions.
• The tokenizers crate cannot read the tokenizer.json.
• The crate splits a text in PROBE differently from the Python split of laghu. PROBE is a set of 46 difficult test texts. laghu logs a warning.

laghu tokenizes a WordPiece model with tokenizers when one of these conditions is true:

• laghu cannot import laghu_rs, or laghu_rs has no WordPiece.
• LAGHU_RUST=0 is set when the model loads.
• The model loads with lean=False.
• The tokenizer.json truncates or pads.
• The Python tokenizers has a different major or minor version from laghu_rs.TOKENIZERS.
• The normalizer, pre-tokenizer and model are not BertNormalizer, BertPreTokenizer and WordPiece.
• The unknown token is not in the vocabulary.
• An added token is normalized, or has lstrip, rstrip or single_word set.
• laghu_rs.WordPiece gives different ids from tokenizers on a text in PROBE. laghu logs a warning.

laghu does not use laghu-rs for BPE and WordLevel models. It does not use laghu-rs for Unigram models with a feature that laghu does not copy.

The ids are the same on every path.

Use

from laghu import load

m = load('minishlab/potion-multilingual-128M')      # a folder, or a Hugging Face repo id (the local cache is tried first)
v = m.encode(['The quick brown fox.', 'कर्मण्येवाधिकारस्ते'])   # float32, (2, 256), unit rows: the same array model2vec returns
m.tokenize(['hello world'])
m.encode_as_sequence('hello world').shape          # (n_tokens, 256)

(2, 256)

load takes a local folder or a Hugging Face repo id. It returns a LeanModel. If the repo is in the local Hugging Face cache, load does not use the network.

If laghu cannot load the model, load logs the reason. Then it tries model2vec’s StaticModel, which needs laghu[fallback]. If model2vec also fails, load raises the first error from laghu. Use fallback=False to get the error without a try with model2vec.

laghu and model2vec cannot read tables stored as bfloat16 or float8, because numpy does not have these types.

laghu keeps the vocabulary file of a Unigram model in $LAGHU_CACHE. If $LAGHU_CACHE is not set, laghu uses $XDG_CACHE_HOME/laghu. If that is not set, laghu uses ~/.cache/laghu. If laghu cannot write to the folder, it puts the file next to tokenizer.json.

Vocabulary file	Time
First load, which builds the file	1 to 2 seconds
Later loads, in any process, which map the file	A few hundredths of a second

If tokenizer.json changes, or if the file is damaged, laghu builds the file again. load(..., shared=False) keeps the vocabulary in a Python dict. This uses about 80 MB more private memory in each process.

To check your own model, use same_as_model2vec(path). It loads the model with laghu and with model2vec, and it compares the results. It uses the built-in test texts of laghu, or the texts that you give:

from laghu import same_as_model2vec
same_as_model2vec('minishlab/potion-base-8M', ['my', 'own texts'])

/Users/71293/code/personal/orgs/laghu/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

{'lean': False,
 'n': 2,
 'ids': True,
 'dtype': True,
 'f32': True,
 'f16': True,
 'exact': True,
 'seq': True,
 'median': True,
 'unk': True,
 'tokens': True}

The result shows, for each check, if the two loaders agree:

Key	What it checks
ids	The token ids
f32	The vectors, changed to float32
f16	The vectors, changed to float16
dtype	The exact data type
exact	The exact bits
seq	encode_as_sequence
median	median_token_length
unk	The unknown token id
tokens	The token list
lean	laghu used its own Unigram tokenizer (not a check)
n	The number of texts (not a check)

What matches, and what it costs

Every model in the table below gives the same ids and vectors as model2vec, bit for bit, on 4,726 texts. The texts include prose in 36 languages and random strings from 30 Unicode blocks. They also include control characters, special tokens, emoji sequences, combining marks, bidirectional text controls and a string of 200,000 characters.

These rules apply to the table:

• Each memory number comes from a new process. On macOS, private memory is phys_footprint after encode, and peak memory is lifetime_max_phys_footprint. On Linux, the same columns would hold RssAnon and VmHWM.
• Speed is encode over 3,226 texts on an M3 Max.
• “laghu tokenizes with” gives the tokenizer that laghu used: its Viterbi search in Python, its Viterbi search in Rust with laghu-rs, or tokenizers.
• In the private memory and speed columns, the first two numbers were measured without laghu-rs. The third number was measured with laghu-rs, in the same way.
• — means that laghu-rs does not change the row. This is true for BPE and WordLevel models. It is also true for potion-code-16M-v2, because its tokenizer.json truncates and pads.
• “not measured” means that nobody measured the row with laghu-rs. These rows were checked without laghu-rs. With laghu-rs, they use Rust when their tokenizer qualifies.
• potion-base-8M and potion-code-16M-v2 were checked again with laghu-rs on 2026-10-24. Both stay identical.

The laghu-rs numbers come from different sessions than the older numbers. Compare each laghu-rs number with the numbers from its own session:

Model	Date	model2vec, texts per second	laghu without laghu-rs, texts per second	laghu with laghu-rs, texts per second	Notes
potion-multilingual-128M	2026-10-23	22,846	14,859	130,007	Best of 3 runs. Load average 6.5. The slowest Rust run gave 41,988 texts per second.
potion-base-8M	2026-10-24	82,027	not measured	650,395	Best of 3 runs. Each Rust run took about 0.003 seconds.

model	tokenizer	laghu tokenizes with	vocab × dims	dtype	norm	identical	private MB, model2vec → laghu → laghu-rs	peak MB, model2vec → laghu	texts/s, model2vec → laghu → laghu-rs
potion-base-2M	WordPiece	tokenizers	29,528 × 64	float32	on	yes	81 → 58 → not measured	82 → 58	84,510 → 92,982 → not measured
potion-base-4M	WordPiece	tokenizers	29,528 × 128	float32	on	yes	94 → 62 → not measured	95 → 62	87,248 → 90,242 → not measured
potion-base-8M	WordPiece	tokenizers, or Rust with laghu-rs	29,528 × 256	float32	on	yes	114 → 70 → 71	115 → 70	79,415 → 81,810 → 650,395
potion-base-32M	WordPiece	tokenizers	63,091 × 512	float32	on	yes	244 → 97 → not measured	244 → 97	75,009 → 81,379 → not measured
potion-retrieval-32M	WordPiece	tokenizers	63,091 × 512	float32	on	yes	244 → 97 → not measured	244 → 97	70,799 → 79,717 → not measured
potion-science-8M	WordPiece	tokenizers	29,528 × 256	float32	on	yes	113 → 70 → not measured	114 → 70	81,008 → 84,532 → not measured
potion-code-16M	WordPiece	tokenizers	61,826 × 256	float32 +map +weights	on	yes	147 → 110 → not measured	211 → 117	10,674 → 10,425 → not measured
potion-code-16M-v2	WordPiece	tokenizers	63,457 × 256	float16	on	yes	108 → 101 → —	186 → 119	21,819 → 22,164 → —
potion-multilingual-128M	Unigram	Python Viterbi, or Rust with laghu-rs	500,353 × 256	float32	on	yes	1,348 → 61 → 59	1,348 → 61	39,105 → 10,902 → 130,007
M2V_base_output	WordPiece	tokenizers	29,528 × 256	float32	off	yes	112 → 69 → not measured	113 → 69	65,460 → 87,100 → not measured
M2V_multilingual_output	WordPiece	tokenizers	501,054 × 256	float32	off	yes	834 → 198 → not measured	842 → 295	42,565 → 90,283 → not measured
M2V_base_glove	WordLevel	tokenizers	400,002 × 256	float32	off	yes	656 → 186 → —	656 → 194	67,028 → 85,058 → —
M2V_base_glove_subword	WordPiece	tokenizers	400,779 × 256	float32	off	yes	647 → 178 → not measured	649 → 195	66,256 → 85,430 → not measured
jina-embeddings-v3-separation-distilled	Unigram	Python Viterbi, or Rust with laghu-rs	250,002 × 256	float32	off	yes	722 → 88 → not measured	722 → 88	51,836 → 20,840 → not measured
multilingual-e5-large-m2v	Unigram	Python Viterbi, or Rust with laghu-rs	250,002 × 512	float32	on	yes	981 → 109 → not measured	981 → 109	24,139 → 19,670 → not measured
m2v-gemma-embedding-300m	Unigram	Python Viterbi, or Rust with laghu-rs	255,732 × 256	float16 +weights	on	yes	542 → 93 → not measured	542 → 93	20,319 → 12,134 → not measured
potion-multilingual-128M-i8-tokenade	Unigram	Python Viterbi, or Rust with laghu-rs	500,353 × 256	int8	on	yes	985 → 138 → not measured	985 → 138	27,676 → 12,090 → not measured
m2v-qwen3-embedding-0.6b-1024d	Unigram	Python Viterbi, or Rust with laghu-rs	151,644 × 1024	float16 +weights	on	yes	602 → 132 → not measured	602 → 132	16,792 → 7,168 → not measured
ovos-m2v-intents-es-ES-bne-v5	Unigram	Python Viterbi, or Rust with laghu-rs	50,259 × 64	float32	on	yes	169 → 52 → not measured	169 → 52	11,786 → 11,265 → not measured
m2v-gte-256-edu	Unigram	tokenizers	313,845 × 256	float32	on	yes	1,028 → 663 → —	1,028 → 675	25,363 → 24,981 → —
bge-m3-m2v-256	Unigram	Python Viterbi, or Rust with laghu-rs	250,002 × 256	float32	off	yes	725 → 89 → not measured	725 → 89	34,518 → 20,670 → not measured
potion-mxbai-micro	WordPiece	tokenizers	2,000 × 256	int8 +map +weights	on	yes	90 → 72 → not measured	91 → 72	68,385 → 70,998 → not measured
NIFE-mxbai-embed-large-v1_model2vec	WordPiece	tokenizers	107,962 × 1024	float32	off	yes	558 → 106 → not measured	572 → 116	41,042 → 26,756 → not measured
ovos-m2v-intents-pt-PT-albertina-v5	BPE	tokenizers	50,262 × 256	float32	on	yes	162 → 90 → —	162 → 90	46,343 → 22,813 → —
F2LLM-v2-330M-model2vec	BPE	tokenizers	151,644 × 512	float16	on	yes	356 → 171 → —	356 → 171	28,795 → 40,132 → —
static-retrieval-multilingual-69m-v1	BPE	tokenizers	179,936 × 384	float32	off	yes	537 → 229 → —	541 → 263	32,235 → 31,848 → —

+map marks a vocabulary-quantized model. This model stores fewer rows than it has tokens, and a map from each token to its row. +weights marks a model that multiplies the vector of each token by a stored weight.

The potion-multilingual-128M row was measured again on 2026-10-13 with the shared vocabulary file. The other Unigram rows were measured with the vocabulary in a dict. For potion-multilingual-128M, the dict used about 80 MB more private memory in each process.

Without laghu-rs, laghu is slower than model2vec on Unigram models. Its speed is between about one third of the speed of model2vec and about the same speed. The reason is that tokenizers uses all cores for a batch, but the Python search uses one core. With laghu-rs, laghu also uses all cores. On potion-multilingual-128M, laghu with laghu-rs is about two times as fast as model2vec. The table in The optional Rust core gives the numbers. The other Unigram models were not measured with laghu-rs. All these numbers are for batches of texts. They do not show the time to encode one query.

laghu keeps a cache of words that it segmented before. The cache helps when the tokenizer first splits the text into words. The jina-v3, bge-m3 and multilingual-e5 tokenizers do this.

Text	Speed with the word cache
Natural text, first pass	1.6 to 1.9 times faster
The same text again	3 to 5 times faster

The bench page has these measurements. It also tells why laghu does not use two faster Rust tokenizers, tokie and gigatoken.

Without laghu-rs, laghu and model2vec tokenize the other models with the same tokenizers code. Each speed sample for these models lasts about 0.1 seconds. The samples change by up to half from one run to the next. Thus, the speed differences in these rows have no meaning.

What is not supported

• laghu does not quantize or truncate a model when it loads the model. These are the model2vec options quantize_to, dimensionality and vocabulary_quantization. Each option makes a new copy of the table, and laghu exists to prevent that copy. Use model2vec if you need these options.
• laghu tokenizes a Unigram model with single_word added tokens with tokenizers. It still maps the table. The reason is that tokenizers finds the end of a word with the Unicode Alphabetic property, and Python’s str.isalpha does not match it.
• laghu accepts use_multiprocessing and ignores it. The result is the same.
• encode([]) returns an empty array of shape (0, dim). model2vec raises an error.

Developing

laghu uses nbdev 3 with hatchling. The notebooks in nbs/ are the source code. uv run nbdev-prepare exports the package, runs the tests and builds this README again. uv run nbdev-test --flags slow also runs the benchmark. The benchmark downloads about 5 GB of models.

Licence

MIT.