fix(classifier): adopt the tokenizer pad_token_id when the model config lacks one

[olesxg]

What

docs/lm_head_to_classifier/convert_lm.py (lines 166-170) lists
Qwen/Qwen3-Reranker-0.6B, -4B and -8B, and the converted checkpoints are published
as michaelfeil/Qwen3-Reranker-0.6B-seq et al. Their config.json carries
architectures: ["Qwen3ForSequenceClassification"] and id2label: {"0": "no", "1": "yes"}
but no pad_token_id key.

transformers' decoder-only sequence-classification forward requires one as soon as the
batch is larger than 1:

ValueError: Cannot handle batch sizes > 1 if no padding token is defined.

infinity hits this during loaded_engine.warmup(batch_size=engine_args.batch_size, ...)
in inference/select_model.py, with the default batch_size=32 — so the server fails
at load, before serving a single request.

michaelfeil/mxbai-rerank-base-v2-seq is unaffected because its config ships
pad_token_id: 151643, which is why this has gone unnoticed.

Fix

Adopt the tokenizer's pad_token_id when the model config does not have one, right
after the pipeline(...) call and before quantization / BetterTransformer / compile
wrap the model. It is a strict no-op for every checkpoint that already carries the key.

The four lines live in a small module-level helper rather than inline so they can be
unit-tested without downloading a model — which matters here, because the test group
pins transformers = "4.47.0", and that version cannot parse model_type: qwen3 at all.

Which path this touches

inference/select_model.py routes on label count: len(id2label) < 2 goes to
RerankEngine, otherwise PredictEngine. These checkpoints have 2 labels, so they go
to SentenceClassifier and never reach CrossEncoderPatched. The crossencoder path is
therefore untouched — the 1-label rerankers that do reach it are BERT-family encoders
that always carry pad_token_id.

Test

Three assertion-level tests appended to the existing
tests/unit_test/transformer/classifier/test_torch_classifer.py, using
types.SimpleNamespace stand-ins: missing -> adopted, already set -> untouched,
tokenizer has none -> no crash. No download, no torch, version-independent, so they run
on the pinned 4.47.0 in CI. No new model id added to conftest.py.

Deliberately not in scope

No change to the transformers floor — I saw #620 was closed for exactly that reason
(ONNX-optimum and BetterTransformer breakage). No causal-LM reranker engine, no
server-side Qwen chat template, no rerank-API instruction field. This is only the
load-time crash.

Related to but does not close #642.

Verified

Config of the published checkpoint (michaelfeil/Qwen3-Reranker-0.6B-seq/raw/main/config.json):

architectures : ['Qwen3ForSequenceClassification']
model_type    : qwen3
id2label      : {'0': 'no', '1': 'yes'}
pad_token_id  : ABSENT

versus michaelfeil/mxbai-rerank-base-v2-seq, which carries pad_token_id: 151643 and works today.

Minimal reproduction, on this repo's own pinned transformers==4.47.0, no download required
(Qwen2 stands in for Qwen3 because 4.47.0 cannot parse model_type: qwen3 — the defect is in
the shared decoder-only sequence-classification path, not in Qwen3 specifically):

cfg = Qwen2Config(vocab_size=64, hidden_size=16, num_hidden_layers=1,
                  num_attention_heads=2, num_key_value_heads=2,
                  num_labels=2, pad_token_id=None)
m = Qwen2ForSequenceClassification(cfg).eval()
m(input_ids=torch.randint(0, 64, (2, 8)))

transformers 4.47.0
config.pad_token_id = None
ValueError: Cannot handle batch sizes > 1 if no padding token is defined.
after fix -> logits (2, 2)

Local test run (Windows, Python 3.11.9):

$ poetry run pytest tests/unit_test/transformer/classifier/test_torch_classifer.py -v
test_classifier PASSED                                                  [ 25%]
test_set_pad_token_id_if_missing_adopts_tokenizer_value PASSED          [ 50%]
test_set_pad_token_id_if_missing_is_noop_when_already_set PASSED        [ 75%]
test_set_pad_token_id_if_missing_tolerates_tokenizer_without_pad PASSED [100%]
4 passed in 18.71s

ruff check and codespell pass; ruff format --check does not flag either changed file;
mypy ./infinity_emb output is identical to main.

I have not benchmarked the converted checkpoint or verified score parity against the original
causal-LM model — this PR is only about the load-time crash.

[greptile-apps]

Greptile Summary

The PR prevents decoder-only sequence classifiers from failing on batched inference when their model configuration omits a padding token ID.

• Adds a helper that adopts the pipeline tokenizer's padding token ID only when the model configuration lacks one.
• Applies the correction immediately after pipeline construction and before optional model transformations.
• Adds focused tests for adoption, preservation of an existing value, and absence of a tokenizer padding value.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code failure identified.

The helper is narrowly scoped to missing configuration values, preserves existing model settings, tolerates tokenizers without padding, and runs before downstream model transformations that retain the updated configuration.

Important Files Changed

Filename	Overview
libs/infinity_emb/infinity_emb/transformer/classifier/torch.py	Safely synchronizes a missing model padding token ID from the tokenizer before classifier optimization and inference.
libs/infinity_emb/tests/unit_test/transformer/classifier/test_torch_classifer.py	Adds isolated coverage for all branches of the padding-token synchronization helper.

_{Reviews (1): Last reviewed commit: "fix(classifier): adopt the tokenizer pad..." | Re-trigger Greptile}