A request field msgpack cannot pack answers 500 and leaks one requests.active permanently (new_user runs before send_one) #681

Before you start

• I have read the FAQ and my problem is not answered there.
• I have read the Roadmap and this is not already planned there.
• I have searched existing issues and found no duplicate.
• I am on the latest release, or on a freshly rebuilt main when building from source.

What happened

A request whose fields contain a value msgpack cannot serialise -- an integer >= 264 or < -263 in chat_template_kwargs, top_k or max_tokens, or a lone UTF-16 surrogate in stop -- is answered with HTTP 500 Internal Server Error instead of a 4xx, and /v1/stats requests.active goes up by one and never comes back down. Every such request leaks one more. The engine keeps serving (later requests are answered and the leaked entries do not use up --max-running-requests), but:

• requests.active no longer means anything: an idle server reports N active requests forever, so a watchdog keyed on "active > 0 and no throughput" (as in Engine wedges under sustained agentic load: active=1, tps=0 until restart (Windows, Qwen3.6-35B-A3B-FP8) #453's setup) will read an idle engine as wedged;
• POST /v1/admin/prepare-stop always runs its whole drain timeout (5.13 s instead of 0.00 s here) before it aborts uids the scheduler never saw;
• a client gets a 500 for its own malformed input, and on /v1/messages a 400 that still leaks.

Reproduced 20/20 (10 request shapes x 2 runs) on main (377a3bc); the code path should be unchanged on current main.

Reproduction

curl -s localhost:1941/v1/stats | python3 -c 'import json,sys; print(json.load(sys.stdin)["requests"]["active"])'
# 0
curl -s -w '\nHTTP %{http_code}\n' localhost:1941/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "gpt-oss-20b", "max_tokens": 8,
  "messages": [{"role": "user", "content": "Say hi."}],
  "chat_template_kwargs": {"x": 18446744073709551616}}'
# Internal Server Error
# HTTP 500
sleep 10
curl -s localhost:1941/v1/stats | python3 -c 'import json,sys; print(json.load(sys.stdin)["requests"]["active"])'
# 1      <- and it stays 1; the same request with {"x": 1} answers 200 and leaves 0

The same with "top_k": 18446744073709551616, "max_tokens": 18446744073709551616, "chat_template_kwargs": {"x": -9223372036854775809}, or "stop": ["\ud800"] (JSON escape of a lone surrogate), on /v1/chat/completions, /v1/completions, /v1/messages (top_k; stop_sequences answers 400 and still leaks) and /v1/responses (top_k). The attached repro_unpackable_field_leak.py (stdlib only) sends all ten shapes, each next to a control with a packable value, and prints requests.active before and after:
repro_unpackable_field_leak.py

FAIL  [hostile] chat ctk int 2**64             /v1/chat/completions   HTTP 500  active 0 -> 1  (leaked 1)  body: 'Internal Server Error'
FAIL  [hostile] chat ctk int < -2**63          /v1/chat/completions   HTTP 500  active 1 -> 2  (leaked 1)
FAIL  [hostile] chat top_k 2**64               /v1/chat/completions   HTTP 500  active 2 -> 3  (leaked 1)
FAIL  [hostile] chat max_tokens 2**64          /v1/chat/completions   HTTP 500  active 3 -> 4  (leaked 1)
FAIL  [hostile] chat stop lone surrogate       /v1/chat/completions   HTTP 500  active 4 -> 5  (leaked 1)
FAIL  [hostile] completions max_tokens 2**64   /v1/completions        HTTP 500  active 5 -> 6  (leaked 1)
FAIL  [hostile] completions top_k 2**64        /v1/completions        HTTP 500  active 6 -> 7  (leaked 1)
FAIL  [hostile] messages top_k 2**64           /v1/messages           HTTP 500  active 7 -> 8  (leaked 1)
FAIL  [hostile] messages stop lone surrogate   /v1/messages           HTTP 400  active 8 -> 9  (leaked 1)
FAIL  [hostile] responses top_k 2**64          /v1/responses          HTTP 500  active 9 -> 10  (leaked 1)
canary chat after the shapes: HTTP 200; requests.active 10 completed 11

(10/10 controls: HTTP 200, no leak.)

What's Expected

A 400 (invalid_request_error, like the server's other validation errors), and requests.active back where it
was.

Actual

500 with a plain-text body, and the server log shows

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "msgpack/_packer.pyx", line 171, in msgpack._cmsgpack.Packer._pack_inner
OverflowError: Python int too large to convert to C unsigned long
The above exception was the direct cause of the following exception:
  ...
  File "freetoken/server/openai_api.py", line 200, in handle_chat_completion
    uid = await submit_generation(spec, state)
  File "freetoken/server/generation.py", line 299, in submit_generation
    await state.send_one(
  File "freetoken/server/api_server.py", line 326, in send_one
    await self.send_tokenizer.put(msg)
  File "freetoken/utils/mp.py", line 46, in put
    event = msgpack.packb(self.encoder(obj), use_bin_type=True)
  File "msgpack/_packer.pyx", line 180, in msgpack._cmsgpack.Packer._pack_inner
OverflowError: Integer value out of range

(the surrogate: UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed).

Possible cause

submit_generation (server/generation.py:293-310) registers the request before it serialises it:

uid = state.new_user()          # ack_map[uid], event_map[uid], stats._inflight.add(uid)
await state.send_one(TokenizeMsg(uid=uid, ..., sampling_params=..., chat_template_kwargs=...))

send_one -> utils/mp.py:46 msgpack.packb(...) raises OverflowError / UnicodeEncodeError. handle_chat_completion (server/openai_api.py:199-202) catches only GenerationError; handle_completion (openai_api.py:408-414, 426-436) calls new_user + send_one with no handler; /v1/messages (anthropic_api.py:114-127) and /v1/responses (responses_api.py:154-163) catch ValueError, which covers UnicodeEncodeError (hence the 400) but not OverflowError, and neither path unregisters the uid. A uid
leaves StatsTracker._inflight only when the scheduler's finished reply for it arrives (stats.py:76-82); this message never reached the tokenizer, so that never happens. Nothing in pydantic stops the values: top_k: int | None and max_tokens: int | None (api_models.py) are unbounded, chat_template_kwargs is dict[str, Any], and resolve_sampling (generation.py:162-193) only checks max_tokens < 1.

What stays behind per leaked request is frontend bookkeeping only -- one entry each in stats._inflight, ack_map (an empty list) and event_map (an asyncio.Event). Measured: no scheduler slot (every prefill logs #running-req: 1 with 20 leaked and --max-running-requests 8), no KV pages (kv.used_pages 0), no VRAM change.

How did you install FreeToken

Built from source

FreeToken version

ft --version: freetoken version 0.1.3 git rev-parse --short HEAD: 377a3bc (main, 2026-10-07, "ci: auto-label amd issues and prs by title (#620)")

OS

Arch Linux

OS details

Ubuntu 20.04.6 LTS, kernel 5.15.0-138-generic, Python 3.12.8.

GPU and driver

NVIDIA GeForce RTX 4090 24 GB (sm_89), driver 580.126.09, CUDA 13.0.

CPU and system RAM

AMD Ryzen Threadripper PRO 5955WX (16 cores / 32 threads), 440 GiB RAM

Checkpoint

openai/gpt-oss-20b (Hugging Face, revision 6cee5e81ee83917806bbde320786a8fb61efebee)

Command

Full log

Anything else

No response