[BlivionIaG]

LFM2 tool calls: typed arguments, JSON list form, per-call streaming, Pythonic grammar for tool_choice

Follows #1246 and #1250.

Changes

1. A call argument is read as the type its schema names.
   • 123 for a string parameter becomes "123".
   • "4" for an integer becomes 4, and 4.0 becomes 4.
   • 1 or 0 for a boolean becomes true or false.
   • A parameter that allows a string and an integer keeps the model's value.
2. The JSON list form [{"name": "f", "arguments": {...}}] is read as calls.
   • parameters is accepted for arguments. arguments can be a JSON string.
   • The offered-tool check and the typing are the same as for the Python form.
   • Anything else stays text.
3. A history call with "arguments": "" went into the prompt as _invalid_arguments=''.
   toolArguments in messages.zig now reads blank text as an empty object, as it does {}.
   This is shared code: the prompt changes the same way for every family, not only LFM2.
4. Each call of a streamed LFM2 block is sent when it ends, not when the whole block ends.
   • Change is in tool_lfm2.zig. chat.zig, tool_stream.zig and every other format are unchanged.
   • An unclosed block is still held back. Only its complete calls are sent.
   • A call is sent when it ends, and the rest of its block is read after it.
     If a later part of the block is not a valid call, the stream drops that part.
     A reply that is not streamed treats a block with an invalid part as text.
5. tool_choice required and named use a Pythonic grammar for LFM2: tool_grammar_lfm2.zig.
   • It forces <|tool_call_start|>[name(key=value, ...)]<|tool_call_end|> with the offered tools.
   • Arguments follow the schema's property order. Required keys are forced. Values are typed by their schema kind.
   • Several calls are allowed unless parallel_tool_calls is false.
   • It is used when a probe call renders as <|tool_call_start|>[tfprobe_fn(tfprobe_k=. Other families keep their grammar.
   • Limits: keys that are not Python identifiers are left out. A tool with no properties can only be called as name().
     The grammar writes the Python form only. Items inside arrays and objects are not typed by their schemas.
6. Goldens: lfm2_typed, lfm2_json and lfm2_two, each whole, streamed, single and single streamed.
   The requests come from freeze_cases.py. The expected responses are written by hand from existing ones.
   For lfm2_typed and lfm2_json, the token counts and token_sha come from the candidate fake server.
   For lfm2_two, they come from fake_text.py, which reproduced the committed numbers of four existing goldens first.
   The goldens do not show timing. The unit tests show the byte at which each call is first sent.
   There is no golden for item 5: the golden fake server has no grammar library and no LFM2 template.

Tests

Head ddc3e53aa, on upstream main at 3d219d215. Apple M4 Pro, 48 GiB, macOS 26.5.1, Zig 0.17.0.

• zig build test test-golden native -Dcpu=apple_m1 -Dversion=1.1.4: 449/449 steps succeeded, exit code 0.
  test-golden: 322/335 equal, 0 unexpected differences. Second set: 26/26 equal.
• zig fmt --check: clean. python3 tools/zig/lean_check.py: 437 problems on this branch and 437 on main.
• New unit tests: tool_lfm2.zig (typing, JSON list), tool_lfm2_stream_test.zig (byte boundaries, early send, refusals),
  tool_grammar_lfm2_test.zig (grammar text, replies that fit it parse through tool_lfm2 and tool_parse), messages.zig (item 3).
• Item 5 with xgrammar (libtfgrammar.dylib, no model): the grammar compiled, accepted conforming replies and masked bad ones.
  This ran on a hand-patched copy of the generated grammar. The exact committed output was not run through it.

Live check

Checkpoint LiquidAI/LFM2.5-24B-A2B-MLX-4bit, revision 667408d494bde191c2cb9dd4f4549827816b9275.
Command: tensorfold-native serve <checkpoint> --prompt-cache-gib 0 (default sampling), libtfgrammar.dylib from a release archive beside the binary.
One model process.

Route	Cases	Result
/v1/chat/completions	one call; two calls; both streamed; argument types; awkward string; parallel_tool_calls: false; tool_choice required and named; no-tool prompt; tool-result round trip	as expected; no tool markup in content
/v1/messages	whole; streamed; history with two tool_result blocks	two tool_use blocks, stop_reason: tool_use; the follow-up uses both results
/v1/responses	whole; streamed; history with two function_call_output items	two function_call items; stream completes; the follow-up uses both results

Not measured

• Whether the model ever writes the JSON list form. Unit tests and goldens cover it.
• The time between the first and second call of a streamed reply on the live model.
• CUDA and HIP. The change is in shared server code and runs in the CPU host tests.
• Decode speed. The change is in text handling after decode.

Overlap

atc overlap --json (ATC 0.7.0, complete: false: some pull request files were unavailable):

• minimax_m3: serve MiniMax-M3.1-Flash (428B-A23B MoE, images) natively on Metal #1235 conflicts in zig/src/server/root.zig. Both add test import lines.
• The other overlapping pull requests it listed are clean.

[BlivionIaG]

Failing test before change 3 (toolArguments in messages.zig from main, the new test unchanged):

error: 'messages.test.a call's arguments become objects: blank text is no arguments, bad text is kept under _invalid_arguments' failed:
expected: {}
found:    {"_invalid_arguments": ""}

With the change the test passes: zig build test-server-cpu gives 738 passed, 3 skipped, 0 failed on the commit that added it.

Full run on head ddc3e53aa, alone on the Mac: zig build test test-golden native, 449/449 steps succeeded, exit code 0.
test-golden 322/335 equal, 0 unexpected differences; second set 26/26. Zig counted 1631/1638 tests for this run (7 skipped); it counts only the tests it re-ran.

[ashhart]

Landed on main in 8086c20..2e33e29, with your records under your authorship. Thank you, @BlivionIaG. LFM2 tool calls now:

• read arguments as the type the schema names;
• parse the JSON list form;
• render empty-string arguments as no arguments;
• stream each call as soon as it ends;
• support required and named calls through a Pythonic grammar.

The goldens cover whole, streamed and single-call-allowed forms, and the host suites and goldens pass.

- Synthema, Ash's AI Assistant.