Skip to content

Apply decision adapters' temperatures and warm their grammar on bind - #281

Merged
webdevtodayjason merged 1 commit into
mainfrom
fable/decide-temperatures
Sep 24, 2026
Merged

webdevtodayjason merged 1 commit into
mainfrom
fable/decide-temperatures

Conversation

@webdevtodayjason

Copy link
Copy Markdown
Contributor

Summary

Two decision-model fixes, one PR, both in the shared decision core (api/decide.py), so /v1/decide and /v1/systemone get them together.

#276: the adapter's per-type temperatures. run_questions now finds the served model's directory in this node's store through models/registry.py::snapshot_dir_for (a model served from an absolute path is its own directory). If that directory carries temperatures.json, each question's label logprobs are divided by its kind's temperature before the softmax, so distribution, confidence, systemone's probabilities, noul, the expected score and the chance-corrected confidence all come from the tempered spread. Kinds are choice, noul and score. On /v1/decide, type: "boolean" maps to noul, type: "score" to score and an options list to choice. Both responses carry a top-level "calibration": {"applied": bool, "temperatures": {...} | null}. A request can send "calibration": "raw" to opt out. Any other value is a 400 on decide and a 422 on systemone. A missing kind, or a value that is not a finite positive number, leaves that kind at the engine's own spread.

#277: warm on load. Every launch path (boot primary, replay, /api/models load, set-model, relaunch) goes through models/api_routes.py::_wait_for_bind. When it reports a bind, decide.py::schedule_decision_warmup checks the model's directory for prompt_contract.json or temperatures.json. If either is there, it sends one minimal constrained question per kind (a two-option choice, a noul, a two-level score) in the background, using the same build_chat_body and ask_one the routes use and a 600 s timeout. It logs the compile time for each kind. An adopted boot primary gets the same warm-up, because the flag does not survive a restart. /api/status gains an instances list where each row carries warm (true, false while warming or after a failed warm-up, null for a model with nothing to warm) and warm_compile_seconds. Readiness is not held back for the warm-up. The route's engine-call timeout goes from 180 s to 300 s. If a call connects and then times out, the 503 now says the engine is compiling the answer grammar, retry instead of unreachable. A connect timeout still reads as unreachable.

AGENTS.md: the systemone rule "nothing on this path rescales, tempers or corrects" is updated to allow this one exception, and the warm-up hook is recorded as the single place a bound engine gets warmed. README: the /v1/decide section documents the calibration block, the raw opt-out and the warm flag.

Closes #276
Closes #277

How it was tested

  • pytest tests/: 3199 passed, 2 skipped, 1 xfailed. main was 3177 passed.
  • ruff check on every touched Python file: clean.
  • tests/test_decide_calibration.py (new): the temperature math against hand-computed sigmoids, the kind mapping, reading temperatures.json (bad entries, bad JSON, a prompt contract alone), and the calibration block. Both routes are also run end to end against the fake vLLM with logprobs, with this node's store pointed at a temp directory: temperatures on, raw opt-out, no file, and a refused value.
  • tests/test_decide_warmup.py (new): a fake engine records the three warm-up requests (constrained to A/B, logprobs on, thinking off, the noul as true/false). A bind through _wait_for_bind flips warm from false to true. A failed warm-up stays cold and names the kind that failed. A non-decision model is left alone and its stale entry is cleared. The warm-up asks by --served-model-name. A slow engine produces the "compiling" message. /api/status reports warm per instance.
  • Not run against a real engine. The compile times and the timeout headroom come from the issue's 60 to 90 s figure, not from a new measurement.

Changelog

Added

Changed

  • A decision request that meets a cold grammar compile gets a 503 saying so, and has longer to finish. The engine-call limit is 300 s, up from 180, and a call that times out after connecting reads "the engine is compiling the answer grammar, retry" instead of "unreachable".

🤖 Generated with Claude Code

/v1/decide and /v1/systemone now read temperatures.json from the served
model's directory in this node's store (resolved through
registry.snapshot_dir_for) and divide each question's label logprobs by
its kind's temperature before the softmax, in run_questions so both
routes get it. Both responses carry a top-level calibration block
{applied, temperatures}; "calibration": "raw" opts out.

A model whose directory carries prompt_contract.json or
temperatures.json is sent one minimal constrained question per kind
(choice, noul, score) through ask_one as soon as its engine binds,
from _wait_for_bind, in the background. Each compile time is logged
and /api/status gains an instances list with warm per instance. The
engine-call timeout goes from 180 s to 300 s, and a connected call
that times out now reads as the grammar compiling, retry, rather than
as an unreachable node.

Closes #276
Closes #277

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@webdevtodayjason
webdevtodayjason merged commit d421bb0 into main Sep 24, 2026
1 check passed
@webdevtodayjason
webdevtodayjason deleted the fable/decide-temperatures branch September 24, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant