Apply decision adapters' temperatures and warm their grammar on bind

README.md

@@ -1278,6 +1278,37 @@ tokenizer does not give a two-letter label its own token, that label's mass is
 its first letter's token mass, which the single-letter label of the same letter
 also claims. Read such a pair as jointly calibrated.
 
+**Decision adapters bring their own temperatures.** When the served model's
+directory in the node's store carries a `temperatures.json`
+(`{"temperatures": {"choice": T, "noul": T, "score": T}}`, which the
+`frontier-infra/jebadiah-*` checkpoints ship), each question's label logprobs are
+divided by the temperature for its kind before the softmax, so `distribution`
+and `confidence` read the way the adapter was fitted. `type: "boolean"` counts as
+`noul`, `type: "score"` as `score` and an `options` list as `choice`. Every
+response says what was applied, and `POST /v1/systemone` carries the same block
+(these temperatures are illustrative, not fitted):
+
+```json
+"calibration": {"applied": true, "temperatures": {"choice": 1.5, "noul": 0.8, "score": 1.2}}
+```
+
+Send `"calibration": "raw"` to get the engine's own spread instead; the block
+then reads `"applied": false` and still lists the temperatures you opted out of,
+so you can refit against the raw numbers. The file is read on the node that
+answers the route, so a model this node routes to a peer without holding a copy
+itself answers raw.
+
+**A decision model warms up when it loads.** The first constrained request per
+question shape makes the engine compile the answer grammar, 60 to 90 s on a
+GB10. So when an engine binds on a model whose directory carries
+`prompt_contract.json` or `temperatures.json`, AINode sends it one small
+question of each kind in the background and logs each compile time. Each row of
+`/api/status`'s `instances` carries `warm` (`true` once warm, `false` while
+warming or after a failed warm-up, `null` for a model with nothing to warm) and
+`warm_compile_seconds`. A request that still meets a cold compile and runs past
+the 300 s engine-call limit gets a `503` saying the engine is compiling the
+answer grammar and to retry.
+
 ### Metrics: `/metrics` (Prometheus) and `/api/metrics` (JSON)
 
 AINode exposes its own metrics on port 3000, the same port as its API. The engine's

ainode/api/server.py

@@ -192,6 +192,9 @@ def create_app(
         app["instances"] = _seed
     app["start_time"] = time.time()
     app["client_session"] = None  # lazy-init in startup
+    # Decision-model warm-up state by engine port (api/decide.py, #277). Created
+    # here because the app is frozen by the time an engine binds.
+    app["decision_warm"] = {}
     app["metrics_collector"] = collector
     # On a unified-memory node the collector has no usage figure of its own to
     # report, and host RAM is not VRAM (#175). Give it the one number this node
@@ -1393,13 +1396,25 @@ async def handle_status(request: web.Request) -> web.Response:
         _own = _manager.by_port(getattr(config, "api_port", 0) or 0)
         engine_adopted = bool(_own is not None and getattr(_own.record, "adopted", False))
 
+    # Every instance this process manages, with whether a decision model's answer
+    # grammar has been compiled yet (#277). ``warm`` is null for a model with
+    # nothing to warm.
+    from ainode.api.decide import instance_warm_fields
+    instances = [{"instance_id": inst.record.instance_id,
+                  "model": inst.record.model,
+                  "api_port": inst.record.api_port,
+                  "status": inst.record.status,
+                  **instance_warm_fields(request.app, inst.record)}
+                 for inst in (_manager.instances() if _manager is not None else [])]
+
     return web.json_response({
         "node_id": config.node_id,
         "node_name": config.node_name,
         "model": config.model,
         "gpu": gpu_info,
         "engine_ready": engine_ready,
         "engine_adopted": engine_adopted,
+        "instances": instances,
         # Coarse engine load phase for the UI launching card (3c), derived from
         # the live /v1/models probe above rather than the engine's own latch:
         # see engine_load_phase.

ainode/api/systemone.py

@@ -8,22 +8,26 @@
 and no fork of the client. ONE adapter, so none of them needs an AINode-shaped
 branch.
 
-What it does NOT promise: calibration. The hosted service's numbers are a
+What it does NOT promise: the hosted service's calibration. Its numbers are a
 property of the model TypeSafe trained and of how they fit it; these are the
 probabilities the served model put on the option labels, read off its logprobs
-and renormalized, and nothing here corrects, tempers or rescales them. A local
-model's confidence is worth what that model's confidence is worth, and the way
-to find out is ``scripts/ainode-bench.py decide``, which scores exactly this
-against labels. So: same wire, same client, same typed answers, and no claim at
-all about how well those answers are spread.
+and renormalized. The one correction applied is the served model's OWN: a
+decision adapter that ships ``temperatures.json`` beside its weights gets each
+question's logprobs divided by the temperature fitted for that question's type,
+in the decision core (``api/decide.py``), and the response's top-level
+``calibration`` block says whether that happened and with which temperatures. A
+request sends ``"calibration": "raw"`` to get the engine's own spread instead. A
+local model's confidence is worth what that model's confidence is worth, and the
+way to find out is ``scripts/ainode-bench.py decide``, which scores exactly this
+against labels.
 
 Two numbers on the way out are inferences rather than readings, and each says so
 where it is computed: ``confidence`` is the hosted service's chance-corrected
 formula (``normalized_confidence``), reproduced from its published examples
 because no document states it, and the usage block falls back to the bench's own
-estimate when an engine reports no usage at all (``usage_block``). The raw
-distribution goes out beside them untouched, so a caller who disagrees with
-either has the numbers they came from.
+estimate when an engine reports no usage at all (``usage_block``). The
+distribution goes out beside them unchanged by either, so a caller who disagrees
+with either has the numbers they came from.
 
 One engine call answers one question, which is also what bounds a question: only
 the top ``MAX_CRITERIA`` labels come back with a probability, so a wider option
@@ -61,8 +65,12 @@
 from aiohttp import web
 
 from ainode.api.decide import (
+    CHOICE,
+    NOUL,
+    SCORE,
     TOP_LOGPROBS,
     DecideError,
+    calibration_mode,
     candidates_for,
     merge_usage,
     normalize_questions,
@@ -72,12 +80,10 @@
     unavailable,
 )
 
-# The three question types the format defines. Anything else is a 422 rather than
-# a guess: a client that asked for a kind of judgement this route does not have is
-# better off being told which kinds it has.
-CHOICE = "choice"
-NOUL = "noul"
-SCORE = "score"
+# The three question types the format defines, which are also the kinds a decision
+# adapter's temperatures are fitted per (``decide.py``). Anything else is a 422
+# rather than a guess: a client that asked for a kind of judgement this route does
+# not have is better off being told which kinds it has.
 QUESTION_TYPES = (CHOICE, NOUL, SCORE)
 
 # A noul's two options, in this order, always. The answer is P(true), so `true`
@@ -283,11 +289,15 @@ def decide_questions(translated: dict[str, Translated]) -> dict[str, dict]:
 
     Goes through the core's own ``normalize_questions`` rather than around it, so
     the option ceiling, the two-option floor and distinct options are checked in
-    one place for both routes.
+    one place for both routes. Each question keeps its Jev type as its ``kind``,
+    which is what picks the adapter's temperature for it.
     """
-    return normalize_questions({key: {"question": item.question,
-                                      "options": item.options}
-                                for key, item in translated.items()})
+    questions = normalize_questions({key: {"question": item.question,
+                                           "options": item.options}
+                                     for key, item in translated.items()})
+    for key, item in translated.items():
+        questions[key]["kind"] = item.kind
+    return questions
 
 
 # ---------------------------------------------------------------- translate out
@@ -322,9 +332,9 @@ def normalized_confidence(top: Any, options: int) -> float:
 
     INFERRED, not specified: it reproduces every example in TypeSafe's published
     docs and SDK types for both choice and score, and Kev's playground authors
-    arrived at the same formula for choice, but no document states it. The raw
-    distribution goes out untouched in ``probabilities`` beside it, so a caller
-    who disagrees with the formula has the numbers it came from.
+    arrived at the same formula for choice, but no document states it. The
+    distribution it came from goes out in ``probabilities`` beside it, so a
+    caller who disagrees with the formula has the numbers it came from.
     """
     spread = probability(top)
     if options < 2:
@@ -461,6 +471,7 @@ async def handle_systemone(request: web.Request) -> web.Response:
         translated = translate_questions(body.get("questions"))
         questions = decide_questions(translated)
         state = serialize_state(body.get("state"))
+        calibration = calibration_mode(body.get("calibration"))
     except DecideError as exc:
         return unprocessable(str(exc))
 
@@ -477,7 +488,8 @@ async def handle_systemone(request: web.Request) -> web.Response:
     # No shared instructions block: in this format a question's own instructions
     # are the whole prompt for it, and the questions of one ask still never see
     # each other's answers.
-    run = await run_questions(request, model, questions, state, None, candidates)
+    run = await run_questions(request, model, questions, state, None, candidates,
+                              calibration)
 
     answers: dict[str, dict] = {}
     failures = list(run.failures)
@@ -503,4 +515,5 @@ async def handle_systemone(request: web.Request) -> web.Response:
         "answers": answers,
         "usage": usage_block(run.payloads, state, translated, len(answers)),
         "latency_ms": round(total_ms, 1),
+        "calibration": run.calibration,
     })

ainode/models/api_routes.py

@@ -1270,15 +1270,31 @@ async def _wait_for_bind(app, port: int, backend, timeout: float = 300.0,
     Every verdict this returns, bound or not, is appended to the node's
     launch-time ledger, the only place a load time is written down where the
     interface can read it back afterwards (see ``record_launch_time``).
+
+    A bind is also where a decision model starts warming its answer grammar
+    (``api/decide.py::schedule_decision_warmup``, #277), in the background and
+    without holding the verdict: every launch path waits here, so no path can
+    skip it.
     """
     bound, reason, alive = await _bind_wait(app, port, backend, timeout,
                                             loading=loading)
     await record_launch_time(app, port, backend, seconds=alive,
                              outcome="ready" if bound else "failed",
                              reason="" if bound else reason)
+    if bound:
+        _schedule_decision_warmup(app, port, backend)
     return bound, reason, alive
 
 
+def _schedule_decision_warmup(app, port: int, backend) -> None:
+    """Start a decision model's warm-up. Never lets a failure reach the bind."""
+    try:
+        from ainode.api.decide import schedule_decision_warmup
+        schedule_decision_warmup(app, port, backend)
+    except Exception:
+        logger.exception("could not schedule the decision warm-up on :%s", port)
+
+
 async def _bind_wait(app, port: int, backend, timeout: float = 300.0,
                      loading: int = 1):
     """The wait itself. The contract, the signals and the verdicts are documented
@@ -1796,6 +1812,9 @@ async def _await_primary_bind(app, config, loading: int = 1) -> bool:
                     "container that outlived the restart (up %s)",
                     adopted_primary.get("model") or model, port,
                     adopted_primary.get("uptime") or "unknown")
+        # The warm flag died with the old process. An engine that already
+        # compiled answers the warm-up at once, and one that did not needs it.
+        _schedule_decision_warmup(app, port, boot_engine)
         return True
     # Otherwise wait for it to serve, retrying it once if it died on the way up:
     # a node that comes back with its main model silently missing is the #96 shape.