Add POST /v1/systemone: TypeSafe's Jev wire format over the decision core

AGENTS.md

@@ -26,8 +26,9 @@ State / architecture / decisions / "why": Obsidian Vault, `AINode` (cluster ops:
 - **The agentic rubric (`ainode/bench/agentic/`, `scripts/ainode-bench.py agentic`) scores on mechanical verdicts only.** No judge model and no pass read by eye: a probe is decided by a regex, a parsed tool call, an executed subprocess or a compared call trace. Every checker is a module-level function of plain values (the reply text, the calls, the delivered tool results) so `tests/test_bench_agentic.py` can canned-response all of them with no network, and a probe that raises is one recorded failure rather than a dead run. **Group C executes model-written code on the machine driving the bench** (temporary directory, 60-second timeout, `sys.executable`), the same trade the harness bench makes when it runs an agent's edit. A run writes an `agentic` block and no `results` block, and a group `--groups` or `--quick` left out is absent from the score, never a zero. New rubric numbers go in that block: the hand-typed `rubric` key in three older records is a historical claim and must not be added to a new record. Probes and flags: `bench/agentic/README.md`.
 - **The decision bench (`ainode/bench/decide/`, `scripts/ainode-bench.py decide`) scores typed decisions against labels, and its confidence numbers are the product.** Accuracy is the weakest number in the block: a wrong answer at 0.95 is the failure mode, so every block carries the Brier score on the labeled option, an expected calibration error with the five-bin reliability table behind it, and the count of wrong answers surviving a 0.8 and a 0.9 gate. **A probability nobody reported is absent, never assumed** (such a row is in the accuracy, out of the calibration, and counted in `no_confidence`), **an item that failed is one row with an `error`** and never a wrong answer, and **cost is a posted vendor rate over reported tokens or `0`** for a local backend, never an estimate. `bench/decide/items.json` is repo data versioned next to its results: loading is strict, a malformed item is a load error rather than a skipped item, and a set's items must share one kind, question and option set because a set is one measurement. Backends are split `request()` / `parse()` as pure functions so `tests/test_bench_decide.py` pins every request shape and response shape with canned payloads and no network. **The TypeSafe key is never printed, never written into a record and never put in a note**: a run reports only which of `--api-key`, `$TYPESAFE_API_KEY` or `~/.jev_api_key` it came from. Sets, metrics and flags: `bench/decide/README.md`.
 - **The speech bench (`ainode/bench/speech/`, `scripts/ainode-bench.py speech`) scores against committed audio, and both halves of that are load-bearing.** A word error rate is only comparable over the same bytes, so the ten clips live in `bench/speech/clips/` as repo data (1.4 MB) rather than being synthesised per run, and **`clips.CLIPS_VERSION` is bumped on any edit to a text, a voice or a WAV**; `--generate-clips` rebuilds the set with macOS `say` plus `afconvert` and is a maintenance step a run never takes. The reference is the exact string handed to `say`, fixed before the run, and **nothing adjusts a reference after a transcript is seen**: a reference edited to match what a model said makes the rate a statement about the editor. The normaliser is part of the measurement, so it is versioned (`metrics.NORMALIZER_VERSION`) and the record carries BOTH rates, `wer` with number words folded to digits and `wer_orthographic` with case and punctuation only, because a transcript that heard every word and wrote "9" for "nine" is not a hearing error and one number alone hides which kind it was. Nothing is folded that changes a word: no stopword list, no stemming, no synonym map, no per-clip exception. `wer` is pooled over words, never a mean of per-clip rates. **A clip that failed is one row with an `error` and nulls for every number**, counted out of every rate, percentile and factor, never folded in as a 100 percent error rate: a transport failure inside a figure a reader takes as the model's is the one mistake this section can make. It is the one bench whose request body is not JSON (`client.py` assembles the multipart itself), so a run through a node's `:3000/v1` exercises the fleet's own audio path; the block is `bench/SCHEMA.md`.
-- **One proxy handler serves every forwarded inference path**: `proxy_to_vllm` is registered for `POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/responses`, `/v1/rerank`, `/v1/score`, `/v1/audio/transcriptions`, `/v1/audio/translations`, `/tokenize` and `/detokenize` (`GET /v1/models` is the federated union and does not forward; `POST /v1/embeddings` keeps its own handler because it validates the body first, and `/v1/decide` composes its own completions). Add a path by registering it on that handler, never by writing a second proxy: routing on the body's `model`, transport failover, the multimodal ordering below, SSE passthrough and header passthrough are all protocol-agnostic and already there. Two of the paths are NOT under `/v1` because vLLM does not serve them there (`/tokenize`, `/detokenize`), so the table is the authority on the path and not a prefix rule. **The two audio paths are the ones whose body is NOT JSON**: OpenAI's speech-to-text API is a `multipart/form-data` upload with the model id as a form field, so the handler reads it with `api/multipart.py::form_fields` over the body it already buffered (never `request.multipart()`, which consumes the stream the proxy still has to forward) and forwards the bytes UNCHANGED under the caller's own `Content-Type`: a multipart body is only parseable against the boundary in its own header, so re-encoding the parts hands the engine a body the forwarded header no longer describes. A multipart body that names no `model` is a 400 naming the field, never a fallback to this node's own model, which would send someone's audio to a chat engine. They are also the first pair whose existence is MODEL-CONDITIONAL: vLLM attaches its speech-to-text router only when the served model reports the `transcription` task, so no chat or pooling engine's `/openapi.json` lists them and the check below cannot be run on one. For a path like that, the evidence is the router's own declaration in the engine image the recipe pins (`entrypoints/openai/speech_to_text/api_router.py`), and the `openapi.json` check still applies the first time such a model actually serves. Forward `request.path_qs`, not `request.path`: Claude Code posts to `/v1/messages?beta=true`. This is a route table and not a catch-all: an unregistered path stays a 404, which is why a new path is added only after `curl http://<node>:<port>/openapi.json` on a real engine says the engine answers it.
-- **`/v1/decide`'s response shape is a contract, not an implementation detail** (`api/decide.py`). Its bench is written against the exact shape (`model`, `node`, `latency_ms`, `decisions[key] = {answer, confidence, distribution, latency_ms}`, `usage = {prompt_tokens, completion_tokens, calls}`), so a `200` always carries every question asked: a bad request is a `400` and an engine that cannot answer is a `503`, never a partial `decisions` block. Probabilities are keyed by the caller's OPTION strings, never by the letters used to constrain the engine. It is the one `/v1` path deliberately NOT on `proxy_to_vllm`, because it composes N grammar-constrained chat completions of its own from one request and has no caller body to forward; it still routes through the proxy's own `_routing_candidates` and the shared `app["client_session"]`, so never give it its own routing rule or HTTP stack. The constraint field is vLLM 0.27.1's `structured_outputs: {"choice": [...]}`: the legacy `guided_choice` is accepted by that image and then silently ignored, so sending it instead would produce free prose with no error.
+- **One proxy handler serves every forwarded inference path**: `proxy_to_vllm` is registered for `POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/responses`, `/v1/rerank`, `/v1/score`, `/v1/audio/transcriptions`, `/v1/audio/translations`, `/tokenize` and `/detokenize` (`GET /v1/models` is the federated union and does not forward; `POST /v1/embeddings` keeps its own handler because it validates the body first, and `/v1/decide` plus `/v1/systemone` compose their own completions). Add a path by registering it on that handler, never by writing a second proxy: routing on the body's `model`, transport failover, the multimodal ordering below, SSE passthrough and header passthrough are all protocol-agnostic and already there. Two of the paths are NOT under `/v1` because vLLM does not serve them there (`/tokenize`, `/detokenize`), so the table is the authority on the path and not a prefix rule. **The two audio paths are the ones whose body is NOT JSON**: OpenAI's speech-to-text API is a `multipart/form-data` upload with the model id as a form field, so the handler reads it with `api/multipart.py::form_fields` over the body it already buffered (never `request.multipart()`, which consumes the stream the proxy still has to forward) and forwards the bytes UNCHANGED under the caller's own `Content-Type`: a multipart body is only parseable against the boundary in its own header, so re-encoding the parts hands the engine a body the forwarded header no longer describes. A multipart body that names no `model` is a 400 naming the field, never a fallback to this node's own model, which would send someone's audio to a chat engine. They are also the first pair whose existence is MODEL-CONDITIONAL: vLLM attaches its speech-to-text router only when the served model reports the `transcription` task, so no chat or pooling engine's `/openapi.json` lists them and the check below cannot be run on one. For a path like that, the evidence is the router's own declaration in the engine image the recipe pins (`entrypoints/openai/speech_to_text/api_router.py`), and the `openapi.json` check still applies the first time such a model actually serves. Forward `request.path_qs`, not `request.path`: Claude Code posts to `/v1/messages?beta=true`. This is a route table and not a catch-all: an unregistered path stays a 404, which is why a new path is added only after `curl http://<node>:<port>/openapi.json` on a real engine says the engine answers it.
+- **`/v1/decide`'s response shape is a contract, not an implementation detail** (`api/decide.py`). Its bench is written against the exact shape (`model`, `node`, `latency_ms`, `decisions[key] = {answer, confidence, distribution, latency_ms}`, `usage = {prompt_tokens, completion_tokens, calls}`), so a `200` always carries every question asked: a bad request is a `400` and an engine that cannot answer is a `503`, never a partial `decisions` block. Probabilities are keyed by the caller's OPTION strings, never by the letters used to constrain the engine. It is the one `/v1` path deliberately NOT on `proxy_to_vllm`, because it composes N grammar-constrained chat completions of its own from one request and has no caller body to forward; it still routes through the proxy's own `_routing_candidates` and the shared `app["client_session"]`, so never give it its own routing rule or HTTP stack. The constraint field is vLLM 0.27.1's `structured_outputs: {"choice": [...]}`: the legacy `guided_choice` is accepted by that image and then silently ignored, so sending it instead would produce free prose with no error. The engine-facing half of a decision request is `decide.py::run_questions`, shared with `/v1/systemone`: one path to the engines and one way the probabilities are read, so a route decides the status and the shape and nothing else.
+- **`/v1/systemone` is TypeSafe's wire format and nothing else** (`api/systemone.py`). It exists so a client written for the hosted System One endpoint (Titanium's JDE `jevJudge({endpoint, model})`, browser-use's jev-ultrafast, the TypeSafe SDK, the playground) answers off a model on this fleet with its endpoint changed and nothing else, which makes the request and response shapes THEIRS and not ours to tidy: `{state, model, questions}` in, `{model, answers, usage: {input_tokens, output_tokens}, latency_ms}` out, a `choice` answering with the caller's own criteria key, a `noul` answering P(true), a `score` answering the expected level with a `legend` from position to level name, a `422` naming the field on a malformed request and a `503` when no node serves the model. JDE's `parseAnswers` discards the WHOLE answer set over one answer it cannot read, so every probability is finite and inside [0, 1] (`systemone.py::probability`) and a question the engine left unanswered is a `503`, never a 200 carrying an invented option. **`confidence` is NOT the picked option's probability**: the hosted service reports it chance corrected, `(n * p_max - 1) / (n - 1)`, which is INFERRED from every example in TypeSafe's published docs and SDK types (`systemone.py::normalized_confidence`) because no document states it, and a caller's bands are tuned against those numbers; the raw distribution goes out untouched beside it, so the formula is revisable against evidence and the numbers it came from are never lost. **A question may carry at most `TOP_LOGPROBS` criteria**, which is the engine request's ceiling and not the format's 255: only the top 20 labels come back with a probability, so a wider option set is a `422` naming the cap rather than a distribution missing its tail, and raising it means a second pass over the remaining labels, which is a measurement and not a constant. Translate in and out around `run_questions`: a gap here is never answered with a second engine path, a second routing rule or a second reading of the logprobs. **Calibration is the model's**, so nothing on this path rescales, tempers or corrects what the engine reported, and the way to find out what a local model's confidence is worth stays `scripts/ainode-bench.py decide`. `tests/test_systemone.py` carries JDE's own reader rule for rule and replays a real case from its blind set, read where it lives (`JDE_COMPLETION_CASES`, default `/Users/sem/code/jde/cases/`) and never copied in: it is someone else's measurement data, and the test skips where it is absent.
 - **A multimodal chat request consults the capability cache before routing** (`api/server.py::proxy_to_vllm`). A body carrying an `image_url` / `input_audio` / `video_url` / `file` part, or the Anthropic Messages spelling (an `image` / `document` block, including one nested in a `tool_result`), is never routed on the model id alone: order candidates accepting-first (cached `vision: true`), then never-probed, and drop instances cached `vision: false`; vLLM's `may be provided in one prompt` 400 is a routing miss, so record `vision: false` and fail over, while every other 4xx goes back to the caller untouched. A request with no media keeps the plain order (local hop first, then peers). There is ONE capability cache: `app["chat_caps_cache"]`, filled by `/api/models/caps` in `api/chat_routes.py`, which probes remote instances directly on their engine port. Never add a second.
 - **Every node-to-node request AINode makes carries the FLEET KEY, and there is one helper that puts it there** (`ainode/auth/fleet.py`: `fleet_headers(app)` off a running app, `fleet_key_headers(secret)` off a config). The key is `HMAC-SHA256(cluster_secret, "ainode-fleet-key-v1")`, so every node holding the secret computes the same one, the join flow already distributes it, rotation follows the secret, and NOTHING new is written to disk. The middleware accepts it as the caller id `fleet` (`auth/middleware.py::identify_caller`), reading the secret LIVE per request, so a node whose secret differs refuses its would-be peers exactly as it drops their datagrams. Auth was enabled-able and unusable on a cluster before this: a node with `auth.enabled` answered 401 to its own fan-outs, so the fleet ran open. Add a peer call and you add the header: an ENGINE port is not a peer call in this sense (the inference proxy, the capability probes and the embeddings route talk to a vLLM container, which never sees this middleware), and `POST /api/cluster/join` is keyless by construction. `tests/test_fleet_auth.py` WALKS THE SOURCE for peer URLs and fails on one whose function does not name the helper, with an exempt list that has to state a reason.
 - **A fresh install requires a key, and an update never changes an installed node's access control** (`scripts/install.sh`). The installer mints one key on a node with no `config.json`, stores the SHA-256 the way `AuthConfig` does, prints the plaintext ONCE in a box, and the summary reads "API protected, one key"; `AINODE_AUTH=off` keeps the old open behaviour and prints what that choice means. Both gates matter: minting on an existing home would lock out every client the operator already pointed at the node, with a key they never saw. Because of this, `ainode update` verifies the running release on `/api/health` and not `/api/status` (health is the one keyless route, which is why it carries `version`): reading a keyed route there made every update on a protected node pull, pin, restart and then report that it had not applied.