Add POST /v1/systemone: TypeSafe's Jev wire format over the decision core

[webdevtodayjason]

A pile of clients are written against TypeSafe's hosted System One endpoint and not against ours. Titanium's JDE asks through jevJudge({endpoint, model}), browser-use's jev-ultrafast, TypeSafe's Python SDK and Kev's playground all post the same body to one path. I added that path to a node, so pointing any of them at a model on this fleet is one string change and no fork of the client.

The route is a translation layer and nothing else. /v1/decide already resolves the model, composes the grammar-constrained completions, fails over off a ghost claim and reads the distribution off the first token's logprobs. I pulled the engine-facing half of that out into decide.py::run_questions and both routes call it, so there is one path to the engines and one way the probabilities get read. /v1/decide's own response shape did not change, which matters because the decision bench is written against it.

Calibration is still the model's. Nothing on this path rescales, tempers or corrects what the engine reported, and the way to find out what a local model's confidence is worth stays scripts/ainode-bench.py decide.

The request

Same body the hosted endpoint takes, with Authorization: Bearer <AINode key>:

{
  "state": "Customer writes: the invoice PDF download 500s on every browser since your Tuesday release. We bill 400 clients on Friday.",
  "model": "ornith-ai/Ornith-1.5-35B-A3B-NVFP4",
  "questions": {
    "queue": {
      "type": "choice",
      "instructions": "Which queue should this ticket go to?",
      "criteria": {
        "billing": "an invoice, a charge or a refund",
        "bug": "the product did something it should not",
        "other": "none of these fit"
      }
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Does this ticket need a human today?",
      "criteria": {
        "true": "a person has to act on it today",
        "false": "it can wait or answer itself"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is it?",
      "criteria": {
        "none": "cosmetic",
        "some": "a workaround exists",
        "bad": "money or data is at risk"
      }
    }
  }
}

state takes text or JSON. A score can also hand its rubric over as an ordered list, "criteria": ["none", "some", "bad"], and a noul can leave criteria out or describe one side only. Every question shares the state and none of them sees another's answer, which was already true of /v1/decide.

The response

{
  "model": "ornith-ai/Ornith-1.5-35B-A3B-NVFP4",
  "answers": {
    "queue": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.985,
      "probabilities": {"billing": 0.99, "bug": 0.007, "other": 0.003}
    },
    "needs_human": {"type": "noul", "noul": 0.993},
    "severity": {
      "type": "score",
      "score": 1.87,
      "confidence": 0.835,
      "legend": {"0": "none", "1": "some", "2": "bad"},
      "probabilities": {"0": 0.02, "1": 0.09, "2": 0.89}
    }
  },
  "usage": {"input_tokens": 300, "output_tokens": 6},
  "latency_ms": 412.5
}

Round numbers to show the shape, not a measurement. A choice answers with the caller's own criteria key and spreads over exactly those keys. A noul answers P(true), whichever way the model went, so a confident no is a low noul rather than a high confidence. A score answers the expected level, so a model split between some and bad reports 1.5 and says more than either would, and legend maps the position back to the level name.

A malformed request is a 422 that names the field, in the error shape the rest of /v1 uses:

{"error": {"message": "question 'severity': a score's 'criteria' needs 2 to 10 ordered levels, got 1", "type": "invalid_request_error"}}

No node serving the model is a 503, and a question the engine could not answer is a 503 too. JDE's parseAnswers throws the whole answer set away over one answer it cannot read, so half an answer set would read as the judge failing anyway. A 200 carries every question or there is no 200.

Pointing JDE at a node

const judge = jevJudge({
  endpoint: "https://<node>:3443/v1/systemone",
  model: "ornith-ai/Ornith-1.5-35B-A3B-NVFP4",
});

The AINode key goes in TYPESAFE_API_KEY, because that is the one variable JDE reads a key from. http://<node>:3000/v1/systemone works the same; 3443 is the TLS listener on a node that has a certificate. It is a /v1 path, so with auth on it wants a key or a session, and the rate limiter covers it.

Its eval harness takes the endpoint too, so a JDE user can prove the route against a case set:

TYPESAFE_API_KEY=<AINode key> node scripts/eval.mjs \
  --cases cases/completion-check-blind.json \
  --endpoint https://<node>:3443/v1/systemone

I have not run the blind sets. Another worker owns those runs, and a number from them belongs in their record and not in this PR.

One thing a JDE user has to change besides the endpoint: timeout_ms. Production is 750 ms per ask and no chat model on this fleet will meet that. Every question is a full prefill of the state plus one constrained token, so hundreds of milliseconds per question is the good case and a cold engine is worse. Size it to the measured p95 of that node.

What needed care

confidence is not the picked option's probability. Across every example in TypeSafe's published docs and SDK types it reproduces as (n * p_max - 1) / (n - 1) for both choice and score, which is the winner's margin over chance: 1/n reports 0 and certainty reports 1, whatever n is. Kev's playground authors landed on the same formula for choice. No document states it, so normalized_confidence says out loud that it is inferred, and the raw distribution goes out untouched beside it. Reporting p_max instead would have read high on every band a JDE user has already tuned.

A question may carry at most 20 criteria, not the format's 255. One engine call reports probabilities for the top 20 tokens, so option 21 comes back at 0 whatever the model thought of it. That is a 422 naming the cap rather than a distribution that quietly lost its tail.

Every probability goes through one clamp that floors a NaN and keeps a rounded value inside [0, 1]. One bad number does not cost this route one answer, it costs it the whole reply, so the insurance is worth the two lines.

What did not map cleanly

• jev-latest gets a 503. model resolves exactly the way /v1/decide resolves it, so a client that leaves the hosted default in place is told no node is serving 'jev-latest'. I did not alias it to this node's default. Answering a request for a hosted model with a local one silently is worse than a 503 that names the id to change.
• More than 20 criteria has no answer today. Forward Observer's taxonomy has 47 categories, so this will come up. Raising the cap means a second pass over the remaining labels, and that is a measurement rather than a constant, so it is a follow-up and not a flag.
• usage is estimated when the engine reports none. With a usage block, the input count is the engine's own, summed over the calls, which counts the state once per question because that is what the engines read. With no usage block at all it falls back to the same 4 characters per token the bench falls back to when it cannot measure a tokenizer, and the docstring says so. Nothing here is a tokenizer.
• A noul may not rename its sides. Missing or partial criteria is fine, {"yes": ..., "no": ...} is a 422. The answer is P(true) and nothing else can stand in for it.
• The response carries no node. /v1/decide names the node that answered. This format is somebody else's contract, so it gets the four keys it defines and nothing extra.
• No top-level instructions. The Jev format puts the wording on each question, so the shared domain-guidance block /v1/decide takes has no equivalent here and the route sends none.
• GET /v1/systemone/models is not in here. GET /v1/models already answers the federated union, so a second list would be a second source for one fact.
• README is untouched. The endpoint reference there is a docs change on a file other branches are in right now. Worth a follow-up once this lands.

Tests

tests/test_systemone.py drives the same fake engine the decide tests drive, imported rather than copied, so both routes are proven against one engine's behaviour.

What it proves: every question type round-trips and its probabilities sum to one over exactly the caller's own keys; a noul's noul is P(true) in both directions; a score's score is the expected level and legend maps positions to level names; a malformed request is a 422 whose message names the field; the route answers 401 without a key when auth is on and gets past the gate with one; an engine with no logprobs still answers every question and reports no spread it did not measure; a NaN cannot poison the answer set. JDE's parseAnswers is in there rewritten rule for rule, and every live response goes through it, so "our 200 parses" is asserted rather than assumed. One test replays a real case from JDE's blind completion-check set, builds the question set the way completionCheck builds it, and reads the answers with that reader. The case file is read where it lives and never copied in, since it is somebody else's measurement data, so that test skips on a machine without it.

52 tests here. Whole suite green at 2813 passed, 2 skipped, 1 xfailed, and ruff clean.

Changelog text for the release PR

Added

• POST /v1/systemone, TypeSafe's System One (Jev) wire format over this node's own decision core, so a client written for the hosted endpoint answers off a model on this fleet with its endpoint changed and nothing else (Titanium's JDE, browser-use's jev-ultrafast, the TypeSafe SDK, the playground). Choice, noul and score questions in, typed answers out: a choice answers with the caller's own criteria key, a noul answers P(true), a score answers the expected level with a legend from position to level name. confidence is the hosted service's chance-corrected (n * p_max - 1) / (n - 1), inferred from its published examples, with the raw distribution reported beside it. A malformed request is a 422 naming the field, no node serving the model is a 503, and a 200 always carries every question asked. Calibration is the model's: nothing on this path rescales what the engine reported. A question carries at most 20 criteria, which is how many labels one engine call reports a probability for.