Skip to content

Add System One routing and risk gating - #39

Merged
merefield merged 10 commits into
mainfrom
system-one-routing-risk
Sep 19, 2026
Merged

merefield merged 10 commits into
mainfrom
system-one-routing-risk

Conversation

@merefield

@merefield merefield commented Sep 19, 2026 •

Copy link
Copy Markdown
Owner

Why this matters

CLAI currently asks the main generative LLM to do several different jobs at once: understand the user's intent, generate a command, explain it, classify risk, and decide whether it is safe enough for risk_appetite auto-run behavior. That works, but intent routing and risk classification are not really generation problems. They are small, repeated judgments where we want typed answers, probabilities, speed, and predictable control flow.

This PR adds optional System One support for those judgments. When configured, CLAI can use TypeSafe / Jev-style APIs for fast typed decisions while keeping the existing LLM provider responsible for command generation and explanations.

The goal is not to replace the LLM. It is to make the surrounding control plane sharper:

  • Speed: small intent/risk judgments can be handled by a focused System One model instead of another broad generative prompt.
  • Cost: cheap typed judgments can avoid overusing the main LLM for routing and safety decisions.
  • Accuracy/control: risk and intent become explicit typed classifications with confidence, instead of buried inside prompt-following JSON generation.
  • Safer auto-run: risk_appetite no longer has to trust only the command-generating model's self-assessed risk.

What changed

  • Add optional System One-compatible config:
    • system_one_key
    • system_one_api
    • system_one_model
  • Add an internal /v1/systemone HTTP client for TypeSafe-compatible Choice judgments.
  • Use System One intent routing when configured, replacing the local ? heuristic for normal requests.
  • Use System One risk auditing after command generation:
    • A higher System One risk label overrides the LLM-provided risk.
    • Low-confidence risk audits force a confirmation prompt before execution.
    • The main LLM still generates the command and explanation.
  • Document TypeSafe Jev configuration and behavior.

Example config

system_one_key=ts-...
system_one_api=https://api.typesafe.ai/v1/systemone
system_one_model=jev-latest

If any of those settings are missing, CLAI keeps the existing behavior.

Testing

  • env GOCACHE=/tmp/clai-go-cache make check

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Confidence validation, clear-history routing, and sensitive error handling issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity

Open (2)
What changed in this PR

Adds optional System One integration for intent routing and command-risk auditing.

Changes:

  • Adds System One configuration and documentation.
  • Implements a TypeSafe-compatible HTTP client.
  • Integrates routing, risk upgrades, confirmation gating, and tests.
File Summary
README.md Documents System One configuration and behavior.
internal/​systemone/​client.go Adds the HTTP client. Critical (1 vote): validate confidence bounds. Moderate (1 vote): sanitize provider errors.
internal/​systemone/​client_test.go Tests client request and response handling.
internal/​config/​config.go Adds System One settings.
internal/​config/​config_test.go Tests configuration defaults.
internal/​app/​app.go Integrates routing and risk auditing. Critical (1 vote): validate confidence bounds. Moderate (1 vote): route clear-history requests through System One when configured.
internal/​app/​app_test.go Tests routing and confirmation behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/app/app.go Outdated
Comment thread internal/systemone/client.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved security and risk-gating findings must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity

Open (2)
Resolved since last review (2)

Comment thread internal/app/app.go
Comment thread internal/systemone/client.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

HTTPS redirects may forward the System One bearer token to another host or subdomain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (2)

Comment thread internal/systemone/client.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical response validation and intent-confidence gating issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread internal/systemone/client.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Low-confidence routing can delete history, and raw service error bodies can reach terminals unsanitized.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity

Open (2)
Resolved since last review (1)

Comment thread internal/app/app.go
Comment thread internal/systemone/client.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Valid TypeSafe responses with rounded probability distributions can currently be rejected.

Review effort: Balanced
Findings: None

Resolved since last review (2)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Relax probability-sum tolerance for two-decimal Choice responses

internal/​systemone/​client.go:243

TypeSafe reports Choice probabilities rounded to two decimal places, so a valid three-option distribution can be 0.33 + 0.33 + 0.33 = 0.99. The current 1e-6 tolerance rejects that normal API response, causing both intent routing and risk auditing to fail. Allow half a unit in the last reported decimal per option (while retaining the strict base tolerance).

@merefield
merefield merged commit 94e287b into main Sep 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants