Skip to content

feat(chat): add opt-in conversation model selection for built-in Bots - #870

Open
samjulien wants to merge 6 commits into
mainfrom
codex/openbot-model-selector
Open

samjulien wants to merge 6 commits into
mainfrom
codex/openbot-model-selector

Conversation

@samjulien

@samjulien samjulien commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

What this changes

Built-in Bots currently run on the deployment's configured model. This adds opt-in model selection for ordinary one-to-one conversations, including the first message: enable Admin → Enterprise controls → Models → Allow model selection, then choose a model beside Send. The setting defaults to disabled.

  • Offer a small curated shortlist for the configured OpenAI or Anthropic API connection, retaining the configured default. No provider switching, credential setup, or full catalog discovery is introduced.
  • Save the choice per person, conversation, and Bot. Resolve it server-side and hold it fixed for each AG-UI run, including its tool loop. Selecting a model does not change other conversations or global configuration.
  • Save a new conversation's choice before starting its first message. Preserve drafts on failures; disable changing models during a run and prevent sending while a selection is being saved.
  • Enforce the existing model allowlist against the effective model and attribute the run to it. Disabling model selection hides the control and makes subsequent runs use the configured default; saved preferences are retained for re-enabling.

Why this is a small first phase

This implements the OpenBot selection, persistence, authorization, and execution boundary without requiring a routing service. It prepares three increments:

  1. This PR: direct API-backed model selection with existing credentials and defaults.
  2. Optional Switchback: reuse the UI and execution boundary, add managed/external router configuration, router credentials, and model capabilities. Keep direct connections available.
  3. Additional providers: support configured connections for providers such as Baseten, Fireworks, Together AI, or OpenRouter. Selection will need to identify a connection as well as a provider/model, since the same model may be reachable through different hosts.

Per-agent token usage and cost reporting remain a future increment. Effective-model attribution is in place here, but token counts, routing attempts, actual served models, pricing, and estimated versus billed costs are not. Switchback could centralize routing telemetry; direct and account-backed calls would still need their own instrumentation.

Account sign-in and scope

Signing in with ChatGPT/OpenAI or Claude/Anthropic does not enable this selector yet, even when the admin setting is on. Existing account-backed execution is preserved, with no fallback to API-key billing. Groups, externally connected agents, deployment-owned remote Bots, and custom API base URLs are also outside this PR's selector scope.

This is an integration gap rather than a claim that subscriptions cannot choose models:

  • PlanModel sends prompts/tools to the owned harness but has no selected-model field or model-capability API. It reports a plan identity rather than the resolved model.
  • The ChatGPT harness selects its model from BOT_MODEL or its Bot configuration and uses LangChain's private _ChatOpenAICodex implementation. It does not expose model discovery or a per-run model override. The official Codex app-server model/list contract provides account-aware options, but this harness does not currently use that transport.
  • The Claude model-only adapter deliberately removes forwarded SDK configuration. The Claude Python SDK supports model selection, but the owned harness would need an explicit validated model contract, capability discovery, and session/tool-continuation handling before exposing it here.

A follow-up can supply harness-specific available models, validate the selected model at the trusted server/harness boundary, and preserve account billing, refresh, and per-run consistency. API model lists should not be treated as subscription entitlements. Account behavior was inspected in source and covered by eligibility/plan tests; no live account-sign-in validation is claimed.

Where it runs

  • State across requests: migration 0066_lazy_dragon_man.sql adds nullable JSONB model_selection to intelligence_channel_mappings. The existing per-user/channel mapping holds {agentId, provider, model}. The admin switch uses the existing shared enterprise-settings store. Most added lines are the generated Drizzle schema snapshot.
  • Second replica: preference reads and run resolution read PostgreSQL; the admin setting uses the existing LISTEN/NOTIFY snapshot refresh. Neither choice depends on process-local conversation storage.
  • Serialization: preference writes use a transaction and row locks. Competing saves serialize; the last committed preference applies to subsequent runs. A running agent keeps its resolved model; this does not add a distributed whole-conversation turn lock.
  • Browser updates: existing query fetch/refetch behavior on navigation, focus, and selector open. No new WebSocket broadcast; an already-open tab may show cached options until refresh, while server validation remains authoritative.
  • Listeners/ports/schedules: none added; the new authenticated GET/PUT resources share the existing API ingress.

Boundary and audit

  • Channel membership, ownership of the thread mapping, Bot visibility, and actual built-in runtime eligibility are checked server-side. The client submits only a model ID or reset, never credentials or an inference endpoint.
  • Existing use-Bots capability checks remain. API-backed built-in runs enforce the model allowlist at execution; account/remote and suggestion paths keep their existing checks.
  • Effective model usage and model-policy refusals use the existing usage/audit stores. Admin setting changes use existing admin auditing. Preference validation/storage failures return bounded errors; this PR does not add separate audit rows for every rejected preference request.

Changelog

An Unreleased entry describes the default-off feature and additive migration.

Proof

  • Changed-file Biome formatting/lint and app/server TypeScript checks pass. Server-test types pass against the repository's existing 240-error baseline.
  • 184 focused server tests and 40 UI/start/composer tests pass, including authorization, persistence, default-off admin enforcement, account-plan exclusion, concurrent runs, provider tool loops, and draft preservation. Checks ran sequentially against an isolated test database with provider-network guards.
  • Server build and app production build pass (Vite reports large-chunk warnings).
  • Local browser verification covered desktop/mobile placement, model changes, reload persistence, reset to default, selection before the first message, and disabling/re-enabling via admin settings.
  • Real GPT-6 Luna inference and history replay succeeded, including a first message with a preselected model. Anthropic model execution was covered by protocol fixtures; live Anthropic/account-backed inference was not tested.
  • Full repository CI passed on 6316d7d7, including tests, real Chromium, all Python harness regressions, builds, migrations, types, platform startup checks, charts, and the final verify job. Conditional image jobs were skipped by the workflow.
  • Rebased onto main at 4bb0c0b4, preserving name personalization alongside per-run model resolution and the updated new-conversation flow. Post-rebase verification passed 225 focused tests (148 server, 77 UI), app/server types, formatting/lint, and both builds. Chromium and Anthropic SDK test fixes are now inherited from main.

@samjulien
samjulien force-pushed the codex/openbot-model-selector branch from d8f48b9 to 6316d7d Compare October 28, 2026 18:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant