fix: prefer the user's saved provider credential over the pod env key

CHANGELOG.md

@@ -62,6 +62,7 @@
 
 ### Bug Fixes
 
+* prefer a credential saved in the box (the active connection / `/connect` entry) over `LITELLM_API_KEY` / `LITELLM_MASTER_KEY` on both the V1 server path and the OpenCode 2 plugin host; the model cache is now isolated per credential
 * **v2:** register Chat Completions models whose deployment accepts Anthropic `cache_control` with `compatibility.cacheFormat: "anthropic"` so prompt caching works for Claude behind LiteLLM
 * **v2:** spell reasoning-effort variants as `thinking` + `effort` for providers on an Anthropic runtime package, where `reasoningEffort` is ignored; models without adaptive thinking get none there
 * restore baseline reasoning-effort variants from sparse LiteLLM metadata (including GPT-6 Astra), respect explicit effort lists and disabled levels

README.md

@@ -78,7 +78,7 @@ also shows how to configure a provider directly in `opencode.json`.
 | 💵 **Real pricing** | Maps `input_cost_per_token` / `output_cost_per_token` (and cache read/write costs) from `/v1/model/info` into OpenCode's `cost` field, so the picker and `/cost` show what the proxy actually bills instead of `$0.00`. Models LiteLLM has no price for are left unpriced, not falsely marked free. |
 | 🧩 **Reasoning-effort variants** | When LiteLLM reports per-model effort support (`supports_low_reasoning_effort`, …), the plugin surfaces each level as a picker variant automatically. |
 | 🧭 **Responses-API routing** | Models LiteLLM reports with `mode: "responses"` are invoked through `/v1/responses` instead of `/v1/chat/completions`, with no extra configuration. OpenCode 2 overrides the runtime package per model; OpenCode 1 gets a `<provider>-responses` sibling. |
-| 🔐 **Auth-aware** | Honours `LITELLM_API_KEY` / `LITELLM_MASTER_KEY`, `providers.litellm.settings.apiKey`, OpenCode `/connect` credentials, and (OpenCode 2) a **LiteLLM** entry in the connect-provider dialog — key plus optional base URL, entered entirely from the web UI. |
+| 🔐 **Auth-aware** | Honours `providers.litellm.settings.apiKey`, OpenCode `/connect` credentials, the box auth store, and `LITELLM_API_KEY` / `LITELLM_MASTER_KEY` on both the V1 server path and the OpenCode 2 plugin host; on OpenCode 2 a **LiteLLM** entry in the connect-provider dialog (key plus optional base URL) connects a gateway entirely from the web UI. A saved credential beats the environment key. |
 | 🌐 **Gateway-friendly** | Supports provider `headers` (and the legacy `customHeaders` option) for proxies behind Cloudflare Access or other API gateways. |
 | 🧩 **Splittable catalog** | `includeModels` / `excludeModels` (glob patterns) let one LiteLLM proxy be divided into several OpenCode providers — e.g. by naming prefix — without hand-maintaining a model list. |
 | 🎚️ **Capability overrides** | `modelCapabilities` forces or retracts per-model capability flags (`supports_vision`, `supports_function_calling`, …) when `/v1/model/info` is unavailable or disagrees with your deployment. |
@@ -332,7 +332,7 @@ If your LiteLLM proxy requires a master key, expose it via any of these approach
 | Env var (alias) | `export LITELLM_MASTER_KEY=sk-...` |
 | Config | `"settings": { "apiKey": "{env:LITELLM_API_KEY}" }` |
 
-The env var path lets you commit `opencode.json` without leaking secrets. On OpenCode 1, `/connect` credentials are read from OpenCode's auth store (`~/.local/share/opencode/auth.json`) and applied to health checks, model discovery, and completion-time requests. On OpenCode 2, a credential saved through `/connect` is a first-class integration connection: the plugin registers the **LiteLLM** integration with an API-key form, uses the stored key for discovery, and lets OpenCode inject it into completion requests — the key never appears in `opencode.json` or provider settings. OpenCode 2 users can still use `settings.apiKey` with an environment substitution or set `LITELLM_API_KEY` on the background service as shown in the Quickstart.
+The env var path lets you commit `opencode.json` without leaking secrets. `/connect` credentials are read from OpenCode's auth store (`~/.local/share/opencode/auth.json`) and applied to health checks, model discovery, and completion-time requests on BOTH OpenCode 1 and OpenCode 2. On OpenCode 2, a credential saved through `/connect` is a first-class integration connection: the plugin registers the **LiteLLM** integration with an API-key form, uses the stored key for discovery, and lets OpenCode inject it into completion requests — the key never appears in `opencode.json` or provider settings. A credential the user saves in the box (the active connection or the auth-store entry) wins over `LITELLM_API_KEY` / `LITELLM_MASTER_KEY`; the environment is a box-level fallback, never an override of a saved key.
 
 #### Web UI connect (OpenCode 2)
 
@@ -342,7 +342,9 @@ With no configuration at all, `/connect` → **LiteLLM** is enough:
 2. Optionally fill the form: **Base URL** (e.g. `https://llm.example.com/v1`) and **Provider ID** (defaults to `litellm`; pick a different id when your config already defines a `litellm` provider, so the connected gateway gets its own entry). Reconnecting with a different id moves the gateway to that provider.
 3. Models appear in the picker without editing `opencode.json` or restarting.
 
-Precedence when several sources are configured: explicit provider/plugin settings always win. For the key, the environment (`LITELLM_API_KEY`/`LITELLM_MASTER_KEY`) is consulted before the dialog credential; for the base URL, the dialog answer beats `LITELLM_BASE_URL` and local port auto-detection. A provider defined in `opencode.json` (without its own key) picks the saved credential up on the next start, while providers created from the dialog update immediately — reconnecting re-points them in place, including a changed base URL or provider id.
+Precedence when several sources are configured: explicit provider/plugin settings always win. For the key, a credential saved in the box (the active connection or the `/connect` entry) beats the environment (`LITELLM_API_KEY`/`LITELLM_MASTER_KEY`), which is only a fallback; for the base URL, the dialog answer beats `LITELLM_BASE_URL` and local port auto-detection. A provider defined in `opencode.json` (without its own key) picks the saved credential up on the next start, while providers created from the dialog update immediately — reconnecting re-points them in place, including a changed base URL or provider id.
+
+The discovered model list is cached per credential: a short hash of the effective key (never the raw key) is part of every cache entry's key, so when the selected credential changes the next discovery starts fresh instead of serving another credential's model list. A `credential.switched` event also recomputes the effective key and reloads the provider registry when it changed — including the first credential saved after setup.
 
 ### Slow proxies (`LITELLM_REQUEST_TIMEOUT_MS`)
 

package.json

@@ -13,8 +13,7 @@
   ],
   "scripts": {
     "typecheck": "tsc --noEmit",
-    "test": "vitest run",
-    "prepublishOnly": "npm run typecheck"
+    "test": "vitest run"
   },
   "keywords": [
     "opencode",

src/plugin/index.ts

@@ -761,9 +761,14 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
         the plugin reads the stored key here and applies it to both its own
         discovery fetches and the completion-time provider options. Without it,
         a key-only proxy would fail discovery even though chat works.
+
+        Precedence rule: the stored credential — the user's explicit
+        choice made in the box — wins over the pod environment. The env is a
+        box-level fallback for users who have not saved a credential — never an
+        override of a user's saved key.
         */
         const storedKey = await getOpenCodeStoredApiKey(providerId)
-        const apiKey = configuredKey ?? envKey ?? storedKey
+        const apiKey = configuredKey ?? storedKey ?? envKey
         // Native `options.headers` and the legacy `customHeaders` both reach
         // discovery and the runtime, so a gateway that only sets one still
         // authenticates the plugin's own fetches.
@@ -829,9 +834,12 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
         // Identity includes the filter/capability/naming config: those
         // are baked into cached entries, so changing them must start a
         // fresh discovery instead of serving the old adjusted view.
+        // The cache key is isolated by the effective credential (a fingerprint,
+        // never the raw key): a credential change starts a fresh discovery
+        // instead of serving another credential's cached model list.
         const cacheKey = buildCacheKey(providerId, baseURL, filters, capabilities, {
           formatModelNames,
-        })
+        }, apiKey)
 
         // Remember how to reach this proxy so the `event` hook can
         // revalidate its cache in the background on new sessions.

src/plugin/v2.ts

@@ -25,6 +25,7 @@ import {
   writeModelCache,
 } from '../utils/model-cache'
 import { normalizeBaseURL, autoDetectLiteLLM } from '../utils/litellm-api'
+import { getOpenCodeStoredApiKey } from '../utils/opencode-auth'
 import { parseModelCapabilities } from '../utils/model-capabilities'
 import type { ModelCapabilities } from '../utils/model-capabilities'
 import {
@@ -341,20 +342,25 @@ async function makeProviderSource(
     CHAT_PROVIDER_ID
   const configuredBaseURL = readString(settings.baseURL)
   const connectedApiKey = connected?.apiKey
-  const configuredApiKey =
-    readString(providerSettings.apiKey) ??
-    readString(pluginOptions.apiKey) ??
-    process.env.LITELLM_API_KEY ??
-    process.env.LITELLM_MASTER_KEY
-  // A provider bound to an integration relies on OpenCode for its credential
-  // even before one is connected. The plugin's own integration only counts
-  // once it actually resolved a key, so env/config setups are unaffected.
-  const usesConnectionCredential =
-    !configuredApiKey && Boolean(provider?.integrationID ?? connectedApiKey)
+  // Precedence rule: an explicit settings/plugin-options apiKey is the
+  // operator's deliberate config and stays first; then a credential the user
+  // saved in the box — the ACTIVE connection or the `/connect` entry in the
+  // box auth store — which wins over the pod environment. The env is a
+  // box-level fallback for users who have not saved a credential, never an
+  // override of the user's explicit choice.
+  const settingsApiKey = readString(providerSettings.apiKey) ?? readString(pluginOptions.apiKey)
+  const envApiKey = process.env.LITELLM_API_KEY ?? process.env.LITELLM_MASTER_KEY
+  // The key OpenCode itself stored for this provider in the box auth store
+  // (/connect journey). A config-defined provider carries no integrationID, so
+  // the host-integration path above cannot see it — read the store directly,
+  // exactly like the v1 entrypoint does. Still ahead of the env: a saved
+  // credential is the user's explicit choice and beats the box-level fallback.
+  const storedKey = await getOpenCodeStoredApiKey(id)
+  const usesConnectionCredential = settingsApiKey === undefined && connectedApiKey !== undefined
   const integrationID =
     provider?.integrationID ??
-    (usesConnectionCredential && connectedApiKey ? LITELLM_INTEGRATION_ID : undefined)
-  const apiKey = configuredApiKey ?? connectedApiKey
+    (usesConnectionCredential ? LITELLM_INTEGRATION_ID : undefined)
+  const apiKey = settingsApiKey ?? connectedApiKey ?? storedKey ?? envApiKey
   const filters = readModelFilters(settings)
   const capabilities = parseModelCapabilities(settings.modelCapabilities)
   const compatibility = parseModelCompatibility(settings.modelCompatibility)
@@ -372,9 +378,12 @@ async function makeProviderSource(
   }
 
   const baseURL = normalizeBaseURL(detectedBaseURL)
+  // The cache key is isolated by the effective credential (a fingerprint, never
+  // the raw key): a credential change starts a fresh discovery instead of
+  // serving another credential's cached model list.
   const cacheKey = buildCacheKey(id, baseURL, filters, capabilities, {
     formatModelNames,
-  })
+  }, apiKey)
   const cacheEntry = readModelCacheEntry(cacheKey)
   let entries = cacheEntry?.models ?? null
   let responsesIds = cacheEntry?.responsesIds ?? []
@@ -531,16 +540,40 @@ async function refreshConnectionCredential(
   context: Context,
   source: ProviderSource,
 ): Promise<boolean> {
-  if (!source.integrationID || !source.usesConnectionCredential) return false
+  // The gate is the integration's presence, not the current credential choice:
+  // setup may run before the user saves a credential, and the first save must
+  // refresh this source instead of leaving it on the environment key.
+  if (!source.integrationID) return false
 
   try {
     const connection = await context.integration.connection.active(source.integrationID)
     const credential = connection
       ? await context.integration.connection.resolve(connection)
       : undefined
-    const apiKey = credential?.type === 'key' ? credential.key : undefined
-    if (apiKey === source.apiKey) return false
-    source.apiKey = apiKey
+    const connectionKey = credential?.type === 'key' ? credential.key : undefined
+    const settingsApiKey = readString(source.settings.apiKey)
+    let nextKey: string | undefined
+    let nextUsesConnection: boolean
+    if (connectionKey !== undefined) {
+      // An explicit settings apiKey stays first; a connection credential takes
+      // over the environment/auto-detected key.
+      nextKey = settingsApiKey ?? connectionKey
+      nextUsesConnection = settingsApiKey === undefined
+    } else {
+      // The connection credential is gone (the connection was removed or its
+      // credential cleared): fall back through the stored credential / env key
+      // — the same precedence `makeProviderSource` uses — instead of retaining
+      // the previous connection key, which `runtimeSettings` would otherwise
+      // materialize into provider settings.
+      const storedKey = await getOpenCodeStoredApiKey(source.id)
+      const envKey = process.env.LITELLM_API_KEY ?? process.env.LITELLM_MASTER_KEY
+      nextKey = settingsApiKey ?? storedKey ?? envKey
+      nextUsesConnection = false
+    }
+    if (nextKey === source.apiKey && nextUsesConnection === source.usesConnectionCredential) return false
+    source.apiKey = nextKey
+    source.usesConnectionCredential = nextUsesConnection
+    source.credentialReloadRequired = true
     return true
   } catch (error) {
     const message = error instanceof Error ? error.message : String(error)

src/utils/model-cache.ts

@@ -111,8 +111,16 @@ export function buildCacheKey(
   filters: { includeModels?: string[]; excludeModels?: string[] },
   capabilities: Record<string, Record<string, boolean>>,
   naming: { formatModelNames?: boolean } = {},
+  credential?: string,
 ): string {
-  const base = `${providerId}@${baseURL}`
+  let base = `${providerId}@${baseURL}`
+  // The credential fingerprint: a short hash (never the raw key) so a
+  // credential change isolates the cached model list — a key that exposes a
+  // different set of models must start a fresh discovery instead of serving
+  // another credential's entries.
+  if (credential) {
+    base += `@${createHash('sha256').update(credential).digest('hex').slice(0, 12)}`
+  }
   const rawNames = naming.formatModelNames === false
   const hasAdjustments =
     (filters.includeModels?.length ?? 0) > 0 ||

test/model-cache.test.ts

@@ -144,6 +144,21 @@ describe('buildCacheKey', () => {
     ).toBe(KEY)
   })
 
+  it('isolates the key by credential fingerprint', () => {
+    // Regression for the CodeRabbit review finding: a model list cached with
+    // one credential must never be served to another — keys expose different
+    // model sets, so a credential change starts a fresh discovery.
+    const without = buildCacheKey('litellm', 'http://localhost:4000', {}, {})
+    const keyA = buildCacheKey('litellm', 'http://localhost:4000', {}, {}, {}, 'sk-key-a')
+    const keyA2 = buildCacheKey('litellm', 'http://localhost:4000', {}, {}, {}, 'sk-key-a')
+    const keyB = buildCacheKey('litellm', 'http://localhost:4000', {}, {}, {}, 'sk-key-b')
+    expect(keyA).toBe(keyA2)
+    expect(keyA).not.toBe(keyB)
+    expect(keyA).not.toBe(without)
+    // The fingerprint never carries the raw key.
+    expect(keyA).not.toContain('sk-key-a')
+  })
+
   it('changes the key when capability overrides are added or changed', () => {
     // Regression for the CodeRabbit review finding: a model previously
     // cached with `supports_vision: false` must not be served again