From 9e6529d5a754a7d1f4a9088e07efc6e2e26c529e Mon Sep 17 00:00:00 2001 From: Mateusz Kowalski Date: Sun, 18 Oct 2026 21:55:00 +0200 Subject: [PATCH 1/2] feat(v2): enable prompt caching for Claude behind LiteLLM OpenCode 2's OpenAI-compatible Chat Completions runtime only places Anthropic-style `cache_control` breakpoints when the model declares `compatibility.cacheFormat`. Discovered models never did, so Claude served through LiteLLM lost prompt caching on OpenCode 2 (#42). Record per model whether `/v1/model/info` lists `cache_control` under `supported_openai_params` and register Chat Completions models that do with `compatibility.cacheFormat: "anthropic"`. Only the Chat Completions runtimes read the setting: the Anthropic Messages runtime adds its own breakpoints and the Responses runtimes do not use it, so models on those packages are left alone. The hint travels with the discovery result and the disk cache (`CACHE_VERSION` 4). Add the `modelCompatibility` provider option, a per-model map like `modelCapabilities`, to set `cacheFormat` to `"anthropic"`, `"alibaba"` or `null`. OpenCode's own `providers..models..compatibility` is merged over the plugin's value, so it can add or change a key but not remove one the plugin inferred; `null` here does that. Values are validated when parsed, because OpenCode validates plugin-registered models only when it serializes them, where one bad value fails the whole model list. It accepts only `cacheFormat`; it is applied when models are registered, so it is not part of the cache key. Fixes #42 Signed-off-by: Mateusz Kowalski Assisted-By: Claude Fable 5.1 --- CHANGELOG.md | 2 + README.md | 29 ++ src/plugin/index.ts | 18 +- src/plugin/v2.ts | 69 ++++- src/utils/model-cache.ts | 52 +++- src/utils/model-compatibility.ts | 65 +++++ test/model-cache.test.ts | 26 ++ test/model-compatibility.test.ts | 67 +++++ test/plugin-v1.test.ts | 61 ++++- test/plugin-v2.test.ts | 453 ++++++++++++++++++++++++++++++- 10 files changed, 808 insertions(+), 34 deletions(-) create mode 100644 src/utils/model-compatibility.ts create mode 100644 test/model-compatibility.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index b82aa49..ffe3f3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,12 +48,14 @@ ### Bug Fixes +* **v2:** register Chat Completions models whose deployment accepts Anthropic `cache_control` with `compatibility.cacheFormat: "anthropic"` so prompt caching works for Claude behind LiteLLM * restore baseline reasoning-effort variants from sparse LiteLLM metadata (including GPT-6 Astra), respect explicit effort lists and disabled levels * avoid inferring reasoning-effort variants from negative-only flags or unsupported request parameters * constrain inferred GPT-5 Pro effort variants to provider-supported levels, including aliased and dated deployments ### Features +* **v2:** add the `modelCompatibility` option to set or remove the inferred `cacheFormat` per model * route models LiteLLM reports with `mode: "responses"` through the OpenAI Responses API (per-model runtime package on OpenCode 2; a `-responses` sibling on OpenCode 1) * support the OpenCode 2 API with the OpenCode 1 server entrypoint * add `formatModelNames` option to keep raw model ids in the picker diff --git a/README.md b/README.md index 609b31e..668413f 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ also shows how to configure a provider directly in `opencode.json`. | 🌐 **Gateway-friendly** | Supports provider `headers` (and the legacy `customHeaders` option) for proxies behind Cloudflare Access or other API gateways. | | 🧩 **Splittable catalog** | `includeModels` / `excludeModels` (glob patterns) let one LiteLLM proxy be divided into several OpenCode providers — e.g. by naming prefix — without hand-maintaining a model list. | | 🎚️ **Capability overrides** | `modelCapabilities` forces or retracts per-model capability flags (`supports_vision`, `supports_function_calling`, …) when `/v1/model/info` is unavailable or disagrees with your deployment. | +| 💾 **Prompt caching (OpenCode 2.0.28+)** | Chat models whose deployment accepts Anthropic `cache_control` markers are registered with `compatibility.cacheFormat: "anthropic"`, so OpenCode 2 places cache breakpoints as it does for a native Anthropic provider. `modelCompatibility` corrects this per model. | | ⏱️ **Bounded startup** | Health checks fail fast (3 s); discovery fetches are capped at **15 s** (configurable via `LITELLM_REQUEST_TIMEOUT_MS`) for slow remote proxies. V2 model refresh runs in the background. | | 📝 **OpenCode logging** | V1 logs go through OpenCode's log API; V2 logs use the V2 plugin host's console logging. | | 🤝 **Non-destructive merge** | Only adds models you don't already have configured. Hand-curated entries are preserved verbatim. | @@ -423,6 +424,33 @@ By default the plugin prettifies each discovered id into a display name — `ant - Like the other options above, `formatModelNames` is part of the cache identity, so toggling it triggers a fresh discovery on the next start instead of serving previously cached names. - To rename just a handful of models while keeping smart formatting for the rest, use the per-model `name` override described in [Overriding or curating individual models](#overriding-or-curating-individual-models-optional). +### Prompt caching (`modelCompatibility`) + +On OpenCode 2.0.28+ (earlier 2.x releases ignore the setting), the OpenAI-compatible Chat Completions runtime only places Anthropic-style `cache_control` breakpoints when a model declares `compatibility.cacheFormat: "anthropic"`. The plugin sets that for every discovered Chat Completions model whose `/v1/model/info` entry lists `cache_control` under `supported_openai_params` — which LiteLLM reports for Anthropic and Vertex AI Anthropic deployments — so prompt caching keeps working for Claude behind the proxy. Only the Chat Completions runtimes read this setting: the Anthropic Messages runtime adds its own cache breakpoints, and the Responses runtimes do not use it, so models on those packages are left alone. + +To correct the inference per model id, set `cacheFormat` to `"anthropic"`, `"alibaba"`, or `null` to remove it: + +```jsonc +{ + "providers": { + "litellm": { + "settings": { + "baseURL": "http://localhost:4000/v1", + "modelCompatibility": { + "bedrock/claude-sonnet-4": { "cacheFormat": "anthropic" }, + "gateway/claude-opus": { "cacheFormat": null } + } + } + } + } +} +``` + +- Keys are exact model ids as they appear in `/v1/models` (not globs). Only `cacheFormat`, the one key the plugin sets, is accepted. +- OpenCode's own `providers..models..compatibility` is merged over the plugin's value, so it can add or change a key but not remove one the plugin inferred; that is what `null` here is for. +- Overrides are applied when models are registered, not during discovery, so changing them does not start a new discovery. +- Neither applies on OpenCode 1. + ## 🔧 How it works ```mermaid @@ -601,6 +629,7 @@ src/ │ ├── model-cache.ts # stale-while-revalidate on-disk model cache │ ├── model-filter.ts # includeModels/excludeModels glob filtering │ ├── model-capabilities.ts # per-model capability flag overrides +│ ├── model-compatibility.ts # cacheFormat inference and per-model overrides │ └── opencode-auth.ts # fallback to OpenCode's /connect-stored credentials └── plugin/ ├── index.ts # OpenCode 1 server entry and shared discovery logic diff --git a/src/plugin/index.ts b/src/plugin/index.ts index 8b4dc3f..4d0f62e 100644 --- a/src/plugin/index.ts +++ b/src/plugin/index.ts @@ -14,6 +14,7 @@ import { import { requiresResponsesAPI } from '../utils/transport' import type { LiteLLMModel, LiteLLMModelInfo } from '../types' import { getOpenCodeStoredApiKey } from '../utils/opencode-auth' +import type { ModelHints } from '../utils/model-cache' import { buildCacheKey, readModelCacheEntry, @@ -331,11 +332,13 @@ export function toConfigModel( * `models` is the `{ id -> entry }` map written into an OpenCode provider * config (or translated into `Model.Info` on OpenCode 2). `responsesIds` * lists the subset that must be invoked through the OpenAI Responses API; - * callers route those accordingly without inspecting each entry. + * callers route those accordingly without inspecting each entry. `hints` + * carries per-model upstream facts OpenCode 2 maps to `compatibility`. */ export interface DiscoveredModels { models: Record responsesIds: string[] + hints: Record } /** @@ -343,7 +346,7 @@ export interface DiscoveredModels { * * Pure with respect to plugin config: it performs the network calls, * classifies + formats each model, and returns the built entries plus the - * ids that require the Responses API. The provider's + * ids that require the Responses API and per-model hints. The provider's * `includeModels`/`excludeModels` filters, `modelCapabilities` overrides and * `formatModelNames` choice are applied here (not at merge time) so every * path that persists or serves a cache — cold discovery and background @@ -411,6 +414,7 @@ export async function discoverModels( const built: Record = {} const responsesIds: string[] = [] + const hints: Record = {} let skipped = 0 let wildcards = 0 let filtered = 0 @@ -440,6 +444,10 @@ export async function discoverModels( } built[model.id] = entry if (requiresResponsesAPI(enriched)) responsesIds.push(model.id) + const params = info?.supported_openai_params + hints[model.id] = { + acceptsCacheControl: Array.isArray(params) && params.includes('cache_control'), + } } if (unmatched.length > 0) { @@ -471,7 +479,7 @@ export async function discoverModels( ')', ) - return { models: built, responsesIds } + return { models: built, responsesIds, hints } } /** @@ -655,7 +663,7 @@ async function backgroundRefresh(cacheKey: string): Promise { DISCOVERY_TIMEOUT_MS, ) if (discovered && Object.keys(discovered.models).length > 0) { - writeModelCache(cacheKey, discovered.models, discovered.responsesIds) + writeModelCache(cacheKey, discovered.models, discovered.responsesIds, discovered.hints) log( 'info', `[opencode-litellm] Background-refreshed model cache for ${ctx.baseURL} (${Object.keys(discovered.models).length} models)`, @@ -913,7 +921,7 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => { customHeaders, ) injectedModelIds.set(cacheKey, new Set(added)) - writeModelCache(cacheKey, discovered.models, discovered.responsesIds) + writeModelCache(cacheKey, discovered.models, discovered.responsesIds, discovered.hints) } } }, diff --git a/src/plugin/v2.ts b/src/plugin/v2.ts index ad0b9ee..1223491 100644 --- a/src/plugin/v2.ts +++ b/src/plugin/v2.ts @@ -21,10 +21,24 @@ import { import { normalizeBaseURL, autoDetectLiteLLM } from '../utils/litellm-api' import { parseModelCapabilities } from '../utils/model-capabilities' import type { ModelCapabilities } from '../utils/model-capabilities' +import { + inferCompatibility, + parseModelCompatibility, + resolveCompatibility, +} from '../utils/model-compatibility' +import type { ModelCompatibility } from '../utils/model-compatibility' import type { ModelFilters } from '../utils/model-filter' +import type { ModelHints } from '../utils/model-cache' const CHAT_PROVIDER_ID = 'litellm' const OPENAI_COMPATIBLE_PACKAGE = '@opencode/ai/providers/openai-compatible' +// Of the packages a LiteLLM provider uses, only the Chat Completions ones read +// `compatibility.cacheFormat`; `openai` (Responses by default), the Responses +// packages and the Anthropic packages do not. +const CHAT_COMPLETIONS_PACKAGES = new Set([ + OPENAI_COMPATIBLE_PACKAGE, + '@opencode/ai/providers/openai/chat', +]) // Runtime package that talks to `/v1/responses`. OpenCode 2 lets one model // override the provider's package, so Responses models stay under their // original provider id and no sibling provider is needed here. @@ -78,6 +92,7 @@ interface ProviderSource { pendingPublication: boolean filters: ModelFilters capabilities: ModelCapabilities + compatibility: ModelCompatibility formatModelNames: boolean cacheKey: string models: Model.Info[] @@ -111,12 +126,21 @@ function numberOrZero(value: unknown): number { return typeof value === 'number' && Number.isFinite(value) ? value : 0 } +interface ToProviderModelsOptions { + /** The provider's runtime package; models not overridden inherit it. */ + providerPackage: string + responsesIds: ReadonlySet + hints: Record + compatibility: ModelCompatibility +} + function toProviderModels( providerID: string, entries: Record, - responsesIds: ReadonlySet = new Set(), + options: ToProviderModelsOptions, ): Model.Info[] { const provider = Provider.ID.make(providerID) + const { responsesIds, hints, compatibility: overrides } = options return Object.entries(entries).flatMap(([modelID, value]) => { const entry = asRecord(value) @@ -152,6 +176,11 @@ function toProviderModels( id: id as Model.VariantID, settings: asRecord(settings), })) as Model.Info['variants'] + const pkg = responsesIds.has(modelID) ? RESPONSES_PACKAGE : options.providerPackage + const compatibility = resolveCompatibility( + inferCompatibility(hints[modelID], CHAT_COMPLETIONS_PACKAGES.has(pkg)), + Object.hasOwn(overrides, modelID) ? overrides[modelID] : undefined, + ) return [ { ...defaults, @@ -159,6 +188,11 @@ function toProviderModels( // runtime package for itself, so it keeps the provider id while // still reaching the Responses API. ...(responsesIds.has(modelID) ? { package: RESPONSES_PACKAGE } : {}), + // `cacheFormat` exists on the host's `Model.Compatibility` (2.0.28+) + // but not in the 2.0.15 schema this package compiles against. + ...(compatibility + ? { compatibility: compatibility as unknown as Model.Info['compatibility'] } + : {}), name: readString(entry.name) ?? modelID, limit: { context: numberOrDefault(limit.context, defaults.limit.context), @@ -228,6 +262,7 @@ function runtimeSettings(source: ProviderSource): Record { 'includeModels', 'excludeModels', 'modelCapabilities', + 'modelCompatibility', 'formatModelNames', 'providerID', ]) { @@ -277,6 +312,7 @@ async function makeProviderSource( const apiKey = configuredApiKey ?? connectedApiKey const filters = readModelFilters(settings) const capabilities = parseModelCapabilities(settings.modelCapabilities) + const compatibility = parseModelCompatibility(settings.modelCompatibility) const formatModelNames = readFormatModelNames(settings) const detectedBaseURL = configuredBaseURL ? normalizeBaseURL(configuredBaseURL) @@ -293,6 +329,7 @@ async function makeProviderSource( const cacheEntry = readModelCacheEntry(cacheKey) let entries = cacheEntry?.models ?? null let responsesIds = cacheEntry?.responsesIds ?? [] + let hints = cacheEntry?.hints ?? {} if (!entries || Object.keys(entries).length === 0) { const discovered = await withTimeout( @@ -310,10 +347,12 @@ async function makeProviderSource( if (discovered && Object.keys(discovered.models).length > 0) { entries = discovered.models responsesIds = discovered.responsesIds - writeModelCache(cacheKey, entries, responsesIds) + hints = discovered.hints + writeModelCache(cacheKey, entries, responsesIds, hints) } else { entries = null responsesIds = [] + hints = {} } } else { logInfo( @@ -321,11 +360,12 @@ async function makeProviderSource( ) } + const providerPackage = provider?.package || OPENAI_COMPATIBLE_PACKAGE return { id, name: provider?.name || 'LiteLLM (proxy)', activation: provider?.activation ?? 'enabled', - package: provider?.package || OPENAI_COMPATIBLE_PACKAGE, + package: providerPackage, settings, headers: customHeaders, integrationID: provider?.integrationID, @@ -339,9 +379,15 @@ async function makeProviderSource( customHeaders: Object.keys(customHeaders).length > 0 ? customHeaders : undefined, filters, capabilities, + compatibility, formatModelNames, cacheKey, - models: toProviderModels(id, entries ?? {}, new Set(responsesIds)), + models: toProviderModels(id, entries ?? {}, { + providerPackage, + responsesIds: new Set(responsesIds), + hints, + compatibility, + }), ownedModelIds: new Set(), } } @@ -395,13 +441,14 @@ async function refreshProviderSource( return } - const models = toProviderModels( - source.id, - discovered.models, - new Set(discovered.responsesIds), - ) + const models = toProviderModels(source.id, discovered.models, { + providerPackage: source.package, + responsesIds: new Set(discovered.responsesIds), + hints: discovered.hints, + compatibility: source.compatibility, + }) if (JSON.stringify(models) === JSON.stringify(source.models)) { - writeModelCache(source.cacheKey, discovered.models, discovered.responsesIds) + writeModelCache(source.cacheKey, discovered.models, discovered.responsesIds, discovered.hints) source.refreshRequired = false return } @@ -415,7 +462,7 @@ async function refreshProviderSource( throw error } - writeModelCache(source.cacheKey, discovered.models, discovered.responsesIds) + writeModelCache(source.cacheKey, discovered.models, discovered.responsesIds, discovered.hints) source.refreshRequired = false logInfo( `[opencode-litellm] Refreshed ${models.length} models for provider "${source.id}"; the provider registry was reloaded.`, diff --git a/src/utils/model-cache.ts b/src/utils/model-cache.ts index 2e4e446..8d80441 100644 --- a/src/utils/model-cache.ts +++ b/src/utils/model-cache.ts @@ -3,12 +3,25 @@ import { readFileSync, mkdirSync, writeFileSync, renameSync, rmSync } from 'node import { homedir, tmpdir } from 'node:os' import { join } from 'node:path' +/** + * Facts about a discovered model's upstream that OpenCode 2 turns into + * `Model.Info.compatibility`. They ride alongside the config-shaped model + * entries (which have no field for them) through discovery and the cache. + */ +export interface ModelHints { + /** + * The deployment accepts Anthropic-style `cache_control` markers on + * Chat Completions (`supported_openai_params` lists `cache_control`). + */ + acceptsCacheControl: boolean +} + /** * Bump when the shape of a cached model entry changes so stale caches * from older plugin versions are ignored rather than deserialized into * an incompatible structure. */ -const CACHE_VERSION = 3 +const CACHE_VERSION = 4 /** * Maximum age of a cache entry before it's treated as a miss. Without a @@ -29,6 +42,11 @@ interface ModelCacheFile { * no transport marker (their shape is the user-facing OpenCode config). */ responsesIds?: string[] + /** + * Per-model facts about the upstream that OpenCode 2 needs but that have + * no place in a provider-config entry (see `ModelHints`). + */ + hints?: Record } /** @@ -134,6 +152,18 @@ function normalizeResponsesIds(value: unknown): string[] { return value.filter((id): id is string => typeof id === 'string') } +/** Keep only well-formed entries of a cached `hints` map. */ +function normalizeHints(value: unknown): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) return {} + const out: Record = {} + for (const [id, hint] of Object.entries(value as Record)) { + if (!hint || typeof hint !== 'object' || Array.isArray(hint)) continue + const { acceptsCacheControl } = hint as Record + out[id] = { acceptsCacheControl: acceptsCacheControl === true } + } + return out +} + /** * Read cached model entries for a cache key. Returns `null` on any * problem (missing file, parse error, version mismatch, or an entry @@ -155,14 +185,18 @@ export function readModelCache( } /** - * Read the models and Responses-API model ids for a cache key in one - * parse. Returns `null` under the same conditions as - * {@link readModelCache} (missing/expired cache), so the two can't + * Read the models, Responses-API model ids and model hints for a cache key + * in one parse. Returns `null` under the same conditions as + * {@link readModelCache} (missing/expired cache), so the three can't * disagree within a single lookup. */ export function readModelCacheEntry( cacheKey: string, -): { models: Record; responsesIds: string[] } | null { +): { + models: Record + responsesIds: string[] + hints: Record +} | null { const parsed = readCacheFile(cacheKey) if (!parsed) return null if ( @@ -171,7 +205,11 @@ export function readModelCacheEntry( ) { return null } - return { models: parsed.models, responsesIds: normalizeResponsesIds(parsed.responsesIds) } + return { + models: parsed.models, + responsesIds: normalizeResponsesIds(parsed.responsesIds), + hints: normalizeHints(parsed.hints), + } } /** @@ -204,6 +242,7 @@ export function writeModelCache( cacheKey: string, models: Record, responsesIds: readonly string[] = [], + hints: Record = {}, ): void { try { mkdirSync(cacheDir(), { recursive: true }) @@ -212,6 +251,7 @@ export function writeModelCache( savedAt: Date.now(), models, responsesIds: [...responsesIds], + hints, } // Write to a temp file and rename so concurrent OpenCode processes // never observe a partially-written JSON file (rename is atomic on diff --git a/src/utils/model-compatibility.ts b/src/utils/model-compatibility.ts new file mode 100644 index 0000000..15cc4f5 --- /dev/null +++ b/src/utils/model-compatibility.ts @@ -0,0 +1,65 @@ +import type { ModelHints } from './model-cache' + +/** + * Values `cacheFormat` accepts. Validated here because OpenCode validates + * plugin-registered models only when it serializes them, where one bad + * value fails the whole model list. + */ +const CACHE_FORMATS: ReadonlySet = new Set(['anthropic', 'alibaba']) + +/** `null` removes the `cacheFormat` the plugin inferred. */ +export interface CompatibilityOverride { + cacheFormat: string | null +} + +/** Per-model compatibility overrides keyed by exact model id. */ +export type ModelCompatibility = Record + +/** + * Defensively parse the `modelCompatibility` provider option. Non-object + * input, non-object per-model values and values outside the accepted set + * are dropped; `null` is kept. + */ +export function parseModelCompatibility(raw: unknown): ModelCompatibility { + if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return {} + const out: ModelCompatibility = {} + for (const [modelId, entry] of Object.entries(raw as Record)) { + if (!entry || typeof entry !== 'object' || Array.isArray(entry)) continue + const { cacheFormat } = entry as Record + if (cacheFormat === null || (typeof cacheFormat === 'string' && CACHE_FORMATS.has(cacheFormat))) { + out[modelId] = { cacheFormat } + } + } + return out +} + +/** + * Compatibility OpenCode 2 needs for a discovered model, from what the + * proxy reported about its upstream. Only the Chat Completions runtimes + * read `cacheFormat`: Anthropic Messages adds its own cache breakpoints + * and the Responses runtimes do not use the setting. + */ +export function inferCompatibility( + hints: ModelHints | undefined, + chatCompletions: boolean, +): Record { + if (chatCompletions && hints?.acceptsCacheControl) return { cacheFormat: 'anthropic' } + return {} +} + +/** + * Overlay a user override on inferred compatibility. `null` deletes the + * key. Returns `undefined` when nothing remains so callers can leave + * `compatibility` unset. + */ +export function resolveCompatibility( + inferred: Record, + override?: CompatibilityOverride, +): Record | undefined { + const out: Record = { ...inferred } + if (override) { + if (override.cacheFormat === null) delete out.cacheFormat + else out.cacheFormat = override.cacheFormat + } + return Object.keys(out).length > 0 ? out : undefined +} diff --git a/test/model-cache.test.ts b/test/model-cache.test.ts index c3f0dee..ef598d0 100644 --- a/test/model-cache.test.ts +++ b/test/model-cache.test.ts @@ -75,9 +75,35 @@ describe('model cache', () => { expect(readModelCacheEntry('entry@http://x')).toEqual({ models: { chat: {}, responses: {} }, responsesIds: ['responses'], + hints: {}, }) }) + it('round-trips model hints and drops malformed ones', () => { + writeModelCache('hints@http://x', { a: {}, b: {} }, [], { + a: { acceptsCacheControl: true }, + b: { acceptsCacheControl: false }, + }) + expect(readModelCacheEntry('hints@http://x')?.hints).toEqual({ + a: { acceptsCacheControl: true }, + b: { acceptsCacheControl: false }, + }) + rewriteCacheFile('hints@http://x', (parsed) => { + parsed.hints = { a: { acceptsCacheControl: 'yes' }, b: null, c: [] } + }) + expect(readModelCacheEntry('hints@http://x')?.hints).toEqual({ a: { acceptsCacheControl: false } }) + }) + + it('treats caches that predate the model hints as a miss', () => { + writeModelCache(KEY, { m: {} }, ['m']) + rewriteCacheFile(KEY, (parsed) => { + parsed.version = 3 + delete parsed.hints + }) + expect(readModelCacheEntry(KEY)).toBeNull() + expect(readModelCache(KEY)).toBeNull() + }) + it('ignores cache files written by a different plugin version', () => { writeModelCache(KEY, { m: {} }) rewriteCacheFile(KEY, (parsed) => { diff --git a/test/model-compatibility.test.ts b/test/model-compatibility.test.ts new file mode 100644 index 0000000..294b374 --- /dev/null +++ b/test/model-compatibility.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it } from 'vitest' +import { + inferCompatibility, + parseModelCompatibility, + resolveCompatibility, +} from '../src/utils/model-compatibility' + +describe('parseModelCompatibility', () => { + it('returns an empty map for non-object input', () => { + expect(parseModelCompatibility(undefined)).toEqual({}) + expect(parseModelCompatibility(null)).toEqual({}) + expect(parseModelCompatibility('cacheFormat')).toEqual({}) + expect(parseModelCompatibility([{ cacheFormat: 'anthropic' }])).toEqual({}) + }) + + it('keeps an accepted cacheFormat or null', () => { + expect( + parseModelCompatibility({ + m1: { cacheFormat: 'anthropic' }, + m2: { cacheFormat: 'alibaba' }, + m3: { cacheFormat: null }, + }), + ).toEqual({ + m1: { cacheFormat: 'anthropic' }, + m2: { cacheFormat: 'alibaba' }, + m3: { cacheFormat: null }, + }) + }) + + it('drops rejected values, other keys and empty entries', () => { + expect( + parseModelCompatibility({ + m1: { cacheFormat: 'Anthropic' }, + m2: { cacheFormat: true }, + m3: { supportsThinkingBlockBinding: false }, + m4: 'anthropic', + m5: {}, + }), + ).toEqual({}) + }) +}) + +describe('inferCompatibility', () => { + it('asks Chat Completions for Anthropic cache markers when the proxy accepts them', () => { + expect(inferCompatibility({ acceptsCacheControl: true }, true)).toEqual({ cacheFormat: 'anthropic' }) + }) + + it('infers nothing without the hint or outside Chat Completions', () => { + expect(inferCompatibility({ acceptsCacheControl: false }, true)).toEqual({}) + expect(inferCompatibility(undefined, true)).toEqual({}) + expect(inferCompatibility({ acceptsCacheControl: true }, false)).toEqual({}) + }) +}) + +describe('resolveCompatibility', () => { + it('lets overrides add, replace and remove keys', () => { + expect(resolveCompatibility({ cacheFormat: 'anthropic' }, { cacheFormat: 'alibaba' })).toEqual({ + cacheFormat: 'alibaba', + }) + expect(resolveCompatibility({}, { cacheFormat: 'anthropic' })).toEqual({ cacheFormat: 'anthropic' }) + }) + + it('returns undefined when nothing remains', () => { + expect(resolveCompatibility({}, undefined)).toBeUndefined() + expect(resolveCompatibility({ cacheFormat: 'anthropic' }, { cacheFormat: null })).toBeUndefined() + }) +}) diff --git a/test/plugin-v1.test.ts b/test/plugin-v1.test.ts index d7ae16f..e569741 100644 --- a/test/plugin-v1.test.ts +++ b/test/plugin-v1.test.ts @@ -3,7 +3,12 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { LiteLLMPlugin } from '../src/plugin' -import { buildCacheKey, writeModelCache } from '../src/utils/model-cache' +import { + buildCacheKey, + readModelCacheEntry, + readModelCacheSavedAt, + writeModelCache, +} from '../src/utils/model-cache' import { __resetOpenCodeAuthCacheForTests } from '../src/utils/opencode-auth' interface ProviderConfig { @@ -59,7 +64,7 @@ describe('OpenCode 1 config hook', () => { process.env.HOME = homeDirectory } - function mockProxy(models: Array<{ id: string; mode?: string }>): void { + function mockProxy(models: Array<{ id: string; mode?: string; params?: string[] }>): void { globalThis.fetch = vi.fn(async (input) => { const url = String(input) if (url.endsWith('/v1/model/info')) { @@ -67,7 +72,11 @@ describe('OpenCode 1 config hook', () => { JSON.stringify({ data: models.map((model) => ({ model_name: model.id, - model_info: { key: model.id, mode: model.mode ?? 'chat' }, + model_info: { + key: model.id, + mode: model.mode ?? 'chat', + supported_openai_params: model.params, + }, })), }), { status: 200 }, @@ -122,6 +131,52 @@ describe('OpenCode 1 config hook', () => { expect(Object.keys(sibling.models ?? {})).toEqual(['responses-model']) }) + it('persists model hints so an OpenCode 2 start can reuse the shared cache', async () => { + useIsolatedState() + const baseURL = 'http://127.0.0.1:45557/v1' + mockProxy([ + { id: 'claude', params: ['max_tokens', 'cache_control'] }, + { id: 'gpt', params: ['max_tokens'] }, + ]) + + await runConfigHook(baseURL) + + const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:45557', {}, {}) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + claude: { acceptsCacheControl: true }, + gpt: { acceptsCacheControl: false }, + }) + }) + + it('keeps model hints in the shared cache across a background refresh', async () => { + useIsolatedState() + const baseURL = 'http://127.0.0.1:45558/v1' + mockProxy([{ id: 'claude', params: ['cache_control'] }]) + const input = { client: { app: { log: vi.fn(async () => {}) } } } + const plugin = await LiteLLMPlugin(input as never) + const config: OpenCodeConfig = { + provider: { + litellm: { + npm: '@ai-sdk/openai-compatible', + name: 'LiteLLM (proxy)', + options: { baseURL, apiKey: 'example-key' }, + models: {}, + }, + }, + } + // Cold start, backdated so the next session is allowed to refresh. + const started = Date.now() + const dateNow = vi.spyOn(Date, 'now').mockReturnValue(started - 10 * 60 * 1000) + await plugin.config!(config as never) + dateNow.mockRestore() + const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:45558', {}, {}) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + + await plugin.event!({ event: { type: 'session.created' } } as never) + await vi.waitFor(() => expect(readModelCacheSavedAt(cacheKey)).toBeGreaterThanOrEqual(started)) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + }) + it('creates no sibling provider when no model needs the Responses API', async () => { useIsolatedState() const baseURL = 'http://127.0.0.1:45552/v1' diff --git a/test/plugin-v2.test.ts b/test/plugin-v2.test.ts index 1e49dc1..2199eeb 100644 --- a/test/plugin-v2.test.ts +++ b/test/plugin-v2.test.ts @@ -4,7 +4,12 @@ import { join } from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import plugin from '../src' import type { Context } from '@opencode/plugin/promise/plugin' -import { buildCacheKey, writeModelCache } from '../src/utils/model-cache' +import { + buildCacheKey, + readModelCacheEntry, + readModelCacheSavedAt, + writeModelCache, +} from '../src/utils/model-cache' import { __resetOpenCodeAuthCacheForTests } from '../src/utils/opencode-auth' function restoreEnv(name: string, value: string | undefined): void { @@ -96,6 +101,7 @@ describe('OpenCode 2 plugin entrypoint', () => { supports_vision: true, supports_reasoning: true, supports_reasoning_efforts: ['low', 'high'], + supported_openai_params: ['max_tokens', 'cache_control', 'reasoning_effort'], input_cost_per_token: 0.000003, output_cost_per_token: 0.000015, cache_read_input_token_cost: 0.000001, @@ -137,7 +143,7 @@ describe('OpenCode 2 plugin entrypoint', () => { supports_minimal_reasoning_effort: false, supports_xhigh_reasoning_effort: true, reasoning_effort_levels: null, - supported_openai_params: ['reasoning'], + supported_openai_params: ['reasoning', 'cache_control'], }, }, ], @@ -228,6 +234,8 @@ describe('OpenCode 2 plugin entrypoint', () => { { id: 'low', settings: { reasoningEffort: 'low' } }, { id: 'high', settings: { reasoningEffort: 'high' } }, ], + // Chat Completions only emits Anthropic cache markers when told to. + compatibility: { cacheFormat: 'anthropic' }, }) expect(registered[0].models[1]).toMatchObject({ @@ -261,11 +269,235 @@ describe('OpenCode 2 plugin entrypoint', () => { expect(registered[0].models[0].package).toBeUndefined() expect(registered[0].models[1].package).toBeUndefined() expect(registered[0].models[2].package).toBeUndefined() + // No `cache_control` reported, or not a Chat Completions model: nothing inferred. + expect(registered[0].models[1].compatibility).toBeUndefined() + expect(registered[0].models[2].compatibility).toBeUndefined() + expect(registered[0].models[3].compatibility).toBeUndefined() + const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:44444', {}, {}) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + 'anthropic/claude-3-5-sonnet': { acceptsCacheControl: true }, + 'gpt-6-astra': { acceptsCacheControl: false }, + 'gpt-5-search-api': { acceptsCacheControl: false }, + 'company-pro': { acceptsCacheControl: true }, + }) await cleanup?.() expect(disposeTransform).toHaveBeenCalledOnce() }) + it('applies modelCompatibility overrides on top of inferred compatibility', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + globalThis.fetch = vi.fn(async (input) => { + const url = String(input) + if (url.endsWith('/v1/model/info')) { + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'claude-direct', + model_info: { + key: 'claude-direct', + mode: 'chat', + supported_openai_params: ['cache_control'], + }, + }, + { + model_name: 'claude-bedrock', + model_info: { + key: 'claude-bedrock', + mode: 'chat', + supported_openai_params: ['max_tokens'], + }, + }, + ], + }), + { status: 200 }, + ) + } + return new Response( + JSON.stringify({ data: [ + { id: 'claude-direct', object: 'model' }, + { id: 'claude-bedrock', object: 'model' }, + ] }), + { status: 200 }, + ) + }) + + const registered: Array<{ info: Record; models: Array> }> = [] + const editor = { + list: () => [], + get: () => undefined, + add: (entry: { info: Record; models: Array> }) => { + registered.push(entry) + }, + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const context = { + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: { + baseURL: 'http://127.0.0.1:44444/v1', + modelCompatibility: { + // `null` removes the inferred key. + 'claude-direct': { cacheFormat: null }, + // Set where the proxy does not advertise `cache_control`. + 'claude-bedrock': { cacheFormat: 'anthropic' }, + // An id the proxy does not list adds no model. + unknown: { cacheFormat: 'anthropic' }, + }, + }, + provider: { + list: vi.fn(async () => ({ data: [] })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + transform(editor as never) + return { dispose: vi.fn(async () => {}) } + }), + reload: vi.fn(async () => {}), + }, + event: { subscribe: () => (async function* () {})() }, + } as unknown as Context + + const cleanup = await plugin.setup(context) + + expect(registered).toHaveLength(1) + expect(registered[0].models.map((model) => [model.id, model.compatibility])).toEqual([ + ['claude-direct', undefined], + ['claude-bedrock', { cacheFormat: 'anthropic' }], + ]) + // runtimeSettings strips plugin-only options. + expect(registered[0].info.settings).not.toHaveProperty('modelCompatibility') + await cleanup?.() + }) + + it('applies compatibility from cached hints without touching the network', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + writeModelCache( + buildCacheKey('litellm', 'http://127.0.0.1:44444', {}, {}), + { claude: { name: 'Claude' }, qwen: { name: 'Qwen' }, constructor: { name: 'Constructor' } }, + [], + { + claude: { acceptsCacheControl: true }, + qwen: { acceptsCacheControl: true }, + constructor: { acceptsCacheControl: true }, + }, + ) + globalThis.fetch = vi.fn(async () => { + throw new Error('the cached path must not hit the network') + }) + + const registered: Array<{ info: Record; models: Array> }> = [] + const editor = { + list: () => [], + get: () => undefined, + add: (entry: { info: Record; models: Array> }) => { + registered.push(entry) + }, + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const context = { + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: { + baseURL: 'http://127.0.0.1:44444/v1', + modelCompatibility: { qwen: { cacheFormat: 'alibaba' } }, + }, + provider: { + list: vi.fn(async () => ({ data: [] })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + transform(editor as never) + return { dispose: vi.fn(async () => {}) } + }), + reload: vi.fn(async () => {}), + }, + event: { subscribe: () => (async function* () {})() }, + } as unknown as Context + + const cleanup = await plugin.setup(context) + + expect(globalThis.fetch).not.toHaveBeenCalled() + expect(registered[0].models.map((model) => [model.id, model.compatibility])).toEqual([ + ['claude', { cacheFormat: 'anthropic' }], + ['qwen', { cacheFormat: 'alibaba' }], + // An id that is also an `Object.prototype` member has no override. + ['constructor', { cacheFormat: 'anthropic' }], + ]) + await cleanup?.() + }) + + it('infers cacheFormat only for Chat Completions runtime packages', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + globalThis.fetch = vi.fn(async (input) => { + const url = String(input) + if (url.endsWith('/v1/model/info')) { + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'claude', + model_info: { key: 'claude', mode: 'chat', supported_openai_params: ['cache_control'] }, + }, + ], + }), + { status: 200 }, + ) + } + return new Response(JSON.stringify({ data: [{ id: 'claude', object: 'model' }] }), { status: 200 }) + }) + + const registered: Array<{ info: Record; models: Array> }> = [] + const editor = { + list: () => [], + get: () => undefined, + add: (entry: { info: Record; models: Array> }) => { + registered.push(entry) + }, + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const context = { + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: {}, + provider: { + list: vi.fn(async () => ({ + data: [ + // `openai` defaults to the Responses API, which does not read `cacheFormat`. + ...[ + ['litellm-openai', '@opencode/ai/providers/openai'], + ['litellm-anthropic', '@opencode/ai/providers/anthropic-compatible'], + ['litellm-chat', '@opencode/ai/providers/openai/chat'], + ].map(([id, pkg]) => ({ + id, + activation: 'enabled', + package: pkg, + settings: { baseURL: 'http://127.0.0.1:44444/v1' }, + })), + ], + })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + transform(editor as never) + return { dispose: vi.fn(async () => {}) } + }), + reload: vi.fn(async () => {}), + }, + event: { subscribe: () => (async function* () {})() }, + } as unknown as Context + + const cleanup = await plugin.setup(context) + + expect(registered.map((entry) => [entry.info.id, entry.models[0].compatibility])).toEqual([ + ['litellm-openai', undefined], + ['litellm-anthropic', undefined], + ['litellm-chat', { cacheFormat: 'anthropic' }], + ]) + await cleanup?.() + }) + it('uses service environment variables to create the default provider', async () => { cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-env-test-')) process.env.XDG_CACHE_HOME = cacheDirectory @@ -348,7 +580,20 @@ describe('OpenCode 2 plugin entrypoint', () => { globalThis.fetch = vi.fn(async (input) => { const url = String(input) if (url.endsWith('/v1/model/info')) { - return new Response(JSON.stringify({ data: [] }), { status: 200 }) + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'anthropic/claude-3-5-sonnet', + model_info: { + key: 'anthropic/claude-3-5-sonnet', + supported_openai_params: ['cache_control'], + }, + }, + ], + }), + { status: 200 }, + ) } return new Response( JSON.stringify({ data: [{ id: 'anthropic/claude-3-5-sonnet', object: 'model' }] }), @@ -361,7 +606,11 @@ describe('OpenCode 2 plugin entrypoint', () => { name: 'Configured LiteLLM', activation: 'enabled', package: '@opencode/ai/providers/openai-compatible', - settings: { baseURL: `${baseURL}/v1`, apiKey: 'example-api-key' }, + settings: { + baseURL: `${baseURL}/v1`, + apiKey: 'example-api-key', + modelCompatibility: { 'anthropic/claude-3-5-sonnet': { cacheFormat: 'alibaba' } }, + }, headers: { 'X-Gateway': 'test' }, } const curatedModel = { id: 'curated-model', name: 'Curated model' } @@ -406,6 +655,8 @@ describe('OpenCode 2 plugin entrypoint', () => { expect(configuredProvider.headers).toMatchObject({ 'X-Gateway': 'test' }) expect(updatedModels.map((model) => model.id)).toContain('curated-model') expect(updatedModels.map((model) => model.id)).toContain('anthropic/claude-3-5-sonnet') + const discovered = updatedModels.find((model) => model.id === 'anthropic/claude-3-5-sonnet') + expect(discovered?.compatibility).toEqual({ cacheFormat: 'alibaba' }) await cleanup?.() }) @@ -423,10 +674,37 @@ describe('OpenCode 2 plugin entrypoint', () => { globalThis.fetch = vi.fn(async (input) => { const url = String(input) if (url.endsWith('/v1/model/info')) { - return new Response(JSON.stringify({ data: [] }), { status: 200 }) + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'anthropic/claude-3-5-sonnet', + model_info: { + key: 'anthropic/claude-3-5-sonnet', + mode: 'chat', + supported_openai_params: ['cache_control'], + }, + }, + { + model_name: 'vertex/claude-sonnet', + model_info: { + key: 'vertex/claude-sonnet', + mode: 'chat', + supported_openai_params: ['cache_control'], + }, + }, + ], + }), + { status: 200 }, + ) } return new Response( - JSON.stringify({ data: [{ id: 'anthropic/claude-3-5-sonnet', object: 'model' }] }), + JSON.stringify({ + data: [ + { id: 'anthropic/claude-3-5-sonnet', object: 'model' }, + { id: 'vertex/claude-sonnet', object: 'model' }, + ], + }), { status: 200 }, ) }) @@ -452,7 +730,10 @@ describe('OpenCode 2 plugin entrypoint', () => { }) const context = { app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, - options: { baseURL: `${baseURL}/v1` }, + options: { + baseURL: `${baseURL}/v1`, + modelCompatibility: { 'anthropic/claude-3-5-sonnet': { cacheFormat: 'alibaba' } }, + }, provider: { list: vi.fn(async () => ({ data: [] })), transform: vi.fn(async (transform: (editor: unknown) => void) => { @@ -474,12 +755,166 @@ describe('OpenCode 2 plugin entrypoint', () => { await vi.waitFor(() => expect(reload).toHaveBeenCalledOnce()) expect(registeredModels).toHaveLength(2) expect(registeredModels[0].map((model) => model.id)).toEqual(['cached-model']) - expect(registeredModels[1].map((model) => model.id)).toEqual([ - 'anthropic/claude-3-5-sonnet', + // Inference and the override are both applied on refresh, and the refreshed hints are persisted. + expect(registeredModels[1].map((model) => [model.id, model.compatibility])).toEqual([ + ['anthropic/claude-3-5-sonnet', { cacheFormat: 'alibaba' }], + ['vertex/claude-sonnet', { cacheFormat: 'anthropic' }], ]) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + 'anthropic/claude-3-5-sonnet': { acceptsCacheControl: true }, + 'vertex/claude-sonnet': { acceptsCacheControl: true }, + }) + await cleanup?.() + }) + + it('infers no cacheFormat for an Anthropic package, including after a refresh', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-refresh-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + const baseURL = 'http://127.0.0.1:44444' + const cacheKey = buildCacheKey('litellm-claude', baseURL, {}, {}) + const now = Date.now() + const dateNow = vi.spyOn(Date, 'now').mockReturnValue(now - 10 * 60 * 1000) + writeModelCache(cacheKey, { 'cached-model': { name: 'Cached Model' } }, [], { + 'cached-model': { acceptsCacheControl: true }, + }) + dateNow.mockRestore() + + globalThis.fetch = vi.fn(async (input) => { + const url = String(input) + if (url.endsWith('/v1/model/info')) { + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'claude', + model_info: { key: 'claude', mode: 'chat', supported_openai_params: ['cache_control'] }, + }, + ], + }), + { status: 200 }, + ) + } + return new Response(JSON.stringify({ data: [{ id: 'claude', object: 'model' }] }), { status: 200 }) + }) + + const registeredModels: Array>> = [] + let providerTransform: ((editor: unknown) => void) | undefined + const editor = { + list: () => [], + get: () => undefined, + add: (entry: { models: Array> }) => { + registeredModels.push(entry.models) + }, + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const reload = vi.fn(async () => { + providerTransform?.(editor) + }) + const context = { + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: {}, + provider: { + list: vi.fn(async () => ({ + data: [ + { + id: 'litellm-claude', + activation: 'enabled', + package: '@opencode/ai/providers/anthropic-compatible', + settings: { baseURL: `${baseURL}/v1` }, + }, + ], + })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + providerTransform = transform + transform(editor) + return { dispose: vi.fn(async () => {}) } + }), + reload, + }, + event: { + subscribe: () => + (async function* () { + yield { type: 'session.created' } + })(), + }, + } as unknown as Context + + const cleanup = await plugin.setup(context) + await vi.waitFor(() => expect(reload).toHaveBeenCalledOnce()) + expect(registeredModels[0].map((model) => model.compatibility)).toEqual([undefined]) + expect(registeredModels[1].map((model) => model.id)).toEqual(['claude']) + expect(registeredModels[1][0].compatibility).toBeUndefined() await cleanup?.() }) + it('keeps hints in the cache when a background refresh finds no changes', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-refresh-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + const baseURL = 'http://127.0.0.1:44444' + const cacheKey = buildCacheKey('litellm', baseURL, {}, {}) + globalThis.fetch = vi.fn(async (input) => { + const url = String(input) + if (url.endsWith('/v1/model/info')) { + return new Response( + JSON.stringify({ + data: [ + { + model_name: 'claude', + model_info: { key: 'claude', mode: 'chat', supported_openai_params: ['cache_control'] }, + }, + ], + }), + { status: 200 }, + ) + } + return new Response(JSON.stringify({ data: [{ id: 'claude', object: 'model' }] }), { status: 200 }) + }) + const editor = { + list: () => [], + get: () => undefined, + add: vi.fn(), + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const reload = vi.fn(async () => {}) + const makeContext = (events: Array<{ type: string }>) => + ({ + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: { baseURL: `${baseURL}/v1` }, + provider: { + list: vi.fn(async () => ({ data: [] })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + transform(editor) + return { dispose: vi.fn(async () => {}) } + }), + reload, + }, + event: { + subscribe: () => + (async function* () { + yield* events + })(), + }, + }) as unknown as Context + + // Cold start, backdated so the next session is allowed to refresh. + const started = Date.now() + const dateNow = vi.spyOn(Date, 'now').mockReturnValue(started - 10 * 60 * 1000) + const first = await plugin.setup(makeContext([])) + dateNow.mockRestore() + await first?.() + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + + const second = await plugin.setup(makeContext([{ type: 'session.created' }])) + await vi.waitFor(() => expect(readModelCacheSavedAt(cacheKey)).toBeGreaterThanOrEqual(started)) + expect(reload).not.toHaveBeenCalled() + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + await second?.() + }) + it('registers providers that only appear after setup (provider.updated)', async () => { cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-late-provider-test-')) process.env.XDG_CACHE_HOME = cacheDirectory From f8f4a1128204bcbd03c70ecd22c9a0d223d80208 Mon Sep 17 00:00:00 2001 From: Mateusz Kowalski Date: Sun, 18 Oct 2026 21:57:00 +0200 Subject: [PATCH 2/2] fix(v2): spell effort variants for Anthropic runtime packages Discovered reasoning-effort variants set `reasoningEffort`, which only the OpenAI runtimes read. On OpenCode 2 a provider can use `@opencode/ai/providers/anthropic` or `anthropic-compatible` to reach Claude through the proxy's `/v1/messages`; there every variant was a silent no-op. Record whether `/v1/model/info` reports `supports_adaptive_thinking` and `thinking_always_on`, and on Anthropic packages emit each level as `{ thinking: { type: "adaptive", display: "summarized" }, effort }`, the shape OpenCode's own catalog uses for adaptive-thinking Claude models. `none` becomes `thinking: { type: "disabled" }` and is dropped when thinking is always on. Models without adaptive thinking get no variants on those packages rather than inert ones. OpenAI packages are unchanged. Cache entries written before this change carry neither hint, so `CACHE_VERSION` moves to 5 and such caches are rediscovered. Signed-off-by: Mateusz Kowalski Assisted-By: Claude Fable 5.1 --- CHANGELOG.md | 1 + README.md | 25 ++++ src/plugin/index.ts | 5 +- src/plugin/v2.ts | 33 ++++- src/types/index.ts | 2 + src/utils/model-cache.ts | 22 +++- test/model-cache.test.ts | 22 ++-- test/model-compatibility.test.ts | 13 +- test/plugin-v1.test.ts | 12 +- test/plugin-v2.test.ts | 209 ++++++++++++++++++++++++++++--- 10 files changed, 302 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ffe3f3a..4296565 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,6 +49,7 @@ ### Bug Fixes * **v2:** register Chat Completions models whose deployment accepts Anthropic `cache_control` with `compatibility.cacheFormat: "anthropic"` so prompt caching works for Claude behind LiteLLM +* **v2:** spell reasoning-effort variants as `thinking` + `effort` for providers on an Anthropic runtime package, where `reasoningEffort` is ignored; models without adaptive thinking get none there * restore baseline reasoning-effort variants from sparse LiteLLM metadata (including GPT-6 Astra), respect explicit effort lists and disabled levels * avoid inferring reasoning-effort variants from negative-only flags or unsupported request parameters * constrain inferred GPT-5 Pro effort variants to provider-supported levels, including aliased and dated deployments diff --git a/README.md b/README.md index 668413f..6f44b05 100644 --- a/README.md +++ b/README.md @@ -245,6 +245,31 @@ between effort levels from the model picker without hand-curating every entry. > `mode: "responses"` the plugin routes it to `/v1/responses` automatically; > otherwise reach it without `reasoningEffort`. See the FAQ entry below. +On OpenCode 2, models whose effective package is `@opencode/ai/providers/anthropic` +or `@opencode/ai/providers/anthropic-compatible` get the same levels in the +shape that runtime reads, `{ thinking: { type: "adaptive", display: "summarized" }, effort }`, +for models LiteLLM reports with `supports_adaptive_thinking`, with `none` +becoming `thinking: { type: "disabled" }` (dropped when `thinking_always_on`). +Models without adaptive thinking get no variants on those packages, since +`reasoningEffort` would be ignored there. To reach Claude over the proxy's +`/v1/messages` while the rest of the catalog keeps its usual transport, split it +with [`includeModels`/`excludeModels`](#splitting-one-proxy-into-multiple-providers-includemodels--excludemodels): + +```jsonc +{ + "providers": { + "litellm": { + "package": "@opencode/ai/providers/openai-compatible", + "settings": { "baseURL": "http://localhost:4000/v1", "excludeModels": ["*claude*"] } + }, + "litellm-claude": { + "package": "@opencode/ai/providers/anthropic-compatible", + "settings": { "baseURL": "http://localhost:4000/v1", "includeModels": ["*claude*"] } + } + } +} +``` + ### Responses API models LiteLLM reports `mode` per model on `/v1/model/info` (the plugin fetches it diff --git a/src/plugin/index.ts b/src/plugin/index.ts index 4d0f62e..197519f 100644 --- a/src/plugin/index.ts +++ b/src/plugin/index.ts @@ -333,7 +333,8 @@ export function toConfigModel( * config (or translated into `Model.Info` on OpenCode 2). `responsesIds` * lists the subset that must be invoked through the OpenAI Responses API; * callers route those accordingly without inspecting each entry. `hints` - * carries per-model upstream facts OpenCode 2 maps to `compatibility`. + * carries per-model upstream facts OpenCode 2 maps to `compatibility` and + * `variants`. */ export interface DiscoveredModels { models: Record @@ -447,6 +448,8 @@ export async function discoverModels( const params = info?.supported_openai_params hints[model.id] = { acceptsCacheControl: Array.isArray(params) && params.includes('cache_control'), + adaptiveThinking: info?.supports_adaptive_thinking === true, + thinkingAlwaysOn: info?.thinking_always_on === true, } } diff --git a/src/plugin/v2.ts b/src/plugin/v2.ts index 1223491..e19f400 100644 --- a/src/plugin/v2.ts +++ b/src/plugin/v2.ts @@ -39,6 +39,11 @@ const CHAT_COMPLETIONS_PACKAGES = new Set([ OPENAI_COMPATIBLE_PACKAGE, '@opencode/ai/providers/openai/chat', ]) +const ANTHROPIC_PACKAGES = new Set([ + '@opencode/ai/providers/anthropic', + '@opencode/ai/providers/anthropic-compatible', +]) +const ADAPTIVE_THINKING = { type: 'adaptive', display: 'summarized' } // Runtime package that talks to `/v1/responses`. OpenCode 2 lets one model // override the provider's package, so Responses models stay under their // original provider id and no sibling provider is needed here. @@ -126,6 +131,29 @@ function numberOrZero(value: unknown): number { return typeof value === 'number' && Number.isFinite(value) ? value : 0 } +/** + * Discovery spells every effort level as `{ reasoningEffort }`, which the + * OpenAI runtimes read. The Anthropic runtimes read `thinking` and `effort` + * instead, so on those packages the levels are respelled the way OpenCode's + * own catalog does for adaptive-thinking Claude models. A model that does + * not report adaptive thinking gets no variants rather than inert ones. + */ +function anthropicVariants( + variants: Model.Info['variants'], + hints: ModelHints | undefined, +): Model.Info['variants'] { + if (!hints?.adaptiveThinking) return [] + return variants.flatMap((variant) => { + const effort = variant.settings?.reasoningEffort + if (typeof effort !== 'string') return [] + if (effort === 'none') { + if (hints.thinkingAlwaysOn) return [] + return [{ ...variant, settings: { thinking: { type: 'disabled' } } }] + } + return [{ ...variant, settings: { thinking: ADAPTIVE_THINKING, effort } }] + }) +} + interface ToProviderModelsOptions { /** The provider's runtime package; models not overridden inherit it. */ providerPackage: string @@ -172,11 +200,14 @@ function toProviderModels( : [] const rawVariants = asRecord(entry.variants) - const variants = Object.entries(rawVariants).map(([id, settings]) => ({ + const openAIVariants = Object.entries(rawVariants).map(([id, settings]) => ({ id: id as Model.VariantID, settings: asRecord(settings), })) as Model.Info['variants'] const pkg = responsesIds.has(modelID) ? RESPONSES_PACKAGE : options.providerPackage + const variants = ANTHROPIC_PACKAGES.has(pkg) + ? anthropicVariants(openAIVariants, hints[modelID]) + : openAIVariants const compatibility = resolveCompatibility( inferCompatibility(hints[modelID], CHAT_COMPLETIONS_PACKAGES.has(pkg)), Object.hasOwn(overrides, modelID) ? overrides[modelID] : undefined, diff --git a/src/types/index.ts b/src/types/index.ts index d35b41e..766ceae 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -71,6 +71,8 @@ export interface LiteLLMModelInfo { supported_openai_params?: string[] | null supports_pdf_input?: boolean supports_audio_input?: boolean + supports_adaptive_thinking?: boolean + thinking_always_on?: boolean input_cost_per_token?: number output_cost_per_token?: number cache_read_input_token_cost?: number diff --git a/src/utils/model-cache.ts b/src/utils/model-cache.ts index 8d80441..355844e 100644 --- a/src/utils/model-cache.ts +++ b/src/utils/model-cache.ts @@ -5,8 +5,9 @@ import { join } from 'node:path' /** * Facts about a discovered model's upstream that OpenCode 2 turns into - * `Model.Info.compatibility`. They ride alongside the config-shaped model - * entries (which have no field for them) through discovery and the cache. + * `Model.Info.compatibility` and `variants`. They ride alongside the + * config-shaped model entries (which have no field for them) through + * discovery and the cache. */ export interface ModelHints { /** @@ -14,6 +15,13 @@ export interface ModelHints { * Chat Completions (`supported_openai_params` lists `cache_control`). */ acceptsCacheControl: boolean + /** + * The model takes `thinking: { type: "adaptive" }` plus an `effort` level + * (`supports_adaptive_thinking`). + */ + adaptiveThinking: boolean + /** Thinking cannot be turned off (`thinking_always_on`). */ + thinkingAlwaysOn: boolean } /** @@ -21,7 +29,7 @@ export interface ModelHints { * from older plugin versions are ignored rather than deserialized into * an incompatible structure. */ -const CACHE_VERSION = 4 +const CACHE_VERSION = 5 /** * Maximum age of a cache entry before it's treated as a miss. Without a @@ -158,8 +166,12 @@ function normalizeHints(value: unknown): Record { const out: Record = {} for (const [id, hint] of Object.entries(value as Record)) { if (!hint || typeof hint !== 'object' || Array.isArray(hint)) continue - const { acceptsCacheControl } = hint as Record - out[id] = { acceptsCacheControl: acceptsCacheControl === true } + const { acceptsCacheControl, adaptiveThinking, thinkingAlwaysOn } = hint as Record + out[id] = { + acceptsCacheControl: acceptsCacheControl === true, + adaptiveThinking: adaptiveThinking === true, + thinkingAlwaysOn: thinkingAlwaysOn === true, + } } return out } diff --git a/test/model-cache.test.ts b/test/model-cache.test.ts index ef598d0..cead715 100644 --- a/test/model-cache.test.ts +++ b/test/model-cache.test.ts @@ -80,25 +80,23 @@ describe('model cache', () => { }) it('round-trips model hints and drops malformed ones', () => { - writeModelCache('hints@http://x', { a: {}, b: {} }, [], { - a: { acceptsCacheControl: true }, - b: { acceptsCacheControl: false }, + const a = { acceptsCacheControl: true, adaptiveThinking: true, thinkingAlwaysOn: false } + const b = { acceptsCacheControl: false, adaptiveThinking: false, thinkingAlwaysOn: true } + writeModelCache('hints@http://x', { a: {}, b: {} }, [], { a, b }) + expect(readModelCacheEntry('hints@http://x')?.hints).toEqual({ a, b }) + rewriteCacheFile('hints@http://x', (parsed) => { + parsed.hints = { a: { acceptsCacheControl: 'yes', adaptiveThinking: 1 }, b: null, c: [] } }) expect(readModelCacheEntry('hints@http://x')?.hints).toEqual({ - a: { acceptsCacheControl: true }, - b: { acceptsCacheControl: false }, - }) - rewriteCacheFile('hints@http://x', (parsed) => { - parsed.hints = { a: { acceptsCacheControl: 'yes' }, b: null, c: [] } + a: { acceptsCacheControl: false, adaptiveThinking: false, thinkingAlwaysOn: false }, }) - expect(readModelCacheEntry('hints@http://x')?.hints).toEqual({ a: { acceptsCacheControl: false } }) }) - it('treats caches that predate the model hints as a miss', () => { + it('treats caches that predate the thinking hints as a miss', () => { writeModelCache(KEY, { m: {} }, ['m']) rewriteCacheFile(KEY, (parsed) => { - parsed.version = 3 - delete parsed.hints + parsed.version = 4 + parsed.hints = { m: { acceptsCacheControl: true } } }) expect(readModelCacheEntry(KEY)).toBeNull() expect(readModelCache(KEY)).toBeNull() diff --git a/test/model-compatibility.test.ts b/test/model-compatibility.test.ts index 294b374..6619ecf 100644 --- a/test/model-compatibility.test.ts +++ b/test/model-compatibility.test.ts @@ -4,6 +4,13 @@ import { parseModelCompatibility, resolveCompatibility, } from '../src/utils/model-compatibility' +import type { ModelHints } from '../src/utils/model-cache' + +const hints = (acceptsCacheControl: boolean): ModelHints => ({ + acceptsCacheControl, + adaptiveThinking: false, + thinkingAlwaysOn: false, +}) describe('parseModelCompatibility', () => { it('returns an empty map for non-object input', () => { @@ -42,13 +49,13 @@ describe('parseModelCompatibility', () => { describe('inferCompatibility', () => { it('asks Chat Completions for Anthropic cache markers when the proxy accepts them', () => { - expect(inferCompatibility({ acceptsCacheControl: true }, true)).toEqual({ cacheFormat: 'anthropic' }) + expect(inferCompatibility(hints(true), true)).toEqual({ cacheFormat: 'anthropic' }) }) it('infers nothing without the hint or outside Chat Completions', () => { - expect(inferCompatibility({ acceptsCacheControl: false }, true)).toEqual({}) + expect(inferCompatibility(hints(false), true)).toEqual({}) expect(inferCompatibility(undefined, true)).toEqual({}) - expect(inferCompatibility({ acceptsCacheControl: true }, false)).toEqual({}) + expect(inferCompatibility(hints(true), false)).toEqual({}) }) }) diff --git a/test/plugin-v1.test.ts b/test/plugin-v1.test.ts index e569741..040238c 100644 --- a/test/plugin-v1.test.ts +++ b/test/plugin-v1.test.ts @@ -143,8 +143,8 @@ describe('OpenCode 1 config hook', () => { const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:45557', {}, {}) expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ - claude: { acceptsCacheControl: true }, - gpt: { acceptsCacheControl: false }, + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + gpt: { acceptsCacheControl: false, adaptiveThinking: false, thinkingAlwaysOn: false }, }) }) @@ -170,11 +170,15 @@ describe('OpenCode 1 config hook', () => { await plugin.config!(config as never) dateNow.mockRestore() const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:45558', {}, {}) - expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + }) await plugin.event!({ event: { type: 'session.created' } } as never) await vi.waitFor(() => expect(readModelCacheSavedAt(cacheKey)).toBeGreaterThanOrEqual(started)) - expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + }) }) it('creates no sibling provider when no model needs the Responses API', async () => { diff --git a/test/plugin-v2.test.ts b/test/plugin-v2.test.ts index 2199eeb..7ac6dc8 100644 --- a/test/plugin-v2.test.ts +++ b/test/plugin-v2.test.ts @@ -101,6 +101,7 @@ describe('OpenCode 2 plugin entrypoint', () => { supports_vision: true, supports_reasoning: true, supports_reasoning_efforts: ['low', 'high'], + supports_adaptive_thinking: true, supported_openai_params: ['max_tokens', 'cache_control', 'reasoning_effort'], input_cost_per_token: 0.000003, output_cost_per_token: 0.000015, @@ -275,10 +276,14 @@ describe('OpenCode 2 plugin entrypoint', () => { expect(registered[0].models[3].compatibility).toBeUndefined() const cacheKey = buildCacheKey('litellm', 'http://127.0.0.1:44444', {}, {}) expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ - 'anthropic/claude-3-5-sonnet': { acceptsCacheControl: true }, - 'gpt-6-astra': { acceptsCacheControl: false }, - 'gpt-5-search-api': { acceptsCacheControl: false }, - 'company-pro': { acceptsCacheControl: true }, + 'anthropic/claude-3-5-sonnet': { + acceptsCacheControl: true, + adaptiveThinking: true, + thinkingAlwaysOn: false, + }, + 'gpt-6-astra': { acceptsCacheControl: false, adaptiveThinking: false, thinkingAlwaysOn: false }, + 'gpt-5-search-api': { acceptsCacheControl: false, adaptiveThinking: false, thinkingAlwaysOn: false }, + 'company-pro': { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, }) await cleanup?.() @@ -379,9 +384,9 @@ describe('OpenCode 2 plugin entrypoint', () => { { claude: { name: 'Claude' }, qwen: { name: 'Qwen' }, constructor: { name: 'Constructor' } }, [], { - claude: { acceptsCacheControl: true }, - qwen: { acceptsCacheControl: true }, - constructor: { acceptsCacheControl: true }, + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + qwen: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + constructor: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, }, ) globalThis.fetch = vi.fn(async () => { @@ -498,6 +503,150 @@ describe('OpenCode 2 plugin entrypoint', () => { await cleanup?.() }) + it('respells effort variants for providers on an Anthropic runtime package', async () => { + cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-test-')) + process.env.XDG_CACHE_HOME = cacheDirectory + globalThis.fetch = vi.fn(async (input) => { + const url = String(input) + if (url.endsWith('/v1/model/info')) { + return new Response( + JSON.stringify({ + data: [ + { + // Adaptive thinking, can be switched off: `none` becomes `disabled`. + model_name: 'claude-adaptive', + model_info: { + key: 'claude-adaptive', + mode: 'chat', + supports_reasoning: true, + supports_adaptive_thinking: true, + reasoning_effort_levels: ['none', 'low', 'max'], + supported_openai_params: ['cache_control', 'reasoning_effort'], + }, + }, + { + // Adaptive thinking that is always on: `none` is dropped. + model_name: 'claude-always-on', + model_info: { + key: 'claude-always-on', + mode: 'chat', + supports_reasoning: true, + supports_adaptive_thinking: true, + thinking_always_on: true, + reasoning_effort_levels: ['none', 'high'], + }, + }, + { + // No adaptive thinking reported: `reasoningEffort` would be inert, so no variants. + model_name: 'claude-manual', + model_info: { + key: 'claude-manual', + mode: 'chat', + supports_reasoning: true, + reasoning_effort_levels: ['low', 'high'], + }, + }, + { + // Responses override on the same provider: the OpenAI runtime keeps `reasoningEffort`. + model_name: 'gpt-responses', + model_info: { + key: 'gpt-responses', + mode: 'responses', + supports_reasoning: true, + supports_adaptive_thinking: true, + reasoning_effort_levels: ['low'], + }, + }, + ], + }), + { status: 200 }, + ) + } + return new Response( + JSON.stringify({ data: [ + { id: 'claude-adaptive', object: 'model' }, + { id: 'claude-always-on', object: 'model' }, + { id: 'claude-manual', object: 'model' }, + { id: 'gpt-responses', object: 'model' }, + ] }), + { status: 200 }, + ) + }) + + const registered: Array<{ info: Record; models: Array> }> = [] + const editor = { + list: () => [], + get: () => undefined, + add: (entry: { info: Record; models: Array> }) => { + registered.push(entry) + }, + update: vi.fn(), + remove: vi.fn(), + models: { set: vi.fn(), update: vi.fn(), remove: vi.fn() }, + } + const context = { + app: { name: 'OpenCode', version: '2.0.14', channel: 'stable' }, + options: { baseURL: 'http://127.0.0.1:44444/v1' }, + provider: { + list: vi.fn(async () => ({ + data: [ + { + id: 'litellm-claude', + name: 'LiteLLM Claude', + activation: 'enabled', + package: '@opencode/ai/providers/anthropic-compatible', + settings: { baseURL: 'http://127.0.0.1:44444/v1' }, + }, + { + id: 'litellm-anthropic', + name: 'LiteLLM Anthropic', + activation: 'enabled', + package: '@opencode/ai/providers/anthropic', + settings: { baseURL: 'http://127.0.0.1:44444/v1' }, + }, + ], + })), + transform: vi.fn(async (transform: (editor: unknown) => void) => { + transform(editor as never) + return { dispose: vi.fn(async () => {}) } + }), + reload: vi.fn(async () => {}), + }, + event: { subscribe: () => (async function* () {})() }, + } as unknown as Context + + const cleanup = await plugin.setup(context) + + expect(registered.map((entry) => entry.info.id)).toEqual(['litellm-claude', 'litellm-anthropic']) + expect(registered[0].info).toMatchObject({ + id: 'litellm-claude', + package: '@opencode/ai/providers/anthropic-compatible', + }) + const adaptive = { type: 'adaptive', display: 'summarized' } + expect(registered[0].models.map((model) => [model.id, model.variants])).toEqual([ + [ + 'claude-adaptive', + [ + { id: 'none', settings: { thinking: { type: 'disabled' } } }, + { id: 'low', settings: { thinking: adaptive, effort: 'low' } }, + { id: 'max', settings: { thinking: adaptive, effort: 'max' } }, + ], + ], + ['claude-always-on', [{ id: 'high', settings: { thinking: adaptive, effort: 'high' } }]], + ['claude-manual', []], + ['gpt-responses', [{ id: 'low', settings: { reasoningEffort: 'low' } }]], + ]) + // Both Anthropic packages get the same spelling. + expect(registered[1].models.map((model) => model.variants)).toEqual( + registered[0].models.map((model) => model.variants), + ) + // The Anthropic runtime places cache breakpoints itself; nothing to declare, + // even though `claude-adaptive` advertises `cache_control`. + const all = registered.flatMap((entry) => entry.models) + expect(all.every((model) => model.compatibility === undefined)).toBe(true) + await cleanup?.() + }) + it('uses service environment variables to create the default provider', async () => { cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-env-test-')) process.env.XDG_CACHE_HOME = cacheDirectory @@ -761,22 +910,33 @@ describe('OpenCode 2 plugin entrypoint', () => { ['vertex/claude-sonnet', { cacheFormat: 'anthropic' }], ]) expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ - 'anthropic/claude-3-5-sonnet': { acceptsCacheControl: true }, - 'vertex/claude-sonnet': { acceptsCacheControl: true }, + 'anthropic/claude-3-5-sonnet': { + acceptsCacheControl: true, + adaptiveThinking: false, + thinkingAlwaysOn: false, + }, + 'vertex/claude-sonnet': { + acceptsCacheControl: true, + adaptiveThinking: false, + thinkingAlwaysOn: false, + }, }) await cleanup?.() }) - it('infers no cacheFormat for an Anthropic package, including after a refresh', async () => { + it('keeps Anthropic variants and no cacheFormat across a background refresh', async () => { cacheDirectory = mkdtempSync(join(tmpdir(), 'opencode-litellm-refresh-test-')) process.env.XDG_CACHE_HOME = cacheDirectory const baseURL = 'http://127.0.0.1:44444' const cacheKey = buildCacheKey('litellm-claude', baseURL, {}, {}) const now = Date.now() const dateNow = vi.spyOn(Date, 'now').mockReturnValue(now - 10 * 60 * 1000) - writeModelCache(cacheKey, { 'cached-model': { name: 'Cached Model' } }, [], { - 'cached-model': { acceptsCacheControl: true }, - }) + writeModelCache( + cacheKey, + { 'cached-model': { name: 'Cached Model', variants: { high: { reasoningEffort: 'high' } } } }, + [], + { 'cached-model': { acceptsCacheControl: true, adaptiveThinking: true, thinkingAlwaysOn: false } }, + ) dateNow.mockRestore() globalThis.fetch = vi.fn(async (input) => { @@ -787,7 +947,14 @@ describe('OpenCode 2 plugin entrypoint', () => { data: [ { model_name: 'claude', - model_info: { key: 'claude', mode: 'chat', supported_openai_params: ['cache_control'] }, + model_info: { + key: 'claude', + mode: 'chat', + supports_reasoning: true, + supports_adaptive_thinking: true, + reasoning_effort_levels: ['high'], + supported_openai_params: ['cache_control'], + }, }, ], }), @@ -843,9 +1010,15 @@ describe('OpenCode 2 plugin entrypoint', () => { const cleanup = await plugin.setup(context) await vi.waitFor(() => expect(reload).toHaveBeenCalledOnce()) + const high = [ + { id: 'high', settings: { thinking: { type: 'adaptive', display: 'summarized' }, effort: 'high' } }, + ] + // Cached hints drive the warm start; discovered hints drive the refresh. expect(registeredModels[0].map((model) => model.compatibility)).toEqual([undefined]) + expect(registeredModels[0].map((model) => model.variants)).toEqual([high]) expect(registeredModels[1].map((model) => model.id)).toEqual(['claude']) expect(registeredModels[1][0].compatibility).toBeUndefined() + expect(registeredModels[1][0].variants).toEqual(high) await cleanup?.() }) @@ -906,12 +1079,16 @@ describe('OpenCode 2 plugin entrypoint', () => { const first = await plugin.setup(makeContext([])) dateNow.mockRestore() await first?.() - expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + }) const second = await plugin.setup(makeContext([{ type: 'session.created' }])) await vi.waitFor(() => expect(readModelCacheSavedAt(cacheKey)).toBeGreaterThanOrEqual(started)) expect(reload).not.toHaveBeenCalled() - expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ claude: { acceptsCacheControl: true } }) + expect(readModelCacheEntry(cacheKey)?.hints).toEqual({ + claude: { acceptsCacheControl: true, adaptiveThinking: false, thinkingAlwaysOn: false }, + }) await second?.() })