[mkowalski]

Summary

On OpenCode 2, the OpenAI-compatible Chat Completions runtime only emits Anthropic-style cache_control breakpoints when a model declares compatibility.cacheFormat: "anthropic". Discovered models never did, so Claude served through LiteLLM lost prompt caching after upgrading from OpenCode 1 (#42).

The plugin now records, per model, whether /v1/model/info lists cache_control under supported_openai_params and registers Chat Completions models that do with compatibility.cacheFormat: "anthropic". LiteLLM reports that for Anthropic and Vertex AI Anthropic deployments. Responses-API models and providers on an Anthropic runtime package are left alone (those runtimes don't read cacheFormat).

A new modelCompatibility provider option (per exact model id, like modelCapabilities) sets cacheFormat to "anthropic", "alibaba", or null. It exists because OpenCode's own providers.<id>.models.<model>.compatibility is merged over the plugin's value, so native config can add or change the key but cannot remove one the plugin inferred. It accepts only cacheFormat, the one key the plugin sets; values are validated when parsed because OpenCode validates plugin-registered models only when it serializes them, where one bad value fails the whole model list. It is applied at registration, so it is not part of the cache key.

The hint travels with the discovery result and the on-disk cache (CACHE_VERSION 3 → 4; one cold discovery on first start after upgrade, as with #40).

Fixes #42

Type of change

• 🐛 Bug fix (non-breaking)
• ✨ New feature (non-breaking)
• 💥 Breaking change
• 📝 Documentation only
• 🔧 Internal / refactor

Checklist

• npm run typecheck passes
• No new runtime dependencies (or justified in this PR description)
• README updated if public API or behavior changed
• CHANGELOG.md updated under ## [Unreleased]
• Commit messages follow Conventional Commits

How was this tested?

• npm test: 141 passing (125 existing + 16 new); tsc --noEmit clean.
• Live: LiteLLM 1.104.0 proxy serving vertex_ai/claude-opus-4-8 (reports cache_control) and github_copilot/* models (do not), OpenCode 2.0.30. After the change, /api/model shows compatibility.cacheFormat: "anthropic" on the Vertex model under an openai-compatible provider, and nothing on the Copilot models, on mode: responses models, or on models under an anthropic-compatible provider.
• New tests cover cold start, warm cache (no network), background refresh with changed and unchanged inventories, a provider registered after setup, the V1 → V2 shared cache, and the override from both plugin options and provider.settings.

Notes

• cacheFormat is read by OpenCode ≥ 2.0.28; earlier 2.x hosts drop the key without error. The README says so.
• as unknown as Model.Info['compatibility']: the lockfile resolves @opencode/schema@2.0.15, whose Model.Compatibility predates cacheFormat. Bumping @opencode/plugin within the existing ^2.0.14 range changes ~900 lockfile lines of unrelated transitive dependencies, so this keeps one documented cast instead. Happy to do the bump as a separate PR if you'd prefer that.
• Discovery-only option keys (includeModels, modelCompatibility, …) still reach the runtime settings of a provider that was configured before plugin setup (editor.update path). That predates this PR and is harmless (runtimes ignore unknown option keys); stripping them there would also be a no-op, since OpenCode's config-provider plugin re-applies providers.<id>.settings after this plugin's transform.

This change was developed with help from an AI coding agent (Claude, running in OpenCode); I reviewed and tested it before opening the PR.

Summary by CodeRabbit

• New Features
  • OpenCode 2 models can now use Anthropic prompt caching when the proxy reports support for cache_control.
  • Added a per-model modelCompatibility option to set or remove inferred cache-format compatibility.
• Bug Fixes
  • Restored reasoning-effort variants and their inference constraints.
• Documentation
  • Added guidance on prompt caching compatibility, overrides, and differences between OpenCode versions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4eb389dd-722c-46a1-8907-253b4f7b3edc

📥 Commits

Reviewing files that changed from the base of the PR and between 0d65275 and 9e6529d.

📒 Files selected for processing: 10

• CHANGELOG.md
• README.md
• src/plugin/index.ts
• src/plugin/v2.ts
• src/utils/model-cache.ts
• src/utils/model-compatibility.ts
• test/model-cache.test.ts
• test/model-compatibility.test.ts
• test/plugin-v1.test.ts
• test/plugin-v2.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The plugin records per-model cache-control support during discovery and stores it in the model cache. OpenCode 2 provider construction uses the hints to infer Anthropic cache compatibility for eligible Chat Completions models. Per-model modelCompatibility settings can replace or remove the inferred format.

Changes: Model cache compatibility

Layer / File(s)	Summary
Discover and persist model hints src/utils/model-cache.ts, src/plugin/index.ts, test/model-cache.test.ts, test/plugin-v1.test.ts	Discovery records whether each model advertises cache_control. Cache version 4 stores and normalizes the hints. Tests cover hint persistence, malformed values, and cache refresh.
Resolve compatibility during provider construction src/utils/model-compatibility.ts, src/plugin/v2.ts, test/model-compatibility.test.ts, test/plugin-v2.test.ts	Compatibility inference is limited to supported Chat Completions runtimes and models with cache-control hints. Per-model overrides can replace or remove the inferred format. Tests cover discovery, cached hints, configured providers, and refreshes.
Document cache compatibility README.md, CHANGELOG.md	The README describes inference conditions, accepted override values, and the OpenCode 1 limitation. The changelog records the compatibility changes and modelCompatibility option.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant LiteLLM
  participant discoverModels
  participant ModelCache
  participant toProviderModels
  LiteLLM->>discoverModels: model info with supported_openai_params
  discoverModels->>ModelCache: models and cache-control hints
  ModelCache->>toProviderModels: cached models and hints
  toProviderModels->>toProviderModels: infer compatibility and apply modelCompatibility override

Loading

Suggested reviewers: temeteke, alexanderwillner

Merge Risk: ⚪ Minimal · up to 9e652

The cache-compatibility change is ready to merge after normal checks; no actionable issue remains identified.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 18 functions across 8 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly summarizes the main change: enabling prompt-caching compatibility for Claude models behind LiteLLM in the v2 plugin.
Linked Issues check		Issue #42 requires discovered Claude models behind LiteLLM to receive compatibility.cacheFormat: "anthropic", with an override for deployments without proxy metadata. The implementation records `sup…
Out of Scope Changes check		The changes stay within issue #42 scope. Discovery metadata, cache propagation and invalidation, runtime filtering, provider registration, documentation, and tests directly support reliable `cacheForm…

Full details: Linked Issues check

Explanation

Issue #42 requires discovered Claude models behind LiteLLM to receive compatibility.cacheFormat: "anthropic", with an override for deployments without proxy metadata. The implementation records supported_openai_params.cache_control in src/plugin/index.ts, carries the hint through cache version 4, and applies cacheFormat: "anthropic" in src/plugin/v2.ts for Chat Completions models. modelCompatibility supports exact model IDs with "anthropic", "alibaba", or null, and parsing validates these values. Responses models and non-Chat-Completions runtime packages are excluded. The added tests cover inference, cache reuse and refresh, runtime restrictions, and overrides.

Full details: Out of Scope Changes check

Explanation

The changes stay within issue #42 scope. Discovery metadata, cache propagation and invalidation, runtime filtering, provider registration, documentation, and tests directly support reliable cacheFormat registration and its override. No unrelated product behavior is demonstrated by the reviewed changes.

Full details: Docstring Coverage

Explanation

Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 18 functions across 8 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}