Repository navigation
Add a Claude Code compatibility matrix and release gate #78
Description
Part of #73.
Blocked by: #75, #76, #77 (native dependencies). This is the release gate: it proves the combination, it does not repair it.
Objective
Prove the combined Claude Code client integration with a versioned compatibility matrix, extend the client E2E matrix with Anthropic Messages cases, and gate the "stable" claim on recorded evidence.
Authority
ADR-0014 "Evidence and release gates": every upstream path must show real-CLI multi-turn text, incremental streaming, tool call/result round trips, cancellation, and error handling; the native Anthropic path additionally proves semantic-field and SSE preservation. Publish verified CLI versions and limitations in English and Chinese, and do not claim Anthropic endorses non-Claude model compatibility.
Scope
1. Real-client matrix
- Extend
scripts/e2e_gateway_client_matrix.pywith Anthropic Messages request, JSON response, and streaming SSE cases. - Run against all three upstream protocols: native Anthropic Messages, OpenAI Responses, and Chat Completions — using provider identities that are explicitly approved for live evidence.
- Cover main-session text, multi-turn history, tool call + result, parallel tools, images, compaction/resume, subagents, thinking/effort, prompt caching, cancellation, and error recovery to the extent Spike Claude Code Messages compatibility through CodexHub #74 classified them; unsupported cases are recorded as unsupported, not skipped silently.
- Record the minimum supported and the latest verified Claude Code versions, with the exact executable and platform for every claim.
2. Credential contract — explicit prerequisite
docs/agents/real-client-e2e.md requires a new Issue and a dedicated credential contract before a client is added to the live matrix; the same rule that governs xAI Grok applies here. Before any live Claude Code row is added, record on this issue:
- the credential contract shape (input file, schema id, provider identity) and the Issue that owns it;
- which provider identities are approved for live evidence and who approved them;
- that no credential material enters this issue, fixtures, logs, or commits.
If approval is not granted, leave this issue blocked and state the concrete blocker. Do not infer consent from a configured provider, and do not substitute mock-only runs for live acceptance.
3. Support table
- Produce a support table distinguishing preserved, adapted (named policy), unsupported, unknown, and provider-specific behavior, traceable to Spike Claude Code Messages compatibility through CodexHub #74's classification and Add inbound Anthropic Messages API for Claude Code clients #75's implementation.
- Unsupported features are visible in the matrix and in the client UI (card compatibility state / request diagnostics). No feature is silently claimed.
- Version drift produces an actionable report with sanitized fixtures; scheduled/report-only upstream compatibility checks become blocking only after stability criteria are approved.
4. Release gate
- Release notes and documentation state the verified Claude Code version range, the experimental scope, and the known limitations in English and Chinese.
- A maintainer records the criteria for moving from experimental to stable, and the change that would satisfy them.
- Relevant Python, Rust, and frontend suites pass at the candidate commit; the quality report runs once and remains non-blocking.
Out of scope
Protocol implementation (#75), client configuration and UI (#76), model contract (#77), and any ACP/AgentProvider work (#85).
Acceptance criteria
- Automated E2E covers the supported text/tool/SSE contract against every approved upstream protocol.
- A real pinned Claude Code CLI passes multi-turn text, incremental streaming, tool call/result round trips, cancellation, and error handling on each approved upstream path.
- The native Anthropic path separately proves semantic-field and SSE preservation.
- The support table is complete, classified, and traceable to evidence for every capability in scope.
- Unsupported features appear in the matrix and the client UI; no silent claim remains.
- Minimum and latest verified Claude Code versions are recorded with exact executables.
- Version drift generates an actionable report with sanitized fixtures.
- The credential contract and its owning Issue are recorded before the first live Claude Code matrix row; no credentials are committed or pasted into issues.
- English and Chinese release documentation state the version range, experimental scope, and limitations.
- A maintainer records the experimental → stable criteria.
- Relevant Python, Rust, and frontend suites pass on the reviewed SHA; report-only quality findings are recorded without becoming a release gate.
Required verification
Risk class: strict — release gate, live runtime evidence, security boundary. Use the repository launcher; never a bare python/pytest.
.\scripts\codexhub-python.cmd -m pytest -q --ignore=tests/test_real_client_e2e.py
.\scripts\codexhub-python.cmd -m pytest -q tests/test_real_client_e2e.py
.\scripts\codexhub-python.cmd scripts/report_quality_gates.py
Push-Location src-tauri
cargo test --locked
cargo clippy --locked --all-targets -- -D warnings
Pop-LocationPush-Location frontend
npm run build
npm run test:ui-contract
Pop-LocationLinux: ./scripts/verify-linux.sh. Real-client acceptance is a local release-operator procedure per docs/agents/real-client-e2e.md; a pass on one operating system never replaces the other.
References
scripts/e2e_gateway_client_matrix.py,tests/test_e2e_gateway_client_matrix.pydocs/agents/real-client-e2e.md,docs/agents/verification-policy.md,docs/agents/release.mddocs/reviews/v0.3-agent-provider-roadmap.md- E2E third-party: official DeepSeek Flash #497 (OpenCode Go credential-contract precedent)
Document status
ADR-0014 is committed in draft PR #556 at ed76f0a6e22d3188879504638b2943f5bb855d41, not yet merged to main. The pinned link is readable now; the document is accepted direction, not live protocol evidence.
Activity
NOirBRight commentedon Sep 21, 2026
PARTIAL: harness/matrix now emits claude-code anthropic-messages rows and the experimental support table is in docs/evidence/claude-code-support-table.md. Live Claude Code rows remain blocked: no dedicated credential-contract Issue has been recorded, so this ticket cannot add live matrix evidence. Pinned observed CLI is 2.1.278 (loopback only).
NOirBRight commentedon Sep 21, 2026
Credential contract recorded as #558. Approved live identities: Official Codex gpt-5.6-luna (Responses conversion) and CommandCode deepseek/deepseek-v4.1-flash (Chat conversion). Native Anthropic upstream is not approved. No credentials in this comment.
NOirBRight commentedon Sep 21, 2026
Linux checkpoint at 011a312/c468a3b lineage, current campaign HEAD after test-fix merge: core pytest 3143 passed, clippy -D warnings, UI contract 7 passed. Live Linux CLI/text/tool/cancel recorded on #558. Windows/release-operator evidence is still required; Linux does not replace it. Draft PR 556 remains experimental, not ready.
NOirBRight commentedon Sep 21, 2026
Windows yoga observation: Claude Code 2.1.232 at C:\Users\noirb.local\bin\claude.exe. Pin is 2.1.278. Version-gate would exit 2 (drift). No Windows live matrix was run; Linux 2.1.278 evidence does not cover yoga. Isolated apply/readback matrix still passes on Linux including claude.
NOirBRight commentedon Sep 21, 2026
yoga Claude Code upgraded 2.1.232 → 2.1.278 via
claude update. Version-gate matches pin. Windows live matrix not started; operator will log in.NOirBRight commentedon Sep 21, 2026
Windows live on yoga: isolated checkout D:\Workstation\CodexHub-claude-live at 9eb0634. Claude Code 2.1.278 print-run through isolated Gateway Official gpt-5.6-luna returned is_error=false, result=ok. Operator worktree D:\Workstation\CodexHub left untouched. CommandCode/DeepSeek are not configured on yoga so those Windows rows were not run. No credentials in this comment.
NOirBRight commentedon Sep 21, 2026
Windows yoga isolated Gateway now has the approved CommandCode and DeepSeek Anthropic providers (copied into the isolated CODEX_HOME only). Claude Code 2.1.278 print-runs: CommandCode result=ok; DeepSeek Anthropic is_error=false result=好的. Luna already passed. Operator D:\Workstation\CodexHub untouched. No credentials in this comment.
docs: align Claude experimental notes with support table