Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
pull_request: {}
workflow_dispatch:
# The tag release calls these same gates for its exact commit.
workflow_call:
Expand All @@ -30,7 +29,7 @@ jobs:
# binary against the mock API and otherwise skips silently.
- name: Install pi
run: |
npm install -g @earendil-works/pi-coding-agent@latest
npm install -g @earendil-works/pi-coding-agent@1.1.0

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinning Pi to 1.1.0 is a coverage regression: 1.2.0 (2026-10-12) and 1.3.0 (2026-10-15, published before this PR was opened) are out, and CI previously tested @latest. A PR that breaks on 1.3.0 now merges green and only shows up in the weekly job, after the fact.

Suggest:

  • pin to the current latest (1.3.0) and make "bump the pins" part of the release checklist, or
  • keep a pinned job as the required gate and add a latest job with continue-on-error: true in PR CI, so drift is visible on the PR rather than a week later.

Either way, CONTRIBUTING should say who bumps the pin and when (a green Latest host compatibility run should be the trigger).

echo "PI_BIN=$(npm prefix -g)/bin/pi" >> "$GITHUB_ENV"
- name: Verify pi starts
run: '"$PI_BIN" --version'
Expand Down Expand Up @@ -69,14 +68,16 @@ jobs:
- run: npm ci
- name: Install Oh My Pi
run: |
npm install -g @oh-my-pi/pi-coding-agent@latest
npm install -g @oh-my-pi/pi-coding-agent@18.11.1
echo "OMP_BIN=$(npm prefix -g)/bin/omp" >> "$GITHUB_ENV"
- name: Verify omp starts
run: '"$OMP_BIN" --version'
- name: Run OMP compatibility suite
env:
OMP_COMPAT_REQUIRED: "1"
run: node tests/test-omp-compat.mjs
run: |
node tests/test-omp-compat.mjs
COMMANDCODE_CONTRACT_HOST=omp npm run test:host-protocols

# ──────────────────────────────────────────────────────────
# Code-level vulnerability scanning (SAST)
Expand Down
36 changes: 36 additions & 0 deletions .github/workflows/latest-hosts.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Latest host compatibility

on:
schedule:
- cron: "43 7 * * 1"
workflow_dispatch:

permissions:
contents: read

jobs:
latest:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- uses: oven-sh/setup-bun@v2.2.0
with:
bun-version: 1.4.0
- run: npm ci
- name: Install latest hosts (separate from pinned PR gates)
run: |
npm install -g @earendil-works/pi-coding-agent@latest @oh-my-pi/pi-coding-agent@latest
echo "PI_BIN=$(npm prefix -g)/bin/pi" >> "$GITHUB_ENV"
echo "OMP_BIN=$(npm prefix -g)/bin/omp" >> "$GITHUB_ENV"
- name: Record versions
run: |
{ "$PI_BIN" --version; "$OMP_BIN" --version; node --version; bun --version; } | tee -a "$GITHUB_STEP_SUMMARY"
- run: npm test

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On its own this job fails silently into the Actions tab (no issue, no notification beyond the default email). Fine if #163 lands; if this merges without it, note in CONTRIBUTING that someone has to look at the weekly run.

env:
PI_LOCAL_REQUIRED: "1"
OMP_COMPAT_REQUIRED: "1"
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

## Unreleased

- Preserve Chat Completions reasoning history on OMP with an explicit replay compatibility flag, verified by real Pi/OMP follow-up requests rather than only visible thinking deltas.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"verified by real Pi/OMP follow-up requests rather than only visible thinking deltas" is verification detail for the PR, not a user-facing note. Suggest: "Preserve Chat Completions reasoning history on OMP; previous reasoning was dropped from follow-up requests."

- Add shared mock protocol contracts for reasoning, system prompts, parallel tools and image results across Chat Completions, Responses and Anthropic Messages, including packed-package checks. Pin supported hosts in PR CI and check latest hosts weekly. Allow explicit multi-model live profiles without enabling paid CI calls.

## 0.7.8 - 2026-10-15

- Redact sensitive text in direct stream errors and normalized context-overflow messages before returning them to the host. This closes two error-reporting paths that did not use the shared redactor; it is defensive hardening, not a claim of a confirmed credential leak ([#155](https://github.com/patlux/pi-commandcode-provider/pull/155)).
Expand Down
8 changes: 8 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@ npm install -g @oh-my-pi/pi-coding-agent
OMP_BIN="$(npm prefix -g)/bin/omp" node tests/test-omp-compat.mjs
```

### Protocol and host contracts

`npm run test:host-protocols` runs both available hosts against loopback mocks. It verifies Chat Completions, Responses and Anthropic Messages, including system instructions, replayed reasoning/signatures, parallel tool-call/result pairing and image results on the follow-up wire. It uses temporary homes and a deterministic tool with no external side effects. Set `COMMANDCODE_CONTRACT_HOST=pi` or `omp` to select one host. Required-host environment variables make missing hosts fail rather than skip.

PR gates pin Pi **1.1.0** and OMP **18.11.1**. The weekly `Latest host compatibility` workflow tests current releases separately; it does not silently move the supported baseline. Update pins only after both the complete suite and packed-package smoke pass. CI also runs for stacked PRs, not only PRs targeting main.

Live model testing remains explicit and may incur costs. Set `COMMANDCODE_E2E_PROVIDER_MODELS` to a comma-separated list to test more than one model sequentially, for example `deepseek/deepseek-v4.1-flash,meta/muse-spark-1.3,claude-sonnet-4-6`. The selected models must be available to the account. Equivalent `_MODELS` variables work for each profile; do not combine `_MODEL` and `_MODELS`. No live credentials or paid calls are used by CI.

Before opening a PR, run:

```sh
Expand Down
7 changes: 6 additions & 1 deletion index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,11 @@ function commandCodeHeaders(): Record<string, string> | undefined {
*/
type CommandCodeProviderConfig = ProviderConfig & { usage?: UsageProvider }

type CommandCodeModelCompat = NonNullable<ProviderConfig["models"]>[number]["compat"]
// OMP requires an explicit opt-in to replay Chat Completions reasoning.
// Pi preserves the streamed signature itself and ignores this extra flag.
type CommandCodeModelCompat = NonNullable<ProviderConfig["models"]>[number]["compat"] & {
replayReasoningContent?: boolean
}

/**
* Compatibility flags per wire. Chat Completions mirrors what the Provider API
Expand Down Expand Up @@ -140,6 +144,7 @@ function compatForModel(model: CommandCodeModel): CommandCodeModelCompat {
supportsStore: false,
supportsDeveloperRole: false,
supportsReasoningEffort: MODEL_EFFORTS[model.id] !== undefined,
replayReasoningContent: model.reasoning,

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flag is now also sent to Pi. The comment says Pi ignores it — a link to the OMP flag definition (or a one-line note on which OMP version introduced it) would make the "OMP requires explicit opt-in" claim verifiable later when either host changes behavior.

maxTokensField: "max_tokens",
}
}
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"LICENSE"
],
"scripts": {
"test": "npm run typecheck && node --test tests/test-release.mjs tests/test-pi-child-cleanup.mjs && tsx tests/test-package-manifest.ts && tsx tests/test-api-key.ts && tsx tests/test-pure-functions.ts && tsx tests/test-models.ts && tsx tests/test-model-metadata-check.ts && tsx tests/test-runtime.ts && tsx tests/test-pricing.ts && tsx tests/test-pricing-check.ts && tsx tests/test-cost.ts && tsx tests/test-oauth.ts && tsx tests/test-abort.ts && tsx tests/test-overflow.ts && tsx tests/test-stream.ts && tsx tests/test-quota.ts && tsx tests/test-quota-command.ts && tsx tests/test-usage.ts && tsx tests/test-retry.ts && tsx tests/test-transport.ts && node tests/test-pi-isolated.mjs && node tests/test-pi-authenticated.mjs && node tests/test-pi-local.mjs && node tests/test-omp-compat.mjs",
"test": "npm run typecheck && node --test tests/test-release.mjs tests/test-pi-child-cleanup.mjs && tsx tests/test-package-manifest.ts && tsx tests/test-api-key.ts && tsx tests/test-pure-functions.ts && tsx tests/test-models.ts && tsx tests/test-model-metadata-check.ts && tsx tests/test-runtime.ts && tsx tests/test-pricing.ts && tsx tests/test-pricing-check.ts && tsx tests/test-cost.ts && tsx tests/test-oauth.ts && tsx tests/test-abort.ts && tsx tests/test-overflow.ts && tsx tests/test-stream.ts && tsx tests/test-quota.ts && tsx tests/test-quota-command.ts && tsx tests/test-usage.ts && tsx tests/test-retry.ts && tsx tests/test-transport.ts && node tests/test-pi-isolated.mjs && node tests/test-pi-authenticated.mjs && node tests/test-pi-local.mjs && node tests/test-omp-compat.mjs && node tests/test-host-protocols.mjs",
"typecheck": "tsc --noEmit",
"format:check": "prettier --check '**/*.{ts,mjs,json,md}'",
"format": "prettier --write '**/*.{ts,mjs,json,md}'",
Expand All @@ -54,6 +54,7 @@
"test:transport": "tsx tests/test-transport.ts",
"test:pi-isolated": "node tests/test-pi-isolated.mjs",
"test:pi-authenticated": "node tests/test-pi-authenticated.mjs",
"test:host-protocols": "node tests/test-host-protocols.mjs",
"test:pi-local": "node tests/test-pi-local.mjs",
"test:smoke": "node tests/test-smoke.mjs",
"test:release": "node --test tests/test-release.mjs",
Expand Down
19 changes: 15 additions & 4 deletions scripts/live-e2e-profile.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,25 @@ async function credentialFor(profile) {
throw new Error(`Set ${prefix}_API_KEY_FILE (recommended) or ${prefix}_API_KEY`)
}

function runProfile(profile, apiKey) {
function modelsFor(profile) {
const modelVariable =
profile === "go"
? "COMMANDCODE_E2E_GO_MODEL"
: profile === "goat"
? "COMMANDCODE_E2E_GOAT_MODEL"
: "COMMANDCODE_E2E_PROVIDER_MODEL"
const model =
const multiple = process.env[`${modelVariable}S`]
if (multiple && process.env[modelVariable])
throw new Error(`${modelVariable} and ${modelVariable}S are mutually exclusive`)
const models = multiple?.split(",").map((model) => model.trim()) ?? [
process.env[modelVariable] ??
(profile === "goat" ? "xai/grok-4.6" : "deepseek/deepseek-v4-flash")
(profile === "goat" ? "xai/grok-4.6" : "deepseek/deepseek-v4-flash"),
]
if (models.some((model) => !model)) throw new Error(`Empty model in ${modelVariable}S`)
return [...new Set(models)]
}

function runProfile(profile, apiKey, model) {
const env = {
...process.env,
COMMAND_CODE_API_KEY: apiKey,
Expand Down Expand Up @@ -79,7 +88,9 @@ function runProfile(profile, apiKey) {

try {
for (const profile of profiles) {
await runProfile(profile, await credentialFor(profile))
const models = modelsFor(profile)
const credential = await credentialFor(profile)
for (const model of models) await runProfile(profile, credential, model)
}
} catch (error) {
console.error(error instanceof Error ? error.message : String(error))
Expand Down
43 changes: 43 additions & 0 deletions tests/fixtures/protocol-tools-extension.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/** Deterministic test tools. No filesystem or network access. */
interface FixtureTool {
name: string
label: string
description: string
parameters: unknown
execute: (
id: string,
args: { image?: string | null },
) => Promise<{
content: ({ type: "text"; text: string } | { type: "image"; data: string; mimeType: string })[]
details: undefined
}>
}
interface FixtureApi {
registerTool(tool: FixtureTool): void
}

export default function (pi: FixtureApi) {
pi.registerTool({
name: "contract_read",
label: "Contract read",
description: "Return a synthetic image and text for protocol contract tests",
parameters: {
type: "object",
properties: { image: { type: ["string", "null"] } },
required: ["image"],
},
async execute(_id, args) {
return {
content: [
{ type: "text", text: `contract-result-${args.image}` },
{
type: "image",
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jRZkAAAAASUVORK5CYII=",
mimeType: "image/png",
},
],
details: undefined,
}
},
})
}
Loading
Loading