-
Notifications
You must be signed in to change notification settings - Fork 1.3k
docs: plan client/proxy train and isolate management API testing #6095
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
3cd2af2
docs(devlog): plan release train 4 clients-proxy lane
lidge-jun 1ec762c
docs(skills): add isolated management API test recipe
lidge-jun 1094ec2
docs(skills): isolate Codex state before management API tests
lidge-jun 6b5d7a8
docs(devlog): dispatch exact-head dev CI after merges
lidge-jun 5752e89
docs(skills): require disposable HOME for proxy QA
lidge-jun 575453f
docs(skills): document sqlite_home precedence in management API recipe
lidge-jun 603a810
docs(devlog): drop internal agent ids from clients-proxy plan
lidge-jun ef8a5c8
docs(devlog): state clients-proxy carry requirements without defect n…
lidge-jun fc6c060
docs(devlog): pin macOS proxy activation and per-transport bypass pre…
lidge-jun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
120 changes: 120 additions & 0 deletions
120
.agents/skills/testing-opencodex-management-api/SKILL.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| --- | ||
| name: testing-opencodex-management-api | ||
| description: Exercise the OpenCodex management API in a disposable, isolated development environment without touching personal client state. | ||
| --- | ||
|
|
||
| # Testing the OpenCodex management API | ||
|
|
||
| ## Isolation is a prerequisite | ||
|
|
||
| Use a disposable OS account, container, or VM with a disposable OS home. Do not run this | ||
| recipe in your normal desktop account merely by changing `OPENCODEX_HOME`. | ||
| That variable relocates OpenCodex state, not every client or shell integration. | ||
| On macOS, even disabling `claudeCode.systemEnv` can remove an existing managed block | ||
| from the OS home's `.zshrc`; `CLAUDE_CONFIG_DIR` does not redirect that file. | ||
| Raycast integration can also update existing OpenCodex-owned entries under the OS home. | ||
| A temporary client directory alone is therefore not a complete isolation boundary. | ||
|
|
||
| Within the disposable environment, allocate a unique scratch directory and set `HOME` | ||
| to a fresh directory inside it before startup. A `HOME` override in a normal desktop | ||
| account is not a substitute for the disposable account, container, or VM. Set all of | ||
| `OPENCODEX_HOME`, `CODEX_HOME`, `CODEX_SQLITE_HOME`, `GROK_HOME`, `CLAUDE_CONFIG_DIR`, and | ||
| `OPENCODEX_CLAUDE_DESKTOP_CONFIG_DIR` to distinct directories inside it before startup. | ||
| Confirm the effective OS home belongs to the disposable account. Do not copy personal | ||
| tokens, client configuration, shell profiles, or keychain contents into this environment. | ||
| Start from a clean environment or inspect inherited path overrides before launching. | ||
| Codex SQLite state resolves in this order: a root `sqlite_home` key in | ||
| `CODEX_HOME/config.toml`, then `CODEX_SQLITE_HOME`, then `CODEX_HOME` itself. Keep the | ||
| scratch `CODEX_HOME/config.toml` free of an outside `sqlite_home`, resolve the effective | ||
| SQLite home with that precedence, and abort unless it is inside the scratch directory. | ||
|
|
||
| Save a scratch `config.json` under `OPENCODEX_HOME` with an unused loopback port: | ||
|
|
||
| ```json | ||
| { | ||
| "port": 19100, | ||
| "hostname": "127.0.0.1", | ||
| "codexAutoStart": false, | ||
| "syncResumeHistory": false, | ||
| "clientIntegrations": {"codex": false, "grok": false, "claude-desktop": false}, | ||
| "claudeCode": {"enabled": false, "injectAgents": false, "systemEnv": false} | ||
| } | ||
| ``` | ||
|
|
||
| Use both redirected client homes and integration disables. Disabled integrations may | ||
| still remove owned artifacts. `codexAutoStart` alone does not disable startup sync: | ||
| desired-state checks also consider integration settings and the hub/loopback-listener | ||
| role. Do not depend on any one flag as an isolation boundary. | ||
|
|
||
| ## Start and authenticate | ||
|
|
||
| Install the repository's locked development dependencies and use the Bun version named | ||
| by `package.json`. Check that the selected Bun executable is available in this shell; | ||
| do not assume a particular developer's PATH layout. Start one foreground instance: | ||
|
|
||
| ```sh | ||
| bun run src/cli/index.ts start --port 19100 | ||
| ``` | ||
|
|
||
| Avoid `ensure`, tray, and service installation paths for this exercise: they can spawn | ||
| detached processes or alter persistent service state. Do not enable live providers or | ||
| submit billable traffic unless that separate test is explicitly authorized. | ||
|
|
||
| Prefer reading the scratch instance's generated `admin-api-token` locally. Alternatively, | ||
| provision a randomly generated `OPENCODEX_ADMIN_AUTH_TOKEN` used only for this test. | ||
| It must differ from every data-plane API key; a collision makes management authentication | ||
| unavailable. Never paste the token into a PR, screenshot, log, or tracked fixture. | ||
|
|
||
| Management requests accept `x-opencodex-api-key: <admin-token>` or | ||
| `Authorization: Bearer <admin-token>`. Missing authorization is refused. A valid token | ||
| does not bypass route-specific origin, session, or policy requirements. Keep requests | ||
| loopback-only and do not follow redirects with credentials. | ||
|
|
||
| ## Focused Lab automation exercise | ||
|
|
||
| Read `GET /api/lab/automation` for policy and live scheduler state; inspect recorded runs | ||
| with `GET /api/lab/automation/runs`. Enabling automation is an explicit state change, | ||
| not a requirement for a basic management-authentication test. | ||
|
|
||
| A policy write uses `PUT /api/lab/automation`, for example: | ||
|
|
||
| ```json | ||
| {"policy":{"enabled":true,"layers":{"protocolConformance":true}}} | ||
| ``` | ||
|
|
||
| Serialize policy writes. The read/merge and save do not share one lock, so concurrent | ||
| writers can overwrite each other's changes even though publication itself is atomic. | ||
| Re-read the policy after changing it. | ||
|
|
||
| A fixture-only manual run uses `POST /api/lab/automation/run` with this request body: | ||
|
|
||
| ```json | ||
| {"evidenceLayer":"protocol_conformance","scenarioId":"responses-core.protocol.request-shape"} | ||
| ``` | ||
|
|
||
| For `live_route_compatibility`, include `providerName` and `modelId` in the POST request | ||
| body, not as substitute top-level configuration fields. The named provider must already | ||
| exist in `config.providers`, and live calls require authorization and suitable test | ||
| credentials. Lab must also be active at proxy startup: a later policy PUT alone does | ||
| not register the live route executor. Enable automation in the disposable home, stop | ||
| the foreground proxy, and start it again before a separately authorized live run. | ||
| The fixture-only protocol exercise above does not need this restart. Consult | ||
| `planManualLabRun` in `src/lab/automation/planner.ts` for accepted | ||
| combinations instead of guessing a scenario or provider. | ||
|
|
||
| The manual endpoint awaits dispatch and returns a run/trigger result. Inspect the returned | ||
| status rather than assuming success or a terminal run. Scheduler work is separate and | ||
| may not appear immediately; read the configured scheduler limits instead of sleeping for | ||
| a hard-coded interval. | ||
|
|
||
| ## Stop and inspect | ||
|
|
||
| Send one interrupt to the foreground process and let its bounded cleanup/drain finish. | ||
| A clean shutdown exits with zero; cleanup or drain failures may exit nonzero. A second | ||
| signal requests forced termination and is not proof of successful cleanup. | ||
| Check that the test listener and any test-owned children have stopped before removing | ||
| the exact scratch tree. Do not clean directories based on a name pattern or age. | ||
| Capture only redacted status, exit code, exact test commands, and observed results. | ||
|
|
||
| This is a development testing recipe. It does not replace the operating reference in | ||
| `skills/ocx/` or the consent rules in `AGENTS_INSTALL.md`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123 changes: 123 additions & 0 deletions
123
devlog/_plan/260927_release_train_4/clients-proxy/000_plan.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| # Release train 4: clients and proxy lane | ||
|
|
||
| At `origin/dev` `24b2f39b77` on 2026-09-27, this lane has a mix of useful client | ||
| integrations, routing changes, and proposals whose current diffs are not safe to | ||
| land. Carry the bounded changes through ordinary PRs to `dev`, correct the observed | ||
| regressions, and leave concrete reasons on proposals that need a new contract. | ||
|
|
||
| ## Loop specification | ||
|
|
||
| - Archetype: satisfy the release-train acceptance contract, one dependency-ordered | ||
| work phase per PABCD cycle. | ||
| - Trigger: the release train 4 `clients-proxy` lane assignment. | ||
| - Goal: land verified client/proxy changes that help the next release and record a | ||
| disposition for every assigned PR and issue. | ||
| - Non-goals: `main`, `preview`, releases, version changes, other lanes, writes to | ||
| contributor forks, automatic third-party installer execution, and eager client | ||
| activation on the three core request paths. | ||
| - Verifier: the focused commands in the decade docs, `bun run test:changed`, | ||
| `bun run typecheck`, `bun run structure:check`, `bun run privacy:scan`, | ||
| `bun run skill:surface:check` when CLI capabilities change, exact-head PR CI, | ||
| and a successful post-merge `dev` CI run. Conditional branches have explicit | ||
| activation cases in their phase documents. | ||
| - Stop condition: every row below has a supported merge or hold decision, each | ||
| landed PR passed its actual required jobs, source PRs/issues received the | ||
| appropriate links and disposition, and the final `dev` run succeeded. | ||
| - Memory artifact: this numbered unit, its phase evidence, and the lane's | ||
| session-bound goalplan/ledger. | ||
| - Terminal outcomes: DONE means that stop condition holds; NOOP means no | ||
| candidate survived review; NEEDS_HUMAN means an external contract or approval | ||
| blocks a specific candidate; BLOCKED means repeated external failure prevents | ||
| all meaningful progress; UNSAFE means validation found an unresolved release | ||
| blocker. There is no user-specified token or wall-clock bound. | ||
| - Escalation: a new scope, an unresolvable security boundary, or a required | ||
| external account decision goes to the coordinator. PR push/merge and issue/PR | ||
| disposition within this lane are already authorized. | ||
|
|
||
| All source edits, Git operations, and tests use this lane's dedicated | ||
| worktree checkout. The native | ||
| session directory is used only for ignored FSM and goalplan state. Local full | ||
| suite may be omitted due to seven concurrent lane worktrees; focused regressions | ||
| remain mandatory, and each PR's Verification section will state the exact | ||
| commands, results, and coverage left to CI. | ||
|
|
||
| ## Source ownership and selection | ||
|
|
||
| `structure/clients/integrations.md:27-55` assigns pure client builders to | ||
| `src/clients/config-export.ts`, detection paths to | ||
| `src/integrations/registry.ts`, and snapshot/classification/writes to the shared | ||
| integration modules. New clients stay explicit and use those seams. The three | ||
| core request files (`src/router.ts`, `src/server/lifecycle.ts`, | ||
| `src/server/responses/core.ts`) must retain the Lab import boundary enforced by | ||
| `tests/lab/core-lab-boundary.test.ts`. `src/config/proxy-env.ts` owns process | ||
| proxy activation (`structure/config-proxy.md:1-20`). | ||
|
|
||
| | Item | Current head/state | Decision and evidence | Work phase | | ||
| | --- | --- | --- | --- | | ||
| | #6051 | `987b8097`, open | Carry the disposable-home management-API recipe with a discoverable contributor link; `.agents/skills/` has no existing entry point. | [010](010_recipe.md) | | ||
| | #5893 / #5853 | `3743320a`, draft | Carry only when every macOS exception maps faithfully onto the bypass variables the active transports read, or discovery refuses before any environment write; an inherited SOCKS proxy keeps its existing path. | [020](020_macos_proxy.md) | | ||
| | #5950 / #5660 | `ef03f5ab`, open | Carry Qoder after current-base revalidation of opt-in config writes, restore, and path handling (`src/clients/config-export/qoder.ts`, PR test). | [030](030_qoder.md) | | ||
| | #5272 | `7dd796d7`, open | Carry Kilo after checking all merged config candidates; first-file-only selection can be overridden by a later legacy file (`src/clients/config-export/kilo.ts:57-63` in PR). | [040](040_kilo.md) | | ||
| | #5193 | `91090f80`, open/conflicting | Reimplement a focused Droid slice on current `dev` only if its client contract and export provenance can be proven. The PR's broad rewrite changes shared loopback export behavior. | [050](050_droid.md) | | ||
| | #5871 | `ba2d2600`, open/conflicting | Carry after conflict repair and an outbound decision-payload regression (`src/combos/jev.ts:588-595` in PR). | [060](060_jev.md) | | ||
| | #5983 / #5982 | `cd45810f`, open | Carry with explicit non-memory metadata taking precedence over the subagent header fallback (`src/server/responses/memory-models.ts:65` in PR). | [070](070_memory.md) | | ||
| | #5905 / #5679 | `19948a38`, draft | Hold: opening regular Cursor integration status can automatically fetch an external installer manifest. Decide explicit opt-in and cover timeout/status before carry. | [080](080_held_items.md) | | ||
| | #3833 | `d47e376b`, draft | Hold: Command Code rejects the exported literal `apiKey` placeholder; the PR test only checks presence. Needs supported client credential form and live client proof. | [080](080_held_items.md) | | ||
| | #4854 | open | Hold OpenScience until its actual config schema and ownership paths are established. Manual OpenAI-compatible endpoint is available. | [080](080_held_items.md) | | ||
| | #3494 | open | Hold VS Code extension integration until one named extension's supported settings and reload lifecycle are verified. | [080](080_held_items.md) | | ||
| | #1416 | open | Hold Orca launch manifest until the stopped-proxy, secret-free consumer contract is pinned; live-catalog config export is the wrong bootstrap path. | [080](080_held_items.md) | | ||
| | #2811 | open | Design only: #5016 was closed because `plan` required `managed: true` that the production inspector never reports. A reachable provenance proof precedes apply. | [080](080_held_items.md) | | ||
|
|
||
| ## Dependency order and merge method | ||
|
|
||
| `010` establishes the verification recipe, `020` owns outbound proxy activation, | ||
| `030` proves the existing client path on current `dev`, and `040`/`050` reuse that | ||
| verified roster with one client at a time. `060` precedes `070` because both touch | ||
| `src/types/config.ts`; that is a merge-conflict dependency, not a runtime one. | ||
| `080` records held items after each applicable outcome. [090](090_final_ci.md) | ||
| checks the latest integrated tree. Each carried source PR becomes a new ordinary | ||
| `dev` PR from this lane, with a `Co-authored-by` trailer in the PR description | ||
| or branch commit. Git authorship alone does not satisfy the carry policy. A | ||
| large or conflicted source diff is reduced before | ||
| landing; the source PR is thanked, linked, and closed only once its replacement | ||
| is merged. No GitHub native stack or tip-only CI exception is selected. | ||
|
|
||
| For every batch, fetch `origin/dev` again, inspect the source PR's current head | ||
| and diff, check the file-size ratchet and merged union/locale/count consumers, | ||
| run focused tests and typecheck, perform explicit security review for any | ||
| credential, proxy, installer, or authentication boundary, then inspect | ||
| required CI at the exact new PR | ||
| head before merging. GUI changes need a screenshot in the PR description from | ||
| the separate `pr-assets` branch, never committed to the PR branch. Merge only | ||
| when the new PR head contains the latest `origin/dev`; dispatch `ci.yml` on | ||
| `dev` manually as specified in [090](090_final_ci.md) and inspect its exact | ||
| head before the next batch. | ||
|
|
||
| ## Consultation and uncertainty | ||
|
|
||
| The architect proposal: D1 existing | ||
| integration ownership and D2 proxy ownership accepted; D3 Cursor discovery | ||
| amended to hold pending opt-in; D4 managed clients accepted with #3833 held; | ||
| D5 new client proposals held pending primary client contracts; D6 JEV then | ||
| memory accepted; D7 Codex updater remains design-only. Four independent source | ||
| PR reviewers examined the candidates. The architect's first reflection | ||
| found three gaps: attribution trailer, explicit security review, and the | ||
| recipe's OS-home isolation condition. All three were folded into this revision | ||
| before independent audit. Their findings are proposals; each carry is | ||
| rechecked on the actual integrated diff and current `dev`. | ||
|
|
||
| Baseline verifier preflight on `24b2f39b77`: `bun run typecheck`, | ||
| `bun run structure:check`, `bun run privacy:scan`, | ||
| `bun run skill:surface:check`, and `bun test | ||
| tests/lab/core-lab-boundary.test.ts` each exited 0; the Lab guard ran 25 | ||
| tests. These check the baseline and this planning tree only. New PR behavior | ||
| still requires the phase-specific commands after the relevant diff is present. | ||
| `bun run test:changed` on this docs-only staged diff selected zero tests and | ||
| exited 1; it is not evidence of test passage. Docs checks and semantic audit | ||
| cover the roadmap, and implementation batches rerun changed tests. | ||
|
|
||
| The same architect rechecked the D2 safety amendment and returned ALIGNED. | ||
| The independent A reviewer first | ||
| reported six blockers, then one remaining test-layout blocker; every finding | ||
| was folded into the relevant decade document and its final verdict was PASS. | ||
| This closes the roadmap design review, not any proposed code change. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.