Skip to content

feat: add read-only ACP provider diagnostics for live config options and env source presence #994

Open
Open
feat: add read-only ACP provider diagnostics for live config options and env source presence#994
Assignees
skygunner
Labels
enhancementNew feature or request

Description

@swjnxyf
swjnxyf
opened on Oct 9, 2026

Context

Follow-up from #990.

#990 was resolved on WebCodex v0.4.6 after confirming a version/catalog mismatch between the Codex CLI bundled by @agentclientprotocol/codex-acp and the standalone Codex CLI selected through CODEX_PATH.

The investigation exposed a diagnostics gap: today an operator may need to attempt a Coding Run, inspect adapter internals, or perform an external ACP handshake before learning what configuration keys/values the live ACP session actually advertises.

Goal

Add a bounded, read-only Coding Agent / ACP provider diagnostic that helps operators verify provider readiness before dispatching a Coding Run.

This should improve supportability without weakening provider policy, exposing credentials, or turning diagnostics into a mutation path.

Proposed capabilities

1. Live ACP config-option diagnostics

For one explicitly selected configured provider, expose a safe projection of the config options advertised by an ephemeral ACP session, for example:

  • option IDs, such as model, mode, collaboration_mode;
  • option kind/type;
  • bounded select-option IDs/labels where safe, such as the live model catalog;
  • current/default selection only if already considered safe by the ACP contract;
  • provider/adapter identity needed to interpret the result.

The diagnostic should initialize the provider and create only the minimum ephemeral ACP session required to obtain its advertised options.

It must not:

  • send session/prompt;
  • start a model-generation turn;
  • execute provider tools;
  • mutate provider configuration;
  • auto-apply a requested option;
  • infer support from stale/static config when the live session disagrees.

A provider that cannot be initialized should return a bounded failure classification rather than raw stderr or local secrets.

2. Safe env_from_env source-presence diagnostics

For a configured provider, allow the operator to determine whether each declared source environment name is present in the effective Runner process environment.

Example safe result:

{
  "CODEX_PATH": { "present": true }
}

Do not return the environment value.

This is especially useful for Windows Desktop/service setups where a persistent User/Machine environment update may not yet be visible to the currently running Desktop/Runner process.

The diagnostic must not expose:

  • values;
  • unrelated environment names;
  • full process environment;
  • credentials/secrets;
  • inherited PATH contents or filesystem search results unless separately requested through an existing safe operator surface.

Suggested UX / tool shape

Exact naming is open, but conceptually one read-only operator surface could report:

Provider: codex
Configured: yes
Active: yes

Environment mappings:
  CODEX_PATH <- CODEX_PATH: present

Live ACP:
  initialize: ok
  session/new: ok
  config options:
    model:
      kind: select
      values: [gpt-6-sol, gpt-6.1-sol, ...]
    mode:
      kind: select
      ...

Possible placements:

  • a Runner/runtime read-only diagnostic tool;
  • Desktop Coding Agents "Diagnose" action;
  • Runtime Console projection;
  • CLI diagnostic command.

The canonical implementation should live in one runtime primitive and be reused by presentation surfaces rather than each UI reimplementing ACP probing.

Security / authority

  • Read-only.
  • Explicit provider ID required.
  • Require the same authority used for safe Runner/provider observation; do not silently grant coding_agent:run, runner:manage, SSH, or project-write authority.
  • Never return environment values, auth tokens, provider credentials, command arguments containing secrets, raw stderr, or arbitrary local file content.
  • No prompt/model generation.
  • No automatic retry loops.
  • No config mutation or restart.

If ACP initialization itself can trigger authentication UI or other side effects for a provider, the diagnostic should classify that state and stop rather than silently proceeding.

Acceptance criteria

  • Operator can inspect live ACP config-option IDs for one provider without starting a Coding Run.
  • Safe select-option values, including model IDs where the provider exposes them, can be observed with bounded output.
  • The result comes from the live ACP session, not only static Runner configuration.
  • Operator can check presence/absence of declared env_from_env source names without seeing values.
  • No prompt/model-generation request is sent.
  • No provider tool is executed.
  • No Runner/provider configuration is modified.
  • Failures use bounded reason codes and do not leak paths, secrets, environment values, or raw provider output.
  • Desktop/CLI/runtime surfaces reuse one canonical diagnostic implementation.
  • Tests cover missing env source, provider init failure, non-advertised option, select-option projection, output bounds, and secret non-disclosure.

Non-goals

  • Changing provider ownership.
  • Automatically upgrading ACP/Codex packages.
  • Automatically selecting or forcing a model.
  • Persisting arbitrary provider environment values inside WebCodex.
  • Replacing allowed_config_options / forced_config policy.
  • Broad environment inspection.

Why this matters

#990 showed that static configuration alone was insufficient to diagnose a real failure:

allowed_config_options = ["model"]
        +
ACP adapter launches an older bundled Codex
        ↓
live model catalog does not contain requested model
        ↓
coding_agent_config_invalid

A live read-only provider diagnostic would make this failure explainable before a Coding Run and would also make Windows environment propagation issues much easier to distinguish from ACP/model-catalog issues.

Refs #990.

Activity

  1. swjnxyf
    added
    enhancementNew feature or request
    on Oct 9, 2026

  2. skygunner commented on Oct 13, 2026

    @skygunner
    skygunner
    on Oct 13, 2026 – with ChatGPT Codex Connector
    Contributor

    I can take a scoped implementation pass for this issue. I checked current main before starting: ACP Runner configuration already has an explicit env_from_env allowlist and an initialize → session/new path which parses actual NewSessionResponse.config_options before any session/prompt. There is no Open PR specifically implementing #994 (I also checked the six currently open PRs).

    Proposed two-stage contribution, both with an explicit provider ID:

    1. Runner-process source presence: read only the names already declared in that provider's env_from_env mapping and report present: boolean, never source values, unrelated environment variables, executable paths or raw errors. It must query the running Runner environment, not a new CLI process's potentially different environment.
    2. Ephemeral ACP live options: use the existing supervised, bounded ACP transport to send only initialize then session/new, project the live option IDs/kinds and bounded allowlisted select choices, and always terminate the ephemeral process tree. Never send session/prompt, session/set_config_option, provider tool calls, or silently retry uncertainty. Sanitize errors before they cross the Runner/Server boundary.

    I would reuse the existing explicit provider/authorization path rather than add an unauthenticated diagnostics HTTP endpoint or rely on static config. Tests should include missing sources, malformed/oversized options, init/new failures, no prompt or config writes, fixed output budgets, and a literal secret-not-present check on every externally visible field.

    I'll keep any candidate on an independent contribution branch and post the first source-backed test evidence before asking for review. If the maintainers prefer a particular already-authorized Runner operation for read-only diagnostics, that would help minimize public API changes.

  3. yyjeqhc commented on Oct 13, 2026

    @yyjeqhc
    yyjeqhc
    on Oct 13, 2026 – with ChatGPT Codex Connector
    Owner

    Thanks for volunteering, @skygunner. This is a useful follow-up to #990, and your proposed two-stage implementation is a good fit for WebCodex. We'd welcome a focused PR (or two independently reviewable PRs).

    A few preferences from the maintainer side:

    1. Environment-source presence first. Resolve only the source names already declared in the explicitly selected provider's env_from_env, against the running Runner process environment. Return bounded present/absent flags only—no values, paths, unrelated names, or raw diagnostics. This can be a small, independently useful first step.
    2. Live ACP options second. Reuse the existing Runner-owned ACP transport and lifecycle rather than introducing a second ACP client or a generic JSON-RPC passthrough. Require an exact configured provider and an authorized Project/cwd; perform only the minimum initialize → session/new sequence, project a bounded allowlisted view of the live options, and clean up the entire ephemeral process tree on every exit path.
    3. Keep authority explicit. A live probe starts a provider process and can have provider-side initialization effects, even without session/prompt. It should not silently run as part of passive status/discovery or be exposed through an unauthenticated endpoint. Please reuse the existing exact Runner/provider identity and authorization boundaries; don't implicitly turn general observation access into arbitrary process-launch authority.
    4. Preserve safety and diagnosability. No prompt, tool execution, session/set_config_option, config mutation, auto-retry, or credential disclosure. Use bounded phase/reason codes instead of raw stderr/ACP errors; test timeouts, malformed/oversized options, stale provider identity, cleanup/cancellation, and secret non-disclosure.

    We'd prefer one canonical Runner diagnostic primitive that a later Desktop/CLI/runtime presentation can reuse, rather than requiring all UI surfaces in the first PR. Please keep the first contribution narrowly scoped and include test evidence; we're happy to review the implementation as it develops.

    Thanks again for taking this on.

  4. yyjeqhc
    assigned
    skygunner
    on Oct 13, 2026
  5. skygunner
    added a commit that references this issue on Oct 16, 2026

    feat(acp): add bounded ephemeral live ACP option probe to Diagnose

    4a450e3
  6. yyjeqhc
    added a commit that references this issue on Oct 16, 2026

    feat(acp): add bounded read-only provider diagnostics (#994) (#1129)

    c7c4731

  7. skygunner commented on Oct 27, 2026

    @skygunner
    skygunner
    on Oct 27, 2026
    Contributor

    Status follow-up from the assigned work: PR #1129 is merged (squash c7c4731e), implementing both maintainer-endorsed stages behind the typed Runner fence:

    • Stage 1 — env_from_env presence. CodingAgentRequest::Diagnose resolves only the source names declared by the explicitly selected provider and returns configured destination → present booleans. No values, no unlisted environment names, no process spawn.
    • Stage 2 — live ACP options (include_live_acp, default false). A managed ephemeral provider process, launched exactly like a Coding Run, performs only initialize → session/new. The projection is a bounded identity view (32 options / 64 choices, 128-byte identities, 256-byte labels, explicit truncation flags) and failures use only the closed outcome/phase vocabulary, with stderr drained to a sink and never captured. The whole process tree is terminated by a drop guard on every exit path; one probe runs at a time per Runner under a single 30 s budget, and a concurrent probe gets a typed busy outcome rather than queueing a later spawn.

    The three boundary weaknesses reproduced in the maintainer review (unbounded frame allocation before read_line returned, out-of-deadline mutex wait, unadvertised currentValue projection) are fixed and regression-covered in the merged code.

    Agreeing with the review, #994 stays open. What remains for acceptance, as scoped in the issue:

    1. a Server/ToolRuntime entry that independently authorizes the exact Runner, provider generation, Project, and process-launch authority before forwarding the typed Diagnose request (no generic public environment-inspection endpoint);
    2. one presentation surface reusing the canonical primitive (Desktop "Diagnose" action, Runtime Console projection, or CLI diagnostic command);
    3. real-operator ACP verification — e.g. reproducing the Support: official ACP Codex model allowlist approval and runner:manage verification #990 model-catalog mismatch through the live projection.

    If the next contribution is welcome, I can take the Server/ToolRuntime entry plus one presentation surface. My default suggestion would be the CLI diagnostic command first (smallest new auth surface, operator-scriptable), with the Desktop "Diagnose" action reusing the same Server entry — but happy to follow whichever surface the maintainers prefer.

  8. swjnxyf commented on Oct 27, 2026

    @swjnxyf
    swjnxyf
    on Oct 27, 2026 – with ChatGPT Codex Connector
    CollaboratorAuthor

    Thanks for the follow-up, @skygunner, and for getting #1129 merged. The remaining scope you outlined makes sense, and you're welcome to take the next contribution.

    I'd prefer the canonical Server/ToolRuntime Diagnose entry first, with a small CLI diagnostic command as its first consumer. The CLI should call the same authorized Server entry that Desktop can reuse later, rather than inspecting its own environment or implementing a separate ACP client.

    A few boundaries to preserve:

    • Resolve the exact Runner instance, configured provider and generation, and authorized Project/cwd before forwarding the typed Diagnose request. Reject stale or replaced identities.
    • Keep the no-spawn env_from_env presence check distinct from include_live_acp = true. The latter launches a provider process, so general read-only observation authority must not implicitly grant process-launch authority.
    • Reuse the existing Runner-owned ACP lifecycle, deadline/cancellation, and bounded response contracts. No prompts, provider tools, configuration writes, raw stderr, or secret/environment values; ensure the ephemeral process tree is cleaned up on every exit path.
    • Include authorization-denial, stale identity, malformed/oversized result, timeout/cancellation, cleanup, and non-disclosure tests. A real-operator check reproducing the Support: official ACP Codex model allowlist approval and runner:manage verification #990 model-catalog mismatch would be useful evidence.

    One focused PR is fine, or two independently reviewable PRs if the Server entry and CLI surface separate cleanly. No need to implement Desktop in this round. Let's keep #994 open until the authorized entry and at least one usable surface are verified.

    Thanks for continuing the work.

  9. skygunner commented on Oct 27, 2026

    @skygunner
    skygunner
    on Oct 27, 2026
    Contributor

    Status follow-up: PR #1329 is open, implementing the endorsed next contribution — the canonical Server/ToolRuntime Diagnose entry (diagnose_coding_agent, observe-level static authority, runtime escalation of include_live_acp to the exact start_coding_agent scope policy plus a fresh per-Project permission decision) and the first consumer surface (webcodex ops diagnose-provider through the existing /api/tools/call bridge). Server-side bound re-verification, stale-identity fail-closed handling, and the non-disclosure projection are regression-covered; suite counts are in the PR description. No Desktop surface in this round, and #994 stays open per the review guidance.

  10. yyjeqhc
    added a commit that references this issue on Oct 28, 2026

    feat(runtime): authorized Server Diagnose entry with a CLI provider d…

    f69dd40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions