feat(agents): register Vercel fx agent - #1053
Closed
AdnanElAssadi wants to merge 4 commits into
Closed
Conversation
Register vercel/ in the provider registry so gateway models (vercel/<creator>/<model>, e.g. vercel/anthropic/claude-sonnet-4.5) route through the LiteLLM proxy with AI_GATEWAY_API_KEY. Declares the OpenAI-compatible completions/responses surfaces at /v1 and the Anthropic Messages surface at the root, so claude-agent-acp, codex-acp, and openhands can all run via the gateway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Comment on lines
+1006
to
+1012
| requires_env=["AI_GATEWAY_API_KEY"], | ||
| # fx speaks the AI SDK gateway wire protocol, so the proxy is skipped | ||
| # via _NATIVE_PROTOCOL_AGENTS in providers/litellm_runtime.py. | ||
| env_mapping={ | ||
| "BENCHFLOW_PROVIDER_API_KEY": "AI_GATEWAY_API_KEY", | ||
| "BENCHFLOW_PROVIDER_MODEL": "FX_MODEL", | ||
| }, |
Contributor
There was a problem hiding this comment.
🔴 fx rejected for vendor-named gateway models
When fx runs with a gateway model whose name contains a known vendor (e.g. the test's vercel/anthropic/claude-sonnet-4.5), infer_env_key_for_model returns ANTHROPIC_API_KEY and resolve_agent_env demands it. fx only takes AI_GATEWAY_API_KEY, which is absent from _EXPLICIT_AGENT_NATIVE_BRIDGE_KEYS, so validation raises and the run aborts before launch.
Prompt for agents
The fx AgentConfig maps its provider API key to AI_GATEWAY_API_KEY (env_mapping BENCHFLOW_PROVIDER_API_KEY -> AI_GATEWAY_API_KEY, requires_env=["AI_GATEWAY_API_KEY"]). But resolve_agent_env in src/benchflow/agents/env.py only special-cases 'oracle' at line 692; fx runs the full model-based key validation. For a gateway model like 'vercel/anthropic/claude-sonnet-4.5', infer_env_key_for_model falls through to the name heuristic and returns ANTHROPIC_API_KEY (registry.py ~1057). The bridge-key check (env.py ~723-738) only treats AI_GATEWAY_API_KEY as satisfying the requirement if it is in _EXPLICIT_AGENT_NATIVE_BRIDGE_KEYS (currently only {"LLM_API_KEY"}) or shares an _AUTH_CONTEXT_GROUPS entry with the inferred key. Neither holds, so validation raises 'ANTHROPIC_API_KEY required ...' and fx never launches for any gateway model whose name contains claude/gpt/gemini/opus/sonnet/haiku/o1/o3. Consider adding AI_GATEWAY_API_KEY to _EXPLICIT_AGENT_NATIVE_BRIDGE_KEYS, or registering a 'vercel' provider whose auth_env is AI_GATEWAY_API_KEY so find_provider resolves the required key correctly, or otherwise exempting native-protocol agents that supply their own gateway key from provider-key inference.
Was this helpful? React with 👍 or 👎 to provide feedback.
| f" {_apt_install('curl', 'ca-certificates')} ) && " | ||
| # Shared prefix so the sandbox user inherits the binary. | ||
| "export FX_INSTALL_DIR=/usr/local/bin && " | ||
| f"curl -fsSL https://fx.sh/setup.sh | bash -s -- {_FX_VERSION} && " |
Contributor
There was a problem hiding this comment.
🟨 Unverified remote install script piped to bash
The fx install fetches https://fx.sh/setup.sh and pipes it into bash -s. The version argument is pinned but the script body is not checksummed or signature-verified, so a compromised or tampered host runs arbitrary code in the sandbox. This matches the repo's existing curl | sh installers, so it is a hardening note rather than a new class of exposure.
Was this helpful? React with 👍 or 👎 to provide feedback.
fx speaks ACP natively (fx acp) and only talks to the Vercel AI Gateway, so it joins _NATIVE_PROTOCOL_AGENTS and skips the LiteLLM proxy; usage comes from the agent protocol response. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AdnanElAssadi
added a commit
to AdnanElAssadi/benchflow
that referenced
this pull request
Aug 23, 2026
Guards PR benchflow-ai#1053 against the review finding that vendor-named gateway models demanded ANTHROPIC_API_KEY when the vercel provider was absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AdnanElAssadi
force-pushed
the
fx-agent
branch
from
August 23, 2026 17:44
cc834ce to
9e5b5e2
Compare
Guards PR benchflow-ai#1053 against the review finding that vendor-named gateway models demanded ANTHROPIC_API_KEY when the vercel provider was absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AdnanElAssadi
force-pushed
the
fx-agent
branch
from
August 23, 2026 17:45
9e5b5e2 to
71a62e1
Compare
Author
|
Folded into #1052 to keep the Vercel AI changes together. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related to https://github.com/benchflow-ai/FrontierPhysics/issues/53
fx speaks ACP natively (fx acp) and only talks to the Vercel AI Gateway, so it joins _NATIVE_PROTOCOL_AGENTS and skips the LiteLLM proxy; usage comes from the agent protocol response.