feat(runtime): support Anthropic, OpenAI, and Gemini model providers by Hweinstock · Pull Request #20 · Hweinstock/agentcore-cli

Hweinstock · 2026-09-02T21:21:53Z

Spec

Problem: The refactored Strands templates weren't confirmed to work end-to-end with non-Bedrock model providers. The --model-provider / --api-key flags (runtime/index.ts) only accepted Bedrock.

Definition of done: pass a non-Bedrock provider (anthropic, openai/open_ai, gemini) with its API key and get a working agent scaffold — creatable, runnable locally, and deployable/invokable remotely.

What changed

--model-provider now accepts Bedrock, Anthropic, OpenAI, Gemini, LiteLLM (case-insensitive; create takes the lowercase open_ai/lite_llm spellings its harness flag already uses). It is optional and defaults to Bedrock; an MCP runtime rejects a provider entirely.
Per-provider auth: Anthropic/OpenAI/Gemini require --api-key; the resolver registers an ApiKeyCredentialProvider credential, wires the template's identity provider, and writes the key to agentcore/.env.local (keyed by credentialEnvVarName) — which serves both local dev (LOCAL_DEV=1) and deploy-time provisioning. LiteLLM's key is optional (keyless routes through Bedrock via IAM).
Template deps pull provider SDKs via strands-agents[anthropic|openai|gemini|litellm] instead of stale direct pins (anthropic ~= 0.30.0 broke on modern httpx; openai ~= 1.0.0 conflicted with mcp's anyio).
LiteLLM is Python-strands only (the TypeScript template rejects it, matching mainline). model-id/api-base/params remain export-path-only, at parity with mainline.

Verification

A reviewer can reproduce the full loop with the steps below. Uses the compiled binary; us-east-1.

0. Build

git fetch <fork> feat/wire-model-providers && git checkout feat/wire-model-providers
bun install
bun run compile:linux-x64
BIN="$PWD/dist/bin/agentcore-linux-x64"

Provide API keys (any provider you want to test):

mkdir -p /tmp/acv && cd /tmp/acv
printf '%s' "$ANTHROPIC_API_KEY" > anthropic.key   # and/or openai.key, gemini.key

1. Create + scaffold (repeat per provider)

# provider ∈ {anthropic, open_ai, gemini}; key file matches
"$BIN" project create --name AnthropicAgent --template agent-python-strands \
  --model-provider anthropic --api-key file:///tmp/acv/anthropic.key --skip-git

Expected: agentcore/agentcore.json contains

"credentials": [{ "authorizerType": "ApiKeyCredentialProvider", "name": "agent_python_strandsAnthropicApiKey" }]

agentcore/.env.local contains AGENTCORE_CREDENTIAL_AGENT_PYTHON_STRANDSANTHROPICAPIKEY='<key>', and app/agent_python_strands/model/load.py renders the AnthropicModel branch (OpenAI/Gemini → OpenAIModel/GeminiModel).

2. Run locally + invoke

cd AnthropicAgent
"$BIN" project dev --mode headless --agent agent_python_strands --port 8080 --no-traces &
# wait for "Application startup complete", then:
curl -s -X POST localhost:8080/invocations -H 'Content-Type: application/json' \
  -d '{"prompt":"Reply with exactly the word PONG and nothing else."}'
kill %1

Expected: a streamed completion ending in "text": "PONG" with real token usage — i.e. a live call to the provider using the key from .env.local.

3. Deploy + invoke remotely (AWS CLI)

"$BIN" project deploy            # logs "Preparing credential provider '...ApiKey'" then "Deployed project"
ARN=$(aws cloudformation describe-stacks --stack-name AgentCore-AnthropicAgent-default \
  --query "Stacks[0].Outputs[?ends_with(OutputKey,'RuntimeArnOutput')].OutputValue" --output text)
printf '%s' '{"prompt":"Reply with exactly PONG."}' > payload.json
aws bedrock-agentcore invoke-agent-runtime --agent-runtime-arn "$ARN" \
  --runtime-session-id "verify-session-$(date +%s)-000000000000" \
  --payload fileb://payload.json --content-type application/json --accept application/json out.bin
cat out.bin

Expected: statusCode 200 and a streamed "text": "PONG" — the deployed agent authenticated via the AgentCore Identity provider provisioned from .env.local. Tear down with aws cloudformation delete-stack --stack-name AgentCore-AnthropicAgent-default (+ aws bedrock-agentcore-control delete-api-key-credential-provider --name agent_python_strandsAnthropicApiKey).

LiteLLM (no key needed)

"$BIN" project create --name LiteLLMAgent --template agent-python-strands --model-provider lite_llm --skip-git

Expected: a runtime with no credential; load.py uses LiteLLMModel (default bedrock/..., IAM) and pyproject.toml pulls strands-agents[litellm]. Passing --api-key additionally provisions a ...LiteLLMApiKey credential.

Author's results

All three keyed providers returned PONG locally (Anthropic 1119 in-tokens, OpenAI 388, Gemini 450). Anthropic was deployed to a dev account and invoked remotely (statusCode 200); resources torn down.

Automated checks

bun test → 2802 pass / 0 fail (provider cases in add/runtime/index.test.ts and project.test.ts); tsc --noEmit, oxlint, prettier --check all clean.

Hweinstock · 2026-09-02T21:50:57Z

+      const envLocal = await Bun.file(join(projectRoot, "agentcore", ".env.local")).text();
+      expect(envLocal).toContain(`${envVarName}='test-api-key'`);
+
+      const loadFile = language === "TypeScript" ? "load.ts" : "load.py";


lets avoid coupling this to the underlying templates. lets just check the agentcore spec, not the scaffolded code.

Hweinstock · 2026-09-02T21:52:50Z

+    },
+  );
+
+  test("normalizes a lowercase --model-provider to canonical casing", async () => {


could this be combined with above by generalizing the test.each slightly.

Hweinstock · 2026-09-02T21:53:18Z

+    ).rejects.toThrow(/API key is required for the Anthropic model provider/);
+  });
+
+  test("rejects a non-Bedrock provider without the strands framework", async () => {


this should be a test.each

Hweinstock · 2026-09-02T21:54:26Z

 ] as const;

-const ModelProviderFlagSchema = z.union([z.literal("Bedrock"), HarnessModelProviderSchema]);
+// Accepts both the harness provider names (bedrock, open_ai, gemini, lite_llm)


remove this comment.

Hweinstock · 2026-09-02T21:54:53Z

      flag(
        "model-provider",
-        "model provider: bedrock, open_ai, gemini, or lite_llm for harnesses; Bedrock for runtime code",
+        "model provider: bedrock, open_ai, gemini, or lite_llm for harnesses; " +


lets accept a consistent format (lowercase) and convert it to the upper case as needed.

Hweinstock · 2026-09-02T21:55:57Z

+  lite_llm: "lite_llm",
+};
+
 function resolveHarnessModelProvider(value: ModelProviderFlag | undefined): HarnessModelProvider {


why can't there a be a single resolveModelProvider function?

Hweinstock · 2026-09-02T21:56:33Z

  });

-  test("rejects non-Bedrock model providers on the runtime path", async () => {
+  test("rejects lite_llm on the runtime path", async () => {


can we make this a test.each with a single entry of lite_llm?

Hweinstock · 2026-09-02T21:56:52Z

    expect(existsSync(join(directory, "MyProject"))).toBe(false);
  });

+  test("scaffolds a runtime with an OpenAI API-key credential", async () => {


this should be a test.each for all of the non-bedrock providers.

Hweinstock · 2026-09-02T21:58:19Z

+        path: ["apiKey"],
+      });
+    }
+    if (modelProvider !== "Bedrock" && framework !== "strands") {


lets remove this. this check should happen when resolving the template. I.e. if I get passed a custom model provider in a template that doesn't support, we should reject, not upfront here.

Hweinstock · 2026-09-02T22:37:14Z

+export const MODEL_PROVIDERS = ["Bedrock", "Anthropic", "OpenAI", "Gemini"] as const;
+export type ModelProvider = (typeof MODEL_PROVIDERS)[number];
+
+const MODEL_PROVIDER_ALIASES: Record<string, ModelProvider> = {


should this be moved to shortcuts?

Hweinstock · 2026-09-02T22:37:26Z

+ * runtime's IAM credentials; the others authenticate with an API key managed
+ * through AgentCore Identity.
+ */
+export const MODEL_PROVIDERS = ["Bedrock", "Anthropic", "OpenAI", "Gemini"] as const;


should be a const enum, then we can use it for the definitions below?

Hweinstock · 2026-09-02T22:38:20Z

  });

-  test("rejects non-Bedrock model providers on the runtime path", async () => {
+  test.each([["lite_llm"]])("rejects the %s provider on the runtime path", async (provider) => {


i see lite_llm in the pyproject.toml? Why don't we support lite_llm for the runtime path?

Hweinstock · 2026-09-02T22:39:18Z

+// path keeps the lowercase names its spec uses, the runtime path takes the
+// title-cased names the templates render against. A value absent from a domain
+// (lite_llm on runtime, anthropic on harness) is unsupported there.
+const MODEL_PROVIDERS: Record<


we should align the names so that we don't need this. Unless they are written as different values to the agentcore.json spec?

Hweinstock · 2026-09-02T22:39:54Z

 ] as const;

-const ModelProviderFlagSchema = z.union([z.literal("Bedrock"), HarnessModelProviderSchema]);
+const ModelProviderFlagSchema = z.enum([...HarnessModelProviderSchema.options, "anthropic"]);


does this mean harness doesn't support anthropic?

Hweinstock · 2026-09-02T22:40:34Z

    ).rejects.toThrow(/API keys are not compatible with Bedrock model providers/);
  });
+
+  // The flag value casing varies deliberately (lowercase and canonical both


remove this comment

Hweinstock · 2026-09-02T22:41:20Z

+ * resolved, rather than by the scaffold schema — the schema cannot know which
+ * template a given framework/language/protocol maps to.
+ */
+function assertBedrockOnly(input: RuntimeResourceConfig, template: string): void {


just inline this, and remove the comment.

Hweinstock · 2026-09-02T22:41:44Z

 import { toPythonPackageName } from "../fsUtils";

+/**
+ * The AgentCore Identity wiring a non-Bedrock model provider needs: a Handlebars


this is way too verbose.

Hweinstock · 2026-09-02T23:08:33Z

 dependencies = [
-    {{#if (eq modelProvider "Anthropic")}}"anthropic ~= 0.30.0",
-    {{/if}}"a2a-sdk[all] >= 0.3.0, < 0.4.0",
+    "a2a-sdk[all] >= 0.3.0, < 0.4.0",


can we swap this ~= compatible releases to be consistent?

Hweinstock · 2026-09-02T23:21:49Z

+  const credentialName = `${input.name}${modelProvider}ApiKey`;
+  const envVarName = credentialEnvVarName(credentialName);
+  return {
+    context: { hasIdentity: true, identityProviders: [{ name: credentialName, envVarName }] },


hasIdentity and identityProviders feel redundant. is there a way we can combine into one, then do length checks to determine?

Hweinstock · 2026-09-02T23:22:25Z

+  const envVarName = credentialEnvVarName(credentialName);
+  return {
+    context: { hasIdentity: true, identityProviders: [{ name: credentialName, envVarName }] },
+    credentials: [{ authorizerType: "ApiKeyCredentialProvider", name: credentialName }],


could we generalize this to spec, and out credentials under it. Then we re-use the same merge spec logic that we do elsewhere.

Hweinstock · 2026-09-02T23:22:45Z


+/** AgentCore Identity wiring for a non-Bedrock provider; empty for Bedrock. */
+type ModelProviderIdentity = {
+  context: { hasIdentity: boolean; identityProviders: { name: string; envVarName: string }[] };


can we rename this more specific, like templateRenderContext

Hweinstock · 2026-09-02T23:24:53Z

    };
  },
  [buildResolverKey("none", "Python", "MCP")]: async (input: RuntimeResourceConfig) => {
+    if (input.scaffoldRuntimeInput.modelProvider !== "Bedrock")


an MCP server, shouldn't even accept a modelProvider. We should make modelProvider optional, validate it exists elsewhere, and reject if its defined here.

Hweinstock · 2026-09-02T23:26:32Z

-function resolveHarnessModelProvider(value: ModelProviderFlag | undefined): HarnessModelProvider {
-  return value === undefined || value === "Bedrock" ? "bedrock" : value;
-}
+// Each accepted flag value maps into whichever domains support it: the harness


make this comment more concise. basically the issue is that runtimes and harness support different models, with different names in their spec configs, Therefore, we map them here behind a consistent interface.

Hweinstock · 2026-09-02T23:27:01Z

+};

-function resolveRuntimeModelProvider(
+function resolveModelProvider(


ok actually split this back into two functions.

Hweinstock · 2026-09-02T23:27:23Z


-function resolveRuntimeModelProvider(
+function resolveModelProvider(
  value: ModelProviderFlag | undefined,


lets do a more specific name than value.

…templates at resolution

…plates

…mments

…ctions

…op redundant hasIdentity

… runtimes

Hweinstock · 2026-09-02T23:49:28Z

 };

+/** Combines several {@link SpecEntries} into one, concatenating each resource collection. */
+export function mergeSpecEntries(entries: SpecEntries[]): SpecEntries {


lets move this to a seperate file called spec.ts in the same directory, it does not belong in types.

Hweinstock · 2026-09-02T23:51:11Z

+    },
+  );
+
+  test.each<[string, string, boolean, RegExp]>([


shouldn't this parameterize over the model provider as well? Then maybe we can just check for InputValidationError to simplify

Hweinstock · 2026-09-02T23:52:34Z

+  if (provider === undefined)
+    throw new InputValidationError(
+      `runtime scaffolding does not support the '${providerFlag}' model provider ` +
+        `(expected bedrock, anthropic, open_ai, or gemini)`,


can we get this list from a type or value somewhere rather than hardcoding it? it may diverge.

Hweinstock · 2026-09-02T23:54:06Z


+/**
+ * Model providers the scaffolded runtime code supports. `Bedrock` uses the
+ * runtime's IAM credentials; the others authenticate with an API key managed


make this more concise

Hweinstock · 2026-09-02T23:54:33Z

    framework: z.enum(["strands", "none"]),
    protocol: ProtocolModeSchema.optional(),
-    modelProvider: z.enum(["Bedrock"]),
+    // Optional: an MCP runtime has no model provider at all, and other runtimes


remove this comment.

Hweinstock · 2026-09-03T01:39:04Z

 import { toPythonPackageName } from "../fsUtils";

+/**
+ * A non-Bedrock provider's contributions to a scaffolded runtime: the template


make this comment more concise.

Hweinstock · 2026-09-03T01:39:06Z

+
+function resolveModelProviderScaffold(input: RuntimeResourceConfig): ModelProviderScaffold {
+  const { modelProvider, apiKey } = input.scaffoldRuntimeInput;
+  // Only a keyed provider needs identity wiring; Bedrock — and a keyless LiteLLM,


delete this comment.

Hweinstock · 2026-09-03T01:42:10Z

+ * spec entry (its ApiKeyCredentialProvider credential), and the .env.local
+ * secret. Empty for Bedrock, which uses the runtime's IAM credentials.
+ */
+type ModelProviderScaffold = {


this name should communicate what the comment above is, but currently it doesn't.

Maybe ModelProviderTemplateConfig or something.

…omments

Hweinstock force-pushed the feat/rename-templates branch from 35468c0 to b5d6e88 Compare September 2, 2026 21:51

Hweinstock commented Sep 2, 2026

View reviewed changes

Hweinstock force-pushed the feat/rename-templates branch from b5d6e88 to 1425fec Compare September 2, 2026 22:10

Hweinstock force-pushed the feat/wire-model-providers branch from f0e9909 to 605088d Compare September 2, 2026 22:32

Hweinstock commented Sep 2, 2026

View reviewed changes

Hweinstock changed the base branch from feat/rename-templates to refactor September 2, 2026 23:29

Hweinstock added 7 commits September 2, 2026 23:30

feat(runtime): support Anthropic, OpenAI, and Gemini model providers

b675fd4

test(runtime): cover non-Bedrock model provider scaffolding

e9f8141

feat(create): support non-Bedrock model providers on the runtime path

7b52f7c

fix(templates): pull provider SDKs via strands-agents extras

41fbb4b

refactor(project): resolve model providers via one function and gate …

c364629

…templates at resolution

test(project): consolidate model-provider tests and decouple from tem…

be435a9

…plates

refactor(runtime): inline the Bedrock-only template guard and trim co…

08116c2

…mments

Hweinstock force-pushed the feat/wire-model-providers branch from ffa5de4 to 08116c2 Compare September 2, 2026 23:32

Hweinstock mentioned this pull request Sep 2, 2026

refactor(templates): reduce strands-http-python to a barebones agent #19

Closed

Hweinstock added 3 commits September 2, 2026 23:40

refactor(create): split model-provider resolution into per-domain fun…

bb56b17

…ctions

refactor(runtime): return provider credentials as spec entries and dr…

c200408

…op redundant hasIdentity

refactor(runtime): make model provider optional and reject it for MCP…

b0a4787

… runtimes

Hweinstock commented Sep 2, 2026

View reviewed changes

feat(runtime): support LiteLLM as a runtime model provider

c50a513

Hweinstock commented Sep 3, 2026

View reviewed changes

refactor(runtime): rename provider template-config type and tighten c…

63a6705

…omments

Hweinstock closed this Sep 3, 2026

Conversation

Hweinstock commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Spec

What changed

Verification

0. Build

1. Create + scaffold (repeat per provider)

2. Run locally + invoke

3. Deploy + invoke remotely (AWS CLI)

LiteLLM (no key needed)

Author's results

Automated checks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hweinstock commented Sep 2, 2026 •

edited

Loading