Skip to content

feat(sandbox): allow VPC network configuration for agentcore runtimes - #1082

Open
beiton wants to merge 3 commits into
benchflow-ai:mainfrom
beiton:feat/agentcore-vpc-network-config
Open

feat(sandbox): allow VPC network configuration for agentcore runtimes#1082
beiton wants to merge 3 commits into
benchflow-ai:mainfrom
beiton:feat/agentcore-vpc-network-config

Conversation

@beiton

@beiton beiton commented Aug 31, 2026

Copy link
Copy Markdown

Runtimes were registered with a hardcoded networkConfiguration of {"networkMode": "PUBLIC"}, putting every agent microVM on the public internet with no way to opt out.

export BENCHFLOW_AGENTCORE_NETWORK_MODE="VPC"        # PUBLIC (default) or VPC
export BENCHFLOW_AGENTCORE_SUBNETS="subnet-aaa,subnet-bbb"
export BENCHFLOW_AGENTCORE_SECURITY_GROUPS="sg-aaa"

PUBLIC stays the default, so existing deployments are unchanged. These follow the existing _lifecycle_configuration() env-override pattern — say the word if you'd rather they live on SandboxConfig as per-task settings.

Setting the id lists without the mode is an error rather than a silent fallback to PUBLIC: a half-configured runtime still reaches READY and looks healthy, so the operator gets no signal that the isolation never took effect. Ids are validated client-side against the documented shapes and the 16-per-list cap.

Adoption

networkConfiguration stopped being a one-key constant the service echoes verbatim and became a nested VpcConfig, which needed two fixes:

  • _verify_adopted_runtime compared the whole returned document. VpcConfig carries requireServiceS3Endpoint, which BenchFlow never sends and cannot set on create — so that comparison would refuse to adopt a healthy VPC runtime, failing a matrix on everything after its first create. It now compares networkMode plus the two id lists, as the lifecycle check above it already does.
  • Id lists are sorted and de-duplicated. They feed _runtime_contract_digest, and json.dumps(sort_keys=True) does not sort list elements, so "a,b" and "b,a" would register two runtimes for identical infrastructure.

Verification

Wire shape checked against the bedrock-agentcore-control botocore model (2023-06-05) — note the runtime nests VpcConfig under networkModeConfig, while Browser and CodeInterpreter use vpcConfig.

pytest -k agentcore 176 passed · ty check src/ clean · ruff check . clean

Not live-validated — no VPC-attached AWS account here, so unlike #937 there is no live-run table. Whether GetAgentRuntime echoes requireServiceS3Endpoint back is inferred from the service model rather than observed; the adoption fix is defensive either way.

Left out

Real but not correctness issues, happy to fold in: an explicit NETWORK_MODE=PUBLIC is indistinguishable from unset (so a deliberate public run with the lists exported gets a misleading error), and validation happens after _images.publish(), so a typo'd id costs an image build first.

enforces_no_network is unchanged — BenchFlow cannot verify a route table from the runtime config, so no-network tasks are still refused on this backend.

beiton and others added 3 commits August 31, 2026 12:40
Runtimes were registered with a hardcoded {"networkMode": "PUBLIC"},
placing every sandbox microVM on the public internet with no way to opt
out. Adds BENCHFLOW_AGENTCORE_NETWORK_MODE / _SUBNETS / _SECURITY_GROUPS
following the existing _lifecycle_configuration() env-override pattern.
PUBLIC stays the default. Setting the lists without the mode is an error
rather than a silent fallback, since a runtime that stays public is
invisible once it reaches READY.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
VpcConfig carries service-managed members that cannot be sent on create,
so whole-document equality would refuse to adopt a healthy VPC runtime --
the path every rollout after the first takes. Compare networkMode plus the
two id lists, matching how the lifecycle check already tolerates extra
keys, and canonicalize those lists so a reordered env var does not
register a second runtime for the same infrastructure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@beiton beiton changed the title feat(sandbox): agentcore vpc network config feat(sandbox): allow VPC network configuration for agentcore runtimes Aug 31, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant