diff --git a/AGENTS.md b/AGENTS.md index 364d6fb72..5772e4aab 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -39,7 +39,7 @@ Do not multiply entities without necessity. The long-term goal is minimal code, - A new Sandbox Provider, Harness, model provider or vendor feature changes only its adapter. It adds no Core execution path, store table or column, migration, deployment or configuration field, API field or Web UI specific to one vendor or Harness. The [Sandbox Provider guide](docs/sandbox-provider.md) and [Harness onboarding](contracts/agents-api/harness-onboarding.md) describe how to add an adapter. - When the protocol cannot express what an adapter needs, change the protocol. Never add an optional side interface for one implementation. -- Implementing the declared `CheckpointProvider` lifecycle in one vendor's Provider is an adapter change. A vendor-only pause interface, a Core path for that vendor, vendor receipts in the store or a vendor idle setting in the deployment is not. +- Implementing the declared checkpoint lifecycle in one vendor's Provider is an adapter change. A vendor-only pause interface, a Core path for that vendor, vendor receipts in the store or a vendor idle setting in the deployment is not. - Fix shared lifecycle, admission, cancellation, reuse and performance problems in the common flow, never in a branch selected by Harness, Runtime or vendor name. Core preparation and execution never branch on operating system or Environment source; platform support requires native CI builds and automated tests. - Each Harness runs its own model and tool loop through a maintained upstream SDK or native protocol, in the Environment's declared workspace directory. Its native history or configuration directory is never the workspace. Never build a second executor, a hand-written model/tool loop or a compatibility framework to fabricate parity. The public API and persistence never depend on one engine's native item types. diff --git a/contracts/agents-api/core.openapi.yaml b/contracts/agents-api/core.openapi.yaml index 27637917c..e26119271 100644 --- a/contracts/agents-api/core.openapi.yaml +++ b/contracts/agents-api/core.openapi.yaml @@ -6714,7 +6714,7 @@ paths: - Sandbox Manager /core/v1/sandbox/runtime-observations: get: - description: Core key only. Each observation is labelled with its owning Project ID. Uses the existing read-only Runtime sampler, with bounded concurrency and no execution or provisioning. A provider with a batch metrics read, such as E2B, samples the page's running sandboxes in one bounded request. + description: Core key only. Each observation is labelled with its owning Project ID. Uses the existing read-only Runtime sampler, with bounded concurrency and no execution or provisioning. parameters: - description: Last Session ID from the preceding page in: query diff --git a/contracts/agents-api/runtime-observability-api.md b/contracts/agents-api/runtime-observability-api.md index 96ee57d21..2e1db7019 100644 --- a/contracts/agents-api/runtime-observability-api.md +++ b/contracts/agents-api/runtime-observability-api.md @@ -145,7 +145,7 @@ Only list rows carry `disk`: null, or `{usage_bytes, limit_bytes}` with the rule | `unsupported` | `runtime_mode_not_observable` | `none` and `self_hosted` Sessions. | | `unavailable` | `allocation_pending` | The managed allocation does not exist yet or is being created. | | `unavailable` | `runtime_not_running` | The allocation is being cleaned up or is released, or the provider reports the Runtime absent, stopped or suspended. | -| `unavailable` | `source_not_configured` | No observation source serves the allocation's provider. | +| `unavailable` | `source_not_configured` | This Core has no managed installation identity. | | `unavailable` | `sample_timeout` | The provider read exceeded its deadline. | | `unavailable` | `sample_unavailable` | The provider could not produce a current sample. | diff --git a/contracts/agents-api/runtime-observability.md b/contracts/agents-api/runtime-observability.md index 31fe8e4ff..a30353900 100644 --- a/contracts/agents-api/runtime-observability.md +++ b/contracts/agents-api/runtime-observability.md @@ -20,13 +20,11 @@ The resolver (`services/core/internal/deployment/observation.go`) reads the Sess Managed Docker, microsandbox and E2B allocations are observed. `none` and `self_hosted` Sessions are `unsupported`; Core never attributes shared host statistics to an `environment:none` Session. -The allocation's persisted `provider_key` selects exactly one configured source, which verifies the allocation's labels or equivalent ownership data before it returns values. Before any provider read, the allocation state decides some rows: `creating` or no allocation yet gives `allocation_pending`, `cleanup_pending` or `released` gives `runtime_not_running`, and a provider key without a source gives `source_not_configured`. A provider read that exceeds its deadline gives `sample_timeout`, a not-running result `runtime_not_running`, and an unavailable result `sample_unavailable`. Any other error, an ownership mismatch or an invalid sample fails the read. +Every managed allocation is read through the deployment's selected Sandbox Provider, which verifies the allocation's installation (`provider_key`) and labels or equivalent ownership data before it returns values. Before any provider read, the allocation state decides some rows: `creating` or no allocation yet gives `allocation_pending`, `cleanup_pending` or `released` gives `runtime_not_running`, and a Core without an installation identity gives `source_not_configured`. A provider read that exceeds its deadline gives `sample_timeout`, a not-running result `runtime_not_running`, and an unavailable result `sample_unavailable`. Any other error, an ownership mismatch or an invalid sample fails the read. -The observation boundary is declared in `services/core/internal/runtimeobs/source.go`. Each registered `SourceResolver` declares supported `ResolveObservationSource`; registration validates this declaration without loading configuration or reading the database. Core resolves each provider key once per page, then validates the returned `Source` and uses that same immutable source for every read of the key on that page. An unconfigured resolver returns typed `ErrUnavailable`, which produces `sample_unavailable` without a provider type. Other resolution errors follow the provider-read error rules above. +`Observe` belongs to the [Sandbox Provider protocol](../../docs/sandbox-provider.md); `services/core/internal/runtimeobs/source.go` owns the observation types and the `Source` view of a Provider. Core loads the selected Provider and its registered kind once per page and uses that same immutable Provider for every read on that page, reading each running target with `Observe`. Without a selection the load returns typed `ErrUnavailable`, which produces `sample_unavailable` without a provider type. Other load errors follow the provider-read error rules above. -A source declares supported `ObservationProviderType`, which returns its immutable telemetry identity: a lowercase letter followed by at most 31 lowercase letters, digits or underscores. Empty identities are invalid. Provider registration and every resolved binding validate the identity and operation declarations before any sample or export. Reconfiguration affects later source resolutions; it cannot change the identity or provider selected for an in-flight page. Generation routers continue to resolve each allocation through its recorded deployment generation. - -A source implements `Observe` and declares `ObserveBatch` in its provider operations. When `ObserveBatch` is declared supported, one call reads up to 100 targets of that provider; when it is declared unsupported, Core reads each target with `Observe`. A failed batch read is never retried target by target. The [Sandbox Provider guide](../../docs/sandbox-provider.md) describes the operation declarations. +An observation's provider type is that registered kind: `docker`, `microsandbox` or `e2b`. Reconfiguration affects later pages; it cannot change the provider type or Provider selected for an in-flight page. Generation routers continue to resolve each allocation through its recorded deployment generation. ## Sample semantics @@ -56,7 +54,7 @@ Cumulative vCPU time, guest memory usage and the effective memory limit come fro ### E2B -One helper `observe` request reads a page of at most 100 allocations: E2B's batch metrics for the sandboxes named in the private receipts, and a labelled listing of the installation's running sandboxes that confirms each one. The [E2B helper](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) owns that request. It never connects to, renews or changes a sandbox and writes no receipts. +Core reads each allocation with one helper `observe` request: E2B's metrics for the sandbox named in its private receipt, and a listing of running sandboxes with the allocation's labels that confirms it. The [E2B helper](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) owns that request. It never connects to, renews or changes a sandbox and writes no receipts. | E2B value | Sample field | | --- | --- | @@ -65,11 +63,11 @@ One helper `observe` request reads a page of at most 100 allocations: E2B's batc | `memUsed`, `memTotal` | Memory usage and limit | | `diskUsed`, `diskTotal` | Disk usage and capacity, kept only when both are present and the total is nonzero | -E2B reports no cumulative CPU time, so CPU seconds stay null. `observed_at` is E2B's point time; a point up to 30 seconds ahead of Core's clock is recorded at Core's time, and a larger lead is `sample_unavailable`. A sandbox missing from the running listing is `runtime_not_running`. A missing or malformed point, an ambiguous listing and an E2B API failure, a rejected key included, are `sample_unavailable`; a malformed point affects only its own row. +E2B reports no cumulative CPU time, so CPU seconds stay null. `observed_at` is E2B's point time; a point up to 30 seconds ahead of Core's clock is recorded at Core's time, and a larger lead is `sample_unavailable`. A sandbox missing from the running listing is `runtime_not_running`. A missing or malformed point, an ambiguous listing and an E2B API failure, a rejected key included, are `sample_unavailable`. ## Read budgets -A current list read handles one page of up to 100 Sessions (default 20) with at most eight concurrent provider reads. Each provider read has two seconds, a batch read at least five, and the whole list request ten; beyond that the list returns 503. A single-Session read has two seconds. No provider call is retried within a request, and Core keeps no observation cache. +A current list read handles one page of up to 100 Sessions (default 20) with at most eight concurrent provider reads. Each provider read has two seconds and the whole list request ten; beyond that the list returns 503. A single-Session read has two seconds. No provider call is retried within a request, and Core keeps no observation cache. ## Durations @@ -123,7 +121,7 @@ With an OTLP endpoint configured, Core exports every record, both `on_read` and | `agents.session.tokens.input` | Gauge, tokens | Measured Session input tokens | | `agents.session.tokens.output` | Gauge, tokens | Measured Session output tokens | | `agents.runtime.sample` | Monotonic delta sum | One per validated result, unavailable and unsupported included | -| `agents.runtime.sample.duration` | Delta histogram, seconds | Provider read duration; a batch read counts once | +| `agents.runtime.sample.duration` | Delta histogram, seconds | Provider read duration | CPU and memory points are exported only when the sample has `started_at`; a missing measurement produces no point. Attributes are `agents.tenant.id`, `agents.session.id`, `agents.environment.id`, `agents.runtime.allocation.id`, `agents.runtime.mode`, `agents.runtime.provider.type`, `agents.runtime.status`, `agents.runtime.reason`, `agents.runtime.collection.source` and nanosecond `agents.runtime.resolved_at_unix_nano`, `agents.runtime.observed_at_unix_nano` and `agents.runtime.compute.started_at_unix_nano`. The nanosecond times keep records joinable when a backend stores event time at lower precision. Provider keys, receipts, native identifiers, raw errors, paths and credentials are never attributes. diff --git a/contracts/agents-api/zh/runtime-observability-api.md b/contracts/agents-api/zh/runtime-observability-api.md index d81203db3..984c5ab7e 100644 --- a/contracts/agents-api/zh/runtime-observability-api.md +++ b/contracts/agents-api/zh/runtime-observability-api.md @@ -1,7 +1,7 @@ --- title: "Runtime 遥测 API" source: contracts/agents-api/runtime-observability-api.md -source_hash: d6111447def530b07c392d457cce0cd553f92dafb2e7cc7f481da47f816fcab2 +source_hash: 6eca80ffefcaf8e26901659e5251518f84d2c6c93349085b378d46f9ac49149d --- Core 通过 `/core/v1` 下的只读管理员路由报告托管 Runtime 和沙箱节点所使用的信息：当前 Runtime 观测值、单个 Session 的已存储 Runtime 历史记录，以及沙箱节点的主机观测值和历史记录。读取操作绝不创建、唤醒、续期或更改计算资源，也绝不向历史记录添加样本。[Runtime observability](runtime-observability.md) 定义了 Core 如何采集和保留这些值；[Console API usage](../../../docs/zh/web/console-api-usage.md) 列出了读取这些值的 Web 页面。 @@ -147,7 +147,7 @@ Authorization: Bearer | `unsupported` | `runtime_mode_not_observable` | `none` 和 `self_hosted` Session。 | | `unavailable` | `allocation_pending` | 托管分配尚不存在或正在创建。 | | `unavailable` | `runtime_not_running` | 分配正在清理或已释放，或者提供方报告 Runtime 不存在、已停止或已暂停。 | -| `unavailable` | `source_not_configured` | 该分配的提供方未配置任何观测源。 | +| `unavailable` | `source_not_configured` | 此 Core 没有托管 installation 标识。 | | `unavailable` | `sample_timeout` | 提供方读取超过其截止时间。 | | `unavailable` | `sample_unavailable` | 提供方无法生成当前样本。 | diff --git a/contracts/agents-api/zh/runtime-observability.md b/contracts/agents-api/zh/runtime-observability.md index ddd10ea91..b4ef0f415 100644 --- a/contracts/agents-api/zh/runtime-observability.md +++ b/contracts/agents-api/zh/runtime-observability.md @@ -1,7 +1,7 @@ --- title: "运行时可观测性" source: contracts/agents-api/runtime-observability.md -source_hash: 103575f3971e77d5ba149cacb27e972e429a46713b2bda7da36cae43bbf313fa +source_hash: d18a476b06248dedfa5630ccf0f8a29dff59677a1e161d1847cc7d05e0c48de8 --- 这是面向贡献者的契约，规定 Core 如何观测 Runtime 并保留其历史。路由和响应字段见 [Runtime telemetry API](runtime-observability-api.md)。代码位于 `services/core/internal/runtimeobs`（解析、源、采样器和导出）、`internal/runtimehistory`（历史查询和 PostgreSQL 存储）以及 `internal/runtimeobs/otlpexporter`。 @@ -22,13 +22,11 @@ none: tenant_id -> session_id (no Session-owned Runtime instance) 托管 Docker、microsandbox 和 E2B 分配均会被观测。`none` 和 `self_hosted` Session 为 `unsupported`；Core 绝不会将共享主机统计信息归属于 `environment:none` Session。 -分配中持久化的 `provider_key` 会选择且仅选择一个已配置源；该源在返回数值前会验证分配标签或等效所有权数据。在读取任何 provider 之前，部分行的结果由分配状态决定：处于 `creating` 状态或尚无分配时得到 `allocation_pending`，处于 `cleanup_pending` 或 `released` 状态时得到 `runtime_not_running`，provider key 没有对应源时得到 `source_not_configured`。provider 读取超出截止时间时得到 `sample_timeout`，返回未运行结果时得到 `runtime_not_running`，返回不可用结果时得到 `sample_unavailable`。任何其他错误、所有权不匹配或无效采样都会使读取失败。 +每个托管分配都通过部署所选的 Sandbox Provider 读取；该 Provider 在返回数值前会验证分配的 installation（`provider_key`）以及分配标签或等效所有权数据。在读取任何 provider 之前，部分行的结果由分配状态决定：处于 `creating` 状态或尚无分配时得到 `allocation_pending`，处于 `cleanup_pending` 或 `released` 状态时得到 `runtime_not_running`，Core 没有 installation 标识时得到 `source_not_configured`。provider 读取超出截止时间时得到 `sample_timeout`，返回未运行结果时得到 `runtime_not_running`，返回不可用结果时得到 `sample_unavailable`。任何其他错误、所有权不匹配或无效采样都会使读取失败。 -观测边界在 `services/core/internal/runtimeobs/source.go` 中声明。每个注册的 `SourceResolver` 都声明支持 `ResolveObservationSource`；注册过程会验证此声明，但不会加载配置或读取数据库。Core 每页只解析每个 provider key 一次，随后验证返回的 `Source`，并在该页上针对此 key 的每次读取中使用同一不可变源。未配置的解析器返回类型化的 `ErrUnavailable`，从而生成不含 provider 类型的 `sample_unavailable`。其他解析错误遵循上述 provider 读取错误规则。 +`Observe` 属于 [Sandbox Provider 协议](../../../docs/zh/sandbox-provider.md)；`services/core/internal/runtimeobs/source.go` 负责观测类型以及 Provider 的 `Source` 视图。Core 每页只加载一次所选 Provider 及其注册 kind，并在该页的每次读取中使用同一不可变 Provider，用 `Observe` 读取每个运行中的目标。没有选择时，加载返回类型化的 `ErrUnavailable`，从而生成不含 provider 类型的 `sample_unavailable`。其他加载错误遵循上述 provider 读取错误规则。 -源会声明支持的 `ObservationProviderType`，该类型返回其不可变遥测标识：一个小写字母，后跟最多 31 个小写字母、数字或下划线。空标识无效。在任何采样或导出之前，provider 注册和每个解析后的绑定都会验证标识及操作声明。重新配置会影响后续的源解析，但无法更改正在处理页面所选定的标识或 provider。Generation 路由器仍会通过每个分配记录的部署代次解析该分配。 - -源实现 `Observe`，并在 provider 操作中声明 `ObserveBatch`。声明支持 `ObserveBatch` 时，一次调用可读取该 provider 的最多 100 个目标；声明不支持时，Core 使用 `Observe` 读取每个目标。批量读取失败后绝不会逐个重试目标。[Sandbox Provider guide](../../../docs/zh/sandbox-provider.md) 说明了这些操作声明。 +观测的 provider 类型即该注册 kind：`docker`、`microsandbox` 或 `e2b`。重新配置会影响后续页面，但无法更改正在处理页面所选定的 provider 类型或 Provider。Generation 路由器仍会通过每个分配记录的部署代次解析该分配。 ## 采样语义 {#sample-semantics} @@ -58,7 +56,7 @@ none: tenant_id -> session_id (no Session-owned Runtime instance) ### E2B {#e2b} -一次 helper `observe` 请求会读取一页最多 100 个分配：读取私有回执中指定 sandbox 的 E2B 批量指标，并获取该 installation 中运行中 sandbox 的带标签列表，以确认每一个 sandbox。[E2B helper](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) 负责此请求。它绝不会连接、续期或更改 sandbox，也不会写入任何回执。 +Core 用一次 helper `observe` 请求读取一个分配：读取私有回执中指定 sandbox 的 E2B 指标，并按该分配的标签列出运行中的 sandbox，以确认该 sandbox。[E2B helper](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) 负责此请求。它绝不会连接、续期或更改 sandbox，也不会写入任何回执。 | E2B 值 | 采样字段 | | --- | --- | @@ -67,11 +65,11 @@ none: tenant_id -> session_id (no Session-owned Runtime instance) | `memUsed`、`memTotal` | 内存使用量和限制 | | `diskUsed`、`diskTotal` | 磁盘使用量和容量；仅当两者都存在且总量非零时保留 | -E2B 不报告累计 CPU 时间，因此 CPU 秒数保持为 null。`observed_at` 为 E2B 的时间点；比 Core 时钟最多领先 30 秒的时间点按 Core 时间记录，领先幅度更大时则为 `sample_unavailable`。未出现在运行列表中的 sandbox 为 `runtime_not_running`。时间点缺失或格式错误、列表存在歧义以及 E2B API 失败（包括 key 被拒绝）均为 `sample_unavailable`；格式错误的时间点只影响其所在行。 +E2B 不报告累计 CPU 时间，因此 CPU 秒数保持为 null。`observed_at` 为 E2B 的时间点；比 Core 时钟最多领先 30 秒的时间点按 Core 时间记录，领先幅度更大时则为 `sample_unavailable`。未出现在运行列表中的 sandbox 为 `runtime_not_running`。时间点缺失或格式错误、列表存在歧义以及 E2B API 失败（包括 key 被拒绝）均为 `sample_unavailable`。 ## 读取预算 {#read-budgets} -当前列表读取处理一页最多 100 个 Session（默认 20 个），provider 读取并发数最多为 8。每次 provider 读取的时限为 2 秒，批量读取至少为 5 秒，整个列表请求为 10 秒；超过这些时限时，列表返回 503。单 Session 读取的时限为 2 秒。一次请求内不会重试任何 provider 调用，Core 也不保留观测缓存。 +当前列表读取处理一页最多 100 个 Session（默认 20 个），provider 读取并发数最多为 8。每次 provider 读取的时限为 2 秒，整个列表请求为 10 秒；超过这些时限时，列表返回 503。单 Session 读取的时限为 2 秒。一次请求内不会重试任何 provider 调用，Core 也不保留观测缓存。 ## 时长 {#durations} @@ -125,7 +123,7 @@ PostgreSQL 存储仅保留周期性的 `openai_hosted` 记录，因此 API 读 | `agents.session.tokens.input` | Gauge，令牌 | 实测 Session 输入令牌 | | `agents.session.tokens.output` | Gauge，令牌 | 实测 Session 输出令牌 | | `agents.runtime.sample` | 单调差值和 | 每个经验证的结果一条，包括 unavailable 和 unsupported | -| `agents.runtime.sample.duration` | Delta 直方图，秒 | Provider 读取时长；批量读取仅计一次 | +| `agents.runtime.sample.duration` | Delta 直方图，秒 | Provider 读取时长 | 仅当采样包含 `started_at` 时才导出 CPU 和内存数据点；缺少测量值不会产生数据点。属性包括 `agents.tenant.id`、`agents.session.id`、`agents.environment.id`、`agents.runtime.allocation.id`、`agents.runtime.mode`、`agents.runtime.provider.type`、`agents.runtime.status`、`agents.runtime.reason`、`agents.runtime.collection.source`，以及以纳秒为单位的 `agents.runtime.resolved_at_unix_nano`、`agents.runtime.observed_at_unix_nano` 和 `agents.runtime.compute.started_at_unix_nano`。这些纳秒时间使记录在后端以较低精度存储事件时间时仍可关联。Provider key、回执、原生标识符、原始错误、路径和凭据绝不会作为属性。 diff --git a/docs/sandbox-provider.md b/docs/sandbox-provider.md index 7b4774a29..08ea0f988 100644 --- a/docs/sandbox-provider.md +++ b/docs/sandbox-provider.md @@ -15,7 +15,7 @@ Core owns durable Environment, allocation, placement and cleanup state; the Prov ## Steps -1. **Read the contract.** Implement the five required operations and give an explicit decision for every extension interface in [Implement the interface](#implement-the-interface). +1. **Read the contract.** Implement every method, support the required operations and declare a decision for each of the others in [Implement the interface](#implement-the-interface). 2. **Write the adapter package** under `services/core/internal/sandbox/`: native SDK calls, ownership checks, identity translation and private configuration. Assert `var _ sandbox.SandboxProvider = (*YourAdapter)(nil)`. An out-of-process helper lives in `services/core/tools/-provider`. 3. **Register the kind** once, following [Register the provider kind](#register-the-provider-kind). Registration is explicit construction, not an init-time plugin registry. 4. **Label owned resources** with the provider ownership labels in the [Runtime names](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#openagentcore-runtime-names) table, and never accept older label names as a fallback. @@ -25,7 +25,7 @@ Core owns durable Environment, allocation, placement and cleanup state; the Prov ## Implement the interface -`SandboxProvider` has five required operations: +`sandbox_provider.go` holds the Core–Sandbox Provider protocol: the `SandboxProvider` interface for allocation, checkpoint and observation, its request and result types, and the setup-time `ConfigurationAdapter` with its typed errors. Value types that Core also uses beyond this boundary, such as `DeploymentSpec` and `CallFence`, live in their own files of the same package. Every method is required at compile time, and `ProviderOperations()` declares which ones the Provider supports. Five operations are always supported: | Operation | Purpose | | --- | --- | @@ -39,26 +39,23 @@ A backend without a native renewable lease, such as Docker, still keeps Core's h ### Explicit operation contracts -Every provider implements the methods of each interface below and returns a complete `ProviderOperations()` declaration. The interface methods are the operation inventory; `sandbox.ValidateOperations` checks the declaration against it without a second hand-kept list. +Every provider returns a complete `ProviderOperations()` declaration with one entry for each `SandboxProvider` method except `ProviderOperations` itself. The interface's method set is the operation inventory, and `sandbox.ValidateOperations` checks the declaration against it. The groups in the table below are lists in the protocol file, and a test keeps them equal to that method set, so a new method must join one of them. -| Contract | Requirement | Responsibility | +| Operations | Requirement | Responsibility | | --- | --- | --- | -| `sandbox.SandboxProvider` | All five operations supported | Allocation lifecycle and bounded commands | -| `sandbox.CheckpointProvider` | Explicit decision for every method, the same for all of them | Exact compute incarnations, capture and restore, retained-source resume and cleanup | -| `runtimeobs.Source` | Explicit decision | Ownership-checked read-only observations | -| `runtimeobs.BatchSource` | Explicit decision; requires `Source` | Bounded observations in input order, with per-target errors | -| `sandbox.SelectionDiscoverer` | Explicit decision | Read-only native configuration discovery before commit | -| `sandbox.CredentialVerifier` | Explicit decision | Verify access to owned resources without mutation | +| `Create`, `GetInfo`, `Renew`, `Kill`, `RunCommand` | Supported | Allocation lifecycle and bounded commands | +| `Observe` | Explicit decision | Ownership-checked read-only observation of one allocation | +| `Initial`, `NewCompute`, `GetCompute`, `Suspend`, `Resume`, `ResumeCompute`, `KillCompute`, `DeleteSnapshot`, `RunCommandCompute` | The same decision for every checkpoint method; supported only by a `nodes` registration | Exact compute incarnations, capture and restore, retained-source resume and cleanup | -`CheckpointProvider` adds `Initial` and `NewCompute` (construct compute references without allocating), `GetCompute`, `Suspend`, `Resume`, `ResumeCompute` (thaw only the same resident instance after an aborted pause), `KillCompute`, `DeleteSnapshot` and `RunCommandCompute`, which runs a bounded command in one exact compute incarnation. Core uses `RunCommandCompute` to wake a parked daemon after a restore ([`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)). +`Initial` and `NewCompute` construct compute references without allocating, `ResumeCompute` thaws only the same resident instance after an aborted pause, and `RunCommandCompute` runs a bounded command in one exact compute incarnation. Core uses `RunCommandCompute` to wake a parked daemon after a restore ([`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)). -Each declaration entry is `state: supported` with no reason, or `state: unsupported` with an authored reason code. Missing, zero, unknown or unsafe entries and missing methods fail validation. Adding a method to an interface requires an explicit decision and implementation in every adapter; never supply a base type or generate blanket unsupported implementations. +Each declaration entry is `state: supported` with no reason, or `state: unsupported` with an authored reason code. Missing, zero, unknown or unsafe entries fail validation. Adding a method to `SandboxProvider` requires an explicit decision and implementation in every adapter; never supply a base type or generate blanket unsupported implementations. An unsupported method returns `providercontract.UnsupportedError` before any native I/O. The error names the exact operation and a safe code, never a native message, resource identity, endpoint or credential. An empty result, a nil error, `Unavailable` or an unknown mutation outcome never stands in for unsupported, and the five required methods can never return it. -Each adapter owns one `Operations()` function, shared by its instance and its registration. `providers.ValidateBinding` checks both against the interfaces and each other, and Runtime admission and node generation loading also reject incomplete providers. An interface assertion establishes only method shape; callers use the declaration to decide support. +Each adapter owns one `Operations()` function, shared by its instance and its registration. `providers.ValidateBinding` checks both against the interface and each other, and Runtime admission and node generation loading also reject incomplete providers. Callers check the declaration with `providercontract.Require` before they call an operation, never a type assertion. -`ObserveBatch` returns an error, not an ambiguous boolean. Only a typed `UnsupportedError` for `ObserveBatch` permits per-target `Observe` calls; an unavailable service, timeout or other failure never does. Observation never renews, starts, prepares or stops compute; see the [observation contract](../contracts/agents-api/runtime-observability.md). +`Observe` reads one allocation and never renews, starts, prepares or stops compute; see the [observation contract](../contracts/agents-api/runtime-observability.md). Contract tests call every method declared unsupported with no native client configured, require its matching error and zero result, and reject incomplete or contradictory declarations. Supported behavior still needs native and lifecycle tests. @@ -110,9 +107,9 @@ Hosted and self-hosted Environments use the same Runtime preparation; a provider A new provider takes these steps: 1. Implement the operation contracts in the adapter package, with native contract tests. -2. Add its specification and resource validators and, for native resource discovery before commit, an optional read-only `SelectionDiscoverer`. Put native credential verification behind `CredentialVerifier`. -3. Implement `sandbox.ConfigurationAdapter` over a typed native configuration. `DecodeInput` strictly parses the separate public `configuration` and write-only `credential` objects of a request. `Encode` produces whitelisted public selectors, read-only observations and separate secret bytes, and never passes request JSON through. `Decode` restores stored selectors, and keeps access to owned resources, without remote admission or new template validation. `Normalize` copies its input before changing it. `ResolveChange`, `Equal` and `WithCredential` own inheritance, identity and credential composition. `Requirements` declares whether a credential and a public Core origin are required and whether configuration discovery is supported. Also implement `ConfigurationDiscoverer`, even when discovery is unsupported: it validates the query and returns a safe catalog, never a mutation or an admission decision, while Core keeps authorization, input limits and deadlines. A node provider accepts only an empty public object, rejects credentials and returns Unsupported for discovery and credential replacement. -4. Register its constructor, policies, configuration adapter, operation declaration and defaults in `providers/registry.go`. Node proxy identity and checkpoint support read this entry. The installer's projection combines the registered policies with the shared field bounds in `sandbox/deployment_contract.go`; regenerate it with `go run ./services/core/cmd/specification-contract -write`. +2. Add its specification and resource validators. +3. Implement `sandbox.ConfigurationAdapter` over a typed native configuration. `DecodeInput` strictly parses the separate public `configuration` and write-only `credential` objects of a request. `Encode` produces whitelisted public selectors, read-only observations and separate secret bytes, and never passes request JSON through. `Decode` restores stored selectors, and keeps access to owned resources, without remote admission or new template validation. `Normalize` copies its input before changing it. `ResolveChange`, `Equal` and `WithCredential` own inheritance, identity and credential composition. `Requirements` declares whether a credential and a public Core origin are required, and which setup operations are supported: `Discovery` for `DiscoverConfiguration`, `SelectionDiscovery` for `DiscoverSelection` and `CredentialVerification` for `VerifyCredential`. `DiscoverConfiguration` validates the query and returns a safe catalog, never a mutation or an admission decision, while Core keeps authorization, input limits and deadlines. `DiscoverSelection` resolves a candidate's omitted native values before commit, and `VerifyCredential` verifies a credential's access to owned resources without mutation. Both receive the candidate's `sandbox.DirectConfig` and build any native client for that call only. A node provider accepts only an empty public object, rejects credentials and returns Unsupported for every setup operation and for credential replacement. +4. Register its constructor, policies, configuration adapter, operation declaration and defaults in `providers/registry.go`. Its key is the provider kind, which also labels the Provider's observations, and checkpoint support reads this entry. The installer's projection combines the registered policies with the shared field bounds in `sandbox/deployment_contract.go`; regenerate it with `go run ./services/core/cmd/specification-contract -write`. 5. Supply the distribution artifacts for the adapter and its helper, and offer the provider to operators through the registered configuration contract. **Known design gap:** Web's setup views carry provider-specific options, such as E2B's views. Exposing another provider through that surface currently requires a shared Web edit. This coupling does not meet [Complexity stays in the adapter](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/AGENTS.md#complexity-stays-in-the-adapter); new integrations must express their configuration through the protocol and keep vendor-specific behavior in the adapter. Never add a Session or Turn scheduling path, a vendor column or API field, or a vendor switch in the store. @@ -126,17 +123,17 @@ A new provider takes these steps: - A `nodes` registration has only `BuildLocal`, and a `direct` registration only `BuildDirect`; missing, mixed or unknown modes are rejected. - The specification and resource validators, the configuration adapter and a complete operation declaration are mandatory, so an incomplete registration cannot publish a partial installer projection. - The Runtime input policy either accepts the pinned Runtime or gives the adapter's fixed reason for rejecting it, never both. -- Checkpoint support requires node mode and positive idle and retention defaults that fit Runtime durations; a provider without checkpoint support configures no suspension defaults. +- Checkpoint is admitted only for a `nodes` registration, because the common lifecycle suspends only node allocations; registration rejects a `direct` Provider that declares it. Checkpoint support also requires positive idle and retention defaults that fit Runtime durations, and a provider without checkpoint support configures no suspension defaults. -The configuration adapter must be non-nil, including its concrete value. Every `ConfigurationRequirements` field needs an explicit valid decision: `Credential` and `PublicOrigin` are `Required` or `NotRequired`, and `Discovery` uses the shared supported or unsupported declaration with a safe reason. A new requirement field or discovery method needs an explicit validation update and never inherits an existing decision. Configuration discovery is distinct from resource selection discovery, and requiring a credential does not promise the `VerifyCredential` operation. These checks establish complete registration, not correct native SDK behavior; constructor and adapter contract tests still apply. +The configuration adapter must be non-nil, including its concrete value. Every `ConfigurationRequirements` field needs an explicit valid decision: `Credential` and `PublicOrigin` are `Required` or `NotRequired`, and `Discovery`, `SelectionDiscovery` and `CredentialVerification` use the shared supported or unsupported declaration with a safe reason. A new requirement field needs an explicit validation update and never inherits an existing decision. Requiring a credential does not promise the `VerifyCredential` operation. These checks establish complete registration, not correct native SDK behavior; constructor and adapter contract tests still apply. ### Configuration storage and construction -Preview and persistence use `providers.Normalize` and `providers.Describe`. `SelectionDiscoverer` resolves omitted native values before commit, and the complete specification is validated again at persistence. `providers.ResolveChange` owns configuration inheritance, and comparisons use normalized selectors, so preview, retry and commit share the same defaults. The store owns transactions, credential encryption, generation fencing, resource ownership and generic object storage: only the adapter interprets `provider_config` and `provider_metadata`, and `provider_credential` holds ciphertext bound to the installation and generation. Retained generations keep their original public configuration and metadata and compose the current credential through the adapter, so a credential replacement never rewrites a retained selector. Database constraints check object structure, not the registration list. +Preview and persistence use `providers.Normalize` and `providers.Describe`. `providers.DiscoverSelection` resolves omitted native values before commit, and the complete specification is validated again at persistence. `providers.ResolveChange` owns configuration inheritance, and comparisons use normalized selectors, so preview, retry and commit share the same defaults. The store owns transactions, credential encryption, generation fencing, resource ownership and generic object storage: only the adapter interprets `provider_config` and `provider_metadata`, and `provider_credential` holds ciphertext bound to the installation and generation. Retained generations keep their original public configuration and metadata and compose the current credential through the adapter, so a credential replacement never rewrites a retained selector. Database constraints check object structure, not the registration list. A direct adapter with a credential verifies all retained generations and allocation references before a key is replaced. The common `sandbox.CallFence` excludes native calls and waits for helper completion, including calls whose callers timed out; execution invokes the prepared verification and fencing callbacks without branching on a vendor. -Vendor deployment validation and SDK setup stay at the construction boundary, and construction never creates an Environment. For node-local adapters `providers.Built` returns the provider, probe, installation identity, backend fingerprint and specification digest, and the factory also returns its close function. `execution.RuntimeProvider` binds the adapter to its kind, installation ID, backend fingerprint, generation, mode and node ownership; the database owns the selection, and the in-memory copy is never another authority. Docker and microsandbox run on nodes, and E2B is constructed directly. The node proxy exposes checkpoint operations only for a backend whose registered declaration supports them, and common lifecycle code admits suspension through `CheckpointProvider`, never through a provider name. +Vendor deployment validation and SDK setup stay at the construction boundary, and construction never creates an Environment. For node-local adapters `providers.Built` returns the provider, probe, installation identity, backend fingerprint and specification digest, and a `Quiescent` check when a helper can outlive its caller; generation collection waits for it. The factory also returns its close function. `execution.RuntimeProvider` binds the adapter to its kind, installation ID, backend fingerprint, generation, mode and node ownership; the database owns the selection, and the in-memory copy is never another authority. Docker and microsandbox run on nodes, and E2B is constructed directly. The node proxy exposes checkpoint operations only for a backend whose registered declaration supports them, and common lifecycle code admits suspension through the checkpoint declaration, never through a provider name. The backend fingerprint identifies a native resource namespace, not capacity. Core keeps deployment generations so that owned allocations keep resolving to their original backend; never repoint retained allocations at a replacement backend. @@ -144,7 +141,7 @@ The backend fingerprint identifies a native resource namespace, not capacity. Co Each node adapter's registration owns its typed `NodeArtifacts` declaration: logical distribution path, release filename suffix and installation role (`node`, `runtime`, `policy` or `image`). Registration rejects missing declarations, unsafe paths and unknown roles. `go run ./services/core/cmd/provider-artifacts -write` generates the shared Web catalog and Python projection. Run the command without `-write` to check freshness. Distribution packaging, Web availability and node installation read this projection; adding a provider's payload does not add a provider-name branch to those consumers. -The launcher supplies `sandbox.ProcessPaths` from the [derived process environment](./configuration.md). Core reads these paths once and passes them to direct construction and configuration discovery. They are fixed distribution properties, not deployment settings or user-selectable helper paths. Each adapter resolves its own relative helper and state locations; E2B uses `e2b/oac-e2b-provider` and `e2b/`. Missing or nonabsolute roots fail before helper execution. Provider construction and discovery never read process environment variables. +The launcher supplies `sandbox.ProcessPaths` from the [derived process environment](./configuration.md). Core reads these paths once and passes them to direct construction and setup operations. They are fixed distribution properties, not deployment settings or user-selectable helper paths. Each adapter resolves its own relative helper and state locations; E2B uses `e2b/oac-e2b-provider` and `e2b/`. Missing or nonabsolute roots fail before helper execution. Provider construction and discovery never read process environment variables. ## Managed lifecycle diff --git a/docs/zh/sandbox-provider.md b/docs/zh/sandbox-provider.md index 539343160..91dc2f563 100644 --- a/docs/zh/sandbox-provider.md +++ b/docs/zh/sandbox-provider.md @@ -1,7 +1,7 @@ --- title: "添加 Sandbox Provider" source: docs/sandbox-provider.md -source_hash: 36ed532c778ec8c1c4c596a011a37613ed2aa0e6ffdf20854ea30ef9a8e0c953 +source_hash: 952ffc561734637ecbae9ad46782575feefaccdfe1c41ed8c6ae1a82c2ff170b --- **Sandbox Provider** 为 Core 管理的 Environment 提供 Runtime daemon 运行所需的外层计算资源，以及启动 daemon 的有界引导流程。本指南说明如何添加 Provider，并作为 Core 驱动 Provider 的参考。接口为 [`SandboxProvider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/sandbox_provider.go)。 @@ -17,7 +17,7 @@ Core 拥有持久 Environment、allocation、placement 和 cleanup 状态；Prov ## 步骤 {#steps} -1. **阅读契约。** 实现五项必需操作，并对[实现接口](#implement-the-interface)中的每个扩展接口作出明确决定。 +1. **阅读契约。** 实现每个方法，支持必需操作，并按[实现接口](#implement-the-interface)对其余每项操作声明决定。 2. **编写 adapter 包**，放在 `services/core/internal/sandbox/`：包括原生 SDK 调用、所有权检查、身份转换和私有配置。声明 `var _ sandbox.SandboxProvider = (*YourAdapter)(nil)`。进程外 helper 放在 `services/core/tools/-provider`。 3. **注册 kind** 一次，遵循[注册 provider kind](#register-the-provider-kind)。注册是明确构造，不是 init 时 plugin registry。 4. **标记所属资源**，使用 [Runtime 名称](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#openagentcore-runtime-names)表中的 provider ownership label，不接受旧 label 名称作为回退。 @@ -27,7 +27,7 @@ Core 拥有持久 Environment、allocation、placement 和 cleanup 状态；Prov ## 实现接口 {#implement-the-interface} -`SandboxProvider` 有五项必需操作： +`sandbox_provider.go` 包含 Core–Sandbox Provider 协议：负责 allocation、checkpoint 和观测的 `SandboxProvider` 接口及其请求与结果类型，以及 setup 阶段的 `ConfigurationAdapter` 及其类型化错误。Core 在此边界之外也使用的值类型，例如 `DeploymentSpec` 和 `CallFence`，位于同一 package 的独立文件中。每个方法在编译期都必须实现，`ProviderOperations()` 声明 Provider 支持哪些方法。以下五项操作始终支持： | 操作 | 用途 | | --- | --- | @@ -41,26 +41,23 @@ Docker 等没有原生可续期租约的 backend 仍遵守 Core 的 hosted expir ### 明确的操作契约 {#explicit-operation-contracts} -每个 provider 实现以下各接口的方法，并返回完整的 `ProviderOperations()` 声明。接口方法就是操作清单；`sandbox.ValidateOperations` 根据接口检查声明，无需第二份手动维护的清单。 +每个 provider 返回完整的 `ProviderOperations()` 声明，`SandboxProvider` 中除 `ProviderOperations` 本身外的每个方法各占一项。接口的方法集就是操作清单，`sandbox.ValidateOperations` 根据它检查声明。下表中的分组是协议文件中的列表，由测试保证它们与该方法集一致，因此新增方法必须加入其中一组。 -| 契约 | 要求 | 职责 | +| 操作 | 要求 | 职责 | | --- | --- | --- | -| `sandbox.SandboxProvider` | 支持全部五项操作 | Allocation 生命周期与有界命令 | -| `sandbox.CheckpointProvider` | 对每个方法明确决定，所有方法一致 | 精确计算实例、捕获与恢复、保留源恢复和清理 | -| `runtimeobs.Source` | 明确决定 | 检查所有权的只读观测 | -| `runtimeobs.BatchSource` | 明确决定；要求 `Source` | 按输入顺序提供有界观测，包含每目标错误 | -| `sandbox.SelectionDiscoverer` | 明确决定 | 提交前只读原生配置发现 | -| `sandbox.CredentialVerifier` | 明确决定 | 验证对所属资源的访问，不修改资源 | +| `Create`、`GetInfo`、`Renew`、`Kill`、`RunCommand` | 支持 | Allocation 生命周期与有界命令 | +| `Observe` | 明确决定 | 检查所有权、只读地观测一个 allocation | +| `Initial`、`NewCompute`、`GetCompute`、`Suspend`、`Resume`、`ResumeCompute`、`KillCompute`、`DeleteSnapshot`、`RunCommandCompute` | 所有 checkpoint 方法决定一致；仅 `nodes` 注册可以支持 | 精确计算实例、捕获与恢复、保留源恢复和清理 | -`CheckpointProvider` 增加 `Initial` 和 `NewCompute`（构造 compute reference，不分配资源）、`GetCompute`、`Suspend`、`Resume`、`ResumeCompute`（暂停中止后仅解冻同一驻留实例）、`KillCompute`、`DeleteSnapshot` 和 `RunCommandCompute`，后者在一个精确 compute incarnation 中运行有界命令。Core 使用 `RunCommandCompute` 在恢复后唤醒 parked daemon（[`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)）。 +`Initial` 和 `NewCompute` 构造 compute reference，不分配资源；`ResumeCompute` 在暂停中止后仅解冻同一驻留实例；`RunCommandCompute` 在一个精确 compute incarnation 中运行有界命令。Core 使用 `RunCommandCompute` 在恢复后唤醒 parked daemon（[`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)）。 -每个声明项为不带 reason 的 `state: supported`，或带 authored reason code 的 `state: unsupported`。缺失、零值、未知或不安全项以及缺失方法都会验证失败。给接口添加方法时，必须在每个 adapter 中明确决定并实现；不提供 base type，也不生成笼统的不支持实现。 +每个声明项为不带 reason 的 `state: supported`，或带 authored reason code 的 `state: unsupported`。缺失、零值、未知或不安全项都会验证失败。给 `SandboxProvider` 添加方法时，必须在每个 adapter 中明确决定并实现；不提供 base type，也不生成笼统的不支持实现。 不支持的方法在任何原生 I/O 前返回 `providercontract.UnsupportedError`。错误指明精确操作和安全 code，不包含原生消息、资源身份、endpoint 或凭据。空结果、nil error、`Unavailable` 或未知 mutation 结果都不能代替 unsupported，五项必需方法不能返回 unsupported。 -每个 adapter 拥有一个 `Operations()` 函数，由实例和注册共享。`providers.ValidateBinding` 根据接口并相互对照检查两者，Runtime admission 与 node generation 加载也拒绝不完整 provider。interface assertion 仅证明方法形态；调用方使用声明决定支持情况。 +每个 adapter 拥有一个 `Operations()` 函数，由实例和注册共享。`providers.ValidateBinding` 根据接口并相互对照检查两者，Runtime admission 与 node generation 加载也拒绝不完整 provider。调用方在调用操作前用 `providercontract.Require` 检查声明，从不使用 type assertion。 -`ObserveBatch` 返回 error，不返回有歧义的 boolean。仅 `ObserveBatch` 的类型化 `UnsupportedError` 允许逐目标调用 `Observe`；服务不可用、超时或其他失败都不允许。观测不续期、启动、准备或停止计算资源；参见[观测契约](../../contracts/agents-api/zh/runtime-observability.md)。 +`Observe` 读取一个 allocation，不续期、启动、准备或停止计算资源；参见[观测契约](../../contracts/agents-api/zh/runtime-observability.md)。 契约测试在未配置原生 client 时调用每个声明不支持的方法，要求匹配错误和零值结果，并拒绝不完整或矛盾的声明。支持的行为仍需要原生和生命周期测试。 @@ -112,9 +109,9 @@ Checkpoint 支持增加 `Compute` generation、name、ID 和 `SnapshotIdentity` 新 provider 执行以下步骤： 1. 在 adapter 包中实现 operation 契约，并编写原生契约测试。 -2. 添加 specification 和 resource validator；提交前需要原生资源发现时，添加可选只读 `SelectionDiscoverer`。原生凭据验证放在 `CredentialVerifier` 后。 -3. 基于类型化原生配置实现 `sandbox.ConfigurationAdapter`。`DecodeInput` 严格解析请求中独立的公开 `configuration` 与只写 `credential` 对象。`Encode` 生成白名单公开 selector、只读观测和独立 secret bytes，不透传请求 JSON。`Decode` 恢复已存储 selector 并保留对所属资源的访问，不做远程 admission 或新模板验证。`Normalize` 修改前复制输入。`ResolveChange`、`Equal` 和 `WithCredential` 负责继承、身份与凭据组合。`Requirements` 声明是否需要凭据和公开 Core origin，以及是否支持配置发现。即使不支持 discovery，也实现 `ConfigurationDiscoverer`：验证 query 并返回安全 catalog，不做 mutation 或 admission decision；Core 保留授权、输入限制与 deadline。node provider 仅接受空公开对象，拒绝凭据，对 discovery 和 credential replacement 返回 Unsupported。 -4. 在 `providers/registry.go` 中注册 constructor、policy、configuration adapter、operation 声明和默认值。Node proxy identity 和 checkpoint 支持读取此项。installer 投影组合已注册 policy 与 `sandbox/deployment_contract.go` 中的共享 field bound；通过 `go run ./services/core/cmd/specification-contract -write` 重新生成。 +2. 添加 specification 和 resource validator。 +3. 基于类型化原生配置实现 `sandbox.ConfigurationAdapter`。`DecodeInput` 严格解析请求中独立的公开 `configuration` 与只写 `credential` 对象。`Encode` 生成白名单公开 selector、只读观测和独立 secret bytes，不透传请求 JSON。`Decode` 恢复已存储 selector 并保留对所属资源的访问，不做远程 admission 或新模板验证。`Normalize` 修改前复制输入。`ResolveChange`、`Equal` 和 `WithCredential` 负责继承、身份与凭据组合。`Requirements` 声明是否需要凭据和公开 Core origin，以及支持哪些 setup 操作：`Discovery` 对应 `DiscoverConfiguration`，`SelectionDiscovery` 对应 `DiscoverSelection`，`CredentialVerification` 对应 `VerifyCredential`。`DiscoverConfiguration` 验证 query 并返回安全 catalog，不做 mutation 或 admission decision；Core 保留授权、输入限制与 deadline。`DiscoverSelection` 在提交前解析候选项省略的原生值，`VerifyCredential` 验证凭据对所属资源的访问，不修改资源。两者都接收候选项的 `sandbox.DirectConfig`，原生 client 只为该次调用构造。node provider 仅接受空公开对象，拒绝凭据，对每项 setup 操作和 credential replacement 返回 Unsupported。 +4. 在 `providers/registry.go` 中注册 constructor、policy、configuration adapter、operation 声明和默认值。其键即 provider kind，也用于标记该 Provider 的观测；checkpoint 支持读取此项。installer 投影组合已注册 policy 与 `sandbox/deployment_contract.go` 中的共享 field bound；通过 `go run ./services/core/cmd/specification-contract -write` 重新生成。 5. 提供 adapter 和 helper 的发行产物，通过已注册 configuration 契约向运维人员提供 provider。 **已知设计缺口：** Web 的 setup view 携带 provider 专有选项，如 E2B 的 view。通过该界面提供另一 provider 目前需要修改共享的 Web。此耦合不符合[复杂性留在 adapter 内](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/AGENTS.md#complexity-stays-in-the-adapter)；新集成必须通过协议表达配置，把厂商专有行为留在 adapter。不得添加 Session 或 Turn 调度路径、厂商专有 column 或 API field，或 store 中的厂商 switch。 @@ -128,17 +125,17 @@ Checkpoint 支持增加 `Compute` generation、name、ID 和 `SnapshotIdentity` - `nodes` 注册仅有 `BuildLocal`，`direct` 注册仅有 `BuildDirect`；缺失、混合或未知 mode 被拒绝。 - specification 和 resource validator、configuration adapter 与完整 operation 声明都是必需项，因此不完整注册不能发布部分 installer projection。 - Runtime input policy 要么接受固定 Runtime，要么给出 adapter 拒绝它的固定原因，不能两者兼有。 -- Checkpoint 支持要求 node mode 和适合 Runtime duration 的正 idle、retention 默认值；不支持 checkpoint 的 provider 不配置 suspension 默认值。 +- Checkpoint 仅准入 `nodes` 注册，因为公共 lifecycle 只暂停 node allocation；声明 checkpoint 的 `direct` Provider 会被注册拒绝。Checkpoint 支持还要求适合 Runtime duration 的正 idle、retention 默认值，不支持 checkpoint 的 provider 不配置 suspension 默认值。 -configuration adapter 必须非 nil，包括其具体值。每个 `ConfigurationRequirements` 字段都需要明确有效的决定：`Credential` 和 `PublicOrigin` 为 `Required` 或 `NotRequired`，`Discovery` 使用共享 supported 或 unsupported 声明并携带安全 reason。新增 requirement field 或 discovery method 需要明确更新验证，不继承已有决定。configuration discovery 与 resource selection discovery 不同，要求凭据也不承诺支持 `VerifyCredential` 操作。这些检查证明注册完整，不证明原生 SDK 行为正确；constructor 和 adapter 契约测试仍然适用。 +configuration adapter 必须非 nil，包括其具体值。每个 `ConfigurationRequirements` 字段都需要明确有效的决定：`Credential` 和 `PublicOrigin` 为 `Required` 或 `NotRequired`，`Discovery`、`SelectionDiscovery` 和 `CredentialVerification` 使用共享 supported 或 unsupported 声明并携带安全 reason。新增 requirement field 需要明确更新验证，不继承已有决定。要求凭据不承诺支持 `VerifyCredential` 操作。这些检查证明注册完整，不证明原生 SDK 行为正确；constructor 和 adapter 契约测试仍然适用。 ### 配置存储与构造 {#configuration-storage-and-construction} -预览和持久化使用 `providers.Normalize` 与 `providers.Describe`。`SelectionDiscoverer` 在提交前解析省略的原生值，持久化时再次验证完整 specification。`providers.ResolveChange` 负责配置继承，比较使用 normalized selector，使预览、重试和提交共享默认值。store 负责事务、凭据加密、generation fencing、资源所有权和通用对象存储：仅 adapter 解释 `provider_config` 与 `provider_metadata`，`provider_credential` 保存绑定到安装实例与 generation 的密文。保留 generation 保持原公开配置和 metadata，通过 adapter 组合当前凭据，因此替换凭据不重写保留 selector。数据库约束检查对象结构，不检查注册列表。 +预览和持久化使用 `providers.Normalize` 与 `providers.Describe`。`providers.DiscoverSelection` 在提交前解析省略的原生值，持久化时再次验证完整 specification。`providers.ResolveChange` 负责配置继承，比较使用 normalized selector，使预览、重试和提交共享默认值。store 负责事务、凭据加密、generation fencing、资源所有权和通用对象存储：仅 adapter 解释 `provider_config` 与 `provider_metadata`，`provider_credential` 保存绑定到安装实例与 generation 的密文。保留 generation 保持原公开配置和 metadata，通过 adapter 组合当前凭据，因此替换凭据不重写保留 selector。数据库约束检查对象结构，不检查注册列表。 具有凭据的 direct adapter 在替换 key 前验证全部保留 generation 与 allocation reference。公共 `sandbox.CallFence` 排除原生调用并等待 helper 完成，包括调用方已超时的调用；execution 调用已准备的 verification 和 fencing callback，不按厂商分支。 -厂商部署验证和 SDK setup 留在构造边界，构造不创建 Environment。node-local adapter 的 `providers.Built` 返回 provider、probe、installation identity、backend fingerprint 和 specification digest，factory 还返回 close 函数。`execution.RuntimeProvider` 将 adapter 绑定到 kind、installation ID、backend fingerprint、generation、mode 和 node ownership；选择由数据库负责，内存副本不构成另一权限来源。Docker 与 microsandbox 在 node 上运行，E2B 直接构造。node proxy 仅对注册声明支持 checkpoint 的 backend 暴露 checkpoint 操作，公共 lifecycle 通过 `CheckpointProvider` 准入 suspension，不通过 provider name。 +厂商部署验证和 SDK setup 留在构造边界，构造不创建 Environment。node-local adapter 的 `providers.Built` 返回 provider、probe、installation identity、backend fingerprint 和 specification digest；helper 可能比调用方存活更久时，还返回 `Quiescent` 检查，generation 回收会等待它。factory 还返回 close 函数。`execution.RuntimeProvider` 将 adapter 绑定到 kind、installation ID、backend fingerprint、generation、mode 和 node ownership；选择由数据库负责，内存副本不构成另一权限来源。Docker 与 microsandbox 在 node 上运行，E2B 直接构造。node proxy 仅对注册声明支持 checkpoint 的 backend 暴露 checkpoint 操作，公共 lifecycle 通过 checkpoint 声明准入 suspension，不通过 provider name。 backend fingerprint 标识原生资源命名空间，不表示容量。Core 保留部署 generation，使所属 allocation 继续解析到原 backend；不要将保留 allocation 重新指向替代 backend。 @@ -146,7 +143,7 @@ backend fingerprint 标识原生资源命名空间，不表示容量。Core 保 每个 node adapter 注册负责其类型化 `NodeArtifacts` 声明：逻辑发行路径、release filename suffix 和安装角色（`node`、`runtime`、`policy` 或 `image`）。注册拒绝缺失声明、不安全路径和未知角色。`go run ./services/core/cmd/provider-artifacts -write` 生成共享 Web catalog 和 Python projection。不带 `-write` 运行可检查是否最新。发行打包、Web availability 和 node 安装读取此投影；添加 provider payload 不在这些消费者中增加 provider-name 分支。 -launcher 从[派生进程环境](configuration.md)提供 `sandbox.ProcessPaths`。Core 读取这些路径一次，并传给 direct construction 和 configuration discovery。它们是固定发行属性，不是部署设置或用户可选 helper 路径。每个 adapter 解析自己的相对 helper 和 state 位置；E2B 使用 `e2b/oac-e2b-provider` 和 `e2b/`。root 缺失或不是绝对路径时，在执行 helper 前失败。Provider 构造与发现不读取进程环境变量。 +launcher 从[派生进程环境](configuration.md)提供 `sandbox.ProcessPaths`。Core 读取这些路径一次，并传给 direct construction 和 setup 操作。它们是固定发行属性，不是部署设置或用户可选 helper 路径。每个 adapter 解析自己的相对 helper 和 state 位置；E2B 使用 `e2b/oac-e2b-provider` 和 `e2b/`。root 缺失或不是绝对路径时，在执行 helper 前失败。Provider 构造与发现不读取进程环境变量。 ## 托管生命周期 {#managed-lifecycle} diff --git a/services/core/cmd/sandbox-node/generations.go b/services/core/cmd/sandbox-node/generations.go index 3cd17d93f..78f41af37 100644 --- a/services/core/cmd/sandbox-node/generations.go +++ b/services/core/cmd/sandbox-node/generations.go @@ -171,7 +171,7 @@ func buildGeneration(registry *providerconfig.Registry, config providerconfig.Co if err != nil { return node.GenerationProvider{}, err } - return node.GenerationProvider{Generation: config.Generation, SpecificationDigest: built.SpecificationDigest, Provider: built.Provider, Probe: built.Probe, Close: closeProvider}, nil + return node.GenerationProvider{Generation: config.Generation, SpecificationDigest: built.SpecificationDigest, Provider: built.Provider, Probe: built.Probe, Quiescent: built.Quiescent, Close: closeProvider}, nil } type generationJournal struct { diff --git a/services/core/cmd/server/main.go b/services/core/cmd/server/main.go index a297e4e58..f45a3a2f9 100644 --- a/services/core/cmd/server/main.go +++ b/services/core/cmd/server/main.go @@ -180,10 +180,10 @@ func run(config processconfig.Config) error { }) defer managedNodes.close() var managed *execution.RuntimeProvider - observationSources := map[string]runtimeobs.SourceResolver{} + var observationSource func(context.Context) (runtimeobs.Source, string, error) if managedNodes != nil { managed = managedNodes.runtime - observationSources[managed.InstallationID] = managedNodes.setup + observationSource = managedNodes.setup.observationSource } observationResolver, err := deployment.NewObservationResolver(sessionStore, deploymentStore) if err != nil { @@ -193,7 +193,7 @@ func run(config processconfig.Config) error { if err != nil { return err } - observationService, err := runtimeobs.NewService(observationResolver, observationSources, history.Options...) + observationService, err := runtimeobs.NewService(observationResolver, observationSource, history.Options...) if err != nil { if history.Exporter != nil { closeCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) diff --git a/services/core/cmd/server/managed_generation_operations.go b/services/core/cmd/server/managed_generation_operations.go index 8c39850fa..2f7c99b65 100644 --- a/services/core/cmd/server/managed_generation_operations.go +++ b/services/core/cmd/server/managed_generation_operations.go @@ -15,11 +15,7 @@ func (p *generationRouter) Initial(ctx context.Context, r sandbox.Reference) (sa return sandbox.Compute{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.Compute{}, err - } - return cp.Initial(ctx, r) + return v.Initial(ctx, r) } func (p *generationRouter) NewCompute(ctx context.Context, r sandbox.Reference, g uint64, snapshot *sandbox.SnapshotIdentity) (sandbox.Compute, error) { if err := providercontract.Require(p, "NewCompute"); err != nil { @@ -30,11 +26,7 @@ func (p *generationRouter) NewCompute(ctx context.Context, r sandbox.Reference, return sandbox.Compute{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.Compute{}, err - } - return cp.NewCompute(ctx, r, g, snapshot) + return v.NewCompute(ctx, r, g, snapshot) } func (p *generationRouter) GetCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute) (sandbox.ComputeState, error) { if err := providercontract.Require(p, "GetCompute"); err != nil { @@ -45,11 +37,7 @@ func (p *generationRouter) GetCompute(ctx context.Context, r sandbox.Reference, return sandbox.ComputeState{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.ComputeState{}, err - } - return cp.GetCompute(ctx, r, c) + return v.GetCompute(ctx, r, c) } func (p *generationRouter) Suspend(ctx context.Context, q sandbox.SuspendRequest) (sandbox.ComputeState, error) { if err := providercontract.Require(p, "Suspend"); err != nil { @@ -60,11 +48,7 @@ func (p *generationRouter) Suspend(ctx context.Context, q sandbox.SuspendRequest return sandbox.ComputeState{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.ComputeState{}, err - } - return cp.Suspend(ctx, q) + return v.Suspend(ctx, q) } func (p *generationRouter) Resume(ctx context.Context, q sandbox.ResumeRequest) (sandbox.ComputeState, error) { if err := providercontract.Require(p, "Resume"); err != nil { @@ -75,11 +59,7 @@ func (p *generationRouter) Resume(ctx context.Context, q sandbox.ResumeRequest) return sandbox.ComputeState{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.ComputeState{}, err - } - return cp.Resume(ctx, q) + return v.Resume(ctx, q) } func (p *generationRouter) KillCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute) error { if err := providercontract.Require(p, "KillCompute"); err != nil { @@ -90,11 +70,7 @@ func (p *generationRouter) KillCompute(ctx context.Context, r sandbox.Reference, return err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return err - } - return cp.KillCompute(ctx, r, c) + return v.KillCompute(ctx, r, c) } func (p *generationRouter) DeleteSnapshot(ctx context.Context, r sandbox.Reference, snapshot sandbox.SnapshotIdentity) error { if err := providercontract.Require(p, "DeleteSnapshot"); err != nil { @@ -105,11 +81,7 @@ func (p *generationRouter) DeleteSnapshot(ctx context.Context, r sandbox.Referen return err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return err - } - return cp.DeleteSnapshot(ctx, r, snapshot) + return v.DeleteSnapshot(ctx, r, snapshot) } func (p *generationRouter) RunCommandCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute, command sandbox.Command) (sandbox.CommandResult, error) { if err := providercontract.Require(p, "RunCommandCompute"); err != nil { @@ -120,11 +92,7 @@ func (p *generationRouter) RunCommandCompute(ctx context.Context, r sandbox.Refe return sandbox.CommandResult{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.CommandResult{}, err - } - return cp.RunCommandCompute(ctx, r, c, command) + return v.RunCommandCompute(ctx, r, c, command) } func (p *generationRouter) ResumeCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute) (sandbox.ComputeState, error) { if err := providercontract.Require(p, "ResumeCompute"); err != nil { @@ -135,15 +103,5 @@ func (p *generationRouter) ResumeCompute(ctx context.Context, r sandbox.Referenc return sandbox.ComputeState{}, err } defer done() - cp, err := sandbox.Checkpoint(v) - if err != nil { - return sandbox.ComputeState{}, err - } - return cp.ResumeCompute(ctx, r, c) -} -func (*generationRouter) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "generation_router_does_not_discover_configuration"} -} -func (*generationRouter) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "generation_router_does_not_verify_configuration"} + return v.ResumeCompute(ctx, r, c) } diff --git a/services/core/cmd/server/managed_generations.go b/services/core/cmd/server/managed_generations.go index 3a349f6ed..e509ec494 100644 --- a/services/core/cmd/server/managed_generations.go +++ b/services/core/cmd/server/managed_generations.go @@ -18,9 +18,8 @@ import ( // immutable specification and current credential. There is no mutable provider // map to unload and no current-generation fallback for a missing historical row. type generationRouter struct { - setup *managedSetup - operations providercontract.Operations - providerType string + setup *managedSetup + operations providercontract.Operations } func (p *generationRouter) route(ctx context.Context, r sandbox.Reference) (sandbox.SandboxProvider, func(), error) { @@ -81,18 +80,10 @@ func (p *generationRouter) RunCommand(ctx context.Context, r sandbox.Reference, return v.RunCommand(ctx, r, c) } -type observedGenerationRouter struct{ *generationRouter } - -func (p *observedGenerationRouter) ObservationProviderType() string { return p.providerType } -func (p *observedGenerationRouter) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} - func (p *generationRouter) ProviderOperations() providercontract.Operations { return maps.Clone(p.operations) } - -func (p *observedGenerationRouter) Observe(ctx context.Context, t runtimeobs.Target) (runtimeobs.Sample, error) { +func (p *generationRouter) Observe(ctx context.Context, t runtimeobs.Target) (runtimeobs.Sample, error) { v, done, err := p.route(ctx, sandbox.Reference{TenantID: t.TenantID, EnvironmentID: t.EnvironmentID, AllocationID: t.Instance.AllocationID}) if err != nil { return runtimeobs.Sample{}, err @@ -101,14 +92,7 @@ func (p *observedGenerationRouter) Observe(ctx context.Context, t runtimeobs.Tar if err := providercontract.Require(v, "Observe"); err != nil { return runtimeobs.Sample{}, err } - source, ok := v.(runtimeobs.Source) - if !ok { - return runtimeobs.Sample{}, providercontract.ErrContract - } - if source.ObservationProviderType() != p.providerType { - return runtimeobs.Sample{}, providercontract.ErrContract - } - return source.Observe(ctx, t) + return v.Observe(ctx, t) } // routeGenerations routes a direct provider's allocations through their own @@ -117,11 +101,7 @@ func (s *managedSetup) routeGenerations(candidate execution.PreparedRuntimeDeplo if setup.Mode == "nodes" { return candidate, nil } - router := &generationRouter{setup: s, providerType: candidate.Config.Provider.(runtimeobs.Source).ObservationProviderType(), operations: candidate.Config.Provider.ProviderOperations()} - router.operations["ObserveBatch"] = providercontract.Support{State: providercontract.Unsupported, Reason: "allocations_require_individual_generation_routing"} - router.operations["DiscoverSelection"] = providercontract.Support{State: providercontract.Unsupported, Reason: "generation_router_does_not_discover_configuration"} - router.operations["VerifyCredential"] = providercontract.Support{State: providercontract.Unsupported, Reason: "generation_router_does_not_verify_configuration"} - candidate.Config.Provider = &observedGenerationRouter{router} + candidate.Config.Provider = &generationRouter{setup: s, operations: candidate.Config.Provider.ProviderOperations()} if err := sandbox.ValidateProvider(candidate.Config.Provider); err != nil { return execution.PreparedRuntimeDeployment{}, err } @@ -142,18 +122,7 @@ func (s *managedSetup) routeGenerations(candidate execution.PreparedRuntimeDeplo // Public template readability cannot establish which team owns a deployment. verify := func(value deployment.Setup, refs []sandbox.Reference) error { value.InstallationID = setup.InstallationID - provider, err := s.provider(value) - if err != nil { - return err - } - if err := providercontract.Require(provider, "VerifyCredential"); err != nil { - return err - } - verifier, ok := provider.(sandbox.CredentialVerifier) - if !ok { - return sandbox.ErrInvalid - } - return verifier.VerifyCredential(ctx, refs) + return s.registry.VerifyCredential(ctx, s.direct(value), refs) } current, err := s.deployment.Setup(ctx) if err != nil { @@ -227,9 +196,3 @@ func (s *managedSetup) routeGenerations(candidate execution.PreparedRuntimeDeplo } return candidate, nil } - -// A batch can contain allocations from different endpoint generations. Let the -// observation worker route each allocation through its immutable generation. -func (p *observedGenerationRouter) ObserveBatch(_ context.Context, _ []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "allocations_require_individual_generation_routing"} -} diff --git a/services/core/cmd/server/managed_setup.go b/services/core/cmd/server/managed_setup.go index 8606525d7..e1328ba2b 100644 --- a/services/core/cmd/server/managed_setup.go +++ b/services/core/cmd/server/managed_setup.go @@ -116,20 +116,20 @@ func (s *managedSetup) prepare(ctx context.Context, setup deployment.Setup) (exe if err != nil { return execution.PreparedRuntimeDeployment{}, err } - selection := sandbox.Selection{Provider: setup.Provider, DeploymentSpec: setup.Specification, Configuration: setup.Configuration} - if err := providercontract.Require(candidate.Config.Provider, "DiscoverSelection"); err == nil { - discoverer := candidate.Config.Provider.(sandbox.SelectionDiscoverer) - selection, err = discoverer.DiscoverSelection(ctx, selection) - if err != nil { + adapter, err := s.registry.Lookup(setup.Provider) + if err != nil { + return execution.PreparedRuntimeDeployment{}, err + } + direct := s.direct(setup) + selection := direct.Selection + if adapter.Configuration.Requirements().SelectionDiscovery.State == providercontract.Supported { + if selection, err = s.registry.DiscoverSelection(ctx, direct); err != nil { return execution.PreparedRuntimeDeployment{}, err } setup.Specification, setup.Configuration = selection.DeploymentSpec, selection.Configuration - candidate, err = s.configuration(setup) - if err != nil { + if candidate, err = s.configuration(setup); err != nil { return execution.PreparedRuntimeDeployment{}, err } - } else if !errors.Is(err, providercontract.ErrUnsupported) { - return execution.PreparedRuntimeDeployment{}, err } candidate.Selection = &selection return s.routeGenerations(candidate, setup) @@ -154,23 +154,17 @@ func (s *managedSetup) configuration(setup deployment.Setup) (execution.Prepared return execution.PreparedRuntimeDeployment{Config: selected, Publish: s.publish}, nil } -func (*managedSetup) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}} -} - -func (s *managedSetup) ResolveObservationSource(ctx context.Context) (runtimeobs.Source, error) { +// observationSource returns the selected Provider and its registered kind for +// Runtime observation. +func (s *managedSetup) observationSource(ctx context.Context) (runtimeobs.Source, string, error) { selected, err := s.load(ctx) if err != nil { - return nil, err + return nil, "", err } if selected == nil { - return nil, runtimeobs.ErrUnavailable - } - source, ok := selected.Provider.(runtimeobs.Source) - if !ok { - return nil, providercontract.ErrContract + return nil, "", runtimeobs.ErrUnavailable } - return source, nil + return selected.Provider, selected.ProviderKind, nil } // provider builds the setup's provider. The setup carries the mode and @@ -180,7 +174,12 @@ func (s *managedSetup) provider(setup deployment.Setup) (sandbox.SandboxProvider if s.hub == nil { return nil, errors.New("sandbox node transport is unavailable") } - return s.hub.GenerationProvider(setup.Provider, setup.Operations, s.deployment.AllocationGeneration), nil + return s.hub.GenerationProvider(setup.Operations, s.deployment.AllocationGeneration), nil } - return s.registry.BuildDirect(providers.DirectConfig{ProcessPaths: s.processPaths, InstallationID: setup.InstallationID, Selection: sandbox.Selection{Provider: setup.Provider, DeploymentSpec: setup.Specification, Configuration: setup.Configuration}, Fence: &s.providerCalls}) + return s.registry.BuildDirect(s.direct(setup)) +} + +// direct is the setup's input to direct-mode construction and setup operations. +func (s *managedSetup) direct(setup deployment.Setup) sandbox.DirectConfig { + return sandbox.DirectConfig{ProcessPaths: s.processPaths, InstallationID: setup.InstallationID, Selection: sandbox.Selection{Provider: setup.Provider, DeploymentSpec: setup.Specification, Configuration: setup.Configuration}, Fence: &s.providerCalls} } diff --git a/services/core/cmd/server/managed_setup_test.go b/services/core/cmd/server/managed_setup_test.go index 54e21dc1a..259ed35cb 100644 --- a/services/core/cmd/server/managed_setup_test.go +++ b/services/core/cmd/server/managed_setup_test.go @@ -11,7 +11,6 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/deployment/placement" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/persistence/postgres/deploymentpg" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/processconfig" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/docker" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/e2b" @@ -261,38 +260,21 @@ func testProviderPaths(t *testing.T, helper, state string) sandbox.ProcessPaths func TestManagedObservationSourceKeepsSelectionAcrossReconfiguration(t *testing.T) { value := deployment.Setup{InstallationID: "installation", Generation: 1} setup := &managedSetup{registry: providers.Builtin(), deployment: &fakeDeploymentSetups{t: t, setup: committedSetup(&value)}, installationID: "installation"} - if source, err := setup.ResolveObservationSource(t.Context()); source != nil || !errors.Is(err, runtimeobs.ErrUnavailable) { + if source, kind, err := setup.observationSource(t.Context()); source != nil || kind != "" || !errors.Is(err, runtimeobs.ErrUnavailable) { t.Fatal("unconfigured setup did not return typed unavailability", source, err) } first := &docker.Provider{} value.Provider, value.Mode, value.Generation = "docker", "nodes", 2 - setup.publish(&execution.RuntimeProvider{Generation: 2, Provider: first}) - source, err := setup.ResolveObservationSource(t.Context()) - if err != nil || source != first { - t.Fatal(source, err) + setup.publish(&execution.RuntimeProvider{Generation: 2, ProviderKind: "docker", Provider: first}) + source, kind, err := setup.observationSource(t.Context()) + if err != nil || source != first || kind != "docker" { + t.Fatal(source, kind, err) } next := µsandbox.Provider{} value.Provider, value.Mode, value.Generation = "microsandbox", "nodes", 3 - setup.publish(&execution.RuntimeProvider{Generation: 3, Provider: next}) - if source.ObservationProviderType() != "docker" { - t.Fatal("in-flight identity changed") - } - selected, err := setup.ResolveObservationSource(t.Context()) - if err != nil || selected != next || selected.ObservationProviderType() != "microsandbox" { - t.Fatal(selected, err) - } -} - -func TestObservationGenerationIdentityMustMatchRoutedAllocation(t *testing.T) { - hub := node.NewHub(node.HubOptions{}) - defer hub.Close() - routed := func(context.Context, sandbox.Reference) (deployment.Setup, error) { - return deployment.Setup{Provider: "docker", Mode: "nodes"}, nil - } - setup := &managedSetup{registry: providers.Builtin(), hub: hub, deployment: &fakeDeploymentSetups{t: t, allocationSetup: routed}, allocations: &fakeGenerationAllocations{t: t}} - source := &observedGenerationRouter{&generationRouter{setup: setup, providerType: "e2b"}} - _, err := source.Observe(t.Context(), runtimeobs.Target{TenantID: "tenant", EnvironmentID: "environment", Instance: runtimeobs.Instance{AllocationID: "allocation"}}) - if !errors.Is(err, providercontract.ErrContract) { - t.Fatal("routed allocation was attributed to another provider", err) + setup.publish(&execution.RuntimeProvider{Generation: 3, ProviderKind: "microsandbox", Provider: next}) + selected, kind, err := setup.observationSource(t.Context()) + if err != nil || selected != next || kind != "microsandbox" { + t.Fatal(selected, kind, err) } } diff --git a/services/core/deploy/e2b/helper_contract_generated.py b/services/core/deploy/e2b/helper_contract_generated.py index ebd7fadf2..04d6c13b3 100644 --- a/services/core/deploy/e2b/helper_contract_generated.py +++ b/services/core/deploy/e2b/helper_contract_generated.py @@ -5,7 +5,6 @@ MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","InstallationID"] MAX_COMMAND_INPUT = 52428832 MAX_CREDENTIAL_REFERENCES = 32 -MAX_OBSERVATION_REFERENCES = 100 MAX_OUTPUT = 1048576 MAX_REQUEST = 75497472 MAX_RESPONSE = 16777216 @@ -14,5 +13,5 @@ PROTOCOL_VERSION = 1 REFERENCE_FIELDS = ["TenantID","EnvironmentID","AllocationID"] REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","RuntimeBootstrap","Command","Deadline"] -RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observations"] +RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] SDK_VERSION = "2.51.0" diff --git a/services/core/internal/api/admin_runtime.go b/services/core/internal/api/admin_runtime.go index f6b1df723..c11dae6aa 100644 --- a/services/core/internal/api/admin_runtime.go +++ b/services/core/internal/api/admin_runtime.go @@ -37,7 +37,7 @@ type AdminRuntimeObservationList struct { } // @Summary List Runtime observations across managed Projects -// @Description Core key only. Each observation is labelled with its owning Project ID. Uses the existing read-only Runtime sampler, with bounded concurrency and no execution or provisioning. A provider with a batch metrics read, such as E2B, samples the page's running sandboxes in one bounded request. +// @Description Core key only. Each observation is labelled with its owning Project ID. Uses the existing read-only Runtime sampler, with bounded concurrency and no execution or provisioning. // @Tags Core Administration // @Produce json // @Security DeploymentAdminAuth diff --git a/services/core/internal/execution/archive_cancellation_cleanup_test.go b/services/core/internal/execution/archive_cancellation_cleanup_test.go index 95353a660..f6e658f96 100644 --- a/services/core/internal/execution/archive_cancellation_cleanup_test.go +++ b/services/core/internal/execution/archive_cancellation_cleanup_test.go @@ -49,7 +49,7 @@ func (waitingCleanupCheckpoint) ProviderOperations() providercontract.Operations } type waitingCleanupCheckpoint struct { - sandbox.CheckpointProvider + sandbox.SandboxProvider beforeKill func() } diff --git a/services/core/internal/execution/provider_operations_fixture_test.go b/services/core/internal/execution/provider_operations_fixture_test.go index c8fc32805..98e951c04 100644 --- a/services/core/internal/execution/provider_operations_fixture_test.go +++ b/services/core/internal/execution/provider_operations_fixture_test.go @@ -10,26 +10,21 @@ import ( func (*lifecycleOnlySandbox) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*lifecycleOnlySandbox) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -62,17 +57,3 @@ func (*lifecycleOnlySandbox) ResumeCompute(context.Context, sandbox.Reference, s func (*lifecycleOnlySandbox) Observe(context.Context, runtimeobs.Target) (runtimeobs.Sample, error) { return runtimeobs.Sample{}, &providercontract.UnsupportedError{Operation: "Observe", Reason: "fixture_operation_not_supported"} } -func (*lifecycleOnlySandbox) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_operation_not_supported"} -} -func (*lifecycleOnlySandbox) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "fixture_operation_not_supported"} -} -func (*lifecycleOnlySandbox) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "fixture_operation_not_supported"} -} - -func (*lifecycleOnlySandbox) ObservationProviderType() string { return "fixture" } -func (p *lifecycleOnlySandbox) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/execution/runtime_compute.go b/services/core/internal/execution/runtime_compute.go index b093ab975..c1aeab3c6 100644 --- a/services/core/internal/execution/runtime_compute.go +++ b/services/core/internal/execution/runtime_compute.go @@ -8,6 +8,7 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/internal/agentdaemon/proto" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/deployment" + "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/google/uuid" ) @@ -44,9 +45,9 @@ func (r *runtimeLifecycle) saveCompute(ctx context.Context, owner deployment.All return r.deployment.SetCompute(ctx, owner, phase, raw, until, idleTimeout) } func (r *runtimeLifecycle) enableCompute(ctx context.Context, owner deployment.Allocation) error { - p, capabilityErr := sandbox.Checkpoint(r.config.Provider) - if capabilityErr != nil { - return capabilityErr + p := r.config.Provider + if err := providercontract.Require(p, "Initial"); err != nil { + return err } initial, err := p.Initial(ctx, runtimeReference(owner)) if err != nil { @@ -64,9 +65,9 @@ func (r *runtimeLifecycle) enableCompute(ctx context.Context, owner deployment.A } func (r *runtimeLifecycle) observeCompute(ctx context.Context, owner deployment.Allocation) error { - p, capabilityErr := sandbox.Checkpoint(r.config.Provider) - if capabilityErr != nil { - return capabilityErr + p := r.config.Provider + if err := providercontract.Require(p, "Initial"); err != nil { + return err } var state runtimeCompute if json.Unmarshal(owner.ComputeState, &state) != nil || state.Current.ID == "" { @@ -103,7 +104,7 @@ func (r *runtimeLifecycle) observeCompute(ctx context.Context, owner deployment. } } -func (r *runtimeLifecycle) idleCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute) error { +func (r *runtimeLifecycle) idleCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute) error { compute, err := p.GetCompute(ctx, runtimeReference(owner), state.Current) if err != nil { return err @@ -170,7 +171,7 @@ func (r *runtimeLifecycle) idleCompute(ctx context.Context, p sandbox.Checkpoint return r.captureCompute(ctx, p, suspending, state, false) } -func (r *runtimeLifecycle) captureCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute, observeOnly bool) error { +func (r *runtimeLifecycle) captureCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute, observeOnly bool) error { result, err := p.Suspend(ctx, sandbox.SuspendRequest{Reference: runtimeReference(owner), OperationID: state.SuspendID, Source: state.Current, Snapshot: state.Snapshot, ObserveOnly: observeOnly}) if err != nil { return err @@ -203,7 +204,7 @@ func (r *runtimeLifecycle) captureCompute(ctx context.Context, p sandbox.Checkpo return err } -func (r *runtimeLifecycle) restoreIdleCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute) error { +func (r *runtimeLifecycle) restoreIdleCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute) error { activity, err := r.reader.Activity(ctx, owner.ID) if err != nil { return err @@ -225,7 +226,7 @@ func (r *runtimeLifecycle) restoreIdleCompute(ctx context.Context, p sandbox.Che } return r.restoreCompute(ctx, p, next, state, false) } -func (r *runtimeLifecycle) restoreCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute, observeOnly bool) error { +func (r *runtimeLifecycle) restoreCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute, observeOnly bool) error { if state.Target == nil || state.Snapshot == nil || state.Rollback { return sandbox.ErrOwnership } diff --git a/services/core/internal/execution/runtime_compute_wake.go b/services/core/internal/execution/runtime_compute_wake.go index 552ede1ed..ed4223d90 100644 --- a/services/core/internal/execution/runtime_compute_wake.go +++ b/services/core/internal/execution/runtime_compute_wake.go @@ -12,7 +12,7 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sessions" ) -func (r *runtimeLifecycle) wakeCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute) error { +func (r *runtimeLifecycle) wakeCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute) error { if state.Rollback { if _, err := p.ResumeCompute(ctx, runtimeReference(owner), state.Current); err != nil { return err @@ -70,7 +70,7 @@ func (r *runtimeLifecycle) wakeCompute(ctx context.Context, p sandbox.Checkpoint return r.observeConnection(ctx, next) } -func (r *runtimeLifecycle) cleanupCompute(ctx context.Context, p sandbox.CheckpointProvider, owner deployment.Allocation, state runtimeCompute) error { +func (r *runtimeLifecycle) cleanupCompute(ctx context.Context, p sandbox.SandboxProvider, owner deployment.Allocation, state runtimeCompute) error { if err := r.lease.CheckOwnership(ctx); err != nil { return err } diff --git a/services/core/internal/execution/sandbox_deployment_drain_test.go b/services/core/internal/execution/sandbox_deployment_drain_test.go index d80fbefa0..4ace349be 100644 --- a/services/core/internal/execution/sandbox_deployment_drain_test.go +++ b/services/core/internal/execution/sandbox_deployment_drain_test.go @@ -92,7 +92,7 @@ func testLifecycleCancellationPreservesLease(t *testing.T, mode string) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return configuration, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -230,7 +230,7 @@ func TestSandboxDeploymentDrainFailureCannotReactivate(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return configuration, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) diff --git a/services/core/internal/execution/sandbox_deployment_setup_test.go b/services/core/internal/execution/sandbox_deployment_setup_test.go index 549f233d3..0337c3199 100644 --- a/services/core/internal/execution/sandbox_deployment_setup_test.go +++ b/services/core/internal/execution/sandbox_deployment_setup_test.go @@ -24,7 +24,7 @@ func TestDeferredSandboxDeploymentLoadsOnceBeforeNodeCreation(t *testing.T) { id := uuid.NewString() var selected atomic.Bool var loads atomic.Int32 - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { loads.Add(1) if !selected.Load() { @@ -94,7 +94,7 @@ func TestDeferredSandboxProviderFailureKeepsRecoveryAvailable(t *testing.T) { id := uuid.NewString() available := false loadErr := ErrExecutionUnavailable - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { if !available { return nil, loadErr @@ -125,7 +125,7 @@ func TestRejectedSandboxCandidatePreservesActiveGeneration(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} rejected := errors.New("candidate provider unavailable") m, err := newRuntimeManager(Owner{Lease: heldLease{}}, unitDeploymentService(t), nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, func(context.Context, deployment.Setup) (PreparedRuntimeDeployment, error) { @@ -199,7 +199,7 @@ func TestCommittedSandboxCandidatePublishesAfterShutdown(t *testing.T) { if err := m.pauseDeployment(t.Context()); err != nil { t.Fatal(err) } - config := &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} var published *RuntimeProvider candidate := PreparedRuntimeDeployment{Config: config, Publish: func(value *RuntimeProvider) { published = value }} m.stop() diff --git a/services/core/internal/execution/sandbox_deployment_switch_test.go b/services/core/internal/execution/sandbox_deployment_switch_test.go index 551d1ec78..6f9a1d14d 100644 --- a/services/core/internal/execution/sandbox_deployment_switch_test.go +++ b/services/core/internal/execution/sandbox_deployment_switch_test.go @@ -18,7 +18,7 @@ func TestSandboxManagerSwitchDrainsBeforeDirectActivation(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -62,7 +62,7 @@ func TestSandboxManagerSwitchDrainsBeforeDirectActivation(t *testing.T) { case <-time.After(time.Second): t.Fatal("switch drain blocked") } - config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} if err := m.activateDeployment(t.Context(), deployment.View{InstallationID: id, Generation: 2, Mode: "direct", Provider: "e2b"}); err != nil { t.Fatal(err) } @@ -100,7 +100,7 @@ func TestSandboxManagerCancelledSwitchCannotResumeBeforeDrain(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -129,7 +129,7 @@ func TestSandboxManagerCancelledSwitchCannotResumeBeforeDrain(t *testing.T) { m.mu.Lock() originalDrain := m.switchDrained m.mu.Unlock() - config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)} + config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} expected := deployment.View{InstallationID: id, Generation: 2, Mode: "direct", Provider: "e2b"} ctx, cancel = context.WithTimeout(t.Context(), 10*time.Millisecond) if err := m.activateDeployment(ctx, expected); !errors.Is(err, context.DeadlineExceeded) { @@ -158,7 +158,7 @@ func TestSandboxActivationCannotBypassOutstandingDrain(t *testing.T) { m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return nil, nil }, func(_ context.Context, setup deployment.Setup) (PreparedRuntimeDeployment, error) { - return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)}}, nil + return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}}, nil })) if err != nil { t.Fatal(err) diff --git a/services/core/internal/execution/sandbox_generations_test.go b/services/core/internal/execution/sandbox_generations_test.go index 32fc48961..733d28f82 100644 --- a/services/core/internal/execution/sandbox_generations_test.go +++ b/services/core/internal/execution/sandbox_generations_test.go @@ -29,7 +29,7 @@ func TestE2BReplacementVerifiesTwiceAndNeverPublishesFailedCommit(t *testing.T) } hub := node.NewHub(node.HubOptions{}) defer hub.Close() - provider := hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1) + provider := hub.Proxy(uuid.NewString(), docker.Operations(), 1) verifyCalls, published, fenced, released := 0, 0, 0, 0 var rejectAt int var rejection error = sandbox.ErrCredentialOwnership diff --git a/services/core/internal/execution/sandbox_reset_test.go b/services/core/internal/execution/sandbox_reset_test.go index 16bd12cc9..439b16dcb 100644 --- a/services/core/internal/execution/sandbox_reset_test.go +++ b/services/core/internal/execution/sandbox_reset_test.go @@ -100,7 +100,7 @@ func TestSandboxResetPageTimeoutRecoversCommittedOwner(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), config) if err != nil { @@ -214,7 +214,7 @@ func TestSandboxResetPublishesCommittedGenerationWithoutReading(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) var published []uint64 config.PublishUnconfigured = func(generation uint64) { diff --git a/services/core/internal/execution/sandbox_snapshot_budget_test.go b/services/core/internal/execution/sandbox_snapshot_budget_test.go index 60a0e1067..eba65917a 100644 --- a/services/core/internal/execution/sandbox_snapshot_budget_test.go +++ b/services/core/internal/execution/sandbox_snapshot_budget_test.go @@ -72,7 +72,7 @@ func TestSandboxResetSnapshotFitsPageBudget(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), "docker", docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), configuration) if err != nil { diff --git a/services/core/internal/providercontract/operations.go b/services/core/internal/providercontract/operations.go index ab1aa7a32..cb24ac2f5 100644 --- a/services/core/internal/providercontract/operations.go +++ b/services/core/internal/providercontract/operations.go @@ -5,7 +5,6 @@ package providercontract import ( "errors" "fmt" - "reflect" "regexp" ) @@ -64,32 +63,3 @@ func Require(p Declared, operation string) error { } return p.ProviderOperations()[operation].Check(operation) } - -// Validate checks the existing small interfaces, including methods declared -// unsupported. Every adapter must explicitly implement those rejections. -// A new method cannot be silently covered by an old declaration or stub. -func Validate(p Declared, contracts ...reflect.Type) error { - if p == nil { - return ErrContract - } - value := reflect.ValueOf(p) - if value.Kind() == reflect.Pointer && value.IsNil() { - return ErrContract - } - operations := p.ProviderOperations() - for _, contract := range contracts { - if !value.Type().Implements(contract) { - return fmt.Errorf("%w: missing %s implementation", ErrContract, contract.Name()) - } - for i := 0; i < contract.NumMethod(); i++ { - name := contract.Method(i).Name - if name == "ProviderOperations" { - continue - } - if err := operations[name].Check(name); err != nil && !errors.Is(err, ErrUnsupported) { - return err - } - } - } - return nil -} diff --git a/services/core/internal/runtimeobs/exporter_independence_test.go b/services/core/internal/runtimeobs/exporter_independence_test.go index 2c3b92078..3a24a74ec 100644 --- a/services/core/internal/runtimeobs/exporter_independence_test.go +++ b/services/core/internal/runtimeobs/exporter_independence_test.go @@ -10,7 +10,7 @@ func TestExporterOutageDoesNotDelayOtherDestinations(t *testing.T) { target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} blocked := &gatedExporter{started: make(chan struct{}, 1), release: make(chan struct{})} records := make(chan ExportRecord, 1) - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now}}}, + service, err := NewService(fixedResolver{target: target}, sourceOf(&fixedSource{sample: Sample{ObservedAt: now}}), WithExporter(blocked, ExportOptions{QueueCapacity: 1, Timeout: time.Second}), WithExporter(channelExporter{records: records}, ExportOptions{QueueCapacity: 1, Timeout: time.Second})) if err != nil { diff --git a/services/core/internal/runtimeobs/operations_fixture_test.go b/services/core/internal/runtimeobs/operations_fixture_test.go index d55ebe8c4..8364c7ad0 100644 --- a/services/core/internal/runtimeobs/operations_fixture_test.go +++ b/services/core/internal/runtimeobs/operations_fixture_test.go @@ -2,46 +2,18 @@ package runtimeobs import ( "context" + "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" ) -func (*fixedSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Supported}, "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_has_no_batch_observation"}} -} -func (*fixedSource) ObserveBatch(context.Context, []Target) ([]BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_has_no_batch_observation"} -} -func (blockingSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Supported}, "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_has_no_batch_observation"}} -} -func (blockingSource) ObserveBatch(context.Context, []Target) ([]BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_has_no_batch_observation"} -} -func (*countingSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Supported}, "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_has_no_batch_observation"}} -} -func (*countingSource) ObserveBatch(context.Context, []Target) ([]BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_has_no_batch_observation"} -} -func (*batchSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Supported}, "ObserveBatch": {State: providercontract.Supported}} +func observedOperations() providercontract.Operations { + return providercontract.Operations{"Observe": {State: providercontract.Supported}} } +func (*fixedSource) ProviderOperations() providercontract.Operations { return observedOperations() } +func (blockingSource) ProviderOperations() providercontract.Operations { return observedOperations() } +func (*countingSource) ProviderOperations() providercontract.Operations { return observedOperations() } -func (s *fixedSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } -func (*fixedSource) ObservationProviderType() string { return "fixture" } - -func (s blockingSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } - -func (s *countingSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } -func (*countingSource) ObservationProviderType() string { return "fixture" } - -func (s *batchSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } -func (*batchSource) ObservationProviderType() string { return "fixture" } - -func (s typedSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } - -func (s *failingBatchSource) ResolveObservationSource(context.Context) (Source, error) { return s, nil } - -func (s *unsupportedObservation) ResolveObservationSource(context.Context) (Source, error) { - return s, nil +// sourceOf selects one fixed docker source for every page. +func sourceOf(source Source) func(context.Context) (Source, string, error) { + return func(context.Context) (Source, string, error) { return source, "docker", nil } } diff --git a/services/core/internal/runtimeobs/operations_test.go b/services/core/internal/runtimeobs/operations_test.go index 3e66de33c..fc9e242d0 100644 --- a/services/core/internal/runtimeobs/operations_test.go +++ b/services/core/internal/runtimeobs/operations_test.go @@ -1,69 +1,20 @@ package runtimeobs import ( - "context" - "errors" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "testing" - "time" -) -type failingBatchSource struct { - *fixedSource - batchErr error - batches int -} - -func (*failingBatchSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Supported}, "ObserveBatch": {State: providercontract.Supported}} -} -func (s *failingBatchSource) ObserveBatch(context.Context, []Target) ([]BatchResult, error) { - s.batches++ - return nil, s.batchErr -} -func TestBatchFallbackRequiresExplicitSafeUnsupported(t *testing.T) { - for _, test := range []struct { - name string - err error - fallback bool - }{ - {"unsupported", &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "native_batch_not_supported"}, true}, - {"unavailable", ErrUnavailable, false}, - {"timeout", context.DeadlineExceeded, false}, - {"failure", errors.New("provider failed"), false}, - {"bare unsupported", providercontract.ErrUnsupported, false}, - {"unsafe unsupported", &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "private endpoint / key"}, false}, - {"wrong operation", &providercontract.UnsupportedError{Operation: "Observe", Reason: "not_supported"}, false}, - } { - t.Run(test.name, func(t *testing.T) { - now := time.Now() - ratio := 0.5 - source := &failingBatchSource{fixedSource: &fixedSource{sample: Sample{ObservedAt: now, CPUUtilizationRatio: &ratio}}, batchErr: test.err} - target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) - if err != nil { - t.Fatal(err) - } - observations, errs := service.ObserveSessions(t.Context(), []SessionIdentity{{TenantID: "tenant", SessionID: "session"}}, PageOptions{}) - if source.batches != 1 || (source.calls == 1) != test.fallback { - t.Fatalf("batch=%d single=%d", source.batches, source.calls) - } - if test.fallback && (errs[0] != nil || observations[0].Status != StatusObserved) { - t.Fatal(observations, errs) - } - }) - } -} + "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" +) type unsupportedObservation struct{ *fixedSource } func (*unsupportedObservation) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": {State: providercontract.Supported}, "ObservationProviderType": {State: providercontract.Supported}, "Observe": {State: providercontract.Unsupported, Reason: "native_metrics_not_supported"}, "ObserveBatch": {State: providercontract.Unsupported, Reason: "native_metrics_not_supported"}} + return providercontract.Operations{"Observe": {State: providercontract.Unsupported, Reason: "native_metrics_not_supported"}} } func TestUnsupportedObservationIsNotUnavailable(t *testing.T) { source := &unsupportedObservation{&fixedSource{}} target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err := NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } diff --git a/services/core/internal/runtimeobs/sampler.go b/services/core/internal/runtimeobs/sampler.go index b55e205db..a89384344 100644 --- a/services/core/internal/runtimeobs/sampler.go +++ b/services/core/internal/runtimeobs/sampler.go @@ -27,8 +27,7 @@ type SessionLister interface { ListRuntimeObservationSessions(context.Context, string, int) (SessionPage, error) } -// HistoryObserver reads one listed page, so providers with a batch read can -// sample the whole page in one bounded request. +// HistoryObserver reads one listed page with bounded concurrency. type HistoryObserver interface { ObserveSessionsForHistory(context.Context, []SessionIdentity, OwnershipChecker, PageOptions) ([]Observation, []error) } diff --git a/services/core/internal/runtimeobs/sampler_test.go b/services/core/internal/runtimeobs/sampler_test.go index 2f1e119e6..da14b77b4 100644 --- a/services/core/internal/runtimeobs/sampler_test.go +++ b/services/core/internal/runtimeobs/sampler_test.go @@ -146,7 +146,7 @@ func TestSamplerSweepsEveryPageAndIsolatesSessionFailures(t *testing.T) { func TestSamplerBoundsConcurrencyAndSourceDeadline(t *testing.T) { source := &countingSource{} target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err := NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } @@ -246,7 +246,7 @@ func TestSamplerPreservesProviderTimeoutAndFinalFenceAfterSlowResolution(t *test records := make(chan ExportRecord, 1) service, err := NewService( resolver, - map[string]SourceResolver{"provider": blockingSource{}}, + sourceOf(blockingSource{}), WithExporter(channelExporter{records: records}, ExportOptions{}), ) if err != nil { diff --git a/services/core/internal/runtimeobs/service.go b/services/core/internal/runtimeobs/service.go index 101c6fe9f..c38db7359 100644 --- a/services/core/internal/runtimeobs/service.go +++ b/services/core/internal/runtimeobs/service.go @@ -4,7 +4,6 @@ import ( "context" "errors" "fmt" - "reflect" "sync" "time" @@ -30,12 +29,16 @@ const ( type Service struct { resolver TargetResolver - sources map[string]SourceResolver + source func(context.Context) (Source, string, error) now func() time.Time exports []*exportDispatcher } -func NewService(resolver TargetResolver, sources map[string]SourceResolver, options ...ServiceOption) (*Service, error) { +// NewService reads managed Runtimes through source, which returns the Sandbox +// Provider of the deployment's current selection and its registered kind, or +// ErrUnavailable while none is selected. A nil source means this Core has no +// managed deployment. +func NewService(resolver TargetResolver, source func(context.Context) (Source, string, error), options ...ServiceOption) (*Service, error) { if resolver == nil { return nil, errors.New("Runtime observation resolver is required") } @@ -48,20 +51,7 @@ func NewService(resolver TargetResolver, sources map[string]SourceResolver, opti return nil, err } } - copySources := make(map[string]SourceResolver, len(sources)) - for key, source := range sources { - if key == "" || source == nil { - return nil, errors.New("invalid Runtime observation source") - } - if err := providercontract.Validate(source, reflect.TypeFor[SourceResolver]()); err != nil { - return nil, err - } - if err := providercontract.Require(source, "ResolveObservationSource"); err != nil { - return nil, fmt.Errorf("%w: observation source resolution must be supported", providercontract.ErrContract) - } - copySources[key] = source - } - service := &Service{resolver: resolver, sources: copySources, now: time.Now} + service := &Service{resolver: resolver, source: source, now: time.Now} for _, export := range config.exporters { service.exports = append(service.exports, newExportDispatcher(export.exporter, export.exportOptions)) } @@ -109,14 +99,12 @@ func (s *Service) ObserveSessionForHistory(ctx context.Context, tenantID, sessio type PageOptions struct { // Concurrency bounds identity resolution and per-target provider reads. Concurrency int - // SourceTimeout bounds each provider read, including one batch read. + // SourceTimeout bounds each provider read. SourceTimeout time.Duration } -// ObserveSessions observes one page of Sessions. Running targets of a source -// that explicitly supports BatchSource share one provider read per MaxBatchTargets; -// other sources are read per target, exactly as ObserveSession reads them. -// Results and errors are aligned with sessions. +// ObserveSessions observes one page of Sessions, reading each running target +// exactly as ObserveSession reads it. Results and errors are aligned with sessions. func (s *Service) ObserveSessions(ctx context.Context, sessions []SessionIdentity, options PageOptions) ([]Observation, []error) { return s.observeSessions(ctx, sessions, CollectionSourceOnRead, nil, options) } @@ -141,8 +129,6 @@ func (s *Service) observeSession(ctx context.Context, tenantID, sessionID string // sourceRead is a resolved running managed target awaiting its provider sample. type sourceRead struct { index int - key string - source Source target Target providerType string } @@ -162,117 +148,39 @@ func (s *Service) observeSessions(ctx context.Context, sessions []SessionIdentit } observations[index], errs[index] = observation, err }) - // A provider key selects one source; keep page order within each group. - var keys []string - groups := map[string][]*sourceRead{} + var pending []*sourceRead for _, read := range reads { - if read == nil { - continue - } - if _, ok := groups[read.key]; !ok { - keys = append(keys, read.key) - } - groups[read.key] = append(groups[read.key], read) - } - for _, key := range keys { - group := groups[key] - sourceCtx, stop := sourceContext(ctx, options.SourceTimeout) - source, err := s.sources[key].ResolveObservationSource(sourceCtx) - stop() - if err == nil { - err = ValidateSource(source) - } - if err != nil { - for _, read := range group { - observations[read.index], errs[read.index] = s.complete(ctx, read, Sample{}, err, 0, collectionSource, owner) - } - continue - } - providerType := source.ObservationProviderType() - for _, read := range group { - read.source, read.providerType = source, providerType - } - for start := 0; start < len(group); start += MaxBatchTargets { - chunk := group[start:min(start+MaxBatchTargets, len(group))] - if s.readBatch(ctx, chunk, observations, errs, collectionSource, owner, options.SourceTimeout) { - continue - } - parallel(len(chunk), options.Concurrency, func(index int) { - read := chunk[index] - sourceCtx, stop := sourceContext(ctx, options.SourceTimeout) - started := time.Now() - var sample Sample - err := providercontract.Require(read.source, "Observe") - if err == nil { - sample, err = read.source.Observe(sourceCtx, read.target) - } - stop() - observations[read.index], errs[read.index] = s.complete(ctx, read, sample, err, time.Since(started), collectionSource, owner) - }) - } - } - return observations, errs -} - -// readBatch reports false, without results, when the source has no batch read -// for its current provider. -func (s *Service) readBatch(ctx context.Context, chunk []*sourceRead, observations []Observation, errs []error, collectionSource CollectionSource, owner OwnershipChecker, sourceTimeout time.Duration) bool { - batch, ok := chunk[0].source.(BatchSource) - if !ok { // NewService rejects this; never treat malformed registration as unsupported. - for _, read := range chunk { - errs[read.index] = providercontract.ErrContract - } - return true - } - if err := providercontract.Require(chunk[0].source, "ObserveBatch"); err != nil { - if errors.Is(err, providercontract.ErrUnsupported) { - return false - } - for _, read := range chunk { - errs[read.index] = err + if read != nil { + pending = append(pending, read) } - return true } - targets := make([]Target, len(chunk)) - for index, read := range chunk { - targets[index] = read.target + if len(pending) == 0 { + return observations, errs } - // One batch read replaces up to MaxBatchTargets single reads, so it may take - // longer than one of them without exceeding the page's overall cost. - if sourceTimeout > 0 { - sourceTimeout = max(sourceTimeout, minBatchSourceTimeout) - } - sourceCtx, stop := sourceContext(ctx, sourceTimeout) - started := time.Now() - results, batchErr := batch.ObserveBatch(sourceCtx, targets) + // One source serves the whole page. + sourceCtx, stop := sourceContext(ctx, options.SourceTimeout) + source, providerType, err := s.source(sourceCtx) stop() - if _, unsupported := providercontract.UnsupportedReason(batchErr, "ObserveBatch"); unsupported { - return false - } - if errors.Is(batchErr, providercontract.ErrUnsupported) { - batchErr = providercontract.ErrContract - } - if batchErr != nil { - for _, read := range chunk { - observations[read.index], errs[read.index] = s.complete(ctx, read, Sample{}, batchErr, 0, collectionSource, owner) + if err != nil { + for _, read := range pending { + observations[read.index], errs[read.index] = s.complete(ctx, read, Sample{}, err, 0, collectionSource, owner) } - return true + return observations, errs } - duration := time.Since(started) - for index, read := range chunk { - if len(results) != len(chunk) { - errs[read.index] = errors.New("Runtime observation batch returned mismatched results") - continue - } - // The rows share one provider read; only the first carries its duration, - // so sample-duration telemetry counts each read once. - rowDuration := time.Duration(0) - if index == 0 { - rowDuration = duration + parallel(len(pending), options.Concurrency, func(index int) { + read := pending[index] + read.providerType = providerType + sourceCtx, stop := sourceContext(ctx, options.SourceTimeout) + started := time.Now() + var sample Sample + err := providercontract.Require(source, "Observe") + if err == nil { + sample, err = source.Observe(sourceCtx, read.target) } - observations[read.index], errs[read.index] = s.complete(ctx, read, results[index].Sample, results[index].Err, rowDuration, collectionSource, owner) - } - return true + stop() + observations[read.index], errs[read.index] = s.complete(ctx, read, sample, err, time.Since(started), collectionSource, owner) + }) + return observations, errs } // resolve returns either a finished observation or a pending provider read. @@ -317,11 +225,10 @@ func (s *Service) resolve(ctx context.Context, tenantID, sessionID string, owner default: return Observation{}, nil, errors.New("invalid managed Runtime allocation state") } - _, ok := s.sources[target.Instance.ProviderKey] - if !ok { + if s.source == nil { return Observation{Target: target, Status: StatusUnavailable, Reason: "source_not_configured", ResolvedAt: resolvedAt}, nil, nil } - return Observation{}, &sourceRead{key: target.Instance.ProviderKey, target: target}, nil + return Observation{}, &sourceRead{target: target}, nil } // complete classifies one provider result and hands it to history export. @@ -352,8 +259,6 @@ func (s *Service) complete(ctx context.Context, read *sourceRead, sample Sample, return s.finish(ctx, observation, collectionSource, owner) } -const minBatchSourceTimeout = 5 * time.Second - func sourceContext(ctx context.Context, timeout time.Duration) (context.Context, context.CancelFunc) { if timeout > 0 { return context.WithTimeout(ctx, timeout) diff --git a/services/core/internal/runtimeobs/service_test.go b/services/core/internal/runtimeobs/service_test.go index ca3d9dbc2..763b4363d 100644 --- a/services/core/internal/runtimeobs/service_test.go +++ b/services/core/internal/runtimeobs/service_test.go @@ -4,7 +4,6 @@ import ( "context" "encoding/json" "errors" - "fmt" "math" "sync" "testing" @@ -38,13 +37,6 @@ func (s *fixedSource) Observe(context.Context, Target) (Sample, error) { return s.sample, s.err } -type typedSource struct { - *fixedSource - providerType string -} - -func (s typedSource) ObservationProviderType() string { return s.providerType } - type blockingSource struct{} func (blockingSource) Observe(ctx context.Context, _ Target) (Sample, error) { @@ -52,8 +44,6 @@ func (blockingSource) Observe(ctx context.Context, _ Target) (Sample, error) { return Sample{}, ctx.Err() } -func (blockingSource) ObservationProviderType() string { return "docker" } - type channelExporter struct { records chan ExportRecord err error @@ -125,7 +115,7 @@ func TestServiceDoesNotCallSourcesForUnsupportedModes(t *testing.T) { if mode == ModeSelfHosted { target.EnvironmentID = "environment" } - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err := NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } @@ -151,7 +141,7 @@ func TestServicePreservesUnavailableAndObservedZero(t *testing.T) { zeroMemory := uint64(0) now := time.Date(2026, 9, 22, 1, 0, 0, 0, time.UTC) source := &fixedSource{sample: Sample{ObservedAt: now, CPUUsageSecondsTotal: &zeroCPU, MemoryUsageBytes: &zeroMemory}} - service, err = NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err = NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } @@ -176,13 +166,13 @@ func TestServiceExportsOnlySanitizedValidatedRecords(t *testing.T) { ProviderState: json.RawMessage(`{"native_id":"must-not-export"}`), }, } - source := typedSource{fixedSource: &fixedSource{sample: Sample{ + source := &fixedSource{sample: Sample{ ObservedAt: now, StartedAt: &startedAt, CPUUsageSecondsTotal: &cpuSeconds, CPUCapacityCores: &cpuCapacity, MemoryUsageBytes: &memoryUsage, MemoryLimitBytes: &memoryLimit, - }}, providerType: "docker"} + }} records := make(chan ExportRecord, 1) - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}, WithExporter(channelExporter{records: records}, ExportOptions{QueueCapacity: 1, Timeout: time.Second})) + service, err := NewService(fixedResolver{target: target}, sourceOf(source), WithExporter(channelExporter{records: records}, ExportOptions{QueueCapacity: 1, Timeout: time.Second})) if err != nil { t.Fatal(err) } @@ -224,7 +214,7 @@ func TestServiceMarksPeriodicHistoryCollection(t *testing.T) { records := make(chan ExportRecord, 1) service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now, StartedAt: &startedAt}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now, StartedAt: &startedAt}}), WithExporter(channelExporter{records: records}, ExportOptions{}), ) if err != nil { @@ -257,7 +247,7 @@ func TestServiceDoesNotExportPeriodicSampleAfterOwnershipLoss(t *testing.T) { records := make(chan ExportRecord, 1) service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now, StartedAt: &startedAt}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now, StartedAt: &startedAt}}), WithExporter(channelExporter{records: records}, ExportOptions{}), ) if err != nil { @@ -278,40 +268,13 @@ func TestServiceDoesNotExportPeriodicSampleAfterOwnershipLoss(t *testing.T) { } } -func TestServiceRejectsUnsafeProviderTypeBeforeSamplingOrExport(t *testing.T) { - now := time.Date(2026, 9, 22, 1, 0, 0, 0, time.UTC) - target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} - source := typedSource{fixedSource: &fixedSource{sample: Sample{ObservedAt: now}}, providerType: "docker native_id=secret"} - records := make(chan ExportRecord, 1) - service, err := NewService( - fixedResolver{target: target}, - map[string]SourceResolver{"provider": source}, - WithExporter(channelExporter{records: records}, ExportOptions{}), - ) - if err != nil { - t.Fatal(err) - } - service.now = func() time.Time { return now } - if _, err := service.ObserveSession(t.Context(), "tenant", "session"); err == nil || source.calls != 0 { - t.Fatalf("unsafe provider type reached sampling: err=%v calls=%d", err, source.calls) - } - if err := service.Close(t.Context()); err != nil { - t.Fatal(err) - } - select { - case record := <-records: - t.Fatalf("unsafe provider type reached exporter: %+v", record) - default: - } -} - func TestServiceExportQueueNeverBlocksOrChangesObservation(t *testing.T) { now := time.Date(2026, 9, 22, 1, 0, 0, 0, time.UTC) target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} exporter := &gatedExporter{started: make(chan struct{}, 1), release: make(chan struct{})} service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now}}), WithExporter(exporter, ExportOptions{QueueCapacity: 1, Timeout: time.Second}), ) if err != nil { @@ -370,7 +333,7 @@ func TestServiceIgnoresExporterFailureAndValidatesOptions(t *testing.T) { target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now}}), WithExporter(channelExporter{records: records, err: errors.New("backend unavailable")}, ExportOptions{}), ) if err != nil { @@ -392,7 +355,7 @@ func TestServiceCloseHonorsItsDeadlineWhenExporterDoesNot(t *testing.T) { target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now}}), WithExporter(exporter, ExportOptions{QueueCapacity: 1, Timeout: time.Millisecond}), ) if err != nil { @@ -422,7 +385,7 @@ func TestServiceIsolatesExporterPanics(t *testing.T) { target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": &fixedSource{sample: Sample{ObservedAt: now}}}, + sourceOf(&fixedSource{sample: Sample{ObservedAt: now}}), WithExporter(exporter, ExportOptions{}), ) if err != nil { @@ -455,9 +418,7 @@ func TestServiceMapsOnlyDeclaredUnavailability(t *testing.T) { {err: context.DeadlineExceeded, wantReason: "sample_timeout"}, {err: errors.New("Docker permission denied"), wantError: true}, } { - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{ - "provider": typedSource{fixedSource: &fixedSource{err: tc.err}, providerType: "docker"}, - }) + service, err := NewService(fixedResolver{target: target}, sourceOf(&fixedSource{err: tc.err})) if err != nil { t.Fatal(err) } @@ -476,7 +437,7 @@ func TestServiceMapsAnActualSourceDeadlineWithoutLeakingIt(t *testing.T) { EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}, } - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": blockingSource{}}) + service, err := NewService(fixedResolver{target: target}, sourceOf(blockingSource{})) if err != nil { t.Fatal(err) } @@ -496,7 +457,7 @@ func TestServiceExportsPeriodicSourceTimeoutAfterFinalOwnershipFence(t *testing. records := make(chan ExportRecord, 1) service, err := NewService( fixedResolver{target: target}, - map[string]SourceResolver{"provider": blockingSource{}}, + sourceOf(blockingSource{}), WithExporter(channelExporter{records: records}, ExportOptions{}), ) if err != nil { @@ -537,7 +498,7 @@ func TestServiceClassifiesResolverAndTerminalAllocationUnavailability(t *testing source := &fixedSource{} target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "creating"}} - service, err = NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err = NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } @@ -547,7 +508,7 @@ func TestServiceClassifiesResolverAndTerminalAllocationUnavailability(t *testing } target = Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "released"}} - service, err = NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": &fixedSource{}}) + service, err = NewService(fixedResolver{target: target}, sourceOf(&fixedSource{})) if err != nil { t.Fatal(err) } @@ -560,7 +521,7 @@ func TestServiceClassifiesResolverAndTerminalAllocationUnavailability(t *testing target = Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{ AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running", AllocationCreatedAt: time.Now().Add(time.Hour), }} - service, err = NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) + service, err = NewService(fixedResolver{target: target}, sourceOf(source)) if err != nil { t.Fatal(err) } @@ -584,7 +545,7 @@ func TestServiceRejectsUnsafeProviderSamples(t *testing.T) { {ObservedAt: now, MemoryUsageBytes: &tooLarge}, {ObservedAt: now, MemoryLimitBytes: &tooLarge}, } { - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": &fixedSource{sample: sample}}) + service, err := NewService(fixedResolver{target: target}, sourceOf(&fixedSource{sample: sample})) if err != nil { t.Fatal(err) } @@ -614,67 +575,6 @@ func TestServiceRejectsMismatchedResolvedOwnership(t *testing.T) { func float64Pointer(value float64) *float64 { return &value } -type batchSource struct { - mu sync.Mutex - batches []int - sample Sample -} - -func (s *batchSource) Observe(context.Context, Target) (Sample, error) { - return Sample{}, errors.New("per-target read used for a batch source") -} - -func (s *batchSource) ObserveBatch(ctx context.Context, targets []Target) ([]BatchResult, error) { - time.Sleep(time.Millisecond) - s.mu.Lock() - s.batches = append(s.batches, len(targets)) - s.mu.Unlock() - results := make([]BatchResult, len(targets)) - for index, target := range targets { - results[index].Sample = s.sample - if target.SessionID == "stopped" { - results[index].Err = ErrNotRunning - } - } - return results, nil -} - -func TestServiceBatchesPageReadsWithinProviderLimit(t *testing.T) { - now := time.Date(2026, 9, 25, 1, 0, 0, 0, time.UTC) - ratio := 0.25 - source := &batchSource{sample: Sample{ObservedAt: now, CPUUtilizationRatio: &ratio}} - target := Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}} - service, err := NewService(fixedResolver{target: target}, map[string]SourceResolver{"provider": source}) - if err != nil { - t.Fatal(err) - } - service.now = func() time.Time { return now } - sessions := make([]SessionIdentity, 150) - for index := range sessions { - sessions[index] = SessionIdentity{TenantID: "tenant", SessionID: fmt.Sprint(index)} - } - sessions[120].SessionID = "stopped" - observations, errs := service.ObserveSessions(t.Context(), sessions, PageOptions{Concurrency: 8, SourceTimeout: time.Second}) - if len(source.batches) != 2 || source.batches[0] != MaxBatchTargets || source.batches[1] != 50 { - t.Fatalf("page was not read in provider-sized batches: %v", source.batches) - } - for index, observation := range observations { - if errs[index] != nil || observation.Target.SessionID != sessions[index].SessionID { - t.Fatalf("batch result %d was not aligned: %+v %v", index, observation, errs[index]) - } - } - if observations[120].Status != StatusUnavailable || observations[120].Reason != "runtime_not_running" || - observations[0].Status != StatusObserved || *observations[0].Sample.CPUUtilizationRatio != .25 { - t.Fatalf("batch results were not classified: %+v %+v", observations[0], observations[120]) - } - // Each provider read reports its duration once, on the first row of its batch. - for index, observation := range observations { - if (observation.SourceDuration > 0) != (index == 0 || index == MaxBatchTargets) { - t.Fatalf("row %d source duration = %v", index, observation.SourceDuration) - } - } -} - func TestSampleWithoutNewerFieldsKeepsItsNodeWireForm(t *testing.T) { observedAt := time.Date(2026, 9, 25, 1, 0, 0, 0, time.UTC) cpu, memory := 1.5, uint64(1024) diff --git a/services/core/internal/runtimeobs/source.go b/services/core/internal/runtimeobs/source.go index 8886a42b1..d29e446bd 100644 --- a/services/core/internal/runtimeobs/source.go +++ b/services/core/internal/runtimeobs/source.go @@ -3,9 +3,6 @@ package runtimeobs import ( "context" "errors" - "fmt" - "reflect" - "regexp" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" ) @@ -15,55 +12,13 @@ var ( ErrNotRunning = errors.New("Runtime is not running") ) -// SourceResolver selects one immutable source for a page of provider reads. -// An unconfigured resolver returns ErrUnavailable. Registration never resolves -// a source; callers validate each selected source before reading it. -type SourceResolver interface { - providercontract.Declared - ResolveObservationSource(context.Context) (Source, error) -} - -// Source reads one provider-owned Runtime instance. Implementations must verify -// ownership before returning data and must not renew, restart, or stop compute. +// Source is the observation half of a Sandbox Provider, which +// services/core/internal/sandbox/sandbox_provider.go defines. type Source interface { providercontract.Declared - // ObservationProviderType is a nonempty, immutable telemetry identity. - ObservationProviderType() string Observe(context.Context, Target) (Sample, error) } -// MaxBatchTargets bounds the targets of one BatchSource read. -const MaxBatchTargets = 100 - -// BatchSource reads multiple instances under the same ownership rules as Observe. -// Unsupported returns a typed providercontract.UnsupportedError before reading; -// only that result permits per-target observation. Unavailable or failed reads -// must not silently retry through Observe. Successful results preserve target order. -type BatchSource interface { - ObserveBatch(context.Context, []Target) (results []BatchResult, err error) -} - -type BatchResult struct { - Sample Sample - Err error -} - type TargetResolver interface { Resolve(context.Context, string, string) (Target, error) } - -var providerTypePattern = regexp.MustCompile(`^[a-z][a-z0-9_]{0,31}$`) - -// ValidateSource checks every read operation and its immutable provider identity. -func ValidateSource(source Source) error { - if err := providercontract.Validate(source, reflect.TypeFor[Source](), reflect.TypeFor[BatchSource]()); err != nil { - return err - } - if err := providercontract.Require(source, "ObservationProviderType"); err != nil { - return fmt.Errorf("%w: observation identity must be supported", providercontract.ErrContract) - } - if !providerTypePattern.MatchString(source.ObservationProviderType()) { - return fmt.Errorf("%w: invalid Runtime observation provider type", providercontract.ErrContract) - } - return nil -} diff --git a/services/core/internal/runtimeobs/source_test.go b/services/core/internal/runtimeobs/source_test.go index e65753320..655d5a64e 100644 --- a/services/core/internal/runtimeobs/source_test.go +++ b/services/core/internal/runtimeobs/source_test.go @@ -10,78 +10,24 @@ import ( ) type selectingSource struct { - source Source - err error - calls int - support providercontract.Support + source Source + providerType string + err error + calls int } -func (s *selectingSource) ProviderOperations() providercontract.Operations { - return providercontract.Operations{"ResolveObservationSource": s.support} -} -func (s *selectingSource) ResolveObservationSource(context.Context) (Source, error) { +func (s *selectingSource) load(context.Context) (Source, string, error) { s.calls++ - return s.source, s.err -} - -type identityDeclaration struct { - typedSource - support providercontract.Support -} - -func (s identityDeclaration) ProviderOperations() providercontract.Operations { - operations := s.typedSource.ProviderOperations() - operations["ObservationProviderType"] = s.support - return operations -} - -func TestSourceBindingRejectsInvalidIdentityBeforeReadOrExport(t *testing.T) { - for _, test := range []struct { - name, identity string - support providercontract.Support - }{ - {"empty", "", providercontract.Support{State: providercontract.Supported}}, - {"unsafe", "secret:provider", providercontract.Support{State: providercontract.Supported}}, - {"undeclared", "docker", providercontract.Support{}}, - {"unsupported", "docker", providercontract.Support{State: providercontract.Unsupported, Reason: "missing_identity"}}, - } { - t.Run(test.name, func(t *testing.T) { - source := identityDeclaration{typedSource: typedSource{fixedSource: &fixedSource{}, providerType: test.identity}, support: test.support} - resolver := &selectingSource{source: source, support: providercontract.Support{State: providercontract.Supported}} - records := make(chan ExportRecord, 1) - service, err := NewService(observableTarget(), map[string]SourceResolver{"provider": resolver}, WithExporter(channelExporter{records: records}, ExportOptions{})) - if err != nil { - t.Fatal(err) - } - if _, err := service.ObserveSession(t.Context(), "tenant", "session"); !errors.Is(err, providercontract.ErrContract) { - t.Fatalf("invalid identity accepted: %v", err) - } - if source.calls != 0 { - t.Fatal("invalid identity reached provider read") - } - if err := service.Close(t.Context()); err != nil { - t.Fatal(err) - } - if len(records) != 0 { - t.Fatal("invalid identity reached export") - } - }) - } + return s.source, s.providerType, s.err } func observableTarget() fixedResolver { return fixedResolver{target: Target{EnvironmentID: "environment", Mode: ModeManaged, Instance: Instance{AllocationID: "allocation", ProviderKey: "provider", AllocationState: "running"}}} } -func TestSourceResolverRegistrationAndUnavailableSelection(t *testing.T) { - for _, support := range []providercontract.Support{{}, {State: providercontract.Unsupported, Reason: "no_source"}} { - resolver := &selectingSource{support: support} - if _, err := NewService(observableTarget(), map[string]SourceResolver{"provider": resolver}); !errors.Is(err, providercontract.ErrContract) || resolver.calls != 0 { - t.Fatal("invalid resolver registration accepted or performed I/O", err) - } - } - resolver := &selectingSource{err: ErrUnavailable, support: providercontract.Support{State: providercontract.Supported}} - service, err := NewService(observableTarget(), map[string]SourceResolver{"provider": resolver}) +func TestUnavailableAndNilSourceSelection(t *testing.T) { + resolver := &selectingSource{err: ErrUnavailable} + service, err := NewService(observableTarget(), resolver.load) if err != nil || resolver.calls != 0 { t.Fatal("registration resolved unconfigured provider", err) } @@ -89,35 +35,34 @@ func TestSourceResolverRegistrationAndUnavailableSelection(t *testing.T) { if err != nil || observation.Status != StatusUnavailable || observation.Reason != "sample_unavailable" || observation.ProviderType != "" { t.Fatal(observation, err) } - var nilSource *fixedSource - resolver.err, resolver.source = nil, nilSource + resolver.err = nil if _, err := service.ObserveSession(t.Context(), "tenant", "session"); !errors.Is(err, providercontract.ErrContract) { - t.Fatal("typed nil source accepted", err) + t.Fatal("nil source accepted", err) } } type reconfiguringSource struct { - typedSource + *fixedSource resolver *selectingSource next Source } func (s *reconfiguringSource) Observe(ctx context.Context, target Target) (Sample, error) { - s.resolver.source = s.next - return s.typedSource.Observe(ctx, target) + s.resolver.source, s.resolver.providerType = s.next, "next" + return s.fixedSource.Observe(ctx, target) } func TestSourceSelectionStaysBoundForWholePage(t *testing.T) { now := time.Now() - next := typedSource{fixedSource: &fixedSource{sample: Sample{ObservedAt: now}}, providerType: "next"} - resolver := &selectingSource{support: providercontract.Support{State: providercontract.Supported}} - previous := &reconfiguringSource{typedSource: typedSource{fixedSource: &fixedSource{sample: Sample{ObservedAt: now}}, providerType: "previous"}, resolver: resolver, next: next} + next := &fixedSource{sample: Sample{ObservedAt: now}} + resolver := &selectingSource{providerType: "previous"} + previous := &reconfiguringSource{fixedSource: &fixedSource{sample: Sample{ObservedAt: now}}, resolver: resolver, next: next} resolver.source = previous - service, err := NewService(observableTarget(), map[string]SourceResolver{"provider": resolver}) + service, err := NewService(observableTarget(), resolver.load) if err != nil { t.Fatal(err) } - sessions := make([]SessionIdentity, MaxBatchTargets+1) + sessions := make([]SessionIdentity, 3) for index := range sessions { sessions[index] = SessionIdentity{TenantID: "tenant", SessionID: "session"} } diff --git a/services/core/internal/sandbox/configuration.go b/services/core/internal/sandbox/configuration.go index e0fbc5cc6..36ace5baa 100644 --- a/services/core/internal/sandbox/configuration.go +++ b/services/core/internal/sandbox/configuration.go @@ -2,74 +2,11 @@ package sandbox import ( "bytes" - "context" "encoding/json" "io" "slices" - - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" -) - -// Requirement has no implicit default: registration must choose either value. -type Requirement string - -const ( - Required Requirement = "required" - NotRequired Requirement = "not_required" ) -type ConfigurationRequirements struct { - Credential Requirement - PublicOrigin Requirement - Discovery providercontract.Support -} - -// Configuration is an adapter-owned typed value, never a request or response DTO. -// Implementations must exclude secrets from JSON and safe diagnostic output. -type Configuration interface { - HasCredential() bool - ReplacesCredential() bool -} - -// ConfigurationRecord separates public selectors, read-only observations and -// secret bytes. Store encrypts Secret with the installation and generation. -// Only adapter codecs may produce Public and Metadata; neither is input passthrough. -type ConfigurationRecord struct { - Public json.RawMessage - Metadata json.RawMessage - Secret []byte `json:"-"` -} - -// ConfigurationAdapter owns all interpretation of provider configuration. -// Decode loads retained ownership without remote discovery or new-build admission. -// Normalize validates a candidate; ResolveChange first applies omitted-field -// inheritance, then normalizes. Equal compares normalized identity, excluding -// discovery metadata and explicit credential-submission intent. -type ConfigurationAdapter interface { - Requirements() ConfigurationRequirements - DecodeInput(public, credential json.RawMessage) (Configuration, error) - Encode(Configuration) (ConfigurationRecord, error) - Decode(ConfigurationRecord) (Configuration, error) - Normalize(Selection) (Selection, error) - ResolveChange(next, previous Selection) (Selection, error) - WithCredential(owner, candidate Configuration) (Configuration, error) - Equal(a, b Configuration) (bool, error) -} - -// ConfigurationDiscoveryInput is a transient read-only request. Query is typed -// and validated by the adapter; it cannot select a compute mutation. -type ConfigurationDiscoveryInput struct { - Configuration json.RawMessage `json:"configuration" swaggertype:"object"` - Credential json.RawMessage `json:"credential" swaggertype:"object"` - Query json.RawMessage `json:"query" swaggertype:"object"` -} - -// ConfigurationDiscoverer is separate from compute and candidate admission. -// Support must also be explicitly declared in ConfigurationRequirements. -type ConfigurationDiscoverer interface { - DiscoverConfiguration(context.Context, ConfigurationDiscoveryInput, ProcessPaths) (json.RawMessage, error) -} - // DecodeConfigurationObject rejects unknown fields, null, nonobjects and trailing // input without exposing submitted content in its error. Missing objects are empty. func DecodeConfigurationObject(raw json.RawMessage, target any, allowed ...string) error { diff --git a/services/core/internal/sandbox/configuration_errors.go b/services/core/internal/sandbox/configuration_errors.go deleted file mode 100644 index 7c02d9728..000000000 --- a/services/core/internal/sandbox/configuration_errors.go +++ /dev/null @@ -1,24 +0,0 @@ -package sandbox - -// ConfigurationError is a fixed safe diagnostic, never SDK text or submitted data. -// Class describes the request outcome; it does not authorize replay. -type ConfigurationError struct { - Class ConfigurationErrorClass - Code, Param, Message string -} -type ConfigurationErrorClass string - -const ( - ConfigurationInvalid ConfigurationErrorClass = "invalid" - ConfigurationConflict ConfigurationErrorClass = "conflict" - ConfigurationUnconfirmed ConfigurationErrorClass = "unconfirmed" -) - -func (e *ConfigurationError) Error() string { return e.Message } - -var ( - ErrCredentialRejected = &ConfigurationError{ConfigurationInvalid, "sandbox_credential_invalid", "credential", "The sandbox provider credential was rejected."} - ErrCredentialOwnership = &ConfigurationError{ConfigurationConflict, "sandbox_credential_ownership", "credential", "The credential cannot manage the retained deployment. Reset before changing accounts."} - ErrConfigurationUnconfirmed = &ConfigurationError{ConfigurationUnconfirmed, "sandbox_verification_unconfirmed", "", "Sandbox provider verification could not be confirmed."} - ErrConfigurationSelection = &ConfigurationError{ConfigurationInvalid, "sandbox_configuration_invalid", "configuration", "Select a ready immutable provider configuration with matching resources."} -) diff --git a/services/core/internal/sandbox/docker/operations.go b/services/core/internal/sandbox/docker/operations.go index c2758008d..2722da0dd 100644 --- a/services/core/internal/sandbox/docker/operations.go +++ b/services/core/internal/sandbox/docker/operations.go @@ -3,38 +3,27 @@ package docker import ( "context" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) -var _ sandbox.CheckpointProvider = (*Provider)(nil) -var _ sandbox.SelectionDiscoverer = (*Provider)(nil) -var _ sandbox.CredentialVerifier = (*Provider)(nil) -var _ runtimeobs.BatchSource = (*Provider)(nil) - // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "Suspend": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "Resume": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Supported}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "docker_does_not_support_batch_observation"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "docker_does_not_support_selection_discovery"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "docker_does_not_support_credential_verification"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Suspend": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Resume": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } @@ -65,12 +54,3 @@ func (p *Provider) RunCommandCompute(context.Context, sandbox.Reference, sandbox func (p *Provider) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: Operations()["ResumeCompute"].Reason} } -func (p *Provider) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: Operations()["ObserveBatch"].Reason} -} -func (p *Provider) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: Operations()["DiscoverSelection"].Reason} -} -func (p *Provider) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: Operations()["VerifyCredential"].Reason} -} diff --git a/services/core/internal/sandbox/docker/resources.go b/services/core/internal/sandbox/docker/resources.go index d48dfcb17..863af750a 100644 --- a/services/core/internal/sandbox/docker/resources.go +++ b/services/core/internal/sandbox/docker/resources.go @@ -13,10 +13,6 @@ import ( "github.com/moby/moby/client" ) -var _ runtimeobs.Source = (*Provider)(nil) - -func (*Provider) ObservationProviderType() string { return "docker" } - // Observe is read-only. Inspect verifies allocation ownership before Docker // statistics are requested; it never renews or changes the container. func (p *Provider) Observe(ctx context.Context, target runtimeobs.Target) (runtimeobs.Sample, error) { @@ -91,7 +87,3 @@ func sampleFromDocker(inspected container.InspectResponse, stats dockerStatsResp } return sample, nil } - -func (p *Provider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/sandbox/e2b/configuration.go b/services/core/internal/sandbox/e2b/configuration.go index 752fd60c7..5a00ed53e 100644 --- a/services/core/internal/sandbox/e2b/configuration.go +++ b/services/core/internal/sandbox/e2b/configuration.go @@ -32,7 +32,8 @@ func configuration(s sandbox.Selection) *DeploymentConfiguration { return c } func (ConfigurationAdapter) Requirements() sandbox.ConfigurationRequirements { - return sandbox.ConfigurationRequirements{Credential: sandbox.Required, PublicOrigin: sandbox.Required, Discovery: providercontract.Support{State: providercontract.Supported}} + return sandbox.ConfigurationRequirements{Credential: sandbox.Required, PublicOrigin: sandbox.Required, Discovery: providercontract.Support{State: providercontract.Supported}, + SelectionDiscovery: providercontract.Support{State: providercontract.Supported}, CredentialVerification: providercontract.Support{State: providercontract.Supported}} } func (ConfigurationAdapter) DecodeInput(public, secret json.RawMessage) (sandbox.Configuration, error) { var p publicConfiguration diff --git a/services/core/internal/sandbox/e2b/credential.go b/services/core/internal/sandbox/e2b/credential.go index 88df2e1b7..42452ff62 100644 --- a/services/core/internal/sandbox/e2b/credential.go +++ b/services/core/internal/sandbox/e2b/credential.go @@ -9,7 +9,11 @@ import ( // VerifyCredential is read-only and bounded. A public readable template alone // does not prove team ownership. References are one bounded Core-owned page. -func (p *Provider) VerifyCredential(ctx context.Context, refs []sandbox.Reference) error { +func (ConfigurationAdapter) VerifyCredential(ctx context.Context, c sandbox.DirectConfig, refs []sandbox.Reference) error { + p, err := newDirect(c) + if err != nil { + return err + } ctx, cancel := context.WithTimeout(ctx, 30*time.Second) defer cancel() if len(refs) > MaxCredentialReferences { @@ -31,7 +35,7 @@ func (p *Provider) VerifyCredential(ctx context.Context, refs []sandbox.Referenc case "team_mismatch", "invalid": return sandbox.ErrCredentialOwnership case "": - if out.DeploymentValid && out.Info == nil && out.Command == nil && out.Observations == nil && out.TemplateBuild == nil { + if out.DeploymentValid && out.Info == nil && out.Command == nil && out.Observation == nil && out.TemplateBuild == nil { return nil } } diff --git a/services/core/internal/sandbox/e2b/deployment.go b/services/core/internal/sandbox/e2b/deployment.go index cc28e153f..3391d3f3e 100644 --- a/services/core/internal/sandbox/e2b/deployment.go +++ b/services/core/internal/sandbox/e2b/deployment.go @@ -70,25 +70,29 @@ func WithTemplateBuild(input sandbox.Selection, build *DeploymentBuild) sandbox. } // DiscoverSelection checks the immutable native build without allocating compute. -func (p *Provider) DiscoverSelection(ctx context.Context, s sandbox.Selection) (sandbox.Selection, error) { +func (ConfigurationAdapter) DiscoverSelection(ctx context.Context, c sandbox.DirectConfig) (sandbox.Selection, error) { + p, err := newDirect(c) + if err != nil { + return sandbox.Selection{}, err + } build, err := p.ValidateDeployment(ctx) if err != nil { if errors.Is(err, sandbox.ErrCredentialRejected) || errors.Is(err, sandbox.ErrCredentialOwnership) { - return s, err + return sandbox.Selection{}, err } if errors.Is(err, sandbox.ErrInvalid) { - return s, sandbox.ErrConfigurationSelection + return sandbox.Selection{}, sandbox.ErrConfigurationSelection } - return s, sandbox.ErrConfigurationUnconfirmed + return sandbox.Selection{}, sandbox.ErrConfigurationUnconfirmed } recorded := &DeploymentBuild{Status: build.Status, CPUs: int32(build.CPUs), MemoryMiB: int32(build.MemoryMiB)} if build.RootDiskMiB != nil && *build.RootDiskMiB <= math.MaxInt32 { disk := int32(*build.RootDiskMiB) recorded.RootDiskMiB = &disk } - s = WithTemplateBuild(s, recorded) + s := WithTemplateBuild(c.Selection, recorded) if err := ValidateSpecification(s.DeploymentSpec); err != nil { - return s, &sandbox.ValidationError{Param: "resources", Message: "E2B template build resources are outside the supported sandbox limits; select another build"} + return sandbox.Selection{}, &sandbox.ValidationError{Param: "resources", Message: "E2B template build resources are outside the supported sandbox limits; select another build"} } return s, nil } diff --git a/services/core/internal/sandbox/e2b/helper_contract.go b/services/core/internal/sandbox/e2b/helper_contract.go index 7e9a96f25..7d701cf51 100644 --- a/services/core/internal/sandbox/e2b/helper_contract.go +++ b/services/core/internal/sandbox/e2b/helper_contract.go @@ -5,7 +5,6 @@ import ( "time" "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) @@ -17,7 +16,6 @@ const MaxOutputBytes = 1024 * 1024 const MaxRequestBytes = 72 * 1024 * 1024 const MaxResponseBytes = 16 * 1024 * 1024 const MaxCredentialReferences = 32 -const MaxObservationReferences = runtimeobs.MaxBatchTargets const MaxCommandInputBytes = sandbox.MaxCommandInputBytes // HelperOperations declares the complete set of one-shot helper operations. @@ -40,17 +38,6 @@ func (q Request) Validate() error { return sandbox.ErrInvalid } switch q.Operation { - case "observe": - if len(q.References) < 1 || len(q.References) > MaxObservationReferences { - return sandbox.ErrInvalid - } - seen := map[sandbox.Reference]bool{} - for _, r := range q.References { - if !validReference(r) || seen[r] { - return sandbox.ErrInvalid - } - seen[r] = true - } case "verify_credential": if len(q.References) > MaxCredentialReferences { return sandbox.ErrInvalid @@ -74,7 +61,7 @@ type Request struct { Operation string Config Config Reference sandbox.Reference - // References lists the allocations of one read-only observe request. + // References lists the allocations of one verify_credential request. References []sandbox.Reference `json:",omitempty"` Bootstrap *sandbox.Bootstrap `json:",omitempty"` RuntimeBootstrap *runtimebootstrap.Connection `json:",omitempty"` @@ -90,7 +77,7 @@ type Response struct { TemplateBuild *TemplateBuild `json:",omitempty"` Templates []TemplateSummary `json:",omitempty"` Builds []ReadyBuild `json:",omitempty"` - Observations []Observation `json:",omitempty"` + Observation *Observation `json:",omitempty"` } func (r Response) Validate() error { diff --git a/services/core/internal/sandbox/e2b/internal/contractgen/main.go b/services/core/internal/sandbox/e2b/internal/contractgen/main.go index 6d79d9544..023dc781e 100644 --- a/services/core/internal/sandbox/e2b/internal/contractgen/main.go +++ b/services/core/internal/sandbox/e2b/internal/contractgen/main.go @@ -83,8 +83,7 @@ func main() { values := map[string]any{ "PROTOCOL_VERSION": e2b.ProtocolVersion, "SDK_VERSION": sdk, "MAX_REQUEST": e2b.MaxRequestBytes, "MAX_RESPONSE": e2b.MaxResponseBytes, - "MAX_OUTPUT": e2b.MaxOutputBytes, "MAX_COMMAND_INPUT": e2b.MaxCommandInputBytes, - "MAX_OBSERVATION_REFERENCES": e2b.MaxObservationReferences, "MAX_CREDENTIAL_REFERENCES": e2b.MaxCredentialReferences, + "MAX_OUTPUT": e2b.MaxOutputBytes, "MAX_COMMAND_INPUT": e2b.MaxCommandInputBytes, "MAX_CREDENTIAL_REFERENCES": e2b.MaxCredentialReferences, "OPERATIONS": e2b.HelperOperations(), "ERROR_CODES": e2b.HelperErrors(), "REQUEST_FIELDS": fields(reflect.TypeFor[e2b.Request]()), "RESPONSE_FIELDS": fields(reflect.TypeFor[e2b.Response]()), "REFERENCE_FIELDS": fields(reflect.TypeFor[sandbox.Reference]()), @@ -131,7 +130,7 @@ func fixtures() []byte { for _, operation := range e2b.HelperOperations() { copy := q copy.Operation = operation - if operation == "observe" || operation == "verify_credential" { + if operation == "verify_credential" { copy.References = []sandbox.Reference{r} } add("request", operation, true, copy) @@ -148,32 +147,22 @@ func fixtures() []byte { value[item.field] = item.value add("request", item.name, false, value) } - for _, operation := range []string{"observe", "verify_credential"} { - limit := e2b.MaxObservationReferences - if operation == "verify_credential" { - limit = e2b.MaxCredentialReferences - } - for _, count := range []int{0, limit, limit + 1} { - copy := q - copy.Operation = operation - copy.References = make([]sandbox.Reference, count) - for index := range copy.References { - copy.References[index] = r - copy.References[index].AllocationID = fmt.Sprintf("%08x-3333-4333-8333-333333333333", index+1) - } - add("request", fmt.Sprintf("%s-count-%d", operation, count), count <= limit && (operation != "observe" || count > 0), copy) - } - for _, refs := range []any{map[string]any{}, "invalid", []any{nil}} { - value := object(q) - value["Operation"] = operation - value["References"] = refs - add("request", operation+"-references-type", false, value) + for _, count := range []int{0, e2b.MaxCredentialReferences, e2b.MaxCredentialReferences + 1} { + copy := q + copy.Operation = "verify_credential" + copy.References = make([]sandbox.Reference, count) + for index := range copy.References { + copy.References[index] = r + copy.References[index].AllocationID = fmt.Sprintf("%08x-3333-4333-8333-333333333333", index+1) } + add("request", fmt.Sprintf("verify_credential-count-%d", count), count <= e2b.MaxCredentialReferences, copy) + } + for _, refs := range []any{map[string]any{}, "invalid", []any{nil}} { + value := object(q) + value["Operation"] = "verify_credential" + value["References"] = refs + add("request", "verify_credential-references-type", false, value) } - duplicate := q - duplicate.Operation = "observe" - duplicate.References = []sandbox.Reference{r, r} - add("request", "duplicate-observation", false, duplicate) for _, code := range e2b.HelperErrors() { add("response", "error-"+code, true, e2b.Response{Version: e2b.ProtocolVersion, ErrorCode: code}) } diff --git a/services/core/internal/sandbox/e2b/observations.go b/services/core/internal/sandbox/e2b/observations.go index e931805ad..063ba1307 100644 --- a/services/core/internal/sandbox/e2b/observations.go +++ b/services/core/internal/sandbox/e2b/observations.go @@ -9,21 +9,16 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) -var ( - _ runtimeobs.Source = (*Provider)(nil) - _ runtimeobs.BatchSource = (*Provider)(nil) -) - // maxClockLead tolerates an E2B metrics timestamp slightly ahead of Core's // clock. A larger lead is treated as an unavailable sample, never as fresh data. const maxClockLead = 30 * time.Second -// Observation is one allocation's latest E2B metrics point. Status is -// observed, not_running, unavailable or ownership; only observed carries -// values. A malformed E2B point is unavailable for its row only. Values keep E2B's units: CPUUsedPct is a percentage of all -// CPUCount cores, and memory and disk are in bytes. +// Observation is the requested allocation's latest E2B metrics point. Status +// is observed, not_running, unavailable or ownership; only observed carries +// values. A malformed E2B point is unavailable. Values keep E2B's units: +// CPUUsedPct is a percentage of all CPUCount cores, and memory and disk are in +// bytes. type Observation struct { - sandbox.Reference Status string ObservedAt, StartedAt *time.Time `json:",omitempty"` CPUCount, CPUUsedPct *float64 `json:",omitempty"` @@ -31,83 +26,46 @@ type Observation struct { DiskUsed, DiskTotal *uint64 `json:",omitempty"` } -func (*Provider) ObservationProviderType() string { return "e2b" } - -// Observe reads one allocation through the same helper request as ObserveBatch. +// Observe reads one allocation with one helper request. The helper takes the +// sandbox ID from its private receipt, confirms the running sandbox by its +// allocation labels and reads E2B's metrics. It never connects to, renews or +// changes a sandbox. func (p *Provider) Observe(ctx context.Context, target runtimeobs.Target) (runtimeobs.Sample, error) { - results, _ := p.ObserveBatch(ctx, []runtimeobs.Target{target}) - return results[0].Sample, results[0].Err -} - -// ObserveBatch reads up to runtimeobs.MaxBatchTargets allocations with one -// helper request. The helper takes sandbox IDs from its private receipts, -// confirms each running sandbox by its allocation labels and reads E2B's batch -// metrics once. It never connects to, renews or changes a sandbox. -func (p *Provider) ObserveBatch(ctx context.Context, targets []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - results := make([]runtimeobs.BatchResult, len(targets)) - references := make([]sandbox.Reference, 0, len(targets)) - positions := make([]int, 0, len(targets)) - for index, target := range targets { - reference := sandbox.Reference{TenantID: target.TenantID, EnvironmentID: target.EnvironmentID, AllocationID: target.Instance.AllocationID} - switch { - case target.Mode != runtimeobs.ModeManaged || !validReference(reference) || len(targets) > runtimeobs.MaxBatchTargets: - results[index].Err = sandbox.ErrInvalid - case target.Instance.ProviderKey != p.config.InstallationID: - results[index].Err = sandbox.ErrOwnership - default: - references = append(references, reference) - positions = append(positions, index) - } - } - if len(references) == 0 { - return results, nil + reference := sandbox.Reference{TenantID: target.TenantID, EnvironmentID: target.EnvironmentID, AllocationID: target.Instance.AllocationID} + if target.Mode != runtimeobs.ModeManaged || !validReference(reference) { + return runtimeobs.Sample{}, sandbox.ErrInvalid } - observations, err := p.observe(ctx, references) - now := p.now() - for offset, index := range positions { - if err != nil { - results[index].Err = err - continue - } - results[index].Sample, results[index].Err = sampleFromObservation(observations[offset], now) + if target.Instance.ProviderKey != p.config.InstallationID { + return runtimeobs.Sample{}, sandbox.ErrOwnership } - return results, nil -} - -func (p *Provider) observe(ctx context.Context, references []sandbox.Reference) ([]Observation, error) { deadline, ok := ctx.Deadline() if !ok { - return nil, sandbox.ErrInvalid + return runtimeobs.Sample{}, sandbox.ErrInvalid } if err := ctx.Err(); err != nil { - return nil, err + return runtimeobs.Sample{}, err } - out, err := p.caller.Call(ctx, Request{Version: ProtocolVersion, Operation: "observe", Config: p.config, References: references, Deadline: deadline}) + out, err := p.caller.Call(ctx, Request{Version: ProtocolVersion, Operation: "observe", Config: p.config, Reference: reference, Deadline: deadline}) if ctxErr := ctx.Err(); ctxErr != nil { - return nil, ctxErr + return runtimeobs.Sample{}, ctxErr } if err != nil || out.Version != ProtocolVersion || out.Info != nil || out.Command != nil || out.DeploymentValid || out.TemplateBuild != nil { - return nil, runtimeobs.ErrUnavailable + return runtimeobs.Sample{}, runtimeobs.ErrUnavailable } switch out.ErrorCode { case "": case "invalid": - return nil, sandbox.ErrInvalid + return runtimeobs.Sample{}, sandbox.ErrInvalid case "ownership": - return nil, sandbox.ErrOwnership + return runtimeobs.Sample{}, sandbox.ErrOwnership default: - // E2B API failures, including a rejected credential, leave rows unavailable. - return nil, runtimeobs.ErrUnavailable - } - if len(out.Observations) != len(references) { - return nil, sandbox.ErrInvalid + // E2B API failures, including a rejected credential, are unavailable. + return runtimeobs.Sample{}, runtimeobs.ErrUnavailable } - for index, observation := range out.Observations { - if observation.Reference != references[index] { - return nil, sandbox.ErrOwnership - } + if out.Observation == nil { + return runtimeobs.Sample{}, sandbox.ErrInvalid } - return out.Observations, nil + return sampleFromObservation(*out.Observation, p.now()) } // sampleFromObservation maps E2B's latest point to the provider-neutral @@ -125,7 +83,7 @@ func sampleFromObservation(observation Observation, now time.Time) (runtimeobs.S // An unknown status breaks Core's own helper protocol. return runtimeobs.Sample{}, sandbox.ErrInvalid } - // Malformed provider data leaves this row unavailable, not the whole page. + // Malformed provider data leaves this sample unavailable. if observation.ObservedAt == nil || observation.StartedAt == nil || observation.CPUCount == nil || observation.CPUUsedPct == nil || observation.MemUsed == nil || observation.MemTotal == nil || !finite(*observation.CPUCount) || *observation.CPUCount <= 0 || !finite(*observation.CPUUsedPct) || *observation.CPUUsedPct < 0 || *observation.MemTotal == 0 { @@ -160,7 +118,3 @@ func sampleFromObservation(observation Observation, now time.Time) (runtimeobs.S } func finite(value float64) bool { return !math.IsNaN(value) && !math.IsInf(value, 0) } - -func (p *Provider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/sandbox/e2b/observations_test.go b/services/core/internal/sandbox/e2b/observations_test.go index c736e862c..04acefd4d 100644 --- a/services/core/internal/sandbox/e2b/observations_test.go +++ b/services/core/internal/sandbox/e2b/observations_test.go @@ -6,56 +6,48 @@ import ( "time" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" - "github.com/google/uuid" ) -func TestObserveBatchMapsMetricsAndKeepsUnmeasuredValuesNull(t *testing.T) { +func TestObserveMapsMetricsAndKeepsUnmeasuredValuesNull(t *testing.T) { p, caller, running := fixture(t) - stopped := sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()} now := time.Date(2026, 9, 25, 10, 31, 37, 0, time.UTC) p.now = func() time.Time { return now } started, observed := now.Add(-3*time.Second), now.Add(time.Second) // E2B's clock leads by one second. count, percent, memoryUsed, memoryTotal, diskUsed, diskTotal := 2.0, 19.55, uint64(183836672), uint64(2079141888), uint64(1593188352), uint64(23511863296) - caller.response.Observations = []Observation{ - {Reference: running, Status: "observed", ObservedAt: &observed, StartedAt: &started, CPUCount: &count, CPUUsedPct: &percent, - MemUsed: &memoryUsed, MemTotal: &memoryTotal, DiskUsed: &diskUsed, DiskTotal: &diskTotal}, - {Reference: stopped, Status: "not_running"}, - } - targets := []runtimeobs.Target{} - for _, reference := range []sandbox.Reference{running, stopped} { - targets = append(targets, runtimeobs.Target{TenantID: reference.TenantID, EnvironmentID: reference.EnvironmentID, Mode: runtimeobs.ModeManaged, - Instance: runtimeobs.Instance{AllocationID: reference.AllocationID, ProviderKey: p.config.InstallationID}}) - } - results, err := p.ObserveBatch(bounded(t), targets) - if err != nil || len(caller.requests) != 1 || caller.requests[0].Operation != "observe" || len(caller.requests[0].References) != 2 || - caller.requests[0].References[1] != stopped || caller.requests[0].Deadline.IsZero() { - t.Fatalf("batch was not one bounded helper request: err=%v %+v", err, caller.requests) - } - sample := results[0].Sample - if results[0].Err != nil || !sample.ObservedAt.Equal(now) || !sample.StartedAt.Equal(started) || + caller.response.Observation = &Observation{Status: "observed", ObservedAt: &observed, StartedAt: &started, CPUCount: &count, CPUUsedPct: &percent, + MemUsed: &memoryUsed, MemTotal: &memoryTotal, DiskUsed: &diskUsed, DiskTotal: &diskTotal} + target := runtimeobs.Target{TenantID: running.TenantID, EnvironmentID: running.EnvironmentID, Mode: runtimeobs.ModeManaged, + Instance: runtimeobs.Instance{AllocationID: running.AllocationID, ProviderKey: p.config.InstallationID}} + sample, err := p.Observe(bounded(t), target) + if err != nil || len(caller.requests) != 1 || caller.requests[0].Operation != "observe" || + caller.requests[0].Reference != running || caller.requests[0].Deadline.IsZero() { + t.Fatalf("observation was not one bounded helper request: err=%v %+v", err, caller.requests) + } + if !sample.ObservedAt.Equal(now) || !sample.StartedAt.Equal(started) || *sample.CPUUtilizationRatio != .1955 || *sample.CPUCapacityCores != 2 || sample.CPUUsageSecondsTotal != nil || *sample.MemoryUsageBytes != memoryUsed || *sample.MemoryLimitBytes != memoryTotal || *sample.DiskUsageBytes != diskUsed || *sample.DiskLimitBytes != diskTotal { - t.Fatalf("metrics were not mapped: %+v %v", sample, results[0].Err) - } - if !errors.Is(results[1].Err, runtimeobs.ErrNotRunning) { - t.Fatalf("absent sandbox = %v", results[1].Err) + t.Fatalf("metrics were not mapped: %+v", sample) } - caller.response.Observations[0].DiskTotal = nil - results, _ = p.ObserveBatch(bounded(t), targets) - if results[0].Err != nil || results[0].Sample.DiskUsageBytes != nil || results[0].Sample.DiskLimitBytes != nil { - t.Fatalf("unreported disk was not null: %+v %v", results[0].Sample, results[0].Err) + caller.response.Observation.DiskTotal = nil + if sample, err = p.Observe(bounded(t), target); err != nil || sample.DiskUsageBytes != nil || sample.DiskLimitBytes != nil { + t.Fatalf("unreported disk was not null: %+v %v", sample, err) + } + caller.response.Observation.MemTotal = nil + if _, err = p.Observe(bounded(t), target); !errors.Is(err, runtimeobs.ErrUnavailable) { + t.Fatalf("malformed point = %v", err) + } + caller.response.Observation = &Observation{Status: "not_running"} + if _, err = p.Observe(bounded(t), target); !errors.Is(err, runtimeobs.ErrNotRunning) { + t.Fatalf("absent sandbox = %v", err) } - caller.response.Observations[0].MemTotal = nil - results, _ = p.ObserveBatch(bounded(t), targets) - if !errors.Is(results[0].Err, runtimeobs.ErrUnavailable) || !errors.Is(results[1].Err, runtimeobs.ErrNotRunning) { - t.Fatalf("a malformed point affected more than its row: %+v", results) + caller.response.Observation = nil + if _, err = p.Observe(bounded(t), target); err == nil { + t.Fatal("a success without an observation was accepted") } caller.response.ErrorCode = "unconfirmed" - results, _ = p.ObserveBatch(bounded(t), targets) - if !errors.Is(results[0].Err, runtimeobs.ErrUnavailable) || !errors.Is(results[1].Err, runtimeobs.ErrUnavailable) { - t.Fatalf("E2B failure was not unavailable: %+v", results) + if _, err = p.Observe(bounded(t), target); !errors.Is(err, runtimeobs.ErrUnavailable) { + t.Fatalf("E2B failure was not unavailable: %v", err) } } diff --git a/services/core/internal/sandbox/e2b/operations.go b/services/core/internal/sandbox/e2b/operations.go index 14530ebf9..a3da4688e 100644 --- a/services/core/internal/sandbox/e2b/operations.go +++ b/services/core/internal/sandbox/e2b/operations.go @@ -3,38 +3,27 @@ package e2b import ( "context" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) -var _ sandbox.CheckpointProvider = (*Provider)(nil) -var _ sandbox.SelectionDiscoverer = (*Provider)(nil) -var _ sandbox.CredentialVerifier = (*Provider)(nil) -var _ runtimeobs.BatchSource = (*Provider)(nil) - // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "Suspend": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "Resume": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Supported}, - "ObserveBatch": {State: providercontract.Supported}, - "DiscoverSelection": {State: providercontract.Supported}, - "VerifyCredential": {State: providercontract.Supported}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Suspend": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Resume": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } diff --git a/services/core/internal/sandbox/e2b/provider.go b/services/core/internal/sandbox/e2b/provider.go index 3328eb0db..c07924250 100644 --- a/services/core/internal/sandbox/e2b/provider.go +++ b/services/core/internal/sandbox/e2b/provider.go @@ -56,7 +56,7 @@ func Discover(ctx context.Context, caller Caller, binary, apiKey, apiURL, domain if out.ErrorCode == "invalid" { return Response{}, sandbox.ErrInvalid } - if out.ErrorCode != "" || out.Info != nil || out.Command != nil || out.TemplateBuild != nil || out.Observations != nil { + if out.ErrorCode != "" || out.Info != nil || out.Command != nil || out.TemplateBuild != nil || out.Observation != nil { return Response{}, sandbox.ErrComputeUnconfirmed } if operation == "list_templates" { @@ -151,6 +151,33 @@ func NewWithCaller(c Config, caller Caller) (*Provider, error) { } return &Provider{config: c, caller: caller, now: time.Now}, nil } + +// BuildDirect builds the Provider for one direct-mode selection. +func BuildDirect(c sandbox.DirectConfig) (sandbox.SandboxProvider, error) { return newDirect(c) } + +func newDirect(c sandbox.DirectConfig) (*Provider, error) { + deployment := configuration(c.Selection) + if deployment == nil { + return nil, errors.New("E2B deployment configuration is unavailable") + } + binary, state, err := InstalledPaths(c.ProcessPaths) + if err != nil { + return nil, err + } + // Only a candidate that omitted its resources has none; its validation + // reads them from the template build before the candidate is rebuilt. + var resources *sandbox.Resources + if c.Selection.DeploymentSpec.Resources != (sandbox.Resources{}) { + resources = &c.Selection.DeploymentSpec.Resources + } + provider, err := NewWithCaller(Config{Binary: binary, StateDir: state, + Resources: resources, InstallationID: c.InstallationID, APIKey: deployment.APIKey, Template: deployment.Template, + APIURL: deployment.APIURL, Domain: deployment.Domain, TimeoutSeconds: 3600}, &ProcessCaller{Fence: c.Fence}) + if err != nil { + return nil, errors.New("E2B provider cannot load; check the installed helper and private state directory") + } + return provider, nil +} func (p *Provider) call(ctx context.Context, operation string, r sandbox.Reference, b *sandbox.Bootstrap, command *sandbox.Command) (Response, error) { deadline, ok := ctx.Deadline() if !ok || (operation != "validate_deployment" && !validReference(r)) { @@ -216,7 +243,7 @@ func (p *Provider) ValidateDeployment(ctx context.Context) (TemplateBuild, error return TemplateBuild{}, err } build := out.TemplateBuild - if !out.DeploymentValid || out.Info != nil || out.Command != nil || out.Observations != nil || build == nil || build.Status != "ready" || + if !out.DeploymentValid || out.Info != nil || out.Command != nil || out.Observation != nil || build == nil || build.Status != "ready" || build.CPUs == 0 || build.MemoryMiB == 0 || build.RootDiskMiB != nil && *build.RootDiskMiB == 0 || p.config.Resources != nil && (build.CPUs != p.config.Resources.CPUs || build.MemoryMiB != p.config.Resources.MemoryMiB) { return TemplateBuild{}, sandbox.ErrComputeUnconfirmed diff --git a/services/core/internal/sandbox/microsandbox/operations.go b/services/core/internal/sandbox/microsandbox/operations.go index f59c9ed02..13e81ff96 100644 --- a/services/core/internal/sandbox/microsandbox/operations.go +++ b/services/core/internal/sandbox/microsandbox/operations.go @@ -1,49 +1,25 @@ package microsandbox -import ( - "context" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" -) - -var _ sandbox.CheckpointProvider = (*Provider)(nil) -var _ sandbox.SelectionDiscoverer = (*Provider)(nil) -var _ sandbox.CredentialVerifier = (*Provider)(nil) -var _ runtimeobs.BatchSource = (*Provider)(nil) +import "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Supported}, - "NewCompute": {State: providercontract.Supported}, - "GetCompute": {State: providercontract.Supported}, - "Suspend": {State: providercontract.Supported}, - "Resume": {State: providercontract.Supported}, - "KillCompute": {State: providercontract.Supported}, - "DeleteSnapshot": {State: providercontract.Supported}, - "RunCommandCompute": {State: providercontract.Supported}, - "ResumeCompute": {State: providercontract.Supported}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Supported}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "microsandbox_does_not_support_batch_observation"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "microsandbox_does_not_support_selection_discovery"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "microsandbox_does_not_support_credential_verification"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Supported}, + "NewCompute": {State: providercontract.Supported}, + "GetCompute": {State: providercontract.Supported}, + "Suspend": {State: providercontract.Supported}, + "Resume": {State: providercontract.Supported}, + "KillCompute": {State: providercontract.Supported}, + "DeleteSnapshot": {State: providercontract.Supported}, + "RunCommandCompute": {State: providercontract.Supported}, + "ResumeCompute": {State: providercontract.Supported}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } -func (p *Provider) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: Operations()["ObserveBatch"].Reason} -} -func (p *Provider) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: Operations()["DiscoverSelection"].Reason} -} -func (p *Provider) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: Operations()["VerifyCredential"].Reason} -} diff --git a/services/core/internal/sandbox/microsandbox/provider.go b/services/core/internal/sandbox/microsandbox/provider.go index 6ba4eeafc..d876702be 100644 --- a/services/core/internal/sandbox/microsandbox/provider.go +++ b/services/core/internal/sandbox/microsandbox/provider.go @@ -16,7 +16,6 @@ type Provider struct { } var _ sandbox.SandboxProvider = (*Provider)(nil) -var _ sandbox.CheckpointProvider = (*Provider)(nil) func NewWithCaller(c Config, caller Caller) (*Provider, error) { if c.Validate() != nil || caller == nil { @@ -213,9 +212,3 @@ func (p *Provider) NewCompute(ctx context.Context, r sandbox.Reference, generati } return c, nil } - -// Quiescent includes a helper that outlived the caller's canceled context. -func (p *Provider) Quiescent() bool { - v, ok := p.caller.(interface{ Quiescent() bool }) - return ok && v.Quiescent() -} diff --git a/services/core/internal/sandbox/microsandbox/resources.go b/services/core/internal/sandbox/microsandbox/resources.go index 6b4619147..ce586b389 100644 --- a/services/core/internal/sandbox/microsandbox/resources.go +++ b/services/core/internal/sandbox/microsandbox/resources.go @@ -10,10 +10,6 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) -var _ runtimeobs.Source = (*Provider)(nil) - -func (*Provider) ObservationProviderType() string { return "microsandbox" } - // Observe reads one point-in-time native metrics snapshot through the existing // one-shot helper. The persisted compute receipt selects the exact generation; // browser input and provider display names never select a sandbox. @@ -74,7 +70,3 @@ func sampleFromMetrics(config Config, metrics Metrics) (runtimeobs.Sample, error MemoryUsageBytes: &memoryUsage, MemoryLimitBytes: &memoryLimit, }, nil } - -func (p *Provider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/sandbox/node/connection_test.go b/services/core/internal/sandbox/node/connection_test.go index 30f88ecd8..abee501b3 100644 --- a/services/core/internal/sandbox/node/connection_test.go +++ b/services/core/internal/sandbox/node/connection_test.go @@ -176,7 +176,7 @@ func TestCopiedIdentityCannotReplaceNodeWithInflightCreate(t *testing.T) { hub.mu.Unlock() ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) defer cancel() - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) r := reference() created := make(chan error, 1) go func() { _, err := proxy.Create(ctx, sandbox.Bootstrap{Reference: r}); created <- err }() diff --git a/services/core/internal/sandbox/node/creation_settlement_test.go b/services/core/internal/sandbox/node/creation_settlement_test.go index 02ef95248..09d8daec5 100644 --- a/services/core/internal/sandbox/node/creation_settlement_test.go +++ b/services/core/internal/sandbox/node/creation_settlement_test.go @@ -73,7 +73,7 @@ func TestNodeCarriesCreationSettlementWithoutConvertingFailureToSuccess(t *testi } }() wait(t, func() bool { return hub.Online(id.NodeID) }) - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) for _, operation := range []func(context.Context) (sandbox.Info, error){ func(ctx context.Context) (sandbox.Info, error) { return proxy.Create(ctx, sandbox.Bootstrap{Reference: ref}) diff --git a/services/core/internal/sandbox/node/docker_live_test.go b/services/core/internal/sandbox/node/docker_live_test.go index 13ab16b51..48695bfff 100644 --- a/services/core/internal/sandbox/node/docker_live_test.go +++ b/services/core/internal/sandbox/node/docker_live_test.go @@ -76,7 +76,7 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { } }() wait(t, func() bool { return hub.Online(id.NodeID) }) - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) r := reference() defer func() { ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) diff --git a/services/core/internal/sandbox/node/generation_connection_test.go b/services/core/internal/sandbox/node/generation_connection_test.go index d39b68459..1c3e59d6c 100644 --- a/services/core/internal/sandbox/node/generation_connection_test.go +++ b/services/core/internal/sandbox/node/generation_connection_test.go @@ -63,7 +63,7 @@ func TestGenerationWireRoutesOldOwnershipAndCurrentTargetSeparately(t *testing.T }() wait(t, func() bool { return hub.Online(id.NodeID) }) for _, generation := range []uint64{1, 17, 9} { - proxy := hub.GenerationProvider("docker", docker.Operations(), func(context.Context, sandbox.Reference) (string, uint64, error) { return id.NodeID, generation, nil }) + proxy := hub.GenerationProvider(docker.Operations(), func(context.Context, sandbox.Reference) (string, uint64, error) { return id.NodeID, generation, nil }) ref := reference() if _, err := proxy.GetInfo(ctx, ref); err != nil { t.Fatal("retained generation info failed", generation, err) diff --git a/services/core/internal/sandbox/node/generations.go b/services/core/internal/sandbox/node/generations.go index 95613b674..047bbe4e1 100644 --- a/services/core/internal/sandbox/node/generations.go +++ b/services/core/internal/sandbox/node/generations.go @@ -15,7 +15,10 @@ type GenerationProvider struct { SpecificationDigest string Provider sandbox.SandboxProvider Probe func(context.Context) error - Close func() + // Quiescent reports that no helper outlived its canceled caller; nil + // means always quiescent. + Quiescent func() bool + Close func() } type GenerationManagerOptions struct { @@ -217,11 +220,7 @@ func (m *GenerationManager) Drop(ctx context.Context, grant sandbox.GenerationRe m.mu.Unlock() return sandbox.ErrOwnership } - quiet := true - if provider, ok := g.value.Provider.(interface{ Quiescent() bool }); ok { - quiet = provider.Quiescent() - } - if !quiet || g.refs != 0 || g.removing || m.target.Generation == grant.Generation || m.target.ServingGeneration != nil && *m.target.ServingGeneration == grant.Generation { + if g.value.Quiescent != nil && !g.value.Quiescent() || g.refs != 0 || g.removing || m.target.Generation == grant.Generation || m.target.ServingGeneration != nil && *m.target.ServingGeneration == grant.Generation { m.mu.Unlock() return ErrUnavailable } @@ -265,7 +264,7 @@ func (m *GenerationManager) prepareLoop() { if candidate == nil || candidate.removing || candidate.collecting || candidate.preparing || candidate.refs != 0 || candidate.value.Provider != nil && !candidate.repairing || time.Now().Before(candidate.retryAt) { continue } - if provider, ok := candidate.value.Provider.(interface{ Quiescent() bool }); ok && !provider.Quiescent() { + if candidate.value.Quiescent != nil && !candidate.value.Quiescent() { continue } g = candidate diff --git a/services/core/internal/sandbox/node/generations_test.go b/services/core/internal/sandbox/node/generations_test.go index 9292bbbb2..8831d5321 100644 --- a/services/core/internal/sandbox/node/generations_test.go +++ b/services/core/internal/sandbox/node/generations_test.go @@ -116,13 +116,6 @@ func TestRetentionReplyMustMatchWholePendingExchange(t *testing.T) { } } -type helperOwnedProvider struct { - *fakeProvider - caller *microsandbox.ProcessCaller -} - -func (p *helperOwnedProvider) Quiescent() bool { return p.caller.Quiescent() } - func TestGrantedDropWaitsForReferencesAndActualHelperExit(t *testing.T) { root := t.TempDir() helper := filepath.Join(root, "helper") @@ -142,7 +135,7 @@ func TestGrantedDropWaitsForReferencesAndActualHelperExit(t *testing.T) { defer func() { _ = os.WriteFile(release, nil, 0600); wait(t, caller.Quiescent) }() m := generationFixture(3) m.target.ServingGeneration = nil - m.values[1].value.Provider = &helperOwnedProvider{fakeProvider: &fakeProvider{}, caller: caller} + m.values[1].value.Provider, m.values[1].value.Quiescent = &fakeProvider{}, caller.Quiescent m.options.Remove = func(context.Context, GenerationProvider) error { return os.Remove(artifact) } _, _, unref, err := m.Acquire(1) if err != nil { diff --git a/services/core/internal/sandbox/node/hub_lifetime_test.go b/services/core/internal/sandbox/node/hub_lifetime_test.go index ed5d0d379..37b07c74b 100644 --- a/services/core/internal/sandbox/node/hub_lifetime_test.go +++ b/services/core/internal/sandbox/node/hub_lifetime_test.go @@ -46,7 +46,7 @@ func assertInfoResponsive(t *testing.T, hub *Hub, id Identity) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() - info, err := hub.Proxy(id.NodeID, id.Provider, docker.Operations(), 1).GetInfo(ctx, reference()) + info, err := hub.Proxy(id.NodeID, docker.Operations(), 1).GetInfo(ctx, reference()) if err != nil || info.ProviderID != "retained" { t.Fatalf("unrelated node RPC blocked: info=%+v err=%v", info, err) } @@ -287,7 +287,7 @@ func TestHubSendQueueRespectsCallerCancellation(t *testing.T) { defer cancel() done := make(chan error, 1) go func() { - _, err := hub.Proxy(id.NodeID, id.Provider, docker.Operations(), 1).GetInfo(ctx, reference()) + _, err := hub.Proxy(id.NodeID, docker.Operations(), 1).GetInfo(ctx, reference()) done <- err }() select { diff --git a/services/core/internal/sandbox/node/node_test.go b/services/core/internal/sandbox/node/node_test.go index aab23036f..9aca8cd5a 100644 --- a/services/core/internal/sandbox/node/node_test.go +++ b/services/core/internal/sandbox/node/node_test.go @@ -125,7 +125,7 @@ func TestLostCreateResponseDoesNotReplayAndReconnectSerializesCleanup(t *testing t.Fatal("running node did not retain lifetime identity lock") } r := reference() - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) createCtx, stopCreate := context.WithTimeout(ctx, 150*time.Millisecond) defer stopCreate() createDone := make(chan error, 1) @@ -168,7 +168,7 @@ func TestLostCreateResponseDoesNotReplayAndReconnectSerializesCleanup(t *testing func TestOfflineIsUnknownAndDockerDoesNotAdvertiseCheckpoint(t *testing.T) { h := NewHub(HubOptions{OwnerEpoch: func(context.Context) (uint64, error) { return 1, nil }}) - p := h.Proxy(uuid.NewString(), "docker", docker.Operations(), 1) + p := h.Proxy(uuid.NewString(), docker.Operations(), 1) if sandbox.SupportsCheckpoint(p) { t.Fatal("docker advertised checkpoint") } diff --git a/services/core/internal/sandbox/node/observations.go b/services/core/internal/sandbox/node/observations.go index 515606640..7129d2f33 100644 --- a/services/core/internal/sandbox/node/observations.go +++ b/services/core/internal/sandbox/node/observations.go @@ -3,16 +3,11 @@ package node import ( "context" "errors" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) -var _ runtimeobs.Source = (*provider)(nil) - -func (p *provider) ObservationProviderType() string { return p.kind } - func observationReference(target runtimeobs.Target) sandbox.Reference { return sandbox.Reference{TenantID: target.TenantID, EnvironmentID: target.EnvironmentID, AllocationID: target.Instance.AllocationID} } @@ -39,18 +34,3 @@ func (p *provider) Observe(ctx context.Context, target runtimeobs.Target) (runti } return *out.Sample, nil } - -func observeProvider(ctx context.Context, provider sandbox.SandboxProvider, target runtimeobs.Target) (runtimeobs.Sample, error) { - if err := providercontract.Require(provider, "Observe"); err != nil { - return runtimeobs.Sample{}, err - } - source, ok := provider.(runtimeobs.Source) - if !ok { - return runtimeobs.Sample{}, providercontract.ErrContract - } - return source.Observe(ctx, target) -} - -func (p *provider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/sandbox/node/observations_test.go b/services/core/internal/sandbox/node/observations_test.go index f9b7e1656..23a5cbfdb 100644 --- a/services/core/internal/sandbox/node/observations_test.go +++ b/services/core/internal/sandbox/node/observations_test.go @@ -89,15 +89,12 @@ func TestObservationsRouteThroughAssignedNodeWithoutLifecycleCalls(t *testing.T) stopSecond := runObservationNode(t, hub, server.URL, second, b) ra, rb := reference(), reference() assignments := map[sandbox.Reference]string{ra: first.NodeID, rb: second.NodeID} - source := hub.GenerationProvider("docker", docker.Operations(), func(_ context.Context, r sandbox.Reference) (string, uint64, error) { + source := hub.GenerationProvider(docker.Operations(), func(_ context.Context, r sandbox.Reference) (string, uint64, error) { if id, ok := assignments[r]; ok { return id, 1, nil } return "", 0, sandbox.ErrOwnership - }).(runtimeobs.Source) - if typed := source.ObservationProviderType(); typed != "docker" { - t.Fatal("provider type lost", typed) - } + }) for _, test := range []struct { ref sandbox.Reference provider *observationProvider diff --git a/services/core/internal/sandbox/node/operations_test.go b/services/core/internal/sandbox/node/operations_test.go index 2cd469fda..e508988b5 100644 --- a/services/core/internal/sandbox/node/operations_test.go +++ b/services/core/internal/sandbox/node/operations_test.go @@ -37,7 +37,7 @@ func TestUnsupportedWireIsExplicitAndDoesNotInvokeProvider(t *testing.T) { } } func TestUnsupportedProxyRejectsBeforeNodeResolution(t *testing.T) { - p := (&Hub{}).GenerationProvider("docker", docker.Operations(), func(context.Context, sandbox.Reference) (string, uint64, error) { + p := (&Hub{}).GenerationProvider(docker.Operations(), func(context.Context, sandbox.Reference) (string, uint64, error) { t.Fatal("unsupported call resolved a node") return "", 0, nil }).(*provider) @@ -47,9 +47,6 @@ func TestUnsupportedProxyRejectsBeforeNodeResolution(t *testing.T) { if _, err := p.Initial(t.Context(), reference()); !errors.Is(err, providercontract.ErrUnsupported) { t.Fatal(err) } - if _, err := p.ObserveBatch(t.Context(), nil); !errors.Is(err, providercontract.ErrUnsupported) { - t.Fatal(err) - } } func TestNodeOperationMappingCoversForwardedMethods(t *testing.T) { for _, method := range []string{"Create", "GetInfo", "Renew", "Kill", "RunCommand", "Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "RunCommandCompute", "ResumeCompute", "Observe"} { diff --git a/services/core/internal/sandbox/node/provider_operations_fixture_test.go b/services/core/internal/sandbox/node/provider_operations_fixture_test.go index 2569a6632..3f892e9bb 100644 --- a/services/core/internal/sandbox/node/provider_operations_fixture_test.go +++ b/services/core/internal/sandbox/node/provider_operations_fixture_test.go @@ -10,26 +10,21 @@ import ( func (*fakeProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*fakeProvider) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -62,46 +57,22 @@ func (*fakeProvider) ResumeCompute(context.Context, sandbox.Reference, sandbox.C func (*fakeProvider) Observe(context.Context, runtimeobs.Target) (runtimeobs.Sample, error) { return runtimeobs.Sample{}, &providercontract.UnsupportedError{Operation: "Observe", Reason: "fixture_operation_not_supported"} } -func (*fakeProvider) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_operation_not_supported"} -} -func (*fakeProvider) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "fixture_operation_not_supported"} -} -func (*fakeProvider) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "fixture_operation_not_supported"} -} func (*observationProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Supported}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Supported}, } } - -func (*fakeProvider) ObservationProviderType() string { return "fixture" } -func (p *fakeProvider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} - -func (*observationProvider) ObservationProviderType() string { return "fixture" } -func (p *observationProvider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/internal/sandbox/node/proxy.go b/services/core/internal/sandbox/node/proxy.go index 01daa2aa1..fc126ebe8 100644 --- a/services/core/internal/sandbox/node/proxy.go +++ b/services/core/internal/sandbox/node/proxy.go @@ -6,23 +6,20 @@ import ( "maps" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) type provider struct { hub *Hub resolveGeneration func(context.Context, sandbox.Reference) (string, uint64, error) - kind string operations providercontract.Operations } var _ sandbox.SandboxProvider = (*provider)(nil) -var _ sandbox.CheckpointProvider = (*provider)(nil) // Proxy binds a fixed node and deployment generation explicitly. -func (h *Hub) Proxy(id, kind string, declared providercontract.Operations, generation uint64) sandbox.SandboxProvider { - return h.GenerationProvider(kind, declared, func(context.Context, sandbox.Reference) (string, uint64, error) { return id, generation, nil }) +func (h *Hub) Proxy(id string, declared providercontract.Operations, generation uint64) sandbox.SandboxProvider { + return h.GenerationProvider(declared, func(context.Context, sandbox.Reference) (string, uint64, error) { return id, generation, nil }) } func (p *provider) call(ctx context.Context, q request) (response, error) { if err := providercontract.Require(p, operationMethod(q.Operation)); err != nil { @@ -72,15 +69,6 @@ func creationSettled(info *sandbox.Info, ref sandbox.Reference) bool { func (p *provider) ProviderOperations() providercontract.Operations { return maps.Clone(p.operations) } -func (*provider) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "node_transport_has_no_batch_observation"} -} -func (p *provider) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "node_configuration_is_core_owned"} -} -func (p *provider) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "node_credentials_are_transport_owned"} -} func (p *provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Info, error) { return p.info(ctx, request{Operation: "create", Reference: b.Reference, Bootstrap: &b}) } @@ -163,13 +151,7 @@ func (p *provider) ResumeCompute(ctx context.Context, r sandbox.Reference, c san // GenerationProvider routes every operation with allocation-owned generation, // distinct from the request's compute generation. declared is the kind's -// registered operations; the transport replaces the ones it owns. -func (h *Hub) GenerationProvider(kind string, declared providercontract.Operations, resolve func(context.Context, sandbox.Reference) (string, uint64, error)) sandbox.SandboxProvider { - operations := maps.Clone(declared) - if operations != nil { - operations["DiscoverSelection"] = providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_is_core_owned"} - operations["VerifyCredential"] = providercontract.Support{State: providercontract.Unsupported, Reason: "node_credentials_are_transport_owned"} - operations["ObserveBatch"] = providercontract.Support{State: providercontract.Unsupported, Reason: "node_transport_has_no_batch_observation"} - } - return &provider{hub: h, kind: kind, operations: operations, resolveGeneration: resolve} +// registered operations. +func (h *Hub) GenerationProvider(declared providercontract.Operations, resolve func(context.Context, sandbox.Reference) (string, uint64, error)) sandbox.SandboxProvider { + return &provider{hub: h, operations: maps.Clone(declared), resolveGeneration: resolve} } diff --git a/services/core/internal/sandbox/node/recovery_test.go b/services/core/internal/sandbox/node/recovery_test.go index d150b6422..78d106da3 100644 --- a/services/core/internal/sandbox/node/recovery_test.go +++ b/services/core/internal/sandbox/node/recovery_test.go @@ -57,10 +57,10 @@ func TestCoreRestartFencesOldConnectionAndNodeRestartKeepsIdentity(t *testing.T) if first.Online(id.NodeID) { t.Fatal("old owner remained online") } - if _, err = first.Proxy(id.NodeID, "docker", docker.Operations(), 1).GetInfo(context.Background(), reference()); !errors.Is(err, sandbox.ErrComputeUnconfirmed) { + if _, err = first.Proxy(id.NodeID, docker.Operations(), 1).GetInfo(context.Background(), reference()); !errors.Is(err, sandbox.ErrComputeUnconfirmed) { t.Fatalf("old owner request = %v", err) } - if _, err = second.Proxy(id.NodeID, "docker", docker.Operations(), 1).GetInfo(context.Background(), reference()); err != nil { + if _, err = second.Proxy(id.NodeID, docker.Operations(), 1).GetInfo(context.Background(), reference()); err != nil { t.Fatal(err) } stop() @@ -156,7 +156,7 @@ func TestHeartbeatAcknowledgementKeepsIdleConnectionAlive(t *testing.T) { if !hub.Online(id.NodeID) { t.Fatal("idle node disconnected") } - if _, err = hub.Proxy(id.NodeID, "docker", docker.Operations(), 1).GetInfo(ctx, reference()); err != nil { + if _, err = hub.Proxy(id.NodeID, docker.Operations(), 1).GetInfo(ctx, reference()); err != nil { t.Fatal(err) } } @@ -234,7 +234,7 @@ func TestDegradedNodeRetainsObservationAndCleanup(t *testing.T) { case <-time.After(time.Second): t.Fatal("missing health") } - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) r := reference() if _, err = proxy.Create(ctx, sandbox.Bootstrap{Reference: r}); !errors.Is(err, sandbox.ErrComputeUnconfirmed) { t.Fatalf("create = %v", err) diff --git a/services/core/internal/sandbox/node/timeout_test.go b/services/core/internal/sandbox/node/timeout_test.go index c849b853d..15f8da3c5 100644 --- a/services/core/internal/sandbox/node/timeout_test.go +++ b/services/core/internal/sandbox/node/timeout_test.go @@ -103,7 +103,7 @@ func TestQueuedMutationExpiresWithoutExecution(t *testing.T) { } }() wait(t, func() bool { return hub.Online(id.NodeID) }) - proxy := hub.Proxy(id.NodeID, "docker", docker.Operations(), 1) + proxy := hub.Proxy(id.NodeID, docker.Operations(), 1) r := reference() createCtx, stopCreate := context.WithTimeout(ctx, 3*time.Second) defer stopCreate() diff --git a/services/core/internal/sandbox/node/wire.go b/services/core/internal/sandbox/node/wire.go index 0194e3c82..03a412951 100644 --- a/services/core/internal/sandbox/node/wire.go +++ b/services/core/internal/sandbox/node/wire.go @@ -307,7 +307,7 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response switch q.Operation { case "observe": var sample runtimeobs.Sample - sample, err = observeProvider(ctx, p, *q.Observation) + sample, err = p.Observe(ctx, *q.Observation) out.Sample = &sample case "create": info, err = p.Create(ctx, *q.Bootstrap) @@ -324,38 +324,33 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response command, err = p.RunCommand(ctx, q.Reference, *q.Command) out.Command = &command default: - cp, checkpointErr := sandbox.Checkpoint(p) - if checkpointErr != nil { - err = checkpointErr - break - } var state sandbox.ComputeState var compute sandbox.Compute switch q.Operation { case "initial": - compute, err = cp.Initial(ctx, q.Reference) + compute, err = p.Initial(ctx, q.Reference) out.Compute = &compute case "new_compute": - compute, err = cp.NewCompute(ctx, q.Reference, q.Generation, q.Snapshot) + compute, err = p.NewCompute(ctx, q.Reference, q.Generation, q.Snapshot) out.Compute = &compute case "compute": - state, err = cp.GetCompute(ctx, q.Reference, *q.Compute) + state, err = p.GetCompute(ctx, q.Reference, *q.Compute) out.State = &state case "suspend": - state, err = cp.Suspend(ctx, *q.Suspend) + state, err = p.Suspend(ctx, *q.Suspend) out.State = &state case "resume": - state, err = cp.Resume(ctx, *q.Resume) + state, err = p.Resume(ctx, *q.Resume) out.State = &state case "kill_compute": - err = cp.KillCompute(ctx, q.Reference, *q.Compute) + err = p.KillCompute(ctx, q.Reference, *q.Compute) case "delete_snapshot": - err = cp.DeleteSnapshot(ctx, q.Reference, *q.Snapshot) + err = p.DeleteSnapshot(ctx, q.Reference, *q.Snapshot) case "resume_compute": - state, err = cp.ResumeCompute(ctx, q.Reference, *q.Compute) + state, err = p.ResumeCompute(ctx, q.Reference, *q.Compute) out.State = &state case "command_compute": - command, err = cp.RunCommandCompute(ctx, q.Reference, *q.Compute, *q.Command) + command, err = p.RunCommandCompute(ctx, q.Reference, *q.Compute, *q.Command) out.Command = &command default: err = sandbox.ErrInvalid diff --git a/services/core/internal/sandbox/operations.go b/services/core/internal/sandbox/operations.go deleted file mode 100644 index 3faf4454b..000000000 --- a/services/core/internal/sandbox/operations.go +++ /dev/null @@ -1,88 +0,0 @@ -package sandbox - -import ( - "errors" - "fmt" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" - "reflect" -) - -// These existing interfaces are the canonical operation inventory. Declarations -// must cover every method, including explicit unsupported implementations. -var providerInterfaces = []reflect.Type{ - reflect.TypeFor[SandboxProvider](), reflect.TypeFor[CheckpointProvider](), - reflect.TypeFor[SelectionDiscoverer](), reflect.TypeFor[CredentialVerifier](), - reflect.TypeFor[runtimeobs.SourceResolver](), reflect.TypeFor[runtimeobs.Source](), reflect.TypeFor[runtimeobs.BatchSource](), -} - -func ValidateProvider(p SandboxProvider) error { - if err := providercontract.Validate(p, providerInterfaces...); err != nil { - return err - } - if err := runtimeobs.ValidateSource(p.(runtimeobs.Source)); err != nil { - return err - } - return ValidateOperations(p.ProviderOperations()) -} - -// ValidateOperations rejects omitted, unknown and contradictory declarations. -func ValidateOperations(operations providercontract.Operations) error { - known := map[string]bool{} - for _, contract := range providerInterfaces { - for i := 0; i < contract.NumMethod(); i++ { - name := contract.Method(i).Name - if name != "ProviderOperations" { - known[name] = true - if err := operations[name].Check(name); err != nil && !errors.Is(err, providercontract.ErrUnsupported) { - return err - } - } - } - } - for name := range operations { - if !known[name] { - return fmt.Errorf("%w: unknown operation %s", providercontract.ErrContract, name) - } - } - required := reflect.TypeFor[SandboxProvider]() - for i := 0; i < required.NumMethod(); i++ { - name := required.Method(i).Name - if name != "ProviderOperations" && operations[name].State != providercontract.Supported { - return fmt.Errorf("%w: required operation %s", providercontract.ErrContract, name) - } - } - for _, name := range []string{"ObservationProviderType", "ResolveObservationSource"} { - if operations[name].State != providercontract.Supported { - return fmt.Errorf("%w: required operation %s", providercontract.ErrContract, name) - } - } - // The checkpoint lifecycle is indivisible: partial cleanup or restore support - // cannot safely own a compute incarnation. - checkpoint := reflect.TypeFor[CheckpointProvider]() - for i := 0; i < checkpoint.NumMethod(); i++ { - name := checkpoint.Method(i).Name - if _, required := reflect.TypeFor[SandboxProvider]().MethodByName(name); !required && operations[name].State != operations["Initial"].State { - return fmt.Errorf("%w: incomplete checkpoint lifecycle", providercontract.ErrContract) - } - } - if operations["ObserveBatch"].State == providercontract.Supported && operations["Observe"].State != providercontract.Supported { - return fmt.Errorf("%w: batch observation requires observation", providercontract.ErrContract) - } - return nil -} - -func SupportsCheckpoint(p SandboxProvider) bool { - return providercontract.Require(p, "Initial") == nil -} - -func Checkpoint(p SandboxProvider) (CheckpointProvider, error) { - if err := providercontract.Require(p, "Initial"); err != nil { - return nil, err - } - cp, ok := p.(CheckpointProvider) - if !ok { - return nil, providercontract.ErrContract - } - return cp, nil -} diff --git a/services/core/internal/sandbox/operations_test.go b/services/core/internal/sandbox/operations_test.go index f1cc091d1..62d2c5b3f 100644 --- a/services/core/internal/sandbox/operations_test.go +++ b/services/core/internal/sandbox/operations_test.go @@ -19,23 +19,13 @@ type changedDeclaration struct { func (p *changedDeclaration) ProviderOperations() providercontract.Operations { return p.operations } -type onlyRequired struct{ sandbox.SandboxProvider } - -func (*onlyRequired) ProviderOperations() providercontract.Operations { return docker.Operations() } - func TestDeclarationsRejectMissingUnknownAndContradictoryOperations(t *testing.T) { for _, mutate := range []struct { name string change func(providercontract.Operations) }{ - {"identity unsupported", func(o providercontract.Operations) { - o["ObservationProviderType"] = providercontract.Support{State: providercontract.Unsupported, Reason: "no_identity"} - }}, - {"resolver unsupported", func(o providercontract.Operations) { - o["ResolveObservationSource"] = providercontract.Support{State: providercontract.Unsupported, Reason: "no_resolver"} - }}, {"omitted", func(o providercontract.Operations) { delete(o, "DeleteSnapshot") }}, - {"zero", func(o providercontract.Operations) { o["ObserveBatch"] = providercontract.Support{} }}, + {"zero", func(o providercontract.Operations) { o["Observe"] = providercontract.Support{} }}, {"unknown", func(o providercontract.Operations) { o["FutureOperation"] = providercontract.Support{State: providercontract.Supported} }}, @@ -46,10 +36,10 @@ func TestDeclarationsRejectMissingUnknownAndContradictoryOperations(t *testing.T o["Initial"] = providercontract.Support{State: providercontract.Supported} }}, {"unsafe reason", func(o providercontract.Operations) { - o["ObserveBatch"] = providercontract.Support{State: providercontract.Unsupported, Reason: "https://private:key@host"} + o["Observe"] = providercontract.Support{State: providercontract.Unsupported, Reason: "https://private:key@host"} }}, {"unknown state", func(o providercontract.Operations) { - o["ObserveBatch"] = providercontract.Support{State: "unavailable"} + o["Observe"] = providercontract.Support{State: "unavailable"} }}, } { t.Run(mutate.name, func(t *testing.T) { @@ -60,9 +50,6 @@ func TestDeclarationsRejectMissingUnknownAndContradictoryOperations(t *testing.T } }) } - if err := sandbox.ValidateProvider(&onlyRequired{}); !errors.Is(err, providercontract.ErrContract) { - t.Fatal("missing extension implementations accepted", err) - } var nilProvider *docker.Provider if err := sandbox.ValidateProvider(nilProvider); !errors.Is(err, providercontract.ErrContract) { t.Fatal("typed nil accepted", err) @@ -100,28 +87,3 @@ func TestEveryUnsupportedNativeOperationRejectsWithoutSideEffects(t *testing.T) } } } - -// An extended interface cannot inherit success through the existing declaration. -type nextContract interface { - sandbox.SandboxProvider - NextOperation(context.Context) error -} -type futureProvider struct{ *docker.Provider } - -func (*futureProvider) NextOperation(context.Context) error { return nil } -func TestNewContractRequiresAnAuthoredDecision(t *testing.T) { - for _, p := range []providercontract.Declared{&docker.Provider{}, &futureProvider{&docker.Provider{}}} { - if err := providercontract.Validate(p, reflect.TypeFor[nextContract]()); !errors.Is(err, providercontract.ErrContract) { - t.Fatal("new operation inherited a default", err) - } - } -} - -type invalidObservationIdentity struct{ *docker.Provider } - -func (*invalidObservationIdentity) ObservationProviderType() string { return "" } -func TestProviderRegistrationRequiresObservationIdentity(t *testing.T) { - if err := sandbox.ValidateProvider(&invalidObservationIdentity{&docker.Provider{}}); !errors.Is(err, providercontract.ErrContract) { - t.Fatal("provider registration accepted empty observation identity", err) - } -} diff --git a/services/core/internal/sandbox/providers/config.go b/services/core/internal/sandbox/providers/config.go index 22e789f49..32a3b1f33 100644 --- a/services/core/internal/sandbox/providers/config.go +++ b/services/core/internal/sandbox/providers/config.go @@ -64,6 +64,8 @@ type Built struct { Provider sandbox.SandboxProvider InstallationID, BackendFingerprint string Probe func(context.Context) error + // Quiescent is nil when no helper can outlive its caller. + Quiescent func() bool } // LocalOptions supplies process-local context without changing persisted configuration. diff --git a/services/core/internal/sandbox/providers/configuration.go b/services/core/internal/sandbox/providers/configuration.go index 910a0cb4c..7658c0668 100644 --- a/services/core/internal/sandbox/providers/configuration.go +++ b/services/core/internal/sandbox/providers/configuration.go @@ -3,6 +3,8 @@ package providers import ( "context" "encoding/json" + "errors" + "fmt" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" @@ -106,11 +108,48 @@ func (r *Registry) DiscoverConfiguration(ctx context.Context, kind string, input if err := a.Configuration.Requirements().Discovery.Check("DiscoverConfiguration"); err != nil { return nil, err } - discovery, ok := a.Configuration.(sandbox.ConfigurationDiscoverer) - if !ok { - return nil, providercontract.ErrContract + result, err := a.Configuration.DiscoverConfiguration(ctx, input, paths) + if err != nil { + return nil, declaredResult("DiscoverConfiguration", err) + } + return result, nil +} + +// DiscoverSelection resolves a direct candidate's omitted native values. +func (r *Registry) DiscoverSelection(ctx context.Context, c sandbox.DirectConfig) (sandbox.Selection, error) { + a, err := r.Lookup(c.Selection.Provider) + if err != nil { + return sandbox.Selection{}, err + } + if err := a.Configuration.Requirements().SelectionDiscovery.Check("DiscoverSelection"); err != nil { + return sandbox.Selection{}, err + } + s, err := a.Configuration.DiscoverSelection(ctx, c) + if err != nil { + return sandbox.Selection{}, declaredResult("DiscoverSelection", err) + } + return s, nil +} + +// VerifyCredential checks a candidate credential against owned resources. +func (r *Registry) VerifyCredential(ctx context.Context, c sandbox.DirectConfig, refs []sandbox.Reference) error { + a, err := r.Lookup(c.Selection.Provider) + if err != nil { + return err } - return discovery.DiscoverConfiguration(ctx, input, paths) + if err := a.Configuration.Requirements().CredentialVerification.Check("VerifyCredential"); err != nil { + return err + } + return declaredResult("VerifyCredential", a.Configuration.VerifyCredential(ctx, c, refs)) +} + +// declaredResult keeps a supported declaration binding: an operation declared +// Supported that reports Unsupported breaks the contract. +func declaredResult(operation string, err error) error { + if errors.Is(err, providercontract.ErrUnsupported) { + return fmt.Errorf("%w: %s is declared supported", providercontract.ErrContract, operation) + } + return err } type nodeConfiguration struct{} @@ -123,7 +162,10 @@ type nodeConfigurationAdapter struct { } func (nodeConfigurationAdapter) Requirements() sandbox.ConfigurationRequirements { - return sandbox.ConfigurationRequirements{Credential: sandbox.NotRequired, PublicOrigin: sandbox.NotRequired, Discovery: providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_has_no_catalog"}} + return sandbox.ConfigurationRequirements{Credential: sandbox.NotRequired, PublicOrigin: sandbox.NotRequired, + Discovery: providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_has_no_catalog"}, + SelectionDiscovery: providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_has_no_catalog"}, + CredentialVerification: providercontract.Support{State: providercontract.Unsupported, Reason: "credentials_not_required"}} } func (nodeConfigurationAdapter) DecodeInput(public, secret json.RawMessage) (sandbox.Configuration, error) { if len(secret) > 0 || sandbox.DecodeConfigurationObject(public, &struct{}{}) != nil { @@ -170,3 +212,9 @@ func (a nodeConfigurationAdapter) Equal(x, y sandbox.Configuration) (bool, error func (nodeConfigurationAdapter) DiscoverConfiguration(context.Context, sandbox.ConfigurationDiscoveryInput, sandbox.ProcessPaths) (json.RawMessage, error) { return nil, &providercontract.UnsupportedError{Operation: "DiscoverConfiguration", Reason: "node_configuration_has_no_catalog"} } +func (nodeConfigurationAdapter) DiscoverSelection(context.Context, sandbox.DirectConfig) (sandbox.Selection, error) { + return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "node_configuration_has_no_catalog"} +} +func (nodeConfigurationAdapter) VerifyCredential(context.Context, sandbox.DirectConfig, []sandbox.Reference) error { + return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "credentials_not_required"} +} diff --git a/services/core/internal/sandbox/providers/configuration_flow_test.go b/services/core/internal/sandbox/providers/configuration_flow_test.go index b61b73c5a..12b338067 100644 --- a/services/core/internal/sandbox/providers/configuration_flow_test.go +++ b/services/core/internal/sandbox/providers/configuration_flow_test.go @@ -31,7 +31,8 @@ func (regionalConfiguration) ReplacesCredential() bool { return false } type regionalCodec struct{} func (regionalCodec) Requirements() sandbox.ConfigurationRequirements { - return sandbox.ConfigurationRequirements{Credential: sandbox.NotRequired, PublicOrigin: sandbox.NotRequired, Discovery: providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_has_no_catalog"}} + unsupported := providercontract.Support{State: providercontract.Unsupported, Reason: "node_configuration_has_no_catalog"} + return sandbox.ConfigurationRequirements{Credential: sandbox.NotRequired, PublicOrigin: sandbox.NotRequired, Discovery: unsupported, SelectionDiscovery: unsupported, CredentialVerification: unsupported} } func (regionalCodec) WithCredential(sandbox.Configuration, sandbox.Configuration) (sandbox.Configuration, error) { return nil, &providercontract.UnsupportedError{Operation: "WithCredential", Reason: "credentials_not_required"} @@ -75,6 +76,12 @@ func (a regionalCodec) Equal(x, y sandbox.Configuration) (bool, error) { func (regionalCodec) DiscoverConfiguration(context.Context, sandbox.ConfigurationDiscoveryInput, sandbox.ProcessPaths) (json.RawMessage, error) { return nil, &providercontract.UnsupportedError{Operation: "DiscoverConfiguration", Reason: "node_configuration_has_no_catalog"} } +func (regionalCodec) DiscoverSelection(context.Context, sandbox.DirectConfig) (sandbox.Selection, error) { + return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "node_configuration_has_no_catalog"} +} +func (regionalCodec) VerifyCredential(context.Context, sandbox.DirectConfig, []sandbox.Reference) error { + return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "node_configuration_has_no_catalog"} +} // A registered native configuration reaches the ordinary API and Store without // adding its fields or kind to either Core package. diff --git a/services/core/internal/sandbox/providers/configuration_test.go b/services/core/internal/sandbox/providers/configuration_test.go index 43588e61a..cfabbadeb 100644 --- a/services/core/internal/sandbox/providers/configuration_test.go +++ b/services/core/internal/sandbox/providers/configuration_test.go @@ -25,11 +25,7 @@ func TestNodeConfigurationExplicitUnsupportedAndStrictEmptyInput(t *testing.T) { t.Fatal(kind, "accepted credential", err) } } - discover, ok := a.Configuration.(sandbox.ConfigurationDiscoverer) - if !ok { - t.Fatal("missing explicit discovery implementation") - } - if _, err := discover.DiscoverConfiguration(t.Context(), sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}); !errors.Is(err, providercontract.ErrUnsupported) { + if _, err := a.Configuration.DiscoverConfiguration(t.Context(), sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}); !errors.Is(err, providercontract.ErrUnsupported) { t.Fatal("discovery did not reject", err) } if _, err := a.Configuration.WithCredential(nil, nil); !errors.Is(err, providercontract.ErrUnsupported) { diff --git a/services/core/internal/sandbox/providers/e2b.go b/services/core/internal/sandbox/providers/e2b.go deleted file mode 100644 index bb3861f7f..000000000 --- a/services/core/internal/sandbox/providers/e2b.go +++ /dev/null @@ -1,56 +0,0 @@ -package providers - -import ( - "errors" - - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/e2b" -) - -type DirectConfig struct { - ProcessPaths sandbox.ProcessPaths - InstallationID string - Selection sandbox.Selection - Fence *sandbox.CallFence -} - -func (r *Registry) BuildDirect(c DirectConfig) (sandbox.SandboxProvider, error) { - a, e := r.Lookup(c.Selection.Provider) - if e != nil { - return nil, e - } - if a.BuildDirect == nil { - return nil, sandbox.ErrInvalid - } - p, err := a.BuildDirect(c) - if err != nil { - return nil, err - } - if err := ValidateBinding(a, p); err != nil { - return nil, err - } - return p, nil -} -func buildE2B(c DirectConfig) (sandbox.SandboxProvider, error) { - configuration, ok := c.Selection.Configuration.(*e2b.DeploymentConfiguration) - if !ok || configuration == nil { - return nil, errors.New("E2B deployment configuration is unavailable") - } - binary, state, err := e2b.InstalledPaths(c.ProcessPaths) - if err != nil { - return nil, err - } - // Only a candidate that omitted its resources has none; its validation - // reads them from the template build before the candidate is rebuilt. - var resources *sandbox.Resources - if c.Selection.DeploymentSpec.Resources != (sandbox.Resources{}) { - resources = &c.Selection.DeploymentSpec.Resources - } - provider, err := e2b.NewWithCaller(e2b.Config{Binary: binary, StateDir: state, - Resources: resources, InstallationID: c.InstallationID, APIKey: configuration.APIKey, Template: configuration.Template, - APIURL: configuration.APIURL, Domain: configuration.Domain, TimeoutSeconds: 3600}, &e2b.ProcessCaller{Fence: c.Fence}) - if err != nil { - return nil, errors.New("E2B provider cannot load; check the installed helper and private state directory") - } - return provider, nil -} diff --git a/services/core/internal/sandbox/providers/generation_test.go b/services/core/internal/sandbox/providers/generation_test.go index 58f961693..e9b1c957c 100644 --- a/services/core/internal/sandbox/providers/generation_test.go +++ b/services/core/internal/sandbox/providers/generation_test.go @@ -81,6 +81,9 @@ func TestMicrosandboxGenerationBindsLeaseIdentity(t *testing.T) { t.Fatal(err) } defer closeProvider() + if built.Quiescent == nil || !built.Quiescent() { + t.Fatal("generation does not report helper quiescence") + } response, err := json.Marshal(sandboxmicro.Response{Version: sandboxmicro.ProtocolVersion}) if err != nil { t.Fatal(err) diff --git a/services/core/internal/sandbox/providers/microsandbox.go b/services/core/internal/sandbox/providers/microsandbox.go index 06dc7c3a4..58fbcb8cf 100644 --- a/services/core/internal/sandbox/providers/microsandbox.go +++ b/services/core/internal/sandbox/providers/microsandbox.go @@ -59,6 +59,7 @@ func configureMicrosandbox(entry Microsandbox, resources sandbox.Resources, call } result.Provider = provider result.Probe = microsandboxProbe(entry, resources) + result.Quiescent = caller.Quiescent result.BackendFingerprint = BackendFingerprint("microsandbox", entry.RuntimeHome) return nil } diff --git a/services/core/internal/sandbox/providers/registration_configuration.go b/services/core/internal/sandbox/providers/registration_configuration.go index 3d8a5dd54..07ba5d285 100644 --- a/services/core/internal/sandbox/providers/registration_configuration.go +++ b/services/core/internal/sandbox/providers/registration_configuration.go @@ -24,32 +24,13 @@ func validateConfigurationAdapter(configuration sandbox.ConfigurationAdapter) er return configurationRegistrationError() } } - discovery := reflect.TypeFor[sandbox.ConfigurationDiscoverer]() - if !value.Type().Implements(discovery) { - return configurationRegistrationError() - } - if err := validateConfigurationDiscoveryInterface(discovery); err != nil { - return err - } return validateConfigurationRequirements(reflect.ValueOf(configuration.Requirements())) } -// Every discovery method needs its own authored requirement. A future method -// cannot inherit the existing Discovery decision merely by being implemented. -func validateConfigurationDiscoveryInterface(discovery reflect.Type) error { - if discovery.NumMethod() != 1 { - return configurationRegistrationError() - } - if _, exists := discovery.MethodByName("DiscoverConfiguration"); !exists { - return configurationRegistrationError() - } - return nil -} - // Check field names as well as values so new requirements cannot bypass the // gate. This owns only configuration requirements, not resource operations. func validateConfigurationRequirements(value reflect.Value) error { - if value.Kind() != reflect.Struct || value.NumField() != 3 { + if value.Kind() != reflect.Struct || value.NumField() != 5 { return configurationRegistrationError() } for i := 0; i < value.NumField(); i++ { @@ -59,12 +40,12 @@ func validateConfigurationRequirements(value reflect.Value) error { if !ok || (requirement != sandbox.Required && requirement != sandbox.NotRequired) { return configurationRegistrationError() } - case "Discovery": + case "Discovery", "SelectionDiscovery", "CredentialVerification": support, ok := value.Field(i).Interface().(providercontract.Support) if !ok { return configurationRegistrationError() } - if err := support.Check("DiscoverConfiguration"); err != nil && !errors.Is(err, providercontract.ErrUnsupported) { + if err := support.Check(value.Type().Field(i).Name); err != nil && !errors.Is(err, providercontract.ErrUnsupported) { return configurationRegistrationError() } default: diff --git a/services/core/internal/sandbox/providers/registration_configuration_test.go b/services/core/internal/sandbox/providers/registration_configuration_test.go index 7fdcda590..4a456d31e 100644 --- a/services/core/internal/sandbox/providers/registration_configuration_test.go +++ b/services/core/internal/sandbox/providers/registration_configuration_test.go @@ -2,6 +2,7 @@ package providers import ( "context" + "encoding/json" "errors" "reflect" "testing" @@ -14,7 +15,6 @@ import ( // call through the nil embedded interfaces fails the test immediately. type registrationConfiguration struct { sandbox.ConfigurationAdapter - sandbox.ConfigurationDiscoverer requirements sandbox.ConfigurationRequirements } @@ -22,17 +22,11 @@ func (a registrationConfiguration) Requirements() sandbox.ConfigurationRequireme return a.requirements } -// A declaration of Unsupported still requires an explicit rejection method. -type missingConfigurationDiscovery struct{ sandbox.ConfigurationAdapter } - -func TestConfigurationRegistrationRejectsNilAndMissingDiscovery(t *testing.T) { +func TestConfigurationRegistrationRejectsNil(t *testing.T) { registry := Builtin() a := registry.adapters["docker"] var typedNil *registrationConfiguration - for _, configuration := range []sandbox.ConfigurationAdapter{ - nil, typedNil, missingConfigurationDiscovery{a.Configuration}, - missingConfigurationDiscovery{registry.adapters["e2b"].Configuration}, - } { + for _, configuration := range []sandbox.ConfigurationAdapter{nil, typedNil} { a.Configuration = configuration if err := ValidateRegistration(a); !errors.Is(err, providercontract.ErrContract) { t.Fatalf("%T: %v", configuration, err) @@ -104,12 +98,7 @@ func TestConfigurationRequirementsDoNotInventDependencies(t *testing.T) { } } -type futureConfigurationDiscovery interface { - sandbox.ConfigurationDiscoverer - NextDiscovery(context.Context) error -} - -func TestFutureConfigurationRequirementAndMethodNeedExplicitHandling(t *testing.T) { +func TestFutureConfigurationRequirementNeedsExplicitHandling(t *testing.T) { registry := Builtin() original := registry.adapters["docker"].Configuration.Requirements() fields := make([]reflect.StructField, 0, 4) @@ -126,32 +115,85 @@ func TestFutureConfigurationRequirementAndMethodNeedExplicitHandling(t *testing. if err := validateConfigurationRequirements(value); !errors.Is(err, providercontract.ErrContract) { t.Fatal("new requirement silently inherited policy", err) } - if err := validateConfigurationDiscoveryInterface(reflect.TypeFor[futureConfigurationDiscovery]()); !errors.Is(err, providercontract.ErrContract) { - t.Fatal("new discovery method inherited support", err) - } } -func TestUnsupportedConfigurationDiscoveryMatchesAuthoredReason(t *testing.T) { +func TestUnsupportedSetupOperationsMatchAuthoredReasons(t *testing.T) { registry := Builtin() for kind, a := range registry.adapters { - support := a.Configuration.Requirements().Discovery - if support.State != providercontract.Unsupported { - continue + requirements := a.Configuration.Requirements() + direct := sandbox.DirectConfig{Selection: sandbox.Selection{Provider: kind}} + discover := func(read func() (json.RawMessage, error)) func() error { + return func() error { + if result, err := read(); result != nil { + return errors.New("fabricated catalog") + } else { + return err + } + } } - native := a.Configuration.(sandbox.ConfigurationDiscoverer) - for _, read := range []func() ([]byte, error){ - func() ([]byte, error) { - return native.DiscoverConfiguration(t.Context(), sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}) - }, - func() ([]byte, error) { - return registry.DiscoverConfiguration(t.Context(), kind, sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}) - }, + for _, operation := range []struct { + name string + support providercontract.Support + calls []func() error + }{ + {"DiscoverConfiguration", requirements.Discovery, []func() error{ + discover(func() (json.RawMessage, error) { + return a.Configuration.DiscoverConfiguration(t.Context(), sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}) + }), + discover(func() (json.RawMessage, error) { + return registry.DiscoverConfiguration(t.Context(), kind, sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}) + }), + }}, + {"DiscoverSelection", requirements.SelectionDiscovery, []func() error{ + func() error { _, err := a.Configuration.DiscoverSelection(t.Context(), direct); return err }, + func() error { _, err := registry.DiscoverSelection(t.Context(), direct); return err }, + }}, + {"VerifyCredential", requirements.CredentialVerification, []func() error{ + func() error { return a.Configuration.VerifyCredential(t.Context(), direct, nil) }, + func() error { return registry.VerifyCredential(t.Context(), direct, nil) }, + }}, } { - result, err := read() - reason, valid := providercontract.UnsupportedReason(err, "DiscoverConfiguration") - if result != nil || !valid || reason != support.Reason { - t.Fatalf("%s: result=%v reason=%s err=%v", kind, result, reason, err) + if operation.support.State != providercontract.Unsupported { + continue + } + for _, call := range operation.calls { + err := call() + if reason, valid := providercontract.UnsupportedReason(err, operation.name); !valid || reason != operation.support.Reason { + t.Fatalf("%s %s: reason=%s err=%v", kind, operation.name, reason, err) + } } } } } + +// contradictingConfiguration declares every setup operation Supported and then +// reports each as Unsupported. +type contradictingConfiguration struct{ registrationConfiguration } + +func (contradictingConfiguration) DiscoverConfiguration(context.Context, sandbox.ConfigurationDiscoveryInput, sandbox.ProcessPaths) (json.RawMessage, error) { + return nil, &providercontract.UnsupportedError{Operation: "DiscoverConfiguration", Reason: "not_ready"} +} +func (contradictingConfiguration) DiscoverSelection(context.Context, sandbox.DirectConfig) (sandbox.Selection, error) { + return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "not_ready"} +} +func (contradictingConfiguration) VerifyCredential(context.Context, sandbox.DirectConfig, []sandbox.Reference) error { + return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "not_ready"} +} + +func TestSupportedSetupOperationsCannotReportUnsupported(t *testing.T) { + registry := Builtin() + a := registry.adapters["docker"] + requirements := a.Configuration.Requirements() + supported := providercontract.Support{State: providercontract.Supported} + requirements.Discovery, requirements.SelectionDiscovery, requirements.CredentialVerification = supported, supported, supported + a.Configuration = contradictingConfiguration{registrationConfiguration{requirements: requirements}} + registry.adapters["docker"] = a + direct := sandbox.DirectConfig{Selection: sandbox.Selection{Provider: "docker"}} + _, discoverErr := registry.DiscoverConfiguration(t.Context(), "docker", sandbox.ConfigurationDiscoveryInput{}, sandbox.ProcessPaths{}) + _, selectionErr := registry.DiscoverSelection(t.Context(), direct) + for _, err := range []error{discoverErr, selectionErr, registry.VerifyCredential(t.Context(), direct, nil)} { + if !errors.Is(err, providercontract.ErrContract) || errors.Is(err, providercontract.ErrUnsupported) { + t.Fatal(err) + } + } +} diff --git a/services/core/internal/sandbox/providers/registration_test.go b/services/core/internal/sandbox/providers/registration_test.go index 93eb5304b..f30c39072 100644 --- a/services/core/internal/sandbox/providers/registration_test.go +++ b/services/core/internal/sandbox/providers/registration_test.go @@ -38,7 +38,7 @@ func TestRegistrationRejectsBeforeCallbacksOrConstruction(t *testing.T) { {"wrong local constructor", func(a *Adapter) { a.Mode = "direct" }}, {"missing direct constructor", func(a *Adapter) { a.Mode = "direct"; a.BuildLocal = nil }}, {"both constructors", func(a *Adapter) { - a.BuildDirect = func(DirectConfig) (sandbox.SandboxProvider, error) { + a.BuildDirect = func(sandbox.DirectConfig) (sandbox.SandboxProvider, error) { t.Fatal("called direct constructor") return nil, nil } @@ -47,7 +47,6 @@ func TestRegistrationRejectsBeforeCallbacksOrConstruction(t *testing.T) { {"missing resource validator", func(a *Adapter) { a.ValidateResources = nil }}, {"missing configuration", func(a *Adapter) { a.Configuration = nil }}, {"typed nil configuration", func(a *Adapter) { var c *registrationConfiguration; a.Configuration = c }}, - {"missing discovery implementation", func(a *Adapter) { a.Configuration = missingConfigurationDiscovery{a.Configuration} }}, {"missing configuration requirement", func(a *Adapter) { a.Configuration = registrationConfiguration{} }}, {"invalid credential requirement", func(a *Adapter) { r := a.Configuration.Requirements() @@ -100,7 +99,7 @@ func TestRegistrationRejectsBeforeCallbacksOrConstruction(t *testing.T) { _, _, err := registry.Build(Config{Provider: kind, Generation: 1, InstallationID: uuid.NewString(), Specification: selection.DeploymentSpec}, LocalOptions{Standalone: true}) return err }}, - {"direct build", func() error { _, err := registry.BuildDirect(DirectConfig{Selection: selection}); return err }}, + {"direct build", func() error { _, err := registry.BuildDirect(sandbox.DirectConfig{Selection: selection}); return err }}, {"binding", func() error { return ValidateBinding(a, &docker.Provider{}) }}, {"projection", func() error { text, err := registry.PythonDeploymentContract() @@ -152,12 +151,12 @@ func TestCompleteRegistrationsPreserveConstruction(t *testing.T) { // Direct providers may legitimately need no remote credential or extra // selection state; registration must not require irrelevant callback stubs. a.Mode, a.BuildLocal, a.NodeArtifacts = "direct", nil, nil - a.BuildDirect = func(DirectConfig) (sandbox.SandboxProvider, error) { + a.BuildDirect = func(sandbox.DirectConfig) (sandbox.SandboxProvider, error) { calls++ return &docker.Provider{}, nil } registry.adapters[kind] = a - p, err := registry.BuildDirect(DirectConfig{Selection: sandbox.Selection{Provider: kind}}) + p, err := registry.BuildDirect(sandbox.DirectConfig{Selection: sandbox.Selection{Provider: kind}}) if err != nil || p == nil || calls != 2 { t.Fatalf("credential-free direct build: %v calls=%d", err, calls) } diff --git a/services/core/internal/sandbox/providers/registry.go b/services/core/internal/sandbox/providers/registry.go index ab92077b5..29bb86b4f 100644 --- a/services/core/internal/sandbox/providers/registry.go +++ b/services/core/internal/sandbox/providers/registry.go @@ -21,7 +21,7 @@ type Adapter struct { Policy sandbox.DeploymentPolicy Configuration sandbox.ConfigurationAdapter BuildLocal func(Config, LocalOptions, *Built) (func(), error) - BuildDirect func(DirectConfig) (sandbox.SandboxProvider, error) + BuildDirect func(sandbox.DirectConfig) (sandbox.SandboxProvider, error) Mode string Operations func() providercontract.Operations IdleSeconds, RetentionSeconds int64 @@ -56,7 +56,7 @@ func Builtin() *Registry { Configuration: nodeConfigurationAdapter{microsandbox.ValidateSpecification}, }, "e2b": { - Policy: e2b.Policy(), Operations: e2b.Operations, Mode: "direct", BuildDirect: buildE2B, + Policy: e2b.Policy(), Operations: e2b.Operations, Mode: "direct", BuildDirect: e2b.BuildDirect, Configuration: e2b.ConfigurationAdapter{}, ValidateSpecification: e2b.ValidateSpecification, ValidateResources: e2b.ValidateResources, }, @@ -75,6 +75,25 @@ func (r *Registry) Lookup(kind string) (Adapter, error) { return a, nil } +// BuildDirect builds a direct-mode Provider and validates its binding. +func (r *Registry) BuildDirect(c sandbox.DirectConfig) (sandbox.SandboxProvider, error) { + a, e := r.Lookup(c.Selection.Provider) + if e != nil { + return nil, e + } + if a.BuildDirect == nil { + return nil, sandbox.ErrInvalid + } + p, err := a.BuildDirect(c) + if err != nil { + return nil, err + } + if err := ValidateBinding(a, p); err != nil { + return nil, err + } + return p, nil +} + // SupportsCheckpoint reports whether the provider declares checkpoint suspension. func (r *Registry) SupportsCheckpoint(kind string) (bool, error) { a, err := r.Lookup(kind) diff --git a/services/core/internal/sandbox/sandbox_provider.go b/services/core/internal/sandbox/sandbox_provider.go index 663526b8e..aac367557 100644 --- a/services/core/internal/sandbox/sandbox_provider.go +++ b/services/core/internal/sandbox/sandbox_provider.go @@ -1,27 +1,36 @@ // Package sandbox defines the Sandbox Provider contract for authorized compute. -// Start at sandbox_provider.go and docs/sandbox-provider.md when adding an adapter. +// This file is the Core–Sandbox Provider protocol; docs/sandbox-provider.md +// describes it. Start here when adding an adapter. Value types that Core also +// uses beyond this boundary, such as DeploymentSpec and CallFence, live in their +// own files of this package. // Core owns durable Environment/allocation state and lifecycle serialization; // SandboxProvider owns compute and bootstrap, Runtime owns capability preparation, // and Harness adapters own native execution. Compute running is not execution ready. // -// Required operations are on SandboxProvider. CheckpointProvider and runtimeobs -// observation remain separate small interfaces. Every registered adapter explicitly -// declares and implements each operation, including safe Unsupported rejections. -// Method-set presence never means an extension is supported. ValidateProvider and -// the common contract tests check declaration completeness and implementation. +// The protocol has two interfaces. SandboxProvider is the allocation-time half +// and ConfigurationAdapter the setup-time half. Every adapter implements every +// method of both and declares which operations it supports: SandboxProvider +// through ProviderOperations, ConfigurationAdapter through Requirements. An +// unsupported method returns a typed providercontract.UnsupportedError; method-set +// presence never means support. // -// Registration is explicit construction, not a global init-time registry. Node-local -// adapters register in sandbox/providers; Core's managed setup -// constructs direct adapters or node proxies. execution.RuntimeProvider binds the -// selected adapter to installation, backend, deployment generation and node identity. -// Keep vendor configuration at those construction boundaries; common lifecycle code -// selects behavior through these contracts, never through a vendor name. +// Registration is explicit construction, not a global init-time registry. Adapters +// register in sandbox/providers; Core's managed setup constructs direct adapters +// or node proxies. execution.RuntimeProvider binds the selected adapter to +// installation, backend, deployment generation and node identity. Keep vendor +// configuration at those construction boundaries; common lifecycle code selects +// behavior through these contracts, never through a vendor name. package sandbox import ( "context" + "encoding/json" "errors" + "fmt" + "reflect" + "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" + "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" ) var ( @@ -30,6 +39,9 @@ var ( ErrExists = errors.New("sandbox allocation already exists") ErrNotFound = errors.New("sandbox allocation not found") ErrCommandUnconfirmed = errors.New("initialization command outcome unconfirmed; reclaim allocation before reuse") + // ErrComputeUnconfirmed requires observation of the retained operation identity; + // it does not authorize another Create, capture, restore, or cold start. + ErrComputeUnconfirmed = errors.New("sandbox lifecycle outcome unconfirmed") ) // Reference must be persisted by the caller before Create. AllocationID is a fresh @@ -90,12 +102,9 @@ type SandboxProvider interface { // idempotent, but nil alone cannot settle an outstanding Create. Kill(context.Context, Reference) error RunCommand(context.Context, Reference, Command) (CommandResult, error) -} -// CheckpointProvider is an explicitly declared extension. It supplies -// exact-incarnation operations; Worker and Store remain the lifecycle owner. -type CheckpointProvider interface { - SandboxProvider + // The checkpoint lifecycle supplies exact-incarnation operations; Worker and + // Store remain the lifecycle owner. Its operations share one declaration. Initial(context.Context, Reference) (Compute, error) NewCompute(context.Context, Reference, uint64, *SnapshotIdentity) (Compute, error) GetCompute(context.Context, Reference, Compute) (ComputeState, error) @@ -106,6 +115,111 @@ type CheckpointProvider interface { RunCommandCompute(context.Context, Reference, Compute, Command) (CommandResult, error) // ResumeCompute thaws only the same resident instance after an aborted pause. ResumeCompute(context.Context, Reference, Compute) (ComputeState, error) + + // Observe reads one owned Runtime instance after verifying its ownership. It + // never renews, restarts or stops compute. + Observe(context.Context, runtimeobs.Target) (runtimeobs.Sample, error) +} + +// requiredOperations are supported by every Provider. +var requiredOperations = []string{"Create", "GetInfo", "Renew", "Kill", "RunCommand"} + +// checkpointOperations are all supported or all unsupported: partial cleanup or +// restore support cannot safely own a compute incarnation. +var checkpointOperations = []string{"Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "RunCommandCompute", "ResumeCompute"} + +// Compute identifies one incarnation of an allocation. Name is provider-derived. +// ID is empty only until the original create or restore result is observed. +type Compute struct { + Generation uint64 + Name string + ID string + RestoredFrom *SnapshotIdentity +} + +// SnapshotIdentity is provider evidence from a verified full snapshot. Core +// persists it unchanged and records consumption separately; it never invents +// paths, checksums, native checkpoint fields, or source identity. +type SnapshotIdentity struct { + Reference string + ID string + Digest string + CheckpointID string + CheckpointRoot string + OperationID string + SourceGeneration uint64 + SourceName string + SourceID string +} + +type ComputeState struct { + Compute Compute + Status string + BootstrapComplete bool + Snapshot *SnapshotIdentity + SourceStopped bool +} +type SuspendRequest struct { + Reference Reference + OperationID string + Source Compute + Snapshot *SnapshotIdentity + // Recovery observes the previous attempt and never starts a new capture. + ObserveOnly bool +} +type ResumeRequest struct { + Reference Reference + OperationID string + Snapshot SnapshotIdentity + Target Compute + // Recovery observes the previous target and never starts a new restore. + ObserveOnly bool +} + +// ValidateProvider checks a constructed Provider's declaration. +func ValidateProvider(p SandboxProvider) error { + if p == nil { + return providercontract.ErrContract + } + if value := reflect.ValueOf(p); value.Kind() == reflect.Pointer && value.IsNil() { + return providercontract.ErrContract + } + return ValidateOperations(p.ProviderOperations()) +} + +// ValidateOperations rejects omitted, unknown and contradictory declarations. +// SandboxProvider's method set is the operation inventory. +func ValidateOperations(operations providercontract.Operations) error { + contract := reflect.TypeFor[SandboxProvider]() + for i := range contract.NumMethod() { + name := contract.Method(i).Name + if name == "ProviderOperations" { + continue + } + if err := operations[name].Check(name); err != nil && !errors.Is(err, providercontract.ErrUnsupported) { + return err + } + } + for name := range operations { + if _, known := contract.MethodByName(name); !known || name == "ProviderOperations" { + return fmt.Errorf("%w: unknown operation %s", providercontract.ErrContract, name) + } + } + for _, name := range requiredOperations { + if operations[name].State != providercontract.Supported { + return fmt.Errorf("%w: required operation %s", providercontract.ErrContract, name) + } + } + for _, name := range checkpointOperations { + if operations[name].State != operations["Initial"].State { + return fmt.Errorf("%w: incomplete checkpoint lifecycle", providercontract.ErrContract) + } + } + return nil +} + +func SupportsCheckpoint(p SandboxProvider) bool { + return providercontract.Require(p, "Initial") == nil } // ProcessPaths locates installed adapter helpers and their private state. The @@ -114,3 +228,123 @@ type ProcessPaths struct { ArtifactRoot string StateRoot string } + +// Requirement has no implicit default: registration must choose either value. +type Requirement string + +const ( + Required Requirement = "required" + NotRequired Requirement = "not_required" +) + +// ConfigurationRequirements is the setup-time declaration. Discovery, +// SelectionDiscovery and CredentialVerification declare DiscoverConfiguration, +// DiscoverSelection and VerifyCredential; requiring a credential does not +// promise VerifyCredential. +type ConfigurationRequirements struct { + Credential Requirement + PublicOrigin Requirement + Discovery providercontract.Support + SelectionDiscovery providercontract.Support + CredentialVerification providercontract.Support +} + +// Configuration is an adapter-owned typed value, never a request or response DTO. +// Implementations must exclude secrets from JSON and safe diagnostic output. +type Configuration interface { + HasCredential() bool + ReplacesCredential() bool +} + +// ConfigurationRecord separates public selectors, read-only observations and +// secret bytes. Store encrypts Secret with the installation and generation. +// Only adapter codecs may produce Public and Metadata; neither is input passthrough. +type ConfigurationRecord struct { + Public json.RawMessage + Metadata json.RawMessage + Secret []byte `json:"-"` +} + +// Selection is the typed deployment configuration shared by preview and commit. +type Selection struct { + DeploymentSpec + ExpectedGeneration uint64 `json:"expected_generation"` + Provider string `json:"provider"` + Configuration Configuration `json:"-"` +} + +func (s Selection) HasCredential() bool { + return s.Configuration != nil && s.Configuration.HasCredential() +} + +func (s Selection) ReplacesCredential() bool { + return s.Configuration != nil && s.Configuration.ReplacesCredential() +} + +// DirectConfig constructs a direct adapter for one selection. Fence counts the +// adapter's helper processes so that a credential commit waits for them. +type DirectConfig struct { + ProcessPaths ProcessPaths + InstallationID string + Selection Selection + Fence *CallFence +} + +// ConfigurationDiscoveryInput is a transient read-only request. Query is typed +// and validated by the adapter; it cannot select a compute mutation. +type ConfigurationDiscoveryInput struct { + Configuration json.RawMessage `json:"configuration" swaggertype:"object"` + Credential json.RawMessage `json:"credential" swaggertype:"object"` + Query json.RawMessage `json:"query" swaggertype:"object"` +} + +// ConfigurationAdapter owns all interpretation of provider configuration. +// Decode loads retained ownership without remote discovery or new-build admission. +// Normalize validates a candidate; ResolveChange first applies omitted-field +// inheritance, then normalizes. Equal compares normalized identity, excluding +// discovery metadata and explicit credential-submission intent. +// +// The three setup operations are read-only native calls, separate from compute +// and candidate admission. Each builds its own native client from its input. +type ConfigurationAdapter interface { + Requirements() ConfigurationRequirements + DecodeInput(public, credential json.RawMessage) (Configuration, error) + Encode(Configuration) (ConfigurationRecord, error) + Decode(ConfigurationRecord) (Configuration, error) + Normalize(Selection) (Selection, error) + ResolveChange(next, previous Selection) (Selection, error) + WithCredential(owner, candidate Configuration) (Configuration, error) + Equal(a, b Configuration) (bool, error) + // DiscoverConfiguration lists the native catalog a credential can use, + // without a saved deployment. + DiscoverConfiguration(context.Context, ConfigurationDiscoveryInput, ProcessPaths) (json.RawMessage, error) + // DiscoverSelection resolves a candidate's omitted native values before + // commit. Loading retained ownership never calls it. + DiscoverSelection(context.Context, DirectConfig) (Selection, error) + // VerifyCredential verifies access to already owned resources without + // mutation. Replacing a credential requires it and a shared CallFence. + VerifyCredential(context.Context, DirectConfig, []Reference) error +} + +// ConfigurationError is a fixed safe diagnostic, never SDK text or submitted data. +// Class describes the request outcome; it does not authorize replay. +type ConfigurationError struct { + Class ConfigurationErrorClass + Code, Param, Message string +} +type ConfigurationErrorClass string + +const ( + ConfigurationInvalid ConfigurationErrorClass = "invalid" + ConfigurationConflict ConfigurationErrorClass = "conflict" + ConfigurationUnconfirmed ConfigurationErrorClass = "unconfirmed" +) + +func (e *ConfigurationError) Error() string { return e.Message } + +var ( + ErrCredentialRejected = &ConfigurationError{ConfigurationInvalid, "sandbox_credential_invalid", "credential", "The sandbox provider credential was rejected."} + ErrCredentialOwnership = &ConfigurationError{ConfigurationConflict, "sandbox_credential_ownership", "credential", "The credential cannot manage the retained deployment. Reset before changing accounts."} + ErrConfigurationUnconfirmed = &ConfigurationError{ConfigurationUnconfirmed, "sandbox_verification_unconfirmed", "", "Sandbox provider verification could not be confirmed."} + ErrConfigurationSelection = &ConfigurationError{ConfigurationInvalid, "sandbox_configuration_invalid", "configuration", "Select a ready immutable provider configuration with matching resources."} +) diff --git a/services/core/internal/sandbox/sandbox_provider_test.go b/services/core/internal/sandbox/sandbox_provider_test.go new file mode 100644 index 000000000..56c23b330 --- /dev/null +++ b/services/core/internal/sandbox/sandbox_provider_test.go @@ -0,0 +1,24 @@ +package sandbox + +import ( + "reflect" + "slices" + "testing" +) + +// The operation groups partition SandboxProvider's operations, so a new method +// cannot escape the required or all-or-nothing checkpoint checks. +func TestOperationGroupsPartitionTheInterface(t *testing.T) { + contract := reflect.TypeFor[SandboxProvider]() + var methods []string + for i := range contract.NumMethod() { + if name := contract.Method(i).Name; name != "ProviderOperations" { + methods = append(methods, name) + } + } + groups := slices.Concat(requiredOperations, checkpointOperations, []string{"Observe"}) + slices.Sort(groups) + if !slices.Equal(methods, groups) { + t.Fatalf("SandboxProvider operations %v, groups %v", methods, groups) + } +} diff --git a/services/core/internal/sandbox/selection.go b/services/core/internal/sandbox/selection.go deleted file mode 100644 index 65fede7d2..000000000 --- a/services/core/internal/sandbox/selection.go +++ /dev/null @@ -1,31 +0,0 @@ -package sandbox - -import "context" - -// Selection is the typed deployment configuration shared by preview and commit. -type Selection struct { - DeploymentSpec - ExpectedGeneration uint64 `json:"expected_generation"` - Provider string `json:"provider"` - Configuration Configuration `json:"-"` -} - -func (s Selection) HasCredential() bool { - return s.Configuration != nil && s.Configuration.HasCredential() -} - -func (s Selection) ReplacesCredential() bool { - return s.Configuration != nil && s.Configuration.ReplacesCredential() -} - -// SelectionDiscoverer is an optional read-only native configuration capability. -// It resolves candidate configuration; loading retained ownership never calls it. -type SelectionDiscoverer interface { - DiscoverSelection(context.Context, Selection) (Selection, error) -} - -// CredentialVerifier verifies access to already owned resources without mutation. -// Replacing credentials requires this capability and a shared CallFence. -type CredentialVerifier interface { - VerifyCredential(context.Context, []Reference) error -} diff --git a/services/core/internal/sandbox/suspension.go b/services/core/internal/sandbox/suspension.go deleted file mode 100644 index 5a600a877..000000000 --- a/services/core/internal/sandbox/suspension.go +++ /dev/null @@ -1,57 +0,0 @@ -package sandbox - -import ( - "errors" -) - -// ErrComputeUnconfirmed requires observation of the retained operation identity; -// it does not authorize another Create, capture, restore, or cold start. -var ErrComputeUnconfirmed = errors.New("sandbox lifecycle outcome unconfirmed") - -// Compute identifies one incarnation of an allocation. Name is provider-derived. -// ID is empty only until the original create or restore result is observed. -type Compute struct { - Generation uint64 - Name string - ID string - RestoredFrom *SnapshotIdentity -} - -// SnapshotIdentity is provider evidence from a verified full snapshot. Core -// persists it unchanged and records consumption separately; it never invents -// paths, checksums, native checkpoint fields, or source identity. -type SnapshotIdentity struct { - Reference string - ID string - Digest string - CheckpointID string - CheckpointRoot string - OperationID string - SourceGeneration uint64 - SourceName string - SourceID string -} - -type ComputeState struct { - Compute Compute - Status string - BootstrapComplete bool - Snapshot *SnapshotIdentity - SourceStopped bool -} -type SuspendRequest struct { - Reference Reference - OperationID string - Source Compute - Snapshot *SnapshotIdentity - // Recovery observes the previous attempt and never starts a new capture. - ObserveOnly bool -} -type ResumeRequest struct { - Reference Reference - OperationID string - Snapshot SnapshotIdentity - Target Compute - // Recovery observes the previous target and never starts a new restore. - ObserveOnly bool -} diff --git a/services/core/tests/integration/provider_operations_fixture_test.go b/services/core/tests/integration/provider_operations_fixture_test.go index 383fde08d..f528d5ae0 100644 --- a/services/core/tests/integration/provider_operations_fixture_test.go +++ b/services/core/tests/integration/provider_operations_fixture_test.go @@ -10,26 +10,21 @@ import ( func (*lifecycleProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*lifecycleProvider) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -62,46 +57,22 @@ func (*lifecycleProvider) ResumeCompute(context.Context, sandbox.Reference, sand func (*lifecycleProvider) Observe(context.Context, runtimeobs.Target) (runtimeobs.Sample, error) { return runtimeobs.Sample{}, &providercontract.UnsupportedError{Operation: "Observe", Reason: "fixture_operation_not_supported"} } -func (*lifecycleProvider) ObserveBatch(context.Context, []runtimeobs.Target) ([]runtimeobs.BatchResult, error) { - return nil, &providercontract.UnsupportedError{Operation: "ObserveBatch", Reason: "fixture_operation_not_supported"} -} -func (*lifecycleProvider) DiscoverSelection(context.Context, sandbox.Selection) (sandbox.Selection, error) { - return sandbox.Selection{}, &providercontract.UnsupportedError{Operation: "DiscoverSelection", Reason: "fixture_operation_not_supported"} -} -func (*lifecycleProvider) VerifyCredential(context.Context, []sandbox.Reference) error { - return &providercontract.UnsupportedError{Operation: "VerifyCredential", Reason: "fixture_operation_not_supported"} -} func (*fakeCheckpointProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Supported}, - "NewCompute": {State: providercontract.Supported}, - "GetCompute": {State: providercontract.Supported}, - "Suspend": {State: providercontract.Supported}, - "Resume": {State: providercontract.Supported}, - "KillCompute": {State: providercontract.Supported}, - "DeleteSnapshot": {State: providercontract.Supported}, - "RunCommandCompute": {State: providercontract.Supported}, - "ResumeCompute": {State: providercontract.Supported}, - "ObservationProviderType": {State: providercontract.Supported}, - "ResolveObservationSource": {State: providercontract.Supported}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ObserveBatch": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DiscoverSelection": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "VerifyCredential": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "RunCommand": {State: providercontract.Supported}, + "Initial": {State: providercontract.Supported}, + "NewCompute": {State: providercontract.Supported}, + "GetCompute": {State: providercontract.Supported}, + "Suspend": {State: providercontract.Supported}, + "Resume": {State: providercontract.Supported}, + "KillCompute": {State: providercontract.Supported}, + "DeleteSnapshot": {State: providercontract.Supported}, + "RunCommandCompute": {State: providercontract.Supported}, + "ResumeCompute": {State: providercontract.Supported}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } - -func (*lifecycleProvider) ObservationProviderType() string { return "fixture" } -func (p *lifecycleProvider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} - -func (*fakeCheckpointProvider) ObservationProviderType() string { return "fixture" } -func (p *fakeCheckpointProvider) ResolveObservationSource(context.Context) (runtimeobs.Source, error) { - return p, nil -} diff --git a/services/core/tools/e2b-provider/README.md b/services/core/tools/e2b-provider/README.md index 34bb368cb..1e7588c76 100644 --- a/services/core/tools/e2b-provider/README.md +++ b/services/core/tools/e2b-provider/README.md @@ -19,14 +19,14 @@ The account key is stored encrypted in Core's database and is write-only. It rea | `command` | `RunCommand` | Runs one bounded command as the Runtime user on a running sandbox whose bootstrap completed; output is limited to 1 MiB per stream | | `validate_deployment` | Deployment setup | Reads the template's builds and requires the exact build to be ready with the configured CPU and memory. Without configured resources the selection adopts the build's CPU and memory. Returns the build's status, CPU, memory and reported disk for Core to record; bounded to 30 seconds | | `list_templates`, `list_builds` | [Configuration discovery](../../../../contracts/agents-api/sandbox-deployment.md#configuration-discovery) | Pages the key's visible templates (`GET /v2/templates`) or one template's ready builds, with a transient key. Results are capped at 200 and write no receipt | -| `observe` | Runtime observations | Up to 100 allocations; see [Observations](#observations) | +| `observe` | Runtime observations | One allocation; see [Observations](#observations) | | `verify_credential` | E2B key replacement | Up to 32 allocation references; see [Credential verification](#credential-verification) | Compatible endpoints must return the SDK 2.51.0 template-list and template-build response models; the helper does not adapt other catalog shapes. E2B has no independently configurable disk limit, and sandbox inspection does not expose a build ID: build provenance comes from the validated create selector. ## Private JSON boundary -[`helper_contract.go`](../../internal/sandbox/e2b/helper_contract.go) owns the adapter-private wire types, version, operation and error vocabulary, and bounds. Its generator projects Python declarations into the helper and template sources, deriving managed-bootstrap fields from the Sandbox Provider types, network access values from `agentnetwork.Policy.Validate`, and the SDK version from the hashed dependency lock. The command-input and observation limits come from their shared Go contracts. The generated modules have no SDK or repository dependency and ship with the frozen helper and protected template startup scripts. +[`helper_contract.go`](../../internal/sandbox/e2b/helper_contract.go) owns the adapter-private wire types, version, operation and error vocabulary, and bounds. Its generator projects Python declarations into the helper and template sources, deriving managed-bootstrap fields from the Sandbox Provider types, network access values from `agentnetwork.Policy.Validate`, and the SDK version from the hashed dependency lock. The command-input limit comes from its shared Go contract. The generated modules have no SDK or repository dependency and ship with the frozen helper and protected template startup scripts. Run `go generate ./services/core/internal/sandbox/e2b` from the repository root after changing these declarations. `make check-e2b-provider` and the Go adapter tests reject stale projections; both languages consume generated valid and invalid exchanges covering wire types, extra fields, operation/reference bounds and managed-bootstrap fields. The helper build copies those fixtures with its source before running the pinned-SDK suite. @@ -56,7 +56,7 @@ Inspection uses SDK metadata and ID reads only. SDK `connect` is never used beca ## Observations -`observe` reads each allocation's sandbox ID from its receipt without taking the allocation lock. It then runs one `GET /sandboxes/metrics` request and one labelled listing of the installation's running sandboxes concurrently, within the caller's deadline; the listing stops once every requested sandbox has appeared. Only a sandbox that the listing confirms for exactly that allocation is reported, with the listing's start time. A malformed metrics point makes only its row unavailable. Observation never connects to, renews or changes a sandbox and writes no receipt. [Runtime observability](../../../../contracts/agents-api/runtime-observability.md) owns the field mapping. +`observe` reads the allocation's sandbox ID from its receipt without taking the allocation lock. It then runs one `GET /sandboxes/metrics` request for that sandbox and one listing of running sandboxes with the allocation's labels concurrently, within the caller's deadline; the listing stops once the sandbox has appeared. Only a sandbox that the listing confirms for exactly that allocation is reported, with the listing's start time. A malformed metrics point is unavailable. Observation never connects to, renews or changes a sandbox and writes no receipt. [Runtime observability](../../../../contracts/agents-api/runtime-observability.md) owns the field mapping. ## Credential verification diff --git a/services/core/tools/e2b-provider/deployment_test.py b/services/core/tools/e2b-provider/deployment_test.py index 4008a4616..27601bbf8 100644 --- a/services/core/tools/e2b-provider/deployment_test.py +++ b/services/core/tools/e2b-provider/deployment_test.py @@ -66,7 +66,7 @@ def test_custom_endpoint_reaches_metrics_client(self, metrics, client): self.config.update(APIURL='https://sandbox-test.sandbase.ai', Domain='sandbox-test.sandbase.ai') metrics.return_value = SimpleNamespace(status_code=503, parsed=None) with self.assertRaises(Failure): - read_metrics(self.config, ['owned-id'], lambda: 5) + read_metrics(self.config, 'owned-id', lambda: 5) configuration = client.call_args.args[0] self.assertEqual((configuration.api_url, configuration.domain), (self.config['APIURL'], self.config['Domain'])) diff --git a/services/core/tools/e2b-provider/helper_contract_generated.py b/services/core/tools/e2b-provider/helper_contract_generated.py index ebd7fadf2..04d6c13b3 100644 --- a/services/core/tools/e2b-provider/helper_contract_generated.py +++ b/services/core/tools/e2b-provider/helper_contract_generated.py @@ -5,7 +5,6 @@ MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","InstallationID"] MAX_COMMAND_INPUT = 52428832 MAX_CREDENTIAL_REFERENCES = 32 -MAX_OBSERVATION_REFERENCES = 100 MAX_OUTPUT = 1048576 MAX_REQUEST = 75497472 MAX_RESPONSE = 16777216 @@ -14,5 +13,5 @@ PROTOCOL_VERSION = 1 REFERENCE_FIELDS = ["TenantID","EnvironmentID","AllocationID"] REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","RuntimeBootstrap","Command","Deadline"] -RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observations"] +RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] SDK_VERSION = "2.51.0" diff --git a/services/core/tools/e2b-provider/observation_test.py b/services/core/tools/e2b-provider/observation_test.py index eb9c09a89..ef204d6d7 100644 --- a/services/core/tools/e2b-provider/observation_test.py +++ b/services/core/tools/e2b-provider/observation_test.py @@ -1,4 +1,4 @@ -"""Read-only batch observation; live metrics qualification remains separate.""" +"""Read-only observation; live metrics qualification remains separate.""" from datetime import datetime, timezone from types import SimpleNamespace from unittest.mock import patch @@ -10,9 +10,8 @@ class ObservationTest(ProviderTest): - def observe(self, references): - request = dict(self.request, Operation='observe', References=references, - Reference={key: '' for key in self.reference}) + def observe(self, reference=None): + request = dict(self.request, Operation='observe', Reference=reference or self.reference) try: return Provider(request).execute() except Failure as error: @@ -30,51 +29,47 @@ def next_items(**options): self.api.list.return_value = paginator return paginator - def test_one_metrics_request_maps_owned_running_sandboxes(self): + def test_one_metrics_request_maps_the_owned_running_sandbox(self): self.assertEqual(self.call('create')['ErrorCode'], '') self.cloud.started_at = datetime(2026, 9, 25, 10, 31, 34, tzinfo=timezone.utc) other = SimpleNamespace(sandbox_id='other-id', metadata={'oac_installationid': self.config['InstallationID']}) paginator = self.listing([other, self.cloud], [other]) - stopped = {key: str(uuid4()) for key in self.reference} point = {'cpuCount': 2, 'cpuUsedPct': 19.55, 'memUsed': 183836672, 'memTotal': 2079141888, 'memCache': 1, 'diskUsed': 1593188352, 'diskTotal': 23511863296, 'timestamp': '2026-09-25T10:31:36.667115804Z', 'timestampUnix': 1790332296} - with patch('provider.read_metrics', return_value={'owned-id': point}) as metrics, \ + with patch('provider.read_metrics', return_value=point) as metrics, \ patch('provider.Receipt') as receipt: - result = self.observe([self.reference, stopped]) + result = self.observe() self.assertEqual(result['ErrorCode'], '') metrics.assert_called_once() - self.assertEqual(metrics.call_args.args[1], ['owned-id']) + self.assertEqual(metrics.call_args.args[1], 'owned-id') receipt.assert_not_called() self.api.connect.assert_not_called() self.api.set_timeout.assert_not_called() - self.assertEqual(self.api.list.call_args.kwargs['query'].metadata, - {'oac_installationid': self.config['InstallationID']}) - owned, missing = result['Observations'] - self.assertEqual(owned, dict(self.reference, Status='observed', + self.assertEqual(self.api.list.call_args.kwargs['query'].metadata, self.cloud.metadata) + self.assertEqual(result['Observation'], dict(Status='observed', ObservedAt='2026-09-25T10:31:36.667115+00:00', StartedAt='2026-09-25T10:31:34+00:00', CPUCount=2, CPUUsedPct=19.55, MemUsed=183836672, MemTotal=2079141888, DiskUsed=1593188352, DiskTotal=23511863296)) - self.assertEqual(missing, dict(stopped, Status='unavailable')) - self.assertEqual(paginator.reads, 1, 'listing continued after every receipt sandbox was found') + self.assertEqual(paginator.reads, 1, 'listing continued after the receipt sandbox was found') + stopped = {key: str(uuid4()) for key in self.reference} + self.assertEqual(self.observe(stopped)['Observation'], {'Status': 'unavailable'}) del point['diskUsed'] point['memTotal'] = -1 - with patch('provider.read_metrics', return_value={'owned-id': point}): + with patch('provider.read_metrics', return_value=point): self.listing([self.cloud]) - self.assertEqual(self.observe([self.reference])['Observations'], [dict(self.reference, Status='unavailable')]) + self.assertEqual(self.observe()['Observation'], {'Status': 'unavailable'}) point['memTotal'] = 2079141888 - with patch('provider.read_metrics', return_value={'owned-id': point}): + with patch('provider.read_metrics', return_value=point): self.listing([self.cloud]) - row = self.observe([self.reference])['Observations'][0] + row = self.observe()['Observation'] self.assertEqual((row['Status'], row['DiskUsed'], row['DiskTotal']), ('observed', None, None)) - def test_absent_listing_is_not_running_and_rejects_oversized_batches(self): + def test_absent_listing_is_not_running(self): self.assertEqual(self.call('create')['ErrorCode'], '') self.listing([]) - with patch('provider.read_metrics', return_value={}): - result = self.observe([self.reference]) - self.assertEqual(result['Observations'], [dict(self.reference, Status='not_running')]) - references = [{key: str(uuid4()) for key in self.reference} for _ in range(101)] - self.assertEqual(self.observe(references)['ErrorCode'], 'invalid') + with patch('provider.read_metrics', return_value=None): + result = self.observe() + self.assertEqual(result['Observation'], {'Status': 'not_running'}) diff --git a/services/core/tools/e2b-provider/provider.py b/services/core/tools/e2b-provider/provider.py index dbf3111f0..9b2fcafe7 100644 --- a/services/core/tools/e2b-provider/provider.py +++ b/services/core/tools/e2b-provider/provider.py @@ -1,5 +1,5 @@ """Five bounded SDK operations for an already authorized Core allocation, plus -read-only deployment validation and batch observation.""" +read-only deployment validation and observation.""" from concurrent.futures import ThreadPoolExecutor import json import math @@ -14,7 +14,7 @@ from sdk import connection_material, definitely_rejected, list_builds, list_templates, read_metrics, restore, run, sdk_options, validate_deployment, verify_team_template from state import Failure, Receipt, private_root, read_receipt from helper_contract_generated import (PROTOCOL_VERSION, OPERATIONS, REQUEST_FIELDS, REFERENCE_FIELDS, - MAX_OBSERVATION_REFERENCES, MAX_CREDENTIAL_REFERENCES, MANAGED_BOOTSTRAP_FIELDS) + MAX_CREDENTIAL_REFERENCES, MANAGED_BOOTSTRAP_FIELDS) PREFIX = 'oac_' FIELDS = ('InstallationID', *REFERENCE_FIELDS) @@ -38,9 +38,9 @@ def utc(value): return value.astimezone(timezone.utc).isoformat() -def observed(reference, cloud, point): - """Map one metrics point without changing E2B units. A malformed point makes - only its own row unavailable. +def observed(cloud, point): + """Map one metrics point without changing E2B units. A malformed point is + unavailable. Disk metrics need a newer envd; unless E2B reports both integer values and a positive total, disk stays unknown rather than an observed zero.""" @@ -56,11 +56,11 @@ def observed(reference, cloud, point): not math.isfinite(cpu_pct) or cpu_pct < 0 or any(type(v) is not int or v < 0 for v in values) or metric.mem_total < 1): raise ValueError('malformed metrics point') - return dict(reference, Status='observed', ObservedAt=utc(metric.timestamp), StartedAt=utc(cloud.started_at), + return dict(Status='observed', ObservedAt=utc(metric.timestamp), StartedAt=utc(cloud.started_at), CPUCount=cpu_count, CPUUsedPct=cpu_pct, MemUsed=metric.mem_used, MemTotal=metric.mem_total, DiskUsed=metric.disk_used if disk_known else None, DiskTotal=metric.disk_total if disk_known else None) except Exception: - return dict(reference, Status='unavailable') + return {'Status': 'unavailable'} class Provider: @@ -80,12 +80,8 @@ def __init__(self, request): if (type(request['Version']) is not int or request['Version'] != PROTOCOL_VERSION or not isinstance(request['Operation'], str) or request['Operation'] not in OPERATIONS or set(request) - set(REQUEST_FIELDS) or - (request['Operation'] not in ('validate_deployment', 'observe', 'list_templates', 'list_builds', 'verify_credential') and + (request['Operation'] not in ('validate_deployment', 'list_templates', 'list_builds', 'verify_credential') and not valid_reference(self.reference)) or - (request['Operation'] == 'observe' and - (not 1 <= len(self.references) <= MAX_OBSERVATION_REFERENCES or - not all(valid_reference(r) for r in self.references) or - len({tuple(sorted(r.items())) for r in self.references}) != len(self.references))) or (request['Operation'] == 'verify_credential' and (len(self.references) > MAX_CREDENTIAL_REFERENCES or not all(valid_reference(r) for r in self.references))) or @@ -279,66 +275,38 @@ def kill(self): self.receipt.save(status='killed', settled=True, bootstrap_complete=False, connection=None) def observe(self): - """Latest metrics of owned running sandboxes, without locks, writes or connect. + """Latest metrics of the owned running sandbox, without locks, writes or connect. - Receipts name each allocation's sandbox so that the one batch metrics - request runs alongside the labelled listing that confirms it is running.""" - root = private_root(self.config) - statuses, candidates = [], {} - for index, reference in enumerate(self.references): - try: - record = read_receipt(self.config, root, reference) or {} - except Failure as error: - statuses.append(error.code) - continue - except Exception: - statuses.append('unavailable') - continue - ids = record.get('ids', []) - if record.get('status') in ('killed', 'rejected'): - statuses.append('not_running') - elif len(ids) == 1 and isinstance(ids[0], str): - statuses.append('unavailable') - candidates[index] = ids[0] - else: - statuses.append('unavailable') - if not candidates: - return [dict(r, Status=status) for r, status in zip(self.references, statuses)] - installation = self.config['InstallationID'] - # One allocation filters by all its labels; a page lists the installation. - metadata = {PREFIX + 'installationid': installation} - if len(self.references) == 1: - metadata = self.metadata_for(self.references[0]) - wanted, seen = set(candidates.values()), set() + The receipt names the allocation's sandbox so that the metrics request + runs alongside the labelled listing that confirms it is running.""" + try: + record = read_receipt(self.config, private_root(self.config), self.reference) or {} + except Failure as error: + return {'Status': error.code} + except Exception: + return {'Status': 'unavailable'} + ids = record.get('ids', []) + if record.get('status') in ('killed', 'rejected'): + return {'Status': 'not_running'} + if len(ids) != 1 or not isinstance(ids[0], str): + return {'Status': 'unavailable'} + found = [] with ThreadPoolExecutor(max_workers=1) as pool: - metrics = pool.submit(read_metrics, self.config, sorted(wanted), self.remaining) - running = {} - paginator = Sandbox.list(query=SandboxQuery(metadata=metadata, state=[SandboxState.RUNNING]), + metrics = pool.submit(read_metrics, self.config, ids[0], self.remaining) + paginator = Sandbox.list(query=SandboxQuery(metadata=self.metadata, state=[SandboxState.RUNNING]), limit=100, **self.options()) - # Stop once every receipt's sandbox has been listed. Detection of a + # Stop once the receipt's sandbox has been listed. Detection of a # second sandbox with the same allocation labels then covers only # the pages read; lifecycle discovery remains exhaustive. - while paginator.has_next and not wanted <= seen: - for cloud in paginator.next_items(**self.options()): - labels = cloud.metadata or {} - if labels.get(PREFIX + 'installationid') == installation: - key = tuple(labels.get(PREFIX + field.lower()) for field in FIELDS[1:]) - running.setdefault(key, []).append(cloud) - seen.add(cloud.sandbox_id) - points = metrics.result() - result = [] - for index, reference in enumerate(self.references): - if index not in candidates: - result.append(dict(reference, Status=statuses[index])) - continue - found = running.get(tuple(reference[field] for field in FIELDS[1:]), []) - if not found: - result.append(dict(reference, Status='not_running')) - elif len(found) != 1 or found[0].sandbox_id != candidates[index] or not isinstance(points.get(candidates[index]), dict): - result.append(dict(reference, Status='unavailable')) - else: - result.append(observed(reference, found[0], points[candidates[index]])) - return result + while paginator.has_next and not any(cloud.sandbox_id == ids[0] for cloud in found): + found += [cloud for cloud in paginator.next_items(**self.options()) + if all((cloud.metadata or {}).get(k) == v for k, v in self.metadata.items())] + point = metrics.result() + if not found: + return {'Status': 'not_running'} + if len(found) != 1 or found[0].sandbox_id != ids[0] or not isinstance(point, dict): + return {'Status': 'unavailable'} + return observed(found[0], point) def metadata_for(self, reference): return {PREFIX + field.lower(): value for field, value in @@ -397,7 +365,7 @@ def execute(self): self.verify_credential() return {'Version': PROTOCOL_VERSION, 'DeploymentValid': True, 'ErrorCode': ''} if self.q['Operation'] == 'observe': - return {'Version': PROTOCOL_VERSION, 'Observations': self.observe(), 'ErrorCode': ''} + return {'Version': PROTOCOL_VERSION, 'Observation': self.observe(), 'ErrorCode': ''} verify_team_template(self.config, self.remaining) build = validate_deployment(self.config, self.remaining) return {'Version': PROTOCOL_VERSION, 'DeploymentValid': True, 'TemplateBuild': build, 'ErrorCode': ''} diff --git a/services/core/tools/e2b-provider/sdk.py b/services/core/tools/e2b-provider/sdk.py index c58236a2a..4b2d3afb7 100644 --- a/services/core/tools/e2b-provider/sdk.py +++ b/services/core/tools/e2b-provider/sdk.py @@ -18,7 +18,7 @@ from state import Failure -from helper_contract_generated import SDK_VERSION, MAX_OUTPUT, MAX_COMMAND_INPUT, MAX_OBSERVATION_REFERENCES +from helper_contract_generated import SDK_VERSION, MAX_OUTPUT, MAX_COMMAND_INPUT def list_templates(config, remaining): @@ -120,16 +120,14 @@ def validate_deployment(config, remaining): raise Failure('unconfirmed') -def read_metrics(config, sandbox_ids, remaining): - """Latest metrics point per sandbox from one batch request of at most 100 IDs.""" - if not 1 <= len(sandbox_ids) <= MAX_OBSERVATION_REFERENCES: - raise Failure('invalid') +def read_metrics(config, sandbox_id, remaining): + """Latest metrics point of one sandbox, or None when E2B reports none.""" client = get_api_client(ConnectionConfig(**sdk_options(config, remaining))) - response = get_sandboxes_metrics.sync_detailed(client=client, sandbox_ids=sandbox_ids) + response = get_sandboxes_metrics.sync_detailed(client=client, sandbox_ids=[sandbox_id]) if (response.status_code != 200 or not isinstance(response.parsed, SandboxesWithMetrics) or not isinstance(response.parsed.sandboxes, dict)): raise Failure('unconfirmed') - return response.parsed.sandboxes + return response.parsed.sandboxes.get(sandbox_id) def connection_material(sandbox): diff --git a/services/core/tools/e2b-provider/testdata/contract.json b/services/core/tools/e2b-provider/testdata/contract.json index aa79da367..0e2c8e8c6 100644 --- a/services/core/tools/e2b-provider/testdata/contract.json +++ b/services/core/tools/e2b-provider/testdata/contract.json @@ -22,19 +22,12 @@ {"kind":"request","name":"command","payload":{"Version":1,"Operation":"command","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, {"kind":"request","name":"config-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":null,"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, {"kind":"request","name":"create","payload":{"Version":1,"Operation":"create","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"duplicate-observation","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":false}, {"kind":"request","name":"extra","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Extra":true,"Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, {"kind":"request","name":"inspect","payload":{"Version":1,"Operation":"inspect","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, {"kind":"request","name":"kill","payload":{"Version":1,"Operation":"kill","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, {"kind":"request","name":"list_builds","payload":{"Version":1,"Operation":"list_builds","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, {"kind":"request","name":"list_templates","payload":{"Version":1,"Operation":"list_templates","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"observe","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"observe-count-0","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":false}, -{"kind":"request","name":"observe-count-100","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000021-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000022-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000023-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000024-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000025-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000026-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000027-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000028-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000029-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000030-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000031-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000032-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000033-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000034-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000035-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000036-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000037-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000038-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000039-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000040-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000041-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000042-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000043-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000044-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000045-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000046-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000047-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000048-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000049-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000050-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000051-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000052-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000053-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000054-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000055-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000056-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000057-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000058-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000059-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000060-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000061-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000062-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000063-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000064-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"observe-count-101","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000021-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000022-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000023-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000024-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000025-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000026-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000027-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000028-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000029-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000002f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000030-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000031-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000032-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000033-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000034-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000035-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000036-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000037-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000038-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000039-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000003f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000040-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000041-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000042-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000043-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000044-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000045-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000046-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000047-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000048-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000049-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000004f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000050-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000051-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000052-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000053-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000054-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000055-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000056-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000057-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000058-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000059-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000005f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000060-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000061-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000062-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000063-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000064-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000065-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":false}, -{"kind":"request","name":"observe-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"observe","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":{},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"observe-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"observe","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":"invalid","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"observe-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"observe","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":[null],"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, +{"kind":"request","name":"observe","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, {"kind":"request","name":"operation-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":null,"Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, {"kind":"request","name":"operation-unknown","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"unsupported","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, {"kind":"request","name":"reference-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":null,"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, diff --git a/services/core/tools/microsandbox-provider/README.md b/services/core/tools/microsandbox-provider/README.md index 554b4fad8..7235c8274 100644 --- a/services/core/tools/microsandbox-provider/README.md +++ b/services/core/tools/microsandbox-provider/README.md @@ -1,6 +1,6 @@ # microsandbox Sandbox Provider helper -microsandbox runs each hosted Session in its own microVM on a Linux amd64 node with KVM, and it is the Sandbox Provider that supports idle suspension. Core forwards provider operations to the node over the [node protocol](../../../../contracts/agents-api/node-generation-protocol.md); the node's adapter ([`sandbox/microsandbox`](../../internal/sandbox/microsandbox)) runs this helper once per operation. The helper links the microsandbox Go SDK v0.7.2 with its FFI library, so Core and the node program stay CGO-free Go binaries. It implements the provider-neutral `sandbox.CheckpointProvider` with full snapshots. It has no daemon, lifecycle database, scheduler or network control plane. +microsandbox runs each hosted Session in its own microVM on a Linux amd64 node with KVM, and it is the Sandbox Provider that supports idle suspension. Core forwards provider operations to the node over the [node protocol](../../../../contracts/agents-api/node-generation-protocol.md); the node's adapter ([`sandbox/microsandbox`](../../internal/sandbox/microsandbox)) runs this helper once per operation. The helper links the microsandbox Go SDK v0.7.2 with its FFI library, so Core and the node program stay CGO-free Go binaries. It implements the checkpoint operations of the provider-neutral `sandbox.SandboxProvider` with full snapshots. It has no daemon, lifecycle database, scheduler or network control plane. [Add a Sandbox Provider](../../../../docs/sandbox-provider.md) owns the provider contract. [Sandbox deployment](../../../../contracts/agents-api/sandbox-deployment.md) owns the resources, Runtime release and suspension policy; the [nodes guide](../../../../docs/getting-started/nodes.md) owns node installation, host requirements, the node's directories and its network policy.