Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ jobs:
revision="$(git rev-parse HEAD)"
if [[ -n "$REQUESTED_REF" && "$REQUESTED_REF" != "$revision" ]]; then exit 1; fi
echo "revision=$revision" >> "$GITHUB_OUTPUT"
- name: Reserve disk space for distribution archives
run: |
# The hosted runner needs room for Docker images, tar exports and the
# microsandbox import. This job does not use these preinstalled SDKs.
df -h /
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /usr/local/.ghcup
df -h /
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
Expand Down Expand Up @@ -79,8 +86,9 @@ jobs:
CORE_DISTRIBUTION_OFFLINE: ${{ inputs.offline && '1' || '0' }}
run: |
inputs="$HOME/.parsar/build/release-inputs/inputs.json"
export AGENTS_RUNTIME_CODEX_PACKAGE="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["codex"])' "$inputs")"
export MCODE_HARNESS_BUILD_DIR="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["mcode"])' "$inputs")"
AGENTS_RUNTIME_CODEX_PACKAGE="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["codex"])' "$inputs")"
MCODE_HARNESS_BUILD_DIR="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["mcode"])' "$inputs")"
export AGENTS_RUNTIME_CODEX_PACKAGE MCODE_HARNESS_BUILD_DIR
export CORE_DISTRIBUTION_RELEASE_BASE_URL="https://github.com/$RELEASE_REPOSITORY/releases/download/$RELEASE_REVISION"
bash scripts/build-core-distribution.sh
mkdir -p "$HOME/.parsar/build/release-upload"
Expand Down
31 changes: 31 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -1558,6 +1558,17 @@ microsandbox runtime/firmware hashes and executable native payloads. Release gen
qualification. A release must be tested from fresh extraction with real models;
no synthetic result may substitute for native execution acceptance.

Distribution `images` records each exported image's config digest;
`image_manifest_digests` records its OCI manifest/index digest. Derive and verify
both from the same archive, including its referenced config and layer bytes, and
require the build host's selected image ID to match one of them. Docker's classic
store identifies images by config, while its containerd store uses the OCI
descriptor. Core, node and self-hosted installers share one resolver for these
required identities: confirm Linux amd64 and the returned immutable local ID,
then use that ID in service/provider configuration and Runtime launches. Tags do
not replace identity verification. The microsandbox-qualified `runtime_ref`
remains independent of Docker's local store identity.

The manifest is the shared download contract for Core, node and self-hosted
installers: flat versioned filenames, compressed Runtime size/hash and unpacked
size/hash, with HTTPS release URLs or the explicit offline payload. Download into
Expand All @@ -1567,6 +1578,11 @@ execution-only payloads. Python zipapps bundle the shared resolver with each
remote bootstrap; the console publishes only fixed non-secret files and declared
artifact names. Release automation builds artifacts and may create an unpublished
draft, but cannot claim real execution qualification or public availability.
Qualify the exact downloaded production artifacts before publishing the draft;
keep the tested asset bytes and source identity unchanged. Never use an acceptance
image containing a private test CA or model credential as a release input.
Repository visibility is independent of publication. Do not add repository
credentials to installed node/Runtime configuration to bypass download access.

Project-authenticated executor-credential extensions remain outside the upstream
API namespace and reuse the existing restricted issuer. They require the exact
Expand All @@ -1576,6 +1592,21 @@ explicit caller credentials on these routes and never substitutes its administra
key. Self-hosted installation reuses Docker Runtime isolation, owns no sandbox
node or Core allocation, and retains user-owned native history after uncertain
launches. Report started, connected and real execution success separately.
Self-hosted installation confirms connection through the private daemon transport
using only its restricted executor credential. The read checks the exact live
Environment/key binding and current authenticated connection; it never enrolls,
allocates, wakes a sandbox or grants project resource access. Console forwarding
preserves this credential without replacing it with an administrator or project
key. Bounded polling and reruns retain the original container and history;
timeout is a diagnostic failure, not permission to relaunch. An explicit installer
`--public-url` supplies both the console origin and the advertised daemon `wss`
origin. Keep local managed Provider routing separate; do not return an internal
Compose hostname to a user-managed Runtime when an external origin was supplied. Bootstrap routing uses the
node bound to the authenticated device's persisted allocation, never request Host
or caller-supplied placement fields. An embedded managed node retains its internal
Core route; remote managed nodes use the selected setup/public route, while
self-hosted devices retain the deployment's advertised public address. This does
not widen sandbox network policies or change credential admission.

The distribution build sets umask 022 for non-root-readable payloads; installation
credentials and state retain their explicit private permissions.
Expand Down
28 changes: 26 additions & 2 deletions contracts/agents-api/environment-executor-credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,32 @@ history volumes belong to the operator. Failed or uncertain launches retain
their volumes and installation receipt for inspection instead of replacing
history or retrying enrollment. Session deletion does not reclaim these volumes.
Rerunning the installer inspects a previously started container only after its
installation and Environment labels match. It reports running separately from
Session connection, or gives a command to start the same stopped container.
installation and Environment labels match. Both first launch and rerun wait up
to 60 seconds for authenticated Core connection confirmation; a running container
alone does not establish connection. A stopped container receives a command to
start that same container before rerunning the installer.
An uncertain launch without a success receipt gives label-filtered container
and volume inspection commands and never creates a replacement. A cached image
with the exact distribution digest and platform skips image download and import.

## Private connection confirmation

`GET /api/v1/agent-daemon/connection?environment_id=UUID` uses the existing
executor bearer, passed unchanged through the console. It is part of the private
daemon transport, not the public Agents API. It reads existing authorization and
binding only; it never enrolls a device, starts execution or changes resources.
The no-store response contains only the requested `environment_id` and `status`
(`connected` or `disconnected`). Connected requires the existing Environment
observation, its exact Session/device binding, current executor authority and a
live gateway socket authenticated with that same credential. A stale observation
or a socket carrying the former rotated key cannot confirm connection.

Invalid, revoked, foreign or deleted-Session authority returns 401; a different
key for an already bound Environment returns 409. Responses do not expose the
actual binding or database diagnostics. The installer derives this HTTPS route
from the validated returned `remote_url`, rejects redirects, retries transient
read failures within its deadline and polls at two-second intervals. Permanent
rejections fail immediately. On timeout or rejection it retains the container,
volumes, credential and receipts, and prints bounded Docker log inspection and
same-command retry guidance. This confirms authenticated connectivity, not model
credentials, harness capabilities or completed execution.
8 changes: 7 additions & 1 deletion deploy/install/configuration.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Deployment files for the existing Core, Runtime and production console."""
from pathlib import Path
from urllib.parse import urlsplit


def bind(source, target, readonly=True):
Expand Down Expand Up @@ -42,13 +43,18 @@ def core_environment(root, state, database_password):
config = str(Path(root) / "config") if native else "/config"
database = f'127.0.0.1:{state["database_port"]}' if native else "database:5432"
daemon_host = f'host.microsandbox.internal:{state["core_port"]}' if native else "core:8091"
daemon_url = f"ws://{daemon_host}/api/v1/agent-daemon/ws"
if state.get("public_url"):
origin = urlsplit(state["public_url"])
daemon_url = origin._replace(scheme="wss" if origin.scheme == "https" else "ws",
path="/api/v1/agent-daemon/ws").geturl()
result = {
"AGENTS_API_DATABASE_URL": f"postgres://agents_api:{database_password}@{database}/agents_api?sslmode=disable",
"AGENTS_API_KEYS_FILE": config + "/keys.json",
"AGENTS_API_CREDENTIAL_KEY_FILE": config + "/credential.key",
"AGENTS_API_ADDR": f'127.0.0.1:{state["core_port"]}' if native else ":8091",
"AGENTS_API_ENGINE": "codex", "AGENTS_API_HARNESSES": "codex,claude_sdk,mcode",
"AGENTS_API_DAEMON_WS_URL": f"ws://{daemon_host}/api/v1/agent-daemon/ws",
"AGENTS_API_DAEMON_WS_URL": daemon_url,
}
result["AGENTS_API_SANDBOX_ADMIN_DIGESTS_FILE"] = (
str(Path(root) / "admin/digests.json") if native else "/admin/digests.json")
Expand Down
50 changes: 50 additions & 0 deletions deploy/install/distribution.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import re
import socket
import stat
import subprocess
import tempfile
import time
import urllib.error
Expand All @@ -20,6 +21,55 @@ class DistributionError(Exception):
pass


def image_identities(manifest, name):
"""Both immutable IDs describe the same archive, as proven by the builder."""
identities = []
for field in ('images', 'image_manifest_digests'):
mapping = manifest.get(field)
value = mapping.get(name) if isinstance(mapping, dict) else None
if not isinstance(value, str) or not re.fullmatch(r'sha256:[0-9a-f]{64}', value):
raise DistributionError('Missing or invalid immutable image identity: ' + name)
identities.append(value)
return tuple(identities)


def docker_command(arguments, timeout=30):
try:
return subprocess.run(arguments, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, text=True, timeout=timeout, check=False)
except (OSError, subprocess.SubprocessError):
raise DistributionError('Cannot inspect or load the distribution image; check Docker access and disk space') from None


def ensure_docker_image(manifest, name, archive, docker=('docker',)):
"""Resolve a proven local ID; obtain a verified archive only on a cache miss."""
expected = image_identities(manifest, name)
docker = list(docker)

def inspect():
for identity in dict.fromkeys(expected):
result = docker_command(docker + ['image', 'inspect', identity, '--format',
'{{.Id}} {{.Os}}/{{.Architecture}}'])
if result.returncode:
continue
fields = result.stdout.strip().split()
if len(fields) != 2 or fields[0] not in expected or fields[1] != 'linux/amd64':
raise DistributionError('Docker image identity or platform differs from the distribution: ' + name)
return fields[0]
return None

identity = inspect()
if identity is not None:
return identity
result = docker_command(docker + ['load', '--input', str(archive())], timeout=1800)
if result.returncode:
raise DistributionError('Cannot load the distribution image; check Docker access and free disk space: ' + name)
identity = inspect()
if identity is None:
raise DistributionError('Cannot verify the loaded distribution image: ' + name)
return identity


def digest(path):
value = hashlib.sha256()
with Path(path).open('rb') as stream:
Expand Down
47 changes: 33 additions & 14 deletions deploy/install/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

from configuration import compose_config, core_environment, managed_config
import native_service
from distribution import DistributionError, artifact, obtain_artifact, runtime_archive
from distribution import DistributionError, artifact, obtain_artifact, runtime_archive, image_identities, ensure_docker_image


class InstallError(Exception):
Expand Down Expand Up @@ -70,9 +70,8 @@ def verify_bundle(bundle):
if not required.issubset(covered):
raise InstallError("Distribution checksum list is incomplete")
manifest = json.loads((bundle / "manifest.json").read_text())
for image in manifest["images"].values():
if not image.startswith("sha256:") or len(image) != 71:
raise InstallError("Distribution must select immutable images")
for name in ("core", "web", "database", "runtime"):
image_identities(manifest, name)
for name in ("images/runtime.tar.gz", "native/bin/parsar-sandbox-node",
"native/bin/agents-api-microsandbox-provider", "native/microsandbox/msb",
"native/microsandbox/libkrunfw.so.5.6.1"):
Expand Down Expand Up @@ -198,6 +197,15 @@ def initialize(root, args, manifest):
actual = (state["mode"], state["provider"], state["core_port"], state["web_port"], state.get("core_url"), state.get("public_url"))
if wanted != actual or state["source_commit"] != manifest["source_commit"]:
raise InstallError("Existing installation differs; preserve it and follow the upgrade/provider-change guide")
services = json.loads((root / "compose.json").read_text())["services"]
for service, config in services.items():
name = "core" if service == "migrate" else service
if config.get("image") != manifest["images"].get(name):
raise InstallError("Retained Docker image differs; preserve the installation and inspect its configuration")
if state["provider"] == "docker":
managed = json.loads((root / "config/managed-runtimes.json").read_text())
if managed.get("docker", {}).get("image") != manifest["images"]["runtime"]:
raise InstallError("Retained Runtime image differs; preserve the installation and inspect its configuration")
return state
if root.exists() and any(root.iterdir()):
raise InstallError("Installation directory is not empty; refusing to overwrite existing state")
Expand Down Expand Up @@ -327,30 +335,41 @@ def main(argv=None):
manifest = verify_bundle(bundle)
if args.provider and not args.web_only:
print("Preparing the selected local sandbox provider...", flush=True)
runtime_archive(manifest, bundle, bundle)
if args.provider == "microsandbox":
runtime_archive(manifest, bundle, bundle)
for name in ("native/bin/agents-api-microsandbox-provider", "native/microsandbox/msb",
"native/microsandbox/libkrunfw.so.5.6.1"):
obtain_artifact(manifest, name, bundle / name, bundle)
native_service.preflight(bundle)
state = initialize(root, args, manifest)
if args.web_only:
images = ["web"]
elif args.provider == "microsandbox":
images = ["database"]
else:
images = ["core", "database"]
if args.provider == "docker":
images.append("runtime")
if not args.core_only and not args.web_only:
images.append("web")
local_images = dict(manifest["images"])
for name in images:
if name == "runtime":
archive = lambda: runtime_archive(manifest, bundle, bundle)
else:
archive = lambda name=name: bundle / f"images/{name}.tar"
local_images[name] = ensure_docker_image(manifest, name, archive)
# Deployment configuration uses Docker's local IDs; published metadata is unchanged.
deployment = dict(manifest, images=local_images)
state = initialize(root, args, deployment)
prepare_node_payload(root, state, bundle)
if state["provider"] == "docker":
seccomp = bundle / "runtime/seccomp.json"
if not (root / "config/seccomp.json").exists():
private_write(root / "config/seccomp.json", seccomp.read_text())
images = ["web"] if state["mode"] == "web-only" else ["core", "database"]
if state["provider"] == "docker":
images.append("runtime")
if native_service.is_native(state):
images = ["database"]
password = (root / "config/database.password").read_text()
environment = core_environment(root, state, password)
native_service.prepare(root, state, bundle, environment)
if state["mode"] == "all":
images.append("web")
for name in images:
run(["docker", "load", "--input", str(bundle / f"images/{name}.tar")], stdout=subprocess.DEVNULL)
import_runtime(root, state, manifest, bundle)
compose(root, "up", "--detach", "--wait")
if native_service.is_native(state):
Expand Down
Loading
Loading