SUPERSEDED — closed — optics monolith split into #145 #147 #146 #148

.github/workflows/ci.yml

@@ -161,3 +161,35 @@ jobs:
         run: |
           python -m compileall -q vantio_install
           python -m unittest discover -s tests -t . -v
+
+      - name: Test Optics docker observe example
+        run: python -m unittest deploy/docker/test_observe_example.py -v
+
+  test-mcp-otel-pe:
+    name: gate-mcp, optics-mcp, otel-i3, pe-*
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+
+      - name: Setup Node.js 22
+        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
+        with:
+          node-version: "22"
+
+      - name: Setup pnpm
+        uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Test gate-mcp, optics-mcp, otel-i3, and pe packages
+        run: |
+          pnpm --filter @vantio/gate-mcp test
+          pnpm --filter @vantio/optics-mcp test
+          node --test tests/optics-otel-i3/*.test.cjs
+          node --test tests/pe-sequential-authority/*.test.cjs
+          node --test tests/pe-progressive-enforcement/*.test.cjs
+          node --test tests/pe-ingress/*.test.cjs
+          node --test tests/pe-host-authority/*.test.cjs
+          node --test tests/pe-egress/*.test.cjs

README.md

@@ -10,7 +10,7 @@ vantio run node agent.js
 That's the integration. Optics intercepts outbound calls to known LLM providers and reports
 destination, process, size, and timing — never prompts or completions.
 
-Python: `pip install vantio-agent-sdk`, then wrap your agent with `@shield`.
+Python: `pip install vantio-agent-sdk`, then wrap your agent with `@shield`. That PyPI package is the Python SDK. The Node package is `@vantio/agent-sdk`. They are two packages with similar names.
 
 Optics is the free Observe tier. [Phantom Engine](https://vantio.ai/phantom-engine) (Enforce + Control)
 extends this to policy enforcement and host-level runtime protection on enrolled Linux systems. Full docs:
@@ -44,7 +44,8 @@ Vantio records *that* a call was made, *when*, *to which provider*, and *how man
 
 - The content of your prompts
 - Model completions or responses
-- Any personally identifiable information
+
+The stored URL path can contain a secret, token, or identifier if the application put one there. Query strings are dropped. The path is kept.
 
 ---
 

architecture_state.md

@@ -1,5 +1,9 @@
 # Vantio Open-Core — Phase I Architecture Ledger
 
+**Historical.** This file is a build log from Phase I. It is not the current product description, not a packaging source, and not a claim about what is shipped. Current product truth lives in the Optics manuals and the package manifests.
+
+
+
 > **Note (2026):** The Tier 02 control plane (`apps/web` API routes, billing, dashboards) has moved to [`vantio-pro`](https://github.com/vantioai/vantio-pro) and the hosted app (`vantio-app`). This ledger retains historical build logs for open-core packages; references to `apps/web` as the live control plane are archival only.
 
 ## Phase I Checklist

deploy/docker/.dockerignore

@@ -0,0 +1,24 @@
+# Strict context. Only the named observe example files are sent to the daemon.
+*
+!Dockerfile.observe
+!agent.js
+!package.json
+!compose.observe.yml
+
+# Secret and VCS names stay excluded even if a later exception is added.
+**/.env
+**/.env.*
+**/.git
+**/.git/**
+**/.ssh
+**/.ssh/**
+**/*.pem
+**/*.key
+**/id_rsa
+**/id_rsa.pub
+**/*credentials*
+**/secrets
+**/secrets/**
+**/.npmrc
+**/.aws
+**/.aws/**

deploy/docker/Dockerfile.observe

@@ -1,15 +1,14 @@
-# Wrap any Node agent image with Vantio Optics (Sight Loop observe).
-# Build: docker build -f deploy/docker/Dockerfile.observe -t my-agent:optics .
-# Run:   docker run --rm -v vantio-runs:/root/.vantio/runs my-agent:optics
+# Optics observe example. Build context is this directory.
+# The CLI pin is exact and matches packages/vantio-cli. It is not a range.
+# CANDIDATE_ONLY_NOT_FOR_PUBLICATION: this file does not publish the package.
 ARG BASE_IMAGE=node:22-bookworm-slim
 FROM ${BASE_IMAGE}
 
-RUN npm install -g @vantio/cli@^0.3.1
+RUN npm install -g @vantio/cli@0.3.25
 
 WORKDIR /app
-COPY . /app
+COPY package.json agent.js ./
 
-# Default: observe the package start script. Override CMD as needed.
-ENV VANTIO_OBSERVE=1
-ENTRYPOINT ["vantio", "run"]
-CMD ["npm", "start"]
+# `vantio run node` attaches the Node interceptor. `npm` is not a wrapped runtime.
+ENTRYPOINT ["vantio", "run", "node"]
+CMD ["agent.js"]

deploy/docker/agent.js

@@ -0,0 +1,3 @@
+// Started as `vantio run node agent.js`. Optics records supported Node traffic
+// from this process. This example does not call the network.
+console.log("vantio optics observe example: node process started");

deploy/docker/compose.observe.yml

@@ -1,18 +1,14 @@
-# Example: run an agent under Optics and persist local run logs.
+# Example: run a Node process under Optics and persist local run logs.
+# Build context is this directory. It does not send the repository root.
 services:
   agent:
     build:
-      context: ../..
-      dockerfile: deploy/docker/Dockerfile.observe
-      args:
-        BASE_IMAGE: node:22-bookworm-slim
-    environment:
-      - VANTIO_HOOKS=0
+      context: .
+      dockerfile: Dockerfile.observe
     volumes:
       - vantio-runs:/root/.vantio/runs
-    # command: ["node", "agent.js"]
 
-  # Optional: sidecar that tails proofs (placeholder — mount runs volume)
+  # Optional: read the local run logs the agent service wrote.
   prove:
     image: node:22-bookworm-slim
     profiles: ["tools"]
@@ -21,7 +17,7 @@ services:
     working_dir: /root
     entrypoint: ["bash", "-lc"]
     command:
-      - npm install -g @vantio/cli && vantio prove --format=md --list || true
+      - npm install -g @vantio/cli@0.3.25 && vantio prove --format=md --list
 
 volumes:
   vantio-runs:

deploy/docker/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "vantio-optics-observe-example",
+  "private": true,
+  "description": "Tiny Node process for the Optics observe image. Run only under vantio run node.",
+  "scripts": {
+    "start": "node agent.js"
+  }
+}

deploy/docker/test_observe_example.py

@@ -0,0 +1,122 @@
+"""Contract for the Optics observe Docker example.
+
+The example must pin an exact CLI version, keep secrets out of the build
+context, and start Node under ``vantio run`` so observation actually attaches.
+"""
+
+from __future__ import annotations
+
+import re
+import unittest
+from pathlib import Path
+
+DOCKER = Path(__file__).resolve().parent
+DOCKERFILE = DOCKER / "Dockerfile.observe"
+COMPOSE = DOCKER / "compose.observe.yml"
+IGNORE = DOCKER / ".dockerignore"
+AGENT = DOCKER / "agent.js"
+
+_PIN = re.compile(r"@vantio/cli@([^\s\"']+)")
+_EXACT = re.compile(r"^\d+\.\d+\.\d+$")
+_ENTRYPOINT = re.compile(r"^ENTRYPOINT\s+(\[.*\])\s*$", re.M)
+_CMD = re.compile(r"^CMD\s+(\[.*\])\s*$", re.M)
+_SECRET_LINES = (
+    "**/.env",
+    "**/.env.*",
+    "**/.git",
+    "**/.git/**",
+    "**/.ssh",
+    "**/.ssh/**",
+    "**/*.pem",
+    "**/*.key",
+    "**/id_rsa",
+    "**/*credentials*",
+    "**/secrets",
+    "**/secrets/**",
+    "**/.npmrc",
+)
+
+
+class ObserveExampleTests(unittest.TestCase):
+    def test_cli_pin_is_exact_and_matches_the_tree(self) -> None:
+        cli = (DOCKER.parents[1] / "packages" / "vantio-cli" / "package.json").read_text(encoding="utf-8")
+        version = re.search(r'"version":\s*"([^"]+)"', cli)
+        self.assertIsNotNone(version)
+        expected = version.group(1)
+        pins = []
+        for path in (DOCKERFILE, COMPOSE):
+            for match in _PIN.finditer(path.read_text(encoding="utf-8")):
+                pins.append((path.name, match.group(1)))
+        self.assertTrue(pins, "the observe example does not pin @vantio/cli")
+        for name, pin in pins:
+            self.assertNotIn("^", pin, name)
+            self.assertNotIn("~", pin, name)
+            self.assertIsNotNone(_EXACT.fullmatch(pin), f"{name} pin {pin} is not exact")
+            self.assertEqual(pin, expected, name)
+        dockerfile = DOCKERFILE.read_text(encoding="utf-8")
+        self.assertNotIn("@vantio/cli@^", dockerfile)
+        self.assertNotIn("@vantio/cli@~", dockerfile)
+        self.assertNotRegex(dockerfile, r"npm install -g @vantio/cli(\s|$)")
+
+    def test_default_command_runs_node_under_vantio_run(self) -> None:
+        text = DOCKERFILE.read_text(encoding="utf-8")
+        entry = _ENTRYPOINT.search(text)
+        cmd = _CMD.search(text)
+        self.assertIsNotNone(entry)
+        self.assertIsNotNone(cmd)
+        self.assertEqual(entry.group(1), '["vantio", "run", "node"]')
+        self.assertNotIn('"npm"', cmd.group(1))
+        self.assertIn("agent.js", cmd.group(1))
+        self.assertNotIn("VANTIO_OBSERVE", text)
+        agent = AGENT.read_text(encoding="utf-8")
+        self.assertNotIn("fetch(", agent)
+        self.assertNotIn("http.request", agent)
+        compose = COMPOSE.read_text(encoding="utf-8")
+        self.assertNotIn("VANTIO_HOOKS=0", compose)
+        self.assertNotIn("|| true", compose)
+
+    def test_build_context_is_strict_and_excludes_secrets(self) -> None:
+        dockerfile = DOCKERFILE.read_text(encoding="utf-8")
+        self.assertNotIn("COPY .", dockerfile)
+        self.assertIn("COPY package.json agent.js", dockerfile)
+        compose = COMPOSE.read_text(encoding="utf-8")
+        self.assertNotIn("../..", compose)
+        self.assertIn("context: .", compose)
+        self.assertTrue(IGNORE.is_file(), ".dockerignore is missing")
+        ignored = IGNORE.read_text(encoding="utf-8")
+        for line in _SECRET_LINES:
+            self.assertIn(line, ignored.splitlines(), line)
+        self.assertIn("\n*\n", f"\n{ignored}")
+        for name in (".env", ".env.local", "id_rsa", "secrets/token", ".git/config", ".ssh/id_rsa", "keys/app.pem"):
+            self.assertTrue(_docker_ignored(ignored, name), name)
+        self.assertFalse(_docker_ignored(ignored, "agent.js"))
+        self.assertFalse(_docker_ignored(ignored, "package.json"))
+        self.assertFalse(_docker_ignored(ignored, "Dockerfile.observe"))
+
+
+def _docker_ignored(text: str, relpath: str) -> bool:
+    """Last-match dockerignore check for the patterns this example uses."""
+    ignored = False
+    for raw in text.splitlines():
+        line = raw.strip()
+        if not line or line.startswith("#"):
+            continue
+        negate = line.startswith("!")
+        pattern = line[1:] if negate else line
+        if _docker_match(pattern, relpath):
+            ignored = not negate
+    return ignored
+
+
+def _docker_match(pattern: str, relpath: str) -> bool:
+    if pattern == "*":
+        return "/" not in relpath
+    regex = re.escape(pattern).replace(r"\*\*/", "(?:.*/)?")
+    regex = regex.replace(r"\*\*", ".*").replace(r"\*", "[^/]*")
+    if pattern.startswith("**/"):
+        return re.fullmatch(regex, relpath) is not None
+    return re.fullmatch(regex, relpath) is not None or re.fullmatch(regex, relpath.split("/")[-1]) is not None
+
+
+if __name__ == "__main__":
+    unittest.main()

docs/governance/VERSION-METADATA.json

@@ -6,10 +6,10 @@
     {
       "id": "cli",
       "name": "@vantio/cli",
-      "version": "0.3.24",
+      "version": "0.3.25",
       "manifest": "packages/vantio-cli/package.json",
       "changelog": "docs/governance/changelogs/cli.md",
-      "changelog_heading": "## 0.3.24"
+      "changelog_heading": "## 0.3.25"
     },
     {
       "id": "node-sdk",
@@ -22,16 +22,16 @@
     {
       "id": "python-sdk",
       "name": "vantio-agent-sdk",
-      "version": "3.1.0",
+      "version": "3.1.1",
       "manifest": "packages/vantio-agent-sdk-py/pyproject.toml",
       "also": [
         {
           "file": "packages/vantio-agent-sdk-py/vantio/__init__.py",
-          "contains": "__version__ = \"3.1.0\""
+          "contains": "__version__ = \"3.1.1\""
         }
       ],
       "changelog": "packages/vantio-agent-sdk-py/CHANGELOG.md",
-      "changelog_heading": "## 3.1.0"
+      "changelog_heading": "## 3.1.1"
     },
     {
       "id": "optics-mcp",
@@ -44,10 +44,10 @@
     {
       "id": "gate-mcp",
       "name": "@vantio/gate-mcp",
-      "version": "0.1.0",
+      "version": "0.1.1",
       "manifest": "packages/vantio-gate-mcp/package.json",
       "changelog": "docs/governance/changelogs/gate-mcp.md",
-      "changelog_heading": "## 0.1.0"
+      "changelog_heading": "## 0.1.1"
     },
     {
       "id": "vscode",

docs/governance/canonical/environment.md

@@ -5,7 +5,7 @@ Free Optics runs with no account and no API key. Telemetry stays off unless `VAN
 | Variable | Role |
 |---|---|
 | `DO_NOT_TRACK` | Set to `1` to keep telemetry off. |
-| `VANTIO_API_BASE` | Base URL for the Gate MCP control-plane client. Default `https://api.vantio.ai`. Not required for free Optics. |
+| `VANTIO_API_BASE` | Base URL for the Gate MCP control-plane client. Default `https://api.vantio.ai`. Not required for free Optics. `gate_get_policy` and `gate_residual_risk` read this from the environment. They do not take a host argument. |
 | `VANTIO_API_KEY` | Control-plane key for Phantom Engine / Enterprise policy and ingest. Not required for free Optics. |
 | `VANTIO_AUDIT_MODE` | Set to `1` to flag events as audit mode. |
 | `VANTIO_CLOUD_INGEST` | Set to `true` or `1` before `reportAnomaly` / `report_anomaly` will send. |

docs/governance/changelogs/cli.md

@@ -2,6 +2,12 @@
 
 This heading exists so a documentation release can require a changelog entry for the version already in `packages/vantio-cli/package.json`. It does not bump that version.
 
+## 0.3.25
+
+CANDIDATE_ONLY_NOT_FOR_PUBLICATION. Source version only. Not an npm release.
+
+CLI readers honor `VANTIO_HOME`. A `VANTIO_INGEST_URL` that is not http(s) is reported, and with an API key in-scope calls fail closed. Streaming byte counts are recorded before the run log is written.
+
 ## 0.3.24
 
 Documentation baseline at `14249ba84ff1f3d5aa8ad7a7366172f29235c76e`. The CLI reads its version from package.json. Product behavior is unchanged by this documentation record.

docs/governance/changelogs/gate-mcp.md

@@ -2,6 +2,12 @@
 
 This heading exists so a documentation release can require a changelog entry for the version already in `packages/vantio-gate-mcp/package.json`. It does not bump that version.
 
+## 0.1.1
+
+CANDIDATE_ONLY_NOT_FOR_PUBLICATION. Source version only. Not an npm release.
+
+`gate_get_policy` and `gate_residual_risk` read `VANTIO_API_KEY` from the environment. They do not take an `api_key` tool argument. They also do not take an `api_base` tool argument. The control-plane host is `VANTIO_API_BASE`, or `https://api.vantio.ai` when that variable is unset or blank. A caller-supplied host is ignored. Host matching uses a DNS suffix. `dry_run: false` names `BLOCKED_*` actions. `dry_run: true` keeps the `DRY_RUN_` prefix. The tools still do not block network traffic.
+
 ## 0.1.0
 
 Documentation baseline at `14249ba84ff1f3d5aa8ad7a7366172f29235c76e`. `@vantio/gate-mcp` remains a legacy compatibility package. Gate is not a separate product. Product behavior is unchanged by this documentation record.