diff --git a/docs/planning/boot-hold/FD-REBOOT-1-LAB-PROCEDURE.md b/docs/planning/boot-hold/FD-REBOOT-1-LAB-PROCEDURE.md new file mode 100644 index 00000000..074ccd2b --- /dev/null +++ b/docs/planning/boot-hold/FD-REBOOT-1-LAB-PROCEDURE.md @@ -0,0 +1,112 @@ +# FD-REBOOT-1 lab procedure — reboot hold reproof + +**Audience:** INTERNAL_RESTRICTED +**Do not run this from the open-core PR.** This file is the procedure for a later authorized Free-plan host reproof. +**Row:** `B1-REBOOT-EXPOSURE` / GAP-FIP-014 +**Until that host reproof:** **NOT_PROVED**. Unknown is never PASS. +**Frozen seal (not retro-claimed):** `e0b19d557891b1ee8bbd20e702df11669d175e4083ef5bbe2f7077cf30093b5e` +**Ceiling:** `INTERNAL_CLEAN_HOST_PROOF` +**Account:** `960577828987` only +**Expected out-of-pocket:** `expected_oop_usd=0` (OOP $0). No Cost Explorer. No Paid instance sizes. No publish. No soak, fleet, or k3s. + +Measured on the short run `2026-09-30T2225ET-b1-short` before this change: `W_unprotected≈23.05s`, `W_race≈12.84s`. Loader down and pins empty during the window. The enrolled timeout was not a Phantom Engine deny. An unenrolled connect completed. This procedure exists so a later run can record hold time and show zero enrolled allows. It does not convert that earlier run into a pass. + +## Guards (same shape as the B1 short) + +1. Fresh cost gate at launch: Free/Active, `expected_oop_usd=0`, `abort=false`, account `960577828987`, no Cost Explorer, no Paid. +2. One lab instance. Prefer `t3.micro`, then `t3.small`. `associate_public_ipv4=false`. `stop_after_minutes` stays inside the short-lab cap. No public IPv4. +3. Region and tags follow the B1 short plan. Re-establish access with SSM or serial. +4. Install the open-core boot hold on the guest (`vantio-boot-hold install`) and set `VANTIO_BOOT_HOLD_LIVE=1` only for the root install and boot units. Enroll one workload and record one unenrolled control with `observe-unenrolled`. +5. Point the loader at `/sys/fs/cgroup/vantio-enrolled.slice` and keep the Phantom Engine container on `--restart=no`, outside that slice. +6. Export timestamps before any terminate. Do not mark the row PASS in the export. A missing timestamp is NOT_PROVED. + +## Clock and metrics + +Record `T_boot`, `T_egress`, and `T_ready` with the guest clock source (chrony/UTC). Also record `T_hold`, the guest time when `vantio-boot-hold` state is `HELD` and the scoped rules are present. + +- `W_unprotected = max(0, T_ready − T_boot)` +- `W_race = T_ready − T_egress` when `T_egress < T_ready`, else `0` +- `W_hold = T_ready − T_hold` when both exist + +Report the measured numbers. Do not invent a maximum-seconds marketing threshold. + +Target for a future PASS, which this procedure does not award: zero enrolled allows at any point during boot, hold time recorded, and either a deny of the first enrolled egress or a documented fail-closed hold while the loader is absent. An unenrolled connect may complete. That shows the host was not placed on a host-wide default-route hold. + +## Matrix + +Run these as separate boots. Each boot exports its own `timestamps.json`, `egress-probe.txt`, `hold-status.json`, and a journal excerpt. The enrolled probe and the unenrolled probe race from t=0 (a unit that fires as soon as its gate allows). + +### 1. Enrolled and unenrolled from t=0 (both mechanisms on) + +Default config: hold on, ordering on. Reboot. From the first userspace moment, attempt enrolled egress and unenrolled egress. + +Expect: enrolled unit does not become active before `vantio-pe-enforce-ready.service`. If it is forced to run, its egress does not complete while `state` is `HELD`. Unenrolled egress may complete. SSH session survives. Record `W_hold`. + +### 2. Loader is running but enforcement is not attached + +Start the loader process without attaching `cgroup_skb_egress_enforce` to `/sys/fs/cgroup/vantio-enrolled.slice` (pins may be present). Reboot or start from a held boot. + +Expect: `vantio-boot-hold release --require-enforce-ready` refuses. Status stays `HELD` / `DEGRADED`. The message says the loader is running and enforcement is not attached. Enrolled workloads stay stopped. Do not treat the running process as enforce-ready. + +### 3. Loader fails, hold stays, SSH works, break-glass releases the start gate + +Leave `/etc/vantio/pe-loader.argv.json` absent or point it at a missing binary. Reboot. + +Expect: `vantio-pe-loader.service` fails, `vantio-pe-enforce-ready.service` fails, enrolled workload stays inactive, `vantio-boot-hold status` shows `HELD` and `DEGRADED` with the operator message. Open an SSH session and a console session. From SSH as root, one command, `vantio-boot-hold release --break-glass --i-am-root-operator`, returns `RELEASED`. Health stays `DEGRADED`. `audit.log` contains `BREAK_GLASS` and names what was released: `packet-hold-ipv4`, `packet-hold-ipv6`, `file-hold`, and `start-gate:` for each enrolled unit. After that command, `systemctl start` of the enrolled unit is not blocked by `Requires=vantio-pe-enforce-ready.service`. Do not call that a reboot PASS. + +Set `deny_probe` in `/etc/vantio/boot-hold.json` before the enforce-ready boots, for example the lab's public IPv6 destination and port 443. A boot that never runs the deny self-check stays held. + +### 4. Docker restart policy + +Create a second container with `--restart=always` and try to enroll it. Expect enroll to refuse until `docker update --restart=no`. After enroll, reboot and confirm `docker ps` does not show the enrolled container before enforce-ready. A container left on `always` and not enrolled is unprotected in status; record it that way rather than calling it held. + +### 5. Hold alone + +As root, `vantio-boot-hold configure --hold on --ordering off`, then reboot. + +Expect: the enrolled unit may start (no `Requires=` on enforce-ready) and its egress still fails while the cgroup and subnet rules are installed. Unenrolled egress may complete. Record hold time. This boot is not a PASS by itself. + +### 6. Ordering alone + +As root, `vantio-boot-hold configure --hold off --ordering on`, then reboot. + +Expect: the packet rules are absent, and the enrolled unit stays inactive until enforce-ready. Record the start time relative to `T_ready`. This boot is not a PASS by itself. Ordering alone leaves a hole if something starts the workload by hand before ready; that is why the hold exists. + +### 7. Both together + +`vantio-boot-hold opt-in` (hold on and ordering on). Reboot. + +Expect: the unit does not start early, and the rules are present until the enforce-ready release. Record `W_hold` and the enrolled and unenrolled probe results. + +### 8. Five or more reboots + +Repeat the both-together boot five times (5+). Each iteration records `W_hold`, `W_unprotected`, and `W_race` in `reboot-N/timestamps.json`. A missing sample is NOT_PROVED for that iteration. Do not average them into a pass. + +### 9. Enrolled workload tries break-glass + +From a process in `vantio-enrolled.slice` (the enrolled unit, including uid 0 inside that cgroup), run `vantio-boot-hold release --break-glass --i-am-root-operator`. + +Expect: exit non-zero, JSON `state` `FAILED_SAFE`, and an audit line with `"result": "REFUSED"`. The hold remains. Repeat as a non-root caller and expect the same refusal. + +## Evidence + +```text +ROW-B1-REBOOT-EXPOSURE/ + timestamps.json + probe-spec.txt + egress-probe.txt + hold-status.json + audit-excerpt.log + journal-boot.txt + matrix/ + hold-alone/ + ordering-alone/ + both/ + loader-fail/ + docker-restart/ + reboot-1/ ... reboot-5/ +``` + +`hold-status.json` is the `vantio-boot-hold status` object. `reboot_row` in that object stays `NOT_PROVED` even when the lab looks clean. A human pass decision waits on the verifier and a fresh seal if Phantom Engine changed. This open-core change does not seal Phantom Engine. + +GAP-BH-002 and GAP-BH-005 through GAP-BH-009 stay open. This procedure does not close them. diff --git a/docs/planning/boot-hold/PE-COMPANION.md b/docs/planning/boot-hold/PE-COMPANION.md new file mode 100644 index 00000000..5fc1617e --- /dev/null +++ b/docs/planning/boot-hold/PE-COMPANION.md @@ -0,0 +1,52 @@ +# Phantom Engine companion note — FD-REBOOT-1 + +**Audience:** INTERNAL_RESTRICTED +**Repo this note belongs to:** open-core packaging. It does not patch `vantio-phantom-engine` and it does not invent a seal. + +## What open-core ships + +`packages/vantio-install` installs these units: + +- `vantio-boot-hold.service` — early, `DefaultDependencies=no`, `Before=docker.service containerd.service`, `WantedBy=sysinit.target` +- `vantio-pe-loader.service` — `After=vantio-boot-hold.service docker.service`, `Before=vantio-pe-enforce-ready.service`, `Restart=no` +- `vantio-pe-enforce-ready.service` — `Requires=vantio-pe-loader.service`, runs `release --require-enforce-ready` +- `vantio-enrolled.slice` and `vantio-enrolled-docker@.service` + +The loader command is not baked into the unit. A root admin writes `/etc/vantio/pe-loader.argv.json` as a JSON list. Example for a container that was created with `--restart=no` and is not in the enrolled slice: + +```json +["/usr/bin/docker", "start", "-a", "vantio-pe"] +``` + +The container's loader arguments need `--enforce`, `--cgroup-skb-enforce`, and: + +```text +--startup-enroll-cgroup /sys/fs/cgroup/vantio-enrolled.slice +``` + +Enrolled systemd units set `Slice=vantio-enrolled.slice`, so their cgroup is `/sys/fs/cgroup/vantio-enrolled.slice/`. + +## What the PE tree does today + +The Phantom Engine tree on this machine has no systemd unit. Startup enrollment is the loop in `vantio-loader/src/main.rs` that calls `resolve_cgroup_spec` for each `--startup-enroll-cgroup` value and inserts that one cgroup id. `cgroup_skb` is then attached to that cgroup path. + +A PE repository change is not required for open-core to install the units. If the parent wants the unit to live in the Phantom Engine repo, add `vantio-pe-loader.service` with the same ordering as `vantio_install/boot_hold/units.py` (`pe_loader_service`) and point `ExecStart` at the loader binary with the slice path above. Do not treat that copy as a seal. A merge that changes loader behavior needs a new seal from the parent. Seal `e0b19d557891b1ee8bbd20e702df11669d175e4083ef5bbe2f7077cf30093b5e` stays frozen and is not retro-claimed. + +## What the parent should re-verify on the next seal + +The boot units set `VANTIO_BOOT_HOLD_LIVE=1` so apply, release, and loader start actually touch the host. Fixture tests leave that variable unset and pass a fake root, so they record commands instead of changing packet filters. The enforce-ready unit does not set `VANTIO_BOOT_HOLD_ALLOW_FACTS` or `VANTIO_BOOT_HOLD_ALLOW_CALLER_FIXTURE`. + +The open-core probe treats enforce-ready as all of the following. A running loader is not enough: + +- a live `vantio-loader` command line containing `--enforce` +- the pinned names `vantio_trace_map`, `vantio_enrolled_cgroups`, `vantio_debug_counters`, `vantio_debug_last_comm`, `vantio_tls_severed_pids` +- `bpftool map show` containing `vantio_enforce` (policy loaded) +- `bpftool cgroup show /sys/fs/cgroup/vantio-enrolled.slice` containing `cgroup_skb_egress_enforce` (attached, not only loaded) +- loader health `OK` (process state R, S, or D) +- a deny self-check: enrolled connect fails and the same connect outside the slice succeeds, after a one-destination exception in the hold chain that is removed before release + +A loader that is running while that cgroup show does not list the program keeps the hold. + +Unknown on any of those keeps the hold. The parent should confirm on the sealed loader that attaching `cgroup_skb` to `/sys/fs/cgroup/vantio-enrolled.slice` covers descendant cgroups under that slice. This note does not record that confirmation. + +A root attacker who can disable host controls is Battery 4. This hold does not detect that tampering. diff --git a/packages/vantio-install/README.md b/packages/vantio-install/README.md index 9b74fbdf..14ba91c9 100644 --- a/packages/vantio-install/README.md +++ b/packages/vantio-install/README.md @@ -17,6 +17,8 @@ Run the commands from the host that will keep the node: - `vantio-install uninstall` on a customer host is the dual-gated command in `docs/UNINSTALL.md`. - `vantio-install verify-removal` checks whether that scope is actually gone, including the known pin names on `/sys/fs/bpf` for a Phantom Engine scope. +`vantio-boot-hold` is the early-boot hold for enrolled workloads. It is on unless a root admin opts out. Read `docs/BOOT-HOLD.md`. The command prints the same JSON envelope, and the reboot exposure row stays `NOT_PROVED` until a later host reproof. + Every command prints one JSON object. `proof_state` stays `NOT_PROVED` in this package. `vantio-verify` reads the evidence directory and the bundle from disk, and it ignores an installer exit code of 0. A live change needs `VANTIO_INSTALL_ALLOW_LIVE=1` and `--i-accept-live-mutations` together, plus `--plan` and `--plan-sha256`, on `apply`, `rollback`, or `uninstall`. The installer then runs an allowlisted observe-only command list and checks the host again before it treats the step as verified. diff --git a/packages/vantio-install/bin/vantio-boot-hold b/packages/vantio-install/bin/vantio-boot-hold new file mode 100755 index 00000000..031fcb55 --- /dev/null +++ b/packages/vantio-install/bin/vantio-boot-hold @@ -0,0 +1,9 @@ +#!/usr/bin/env python3 +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from vantio_install.boot_hold.cli import main + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/packages/vantio-install/docs/BOOT-HOLD.md b/packages/vantio-install/docs/BOOT-HOLD.md new file mode 100644 index 00000000..b74f376f --- /dev/null +++ b/packages/vantio-install/docs/BOOT-HOLD.md @@ -0,0 +1,160 @@ +# Boot hold + +Enrolled workloads stay held from early boot until Phantom Engine is enforce-ready. The hold is on by default. It covers enrolled workloads only: their network egress and the protected paths you listed at enrollment. SSH, cloud agents, DHCP, DNS, and every unenrolled process keep working. The host default route stays up. + +The hold is a host mechanism. It is installed by `vantio-boot-hold.service` before Docker and containerd start. It does not read Phantom Engine BPF pins to install itself, and it does not need the Phantom Engine container to be running. Those pins are empty after a reboot until the loader attaches. + +`vantio-install apply` installs and enables this hold. You do not run a second install command to get it. A trusted opt-out file is left alone, so apply does not turn the hold back on. + +`vantio-boot-hold status` prints one JSON object. The same object is copied into the installer `HEALTH.json` as `boot_hold`. While the hold is in the packet filter and Phantom Engine is not enforce-ready, `state` is `HELD` and `health` is `DEGRADED`. The message tells you SSH is up and how a root operator releases the hold. `reboot_row` stays `NOT_PROVED`. This command does not mark a reboot exposure run as passed. + +## What runs, and in what order + +1. `vantio-boot-hold.service` runs from `sysinit.target`, before `docker.service` and `containerd.service`. It creates `vantio-enrolled.slice`, loads the AppArmor profile `vantio-boot-hold` in deny mode for the protected paths you enrolled, and inserts scoped iptables and ip6tables rules. +2. Docker starts. A drop-in orders Docker after the hold. Docker does not `Requires=` the hold, so a hold failure does not take Docker down. Enrolled containers stay stopped because their restart policy is `no`. +3. `vantio-pe-loader.service` starts Phantom Engine after the hold and after Docker. The loader command is a root-owned JSON list in `/etc/vantio/pe-loader.argv.json`. If that file is missing, the loader unit fails and enrolled workloads stay held. +4. `vantio-pe-enforce-ready.service` releases the hold only after a live check: the loader is up with `--enforce`, the pinned map names are present, policy is loaded (`vantio_enforce` is in the map list), `cgroup_skb_egress_enforce` is attached to `vantio-enrolled.slice`, loader health is OK, and a deny self-check sees the enrolled connect fail while the same connect from outside the slice succeeds. The self-check opens one scoped exception in the hold chain for that destination, then removes it. A loader that is running but not attached does not pass. If the check fails, the unit fails, the hold stays, and enrolled units that require it stay stopped. Set `deny_probe` in `/etc/vantio/boot-hold.json` to the host and port the self-check uses. Without that probe target the check does not pass and the hold stays. +5. Enrolled workloads start from their own systemd units after enforce-ready. + +Ordering and the hold are both on unless a root admin changes them. You can exercise one at a time with `configure`. Turning both off is an opt-out. + +## Packet rules + +The boot units set `VANTIO_BOOT_HOLD_LIVE=1` so the packet rules, the release, and the loader start run on the host. An admin shell that changes the hold exports the same variable. If `iptables` or `ip6tables` cannot insert the enrolled rules, the hold unit fails and enrolled units that require it stay stopped. + +The filter chain is `VANTIO_BOOT_HOLD`. It drops two classes of traffic: + +- OUTPUT packets whose socket is in `vantio-enrolled.slice` or a child of that slice (IPv4 and IPv6). +- FORWARD packets from `10.250.250.0/24` (IPv4) or `fd76:616e:7469::/64` (IPv6). + +SSH, DHCP, and DNS are not in that slice and do not use that subnet, so their packets stay on the normal path. The rules do not change the default route. + +## Protected files + +Each enrolled systemd unit gets `Slice=vantio-enrolled.slice`. While the hold is up, the unit also gets `InaccessiblePaths=` for each protected path and `AppArmorProfile=vantio-boot-hold` when AppArmor loaded. Release removes the `InaccessiblePaths=` lines, reloads systemd, and replaces the AppArmor profile with an allow profile of the same name so the workload can start. Phantom Engine policy is the control after the enforce-ready probe. + +A protected path is an absolute directory at least three levels deep, outside SSH keys and system directories such as `/etc`, `/usr`, and `/home` itself. `/var/lib/app/secrets` is a usable example. `/`, `/etc/ssh`, and `/root/.ssh` are refused. + +## Plain Docker + +Create the network and the container while the machine is up, with restart policy `no`. The systemd unit starts the container later. + +```bash +docker network create --subnet 10.250.250.0/24 vantio-enrolled +docker run -d --name agent \ + --restart=no \ + --cgroup-parent=/vantio-enrolled.slice \ + --security-opt apparmor=vantio-boot-hold \ + --network vantio-enrolled \ + your-image +vantio-boot-hold enroll-docker \ + --name agent \ + --restart-policy no \ + --cgroup-parent /vantio-enrolled.slice \ + --protected-path /var/lib/app/secrets +``` + +`vantio-enrolled-docker@agent.service` runs `docker start agent` after `vantio-pe-enforce-ready.service`. A restart policy of `always`, `unless-stopped`, or `on-failure` is refused, because Docker would start that container in parallel with Phantom Engine. `enroll-docker --set-restart-no` runs `docker update --restart=no` and then enrolls. + +`vantio-enrolled-network.service` creates the network after Docker is up. It does not start enrolled containers. + +## Compose + +The compose file uses restart `no`, the enrolled cgroup parent, the AppArmor profile, and the enrolled subnet. `vantio-boot-hold enroll-compose` refuses the file until those are present, and it installs a systemd unit whose `ExecStart` is `docker compose start`. + +```yaml +services: + agent: + image: your-image + restart: "no" + cgroup_parent: /vantio-enrolled.slice + security_opt: + - apparmor:vantio-boot-hold + networks: + - enrolled +networks: + enrolled: + name: vantio-enrolled + ipam: + config: + - subnet: 10.250.250.0/24 +``` + +```bash +vantio-boot-hold enroll-compose \ + --project-dir /var/lib/app/compose \ + --compose-file /var/lib/app/compose/compose.yaml +``` + +Create the containers with `docker compose create` in that directory before reboot. The systemd unit starts them after enforce-ready. + +## systemd services + +```bash +vantio-boot-hold enroll-systemd \ + --unit my-agent.service \ + --protected-path /var/lib/app/secrets +``` + +The drop-in `/etc/systemd/system/my-agent.service.d/vantio-boot-hold.conf` sets `Slice=vantio-enrolled.slice`, `After=` and `Requires=` `vantio-boot-hold.service`, and, when ordering is on, `After=` and `Requires=` `vantio-pe-enforce-ready.service`. + +The Phantom Engine loader enrolls that slice. A root-owned `/etc/vantio/pe-loader.argv.json` can be: + +```json +["/usr/bin/docker", "start", "-a", "vantio-pe"] +``` + +Create the Phantom Engine container first with `--restart=no`, host networking, and `--startup-enroll-cgroup /sys/fs/cgroup/vantio-enrolled.slice` in the loader arguments. Keep that container out of `vantio-enrolled.slice`. The open-core unit starts it. The companion note for a Phantom Engine repository change is in the planning doc shipped beside this package's tests, and this package does not invent a seal. + +SSH, `systemd-networkd`, resolved, SSM, Docker, and the hold units themselves cannot be enrolled. + +## When the loader does not start + +Enrolled workloads stay held. `vantio-boot-hold status` shows `HELD` and `DEGRADED`, with the operator message. SSH and the console still come up, because those units are not ordered behind the hold and they are not in the enrolled slice. + +Release is a root action on the host and it is written to `/var/lib/vantio/boot-hold/audit.log`. + +- After the probe passes, `vantio-pe-enforce-ready.service` runs `vantio-boot-hold release --require-enforce-ready`. +- If the probe fails, that command refuses and the hold stays. +- A root operator at the console can run `vantio-boot-hold release --break-glass --i-am-root-operator`. That one action removes the packet hold, clears the file hold, and drops `Requires=` on enforce-ready and on the boot-hold unit for enrolled systemd, Docker, and Compose units. The audit line lists each item released, including `packet-hold-ipv4`, `packet-hold-ipv6`, `file-hold`, and `start-gate:`. Health stays `DEGRADED` because Phantom Engine was not enforce-ready. The next boot installs the hold again unless you opted out. + +The command refuses a caller whose cgroup is under `vantio-enrolled.slice`, a caller outside the host init namespace, and any caller who is not root. An enrolled workload cannot release the hold. There is no silent release. + +## Default and opt-out + +A missing `/etc/vantio/boot-hold.json` means the hold and ordering are on. Opt out with: + +```bash +vantio-boot-hold opt-out --reason "maintenance window approved by the host admin" +``` + +The file must be owned by root and must not be group or world writable. A looser file is ignored and the hold stays on. `status` shows `OPTED_OUT`. Turn it back on with `vantio-boot-hold opt-in`. + +Hold alone, or ordering alone: + +```bash +vantio-boot-hold configure --hold on --ordering off +vantio-boot-hold configure --hold off --ordering on +``` + +Both of those are logged. `configure` refuses to turn both off. Use `opt-out` for that. + +Live packet changes on the host run only when you are root, `--root /`, and `VANTIO_BOOT_HOLD_LIVE=1`. Any other invocation records the commands and writes under `--root` so a fixture can exercise them. + +## Workloads that are not enrolled + +A workload that is missing from `/var/lib/vantio/boot-hold/registry.json` is unprotected. `status --lookup NAME` prints `protection: UNPROTECTED` for that name. Record a known unenrolled control explicitly: + +```bash +vantio-boot-hold observe-unenrolled --id unenrolled-probe --kind systemd +``` + +The hold does not apply to it. Status keeps showing it as unprotected. + +## Install the units + +```bash +vantio-boot-hold install +``` + +That writes the units, the Docker and containerd `After=vantio-boot-hold.service` drop-ins, and the enable symlinks. The next boot runs the hold before Docker. `status` before `apply-boot` says `CONFIGURED`, which means the packet filter is not installed yet. diff --git a/packages/vantio-install/docs/INSTALL.md b/packages/vantio-install/docs/INSTALL.md index d1a2c78f..f7e775c3 100644 --- a/packages/vantio-install/docs/INSTALL.md +++ b/packages/vantio-install/docs/INSTALL.md @@ -27,3 +27,5 @@ Apply loads the sealed Phantom Engine OCI tar with `docker load`. On Ubuntu 24.0 Apply is finished only when `state` is `HEALTHY` or `DEGRADED` and `HEALTH.json` records that result. `APPLIED` means the steps ran and health is not confirmed yet. A process exit of 0 from Docker, npm, or pip is not that result. After the Optics npm install, the host check requires `/bin/vantio` and the pinned CLI version (`vantio --version`, or the installed package manifest when the binary does not print a version). After the Agent SDK npm install, the host check requires `/lib/node_modules/@vantio/agent-sdk` and the pinned version in that package manifest. After the Agent SDK pip install, the host check requires the `vantio` module under the prefix and the pinned version. Debian and Ubuntu pip write that module at `local/lib/python3.X/dist-packages`. An upstream prefix layout writes it at `lib/python3.X/site-packages`. When the module does not declare a version, the check reads the matching dist-info metadata. `install_agent_sdks` is checkpointed only after both SDK checks pass. Before the observe-only Phantom Engine container starts, the installer loads AppArmor profile `vantio-pe-observe` and passes `--security-opt apparmor=vantio-pe-observe`. The same start bind-mounts host `/sys/fs/bpf` and host `/sys/kernel/tracing`. Preflight records the tracing mount as PF-TRACEFS and blocks when that directory is missing or empty. The host check for that container reads the container after `docker run -d` returns. That command exits 0 when the daemon accepts the container, including when the process has already stopped. A stopped container is not verified. A detached container that is still starting is read again until the loader process is visible, the known bpffs pins are present, and clsact is on the interface, or until that wait ends. A detached container that reaches that state is verified. A container that stays up without those facts is not verified. On Ubuntu 24.04 the `nodejs` package does not include npm. When npm is already on `PATH`, `ensure_node` runs `npm --version`. When npm is missing and the installer is root, `ensure_node` installs the Ubuntu `npm` package before the Optics CLI install. When npm is missing and the installer is not root, `plan` stops and `PLAN.json` names that package. + +`vantio-install apply` installs and enables `vantio-boot-hold` before the observe container starts. A trusted opt-out in `/etc/vantio/boot-hold.json` stays in place. The hold keeps enrolled workloads stopped until Phantom Engine is enforce-ready, which includes a live deny check. A running loader that is not attached does not release the hold. The hold matches the enrolled cgroup and the enrolled subnet `10.250.250.0/24`. SSH, DHCP, DNS, and unenrolled processes keep working. `docs/BOOT-HOLD.md` is the procedure for plain Docker, Compose, and systemd. The reboot exposure row stays `NOT_PROVED` until a host reproof records it. diff --git a/packages/vantio-install/docs/LIMITATIONS.md b/packages/vantio-install/docs/LIMITATIONS.md index 9a340ef2..5be8e1f8 100644 --- a/packages/vantio-install/docs/LIMITATIONS.md +++ b/packages/vantio-install/docs/LIMITATIONS.md @@ -24,4 +24,6 @@ Live changes on a host run only when you set `VANTIO_INSTALL_ALLOW_LIVE=1` and p The live privilege check accepts effective uid 0 only. `privilege_mode` `sudo` with `sudo` on `PATH`, and `privilege_mode` `docker_group` with write access to `/var/run/docker.sock`, are recorded facts and are not a live grant. `privilege_mode` `UNKNOWN` blocks `PF-DOCKER-PERM`. When the effective uid is not 0, the live command returns `FAILED_SAFE` and the message `Live mutations need effective root. sudo on PATH is not privilege.` `PREFLIGHT.md` records the symptoms. `sudo` is not an allowlisted executable. Raw `docker` and raw `sudo docker` outside the installer argv list are forbidden. The installer does not insert `sudo` in front of `docker`. A live residual check reports `RESIDUAL_FOUND` when something from that scope is still on the host. From that state, the same dual-gated rollback, or uninstall with `--scope optics` or `--scope all`, removes a leftover Optics CLI or Agent SDK tree under the prefix. Rollback, or uninstall with `--scope pe` or `--scope all`, also unlinks the known bpffs pin names when they are still present. Apply stays refused until `verify-removal` reports an empty residual list. +`vantio-install apply` installs and enables the boot hold. Release waits for attached enforcement and a live deny check, not for a running loader alone. `vantio-boot-hold` defaults on for workloads you enroll. A root admin can opt out, and that opt-out is logged and shown in `vantio-boot-hold status`. The hold applies to the enrolled cgroup `vantio-enrolled.slice` and to the enrolled subnet `10.250.250.0/24`. It leaves the host default route in place so SSH, DHCP, and DNS can come up. A workload that is not in the enrollment registry is unprotected in that status. This package does not mark the reboot exposure row proved. `proof_state` stays `NOT_PROVED` until a later host reproof. + Fixture tests exercise the transaction without those live commands. Those tests are internal. They are not a customer rehearsal, and they do not make `--fixture-host` a customer flag. `proof_state` stays `NOT_PROVED`. The second-lab gate stays closed until a later authorization. The proof ceiling stays `INTERNAL_CLEAN_HOST_PROOF`. diff --git a/packages/vantio-install/docs/PACKAGING.md b/packages/vantio-install/docs/PACKAGING.md index 77a297f5..72fba9ac 100644 --- a/packages/vantio-install/docs/PACKAGING.md +++ b/packages/vantio-install/docs/PACKAGING.md @@ -10,6 +10,7 @@ Optics CLI 0.3.24, Agent SDK npm 0.2.4, and Agent SDK Python 3.1.0 are pins in ` - `vantio-install` calls `vantio_install.cli:main` - `vantio-verify` calls `vantio_install.verifier:main` +- `vantio-boot-hold` calls `vantio_install.boot_hold.cli:main` `bin/vantio-install` is a source launcher. The sdist include list omits `bin/`. The sealed wheel's console scripts are the customer commands. diff --git a/packages/vantio-install/packaging/boot-hold/containerd.service.d-vantio-boot-hold.conf b/packages/vantio-install/packaging/boot-hold/containerd.service.d-vantio-boot-hold.conf new file mode 100644 index 00000000..fb463ce4 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/containerd.service.d-vantio-boot-hold.conf @@ -0,0 +1,2 @@ +[Unit] +After=vantio-boot-hold.service diff --git a/packages/vantio-install/packaging/boot-hold/docker.service.d-vantio-boot-hold.conf b/packages/vantio-install/packaging/boot-hold/docker.service.d-vantio-boot-hold.conf new file mode 100644 index 00000000..fb463ce4 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/docker.service.d-vantio-boot-hold.conf @@ -0,0 +1,2 @@ +[Unit] +After=vantio-boot-hold.service diff --git a/packages/vantio-install/packaging/boot-hold/pe-loader.argv.example.json b/packages/vantio-install/packaging/boot-hold/pe-loader.argv.example.json new file mode 100644 index 00000000..850fdbcd --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/pe-loader.argv.example.json @@ -0,0 +1,6 @@ +[ + "/usr/bin/docker", + "start", + "-a", + "vantio-pe" +] diff --git a/packages/vantio-install/packaging/boot-hold/vantio-boot-hold.service b/packages/vantio-install/packaging/boot-hold/vantio-boot-hold.service new file mode 100644 index 00000000..290cda60 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-boot-hold.service @@ -0,0 +1,16 @@ +[Unit] +Description=Hold enrolled workloads before Docker and before workload services +DefaultDependencies=no +After=local-fs.target +Before=docker.service containerd.service vantio-pe-loader.service +# SSH, systemd-networkd, resolved, and SSM are not ordered behind this unit. + +[Service] +Type=oneshot +RemainAfterExit=yes +TimeoutStartSec=30 +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart=/usr/bin/python3 -m vantio_install.boot_hold apply-boot + +[Install] +WantedBy=sysinit.target diff --git a/packages/vantio-install/packaging/boot-hold/vantio-enrolled-docker@.service b/packages/vantio-install/packaging/boot-hold/vantio-enrolled-docker@.service new file mode 100644 index 00000000..4839ad51 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-enrolled-docker@.service @@ -0,0 +1,13 @@ +[Unit] +Description=Start enrolled container %i after the boot gate +After=docker.service vantio-enrolled-network.service vantio-boot-hold.service vantio-pe-enforce-ready.service +Requires=docker.service vantio-enrolled-network.service vantio-boot-hold.service vantio-pe-enforce-ready.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/docker start %i +ExecStop=/usr/bin/docker stop %i + +[Install] +WantedBy=multi-user.target diff --git a/packages/vantio-install/packaging/boot-hold/vantio-enrolled-network.service b/packages/vantio-install/packaging/boot-hold/vantio-enrolled-network.service new file mode 100644 index 00000000..b5bf31aa --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-enrolled-network.service @@ -0,0 +1,13 @@ +[Unit] +Description=Create the enrolled Docker network vantio-enrolled (10.250.250.0/24) +After=docker.service vantio-boot-hold.service +Requires=docker.service + +[Service] +Type=oneshot +RemainAfterExit=yes +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart=/usr/bin/python3 -m vantio_install.boot_hold ensure-network + +[Install] +WantedBy=multi-user.target diff --git a/packages/vantio-install/packaging/boot-hold/vantio-enrolled.slice b/packages/vantio-install/packaging/boot-hold/vantio-enrolled.slice new file mode 100644 index 00000000..32b7e52c --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-enrolled.slice @@ -0,0 +1,6 @@ +[Unit] +Description=Enrolled workloads +DefaultDependencies=no +Before=slices.target + +[Slice] diff --git a/packages/vantio-install/packaging/boot-hold/vantio-pe-enforce-ready.service b/packages/vantio-install/packaging/boot-hold/vantio-pe-enforce-ready.service new file mode 100644 index 00000000..f188e129 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-pe-enforce-ready.service @@ -0,0 +1,15 @@ +[Unit] +Description=Release the enrolled hold only after Phantom Engine is enforce-ready +DefaultDependencies=no +After=vantio-pe-loader.service vantio-boot-hold.service +Requires=vantio-pe-loader.service + +[Service] +Type=oneshot +RemainAfterExit=yes +TimeoutStartSec=180 +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart=/usr/bin/python3 -m vantio_install.boot_hold release --require-enforce-ready --wait-seconds 120 + +[Install] +WantedBy=multi-user.target diff --git a/packages/vantio-install/packaging/boot-hold/vantio-pe-loader.service b/packages/vantio-install/packaging/boot-hold/vantio-pe-loader.service new file mode 100644 index 00000000..99182755 --- /dev/null +++ b/packages/vantio-install/packaging/boot-hold/vantio-pe-loader.service @@ -0,0 +1,16 @@ +[Unit] +Description=Start Phantom Engine as early as the enrolled hold allows +DefaultDependencies=no +After=local-fs.target vantio-boot-hold.service docker.service +Wants=docker.service +Before=vantio-pe-enforce-ready.service + +[Service] +Type=simple +Restart=no +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart=/usr/bin/python3 -m vantio_install.boot_hold start-loader +ExecStop=/bin/kill -TERM $MAINPID + +[Install] +WantedBy=multi-user.target diff --git a/packages/vantio-install/pyproject.toml b/packages/vantio-install/pyproject.toml index ddf1d156..d0669451 100644 --- a/packages/vantio-install/pyproject.toml +++ b/packages/vantio-install/pyproject.toml @@ -32,6 +32,7 @@ classifiers = [ [project.scripts] vantio-install = "vantio_install.cli:main" vantio-verify = "vantio_install.verifier:main" +vantio-boot-hold = "vantio_install.boot_hold.cli:main" # Sealed sdist file set. Independent of a git checkout. The source launcher # under bin/ is omitted on purpose. diff --git a/packages/vantio-install/tests/test_boot_hold.py b/packages/vantio-install/tests/test_boot_hold.py new file mode 100644 index 00000000..51142fe0 --- /dev/null +++ b/packages/vantio-install/tests/test_boot_hold.py @@ -0,0 +1,620 @@ +"""Boot-hold tests. No EC2 and no live packet filter.""" + +from __future__ import annotations + +import json +import os +import shutil +import stat +import subprocess +import sys +import tempfile +import unittest +from contextlib import redirect_stdout +from io import StringIO +from pathlib import Path + +PACKAGE = Path(__file__).resolve().parents[1] +REPO = PACKAGE.parents[1] +sys.path.insert(0, str(PACKAGE)) + +from vantio_install.boot_hold.cli import main # noqa: E402 +from vantio_install.boot_hold.constants import BPF_PINS, HELD_MESSAGE, SLICE # noqa: E402 +from vantio_install.boot_hold.errors import BootHoldError # noqa: E402 +from vantio_install.boot_hold.files import allow_profile, deny_profile # noqa: E402 +from vantio_install.boot_hold.graph import graph_errors, parse_unit # noqa: E402 +from vantio_install.boot_hold.identity import Caller # noqa: E402 +from vantio_install.boot_hold.net import commands_are_scoped # noqa: E402 +from vantio_install.boot_hold.policy import _trusted, default_policy, load_policy, save_policy # noqa: E402 +from vantio_install.boot_hold.readiness import evaluate_ready, perform_deny_self_check, probe_loader # noqa: E402 +from vantio_install.boot_hold.service import ( # noqa: E402 + apply_boot, + configure, + enable_from_apply, + enroll_compose, + enroll_docker, + enroll_systemd, + loader_argv, + observe_unenrolled, + opt_out, + release, + status_body, +) +from vantio_install.boot_hold.units import static_units # noqa: E402 +from vantio_install.constants import PROOF_CEILING, PROOF_STATE # noqa: E402 + +READY = { + "bpf_pins": list(BPF_PINS), + "loader_cmdline": "/vantio-loader --iface ens5 --enforce --cgroup-skb-enforce", + "loader_health": "OK", + "bpf_programs": "cgroup_skb_egress_enforce tag abc", + "enforcement_attachment": { + "attached": True, + "program": "cgroup_skb_egress_enforce", + "cgroup": "/sys/fs/cgroup/vantio-enrolled.slice", + }, + "policy_loaded": True, + "deny_self_check": { + "attempted": True, + "enrolled_denied": True, + "unenrolled_allowed": True, + "mechanism": "phantom-engine", + "hold_bypassed": True, + }, +} +NOT_READY = {"bpf_pins": [], "loader_cmdline": "", "loader_health": "", "bpf_programs": ""} + + +class Rec: + def __init__(self) -> None: + self.calls: list[list[str]] = [] + + def __call__(self, argv: list[str]) -> int: + self.calls.append(list(argv)) + if len(argv) >= 2 and argv[1] in {"-C", "-N", "-D", "-X"}: + return 1 + if "inspect" in argv: + return 1 + if argv[:2] == ["/usr/sbin/apparmor_parser", "-V"]: + return 1 + if argv[:2] == ["apparmor_parser", "-V"]: + return 0 if shutil.which("apparmor_parser") else 1 + if argv[:3] == ["systemctl", "is-active", "stay-down.service"]: + return 3 + return 0 + + +def root_caller() -> Caller: + return Caller(0, 10, "vantio-boot-hold", "0::/system.slice/ssh.service\n", "systemd") + + +def enrolled_root() -> Caller: + return Caller(0, 11, "agent", f"0::/{SLICE}/my-agent.service\n", "systemd") + + +def user_caller() -> Caller: + return Caller(1000, 12, "user", "0::/user.slice/user.service\n", "systemd") + + +def container_root() -> Caller: + return Caller(0, 13, "agent", "0::/\n", "agent") + + +class BootHoldTest(unittest.TestCase): + def setUp(self) -> None: + self.root = Path(tempfile.mkdtemp(prefix="vantio-boot-hold-")) + self.addCleanup(lambda: shutil.rmtree(self.root, ignore_errors=True)) + self.runner = Rec() + self.caller = root_caller() + self.python = "/usr/bin/python3" + + def test_missing_config_defaults_on(self) -> None: + policy, notes = load_policy(self.root) + self.assertTrue(policy["enabled"]) + self.assertTrue(policy["hold"]) + self.assertTrue(policy["ordering"]) + self.assertTrue(any("defaults" in note or "absent" in note or "stays on" in note for note in notes)) + body = status_body(self.root) + self.assertEqual(body["reboot_row"], "NOT_PROVED") + self.assertEqual(body["proof_state"], PROOF_STATE) + self.assertEqual(body["proof_ceiling"], PROOF_CEILING) + self.assertFalse(body["host_wide_default_route_hold"]) + self.assertEqual(body["state"], "CONFIGURED") + + def test_world_writable_opt_out_is_ignored(self) -> None: + save_policy(self.root, {"enabled": False, "hold": False, "ordering": False}) + path = self.root / "etc/vantio/boot-hold.json" + os.chmod(path, 0o666) + policy, notes = load_policy(self.root) + self.assertTrue(policy["enabled"]) + self.assertTrue(policy["hold"]) + self.assertTrue(any("ignored" in note for note in notes)) + + def test_non_root_owned_config_is_not_trusted_on_the_host_root(self) -> None: + path = self.root / "boot-hold.json" + path.write_text("{}\n", encoding="utf-8") + os.chmod(path, 0o644) + self.assertFalse(_trusted(Path("/"), path)) + self.assertNotEqual(path.stat().st_uid, 0) + + def test_enroll_and_unenrolled_lookup(self) -> None: + enroll_systemd(self.root, self.caller, "my-agent.service", ["/var/lib/app/secrets"], self.python) + observe_unenrolled(self.root, self.caller, "control", "systemd") + apply_boot(self.root, self.caller, self.runner, self.python) + body = status_body(self.root, lookup="nobody") + by_id = {row["id"]: row for row in body["workloads"]} + self.assertEqual(by_id["my-agent.service"]["protection"], "HELD") + self.assertEqual(by_id["control"]["protection"], "UNPROTECTED") + self.assertEqual(by_id["control"]["enrolled"], "no") + self.assertEqual(by_id["nobody"]["protection"], "UNPROTECTED") + self.assertEqual(body["state"], "HELD") + self.assertEqual(body["health"], "DEGRADED") + self.assertIn("SSH", body["message"]) + dropin = (self.root / "etc/systemd/system/my-agent.service.d/vantio-boot-hold.conf").read_text(encoding="utf-8") + self.assertIn("Slice=vantio-enrolled.slice", dropin) + self.assertIn("InaccessiblePaths=/var/lib/app/secrets", dropin) + self.assertIn("Requires=vantio-boot-hold.service vantio-pe-enforce-ready.service", dropin) + + def test_protected_paths_and_recovery_units(self) -> None: + for bad in ["/", "/etc/ssh", "/root/.ssh", "/var/lib", "relative/secrets", "/var/lib/app/../secrets"]: + with self.assertRaises(BootHoldError): + enroll_systemd(self.root, self.caller, "my-agent.service", [bad], self.python) + with self.assertRaises(BootHoldError): + enroll_systemd(self.root, self.caller, "ssh.service", ["/var/lib/app/secrets"], self.python) + + def test_packet_rules_are_enrolled_only(self) -> None: + apply_boot(self.root, self.caller, self.runner, self.python) + blob = [" ".join(call) for call in self.runner.calls] + joined = "\n".join(blob) + self.assertIn("--path vantio-enrolled.slice", joined) + self.assertIn("-s 10.250.250.0/24", joined) + self.assertIn("-s fd76:616e:7469::/64", joined) + self.assertEqual(commands_are_scoped(self.runner.calls), []) + self.assertNotIn("route", joined) + self.assertNotIn("blackhole", joined) + self.assertNotIn("0.0.0.0/0", joined) + for call in self.runner.calls: + self.assertNotIn("-P", call) + + def test_release_requires_ready_or_break_glass(self) -> None: + apply_boot(self.root, self.caller, Rec(), self.python) + fresh = Rec() + with self.assertRaises(BootHoldError): + release( + self.root, + self.caller, + fresh, + require_ready=False, + break_glass=False, + operator_flag=False, + facts_probe=lambda: NOT_READY, + ) + self.assertFalse(any(call[1:2] == ["-D"] for call in fresh.calls)) + audit = (self.root / "var/lib/vantio/boot-hold/audit.log").read_text(encoding="utf-8") + self.assertIn("REFUSED", audit) + + def test_loader_failure_stays_held(self) -> None: + apply_boot(self.root, self.caller, Rec(), self.python) + fresh = Rec() + with self.assertRaises(BootHoldError) as caught: + release( + self.root, + self.caller, + fresh, + require_ready=True, + break_glass=False, + operator_flag=False, + facts_probe=lambda: NOT_READY, + wait_seconds=0, + ) + self.assertIn("held", str(caught.exception).lower()) + body = status_body(self.root) + self.assertEqual(body["state"], "HELD") + self.assertEqual(body["health"], "DEGRADED") + self.assertIn(HELD_MESSAGE, body["message"]) + self.assertFalse(any(len(call) > 1 and call[1] == "-D" for call in fresh.calls)) + + def test_ready_release_and_break_glass(self) -> None: + enroll_systemd(self.root, self.caller, "stay-down.service", ["/var/lib/app/secrets"], self.python) + apply_boot(self.root, self.caller, Rec(), self.python) + fresh = Rec() + body = release( + self.root, + self.caller, + fresh, + require_ready=True, + break_glass=False, + operator_flag=False, + facts_probe=lambda: READY, + ) + self.assertEqual(body["state"], "RELEASED") + self.assertEqual(body["reboot_row"], "NOT_PROVED") + self.assertEqual(body["audit_reason"], "enforce-ready") + self.assertTrue(any(call[1:2] == ["-D"] for call in fresh.calls)) + self.assertEqual(commands_are_scoped(fresh.calls), []) + dropin = (self.root / "etc/systemd/system/stay-down.service.d/vantio-boot-hold.conf").read_text(encoding="utf-8") + self.assertNotIn("InaccessiblePaths=", dropin) + self.assertIn("Requires=vantio-boot-hold.service vantio-pe-enforce-ready.service", dropin) + again = Rec() + glass = release( + self.root, + self.caller, + again, + require_ready=False, + break_glass=True, + operator_flag=True, + facts_probe=lambda: NOT_READY, + ) + self.assertEqual(glass["audit_reason"], "BREAK_GLASS") + self.assertEqual(glass["health"], "DEGRADED") + self.assertIn("packet-hold-ipv4", glass["released"]) + self.assertIn("packet-hold-ipv6", glass["released"]) + self.assertIn("file-hold", glass["released"]) + self.assertIn("start-gate:stay-down.service", glass["released"]) + cleared = (self.root / "etc/systemd/system/stay-down.service.d/vantio-boot-hold.conf").read_text(encoding="utf-8") + self.assertNotIn("vantio-pe-enforce-ready.service", cleared) + self.assertNotIn("Requires=", cleared) + docker_unit = (self.root / "etc/systemd/system/vantio-enrolled-docker@.service").read_text(encoding="utf-8") + self.assertNotIn("vantio-pe-enforce-ready.service", docker_unit) + audit = (self.root / "var/lib/vantio/boot-hold/audit.log").read_text(encoding="utf-8") + self.assertIn("start-gate:stay-down.service", audit) + self.assertIn("packet-hold-ipv4", audit) + self.assertEqual(status_body(self.root)["reboot_row"], "NOT_PROVED") + with self.assertRaises(BootHoldError): + release( + self.root, + self.caller, + Rec(), + require_ready=False, + break_glass=True, + operator_flag=False, + facts_probe=lambda: NOT_READY, + ) + + def test_enrolled_and_non_root_release_refused(self) -> None: + apply_boot(self.root, self.caller, Rec(), self.python) + for caller in (enrolled_root(), user_caller(), container_root()): + with self.assertRaises(BootHoldError): + release( + self.root, + caller, + Rec(), + require_ready=False, + break_glass=True, + operator_flag=True, + facts_probe=lambda: READY, + ) + audit = (self.root / "var/lib/vantio/boot-hold/audit.log").read_text(encoding="utf-8") + self.assertGreaterEqual(audit.count('"result": "REFUSED"'), 3) + self.assertNotIn('"result": "BREAK_GLASS"', audit) + self.assertEqual(status_body(self.root)["state"], "HELD") + + def test_hold_alone_ordering_alone_and_both(self) -> None: + enroll_systemd(self.root, self.caller, "my-agent.service", ["/var/lib/app/secrets"], self.python) + configure(self.root, self.caller, self.runner, hold=True, ordering=False, python=self.python) + dropin = (self.root / "etc/systemd/system/my-agent.service.d/vantio-boot-hold.conf").read_text(encoding="utf-8") + parsed = parse_unit(dropin) + requires = parsed["Unit"].get("Requires", []) + self.assertIn("vantio-boot-hold.service", requires) + self.assertNotIn("vantio-pe-enforce-ready.service", requires) + self.assertIn("InaccessiblePaths=/var/lib/app/secrets", dropin) + self.assertTrue(any(call[1:2] == ["-I"] for call in self.runner.calls)) + units = static_units(self.python, {"enabled": True, "hold": True, "ordering": False, **{k: default_policy()[k] for k in ("cgroup_slice", "enrolled_subnet_v4", "enrolled_subnet_v6")}}) + self.assertEqual(graph_errors(units, ordering=False, hold=True), []) + + ordering_runner = Rec() + configure(self.root, self.caller, ordering_runner, hold=False, ordering=True, python=self.python) + self.assertFalse(any(call[1:2] == ["-I"] for call in ordering_runner.calls)) + dropin = (self.root / "etc/systemd/system/my-agent.service.d/vantio-boot-hold.conf").read_text(encoding="utf-8") + self.assertNotIn("InaccessiblePaths=", dropin) + self.assertIn("vantio-pe-enforce-ready.service", parse_unit(dropin)["Unit"].get("Requires", [])) + off_units = static_units(self.python, load_policy(self.root)[0]) + self.assertEqual(graph_errors(off_units, ordering=True, hold=False), []) + + both = Rec() + configure(self.root, self.caller, both, hold=True, ordering=True, python=self.python) + self.assertTrue(any("--path" in call and "vantio-enrolled.slice" in call for call in both.calls)) + both_units = static_units(self.python, default_policy()) + self.assertEqual(graph_errors(both_units, ordering=True, hold=True), []) + with self.assertRaises(BootHoldError): + configure(self.root, self.caller, Rec(), hold=False, ordering=False, python=self.python) + + def test_docker_restart_policy_and_cgroup_parent(self) -> None: + with self.assertRaises(BootHoldError): + enroll_docker( + self.root, + self.caller, + "agent", + "always", + "/vantio-enrolled.slice", + ["/var/lib/app/secrets"], + self.python, + set_restart_no=False, + runner=Rec(), + ) + body = enroll_docker( + self.root, + self.caller, + "agent", + "always", + "/vantio-enrolled.slice", + ["/var/lib/app/secrets"], + self.python, + set_restart_no=True, + runner=self.runner, + ) + self.assertEqual(body["state"], "ENROLLED") + self.assertIn(["docker", "update", "--restart=no", "agent"], self.runner.calls) + unit = (self.root / "etc/systemd/system/vantio-enrolled-docker@.service").read_text(encoding="utf-8") + self.assertIn("Requires=docker.service vantio-enrolled-network.service vantio-boot-hold.service vantio-pe-enforce-ready.service", unit) + self.assertIn("ExecStart=/usr/bin/docker start %i", unit) + with self.assertRaises(BootHoldError): + enroll_docker( + self.root, + self.caller, + "other", + "no", + "/system.slice", + [], + self.python, + set_restart_no=False, + runner=Rec(), + ) + + def test_compose_enroll_refuses_restart_always(self) -> None: + with self.assertRaises(BootHoldError): + enroll_compose(self.root, self.caller, "/var/lib/app/compose", "restart: always\n", self.python) + text = "\n".join( + [ + "services:", + " agent:", + " restart: \"no\"", + " cgroup_parent: /vantio-enrolled.slice", + " security_opt:", + " - apparmor:vantio-boot-hold", + "networks:", + " enrolled:", + " subnet: 10.250.250.0/24", + ] + ) + body = enroll_compose(self.root, self.caller, "/var/lib/app/compose", text, self.python) + self.assertEqual(body["state"], "ENROLLED") + unit = (self.root / "etc/systemd/system/vantio-enrolled-compose-compose.service").read_text(encoding="utf-8") + self.assertIn("ExecStart=/usr/bin/docker compose start", unit) + self.assertIn("vantio-pe-enforce-ready.service", parse_unit(unit)["Unit"].get("Requires", [])) + + def test_packaged_units_match_the_renderer(self) -> None: + rendered = static_units("/usr/bin/python3", default_policy()) + mapping = { + "etc/systemd/system/vantio-boot-hold.service": "vantio-boot-hold.service", + "etc/systemd/system/vantio-pe-loader.service": "vantio-pe-loader.service", + "etc/systemd/system/vantio-pe-enforce-ready.service": "vantio-pe-enforce-ready.service", + "etc/systemd/system/vantio-enrolled.slice": "vantio-enrolled.slice", + "etc/systemd/system/vantio-enrolled-docker@.service": "vantio-enrolled-docker@.service", + "etc/systemd/system/vantio-enrolled-network.service": "vantio-enrolled-network.service", + "etc/systemd/system/docker.service.d/vantio-boot-hold.conf": "docker.service.d-vantio-boot-hold.conf", + "etc/systemd/system/containerd.service.d/vantio-boot-hold.conf": "containerd.service.d-vantio-boot-hold.conf", + } + base = PACKAGE / "packaging/boot-hold" + for rel, name in mapping.items(): + self.assertEqual((base / name).read_text(encoding="utf-8"), rendered[rel], name) + + def test_graph_keeps_ssh_off_the_hold(self) -> None: + units = static_units(self.python, default_policy()) + self.assertEqual(graph_errors(units), []) + hold = units["etc/systemd/system/vantio-boot-hold.service"] + self.assertIn("Before=docker.service containerd.service", hold) + self.assertIn("Environment=VANTIO_BOOT_HOLD_LIVE=1", hold) + self.assertIn("Environment=VANTIO_BOOT_HOLD_LIVE=1", units["etc/systemd/system/vantio-pe-loader.service"]) + self.assertIn("Environment=VANTIO_BOOT_HOLD_LIVE=1", units["etc/systemd/system/vantio-pe-enforce-ready.service"]) + self.assertNotIn("ssh.service", hold) + self.assertNotIn("systemd-networkd.service", hold) + docker_dropin = parse_unit(units["etc/systemd/system/docker.service.d/vantio-boot-hold.conf"]) + self.assertEqual(docker_dropin["Unit"].get("Requires", []), []) + ready = units["etc/systemd/system/vantio-pe-enforce-ready.service"] + self.assertIn("--require-enforce-ready", ready) + self.assertIn("Requires=vantio-pe-loader.service", ready) + + def test_loader_argv_missing_does_not_release(self) -> None: + apply_boot(self.root, self.caller, Rec(), self.python) + with self.assertRaises(BootHoldError) as caught: + loader_argv(self.root) + self.assertIn("stay held", str(caught.exception)) + self.assertEqual(status_body(self.root)["state"], "HELD") + + def test_opt_out_is_logged_and_visible(self) -> None: + apply_boot(self.root, self.caller, Rec(), self.python) + body = opt_out(self.root, self.caller, Rec(), "maintenance window", self.python) + self.assertEqual(body["state"], "OPTED_OUT") + self.assertEqual(status_body(self.root)["health"], "OPTED_OUT") + audit = (self.root / "var/lib/vantio/boot-hold/audit.log").read_text(encoding="utf-8") + self.assertIn("opt-out", audit) + self.assertIn("maintenance window", audit) + mode = stat.S_IMODE((self.root / "etc/vantio/boot-hold.json").stat().st_mode) + self.assertEqual(mode & 0o022, 0) + + def test_running_loader_without_attachment_stays_held(self) -> None: + unattached = dict(READY) + unattached["enforcement_attachment"] = {"attached": False, "program": "", "cgroup": ""} + verdict = evaluate_ready(unattached) + self.assertFalse(verdict["enforce_ready"]) + self.assertEqual(verdict["reason"], "loader-running-not-attached") + self.assertTrue(verdict["loader_up"]) + self.assertTrue(verdict["program_loaded"]) + apply_boot(self.root, self.caller, Rec(), self.python) + with self.assertRaises(BootHoldError) as caught: + release( + self.root, + self.caller, + Rec(), + require_ready=True, + break_glass=False, + operator_flag=False, + facts_probe=lambda: unattached, + ) + self.assertIn("not attached", str(caught.exception)) + self.assertEqual(status_body(self.root)["state"], "HELD") + no_deny = dict(READY) + no_deny["deny_self_check"] = dict(READY["deny_self_check"]) + no_deny["deny_self_check"]["enrolled_denied"] = False + self.assertEqual(evaluate_ready(no_deny)["reason"], "deny-self-check-failed") + self.assertFalse(evaluate_ready(no_deny)["enforce_ready"]) + + def test_deny_self_check_requires_both_paths(self) -> None: + calls: list[list[str]] = [] + + def runner(argv: list[str]) -> int: + calls.append(list(argv)) + if argv[:2] == ["iptables", "-I"]: + return 0 + if "--slice" in argv and "vantio-enrolled.slice" in argv: + return 1 + if "--slice" in argv and "system.slice" in argv: + return 0 + return 0 + + result = perform_deny_self_check(runner, host="192.0.2.1", port=443, attached=True) + self.assertTrue(result["hold_bypassed"]) + self.assertTrue(result["enrolled_denied"]) + self.assertTrue(result["unenrolled_allowed"]) + self.assertTrue(any(call[0] == "iptables" and "-D" in call for call in calls)) + self.assertTrue(any("--path" in call and "vantio-enrolled.slice" in call for call in calls)) + before = len(calls) + skipped = perform_deny_self_check(runner, host="192.0.2.1", port=443, attached=False) + self.assertFalse(skipped["hold_bypassed"]) + self.assertEqual(len(calls), before) + + def test_apply_enables_hold_and_keeps_opt_out(self) -> None: + body = enable_from_apply(self.root, self.caller, self.runner, self.python) + self.assertEqual(body["state"], "HELD") + unit = self.root / "etc/systemd/system/vantio-boot-hold.service" + link = self.root / "etc/systemd/system/sysinit.target.wants/vantio-boot-hold.service" + self.assertTrue(unit.is_file()) + self.assertTrue(link.is_symlink()) + self.assertTrue(any("--path" in call and "vantio-enrolled.slice" in call for call in self.runner.calls)) + opted = self.root / "opt-out-host" + save_policy(opted, {"enabled": False, "hold": True, "ordering": True}) + quiet = Rec() + again = enable_from_apply(opted, self.caller, quiet, self.python) + self.assertNotEqual(again["state"], "HELD") + self.assertFalse(any(len(call) > 1 and call[1] == "-I" for call in quiet.calls)) + self.assertFalse(json.loads((opted / "etc/vantio/boot-hold.json").read_text(encoding="utf-8"))["enabled"]) + self.assertEqual(status_body(opted)["health"], "OPTED_OUT") + + def test_evaluate_ready_rejects_partial_facts(self) -> None: + self.assertTrue(evaluate_ready(READY)["enforce_ready"]) + almost = dict(READY) + almost["bpf_programs"] = "" + self.assertFalse(evaluate_ready(almost)["enforce_ready"]) + almost = dict(READY) + almost["loader_cmdline"] = "/vantio-loader --iface ens5" + self.assertFalse(evaluate_ready(almost)["enforce_ready"]) + proc = self.root / "proc/4242" + proc.mkdir(parents=True) + (proc / "cmdline").write_bytes(b"/vantio-loader\x00--enforce\x00") + (proc / "stat").write_text("4242 (vantio-loader) S 1 1 1\n", encoding="utf-8") + cmdline, health = probe_loader(self.root) + self.assertIn("--enforce", cmdline) + self.assertEqual(health, "OK") + + def test_profile_denies_only_the_protected_path(self) -> None: + text = deny_profile(["/var/lib/app/secrets"]) + self.assertIn("deny /var/lib/app/secrets rwmlkx,", text) + self.assertNotIn("/etc/ssh", text) + self.assertIn("profile vantio-boot-hold", allow_profile()) + self.assertNotIn("deny ", allow_profile()) + if shutil.which("apparmor_parser"): + path = self.root / "profile" + path.write_text(text, encoding="utf-8") + completed = subprocess.run( + ["apparmor_parser", "-Q", "-K", "-T", str(path)], + check=False, + capture_output=True, + text=True, + ) + self.assertEqual(completed.returncode, 0, completed.stderr) + + def test_docs_and_lab_procedure(self) -> None: + doc = (PACKAGE / "docs/BOOT-HOLD.md").read_text(encoding="utf-8") + for needle in ( + "--cgroup-parent=/vantio-enrolled.slice", + "--restart=no", + "apparmor=vantio-boot-hold", + "10.250.250.0/24", + "Slice=vantio-enrolled.slice", + "vantio-pe-enforce-ready.service", + "docker compose start", + "NOT_PROVED", + "--break-glass --i-am-root-operator", + "observe-unenrolled", + ): + self.assertIn(needle, doc) + self.assertNotIn("/home/vantioai", doc) + lab = (REPO / "docs/planning/boot-hold/FD-REBOOT-1-LAB-PROCEDURE.md").read_text(encoding="utf-8") + for needle in ( + "960577828987", + "expected_oop_usd=0", + "NOT_PROVED", + "Hold alone", + "Ordering alone", + "Both together", + "5+", + "from t=0", + "BREAK_GLASS", + "restart", + "unenrolled", + "SSH", + "Unknown is never PASS", + "e0b19d55", + "INTERNAL_CLEAN_HOST_PROOF", + "t3.small", + ): + self.assertIn(needle, lab) + companion = (REPO / "docs/planning/boot-hold/PE-COMPANION.md").read_text(encoding="utf-8") + self.assertIn("resolve_cgroup_spec", companion) + self.assertIn("/sys/fs/cgroup/vantio-enrolled.slice", companion) + self.assertIn("does not invent a seal", companion) + + def test_cli_refuses_enrolled_identity(self) -> None: + fixture = self.root / "caller.json" + fixture.write_text( + json.dumps( + { + "euid": 0, + "pid": 50, + "comm": "agent", + "cgroup_text": f"0::/{SLICE}/job.service", + "pid1_comm": "systemd", + } + ), + encoding="utf-8", + ) + env_keys = { + "VANTIO_BOOT_HOLD_ALLOW_CALLER_FIXTURE": "1", + "VANTIO_BOOT_HOLD_CALLER_FIXTURE": str(fixture), + } + previous = {key: os.environ.get(key) for key in env_keys} + os.environ.update(env_keys) + self.addCleanup(lambda: _restore_env(previous)) + buffer = StringIO() + with redirect_stdout(buffer): + code = main(["release", "--root", str(self.root), "--break-glass", "--i-am-root-operator"]) + self.assertNotEqual(code, 0) + payload = json.loads(buffer.getvalue()) + self.assertEqual(payload["state"], "FAILED_SAFE") + self.assertEqual(payload["reboot_row"], "NOT_PROVED") + self.assertIn("enrolled workload", payload["message"]) + audit = (self.root / "var/lib/vantio/boot-hold/audit.log").read_text(encoding="utf-8") + self.assertIn("REFUSED", audit) + + +def _restore_env(previous: dict[str, str | None]) -> None: + for key, value in previous.items(): + if value is None: + os.environ.pop(key, None) + else: + os.environ[key] = value + + +if __name__ == "__main__": + unittest.main() diff --git a/packages/vantio-install/tests/test_live_executor.py b/packages/vantio-install/tests/test_live_executor.py index 77ba949b..89086a13 100644 --- a/packages/vantio-install/tests/test_live_executor.py +++ b/packages/vantio-install/tests/test_live_executor.py @@ -168,7 +168,7 @@ def verify(self, op_type: str, grant) -> str: return "VERIFIED" if not self.loaded and self.tagged is None else "NOT_VERIFIED" if op_type == "tc_clsact_del": return "VERIFIED" if grant.iface not in self.clsact else "NOT_VERIFIED" - if op_type.startswith("remove_"): + if op_type.startswith("remove_") or op_type in {"install_boot_hold", "remove_boot_hold"}: return "VERIFIED" return "NOT_VERIFIED" diff --git a/packages/vantio-install/tests/test_stage_a.py b/packages/vantio-install/tests/test_stage_a.py index 5d2f5afe..30fdac74 100644 --- a/packages/vantio-install/tests/test_stage_a.py +++ b/packages/vantio-install/tests/test_stage_a.py @@ -318,6 +318,46 @@ def test_apply_healthy_and_idempotent(self) -> None: self.assertTrue(replay.get("replayed")) self.assertEqual(len(harness.snapshot()["containers"]), 1) + def test_apply_installs_and_enables_boot_hold(self) -> None: + harness = self.make() + plan_code, plan_body = harness.run("plan") + self.assertEqual(plan_code, 0, plan_body) + code, body = harness.run("apply", yes=True) + self.assertEqual(code, 0, body) + self.assertEqual(body["state"], "HEALTHY") + self.assertEqual(body["enforcement"], "NOT_ENABLED") + layout = harness.state / "boot-hold-host" + self.assertTrue((layout / "etc/systemd/system/vantio-boot-hold.service").is_file()) + self.assertTrue((layout / "etc/systemd/system/sysinit.target.wants/vantio-boot-hold.service").is_symlink()) + health = json.loads(harness.tx_file("HEALTH.json").read_text(encoding="utf-8")) + self.assertEqual(health["reboot_row"], "NOT_PROVED") + self.assertEqual(health["boot_hold"]["state"], "HELD") + self.assertEqual(health["boot_hold"]["reboot_row"], "NOT_PROVED") + joined = " ".join(" ".join(item) for item in harness.snapshot()["boot_hold"]["commands"]) + self.assertIn("--path vantio-enrolled.slice", joined) + self.assertNotIn("0.0.0.0/0", joined) + + def test_apply_honors_boot_hold_opt_out(self) -> None: + harness = self.make() + layout = harness.state / "boot-hold-host" + config = layout / "etc/vantio/boot-hold.json" + config.parent.mkdir(parents=True) + config.write_text( + json.dumps({"enabled": False, "hold": True, "ordering": True, "cgroup_slice": "vantio-enrolled.slice", "enrolled_subnet_v4": "10.250.250.0/24", "enrolled_subnet_v6": "fd76:616e:7469::/64"}) + + "\n", + encoding="utf-8", + ) + os.chmod(config, 0o644) + code, body = harness.run("plan") + self.assertEqual(code, 0, body) + code, body = harness.run("apply", yes=True) + self.assertEqual(code, 0, body) + health = json.loads(harness.tx_file("HEALTH.json").read_text(encoding="utf-8")) + self.assertEqual(health["boot_hold"]["health"], "OPTED_OUT") + self.assertFalse(json.loads(config.read_text(encoding="utf-8"))["enabled"]) + joined = " ".join(" ".join(item) for item in harness.snapshot()["boot_hold"]["commands"]) + self.assertNotIn(" -I ", f" {joined} ") + def test_interrupted_resume_and_digest_mismatch(self) -> None: harness = self.make() harness.run("plan") diff --git a/packages/vantio-install/vantio_install/boot_hold/__init__.py b/packages/vantio-install/vantio_install/boot_hold/__init__.py new file mode 100644 index 00000000..a5d49c5f --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/__init__.py @@ -0,0 +1,9 @@ +"""Early boot hold for enrolled workloads. + +The hold is a host mechanism. It does not read Phantom Engine BPF pins to +install itself, and it does not mark the reboot exposure row proved. +""" + +from vantio_install.boot_hold.constants import REBOOT_ROW + +__all__ = ["REBOOT_ROW"] diff --git a/packages/vantio-install/vantio_install/boot_hold/__main__.py b/packages/vantio-install/vantio_install/boot_hold/__main__.py new file mode 100644 index 00000000..77f56fac --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/__main__.py @@ -0,0 +1,6 @@ +"""python -m vantio_install.boot_hold""" + +from vantio_install.boot_hold.cli import main + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/packages/vantio-install/vantio_install/boot_hold/audit.py b/packages/vantio-install/vantio_install/boot_hold/audit.py new file mode 100644 index 00000000..d6a52a9c --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/audit.py @@ -0,0 +1,33 @@ +"""Append-only audit log for hold changes.""" + +from __future__ import annotations + +import json +import os +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +from vantio_install.boot_hold.constants import AUDIT_REL +from vantio_install.boot_hold.identity import Caller + + +def append_audit(root: Path, caller: Caller, action: str, result: str, detail: dict[str, Any] | None = None) -> None: + path = root / AUDIT_REL + path.parent.mkdir(parents=True, exist_ok=True) + event = { + "action": action, + "result": result, + "at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), + "euid": caller.euid, + "pid": caller.pid, + "comm": caller.comm, + "pid1_comm": caller.pid1_comm, + "cgroup": caller.cgroup_text.strip().replace("\n", " | "), + "detail": detail or {}, + } + with path.open("a", encoding="utf-8") as handle: + handle.write(json.dumps(event, sort_keys=True) + "\n") + os.chmod(path, 0o600) + if root == Path("/"): + os.chown(path, 0, 0) diff --git a/packages/vantio-install/vantio_install/boot_hold/cli.py b/packages/vantio-install/vantio_install/boot_hold/cli.py new file mode 100644 index 00000000..9b1029a1 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/cli.py @@ -0,0 +1,257 @@ +"""CLI for the enrolled boot hold. Stdout is one JSON object.""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path + +from vantio_install.boot_hold.errors import BootHoldError +from vantio_install.boot_hold.graph import graph_errors +from vantio_install.boot_hold.identity import read_host_caller +from vantio_install.boot_hold.policy import default_policy +from vantio_install.boot_hold.policy import load_policy +from vantio_install.boot_hold.readiness import assemble_host_facts, default_bpftool, probe_host +from vantio_install.boot_hold.service import ( + apply_boot, + configure, + enroll_compose, + enroll_docker, + enroll_systemd, + ensure_network, + envelope, + install, + loader_argv, + observe_unenrolled, + opt_in, + opt_out, + release, + start_loader, + status_body, + live_boot_hold_runner, +) +from vantio_install.boot_hold.units import static_units +from vantio_install import constants as install_constants + + +class RecordingRunner: + """Runs host commands only when the live gate is set. Otherwise it records them.""" + + def __init__(self, live: bool) -> None: + self.live = live + self.calls: list[list[str]] = [] + + def __call__(self, argv: list[str]) -> int: + self.calls.append(list(argv)) + if self.live: + completed = subprocess.run(argv, check=False) + return int(completed.returncode) + if len(argv) >= 2 and argv[1] in {"-C", "-N", "-D", "-X"}: + return 1 + if argv and argv[0] == "docker": + return 1 + if "inspect" in argv: + return 1 + if argv[:2] == ["/usr/sbin/apparmor_parser", "-V"]: + return 1 + if argv[:2] == ["apparmor_parser", "-V"]: + return 0 if shutil.which("apparmor_parser") else 1 + return 0 + + +def _live(root: Path) -> bool: + return root == Path("/") and os.geteuid() == 0 and os.environ.get("VANTIO_BOOT_HOLD_LIVE") == "1" + + +def _capture(argv: list[str]) -> str: + try: + completed = subprocess.run(argv, check=False, capture_output=True, text=True, timeout=8) + except (OSError, subprocess.TimeoutExpired): + return "" + if completed.returncode != 0: + return "" + return completed.stdout or "" + + +def _facts_probe(root: Path, facts_path: str | None): + def probe() -> dict: + if facts_path and os.environ.get("VANTIO_BOOT_HOLD_ALLOW_FACTS") == "1": + data = json.loads(Path(facts_path).read_text(encoding="utf-8")) + if not isinstance(data, dict): + return {} + return data + if not _live(root): + return probe_host(root, bpftool=default_bpftool) + policy, _notes = load_policy(root) + return assemble_host_facts( + root, + prog_show=_capture(["bpftool", "prog", "show"]), + cgroup_show=_capture(["bpftool", "cgroup", "show", "/sys/fs/cgroup/vantio-enrolled.slice"]), + map_show=_capture(["bpftool", "map", "show"]), + runner=live_boot_hold_runner, + deny_probe=policy.get("deny_probe") if isinstance(policy.get("deny_probe"), dict) else None, + ) + + return probe + + +def _parser() -> argparse.ArgumentParser: + parent = argparse.ArgumentParser(add_help=False) + parent.add_argument("--root", default="/") + parent.add_argument("--json", action="store_true") + parser = argparse.ArgumentParser(prog="vantio-boot-hold") + sub = parser.add_subparsers(dest="command", required=True) + + for name in ("install", "apply-boot", "start-loader", "ensure-network", "opt-in", "graph-check"): + sub.add_parser(name, parents=[parent]) + + status = sub.add_parser("status", parents=[parent]) + status.add_argument("--lookup", default="") + + release_cmd = sub.add_parser("release", parents=[parent]) + release_cmd.add_argument("--require-enforce-ready", action="store_true") + release_cmd.add_argument("--break-glass", action="store_true") + release_cmd.add_argument("--i-am-root-operator", action="store_true") + release_cmd.add_argument("--wait-seconds", type=int, default=0) + release_cmd.add_argument("--facts", default="") + + enroll_unit = sub.add_parser("enroll-systemd", parents=[parent]) + enroll_unit.add_argument("--unit", required=True) + enroll_unit.add_argument("--protected-path", action="append", default=[]) + + enroll_d = sub.add_parser("enroll-docker", parents=[parent]) + enroll_d.add_argument("--name", required=True) + enroll_d.add_argument("--restart-policy", default="no") + enroll_d.add_argument("--cgroup-parent", default="") + enroll_d.add_argument("--protected-path", action="append", default=[]) + enroll_d.add_argument("--set-restart-no", action="store_true") + + enroll_c = sub.add_parser("enroll-compose", parents=[parent]) + enroll_c.add_argument("--project-dir", required=True) + enroll_c.add_argument("--compose-file", required=True) + + observe = sub.add_parser("observe-unenrolled", parents=[parent]) + observe.add_argument("--id", required=True) + observe.add_argument("--kind", required=True) + + opt = sub.add_parser("opt-out", parents=[parent]) + opt.add_argument("--reason", required=True) + + config = sub.add_parser("configure", parents=[parent]) + config.add_argument("--hold", required=True, choices=["on", "off"]) + config.add_argument("--ordering", required=True, choices=["on", "off"]) + return parser + + +def _dump(payload: dict) -> int: + if payload.get("proof_state") in install_constants.FORBIDDEN_PROOF_STATES: + payload["proof_state"] = install_constants.PROOF_STATE + payload["state"] = "FAILED_SAFE" + payload["reboot_row"] = "NOT_PROVED" + json.dump(payload, sys.stdout, indent=2, sort_keys=True) + sys.stdout.write("\n") + if payload.get("state") in {"FAILED_SAFE", "UNKNOWN"}: + return 4 + if payload.get("state") == "HELD" and payload.get("command") == "release": + return 4 + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = _parser() + try: + args = parser.parse_args(argv) + except SystemExit as exc: + code = exc.code + return int(code) if isinstance(code, int) else 10 + root = Path(args.root) + caller = read_host_caller() + runner = RecordingRunner(_live(root)) + python = sys.executable or "/usr/bin/python3" + command = args.command + try: + if command == "graph-check": + units = static_units(python, default_policy()) + errors = graph_errors(units, ordering=True, hold=True) + payload = envelope( + "graph-check", + "FAILED_SAFE" if errors else "OK", + "Dependency graph check failed." if errors else "Boot hold ordering keeps SSH, Docker, and enrolled workloads in the required order.", + errors=errors, + ) + return _dump(payload) if not errors else (_dump(payload) or 1) + if command == "status": + payload = status_body(root, lookup=args.lookup or None) + elif command == "install": + payload = install(root, caller, python) + elif command == "apply-boot": + payload = apply_boot(root, caller, runner, python) + elif command == "start-loader": + payload = start_loader(root, caller) + if _live(root): + argv_exec = loader_argv(root) + os.execv(argv_exec[0], argv_exec) + elif command == "ensure-network": + payload = ensure_network(root, caller, runner) + elif command == "release": + payload = release( + root, + caller, + runner, + require_ready=bool(args.require_enforce_ready), + break_glass=bool(args.break_glass), + operator_flag=bool(args.i_am_root_operator), + facts_probe=_facts_probe(root, args.facts or None), + wait_seconds=int(args.wait_seconds), + python=python, + ) + elif command == "enroll-systemd": + payload = enroll_systemd(root, caller, args.unit, list(args.protected_path), python) + elif command == "enroll-docker": + payload = enroll_docker( + root, + caller, + args.name, + args.restart_policy, + args.cgroup_parent, + list(args.protected_path), + python, + set_restart_no=bool(args.set_restart_no), + runner=runner, + ) + elif command == "enroll-compose": + text = Path(args.compose_file).read_text(encoding="utf-8") + payload = enroll_compose(root, caller, args.project_dir, text, python) + elif command == "observe-unenrolled": + payload = observe_unenrolled(root, caller, args.id, args.kind) + elif command == "opt-out": + payload = opt_out(root, caller, runner, args.reason, python) + elif command == "opt-in": + payload = opt_in(root, caller, runner, python) + elif command == "configure": + payload = configure( + root, + caller, + runner, + hold=args.hold == "on", + ordering=args.ordering == "on", + python=python, + ) + else: + payload = envelope(command, "FAILED_SAFE", "Unknown command.") + return _dump(payload) or 10 + except BootHoldError as exc: + payload = envelope(command, exc.state, str(exc)) + json.dump(payload, sys.stdout, indent=2, sort_keys=True) + sys.stdout.write("\n") + return exc.exit_code + except Exception as exc: # noqa: BLE001 — last-resort crash envelope + payload = envelope(command, "FAILED_SAFE", exc.__class__.__name__) + json.dump(payload, sys.stdout, indent=2, sort_keys=True) + sys.stdout.write("\n") + return install_constants.EXIT_CRASH + return _dump(payload) diff --git a/packages/vantio-install/vantio_install/boot_hold/constants.py b/packages/vantio-install/vantio_install/boot_hold/constants.py new file mode 100644 index 00000000..550cec73 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/constants.py @@ -0,0 +1,58 @@ +"""Pins for the enrolled-only boot hold.""" + +from __future__ import annotations + +from vantio_install import constants as install_constants + +REBOOT_ROW = "NOT_PROVED" +PROOF_STATE = install_constants.PROOF_STATE +PROOF_CEILING = install_constants.PROOF_CEILING + +SLICE = "vantio-enrolled.slice" +CGROUP_ROOT = "/sys/fs/cgroup" +SLICE_PATH = f"{CGROUP_ROOT}/{SLICE}" +CHAIN = "VANTIO_BOOT_HOLD" +SUBNET_V4 = "10.250.250.0/24" +SUBNET_V6 = "fd76:616e:7469::/64" +NETWORK_NAME = "vantio-enrolled" +APPARMOR_PROFILE = "vantio-boot-hold" +ENFORCE_PROGRAM = "cgroup_skb_egress_enforce" +BPF_PINS = install_constants.BPF_PINS + +STATE_DIR = "var/lib/vantio/boot-hold" +CONFIG_REL = "etc/vantio/boot-hold.json" +REGISTRY_REL = f"{STATE_DIR}/registry.json" +AUDIT_REL = f"{STATE_DIR}/audit.log" +HEALTH_REL = "run/vantio/boot-hold-health.json" +HOLD_STATE_REL = f"{STATE_DIR}/hold-state.json" +LOADER_ARGV_REL = "etc/vantio/pe-loader.argv.json" +PROFILE_REL = "etc/apparmor.d/vantio-boot-hold" +DROPIN_NAME = "vantio-boot-hold.conf" + +HELD_MESSAGE = ( + "Enrolled workloads are held. Phantom Engine is not enforce-ready, so the hold stays on. " + "SSH and the console stay up. A root operator on the host can release with " + "`vantio-boot-hold release --break-glass --i-am-root-operator`. " + "An enrolled workload cannot release this hold." +) + +RECOVERY_UNITS = frozenset( + { + "ssh.service", + "sshd.service", + "ssh.socket", + "systemd-networkd.service", + "systemd-networkd-wait-online.service", + "systemd-resolved.service", + "systemd-timesyncd.service", + "dhcpcd.service", + "amazon-ssm-agent.service", + "snap.amazon-ssm-agent.amazon-ssm-agent.service", + "docker.service", + "containerd.service", + "vantio-boot-hold.service", + "vantio-pe-loader.service", + "vantio-pe-enforce-ready.service", + "vantio-enrolled-network.service", + } +) diff --git a/packages/vantio-install/vantio_install/boot_hold/errors.py b/packages/vantio-install/vantio_install/boot_hold/errors.py new file mode 100644 index 00000000..536af253 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/errors.py @@ -0,0 +1,10 @@ +"""Errors for the boot-hold command.""" + +from __future__ import annotations + + +class BootHoldError(Exception): + def __init__(self, message: str, *, exit_code: int = 4, state: str = "FAILED_SAFE") -> None: + super().__init__(message) + self.exit_code = exit_code + self.state = state diff --git a/packages/vantio-install/vantio_install/boot_hold/files.py b/packages/vantio-install/vantio_install/boot_hold/files.py new file mode 100644 index 00000000..84347f3e --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/files.py @@ -0,0 +1,72 @@ +"""Protected-path hold for enrolled workloads. + +Systemd units get InaccessiblePaths while the hold is up. Docker and systemd +both name the AppArmor profile vantio-boot-hold. Boot loads the deny profile. +Release replaces that same profile with an allow profile so a later start can +see the paths, and Phantom Engine policy is the control after enforce-ready. +""" + +from __future__ import annotations + +from vantio_install.boot_hold.constants import APPARMOR_PROFILE + + +def deny_profile(paths: list[str]) -> str: + lines = [ + f"profile {APPARMOR_PROFILE} flags=(attach_disconnected) {{", + " file,", + " network,", + " capability,", + " signal,", + " unix,", + ] + for path in paths: + lines.append(f" deny {path} rwmlkx,") + lines.append(f" deny {path}/** rwmlkx,") + lines.append("}") + lines.append("") + return "\n".join(lines) + + +def allow_profile() -> str: + return ( + f"profile {APPARMOR_PROFILE} flags=(attach_disconnected) {{\n" + " file,\n" + " network,\n" + " capability,\n" + " signal,\n" + " unix,\n" + "}\n" + ) + + +def dropin_text( + *, + hold: bool, + ordering: bool, + protected_paths: list[str], + apparmor: bool, + hide_paths: bool, + start_gate: bool = True, +) -> str: + after: list[str] = [] + requires: list[str] = [] + if start_gate and hold: + after.append("vantio-boot-hold.service") + requires.append("vantio-boot-hold.service") + if start_gate and ordering: + after.append("vantio-pe-enforce-ready.service") + requires.append("vantio-pe-enforce-ready.service") + lines = ["[Unit]", "Description=Vantio enrolled workload gate"] + if after: + lines.append("After=" + " ".join(after)) + if requires: + lines.append("Requires=" + " ".join(requires)) + lines.extend(["", "[Service]", "Slice=vantio-enrolled.slice"]) + if hide_paths: + for path in protected_paths: + lines.append(f"InaccessiblePaths={path}") + if apparmor: + lines.append(f"AppArmorProfile={APPARMOR_PROFILE}") + lines.append("") + return "\n".join(lines) diff --git a/packages/vantio-install/vantio_install/boot_hold/graph.py b/packages/vantio-install/vantio_install/boot_hold/graph.py new file mode 100644 index 00000000..93f4ccb1 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/graph.py @@ -0,0 +1,95 @@ +"""Dependency checks for the boot-hold units.""" + +from __future__ import annotations + +_RECOVERY = ( + "ssh.service", + "sshd.service", + "ssh.socket", + "systemd-networkd.service", + "systemd-resolved.service", + "systemd-networkd-wait-online.service", + "dhcpcd.service", + "amazon-ssm-agent.service", + "snap.amazon-ssm-agent.amazon-ssm-agent.service", +) + +_HOST_WIDE = ("ip route", "route del", "blackhole", "-P OUTPUT DROP", "-P FORWARD DROP", "0.0.0.0/0", "::/0") + + +def parse_unit(text: str) -> dict[str, dict[str, list[str]]]: + sections: dict[str, dict[str, list[str]]] = {} + section: str | None = None + for raw in text.splitlines(): + line = raw.split("#", 1)[0].strip() + if not line: + continue + if line.startswith("[") and line.endswith("]"): + section = line[1:-1] + sections.setdefault(section, {}) + continue + if section is None or "=" not in line: + continue + key, value = line.split("=", 1) + bucket = sections[section].setdefault(key.strip(), []) + bucket.extend(value.split()) + return sections + + +def _deps(parsed: dict[str, dict[str, list[str]]], key: str) -> list[str]: + found: list[str] = [] + for section in parsed.values(): + found.extend(section.get(key, [])) + return found + + +def graph_errors(units: dict[str, str], *, ordering: bool = True, hold: bool = True) -> list[str]: + errors: list[str] = [] + for name, text in units.items(): + for needle in _HOST_WIDE: + if needle in text: + errors.append(f"{name} contains {needle}") + hold_text = units.get("etc/systemd/system/vantio-boot-hold.service", "") + hold_parsed = parse_unit(hold_text) + before = set(_deps(hold_parsed, "Before")) + if hold and "docker.service" not in before: + errors.append("boot hold is not Before=docker.service") + if hold and "containerd.service" not in before: + errors.append("boot hold is not Before=containerd.service") + for key in ("Before", "Requires", "Wants", "After"): + for dep in _deps(hold_parsed, key): + if dep in _RECOVERY: + errors.append(f"boot hold {key} includes {dep}") + loader = parse_unit(units.get("etc/systemd/system/vantio-pe-loader.service", "")) + if "vantio-boot-hold.service" not in _deps(loader, "After"): + errors.append("Phantom Engine loader is not After=vantio-boot-hold.service") + if "vantio-pe-enforce-ready.service" not in _deps(loader, "Before"): + errors.append("Phantom Engine loader is not Before=vantio-pe-enforce-ready.service") + ready = parse_unit(units.get("etc/systemd/system/vantio-pe-enforce-ready.service", "")) + if "vantio-pe-loader.service" not in _deps(ready, "After"): + errors.append("enforce-ready is not After= the loader") + if "vantio-pe-loader.service" not in _deps(ready, "Requires"): + errors.append("enforce-ready does not Requires= the loader") + if "--require-enforce-ready" not in units.get("etc/systemd/system/vantio-pe-enforce-ready.service", ""): + errors.append("enforce-ready unit does not require the enforce-ready probe") + docker_unit = units.get("etc/systemd/system/vantio-enrolled-docker@.service", "") + docker_parsed = parse_unit(docker_unit) + if ordering and "vantio-pe-enforce-ready.service" not in _deps(docker_parsed, "Requires"): + errors.append("enrolled docker unit does not Requires= enforce-ready") + if not ordering and "vantio-pe-enforce-ready.service" in _deps(docker_parsed, "Requires"): + errors.append("ordering-off docker unit still Requires= enforce-ready") + if hold and "vantio-boot-hold.service" not in _deps(docker_parsed, "Requires"): + errors.append("enrolled docker unit does not Requires= the boot hold") + docker_dropin = units.get("etc/systemd/system/docker.service.d/vantio-boot-hold.conf", "") + dropin_parsed = parse_unit(docker_dropin) + if "vantio-boot-hold.service" not in _deps(dropin_parsed, "After"): + errors.append("docker drop-in is not After= the boot hold") + if _deps(dropin_parsed, "Requires"): + errors.append("docker.service must not Requires= the boot hold") + if SLICE_MISSING(units.get("etc/systemd/system/vantio-enrolled.slice", "")): + errors.append("enrolled slice unit is missing") + return errors + + +def SLICE_MISSING(text: str) -> bool: + return "[Slice]" not in text diff --git a/packages/vantio-install/vantio_install/boot_hold/identity.py b/packages/vantio-install/vantio_install/boot_hold/identity.py new file mode 100644 index 00000000..d30b2f35 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/identity.py @@ -0,0 +1,52 @@ +"""Who may change the boot hold.""" + +from __future__ import annotations + +import json +import os +from dataclasses import dataclass +from pathlib import Path + +from vantio_install.boot_hold.constants import SLICE +from vantio_install.boot_hold.errors import BootHoldError + + +@dataclass(frozen=True) +class Caller: + euid: int + pid: int + comm: str + cgroup_text: str + pid1_comm: str + + def refusal(self) -> str | None: + if self.euid != 0: + return "This action needs a root operator on the host." + if self.pid1_comm not in {"systemd", "init"}: + return "This action was refused because the caller is outside the host init namespace." + if SLICE in self.cgroup_text: + return "This action was refused because the caller is an enrolled workload." + return None + + +def read_host_caller() -> Caller: + fixture = os.environ.get("VANTIO_BOOT_HOLD_CALLER_FIXTURE") + if os.environ.get("VANTIO_BOOT_HOLD_ALLOW_CALLER_FIXTURE") == "1" and fixture: + data = json.loads(Path(fixture).read_text(encoding="utf-8")) + return Caller( + euid=int(data["euid"]), + pid=int(data.get("pid", 1)), + comm=str(data.get("comm", "vantio-boot-hold")), + cgroup_text=str(data.get("cgroup_text", "0::/system.slice/ssh.service")), + pid1_comm=str(data.get("pid1_comm", "systemd")), + ) + cgroup = Path("/proc/self/cgroup").read_text(encoding="utf-8", errors="replace") + pid1 = Path("/proc/1/comm").read_text(encoding="utf-8", errors="replace").strip() + comm = Path("/proc/self/comm").read_text(encoding="utf-8", errors="replace").strip() + return Caller(euid=os.geteuid(), pid=os.getpid(), comm=comm or "unknown", cgroup_text=cgroup, pid1_comm=pid1 or "unknown") + + +def require_operator(caller: Caller) -> None: + reason = caller.refusal() + if reason: + raise BootHoldError(reason, exit_code=4, state="FAILED_SAFE") diff --git a/packages/vantio-install/vantio_install/boot_hold/net.py b/packages/vantio-install/vantio_install/boot_hold/net.py new file mode 100644 index 00000000..176585ef --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/net.py @@ -0,0 +1,95 @@ +"""Enrolled-only packet hold. No default-route change and no host-wide policy drop.""" + +from __future__ import annotations + +from collections.abc import Callable + +from vantio_install.boot_hold.constants import CHAIN +from vantio_install.boot_hold.errors import BootHoldError + +Runner = Callable[[list[str]], int] + +_FORBIDDEN = ( + "route", + "blackhole", + "0.0.0.0/0", + "::/0", + "-P", + "iptables-restore", + "ip6tables-restore", +) + + +def families(policy: dict) -> tuple[tuple[str, str], ...]: + return ( + ("iptables", str(policy["enrolled_subnet_v4"])), + ("ip6tables", str(policy["enrolled_subnet_v6"])), + ) + + +def commands_are_scoped(calls: list[list[str]]) -> list[str]: + """Return problems if a command would hold the whole host.""" + + problems: list[str] = [] + for argv in calls: + blob = " ".join(argv) + for token in _FORBIDDEN: + if token in blob: + problems.append(f"command contains {token!r}: {blob}") + if "FORWARD" in argv or "OUTPUT" in argv: + scoped = "cgroup" in argv or "-s" in argv + if not scoped and "-D" not in argv and "-F" not in argv and "-X" not in argv and "-N" not in argv: + problems.append(f"filter jump is not scoped to a cgroup or enrolled subnet: {blob}") + return problems + + +def _required(runner: Runner, argv: list[str]) -> None: + if runner(argv) != 0: + raise BootHoldError(f"Host command failed: {' '.join(argv)}") + + +def install_hold(runner: Runner, policy: dict) -> list[list[str]]: + issued: list[list[str]] = [] + + def run(argv: list[str]) -> int: + issued.append(list(argv)) + return runner(argv) + + slice_path = str(policy["cgroup_slice"]) + for binary, subnet in families(policy): + run([binary, "-N", CHAIN]) + _required(run, [binary, "-F", CHAIN]) + _required(run, [binary, "-A", CHAIN, "-j", "DROP"]) + check_out = [binary, "-C", "OUTPUT", "-m", "cgroup", "--path", slice_path, "-j", CHAIN] + if run(check_out) != 0: + _required(run, [binary, "-I", "OUTPUT", "1", "-m", "cgroup", "--path", slice_path, "-j", CHAIN]) + check_fwd = [binary, "-C", "FORWARD", "-s", subnet, "-j", CHAIN] + if run(check_fwd) != 0: + _required(run, [binary, "-I", "FORWARD", "1", "-s", subnet, "-j", CHAIN]) + problems = commands_are_scoped(issued) + if problems: + raise BootHoldError("Refusing a host-wide hold. " + problems[0]) + return issued + + +def remove_hold(runner: Runner, policy: dict) -> list[list[str]]: + issued: list[list[str]] = [] + + def run(argv: list[str]) -> int: + issued.append(list(argv)) + return runner(argv) + + slice_path = str(policy["cgroup_slice"]) + for binary, subnet in families(policy): + for _ in range(8): + if run([binary, "-D", "OUTPUT", "-m", "cgroup", "--path", slice_path, "-j", CHAIN]) != 0: + break + for _ in range(8): + if run([binary, "-D", "FORWARD", "-s", subnet, "-j", CHAIN]) != 0: + break + run([binary, "-F", CHAIN]) + run([binary, "-X", CHAIN]) + problems = commands_are_scoped(issued) + if problems: + raise BootHoldError("Refusing a host-wide hold change. " + problems[0]) + return issued diff --git a/packages/vantio-install/vantio_install/boot_hold/policy.py b/packages/vantio-install/vantio_install/boot_hold/policy.py new file mode 100644 index 00000000..4733160d --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/policy.py @@ -0,0 +1,106 @@ +"""Boot-hold configuration. Missing config means the hold is on.""" + +from __future__ import annotations + +import json +import os +from pathlib import Path + +from vantio_install.boot_hold.constants import CONFIG_REL, SLICE, SUBNET_V4, SUBNET_V6 +from vantio_install.boot_hold.errors import BootHoldError + + +def default_policy() -> dict: + return { + "enabled": True, + "hold": True, + "ordering": True, + "cgroup_slice": SLICE, + "enrolled_subnet_v4": SUBNET_V4, + "enrolled_subnet_v6": SUBNET_V6, + } + + +def config_path(root: Path) -> Path: + return root / CONFIG_REL + + +def _trusted(root: Path, path: Path) -> bool: + mode = path.stat().st_mode + if mode & 0o022: + return False + if root == Path("/"): + return path.stat().st_uid == 0 + return True + + +def load_policy(root: Path) -> tuple[dict, list[str]]: + """Return the policy and operator notes. + + A missing file, an unreadable file, or a file that is not a trusted + root-owned config keeps the hold on. Opt-out is honored only from a + trusted file that sets ``enabled`` to false. + """ + + path = config_path(root) + if not path.exists(): + return default_policy(), ["Boot hold config is absent, so the hold stays on."] + if not _trusted(root, path): + return default_policy(), [ + "Boot hold config is not a root-owned file with mode 0644 or stricter. Opt-out is ignored and the hold stays on." + ] + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError: + return default_policy(), ["Boot hold config is not valid JSON. Opt-out is ignored and the hold stays on."] + if not isinstance(data, dict): + return default_policy(), ["Boot hold config must be a JSON object. The hold stays on."] + policy = default_policy() + if data.get("enabled") is False: + policy["enabled"] = False + if data.get("hold") is False: + policy["hold"] = False + if data.get("ordering") is False: + policy["ordering"] = False + probe = data.get("deny_probe") + if isinstance(probe, dict) and isinstance(probe.get("host"), str) and probe.get("host") and isinstance(probe.get("port"), int): + if probe["host"] not in {"0.0.0.0", "::", "0.0.0.0/0", "::/0"} and 0 < int(probe["port"]) < 65536: + policy["deny_probe"] = {"host": probe["host"], "port": int(probe["port"])} + notes: list[str] = [] + if policy["enabled"] is False: + notes.append("A root admin opted out of the boot hold.") + elif policy["hold"] is False or policy["ordering"] is False: + notes.append("A root admin changed hold or ordering. The change is explicit.") + return policy, notes + + +def save_policy(root: Path, policy: dict) -> None: + path = config_path(root) + path.parent.mkdir(parents=True, exist_ok=True) + body = default_policy() + body["enabled"] = bool(policy.get("enabled", True)) + body["hold"] = bool(policy.get("hold", True)) + body["ordering"] = bool(policy.get("ordering", True)) + if isinstance(policy.get("deny_probe"), dict): + body["deny_probe"] = policy["deny_probe"] + path.write_text(json.dumps(body, indent=2, sort_keys=True) + "\n", encoding="utf-8") + os.chmod(path, 0o644) + if root == Path("/"): + os.chown(path, 0, 0) + + +def mechanisms_active(policy: dict) -> tuple[bool, bool]: + """Return (hold_on, ordering_on) after the opt-out switch.""" + + if not policy.get("enabled", True): + return False, False + return bool(policy.get("hold", True)), bool(policy.get("ordering", True)) + + +def require_explicit_matrix(hold: bool, ordering: bool) -> None: + if not hold and not ordering: + raise BootHoldError( + "Turning the hold and ordering off together is an opt-out. Use `vantio-boot-hold opt-out --reason ...`.", + exit_code=10, + state="FAILED_SAFE", + ) diff --git a/packages/vantio-install/vantio_install/boot_hold/readiness.py b/packages/vantio-install/vantio_install/boot_hold/readiness.py new file mode 100644 index 00000000..778a3ea6 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/readiness.py @@ -0,0 +1,276 @@ +"""Enforce-ready probe. Unknown facts are not ready.""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + +from vantio_install.boot_hold.constants import BPF_PINS, ENFORCE_PROGRAM, SLICE + + +def _attachment(facts: dict) -> dict: + raw = facts.get("enforcement_attachment") + return raw if isinstance(raw, dict) else {} + + +def _self_check(facts: dict) -> dict: + raw = facts.get("deny_self_check") + return raw if isinstance(raw, dict) else {} + + +def evaluate_ready(facts: dict) -> dict: + """Ready only when enforcement is attached and a live deny check passed. + + A running loader is not enough. A loaded program that is not attached to + the enrolled cgroup keeps the hold. + """ + + pins = list(facts.get("bpf_pins") or []) + missing = [name for name in BPF_PINS if name not in pins] + cmdline = str(facts.get("loader_cmdline") or "") + tokens = cmdline.split() + loader_up = "vantio-loader" in cmdline and "--enforce" in tokens + programs = str(facts.get("bpf_programs") or "") + program_loaded = ENFORCE_PROGRAM in programs + attachment = _attachment(facts) + cgroup = str(attachment.get("cgroup") or "").rstrip("/") + attached = ( + attachment.get("attached") is True + and attachment.get("program") == ENFORCE_PROGRAM + and cgroup.endswith(SLICE) + ) + policy_loaded = facts.get("policy_loaded") is True and not missing + health_ok = str(facts.get("loader_health") or "") == "OK" + check = _self_check(facts) + live_deny = ( + check.get("attempted") is True + and check.get("enrolled_denied") is True + and check.get("unenrolled_allowed") is True + and check.get("mechanism") == "phantom-engine" + and check.get("hold_bypassed") is True + ) + ready = bool(loader_up and program_loaded and attached and policy_loaded and health_ok and live_deny) + if ready: + reason = "enforce-ready" + elif loader_up and program_loaded and not attached: + reason = "loader-running-not-attached" + elif loader_up and not live_deny: + reason = "deny-self-check-failed" + else: + reason = "not-enforce-ready" + return { + "enforce_ready": ready, + "missing_pins": missing, + "loader_up": loader_up, + "program_loaded": program_loaded, + "program_attached": attached, + "policy_loaded": policy_loaded, + "loader_health_ok": health_ok, + "live_deny": live_deny, + "reason": reason, + } + + +def interpret_self_check(*, attached: bool, enrolled_rc: int | None, unenrolled_rc: int | None, hold_bypassed: bool) -> dict: + """Turn probe exit codes into a deny-check record. Unknown stays not-ready.""" + + if not attached or enrolled_rc is None or unenrolled_rc is None or not hold_bypassed: + return { + "attempted": bool(attached and hold_bypassed and enrolled_rc is not None), + "enrolled_denied": False, + "unenrolled_allowed": False, + "mechanism": "phantom-engine", + "hold_bypassed": bool(hold_bypassed), + } + return { + "attempted": True, + "enrolled_denied": enrolled_rc != 0, + "unenrolled_allowed": unenrolled_rc == 0, + "mechanism": "phantom-engine", + "hold_bypassed": True, + } + + +def probe_pins(root: Path) -> list[str]: + base = root / "sys/fs/bpf" + found: list[str] = [] + if not base.is_dir(): + return found + for name in BPF_PINS: + path = base / name + if not path.exists() or path.is_symlink() or path.is_dir(): + continue + found.append(name) + return found + + +def probe_loader(root: Path) -> tuple[str, str]: + proc = root / "proc" + if not proc.is_dir(): + return "", "" + cmdline = "" + health = "" + for entry in proc.iterdir(): + if not entry.name.isdigit(): + continue + raw_path = entry / "cmdline" + try: + raw = raw_path.read_bytes() + except OSError: + continue + text = raw.replace(b"\x00", b" ").decode("utf-8", errors="replace") + if "vantio-loader" not in text: + continue + cmdline = text.strip() + health = _proc_health(entry / "stat") + break + return cmdline, health + + +def _proc_health(stat_path: Path) -> str: + try: + text = stat_path.read_text(encoding="utf-8", errors="replace") + except OSError: + return "" + # comm is inside parentheses; state is the next field. + end = text.rfind(")") + if end < 0 or end + 2 >= len(text): + return "" + state = text[end + 2 :].split() + if not state: + return "" + if state[0] in {"R", "S", "D"}: + return "OK" + return state[0] + + +def probe_programs(bpftool) -> str: + if bpftool is None: + return "" + try: + return bpftool() + except OSError: + return "" + + +def attachment_from_show(text: str) -> dict: + attached = ENFORCE_PROGRAM in text + return { + "attached": attached, + "program": ENFORCE_PROGRAM if attached else "", + "cgroup": f"/sys/fs/cgroup/{SLICE}" if attached else "", + } + + +def policy_from_maps(text: str, pins_complete: bool) -> bool: + return bool(pins_complete and "vantio_enforce" in text) + + +def _connect_script(host: str, port: int) -> str: + return ( + "import socket,sys\n" + "s=socket.socket()\n" + "s.settimeout(2)\n" + "try:\n" + f" s.connect(({host!r}, {int(port)}))\n" + "except OSError:\n" + " sys.exit(1)\n" + "s.close()\n" + ) + + +def connect_argv(host: str, port: int, *, enrolled: bool) -> list[str]: + argv = ["systemd-run", "--quiet", "--wait", "--pipe", "--collect"] + argv.extend(["--slice", SLICE if enrolled else "system.slice"]) + argv.extend(["python3", "-c", _connect_script(host, port)]) + return argv + + +def exception_argv(binary: str, host: str, port: int, *, delete: bool) -> list[str]: + action = ["-D", "VANTIO_BOOT_HOLD"] if delete else ["-I", "VANTIO_BOOT_HOLD", "1"] + return [binary, *action, "-p", "tcp", "-d", host, "--dport", str(port), "-m", "cgroup", "--path", SLICE, "-j", "RETURN"] + + +def perform_deny_self_check(runner, *, host: str, port: int, attached: bool) -> dict: + """Prove a deny on the enrolled path after the boot-hold rule returns for that one flow. + + The exception is one destination and one port, only for the enrolled cgroup, + and it is removed before this function returns. If enforcement is not + attached, the exception is not inserted. + """ + + if not attached or not host or not port: + return interpret_self_check(attached=False, enrolled_rc=None, unenrolled_rc=None, hold_bypassed=False) + binary = "ip6tables" if ":" in host else "iptables" + inserted = runner(exception_argv(binary, host, port, delete=False)) == 0 + if not inserted: + return interpret_self_check(attached=True, enrolled_rc=None, unenrolled_rc=None, hold_bypassed=False) + enrolled_rc = 1 + unenrolled_rc = 1 + try: + enrolled_rc = runner(connect_argv(host, port, enrolled=True)) + unenrolled_rc = runner(connect_argv(host, port, enrolled=False)) + finally: + runner(exception_argv(binary, host, port, delete=True)) + return interpret_self_check( + attached=True, + enrolled_rc=enrolled_rc, + unenrolled_rc=unenrolled_rc, + hold_bypassed=True, + ) + + +def assemble_host_facts( + root: Path, + *, + prog_show: str, + cgroup_show: str, + map_show: str, + runner=None, + deny_probe: dict | None = None, +) -> dict: + pins = probe_pins(root) + cmdline, health = probe_loader(root) + attachment = attachment_from_show(cgroup_show) + pins_complete = all(name in pins for name in BPF_PINS) + policy_loaded = policy_from_maps(map_show, pins_complete) + check = interpret_self_check(attached=False, enrolled_rc=None, unenrolled_rc=None, hold_bypassed=False) + if attachment["attached"] and runner is not None and isinstance(deny_probe, dict): + check = perform_deny_self_check( + runner, + host=str(deny_probe.get("host") or ""), + port=int(deny_probe.get("port") or 0), + attached=True, + ) + return { + "bpf_pins": pins, + "loader_cmdline": cmdline, + "loader_health": health, + "bpf_programs": prog_show, + "enforcement_attachment": attachment, + "policy_loaded": policy_loaded, + "deny_self_check": check, + } + + +def probe_host(root: Path, bpftool=None) -> dict: + cmdline, health = probe_loader(root) + return { + "bpf_pins": probe_pins(root), + "loader_cmdline": cmdline, + "loader_health": health, + "bpf_programs": probe_programs(bpftool), + } + + +def default_bpftool() -> str: + completed = subprocess.run( + ["bpftool", "prog", "show"], + check=False, + capture_output=True, + text=True, + timeout=8, + ) + if completed.returncode != 0: + return "" + return completed.stdout or "" diff --git a/packages/vantio-install/vantio_install/boot_hold/registry.py b/packages/vantio-install/vantio_install/boot_hold/registry.py new file mode 100644 index 00000000..2c65b221 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/registry.py @@ -0,0 +1,174 @@ +"""Persistent enrollment registry. It survives reboot and does not use BPF pins.""" + +from __future__ import annotations + +import json +import os +import re +from pathlib import Path + +from vantio_install.boot_hold.constants import RECOVERY_UNITS, REGISTRY_REL, SLICE +from vantio_install.boot_hold.errors import BootHoldError + +_UNIT_NAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9:_.@\\-]{0,200}\.service$") +_DOCKER_NAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.-]{0,120}$") +_RESTART_BAD = re.compile(r"(?im)^\s*restart\s*:\s*(always|unless-stopped|on-failure\b)") + +_BLOCKED_EXACT = frozenset( + { + "/", + "/etc", + "/usr", + "/bin", + "/sbin", + "/lib", + "/lib64", + "/boot", + "/root", + "/home", + "/var", + "/var/lib", + "/sys", + "/proc", + "/dev", + "/run", + "/tmp", + "/etc/ssh", + "/root/.ssh", + } +) +_BLOCKED_PREFIXES = ( + "/etc/", + "/usr/", + "/bin/", + "/sbin/", + "/lib/", + "/lib64/", + "/boot/", + "/sys/", + "/proc/", + "/dev/", + "/run/", + "/root/.ssh/", + "/etc/ssh/", +) + + +def registry_path(root: Path) -> Path: + return root / REGISTRY_REL + + +def empty_registry() -> dict: + return {"version": 1, "workloads": []} + + +def load_registry(root: Path) -> dict: + path = registry_path(root) + if not path.exists(): + return empty_registry() + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise BootHoldError("The enrollment registry is not valid JSON. Status is UNKNOWN.", state="UNKNOWN") from exc + if not isinstance(data, dict) or not isinstance(data.get("workloads"), list): + raise BootHoldError("The enrollment registry is unreadable. Status is UNKNOWN.", state="UNKNOWN") + return data + + +def save_registry(root: Path, registry: dict) -> None: + path = registry_path(root) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(registry, indent=2, sort_keys=True) + "\n", encoding="utf-8") + os.chmod(path, 0o600) + if root == Path("/"): + os.chown(path, 0, 0) + os.chown(path.parent, 0, 0) + os.chmod(path.parent, 0o700) + + +def validate_protected_path(raw: str) -> str: + if not isinstance(raw, str) or not raw.startswith("/") or "\x00" in raw: + raise BootHoldError( + "A protected path is an absolute workload directory, three or more levels deep.", + exit_code=10, + ) + if ".." in Path(raw).parts: + raise BootHoldError("Choose a specific workload directory without dot segments.", exit_code=10) + path = os.path.normpath(raw) + if path != raw.rstrip("/"): + raise BootHoldError("Choose a specific workload directory without dot segments.", exit_code=10) + parts = [part for part in path.split("/") if part] + if path in _BLOCKED_EXACT or any(path.startswith(prefix) for prefix in _BLOCKED_PREFIXES): + raise BootHoldError( + "Choose a workload directory outside SSH, system, and recovery paths.", + exit_code=10, + ) + if ".ssh" in parts: + raise BootHoldError("Choose a workload directory outside SSH paths.", exit_code=10) + if len(parts) < 3: + raise BootHoldError( + "Choose a specific workload directory at least three levels deep.", + exit_code=10, + ) + return path + + +def validate_unit_name(unit: str) -> str: + if not _UNIT_NAME.match(unit): + raise BootHoldError("The systemd unit name is not usable for enrollment.", exit_code=10) + if unit in RECOVERY_UNITS or unit.startswith("ssh"): + raise BootHoldError( + f"{unit} stays outside the enrolled slice so SSH, DNS, DHCP, and host agents keep working.", + exit_code=10, + ) + return unit + + +def validate_docker_name(name: str) -> str: + if not _DOCKER_NAME.match(name): + raise BootHoldError("The container name is not usable for enrollment.", exit_code=10) + return name + + +def restart_is_gated(policy: str | None) -> None: + text = (policy or "no").strip() + if text in {"", "no", "none"}: + return + raise BootHoldError( + "Docker restart policy must be `no` for an enrolled container. " + "A restart policy of always, unless-stopped, or on-failure starts the container with the daemon, " + "ahead of enforce-ready. Set it with `docker update --restart=no ` " + "and let `vantio-enrolled-docker@.service` start it.", + exit_code=10, + ) + + +def compose_text_gated(text: str) -> None: + if _RESTART_BAD.search(text): + raise BootHoldError( + "Compose restart is `always`, `unless-stopped`, or `on-failure`. " + "Set `restart: \"no\"` so the systemd unit starts the project after enforce-ready.", + exit_code=10, + ) + required = ( + ("cgroup_parent", "Set `cgroup_parent: /vantio-enrolled.slice` on the enrolled service."), + (SLICE, "Point `cgroup_parent` at /vantio-enrolled.slice."), + ("vantio-boot-hold", "Set `security_opt: apparmor:vantio-boot-hold` on the enrolled service."), + ("10.250.250.0/24", "Attach the service to a network whose subnet is 10.250.250.0/24."), + ) + for token, message in required: + if token not in text: + raise BootHoldError(message, exit_code=10) + + +def upsert(registry: dict, record: dict) -> dict: + workloads = [row for row in registry.get("workloads", []) if row.get("id") != record["id"]] + workloads.append(record) + return {"version": 1, "workloads": workloads} + + +def find_workload(registry: dict, name: str) -> dict | None: + for row in registry.get("workloads") or []: + if row.get("id") == name or row.get("unit") == name or row.get("container") == name: + return row + return None diff --git a/packages/vantio-install/vantio_install/boot_hold/service.py b/packages/vantio-install/vantio_install/boot_hold/service.py new file mode 100644 index 00000000..e5f6e5cb --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/service.py @@ -0,0 +1,810 @@ +"""Boot-hold operations. Filesystem edits stay under the layout root.""" + +from __future__ import annotations + +import json +import os +import subprocess +import time +from collections.abc import Callable +from pathlib import Path +from typing import Any + +from vantio_install.boot_hold.audit import append_audit +from vantio_install.boot_hold.constants import ( + DROPIN_NAME, + HEALTH_REL, + HELD_MESSAGE, + HOLD_STATE_REL, + LOADER_ARGV_REL, + NETWORK_NAME, + PROFILE_REL, + PROOF_CEILING, + PROOF_STATE, + REBOOT_ROW, + SLICE, + SLICE_PATH, +) +from vantio_install.boot_hold.errors import BootHoldError +from vantio_install.boot_hold.files import allow_profile, deny_profile, dropin_text +from vantio_install.boot_hold.identity import Caller, require_operator +from vantio_install.boot_hold.net import install_hold, remove_hold +from vantio_install.boot_hold.policy import ( + config_path, + load_policy, + mechanisms_active, + require_explicit_matrix, + save_policy, +) +from vantio_install.boot_hold.readiness import evaluate_ready +from vantio_install.boot_hold.registry import ( + compose_text_gated, + empty_registry, + find_workload, + load_registry, + restart_is_gated, + save_registry, + upsert, + validate_docker_name, + validate_protected_path, + validate_unit_name, +) +from vantio_install.boot_hold.units import compose_service, enrolled_docker_service, static_units, wants_links + +Runner = Callable[[list[str]], int] + + +def envelope(command: str, state: str, message: str, **extra: Any) -> dict[str, Any]: + body: dict[str, Any] = { + "command": command, + "state": state, + "message": message, + "proof_state": PROOF_STATE, + "proof_ceiling": PROOF_CEILING, + "reboot_row": REBOOT_ROW, + "host_wide_default_route_hold": False, + } + body.update(extra) + return body + + +def _write_json(path: Path, body: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(body, indent=2, sort_keys=True) + "\n", encoding="utf-8") + + +def _read_json(path: Path) -> dict: + if not path.exists(): + return {} + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError: + return {} + return data if isinstance(data, dict) else {} + + +def load_hold_state(root: Path) -> dict: + return _read_json(root / HOLD_STATE_REL) + + +def save_hold_state(root: Path, body: dict) -> None: + path = root / HOLD_STATE_REL + _write_json(path, body) + os.chmod(path, 0o600) + + +def protected_paths(registry: dict) -> list[str]: + found: list[str] = [] + for row in registry.get("workloads") or []: + if not row.get("enrolled"): + continue + for path in row.get("protected_paths") or []: + if path not in found: + found.append(path) + return found + + +def _write_profile(root: Path, text: str, runner: Runner) -> str: + path = root / PROFILE_REL + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + os.chmod(path, 0o644) + binary = "/usr/sbin/apparmor_parser" + if runner([binary, "-V"]) != 0: + binary = "apparmor_parser" + if runner([binary, "-V"]) != 0: + return "UNAVAILABLE" + if runner([binary, "-r", "-K", "-T", str(path)]) != 0: + return "FAILED" + return "LOADED" + + +def _unit_dropin_path(root: Path, unit: str) -> Path: + return root / "etc/systemd/system" / f"{unit}.d" / DROPIN_NAME + + +def write_workload_dropins( + root: Path, + registry: dict, + policy: dict, + *, + file_hold: bool, + apparmor: bool, + start_gate: bool = True, +) -> None: + hold, ordering = mechanisms_active(policy) + for row in registry.get("workloads") or []: + if not row.get("enrolled") or row.get("kind") != "systemd": + continue + unit = str(row["unit"]) + text = dropin_text( + hold=hold, + ordering=ordering, + protected_paths=list(row.get("protected_paths") or []), + apparmor=apparmor and file_hold and hold, + hide_paths=file_hold and hold, + start_gate=start_gate, + ) + if not hold and not ordering: + path = _unit_dropin_path(root, unit) + if path.exists(): + path.unlink() + continue + path = _unit_dropin_path(root, unit) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def install_tree(root: Path, python: str, policy: dict) -> list[str]: + written: list[str] = [] + for rel, text in static_units(python, policy).items(): + path = root / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + written.append(rel) + for rel, target in wants_links().items(): + path = root / rel + path.parent.mkdir(parents=True, exist_ok=True) + if path.is_symlink() or path.exists(): + path.unlink() + path.symlink_to(target) + written.append(rel) + cgroup = root / "sys/fs/cgroup" / SLICE + cgroup.mkdir(parents=True, exist_ok=True) + return written + + +def _daemon_reload(runner: Runner) -> None: + runner(["systemctl", "daemon-reload"]) + + +def _try_restart_active(runner: Runner, unit: str) -> None: + if runner(["systemctl", "is-active", unit]) == 0: + runner(["systemctl", "try-restart", unit]) + + +def workload_view(registry: dict, *, held: bool) -> list[dict[str, str]]: + rows: list[dict[str, str]] = [] + for row in registry.get("workloads") or []: + if row.get("enrolled"): + protection = "HELD" if held else "UNPROTECTED" + else: + protection = "UNPROTECTED" + rows.append( + { + "id": str(row.get("id", "")), + "kind": str(row.get("kind", "")), + "enrolled": "yes" if row.get("enrolled") else "no", + "protection": protection, + } + ) + return rows + + +def status_body(root: Path, *, lookup: str | None = None) -> dict[str, Any]: + policy, notes = load_policy(root) + try: + registry = load_registry(root) + registry_state = "READ" + except BootHoldError as exc: + registry = empty_registry() + registry_state = "UNKNOWN" + notes = notes + [str(exc)] + hold_on, ordering_on = mechanisms_active(policy) + state = load_hold_state(root) + released = bool(state.get("released")) + network_installed = bool(state.get("network_hold")) + units_installed = (root / "etc/systemd/system/vantio-boot-hold.service").exists() + held = False + if registry_state == "UNKNOWN": + health = "UNKNOWN" + message = "Enrollment status is UNKNOWN. Unknown is not a pass." + overall = "UNKNOWN" + elif not policy.get("enabled", True): + health = "OPTED_OUT" + overall = "OPTED_OUT" + message = "Boot hold is off because a root admin opted out. Enrolled workloads are not held. The audit log records that opt-out." + elif network_installed and not released: + health = "DEGRADED" + overall = "HELD" + message = HELD_MESSAGE + held = True + elif state.get("last_reason") == "BREAK_GLASS": + health = "DEGRADED" + overall = "RELEASED" + message = ( + "A root operator released the hold with break-glass while Phantom Engine was not enforce-ready. " + "The reboot row stays NOT_PROVED." + ) + elif released and state.get("last_reason") == "enforce-ready": + health = "HANDOFF" + overall = "RELEASED" + message = ( + "The hold released after the enforce-ready probe passed. " + "This status does not mark the reboot exposure row proved." + ) + elif hold_on and not units_installed: + health = "DEGRADED" + overall = "CONFIGURED" + message = ( + "Boot hold defaults on. The units are not installed yet, so the packet filter is not holding enrolled workloads. " + "Install them with `vantio-boot-hold install`." + ) + elif hold_on and not network_installed: + health = "DEGRADED" + overall = "CONFIGURED" + message = ( + "Boot hold is configured on. `vantio-boot-hold.service` applies the enrolled packet hold at boot, before Docker. " + "Until that service runs, status stays CONFIGURED." + ) + else: + health = "DEGRADED" + overall = "HOLD_OFF" + message = "The network hold is off because a root admin set hold to off. Ordering may still gate starts. The audit log records that change." + workloads = workload_view(registry, held=held) + if lookup: + match = find_workload(registry, lookup) + if match is None or not match.get("enrolled"): + workloads.append( + { + "id": lookup, + "kind": "lookup", + "enrolled": "no", + "protection": "UNPROTECTED", + } + ) + return envelope( + "status", + overall, + message, + health=health, + enabled=bool(policy.get("enabled", True)), + hold=hold_on, + ordering=ordering_on, + released=released, + registry=registry_state, + workloads=workloads, + notes=notes, + unprotected_rule="A workload that is absent from the enrollment registry is unprotected.", + file_profile=state.get("file_profile", "UNKNOWN"), + ) + + +def _audit_refusal(root: Path, caller: Caller, action: str, exc: BootHoldError) -> None: + try: + append_audit(root, caller, action, "REFUSED", {"message": str(exc)}) + except OSError: + return + + +def _authorize(root: Path, caller: Caller, action: str) -> None: + try: + require_operator(caller) + except BootHoldError as exc: + _audit_refusal(root, caller, action, exc) + raise + + +def install(root: Path, caller: Caller, python: str) -> dict[str, Any]: + _authorize(root, caller, "install") + policy, _notes = load_policy(root) + if not (root / "etc/vantio/boot-hold.json").exists(): + save_policy(root, policy) + written = install_tree(root, python, policy) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + save_registry(root, registry) + append_audit(root, caller, "install", "OK", {"units": len(written)}) + return envelope("install", "INSTALLED", "Boot hold units are installed. The hold defaults on.", units=written) + + +def apply_boot(root: Path, caller: Caller, runner: Runner, python: str) -> dict[str, Any]: + _authorize(root, caller, "apply-boot") + policy, notes = load_policy(root) + hold_on, _ordering = mechanisms_active(policy) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + install_tree(root, python, policy) + if not hold_on: + remove_hold(runner, policy) + if protected_paths(registry): + _write_profile(root, allow_profile(), runner) + save_hold_state( + root, + {"network_hold": False, "file_profile": "off", "released": True, "last_reason": "hold-off"}, + ) + write_workload_dropins(root, registry, policy, file_hold=False, apparmor=False) + _daemon_reload(runner) + message = notes[0] if notes else "The boot hold is off by root admin configuration." + body = envelope("apply-boot", "HOLD_OFF", message, health="OPTED_OUT" if not policy.get("enabled", True) else "DEGRADED") + _write_json(root / HEALTH_REL, body) + append_audit(root, caller, "apply-boot", "HOLD_OFF", {}) + return body + paths = protected_paths(registry) + profile_state = "NO_PATHS" + if paths: + profile_state = _write_profile(root, deny_profile(paths), runner) + write_workload_dropins(root, registry, policy, file_hold=True, apparmor=profile_state == "LOADED") + try: + install_hold(runner, policy) + except BootHoldError as exc: + save_hold_state( + root, + {"network_hold": False, "file_profile": profile_state, "released": False, "last_reason": "hold-failed"}, + ) + body = envelope("apply-boot", "HELD", HELD_MESSAGE + " " + str(exc), health="DEGRADED") + _write_json(root / HEALTH_REL, body) + append_audit(root, caller, "apply-boot", "FAILED", {"message": str(exc)}) + raise + (root / "sys/fs/cgroup" / "vantio-enrolled.slice").mkdir(parents=True, exist_ok=True) + _daemon_reload(runner) + save_hold_state( + root, + { + "network_hold": True, + "file_profile": "deny" if profile_state == "LOADED" else profile_state, + "released": False, + "last_reason": "boot-hold", + }, + ) + body = envelope( + "apply-boot", + "HELD", + HELD_MESSAGE, + health="DEGRADED", + file_profile=profile_state, + enrolled_slice=SLICE_PATH, + ) + _write_json(root / HEALTH_REL, body) + append_audit(root, caller, "apply-boot", "HELD", {"file_profile": profile_state}) + return body + + +def _wait_ready(facts_probe: Callable[[], dict], wait_seconds: int, sleep: Callable[[float], None]) -> dict: + facts = facts_probe() + verdict = evaluate_ready(facts) + if wait_seconds <= 0 or verdict["enforce_ready"]: + return verdict + deadline = time.monotonic() + wait_seconds + while time.monotonic() < deadline and not verdict["enforce_ready"]: + sleep(1) + verdict = evaluate_ready(facts_probe()) + return verdict + + +def release( + root: Path, + caller: Caller, + runner: Runner, + *, + require_ready: bool, + break_glass: bool, + operator_flag: bool, + facts_probe: Callable[[], dict], + wait_seconds: int = 0, + sleep: Callable[[float], None] = time.sleep, + python: str = "/usr/bin/python3", +) -> dict[str, Any]: + _authorize(root, caller, "release") + if require_ready and break_glass: + raise BootHoldError("Pass either --require-enforce-ready or --break-glass.", exit_code=10) + if not require_ready and not break_glass: + exc = BootHoldError( + "Release needs --require-enforce-ready after the probe passes, or --break-glass --i-am-root-operator. The hold does not release on its own.", + exit_code=10, + ) + _audit_refusal(root, caller, "release", exc) + raise exc + if break_glass and not operator_flag: + exc = BootHoldError( + "Break-glass release needs --i-am-root-operator on a host root shell.", + exit_code=10, + ) + _audit_refusal(root, caller, "release", exc) + raise exc + policy, _notes = load_policy(root) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + verdict = _wait_ready(facts_probe, wait_seconds if require_ready else 0, sleep) + if require_ready and not verdict["enforce_ready"]: + save_hold_state( + root, + { + "network_hold": True, + "file_profile": load_hold_state(root).get("file_profile", "deny"), + "released": False, + "last_reason": verdict.get("reason") or "not-enforce-ready", + }, + ) + message = HELD_MESSAGE + if verdict.get("reason") == "loader-running-not-attached": + message += " The loader is running and enforcement is not attached, so the hold stays." + elif verdict.get("reason") == "deny-self-check-failed": + message += " The live deny check did not pass, so the hold stays." + body = envelope("release", "HELD", message, health="DEGRADED", ready=verdict) + _write_json(root / HEALTH_REL, body) + append_audit(root, caller, "release", "REFUSED", {"ready": verdict}) + raise BootHoldError(message, exit_code=4, state="HELD") + reason = "enforce-ready" if require_ready else "BREAK_GLASS" + released = _lift_hold( + root, + registry, + policy, + runner, + python, + clear_start_gate=break_glass, + ) + save_hold_state(root, {"network_hold": False, "file_profile": "allow", "released": True, "last_reason": reason}) + if reason == "BREAK_GLASS": + released_text = ", ".join(released) + message = ( + "A root operator released the boot hold in one action. " + f"Released: {released_text}. " + "Start gates no longer require enforce-ready. " + "The audit log records that list. The reboot row stays NOT_PROVED." + ) + health = "DEGRADED" + state = "RELEASED" + else: + message = ( + "The hold released after the enforce-ready check, including the live deny check. " + "The reboot row stays NOT_PROVED." + ) + health = "HANDOFF" + state = "RELEASED" + body = envelope( + "release", + state, + message, + health=health, + ready=verdict, + audit_reason=reason, + released=released, + ) + _write_json(root / HEALTH_REL, body) + append_audit(root, caller, "release", reason, {"ready": verdict, "released": released}) + return body + + +def _lift_hold( + root: Path, + registry: dict, + policy: dict, + runner: Runner, + python: str, + *, + clear_start_gate: bool, +) -> list[str]: + """Remove the packet hold and, on break-glass, the start dependencies.""" + + released = ["packet-hold-ipv4", "packet-hold-ipv6", "file-hold"] + if protected_paths(registry): + _write_profile(root, allow_profile(), runner) + remove_hold(runner, policy) + write_workload_dropins( + root, + registry, + policy, + file_hold=False, + apparmor=False, + start_gate=not clear_start_gate, + ) + for row in registry.get("workloads") or []: + if not row.get("enrolled"): + continue + if row.get("kind") == "systemd": + unit = str(row["unit"]) + if clear_start_gate: + released.append(f"start-gate:{unit}") + runner(["systemctl", "reset-failed", unit]) + else: + _try_restart_active(runner, unit) + elif row.get("kind") == "compose" and clear_start_gate: + directory = Path(str(row.get("project_dir") or "")) + unit_name = f"vantio-enrolled-compose-{directory.name}.service" + unit_path = root / "etc/systemd/system" / unit_name + if directory.name: + unit_path.write_text(compose_service(directory.name, str(directory), policy, start_gate=False), encoding="utf-8") + released.append(f"start-gate:{unit_name}") + runner(["systemctl", "reset-failed", unit_name]) + if clear_start_gate: + docker_unit = root / "etc/systemd/system/vantio-enrolled-docker@.service" + docker_unit.parent.mkdir(parents=True, exist_ok=True) + docker_unit.write_text(enrolled_docker_service(policy, start_gate=False), encoding="utf-8") + released.append("start-gate:vantio-enrolled-docker@.service") + for row in registry.get("workloads") or []: + if row.get("enrolled") and row.get("kind") == "docker": + unit = f"vantio-enrolled-docker@{row.get('container')}.service" + released.append(f"start-gate:{unit}") + runner(["systemctl", "reset-failed", unit]) + else: + install_tree(root, python, policy) + _daemon_reload(runner) + return released + + +def enable_from_apply(root: Path, caller: Caller, runner: Runner, python: str) -> dict[str, Any]: + """Install and enable the hold during vantio-install apply. + + A trusted opt-out file is left as the admin wrote it. Apply does not + turn the hold back on. + """ + + if not config_path(root).exists(): + save_policy(root, {"enabled": True, "hold": True, "ordering": True}) + return apply_boot(root, caller, runner, python) + + +def live_boot_hold_runner(argv: list[str]) -> int: + """Run one boot-hold command on the host. Host-wide packet changes are refused.""" + + allowed_root = {"iptables", "ip6tables", "apparmor_parser", "/usr/sbin/apparmor_parser", "systemctl", "systemd-run", "bpftool"} + if not argv or argv[0] not in allowed_root: + raise BootHoldError("Refusing a boot-hold command outside the enrolled hold.") + if argv[0] == "systemctl" and (len(argv) < 2 or argv[1] not in {"daemon-reload", "is-active", "try-restart", "reset-failed"}): + raise BootHoldError("Refusing a systemctl command that is not part of the boot hold.") + blob = " ".join(argv) + if "-P" in argv or "0.0.0.0/0" in blob or "::/0" in blob: + raise BootHoldError("Refusing a host-wide hold command.") + completed = subprocess.run(argv, check=False) + return int(completed.returncode) + + +def remove_from_apply(root: Path, caller: Caller, runner: Runner) -> None: + """Rollback/uninstall removes units and packet rules. An opt-out file stays.""" + + _authorize(root, caller, "remove") + policy, _notes = load_policy(root) + remove_hold(runner, policy) + for rel in list(static_units("/usr/bin/python3", policy)) + list(wants_links()): + path = root / rel + if path.is_symlink() or path.is_file(): + path.unlink() + save_hold_state(root, {"network_hold": False, "file_profile": "off", "released": True, "last_reason": "removed"}) + append_audit(root, caller, "remove", "OK", {"released": ["packet-hold-ipv4", "packet-hold-ipv6", "units"]}) + + +def enroll_systemd(root: Path, caller: Caller, unit: str, paths: list[str], python: str) -> dict[str, Any]: + _authorize(root, caller, "enroll") + unit = validate_unit_name(unit) + clean = [validate_protected_path(path) for path in paths] + policy, _notes = load_policy(root) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + record = { + "id": unit, + "kind": "systemd", + "enrolled": True, + "unit": unit, + "protected_paths": clean, + "cgroup_parent": SLICE, + } + registry = upsert(registry, record) + save_registry(root, registry) + apparmor = (root / PROFILE_REL).exists() + write_workload_dropins(root, registry, policy, file_hold=True, apparmor=apparmor) + install_tree(root, python, policy) + append_audit(root, caller, "enroll", "OK", {"unit": unit, "paths": clean}) + return envelope( + "enroll", + "ENROLLED", + f"{unit} is enrolled. It joins {SLICE} and waits for the boot gate.", + unit=unit, + protected_paths=clean, + ) + + +def enroll_docker( + root: Path, + caller: Caller, + name: str, + restart_policy: str, + cgroup_parent: str, + paths: list[str], + python: str, + *, + set_restart_no: bool, + runner: Runner, +) -> dict[str, Any]: + _authorize(root, caller, "enroll") + name = validate_docker_name(name) + clean = [validate_protected_path(path) for path in paths] + if set_restart_no and restart_policy not in {"", "no", "none"}: + if runner(["docker", "update", "--restart=no", name]) != 0: + raise BootHoldError(f"docker update --restart=no {name} failed. The container stays unenrolled.") + restart_policy = "no" + restart_is_gated(restart_policy) + if cgroup_parent.rstrip("/") not in {f"/{SLICE}", SLICE}: + raise BootHoldError( + "Create the container with --cgroup-parent=/vantio-enrolled.slice --restart=no " + "--security-opt apparmor=vantio-boot-hold --network vantio-enrolled.", + exit_code=10, + ) + policy, _notes = load_policy(root) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + registry = upsert( + registry, + { + "id": name, + "kind": "docker", + "enrolled": True, + "container": name, + "protected_paths": clean, + "cgroup_parent": SLICE, + "restart": "no", + }, + ) + save_registry(root, registry) + install_tree(root, python, policy) + wants = root / "etc/systemd/system/multi-user.target.wants" / f"vantio-enrolled-docker@{name}.service" + wants.parent.mkdir(parents=True, exist_ok=True) + if wants.exists() or wants.is_symlink(): + wants.unlink() + wants.symlink_to(f"../vantio-enrolled-docker@{name}.service") + append_audit(root, caller, "enroll", "OK", {"container": name}) + return envelope( + "enroll", + "ENROLLED", + f"Container {name} is enrolled. Docker restart stays no. " + f"vantio-enrolled-docker@{name}.service starts it after the boot gate.", + container=name, + ) + + +def enroll_compose(root: Path, caller: Caller, project_dir: str, compose_text: str, python: str) -> dict[str, Any]: + _authorize(root, caller, "enroll") + directory = Path(project_dir) + if not directory.is_absolute() or any(char in project_dir for char in " \n\t;|&$`"): + raise BootHoldError("The compose project directory must be an absolute path without spaces.", exit_code=10) + compose_text_gated(compose_text) + policy, _notes = load_policy(root) + name = directory.name + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + registry = upsert( + registry, + { + "id": f"compose-{name}", + "kind": "compose", + "enrolled": True, + "project_dir": str(directory), + "protected_paths": [], + "cgroup_parent": SLICE, + }, + ) + save_registry(root, registry) + install_tree(root, python, policy) + unit_name = f"vantio-enrolled-compose-{name}.service" + unit_path = root / "etc/systemd/system" / unit_name + unit_path.write_text(compose_service(name, str(directory), policy), encoding="utf-8") + wants = root / "etc/systemd/system/multi-user.target.wants" / unit_name + wants.parent.mkdir(parents=True, exist_ok=True) + if wants.exists() or wants.is_symlink(): + wants.unlink() + wants.symlink_to(f"../{unit_name}") + append_audit(root, caller, "enroll", "OK", {"compose": str(directory)}) + return envelope( + "enroll", + "ENROLLED", + f"Compose project {directory} is enrolled. `docker compose start` runs from {unit_name} after the boot gate.", + unit=unit_name, + ) + + +def observe_unenrolled(root: Path, caller: Caller, name: str, kind: str) -> dict[str, Any]: + _authorize(root, caller, "observe-unenrolled") + if kind not in {"systemd", "docker", "compose", "process"}: + raise BootHoldError("Kind must be systemd, docker, compose, or process.", exit_code=10) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + registry = upsert( + registry, + {"id": name, "kind": kind, "enrolled": False, "protected_paths": [], "note": "observed unenrolled"}, + ) + save_registry(root, registry) + append_audit(root, caller, "observe-unenrolled", "OK", {"id": name}) + return envelope( + "observe-unenrolled", + "UNPROTECTED", + f"{name} is recorded as unenrolled. It is unprotected and the hold does not apply to it.", + id=name, + protection="UNPROTECTED", + ) + + +def opt_out(root: Path, caller: Caller, runner: Runner, reason: str, python: str) -> dict[str, Any]: + _authorize(root, caller, "opt-out") + if not reason.strip(): + raise BootHoldError("Opt-out needs a reason.", exit_code=10) + policy, _notes = load_policy(root) + policy["enabled"] = False + save_policy(root, policy) + registry = load_registry(root) if (root / "var/lib/vantio/boot-hold/registry.json").exists() else empty_registry() + remove_hold(runner, policy) + if protected_paths(registry): + _write_profile(root, allow_profile(), runner) + write_workload_dropins(root, registry, policy, file_hold=False, apparmor=False) + install_tree(root, python, policy) + _daemon_reload(runner) + save_hold_state(root, {"network_hold": False, "file_profile": "off", "released": True, "last_reason": "OPT_OUT"}) + append_audit(root, caller, "opt-out", "OK", {"reason": reason}) + body = envelope( + "opt-out", + "OPTED_OUT", + "Boot hold is off. A root admin opted out, and the audit log records the reason. Enrolled workloads are not held.", + health="OPTED_OUT", + reason=reason, + ) + _write_json(root / HEALTH_REL, body) + return body + + +def opt_in(root: Path, caller: Caller, runner: Runner, python: str) -> dict[str, Any]: + _authorize(root, caller, "opt-in") + save_policy(root, {"enabled": True, "hold": True, "ordering": True}) + append_audit(root, caller, "opt-in", "OK", {}) + return apply_boot(root, caller, runner, python) + + +def configure(root: Path, caller: Caller, runner: Runner, *, hold: bool, ordering: bool, python: str) -> dict[str, Any]: + _authorize(root, caller, "configure") + require_explicit_matrix(hold, ordering) + save_policy(root, {"enabled": True, "hold": hold, "ordering": ordering}) + append_audit(root, caller, "configure", "OK", {"hold": hold, "ordering": ordering}) + return apply_boot(root, caller, runner, python) + + +def ensure_network(root: Path, caller: Caller, runner: Runner) -> dict[str, Any]: + _authorize(root, caller, "ensure-network") + policy, _notes = load_policy(root) + if runner(["docker", "network", "inspect", NETWORK_NAME]) != 0: + if runner(["docker", "network", "create", "--subnet", str(policy["enrolled_subnet_v4"]), NETWORK_NAME]) != 0: + raise BootHoldError(f"Creating Docker network {NETWORK_NAME} failed.") + append_audit(root, caller, "ensure-network", "OK", {"subnet": policy["enrolled_subnet_v4"]}) + return envelope( + "ensure-network", + "NETWORK_READY", + f"Docker network {NETWORK_NAME} uses {policy['enrolled_subnet_v4']}. Creating the network does not start enrolled containers.", + ) + + +def loader_argv(root: Path) -> list[str]: + path = root / LOADER_ARGV_REL + if not path.exists(): + raise BootHoldError( + "Phantom Engine loader command is not configured, so the loader unit fails and enrolled workloads stay held. " + "A root admin writes a JSON list of strings to /etc/vantio/pe-loader.argv.json.", + state="HELD", + ) + if path.stat().st_mode & 0o022: + raise BootHoldError("The loader argv file is writable by group or other. The loader stays stopped and the hold stays on.", state="HELD") + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise BootHoldError("pe-loader.argv.json is not valid JSON. The loader stays stopped.", state="HELD") from exc + if not isinstance(data, list) or not data or not all(isinstance(item, str) and item for item in data): + raise BootHoldError("pe-loader.argv.json must be a JSON list of strings.", state="HELD") + if not str(data[0]).startswith("/"): + raise BootHoldError("The loader command must start with an absolute path.", state="HELD") + return [str(item) for item in data] + + +def start_loader(root: Path, caller: Caller) -> dict[str, Any]: + _authorize(root, caller, "start-loader") + argv = loader_argv(root) + append_audit(root, caller, "start-loader", "EXEC", {"argv0": argv[0]}) + return envelope("start-loader", "EXEC", "Starting the Phantom Engine loader command.", argv=argv) diff --git a/packages/vantio-install/vantio_install/boot_hold/units.py b/packages/vantio-install/vantio_install/boot_hold/units.py new file mode 100644 index 00000000..ffcda8e5 --- /dev/null +++ b/packages/vantio-install/vantio_install/boot_hold/units.py @@ -0,0 +1,184 @@ +"""systemd units for the boot hold, early Phantom Engine start, and workload gate.""" + +from __future__ import annotations + +from vantio_install.boot_hold.constants import NETWORK_NAME, SUBNET_V4 +from vantio_install.boot_hold.policy import mechanisms_active + + +def _py(python: str, args: str) -> str: + return f"{python} -m vantio_install.boot_hold {args}" + + +def boot_hold_service(python: str) -> str: + return f"""[Unit] +Description=Hold enrolled workloads before Docker and before workload services +DefaultDependencies=no +After=local-fs.target +Before=docker.service containerd.service vantio-pe-loader.service +# SSH, systemd-networkd, resolved, and SSM are not ordered behind this unit. + +[Service] +Type=oneshot +RemainAfterExit=yes +TimeoutStartSec=30 +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart={_py(python, "apply-boot")} + +[Install] +WantedBy=sysinit.target +""" + + +def pe_loader_service(python: str) -> str: + return f"""[Unit] +Description=Start Phantom Engine as early as the enrolled hold allows +DefaultDependencies=no +After=local-fs.target vantio-boot-hold.service docker.service +Wants=docker.service +Before=vantio-pe-enforce-ready.service + +[Service] +Type=simple +Restart=no +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart={_py(python, "start-loader")} +ExecStop=/bin/kill -TERM $MAINPID + +[Install] +WantedBy=multi-user.target +""" + + +def enforce_ready_service(python: str) -> str: + return f"""[Unit] +Description=Release the enrolled hold only after Phantom Engine is enforce-ready +DefaultDependencies=no +After=vantio-pe-loader.service vantio-boot-hold.service +Requires=vantio-pe-loader.service + +[Service] +Type=oneshot +RemainAfterExit=yes +TimeoutStartSec=180 +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart={_py(python, "release --require-enforce-ready --wait-seconds 120")} + +[Install] +WantedBy=multi-user.target +""" + + +def enrolled_slice() -> str: + return f"""[Unit] +Description=Enrolled workloads +DefaultDependencies=no +Before=slices.target + +[Slice] +""" + + +def docker_ordering_dropin() -> str: + return """[Unit] +After=vantio-boot-hold.service +""" + + +def enrolled_docker_service(policy: dict, *, start_gate: bool = True) -> str: + hold, ordering = mechanisms_active(policy) + if not start_gate: + hold, ordering = False, False + after = ["docker.service", "vantio-enrolled-network.service"] + requires = ["docker.service", "vantio-enrolled-network.service"] + if hold: + after.append("vantio-boot-hold.service") + requires.append("vantio-boot-hold.service") + if ordering: + after.append("vantio-pe-enforce-ready.service") + requires.append("vantio-pe-enforce-ready.service") + return f"""[Unit] +Description=Start enrolled container %i after the boot gate +After={' '.join(after)} +Requires={' '.join(requires)} + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/docker start %i +ExecStop=/usr/bin/docker stop %i + +[Install] +WantedBy=multi-user.target +""" + + +def enrolled_network_service(python: str) -> str: + return f"""[Unit] +Description=Create the enrolled Docker network {NETWORK_NAME} ({SUBNET_V4}) +After=docker.service vantio-boot-hold.service +Requires=docker.service + +[Service] +Type=oneshot +RemainAfterExit=yes +Environment=VANTIO_BOOT_HOLD_LIVE=1 +ExecStart={_py(python, "ensure-network")} + +[Install] +WantedBy=multi-user.target +""" + + +def compose_service(name: str, project_dir: str, policy: dict, *, start_gate: bool = True) -> str: + hold, ordering = mechanisms_active(policy) + if not start_gate: + hold, ordering = False, False + after = ["docker.service", "vantio-enrolled-network.service"] + requires = ["docker.service", "vantio-enrolled-network.service"] + if hold: + after.append("vantio-boot-hold.service") + requires.append("vantio-boot-hold.service") + if ordering: + after.append("vantio-pe-enforce-ready.service") + requires.append("vantio-pe-enforce-ready.service") + safe_dir = project_dir + return f"""[Unit] +Description=Start enrolled compose project {name} after the boot gate +After={' '.join(after)} +Requires={' '.join(requires)} + +[Service] +Type=oneshot +RemainAfterExit=yes +WorkingDirectory={safe_dir} +ExecStart=/usr/bin/docker compose start +ExecStop=/usr/bin/docker compose stop + +[Install] +WantedBy=multi-user.target +""" + + +def static_units(python: str, policy: dict) -> dict[str, str]: + return { + "etc/systemd/system/vantio-boot-hold.service": boot_hold_service(python), + "etc/systemd/system/vantio-pe-loader.service": pe_loader_service(python), + "etc/systemd/system/vantio-pe-enforce-ready.service": enforce_ready_service(python), + "etc/systemd/system/vantio-enrolled.slice": enrolled_slice(), + "etc/systemd/system/vantio-enrolled-docker@.service": enrolled_docker_service(policy), + "etc/systemd/system/vantio-enrolled-network.service": enrolled_network_service(python), + "etc/systemd/system/docker.service.d/vantio-boot-hold.conf": docker_ordering_dropin(), + "etc/systemd/system/containerd.service.d/vantio-boot-hold.conf": docker_ordering_dropin(), + } + + +def wants_links() -> dict[str, str]: + """Symlink path -> relative target, as systemd enable would write them.""" + + return { + "etc/systemd/system/sysinit.target.wants/vantio-boot-hold.service": "../vantio-boot-hold.service", + "etc/systemd/system/multi-user.target.wants/vantio-pe-loader.service": "../vantio-pe-loader.service", + "etc/systemd/system/multi-user.target.wants/vantio-pe-enforce-ready.service": "../vantio-pe-enforce-ready.service", + "etc/systemd/system/multi-user.target.wants/vantio-enrolled-network.service": "../vantio-enrolled-network.service", + } diff --git a/packages/vantio-install/vantio_install/constants.py b/packages/vantio-install/vantio_install/constants.py index df636bad..2628b1c3 100644 --- a/packages/vantio-install/vantio_install/constants.py +++ b/packages/vantio-install/vantio_install/constants.py @@ -96,6 +96,7 @@ "stage_pe_archive", "docker_load", "write_observe_config", + "install_boot_hold", "start_pe_observe", "mark_applied", "collect_health", @@ -109,6 +110,7 @@ "stage_pe_archive", "docker_load", "write_observe_config", + "install_boot_hold", "start_pe_observe", } ) diff --git a/packages/vantio-install/vantio_install/engine.py b/packages/vantio-install/vantio_install/engine.py index 613f5932..e54dd789 100644 --- a/packages/vantio-install/vantio_install/engine.py +++ b/packages/vantio-install/vantio_install/engine.py @@ -9,6 +9,7 @@ from vantio_install import bpf_pins, constants from vantio_install.errors import InstallError +from vantio_install.boot_hold.service import status_body from vantio_install.health import component_template, derive from vantio_install.host import load_fixture, probe_live from vantio_install.manifest import artifact_paths, bundle_digest, load_manifest, missing_manifest_fields @@ -509,7 +510,13 @@ def _live_grant(ctx: dict, tx: dict, tx_dir: Path, config: dict, bundle: Path, h ) -def _step_ctx(tx: dict, config: dict, bundle: Path, hooks: dict, observe_path: Path) -> dict: +def _boot_hold_root(ctx: dict, state_dir: Path) -> str: + if ctx.get("fixture_host"): + return str(state_dir / "boot-hold-host") + return "/" + + +def _step_ctx(tx: dict, config: dict, bundle: Path, hooks: dict, observe_path: Path, boot_hold_root: str) -> dict: return { "transaction_id": tx["transaction_id"], "config": config, @@ -519,6 +526,7 @@ def _step_ctx(tx: dict, config: dict, bundle: Path, hooks: dict, observe_path: P "simulate_probe_errors": hooks.get("simulate_probe_errors") or [], "uninstall_leave": hooks.get("uninstall_leave") or [], "bpffs_root": hooks.get("bpffs_root") or "", + "boot_hold_root": boot_hold_root, } @@ -592,7 +600,7 @@ def apply(ctx: dict) -> tuple[int, dict]: _move(tx, "APPLYING", stamp) mutator, stage, prefix = _mutator(ctx, snapshot, config, grant) observe_path = tx_dir / "observe-config.json" - step_ctx = _step_ctx(tx, config, bundle, hooks, observe_path) + step_ctx = _step_ctx(tx, config, bundle, hooks, observe_path, _boot_hold_root(ctx, state_dir)) tx["mutation_in_progress"] = True tx["owner_pid"] = os.getpid() tx["phase"] = "apply" @@ -655,6 +663,8 @@ def apply(ctx: dict) -> tuple[int, dict]: "transaction_id": tx["transaction_id"], "as_of_et": stamp, "evidence_paths": [str(evidence)], + "boot_hold": status_body(Path(step_ctx["boot_hold_root"])), + "reboot_row": "NOT_PROVED", } ) write_json(tx_dir / "HEALTH.json", health) @@ -818,7 +828,7 @@ def rollback(ctx: dict) -> tuple[int, dict]: _move(tx, "ROLLING_BACK", stamp) mutator, _stage, _prefix = _mutator(ctx, snapshot, config, grant) prefix = getattr(mutator, "prefix", _prefix) - step_ctx = _step_ctx(tx, config, bundle, hooks, tx_dir / "observe-config.json") + step_ctx = _step_ctx(tx, config, bundle, hooks, tx_dir / "observe-config.json", _boot_hold_root(ctx, ctx["state_dir"])) tx["mutation_in_progress"] = True tx["owner_pid"] = os.getpid() tx["phase"] = "rollback" @@ -912,7 +922,7 @@ def uninstall(ctx: dict) -> tuple[int, dict]: if tx["state"] == "INTERRUPTED" or tx["state"] in {"HEALTHY", "DEGRADED", "FAILED_SAFE"} or tx["state"] in RESIDUAL_STATES: _move(tx, "UNINSTALLING", stamp) mutator, _stage, _prefix = _mutator(ctx, snapshot, config, grant) - step_ctx = _step_ctx(tx, config, bundle, hooks, tx_dir / "observe-config.json") + step_ctx = _step_ctx(tx, config, bundle, hooks, tx_dir / "observe-config.json", _boot_hold_root(ctx, ctx["state_dir"])) tx["scope"] = scope tx["phase"] = "uninstall" tx["mutation_in_progress"] = True diff --git a/packages/vantio-install/vantio_install/live_executor.py b/packages/vantio-install/vantio_install/live_executor.py index ae020d0a..0d72fe2f 100644 --- a/packages/vantio-install/vantio_install/live_executor.py +++ b/packages/vantio-install/vantio_install/live_executor.py @@ -11,12 +11,15 @@ import re import shutil import subprocess +import sys import time from collections.abc import Callable from dataclasses import dataclass from pathlib import Path from vantio_install import bpf_pins, constants +from vantio_install.boot_hold.identity import read_host_caller +from vantio_install.boot_hold.service import enable_from_apply, live_boot_hold_runner, remove_from_apply from vantio_install.docker_object import ( FAILED_SAFE as DOCKER_FAILED_SAFE, IDEMPOTENT_ABSENT, @@ -96,6 +99,7 @@ "stage_pe_archive": ("mkdir_stage", "stage_pe_archive"), "docker_load": ("docker_load", "docker_tag"), "write_observe_config": ("mkdir_evidence", "write_observe_config", "o7_init"), + "install_boot_hold": ("install_boot_hold",), "start_pe_observe": ("tc_clsact", "write_pe_apparmor", "load_pe_apparmor", "start_pe_observe"), } @@ -111,6 +115,7 @@ "docker_load": ("docker_rmi",), "stage_pe_archive": ("remove_stage",), "write_observe_config": ("remove_observe_config", "remove_o7_record"), + "install_boot_hold": ("remove_boot_hold",), "install_agent_sdks": ("remove_sdks",), "install_optics_cli": ("remove_optics",), } @@ -379,6 +384,8 @@ def catalog_argv(op_type: str, grant: LiveGrant) -> list[str] | None: pin["pe_local_tag"], ), "write_observe_config": None, + "install_boot_hold": None, + "remove_boot_hold": None, "o7_init": None, "tc_clsact": tc_clsact_argv(iface), "write_pe_apparmor": None, @@ -518,6 +525,14 @@ def _filesystem(op_type: str, grant: LiveGrant) -> None: if sha256_file(target) != pin["pe_archive_sha256"]: _fail("The staged archive does not match the pin after copy.", failure_class="ROLLBACK_REQUIRED") return + if op_type == "install_boot_hold": + if os.geteuid() == 0: + enable_from_apply(Path("/"), read_host_caller(), live_boot_hold_runner, sys.executable or "/usr/bin/python3") + return + if op_type == "remove_boot_hold": + if os.geteuid() == 0: + remove_from_apply(Path("/"), read_host_caller(), live_boot_hold_runner) + return if op_type == "write_observe_config": path = confine(grant.observe_config, [grant.tx_dir]) payload = { @@ -1020,6 +1035,13 @@ def verify(self, op_type: str, grant: LiveGrant) -> str: if target.is_file() and sha256_file(target) == constants.FROZEN_PINS["pe_archive_sha256"]: return "VERIFIED" return "NOT_VERIFIED" + if op_type == "install_boot_hold": + unit = Path("/etc/systemd/system/vantio-boot-hold.service") + link = Path("/etc/systemd/system/sysinit.target.wants/vantio-boot-hold.service") + return "VERIFIED" if unit.is_file() and link.is_symlink() else "NOT_VERIFIED" + if op_type == "remove_boot_hold": + unit = Path("/etc/systemd/system/vantio-boot-hold.service") + return "VERIFIED" if not unit.exists() else "NOT_VERIFIED" if op_type == "write_observe_config": payload = read_json(grant.observe_config) if payload.get("enforcement") == "NOT_ENABLED" and "--enforce" not in (payload.get("cmd") or []): diff --git a/packages/vantio-install/vantio_install/mutator.py b/packages/vantio-install/vantio_install/mutator.py index 3bfdf1b9..48d7ce56 100644 --- a/packages/vantio-install/vantio_install/mutator.py +++ b/packages/vantio-install/vantio_install/mutator.py @@ -21,6 +21,8 @@ observe_env, ) from vantio_install.pe_apparmor import pe_apparmor_profile_path +from vantio_install.boot_hold.identity import Caller +from vantio_install.boot_hold.service import enable_from_apply, remove_from_apply from vantio_install.errors import InstallError from vantio_install.oci_load import OciArchiveError, materialize, plan_load from vantio_install.stage_remove import remove_stage_nofollow @@ -46,6 +48,7 @@ def apply_step(self, step_id: str, ctx: dict) -> None: "stage_pe_archive": self._stage_pe, "docker_load": self._docker_load, "write_observe_config": self._write_config, + "install_boot_hold": self._install_boot_hold, "start_pe_observe": self._start, }.get(step_id) if handler: @@ -58,6 +61,7 @@ def rollback_step(self, step_id: str, ctx: dict) -> None: "stage_pe_archive": self._remove_stage, "docker_load": self._docker_rmi, "write_observe_config": self._remove_config, + "install_boot_hold": self._remove_boot_hold, "start_pe_observe": self._stop, }.get(step_id) if handler: @@ -69,6 +73,7 @@ def rollback_step(self, step_id: str, ctx: dict) -> None: def uninstall(self, scope: str, ctx: dict) -> None: leave = set(ctx.get("uninstall_leave") or []) if scope in {"pe", "all"}: + self._remove_boot_hold(ctx) if "container" not in leave: self._stop(ctx) if "image" not in leave: @@ -202,6 +207,36 @@ def _docker_rmi(self, ctx: dict) -> None: row for row in self.snapshot.get("images") or [] if row.get("tag") != tag ] + def _boot_caller(self) -> Caller: + return Caller(0, 1, "vantio-install", "0::/system.slice/vantio-install.service\n", "systemd") + + def _boot_runner(self, calls: list[list[str]]): + def run(argv: list[str]) -> int: + calls.append(list(argv)) + if len(argv) >= 2 and argv[1] in {"-C", "-N", "-D", "-X"}: + return 1 + if argv and argv[0] in {"apparmor_parser", "/usr/sbin/apparmor_parser"} and "-V" in argv: + return 1 + return 0 + + return run + + def _install_boot_hold(self, ctx: dict) -> None: + root = Path(str(ctx.get("boot_hold_root") or "")) + if not str(root): + raise InstallError("Boot hold install is missing its layout root.", exit_code=4, state="FAILED_SAFE") + calls: list[list[str]] = [] + body = enable_from_apply(root, self._boot_caller(), self._boot_runner(calls), "python3") + self.snapshot["boot_hold"] = {"state": body.get("state"), "health": body.get("health"), "commands": calls} + + def _remove_boot_hold(self, ctx: dict) -> None: + root = Path(str(ctx.get("boot_hold_root") or "")) + if not str(root): + return + calls: list[list[str]] = [] + remove_from_apply(root, self._boot_caller(), self._boot_runner(calls)) + self.snapshot["boot_hold"] = {"state": "REMOVED", "commands": calls} + def _write_config(self, ctx: dict) -> None: path = Path(ctx["observe_config_path"]) payload = { @@ -335,7 +370,7 @@ def rollback_step(self, step_id: str, ctx: dict) -> None: def uninstall(self, scope: str, ctx: dict) -> None: if scope in {"pe", "all"}: - for step_id in ("start_pe_observe", "docker_load", "stage_pe_archive", "write_observe_config"): + for step_id in ("install_boot_hold", "start_pe_observe", "docker_load", "stage_pe_archive", "write_observe_config"): self._run(step_id, "rollback") if scope in {"optics", "all"}: for step_id in ("install_agent_sdks", "install_optics_cli"):