Run VM guests on macOS with Hypervisor.framework

[drk1rd]

VM instances on macOS used to run emulated (TCG). Docker Desktop and OrbStack run containers in a Linux VM that has no nested virtualization. I checked OrbStack 2.2.4 on an M5 Pro with macOS 27: containers get no /dev/kvm, and the kernel logs kvm: HYP mode not available. So passing the device through doesn't work here.

On an Apple Silicon Mac with QEMU installed (brew install qemu), guests now run on the Mac itself with -accel hvf -cpu host. Instances report virtualization: "hvf". Part of #3.

Design

• The VM container stays and keeps everything that makes the guest an instance: the private address and passt, security groups, published ports, the metadata route, the disks in their Docker volumes, and the console log.
• With HC_VM_HOST=1, hc-vm-run prepares disks and seed as before and starts passt. It then serves the root qcow2 overlay, the extra volumes, the seed and the UEFI variable store as NBD exports of qemu-storage-daemon on a Unix socket, and waits.
• Host runner (homecloud vm-host DIR, hidden command): a detached process per guest that holds a lock on the guest's run directory (mode 0700).
  • It starts QEMU and relays QEMU's network card (-netdev stream) and NBD disks into the container. Each connection goes through docker exec socat - UNIX-CONNECT:....
  • Nothing listens on a TCP port. I tested two alternatives first: Unix sockets don't cross OrbStack bind mounts, and published ports are reachable from other containers through host.docker.internal.
  • It copies the serial console into the container log, so GetConsoleOutput and boot detection are unchanged.
  • The container exits when QEMU exits, and QEMU is killed when the container stops. Stop, start, reboot, rebuilds, volume attach, snapshots, images, backup and reconciliation therefore keep using the container code paths.
• Server side: a loop starts a runner for every running homecloud.vm-host=1 container that has none. Because the runner is its own process, guests survive a server restart. Run-command, guest metrics, the serial terminal and the ACPI power button connect directly to QEMU's sockets in the run directory.
• Choosing the backend is automatic: macOS/arm64, not in a container, an arm64 Docker host, QEMU and its firmware found, and an hvf probe succeeds. Otherwise HomeCloud falls back to the container and logs why. serve --vm-accel auto|hvf|container|kvm|tcg (saved as vm_accel) or HC_VM_ACCEL forces a backend. If passt cannot start for a guest on the host, the guest moves into the container, emulated.
• Firmware: QEMU's own edk2-aarch64-code.fd. Debian's AAVMF 2025.02, which the container uses, hangs before its boot manager under hvf. I reproduced this standalone, without NBD.
• Guest image architecture: already follows the Docker host, so it is arm64 on Apple Silicon.
• Linux is unchanged: no /dev/kvm or HC_VM_ACCEL behaviour changes, and the runner image Dockerfile is the same (qemu-storage-daemon was already in qemu-utils).
• The console shows a "VM · HVF" badge. The docs cover the backend in architecture.md and install-server.md.
• Also fixed: unit tests pinned hostArch for the whole test binary. On a Mac that made later live VM tests download and boot the amd64 image.

Measured on an M5 Pro, OrbStack, Ubuntu 24.04 VM AMI

	before (TCG in OrbStack)	after (hvf)
Container start to instance running (cloud-init done)	145 s	21 s (QEMU start to running)
run-instances to running, image cached	~150 s	31 s
Stop + start	96 s	24–30 s
TestVMInstanceLifecycle (image cached, after the first runner build)	~440 s	135–144 s

Verified on this Mac

• With HC_TEST_VM=1 and OrbStack, all VM tests pass: TestVMInstanceLifecycle and TestVMDisksImagesAndBackup. They cover:
  • SSH through the published port, the private address in the guest, IMDS, DNS, user data and console output
  • run-command, guest metrics and the serial terminal
  • stop/start, security-group rebuilds and terminate
  • extra volumes, snapshot restore and attach, CreateImage, backup and restore
• With Docker set, go test ./internal/svc/ec2 ./cmd passes. go test ./..., go vet ./..., and GOOS=linux / GOOS=windows go build ./... also pass.
• Real homecloud serve plus the CLI and AWS CLI:
  • run-instances gave virtualization: hvf and vm_network: passt.
  • SSH worked: uname -m reported aarch64, the guest had its private IP, and IMDS answered.
  • homecloud ec2 exec and ec2 logs worked.
  • The guest kept running across a server restart (same QEMU pid, SSH still worked).
  • Stop took 2 s and start took 21 s; terminate cleaned up QEMU, the runner and the run directory.

Not verified

• Intel Macs and Docker Desktop. Auto-selection is limited to Apple Silicon; Docker Desktop should work the same way since only docker exec is used, but I didn't test it.
• The fallback into the container when passt fails for a guest on the host.
• The console TypeScript change was not built here (no node_modules).

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: bcf13a27-7851-4127-b609-5db82dd1502f

📥 Commits

Reviewing files that changed from the base of the PR and between 7e466a5 and 7940f8e.

📒 Files selected for processing: 25

• README.md
• cli/cmd/serve.go
• cli/cmd/vmhost.go
• cli/internal/core/core.go
• cli/internal/runtime/self.go
• cli/internal/svc/ec2/capabilities.go
• cli/internal/svc/ec2/ec2.go
• cli/internal/svc/ec2/terminal.go
• cli/internal/svc/ec2/vm.go
• cli/internal/svc/ec2/vm/hc-vm-run
• cli/internal/svc/ec2/vm/host.go
• cli/internal/svc/ec2/vm/host_test.go
• cli/internal/svc/ec2/vm_access.go
• cli/internal/svc/ec2/vm_host.go
• cli/internal/svc/ec2/vm_host_unix.go
• cli/internal/svc/ec2/vm_host_unix_test.go
• cli/internal/svc/ec2/vm_host_windows.go
• cli/internal/svc/ec2/vm_live_test.go
• cli/internal/svc/ec2/vm_security_test.go
• console/components/ec2/instance-actions.tsx
• console/components/ec2/instance-detail.tsx
• console/components/ec2/launch-wizard.tsx
• console/lib/types/ec2.ts
• docs/architecture.md
• docs/install-server.md

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cloudflare-workers-and-pages]

Deploying homecloud with    Cloudflare Pages

Latest commit:	7940f8e
Status:	✅  Deploy successful!
Preview URL:	https://d11ac770.homecloud.pages.dev
Branch Preview URL:	https://vm-macos-hvf.homecloud.pages.dev

View logs