feat(inference): run local models on the Apple Silicon GPU

[JArmandoAnaya]

Closes #573.

What changed

mps joins the device vocabulary, so a local connection can run on the Apple Silicon GPU. The kernel gains the value, the adapters honour it, and the catalog screen offers it — with fp32 as the only precision, because half precision applies on CUDA only.

The two local adapters each carried a private _resolved_device and an identical CPU_FALLBACK_WARNING. Adding a third device to two copies is how two copies become two answers, so the rule is promoted into inference/_device.py — the move require_move and require_draft already made in the kernel.

The part worth reviewing: available is not usable

Caught by running the branch on real hardware rather than by a test.

_present originally asked torch.backends.mps.is_available() alone, and argued in its own docstring that asking is_built too would be "asking one question twice". That is right about is_built and wrong about the question. is_available answers is there a Metal device; what the adapter needs is can I put a tensor on it.

On an Intel Mac with a discrete GPU those diverge:

mps built True
mps avail True
>>> torch.zeros(1, device="mps")
RuntimeError: MPS backend is only supported on devices with unified memory

So mps was waved through, the documented CPU fallback never ran, zero fallback warnings were logged, and the model died moving its weights to the device — POST /inference/suggest → 500 with an incident id, surfacing in the editor as "That suggestion could not be made". Measured on an i9 MacBook Pro, macOS 26.6, torch 2.13.0 (conda-forge osx-64).

The fix asks the second question by doing it — one one-element allocation, RuntimeError means absent — which is the run-time shape this module already argues for: "a device this particular machine does not have is answered at the moment of the call."

After the fix, on the same machine:

WARNING inference connection 'sam2 small' asks for device 'mps',
        which this machine does not offer; running on the CPU in full precision instead
resolved() answered device='cpu' half=False

It costs the other paths nothing, and that was measured

resolved() runs inside _load(), which is memoised — once per model load, never per suggestion. The probe is additionally cached on the array library it was handed. Probe allocations across 50 resolved() calls:

Path	Allocations	Result
cpu	0 — branch never reached	unchanged
cuda	0 — branch never reached	unchanged
mps, Apple Silicon	1 per process	still ('mps', False) — unchanged
mps, Intel Mac	1 per process	now ('cpu', False) + warning

Timings on the failing path: first resolved() 25.6 ms including the probe, every call after it 0.0007 ms, cpu 0.0003 ms with no probe at all. On Apple Silicon that one allocation initialises the Metal context — work the first real inference does moments later regardless, so it is moved earlier rather than added.

Caching on the torch object rather than in a module global is also what keeps the probe as injectable as the two availability flags beside it: a test handing in a different stub gets a different answer without clearing anything.

Found, not fixed

Nothing unrelated surfaced.

Test plan

bash scripts/check.sh run in stages (the harness kills a command at ~10 minutes), each verdict recorded, then re-run in full after rebasing onto 69dbcc8 because that brought in four commits from other work:

Stage	Verdict
check.sh python frontend generated docs	PASSED — 3339 passed, 13 skipped; ruff, mypy, import-linter, every drift gate, docs build
check.sh browser	PASSED — annotator e2e and browser cycle

New coverage in tests/inference/test_device.py, all on the existing stub so it runs on a machine with neither GPU:

• Metal that reports available and then refuses a tensor falls back to the CPU with the warning — the case is_available alone gets wrong.
• The probe is never reached on a machine reporting no Metal, asserted with a stub whose zeros raises AssertionError rather than described in a comment.
• A non-RuntimeError surprise from the probe propagates instead of being read as absent hardware.

StubTorch gains mps_usable as a third axis, separate from mps, because on a real machine the two genuinely disagree.

Three mutations, each reddening a named test. One came back green first — widening except RuntimeError to except Exception was invisible, so the docstring's claim that a half-installed runtime propagates rather than becoming a silent CPU fallback was a description and not a rule. That is the third test above.

What still needs an M-series Mac

The fallback is verified on hardware that cannot run MPS. mps actually accelerating anything is not verified by this PR — no runner in CI is an Apple Silicon Mac and neither is the machine this was written on. The path is exercised by stub, and the real thing wants one manual run on an M-series machine before anybody relies on the speedup.