Theme E follow-up — attack the software-path VNC ~1s lag
The Theme E spike (PR #5, docs/specs/04-gazebo-gpu-wsl2-research.md) proved that enabling the GPU on WSL2 Intel makes the interactive VNC stream worse (~1s software → 20s+ GPU) because every frame must be read back across the /dev/dxg (Mesa D3D12/dozen) boundary. GPU is rejected for the interactive path.
This issue tracks the 1s-lag attack path: driving the software (llvmpipe) rendering path down from ~1s toward interactive (<200ms) without touching the GPU readback path.
Full research: .copilot-tracking/research/2026-07-19/gazebo-vnc-1s-lag-attack-path-research.md.
Where the ~1s goes (software path)
Dominant, removable costs are (1) the full-frame -noshm copy, (2) full-screen polling with no XDAMAGE, and (3) full-frame CPU encode. Never leaves system RAM.
Attack path (ordered, cheapest / highest-ROI first)
- L1 — Enable MIT-SHM in x11vnc (drop
-noshm, add -shm): removes the per-poll full X-protocol framebuffer copy. Biggest single win on the current stack.
- L2 — Reduce 1920x1080 → 1280x720: ~2.25x fewer pixels to scan/copy/encode; compounds with L1. Keep configurable.
- L3 — Swap to a direct-capture X server (TurboVNC/Xvnc or KasmVNC): eliminates the screen-scrape polling model; damage-aware, region-only encode. Highest ceiling, biggest integration change (Dockerfile, entrypoint, proxy,
tests/acceptance/s4-novnc-frame.spec.js). Sequence after L1/L2.
- L4 — Tune encoder quality/FPS (cap 15–24 FPS, WebP/JPEG-turbo).
- L5 — Right-size
/dev/shm + CPU budget to guard the L1 win.
Recommended ordering
- L1 + L2 first (small reversible config on existing x11vnc stack; measure the delta).
- L3 only if L1+L2 miss the target.
- L4/L5 as tuning passes.
Acceptance targets
- Interactive motion-to-photon perceptibly below ~1s; stretch <200ms on 1280x720 software path.
- Probes: confirm SHM attach in x11vnc startup log (not "falling back to XGetImage");
ls -l /dev/dri (expected absent on WSL2 → validates rejecting VA-API/VirtualGL); A/B camera-orbit latency before/after each lever.
- Extend
tests/acceptance/s4-novnc-frame.spec.js with a latency/interaction smoke assertion.
Rejected / deferred
- GPU (Mesa D3D12/dozen) for interactive path — REJECTED (per-frame readback).
- VirtualGL + TurboVNC and VA-API/WebRTC HW encode — DEFERRED on WSL2 (no reliable
/dev/dri).
Links
Theme E follow-up — attack the software-path VNC ~1s lag
The Theme E spike (PR #5,
docs/specs/04-gazebo-gpu-wsl2-research.md) proved that enabling the GPU on WSL2 Intel makes the interactive VNC stream worse (~1s software → 20s+ GPU) because every frame must be read back across the/dev/dxg(Mesa D3D12/dozen) boundary. GPU is rejected for the interactive path.This issue tracks the 1s-lag attack path: driving the software (llvmpipe) rendering path down from ~1s toward interactive (<200ms) without touching the GPU readback path.
Full research:
.copilot-tracking/research/2026-07-19/gazebo-vnc-1s-lag-attack-path-research.md.Where the ~1s goes (software path)
Dominant, removable costs are (1) the full-frame
-noshmcopy, (2) full-screen polling with no XDAMAGE, and (3) full-frame CPU encode. Never leaves system RAM.Attack path (ordered, cheapest / highest-ROI first)
-noshm, add-shm): removes the per-poll full X-protocol framebuffer copy. Biggest single win on the current stack.tests/acceptance/s4-novnc-frame.spec.js). Sequence after L1/L2./dev/shm+ CPU budget to guard the L1 win.Recommended ordering
Acceptance targets
ls -l /dev/dri(expected absent on WSL2 → validates rejecting VA-API/VirtualGL); A/B camera-orbit latency before/after each lever.tests/acceptance/s4-novnc-frame.spec.jswith a latency/interaction smoke assertion.Rejected / deferred
/dev/dri).Links
docs/brds/uberos-workspace-enhancements-brd.md(Theme E).copilot-tracking/research/2026-07-19/gazebo-vnc-1s-lag-attack-path-research.md