Skip to content

Enable per-worker iGPU execution and contention-aware learning - #4

Merged
kleedaisuki merged 1 commit into
mainfrom
codex/igpu-worker-concurrency
Sep 27, 2026
Merged

kleedaisuki merged 1 commit into
mainfrom
codex/igpu-worker-concurrency

Conversation

@kleedaisuki

Copy link
Copy Markdown
Owner

Purpose

Remove the artificial pool-wide iGPU task bottleneck. Every eligible content worker now owns CPU SIMD, a CUDA stream, and a private OpenCL queue/kernel/device buffers, with one bounded shared budget and one online model per worker.

Design

  • Share only the compiled OpenCL program; each owner reuses its existing CPU file-I/O buffers and reserves its own device quota.
  • Keep automatic discovery nonblocking, but make forced iGPU and explicit train wait for a peer's one-time probe rather than silently using CPU.
  • Retire only the failing worker's accelerator; preserve CUDA peak/contention metric meanings and add iGPU-specific telemetry.
  • Fit the model with observed CPU/iGPU or CUDA overlap while measuring error against the decision-time prediction; keep exact digest verification and fallback contracts.

Verification

  • Windows MSVC Debug CTest 27/27, iGPU routing repeated 100 times.
  • Physical Windows OpenCL: three concurrent admitted iGPU file tasks, 3 x 32 MiB, zero CPU fallbacks; complete BLAKE3 digests match CPU and CUDA scans.
  • Physical three-backend train: 3 samples each CPU/CUDA/iGPU, 8 prior hits, 3 saved keys. Forced CUDA also reached 3 concurrent tasks and produced identical digests.
  • Two interleaved 5,000-small-file CPU batches differed by ~2% in opposite directions; no established performance win or regression.

Limits

Admitted iGPU file tasks do not prove concurrent hardware kernel execution. CPU and iGPU still contend for shared memory; general whole-scan speedup is not claimed. CI must verify Linux/macOS/Windows and PoCL kernel paths before merge. Details: docs/igpu-worker-concurrency.md.

Replace the pool-wide iGPU file-task lease with worker-owned OpenCL backends while reusing each worker's CPU I/O buffers and dividing host/device quotas across all workers. Preserve CUDA telemetry meanings and add separate iGPU concurrency/setup diagnostics. Fit online models on observed overlap while retaining decision-time predictions for error accounting; explicit training and forced iGPU wait for one-time discovery rather than accepting a busy-probe CPU fallback.

Verified Windows Debug CTest 27/27, 100 repeated iGPU routing tests, physical three-worker iGPU/CUDA/CPU digest equivalence, three-backend train persistence, and paired exploratory CPU small-file runs. Hardware overlap and general speedup are not claimed; cross-platform Actions and PoCL validation remain release gates.
@kleedaisuki
kleedaisuki merged commit 012ddfc into main Sep 27, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant