MoE cache and CPU expert refactor #671

Tracks the refactor of the MoE expert cache and the CPU expert path: mixed quant formats per layer, GGUF experts, batch-size > 1 decode and AMD support.

Where it stands

• The GPU slot cache is one LRU pool shared by all MoE layers and assumes one bank layout (one quant_format, one bank schema) for the whole model.
• The engine takes the quant method of the first MoE layer for every layer.
• The CPU expert executor serves one weight format per model.

Work items

Cache

• Slot pools: per-layer pools and pools per bank layout, sized equally or from an expert-use profile. @jason-fxz
• Bank sets that differ per layer, e.g. NVFP4 layers next to BF16 layers left unquantized. Slot pools key a layout by dtype and row shape over one bank schema, so they cover GGUF mixes (same banks, different row widths) but not formats with different banks. @jason-fxz

Mixed quant per layer

• Carry the quant format per layer: the engine reads each layer's quant method, loaders and FTW checkpoints record it, and the CPU executor picks a format per (layer, bank).

CPU expert executor

• Split the executor from the weight formats and the GPU handshake (refactor(cpu-moe): split the executor into weight formats and gpu sync backends #670). @jason-fxz
• Kernel IO for the handshake: a publish kernel writes a step's inputs into the pinned IO block and rings the doorbell, or skips a step without CPU routes; a combine kernel reads the CPU rows back. The coordinator skips tasks without routed experts (refactor(cpu-moe): split the executor into weight formats and gpu sync backends #670). @jason-fxz
• Hybrid decode: split the routes between GPU and CPU inside the hybrid ensure kernel instead of torch ops (perf(moe): split hybrid decode routes inside the ensure kernel #685). @jason-fxz
• Batch size > 1: split the work by expert, so an expert shared by several tokens is read once.

GGUF experts

• CPU: vendor ggml-cpu quants under kernels/gguf/, built once per ISA tier, behind one GgufFormat(ggml_type); every type with a ggml-cpu vec_dot works through ggml's type traits. One generic gguf bank schema (gate_up, down).
• GPU: prefill and hybrid need GPU kernels per GGUF type; today they cover Q4_0, Q8_0 and Q6_K.

AMD

• Validate the HIP handshake on ROCm 10 (native capture) and ROCm 7.14 (explicit graph nodes). [ROCm] CpuMoeExecutor decode is silently wrong under CUDA-graph replay (rel err 1.36, no exception) #350, fix(rocm): make CPU/Hybrid MoE graph replay safe #378, fix(rocm): verify native cpu moe graph handshake on rocm 10 #667.