A scalable, agentic-first, and HuggingFace-native RL framework for research (9k lines).
Educational distributed training and inference library for heterogeneous local hardware: Mac minis, Raspberry Pis and GPUs over plain Python sockets. FSDP, GRPO, parameter-server, expert and data parallelism.
Profile-guided layout optimization for llama.cpp GGUF MoE models — BOLT/PGO for LLM weights. Trace expert routing, rewrite the file so co-activated experts sit together, prefetch as merged reads.
Run large LLMs on a Mac cluster for massive speed improvements.
Experimental recovery coordination for expert-parallel inference groups in llm-d.
Low-cost AMD/ROCm stationary-expert fabric for configurable trillion-parameter sparse MoE inference, with packed-INT4 HIP kernels, activation-only RoCE transport, hardware/BOM design, release gates, and reproducible roofline tooling.
Sovereign Sub-Millisecond Speculative Expert Dispatcher, Waterfill Affinity Load-Balancer & All-to-All Overlap Engine for DeepSeek R1 & NVIDIA Blackwell
CuTe MoE Communication and Overlap Kernels
Completed negative-result study of request-level route-aware scheduling for native diffusion MoE inference
Correctness-first PyTorch/CUDA lab for FlashAttention, DeepSeek MLA/MoE, expert parallelism, reproducible benchmarks, and HPC-for-AI notes.
Simulates Mixture-of-Experts routing strategies (TopK, NoisyTopK, ExpertChoice, Hash) across 432 configurations. Key findings: ExpertChoice achieves best practical balance (imbalance=1.40 vs TopK=5.52); NoisyTopK reduces drops by 40% on skewed distributions; capacity factor is the strongest control knob.
An inference engine for the token-routing layer of Mixture-of-Experts models
test
Hand-built MoE expert parallelism: routing-skew capture, fused grouped-GEMM Triton kernels, 2-all2all dispatch/combine, overlap ablations — proving by counter-example why DeepEP exists.
Distributed training framework for DeepSeek-V3 (Multi-Head Latent Attention, DeepSeekMoE, auxiliary-loss-free load balancing) with composable DP/FSDP/HSDP/TP/PP/CP/EP parallelism, torchtitan-inspired.
Fork of deepseek-ai/DeepEP; hybrid EP build compatibility work (upstream PR #699).
To associate your repository with the expert-parallelism topic, visit your repo's landing page and select "manage topics."