Issues

• MoE cache and CPU expert refactor

  refactor
  FlashML-org/FreeToken#671 · Filter by author jason-fxz opened on Oct 13, 2026

• MiniCPM-V 4.7 35B-A3B

  Desktopfeature
  FlashML-org/FreeToken#666 · Filter by author Zeda1002 opened on Oct 12, 2026

• bench bw verdict misses in the other direction: auto stays on offload (ratio ~1.9), while hybrid decodes 21% / 55% faster (Qwen3.8-Flash-Next NVFP4, RTX 4090 + dual Xeon)

  FlashML-org/FreeToken#664 · Filter by author oscarlius opened on Oct 12, 2026

• sm75 (Turing) fork: 35B MoE on an 8 GB RTX 2080 — 350/25 t/s, 64K context

  FlashML-org/FreeToken#663 · Filter by author ljmbot opened on Oct 12, 2026

• Requests stall and time out above --max-running-requests 8; GPU stays at ~28% of TDP and only one CPU thread is busy

  FlashML-org/FreeToken#661 · Filter by author yakold opened on Oct 11, 2026

• Support Qwen3.8-Flash-Next-GSQ-RCO GGUF and integrate Strata + KVMem

  Desktopfeature
  FlashML-org/FreeToken#658 · Filter by author yutils opened on Oct 10, 2026

• MXFP4 expert loaders accept the e8m0 NaN scale code (0xFF) and serve NaN blocks

  FlashML-org/FreeToken#655 · Filter by author Avicennasis opened on Oct 9, 2026

• test_gemv_path_matches_the_transcription fails on sm_120 (bf16 rounding tie exceeds the tolerance)

  FlashML-org/FreeToken#649 · Filter by author gdevenyi opened on Oct 8, 2026

• Qwen3.8-Flash-Next: transformers 5.16 drops \p{M} from the pre-tokenizer, so combining-mark text gets different ids

  FlashML-org/FreeToken#648 · Filter by author gdevenyi opened on Oct 8, 2026

• /v1/chat/completions returns 400 for the OpenAI developer role

  FlashML-org/FreeToken#646 · Filter by author gdevenyi opened on Oct 8, 2026

• Streamed Anthropic thinking content_block_start has no signature field

  FlashML-org/FreeToken#645 · Filter by author gdevenyi opened on Oct 8, 2026

• Non-streaming requests keep decoding after the client disconnects

  FlashML-org/FreeToken#644 · Filter by author gdevenyi opened on Oct 8, 2026

• Live cache rebuild grows MoE before shrinking KV/mamba, so a resize that passes the fit check can OOM

  FlashML-org/FreeToken#643 · Filter by author gdevenyi opened on Oct 8, 2026
  1

• --moe-cache-auto asserts "cache budget too small" instead of falling back to prefill overlap off

  FlashML-org/FreeToken#642 · Filter by author gdevenyi opened on Oct 8, 2026

• Measurements: TP=2 over PCIe on consumer GPUs (no NVLink) - the all-reduce costs 26% of SM time in prefill

  FlashML-org/FreeToken#640 · Filter by author alvarorsouza-arch opened on Oct 8, 2026

• Pre-sm70: remaining launch-config fixes on the Qwen3.8-Flash-Next path

  FlashML-org/FreeToken#628 · Filter by author gdevenyi opened on Oct 7, 2026

• Size Triton's autotune flush buffer from the device L2 instead of a fixed 256 MB

  FlashML-org/FreeToken#627 · Filter by author gdevenyi opened on Oct 7, 2026

• CPU MoE: place expert rows per NUMA node and schedule tiles on their node

  FlashML-org/FreeToken#626 · Filter by author gdevenyi opened on Oct 7, 2026

• CPU MoE: AVX2 int8-activation NVFP4 kernel for CPUs without VNNI

  FlashML-org/FreeToken#625 · Filter by author gdevenyi opened on Oct 7, 2026

• Load-time MXFP8 requantization of bf16 dense, hyper-connection and lm_head weights

  FlashML-org/FreeToken#624 · Filter by author gdevenyi opened on Oct 7, 2026

• mxfp8_linear dequantizes the whole weight past the GEMV cap (2.4 GiB transient for a vocab-sized projection)

  FlashML-org/FreeToken#623 · Filter by author gdevenyi opened on Oct 7, 2026

• QSA host KV tier: compact selected pages with a column bitmap and scan (7.5x)

  FlashML-org/FreeToken#622 · Filter by author gdevenyi opened on Oct 7, 2026

• QSA host KV tier: give each layer its own slot map (decode at 231K 6.8 -> 15.6 tok/s)

  FlashML-org/FreeToken#621 · Filter by author gdevenyi opened on Oct 7, 2026

• GLM-5.3-Flash-NVFP4 CUDA OOM during load on RTX 5070 Ti 16 GB (sm_120)

  FlashML-org/FreeToken#611 · Filter by author ogino opened on Oct 6, 2026

• Windows wheels resolve unpinned torch dependency to CPU-only build: 'cannot use CUDA device 0: only 0 device(s) visible'

  FlashML-org/FreeToken#607 · Filter by author pablosilva87 opened on Oct 5, 2026
  1