Skip to content

[Engine] Rebase Qwen3.6 bounded low-memory expert cache onto main - #3388

Open
CC-David-CC wants to merge 1 commit into
Niko1221:mainfrom
CC-David-CC:work/qwen36-low-memory-main-20261020
Open

CC-David-CC wants to merge 1 commit into
Niko1221:mainfrom
CC-David-CC:work/qwen36-low-memory-main-20261020

Conversation

@CC-David-CC

@CC-David-CC CC-David-CC commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem and result

Qwen3.6 A3B low-memory expert caching from #2925 was based on older main. This branch rebases its unchanged, opt-in patch onto e88ee39.

Related discussion

#2925, including #2925 (comment).

Approach

Keep --low-memory and --expert-ram-gib for bounded GGUF-backed expert caching and skip unused MTP weights. One rebased commit; two changed files.

Validation

Passed:

  • Clean rebase; range-diff confirms the patch is unchanged; git diff --check.
  • Built strata and strata-q36 on middle-child: GTX 1050 Ti, CUDA 12.9, sm_61, AVX CPU floor.
  • Existing bounded-cache regression passed with AddressSanitizer and UBSan; CLI startup checked.

Failed:

  • No build or cache regression failures observed.

Not run:

  • Model inference, token parity, performance benchmarks, HIP and other backend builds.

Limits

Build and cache checks do not establish inference correctness or performance on this GPU. No new speed or memory measurements are claimed. Follow up in the original discussion before resubmitting the closed proposal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant