Repository navigation

Rebase to Upstream - #31

Closed

gmlwns2000 wants to merge 639 commits into

deepauto/devfrom

feat/rebase-0912

Closed

Rebase to Upstream#31
gmlwns2000 wants to merge 639 commits into
deepauto/devfrom
feat/rebase-0912

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 6, 2025
Commits on Sep 7, 2025
- Optimize moe_sum_reduce_kernel (#9477)
  
  authored
  cb3918a
- [Feature] LMCache Connector Integration (#9741)
  
  Oasis-Git
  and
  xiezhq-hermann
  authored
  9a7ced4
- CUTLASS fp8 blockwise gemm support of sm120 (#9969)
  jianyingzhu
  authored
  dd1e268
- Optimize nvfp4 block scaled gemm kernel when M is small. (#10101)
  HydraQYH
  authored
  85ed8e0
- Fix cuda graph mode in flashinfer attn backend (#10056)
  benbarsdell
  authored
  a12061d
- [HiCache] fix: check clear() method for storage backend (#10096)
  
  stmatengss
  and
  hzh0425
  authored
  41628dc
- add dataset_path for bench_one_batch_server.py (#10113)
  
  miter6
  and
  rj-linhuang
  authored
  111b137
- [Auto Sync] Update parallel_state.py (20250907) (#10126)
  
  authored
  617aa2b
- [Minor] fix lint in main (#10128)
  DarkSharpness
  authored
  0672468
- [1/2] Refactor multi-tokenizer manager (#10074)
  hnyls2002
  authored
  e719bb0
- Fix flashinfer version in sgl-kernel (#10135)
  merrymercy
  authored
  76a2c86
- [DOC]: some minor updates (#10134)
  yyihuang
  authored
  b0fcbb7
Commits on Sep 8, 2025
- [BUG FIX] add fail check when get fail in case wait complete block (#9971)
  
  authored
  33467c0
- [MoE] fix: incorrect weight initialization for cutlass_fused_experts_fp8 (#10144)
  ch-wan
  authored
  5a7e10f
- vlm: enable GLM4.1V server testing & fix video processing (#10095)
  
  JustinTong0323
  and
  byjiang1996
  authored
  f3440ad
- Fix slow fused add RMSNorm (#10141)
  fzyzcjy
  authored
  bc5fc33
- fix the fp8 topk_config.correction_bias is none bug (#10040)
  rainj-me
  authored
  7802586
- Qwen2.5-VL eagle3 infer (#8801)
  Lzhang-hub
  authored
  37d83c6
- Fix run time error in dsv3-fp8 model on mi35x (#10104)
  
  authored
  400d3b9
- Standalone speculative decoding (#10090)
  Qiaolin-Yu
  authored
  8cda5a6
- Add graph runner support with torch compile on CPU (#7843)
  CaoE
  authored
  7577f0e
- move compile threads to an option to avoid OOM on low memory host (#10123)
  rainj-me
  authored
  6049ca2
- [1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel, fixed) (#10108)
  yuhyao
  authored
  ee0b3c5
- [Bugfix] Retract not releasing enough memory when page size > 1 (#9989)
  xiezhq-hermann
  authored
  3b99f23
- Add speculator attention backend switch (#9981)
  cicirori
  authored
  8c5930f
- Fix: (glm4v) Add missing field (#10147)
  JustinTong0323
  authored
  8116804
- [Bugfix] Qwen3MoE aclrtMemcpy failed with NPUGraph (#10013)
  iforgetmyname
  authored
  b67c277
- enable auto-round quantization model (#6226)
  WeiweiZhang1
  authored
  c8295d2
- Revert "enable auto-round quantization model (#6226)" (#10148)
  zhyncs
  authored
  b7d1f17
- enable llama3.1-8B on xpu (#9434)
  huaiyuzh
  authored
  ee21817
- [Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU (#9871)
  
  ssshinigami
  and
  Makcum888e
  authored
  5dd8c64
- update xgrammar 0.1.24 and transformers 4.56.1 (#10155)
  Swipe4057
  authored
  bfd7a18
- [1/N] DP-Refactor: move communicators into `tokenizer_communicator_mixin` (#10028)
  hnyls2002
  authored
  78f1398
- [Hicache]: Add E2E CI For 3FS-KVStore (#10131)
  hzh0425
  authored
  ec99668
- Monkey patch uvicorn multi worker `is_alive` timeout (#10159)
  
  hnyls2002
  and
  LLLL114
  authored
  72f9fc5
- [CI] fix ambiguous argument in testing hybrid attentions. (#10161)
  hnyls2002
  authored
  2c2b19b
- [1/2] Speed up prefill mla attention (#10156)
  fzyzcjy
  authored
  0096798
- [Bug fix] Fix ascend mla in aclgraph (#9925)
  alanhe151220037
  authored
  8085aca
- pref: Add H20 fp8 fused MoE kernel configs for Qwen3 (#10166)
  
  Zhiy-Zhang
  and
  qiufan.zzy
  authored
  91f0fd9
- [fix] Relax white space rules in EBNFComposer (#9595)
  LukasBluebaum
  authored
  9a18aa5
- Revert "[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712)" (#10176)
  zhyncs
  authored
  45b3a6a
- [Bench] feat: mooncake trace integration (#9839)
  
  stmatengss
  and
  XucSh
  authored
  a02071a
- fix: resolve lint issue (#10181)
  zhyncs
  authored
  19d64f2
- fix the cutlass moe tests (#10182)
  rainj-me
  authored
  7a40e4f
Commits on Sep 9, 2025
- gb200: update dockerfile to latest kernel (#9522)
  ishandhanani
  authored
  148022f
- Cleaning codes for speculative attention mode (#10149)
  Fridge003
  authored
  8ad700f
- Revert "feat: add fused moe config for Qwen3-30B-A3B on B200" (#10185)
  rainj-me
  authored
  df5407f
- [Fix] Orphan process in data parallel (#7995)
  Capronir
  authored
  96784a6
- Update link for EAGLE speculative decoding (#10191)
  gerayking
  authored
  ba066ca
- [CPU] Fix phi4-mm prompt issue in bench_serving (#9900)
  blzheng
  authored
  97fff98
- Updated Nvidia Jetson docs (#4422)
  shahizat
  authored
  2fe1773
- [1/N]DP refactor: Improve dp rank scheduling in PD disaggregation mode. (#10169)
  hnyls2002
  authored
  83d55ac
- Support opt model (#10165)
  wenhuipeng
  authored
  16ff3d4
- feat: use sgl-kernel cu129 as default (#10188)
  zhyncs
  authored
  cdc56ef
- [Refactor] Remove Hicache Load & Write threads (#10127)
  
  DarkSharpness
  and
  xiezhq-hermann
  authored
  948b01a
- Explicitly export CMAKE_BUILD_PARALLEL_LEVEL (#10193)
  key4ng
  authored
  718f25a
- [CPU] Add gelu_and_mul kernel in sgl-kernel and add ut (#9300)
  blzheng
  authored
  d1d4074
- feat: support fa cute in sgl-kernel (#10205)
  
  zhyncs
  and
  cicirori
  authored
  94fb4e9
- Refactor fused_add_rmsnorm import logic (#10207)
  ShangmingCai
  authored
  f5f6b3b
- tool-call(dsv3): Fixed a parse problem when there are multiple function definitions in tool_calls (#10209)
  Missmiaom
  authored
  2cd94dd
- [Auto Sync] Update sampling_batch_info.py (20250909) (#10212)
  
  authored
  71133a0
- chore: bump v0.3.9 sgl-kernel (#10208)
  zhyncs
  authored
  f3817cb
- add variable TP Decode > Prefill size support (#9960)
  shaharmor98
  authored
  9ab72f9
- [Fix] KV-cache eviction mismatch across PP ranks in DeepSeek V3/R1 (#10214)
  qhsc
  authored
  71fc7b7
- chore: upgrade v0.3.9 sgl-kernel (#10220)
  zhyncs
  authored
  d3ee709
- Revert the changes on NCCL symmetric memory (#10210)
  
  merrymercy
  and
  zhyncs
  authored
  d352c29
- Revert "Revert the changes on NCCL symmetric memory" (#10238)
  merrymercy
  authored
  4582931
- [HiCache] feat: add mooncake backend extra config (#10213)
  stmatengss
  authored
  8471e5e
- Add mamba kernel (#10234)
  yizhang2077
  authored
  8cbe153
- [Auto Sync] Update io_struct.py (20250909) (#10236)
  
  authored
  bf72b80
Commits on Sep 10, 2025
Commits on Sep 11, 2025
Commits on Sep 12, 2025
- Support HiP Attention
  
  committed
  b44c565
- Improve for review comments
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  7c6f805
- remove rope reuse code
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  9953c31
- update exaone
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  f1e50ab
- add error messages for unsupported models for hip attention
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  0de8355
- Implement abstract methods
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  bf47a4f
- fix eagle draft cuda graph runner
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  51e1379
- support deepseek-v2
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  03a2e21
- giupdate pyproject.toml to include hip-attn
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  69de45f
- fix test flag
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  5bdce3a
- move for loop over batch dim into hip-attention library
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  4123abf
- feat: bump hip-attn
  
  kbumsik
  authored and
  gmlwns2000
  committed
  1494948
- bump hip-attn version
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  aa867f1
- cleanup rebase
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  0a71bf5
- fix for MLA
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  768a1b0
- fix args
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  9fd6dd6
- update tests
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  848edf7
- bump hip-attn version
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  51a00a3
- cleanup after rebase
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  3809fc0
- remove redundant code
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  83853fa
- fix
  gmlwns2000
  committed
  4522988
- fix bug
  gmlwns2000
  committed
  28d8db3
- handling chunk attention in hip
  gmlwns2000
  committed
  efa2ede
- PASSKEY, you should remove before PR
  gmlwns2000
  committed
  6f9406a
- fix
  gmlwns2000
  committed
  8fd77ff
- fix
  gmlwns2000
  committed
  9997ae5
- fix
  gmlwns2000
  committed
  fd898d4
- fix
  gmlwns2000
  committed
  b320ef3
- cleaning
  gmlwns2000
  committed
  1d50cca
- fmt
  gmlwns2000
  committed
  457219b
- support qwen2.5 vL
  gmlwns2000
  committed
  bfb06b7
- cleanup
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  2c8362b
- run auto format
  
  daniel-geon-park
  authored and
  gmlwns2000
  committed
  ec58af8
- fix
  gmlwns2000
  committed
  17dd1c3
- bug fix
  gmlwns2000
  committed
  312a761
- support video input for qwen-vl
  
  mickqian
  authored and
  gmlwns2000
  committed
  26599bc
- rebase
  
  mickqian
  authored and
  gmlwns2000
  committed
  640d32d
- simplify processor
  
  mickqian
  authored and
  gmlwns2000
  committed
  ff7a50c
- fix bug
  gmlwns2000
  committed
  3d381a2
- fix
  gmlwns2000
  committed
  5e8dfbb
- fix bug
  gmlwns2000
  committed
  40a50b6
- fix
  gmlwns2000
  committed
  edb066a
- fix for qwen2 3 moe
  gmlwns2000
  committed
  8ec3b73
- fix
  gmlwns2000
  committed
  e081af7
- fix
  gmlwns2000
  committed
  6ea2d82
- support chunked
  gmlwns2000
  committed
  1ae4b81
- fix
  gmlwns2000
  committed
  6a282a5
- fix
  gmlwns2000
  committed
  a400a48
- fix
  gmlwns2000
  committed
  b0c7ae2
- fix for qwen3
  gmlwns2000
  committed
  95d9d17
- fix
  gmlwns2000
  committed
  2ed92cb
- fix
  gmlwns2000
  committed
  66d4f58
- fix
  gmlwns2000
  committed
  b192c87
- fix
  gmlwns2000
  committed
  819ddde
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  9ebf011
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  d47a72c
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  4737bcd
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  1f8f40d
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  0e43744
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  9d99433
- fix
  
  kbumsik
  authored and
  gmlwns2000
  committed
  a741fdc
- dense decode ...?
  
  kbumsik
  authored and
  gmlwns2000
  committed
  96e6ec0
- fix
  gmlwns2000
  committed
  5821d0a
- fix deepseekv2
  gmlwns2000
  committed
  2462fda
- fix
  gmlwns2000
  committed
  4a94ea3
- temporary fix
  gmlwns2000
  committed
  1335895
- fix bug
  gmlwns2000
  committed
  aff3b33
- fix
  gmlwns2000
  committed
  ac70f0c
- fmt
  gmlwns2000
  committed
  1da303e
- handle mla spec decode
  gmlwns2000
  committed
  674adfe
- fmt
  gmlwns2000
  committed
  6aeb1de
- fix bug
  gmlwns2000
  committed
  00a14fd
- fix
  gmlwns2000
  committed
  c8008a4
- fix radix
  gmlwns2000
  committed
  d179563
- fmt
  gmlwns2000
  committed
  a035721
- fix deepseek bug
  gmlwns2000
  committed
  a3b2662
- fmt
  gmlwns2000
  committed
  590fd97
- support fp8
  gmlwns2000
  committed
  d67aa00
- fix
  gmlwns2000
  committed
  01a27da
- fix
  gmlwns2000
  committed
  247e1a3
- improve warmup
  gmlwns2000
  committed
  8e70c9a
- fix for fp8
  gmlwns2000
  committed
  640c582
- run
  gmlwns2000
  committed
  e2ca229
- add delta
  gmlwns2000
  committed
  5384def
- support fp8e4m3
  gmlwns2000
  committed
  fbf53f0
- fa3 decode is handled in hip-attention lib
  gmlwns2000
  committed
  eb24443
- update
  gmlwns2000
  committed
  c384d50
- add self extend scale
  gmlwns2000
  committed
  63557ef
- fmt
  gmlwns2000
  committed
  94ce5f1
- this is bad idea...
  gmlwns2000
  committed
  52c9387
- fix
  gmlwns2000
  committed
  035082b
- fix
  gmlwns2000
  committed
  e7b1140
- support GLM45
  gmlwns2000
  committed
  0473f32
- fix warmup
  gmlwns2000
  committed
  497c9f3
- handling gpt-oss sk
  gmlwns2000
  committed
  e5cc3bb
- fmt
  gmlwns2000
  committed
  1fca935
- support gpt oss
  gmlwns2000
  committed
  7c7499f
- fix gpt-oss
  gmlwns2000
  committed
  edb5a66
- fix
  gmlwns2000
  committed
  916614c
- add bug message
  gmlwns2000
  committed
  00a2a9f
- fmt
  gmlwns2000
  committed
  2e023a5
- fmt
  gmlwns2000
  committed
  02767c6
- fix
  gmlwns2000
  committed
  deaa3cf
- fmt
  gmlwns2000
  committed
  1ce26a6
- fix
  gmlwns2000
  committed
  5cb816d
- fmt
  gmlwns2000
  committed
  83d8acb
- fix
  gmlwns2000
  committed
  402ea03
- add global var
  gmlwns2000
  committed
  0fa639c
- fix
  gmlwns2000
  committed
  2841661
- suport dual chunk attention
  gmlwns2000
  committed
  57aa4ac
- fix
  gmlwns2000
  committed
  3695b2c
Commits on Sep 25, 2025
- try to support qwen3 next
  gmlwns2000
  committed
  593e888

Commits

Commits on Sep 6, 2025

Commits on Sep 7, 2025

Commits on Sep 8, 2025

Commits on Sep 9, 2025

Commits on Sep 10, 2025

Commits on Sep 11, 2025

Commits on Sep 12, 2025