Skip to content

[bugfix] Use HF leaf names for existing Miles LoRA targets - #70

Closed
kevintli wants to merge 3 commits into
mainfrom
devin/1790628737-miles-lora-hf-targets
Closed

kevintli wants to merge 3 commits into
mainfrom
devin/1790628737-miles-lora-hf-targets

Conversation

@kevintli

@kevintli kevintli commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes our Qwen Miles LoRA definitions, broken since radixark/miles#3318 (merged 2026-09-23).

Our TARGET_MODULES use Megatron selectors (linear_qkv, linear_proj, linear_fc1, linear_fc2). Before #3318, Miles mapped these to HF names with a static table (linear_fc1 -> gate_proj, up_proj, etc.). Since #3318, Miles matches each selector against both the Megatron and the HF name of every module. The Qwen vision tower's HF modules are named model.visual.blocks.N.mlp.linear_fc1/linear_fc2, so they now match too and end up in the adapter.

The SGLang rollout pool is launched with --lora-target-modules = ROLLOUT_LORA_TARGET_MODULES, which only lists LM modules, so it rejects the adapter with a 400 on the first sample.

Fix: use the HF names in TARGET_MODULES, the same names as in ROLLOUT_LORA_TARGET_MODULES. These names only exist in the LM.

("linear_qkv", "linear_proj", "linear_fc1", "linear_fc2"[, "output_layer"])
-> ("q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"[, "lm_head"])

Applies to 9B 16k/16k_dp2, 9B-Base 2k/16k, 27B 16k/64k/128k/256k.

Other changes:

  • miles_config.py: add the HF names to _ATTN_LEAVES/_MLP_LEAVES/_UNEMBED_LEAVES. These sets aren't passed to Miles. lora_target_flags uses them to derive train_attn/train_mlp/train_unembed, which create_model checks against the request. Without this, HF targets would give (False, False, False).
  • New test: TARGET_MODULES ⊆ ROLLOUT_LORA_TARGET_MODULES for every Miles LoRA definition. It catches lilo-side mismatches (e.g. lm_head added to only one list), not Miles behavior changes.

--exclude-modules model.visual.* doesn't work instead: Miles asserts when an exclusion overlaps an explicit target like linear_fc1.

Link to Devin session: https://modal.devinenterprise.com/sessions/942b72932a7445349a6c123a326c5acd
Open in Devin Desktop: https://modal.devinenterprise.com/desktop/session/942b72932a7445349a6c123a326c5acd?variant=devin
Requested by: @kevintli

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot changed the title Use HF leaf names for existing Miles LoRA targets (fix vision-tower adapters) Use HF leaf names for existing Miles LoRA targets Sep 28, 2026
kevintli and others added 2 commits September 28, 2026 21:06
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@kevintli kevintli changed the title Use HF leaf names for existing Miles LoRA targets [bugfix] Use HF leaf names for existing Miles LoRA targets Sep 28, 2026

@micahtyong micahtyong left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like you'll need to rebase now that #55 is in but this lgtm

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Closing as superseded by #55, which already contains everything this PR changes:

  • The providers/modal/definitions/ files this PR edited are gone. Every LoRA recipe in src/lilo/configs/*.py now uses HF target_modules (q_proj … down_proj[, lm_head]), and none of them uses linear_* or output_layer anymore.
  • miles_config.py on main already adds the HF leaf names to _ATTN/_MLP/_UNEMBED_LEAVES.
  • test_all_packaged_recipes_validate_offline asserts that each recipe's trainer targets are equal to the inference lora_target_modules, which the deployment now derives from those same targets. That covers the TARGET_MODULES ⊆ ROLLOUT_LORA_TARGET_MODULES test from this PR.

The only line not on main is gate_up_proj in _MLP_LEAVES. It isn't needed: the only recipe that uses it, gpt_oss_20b_lora_64k, also targets down_proj, so lora_target_flags already returns (True, True, False).

The investigation (Miles radixark/miles#3318 pulling in model.visual.* via the linear_fc1/fc2 HF names) is still in the PR description for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants