fix(smollm3): resolve the NoPE schedule from the shared ModelConfig - #1
Open
Moviw wants to merge 1 commit into
Open
Conversation
The engine builders call rope_layer_schedule() on the config object the production build pipeline constructs, which is the shared tensorrt_model_connect.config.ModelConfig, not the family-local copy that defines the method. Both standard and dual-profile builds therefore raise AttributeError before any engine is built. Resolve the schedule from config.raw in a module-level helper and route both builders plus build routing through it. Refs: NVIDIA#1146, NVIDIA#1158 Signed-off-by: Moviw <xvzimo@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Hi @ruiling-smartbear — huge thanks for the SmolLM3 family work; I'm not trying to duplicate it. While helping run the missing engine-build step on a real TensorRT box (RTX 3090, TensorRT 11.2.1.2), I hit a blocker in the builders and this is a small fix for it, offered against your
feat/smollm3branch so it can fold straight into NVIDIA#1158.The engine builders call
rope_layer_schedule()on the config object the production build pipeline hands them, which is the sharedtensorrt_model_connect.config.ModelConfig(fromengine_builder.py/build_cli.py), not the family-local copy where the method is defined. So both builders raiseAttributeError: 'ModelConfig' object has no attribute 'rope_layer_schedule'before any engine is built.What changed
families/smollm3/config.py: moved the schedule resolution into a module-levelrope_layer_schedule(config)helper that readsconfig.raw/config.num_hidden_layersand therefore works on the sharedModelConfigtoo; theModelConfigmethod now delegates to it.standard_decoder_builder.pyanddual_profile_decoder_builder.py: call the module-level helper instead of the family-local method.build_routing.py: call the helper directly instead ofgetattr(config, "rope_layer_schedule", None), so malformed-schedule validation actually runs instead of silently no-oping.No shared infrastructure is touched; all changes stay under
families/smollm3/.Validation
The engine-building failures above are the same six the PR describes as "need a real TensorRT module" — on a TRT box they were not skipped, they were failing with the
AttributeErrorabove, and now pass.Notes
ruiling-smartbear:feat/smollm3(this PR targets your branch, notNVIDIA:main).test_model_e2e[smollm3-3b]parity run still needs the 3B checkpoint + engine build; I did not run it (needs/mnt/storage+ model download), which is unchanged from your remaining gap.Refs: NVIDIA#1146, NVIDIA#1158