MoE histc / _grouped_mm

[jloftin-nv]

Description

Default transformers MoE forwarding emits aten.histc and aten._grouped_mm per layer. Neither has a converter, so MoE graphs shatter into roughly 3 * layers + 1 engines despite high reported coverage.

Add converters for histc and dense _grouped_mm so MoE blocks can stay in a single TensorRT engine where shapes allow.

Type of change

Please delete options that are not relevant and/or add your own.

• New feature (non-breaking change which adds functionality)

Checklist:

• [ X] My code follows the style guidelines of this project (You can use the linters)
• [ X] I have performed a self-review of my own code
• [ X] I have commented my code, particularly in hard-to-understand areas and hacks
• [ X] I have made corresponding changes to the documentation
• [ X] I have added tests to verify my fix or my feature
• [ X] New and existing unit tests pass locally with my changes
• [ X] I have added the relevant labels to my PR in so that relevant reviewers are notified

[zewenli98]

Hi @jloftin-nv I didn't read the PR thoroughly yet. just wanted to know if we can use TRT IMoELayer somewhere?

[jloftin-nv]

Hi @jloftin-nv I didn't read the PR thoroughly yet. just wanted to know if we can use TRT IMoELayer somewhere?

I think we can in a follow up PR