dryrun stats observable - #4519
Conversation
micwill755
left a comment
There was a problem hiding this comment.
Request changes, for these:
-
Docs missing. Checklist says docs were updated; they were not. Add the callback recipe to dryrun docs / observer docs.
-
Hook does not always run. dryrun_stats_display is skipped on some compile_module early returns including num_supported_ops == 0 even with dryrun=True. Coverage sweeps of unsupported models would get no tracker. Call display or emit the tracker on those paths too.
| trackers.append(ctx.args[0]) | ||
|
|
||
| with dryrun_stats_display.observers.pre.add(capture): | ||
| torch_tensorrt.dynamo.compile( |
There was a problem hiding this comment.
dynamo.compile() needs an ExportedProgram, not an nn.Module. Export first with torch.export.export(model, (x,)), then pass that into compile.
Also drop enabled_precisions={torch.float32}, enabled_precisions is deprecated as of TRT 11. Dynamo uses dtypes on the model and input so this FP32 test needs no precision argument.
Description
dryrun=True builds a full DryRunTracker (coverage, unsupported ops, engine count) and only prints it. There is no supported way to read those stats for CI or sweeps without monkeypatching.
Mark dryrun_stats_display with @observable() so callers can register on observers.pre / post and capture the tracker without a new public API.
Fixes # (issue)
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: