Skip to content

[Feature]: Remove the autotuner invocation from AD's nvfp4_gemm operator #9496

@nzmora-nvidia

Description

@nzmora-nvidia

🚀 The feature, motivation and pitch

The autotuner is used when AD record cuda-graphs, and per-shape tactics are cached.
The operator should not explicitly use the auto-tuner context because that is the responsibility of the AD engine.
TRTLLM's nvfp4_gemm_runner is responsible for auto-tuning and caching a tactic and uses this tactic during inference, but only if the auto-tuner is not inside an auto-tuner context.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Metadata

Metadata

Assignees

Labels

AutoDeploy<NV> AutoDeploy BackendCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.bugSomething isn't workingfeature requestNew feature or request. This includes new model, dtype, functionality supporttriagedIssue has been triaged by maintainers

Type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions