Skip to content

[Feature]: AutoDeploy: align AD LlmArgs more closely with PT LlmArgs #9306

@lucaslie

Description

@lucaslie

🚀 The feature, motivation and pitch

Now that PT LlmArgs have mostly stabilized, let's see if we can more closely align AD LlmArgs with PT LlmArgs:

  1. Deprecate AutoDeployConfig as separate class and just move it all into AD's LlmArgs class
  2. Align any field in AD's LlmArgs if there is a corresponding field in PT
  3. Consider inheriting from PT's LlmArgs instead of BaseLlmArgs

This help ensuring AD can serve as a drop-in replacement for PT backend where needed and simplify integration management (trtllm-serve, trtllm-bench, dynamo, NIM, ...)

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Metadata

Metadata

Assignees

Labels

AutoDeploy<NV> AutoDeploy Backendfeature requestNew feature or request. This includes new model, dtype, functionality support

Type

Projects

Status

Ready

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions