-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Closed
Labels
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support
Description
🚀 The feature, motivation and pitch
The HugginFace Hub support revisions (git tags) for model versions.
You can now use the command
# download manually the exact revision
huggingface-cli download YourOrg/YourHuggingFaceModel \
--revision YourRevision \
--local-dir ./YourHuggingFaceModel-YourRevision
# serve the exact revision from local file
trtllm-serve ./YourHuggingFaceModel-YourRevision
I can now serve the model directly from the hugginface but only from the main default branch
trtllm-serve YourOrg/YourHuggingFaceModel
It will be great to add to start serving the exact version directly (very useful for Docker deployment)
trtllm-serve YourOrg/YourHuggingFaceModel --revision YourRevision # add the revision flag
trtllm-serve YourOrg/YourHuggingFaceModel:YourRevision # or Docker style revision specification
See HF docs about model version.
See Olmo model checkpoints versions named after step number
Alternatives
The revision parameter is supported in the llmapi
TensorRT-LLM/tensorrt_llm/llmapi/llm_args.py
Line 1725 in b7308a4
| revision: Optional[str] = Field( |
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support