Add tests for pulling vLLM-compatible models from HuggingFace #76
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The ability to pull vLLM-compatible (safetensors) models from HuggingFace was already implemented but lacked test coverage. This PR adds tests to document and verify the feature.
Changes
cmd/cli/commands/utils_test.go: Added test cases for vLLM-compatible HuggingFace model name normalization
hf.co/meta-llama/Llama-3.1-8B-Instruct→huggingface.co/meta-llama/llama-3.1-8b-instruct:latesthf.co/Qwen/Qwen2.5-3B-Instruct:FP8→huggingface.co/qwen/qwen2.5-3b-instruct:fp8pkg/inference/models/handler_test.go: Added two test functions
TestPullSafetensorsModel: Full pull flow for safetensors models usingNormalizeModelNamefor tag generationTestNormalizeHuggingFaceVLLMModel: Coverage for Llama, Qwen, Mistral, DeepSeek model patternsExample
The existing implementation supports:
hf.co/prefix normalization tohuggingface.co/with lowercaseHF_TOKENauthentication for private modelsWarning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
cloud.google.com/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)dario.cat/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)go.googlesource.com/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)go.opentelemetry.io/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)go.uber.org/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)go.yaml.in/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)gonum.org/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)google.golang.org/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)gopkg.in/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)gotest.tools/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)howett.net/update-job-proxy /update-job-proxy(dns block)/update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet(dns block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.