Skip to content

Conversation

Copy link

Copilot AI commented Dec 3, 2025

The ability to pull vLLM-compatible (safetensors) models from HuggingFace was already implemented but lacked test coverage. This PR adds tests to document and verify the feature.

Changes

  • cmd/cli/commands/utils_test.go: Added test cases for vLLM-compatible HuggingFace model name normalization

    • hf.co/meta-llama/Llama-3.1-8B-Instructhuggingface.co/meta-llama/llama-3.1-8b-instruct:latest
    • hf.co/Qwen/Qwen2.5-3B-Instruct:FP8huggingface.co/qwen/qwen2.5-3b-instruct:fp8
  • pkg/inference/models/handler_test.go: Added two test functions

    • TestPullSafetensorsModel: Full pull flow for safetensors models using NormalizeModelName for tag generation
    • TestNormalizeHuggingFaceVLLMModel: Coverage for Llama, Qwen, Mistral, DeepSeek model patterns

Example

// HuggingFace vLLM-compatible models are normalized correctly
result := NormalizeModelName("hf.co/meta-llama/Llama-3.1-8B-Instruct")
// Returns: "huggingface.co/meta-llama/llama-3.1-8b-instruct:latest"

The existing implementation supports:

  • hf.co/ prefix normalization to huggingface.co/ with lowercase
  • Safetensors format detection and storage
  • HF_TOKEN authentication for private models
  • Automatic vLLM backend selection for safetensors models

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cloud.google.com
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • dario.cat
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • go.googlesource.com
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • go.opentelemetry.io
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • go.uber.org
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • go.yaml.in
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • gonum.org
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • google.golang.org
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • gopkg.in
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • gotest.tools
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)
  • howett.net
    • Triggering command: /update-job-proxy /update-job-proxy (dns block)
    • Triggering command: /update-job-proxy /update-job-proxy x64/pkg/tool/linux_amd64/vet (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Pull vllm-compatible models from HuggingFace</issue_title>
<issue_description></issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Add vllm-compatible models from HuggingFace Add tests for pulling vLLM-compatible models from HuggingFace Dec 3, 2025
Copilot AI requested a review from ericcurtin December 3, 2025 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pull vllm-compatible models from HuggingFace

2 participants