Releases · huggingface/huggingface_hub

✨ The huggingface_hub library now includes an MCP Client, designed to empower Large Language Models (LLMs) with the ability to interact with external Tools via Model Context Protocol (MCP). This client extends the InfrenceClient and provides a seamless way to connect LLMs to both local and remote tool servers!

pip install -U huggingface_hub[mcp]

In the following example, we use the Qwen/Qwen2.5-72B-Instruct model via the Nebius inference provider. We then add a remote MCP server, in this case, an SSE server which makes the Flux image generation tool available to the LLM:

import os

from huggingface_hub import ChatCompletionInputMessage, ChatCompletionStreamOutput, MCPClient

async def main():
    async with MCPClient(
        provider="nebius",
        model="Qwen/Qwen2.5-72B-Instruct",
        api_key=os.environ["HF_TOKEN"],
    ) as client:
        await client.add_mcp_server(type="sse", url="https://evalstate-flux1-schnell.hf.space/gradio_api/mcp/sse")
        messages = [
            {
                "role": "user",
                "content": "Generate a picture of a cat on the moon",
            }
        ]
        async for chunk in client.process_single_turn_with_tools(messages):
            # Log messages
            if isinstance(chunk, ChatCompletionStreamOutput):
                delta = chunk.choices[0].delta
                if delta.content:
                    print(delta.content, end="")

            # Or tool calls
            elif isinstance(chunk, ChatCompletionInputMessage):
                print(
                    f"\nCalled tool '{chunk.name}'. Result: '{chunk.content if len(chunk.content) < 1000 else chunk.content[:1000] + '...'}'"
                )

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

For even simpler development, we now also offer a higher-level Agent class. These 'Tiny Agents' simplify creating conversational Agents by managing the chat loop and state, essentially acting as a user-friendly wrapper around MCPClient. It's designed to be a simple while loop built right on top of an MCPClient.

You can run these Agents directly from the command line:

> tiny-agents run --help
                                                                                                                                                                                     
 Usage: tiny-agents run [OPTIONS] [PATH] COMMAND [ARGS]...                                                                                                                           
                                                                                                                                                                                     
 Run the Agent in the CLI                                                                                                                                                            
                                                                                                                                                                                     
                                                                                                                                                                                     
╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│   path      [PATH]  Path to a local folder containing an agent.json file or a built-in agent stored in the 'tiny-agents/tiny-agents' Hugging Face dataset                         │
│                     (https://huggingface.co/datasets/tiny-agents/tiny-agents)                                                                                                     │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                                                                                                       │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

You can run these Agents using your own local configs or load them directly from the Hugging Face dataset tiny-agents.

This is an early version of the MCPClient, and community contributions are welcome 🤗

[MCP] Add documentation by @hanouticelina in #3102
[MCP] add support for SSE + HTTP by @Wauplin in #3099
[MCP] Tiny Agents in Python by @hanouticelina in #3098
PoC: InferenceClient is also a MCPClient by @julien-c in #2986

⚡ Inference Providers

Thanks to @diadorer, feature extraction (embeddings) inference is now supported with Nebius provider!

[Inference Providers] Add feature extraction task for Nebius by @diadorer in #3057

We’re thrilled to introduce Nscale as an official inference provider! This expansion strengthens the Hub as the go-to entry point for running inference on open-weight models 🔥

🗿 adding support for Nscale inference provider by @nbarr07 in #3068

We also fixed compatibility issues with structured outputs across providers by ensuring the InferenceClient follows the OpenAI API specs structured output.

[Inference Providers] Fix structured output schema in chat completion by @hanouticelina in #3082

💾 Serialization

We've introduced a new @strict decorator for dataclasses, providing robust validation capabilities to ensure data integrity both at initialization and during assignment. Here is a basic example:

from dataclasses import dataclass
from huggingface_hub.dataclasses import strict, as_validated_field

# Custom validator to ensure a value is positive
def positive_int(value: int):
    if not value > 0:
        raise ValueError(f"Value must be positive, got {value}")


class Config:
    model_type: str
    hidden_size: int = positive_int(default=16)
    vocab_size: int = 32  # Default value

    # Methods named `validate_xxx` are treated as class-wise validators
    def validate_big_enough_vocab(self):
        if self.vocab_size < self.hidden_size:
            raise ValueError(f"vocab_size ({self.vocab_size}) must be greater than hidden_size ({self.hidden_size})")

config = Config(model_type="bert", hidden_size=24)   # Valid
config = Config(model_type="bert", hidden_size=-1)   # Raises StrictDataclassFieldValidationError

# `vocab_size` too small compared to `hidden_size`
config = Config(model_type="bert", hidden_size=32, vocab_size=16)   # Raises StrictDataclassClassValidationError

This feature also includes support for custom validators, class-wise validation logic, handling of additional keyword arguments, and automatic validation based on type hints. Documentation can be found here.

New @strict decorator for dataclass validation by @Wauplin in #2895

This release brings also support for DTensor in _get_unique_id / get_torch_storage_size helpers, allowing transformers to seamlessly use save_pretrained with DTensor.

Feat: support DTensor when saving by @S1ro1 in #3042

✨ HF API

When creating an Endpoint, the default for scale_to_zero_timeout is now None, meaning endpoints will no longer scale to zero by default unless explicitly configured.

Dont set scale to zero as default when creating an Endpoint by @tomaarsen in #3062

We've also introduced experimental helpers to manage OAuth within FastAPI applications, bringing functionality previously used in Gradio to a wider range of frameworks for easier integration.

Add helpers to handle OAuth in a FastAPI app by @Wauplin in #2684

📚 Documentation

We now have much more detailed documentation for Inference! This includes more detailed explanations and examples to clarify that the InferenceClient can also be effectively used with local endpoints (llama.cpp, vllm, MLX..etc).

[Inference] Mention local endpoints inference + remove separate HF Inference API mentions by @hanouticelina in #3085

🛠️ Small fixes and maintenance

😌 QoL improvements

bump hf-xet min version by @hanouticelina in #3078
Add api.endpoint to arguments for _get_upload_mode by @matthewgrossman in #3077
surface 401 unauthorized errors more directly in snapshot_download by @hanouticelina in #3092

🐛 Bug and typo fixes

[HfFileSystem] Fix end-of-file read() by @lhoestq in #3080
[Inference Endpoints] fix inference endpoint creation with custom image by @hanouticelina in #3076
Expand file lock scope to resolve concurrency issues during downloads by @humengyu2012 in #3063
Documentation Issue by @thanosKivertzikidis in #3091
Do not fetch /preupload if already done in upload-large-folder by @Wauplin in #3100

🏗️ internal

[Internal] make hf-xet (again) a required dependency #3103
fix conda by @han...

Contributors

julien-c, matthewgrossman, and 9 other contributors

Assets 2

19 May 09:48

hanouticelina

v0.31.4

2f4ee4b

[v0.31.4]: strict dataclasses, support `DTensor` saving & some bug fixes

This release includes some new features and bug fixes:

New strict decorators for runtime dataclass validation with custom and type-based checks. by @Wauplin in #2895.
Added DTensor support to _get_unique_id / get_torch_storage_size helpers, enabling transformers to use save_pretrained with DTensor. by @S1ro1 in #3042.
Some bug fixes: #3080 & #3076.

Full Changelog: v0.31.2...v0.31.4

Contributors

Wauplin and S1ro1

Assets 2

13 May 09:50

hanouticelina

v0.31.2

2e5dc96

[v0.31.2] Hot-fix: make `hf-xet` optional again and bump the min version of the package

Patch release to make hf-xet optional. More context in #3079 and #3078.

Full Changelog: v0.31.1...v0.31.2

Assets 2

06 May 20:59

hanouticelina

v0.31.0

153f159

[v0.31.0] LoRAs with Inference Providers, `auto` mode for provider selection, embeddings models and more

🧑‍🎨 Introducing LoRAs with fal.ai and Replicate providers

We're introducing blazingly fast LoRA inference powered by
fal.ai and Replicate through Hugging Face Inference Providers! You can use any compatible LoRA available on the Hugging Face Hub and get generations at lightning fast speed ⚡

from huggingface_hub import InferenceClient

client = InferenceClient(provider="fal-ai") # or provider="replicate"

# output is a PIL.Image object
image = client.text_to_image(
    "a boy and a girl looking out of a window with a cat perched on the window sill. There is a bicycle parked in front of them and a plant with flowers to the right side of the image. The wall behind them is visible in the background.",
    model="openfree/flux-chatgpt-ghibli-lora",
)

[Inference Providers] LoRAs with Replicate by @hanouticelina in #3054
[Inference Providers] Support for LoRAs with fal by @hanouticelina in #3005

⚙️ `auto` mode for provider selection

You can now automatically select a provider for a model using auto mode — it will pick the first available provider based on your preferred order set in https://hf.co/settings/inference-providers.

from huggingface_hub import InferenceClient

# will select the first provider available for the model, sorted by your order.
client = InferenceClient(provider="auto") 

completion = client.chat.completions.create(
    model="Qwen/Qwen3-235B-A22B",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
)

print(completion.choices[0].message)

⚠️ Note: This is now the default value for the provider argument. Previously, the default was hf-inference, so this change may be a breaking one if you're not specifying the provider name when initializing InferenceClient or AsyncInferenceClient.

PoC: provider="auto" by @julien-c in #3011

🧠 Embeddings support with Sambanova (feature-extraction)

We added support for feature extraction (embeddings) inference with sambanova provider.

[Inference Providers] sambanova supports feature extraction by @hanouticelina in #3037

⚡ Other Inference features

HF Inference API provider is now fully integrated as an Inference Provider, this means it only supports a predefined list of deployed models, selected based on popularity.
Cold-starting arbitrary models from the Hub is no longer supported — if a model isn't already deployed, it won’t be available via HF Inference API.

Miscellaneous improvements and some bug fixes:

Fix 'sentence-transformers/all-MiniLM-L6-v2' doesn't support task 'feature-extraction' by @Wauplin in #2968
fix text generation by @hanouticelina in #2982
Fix HfInference conversational by @Wauplin in #2985
Fix 'sentence_similarity' on InferenceClient by @tomaarsen in #3004
Update inference types (automated commit) by @HuggingFaceInfra in #3015
update text to speech input by @hanouticelina in #3025
[Inference Providers] fix inference with URL endpoints by @hanouticelina in #3041
Update inference types (automated commit) by @HuggingFaceInfra in #3051

✅ Of course, all of those inference changes are available in the AsyncInferenceClient async equivalent 🤗

🚀 Xet

Thanks to @bpronan's PR, Xet now supports uploading byte arrays:

from huggingface_hub import upload_file

file_content = b"my-file-content"
repo_id = "username/model-name" # `hf-xet` should be installed and Xet should be enabled for this repo

upload_file(
    path_or_fileobj=file_content,
    repo_id=repo_id,
)

Xet Upload with byte array by @bpronan in #3035

Additionally, we’ve added documentation for environment variables used by hf-xet to optimize file download/upload performance — including options for caching (HF_XET_CHUNK_CACHE_SIZE_BYTES), concurrency (HF_XET_NUM_CONCURRENT_RANGE_GETS), high-performance mode (HF_XET_HIGH_PERFORMANCE), and sequential writes (HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY).

Docs for xet env variables by @rajatarya in #3024
Minor xet changes: HF_HUB_DISABLE_XET flag, suppress logger.info by @rajatarya in #3039

Miscellaneous improvements:

Removing workaround for deprecated refresh route headers by @bpronan in #2993

✨ HF API

We added HTTP download support for files larger than 50GB — enabling more reliable handling of large file downloads.

Add HTTP Download support for files > 50GB by @rajatarya in #2991

We also added dynamic batching to upload_large_folder, replacing the fixed 50-files-per-commit rule with an adaptive strategy that adjusts based on commit success and duration — improving performance and reducing the risk of hitting the commits rate limit on large repositories.

Fix dynamic commit size by @maximizemaxwell in #3016

We added support for new arguments when creating or updating Hugging Face Inference Endpoints.

add route payload to deploy Inference Endpoints by @Vaibhavs10 in #3013
Add the 'env' parameter to creating/updating Inference Endpoints by @tomaarsen in #3045

💔 Breaking changes

The default value of the provider argument in InferenceClient and AsyncInferenceClient is now "auto" instead of "hf-inference" (HF Inference API). This means provider selection will now follow your preferred order set in your inference provider settings.
If your code relied on the previous default ("hf-inference"), you may need to update it explicitly to avoid unexpected behavior.
HF Inference API Routing Update: The inference URL path for feature-extraction and sentence-similarity tasks has changed from https://router.huggingface.co/hf-inference/pipeline/{task}/{model}to https://router.huggingface.co/hf-inference/models/{model}/pipeline/{task}.

[inference] Necessary breaking change: nest task-specific route inside of model route by @julien-c in #3044

🛠️ Small fixes and maintenance

😌 QoL improvements

Unlist TPUs from SpaceHardware by @Wauplin in #2973
dev(narugo): disable hf_transfer when custom 'Range' header is assigned by @narugo1992 in #2979
Improve error handling for invalid eval results in model cards by @hanouticelina in #3000
Handle Rate Limits in Pagination with Automatic Retries by @Weyaxi in #2970
Add example for downloading files in subdirectories, related to #3014 by @mixer3d in #3023
Super-micro-tiny-PR to allow for direct copy-paste :) by @fracapuano in #3030
Migrate to logger.warning usage by @emmanuel-ferdman in #3056

🐛 Bug and typo fixes

Retry on transient error in download workflow by @Wauplin in #2976
fix snapshot download behavior in offline mode when downloading to a local dir by @hanouticelina in #3009
fix docstring by @hanouticelina in #3040
fix default CACHE_DIR by @albertcthomas in #3050

🏗️ internal

fix: fix test_get_hf_file_metadata_from_a_lfs_file as since xet migration by @XciD in #2972
A better security-wise style bot GH Action by @hanouticelina in #2914
prepare for next release by @hanouticelina in #2983
Bump hf_xet min version to 1.0.0 + make it required dep on 64 bits by @hanouticelina in #2971
fix permissions for style bot by @hanouticelina in #3012
remove (inference only) VCR tests by @hanouticelina in #3021
remove test by @hanouticelina in #3028

Community contributions

The following contributors have made significant changes to the library over the last release:

@bpronan
- Removing workaround for deprecated refresh route headers (#2993)
- Xet Upload with byte array (#3035)
@tomaarsen
- Fix 'sentence_similarity' on InferenceClient (#3004)
- Add the 'env' parameter to creating/updating Inference Endpoints (#3045)
@Weyaxi
- Handle Rate Limits in Pagination with Automatic Retries (#2970)
@rajatarya
- Add HTTP Download support for files > 50GB (#2991)
- Docs for xet env variables (#3024)
- Minor xet changes: HF_HUB_DISABLE_XET flag, suppress logger.info (#3039)
@Vaibhavs10
- add route payload to deploy Inference Endpoints (#3013)
@maximizemaxwell
- Fix dynamic commit size (#3016)
@emmanuel-ferdman
- Migrate to logger.warning usage (#3056)