Skip to content

Commit 3bfaf48

Browse files
burtenshawSBrandeisWauplin
authored
update HF-Inference to inference providers only (#1809)
* update page to inference providers only * update ToC * Update docs/hub/models-inference.md Co-authored-by: Simon Brandeis <[email protected]> * Update docs/hub/models-inference.md Co-authored-by: Simon Brandeis <[email protected]> * Update docs/hub/models-inference.md Co-authored-by: Lucain <[email protected]> * add datastudio * add openai and image generation * drop intro paragraph * fix tip notes with mdx * use openai client --------- Co-authored-by: Simon Brandeis <[email protected]> Co-authored-by: Lucain <[email protected]>
1 parent 3519e41 commit 3bfaf48

File tree

2 files changed

+130
-17
lines changed

2 files changed

+130
-17
lines changed

docs/hub/_toctree.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@
127127
- local: models-widgets-examples
128128
title: Widget Examples
129129
- local: models-inference
130-
title: Inference API docs
130+
title: Model Inference
131131
- local: models-download-stats
132132
title: Models Download Stats
133133
- local: models-faq

docs/hub/models-inference.md

Lines changed: 129 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,143 @@
11
# Inference Providers
22

3-
Please refer to the [Inference Providers Documentation](https://huggingface.co/docs/inference-providers) for detailed information.
3+
Hugging Face's model pages have pay-as-you-go inference for thousands of models, so you can try them all out right in the browser. Service is powered by Inference Providers and includes a free-tier.
44

5-
## What is HF-Inference API?
5+
Inference Providers give developers streamlined, unified access to hundreds of machine learning models, powered by the best serverless inference partners. 👉 **For complete documentation, visit the [Inference Providers Documentation](https://huggingface.co/docs/inference-providers)**.
66

7-
HF-Inference API is one of the many providers available on the Hugging Face Hub.
8-
It is deployed by Hugging Face ourselves, using text-generation-inference for LLMs for instance. This service used to be called “Inference API (serverless)” prior to Inference Providers.
7+
## Inference Providers on the Hub
98

10-
For more details about the HF-Inference API, check out its [dedicated page](https://huggingface.co/docs/inference-providers/providers/hf-inference).
9+
Inference Providers is deeply integrated with the Hugging Face Hub, and you can use it in a few different ways:
1110

12-
## What technology do you use to power the HF-Inference API?
11+
- **Interactive Widgets** - Test models directly on model pages with interactive widgets that use Inference Providers under the hood. Check out the [DeepSeek-R1-0528 model page](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528) for an example.
12+
- **Inference Playground** - Easily test and compare chat completion models with your prompts. Check out the [Inference Playground](https://huggingface.co/playground) to get started.
13+
- **Search** - Filter models by inference provider on the [models page](https://huggingface.co/models?inference_provider=all) to find models available through specific providers.
14+
- **Data Studio** - Use AI to explore datasets on the Hub. Check out [Data Studio](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts/viewer?views%5B%5D=train) on your favorite dataset.
1315

14-
The HF-Inference API is powered by [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index) under the hood.
16+
## Build with Inference Providers
1517

16-
## Why don't I see an inference widget, or why can't I use the API?
18+
You can integrate Inference Providers into your own applications using our SDKs or HTTP clients. Here's a quick start with Python and JavaScript, for more details, check out the [Inference Providers Documentation](https://huggingface.co/docs/inference-providers).
1719

18-
For some tasks, there might not be support by any Inference Provider, and hence, there is no widget.
20+
<hfoptions id="inference-providers-quick-start">
1921

20-
## How can I see my usage?
22+
<hfoption id="python">
2123

22-
To check usage across all providers, check out your [billing page](https://huggingface.co/settings/billing).
24+
You can use our Python SDK to interact with Inference Providers.
2325

24-
To check your HF-Inference usage specifically, check out the [Inference Dashboard](https://ui.endpoints.huggingface.co/endpoints). The dashboard shows both your serverless and dedicated endpoints usage.
26+
```python
27+
from huggingface_hub import InferenceClient
2528

26-
## Is there programmatic access to Inference Providers?
29+
import os
2730

28-
Yes! We provide client wrappers in both JS and Python:
29-
- [JS (`@huggingface/inference`)](https://huggingface.co/docs/huggingface.js/inference/classes/InferenceClient)
30-
- [Python (`huggingface_hub`)](https://huggingface.co/docs/huggingface_hub/guides/inference)
31+
client = InferenceClient(
32+
api_key=os.environ["HF_TOKEN"],
33+
provider="auto", # Automatically selects best provider
34+
)
35+
36+
# Chat completion
37+
completion = client.chat.completions.create(
38+
model="deepseek-ai/DeepSeek-V3-0324",
39+
messages=[{"role": "user", "content": "A story about hiking in the mountains"}]
40+
)
41+
42+
# Image generation
43+
image = client.text_to_image(
44+
prompt="A serene lake surrounded by mountains at sunset, photorealistic style",
45+
model="black-forest-labs/FLUX.1-dev"
46+
)
47+
48+
```
49+
50+
Or, you can just use the OpenAI API compatible client.
51+
52+
```python
53+
import os
54+
from openai import OpenAI
55+
56+
client = OpenAI(
57+
base_url="https://router.huggingface.co/v1",
58+
api_key=os.environ["HF_TOKEN"],
59+
)
60+
61+
completion = client.chat.completions.create(
62+
model="deepseek-ai/DeepSeek-V3-0324",
63+
messages=[
64+
{
65+
"role": "user",
66+
"content": "A story about hiking in the mountains"
67+
}
68+
],
69+
)
70+
```
71+
72+
<Tip warning={true}>
73+
74+
The OpenAI API compatible client is not supported for image generation.
75+
76+
</Tip>
77+
78+
</hfoption>
79+
80+
<hfoption id="javascript">
81+
82+
You can use our JavaScript SDK to interact with Inference Providers.
83+
84+
```javascript
85+
import { InferenceClient } from "@huggingface/inference";
86+
87+
const client = new InferenceClient(process.env.HF_TOKEN);
88+
89+
const chatCompletion = await client.chatCompletion({
90+
provider: "auto", // Automatically selects best provider
91+
model: "deepseek-ai/DeepSeek-V3-0324",
92+
messages: [{ role: "user", content: "Hello!" }]
93+
});
94+
95+
const imageBlob = await client.textToImage({
96+
model: "black-forest-labs/FLUX.1-dev",
97+
inputs:
98+
"A serene lake surrounded by mountains at sunset, photorealistic style",
99+
});
100+
```
101+
102+
Or, you can just use the OpenAI API compatible client.
103+
104+
```javascript
105+
import { OpenAI } from "openai";
106+
107+
const client = new OpenAI({
108+
baseURL: "https://router.huggingface.co/v1",
109+
apiKey: process.env.HF_TOKEN,
110+
});
111+
112+
const completion = await client.chat.completions.create({
113+
model: "meta-llama/Llama-3.1-8B-Instruct",
114+
messages: [{ role: "user", content: "A story about hiking in the mountains" }],
115+
});
116+
117+
```
118+
119+
<Tip warning={true}>
120+
121+
The OpenAI API compatible client is not supported for image generation.
122+
123+
</Tip>
124+
125+
</hfoption>
126+
127+
</hfoptions>
128+
129+
You'll need a Hugging Face token with inference permissions. Create one at [Settings > Tokens](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained).
130+
131+
### How Inference Providers works
132+
133+
To dive deeper into Inference Providers, check out the [Inference Providers Documentation](https://huggingface.co/docs/inference-providers). Here are some key resources:
134+
135+
- **[Quick Start](https://huggingface.co/docs/inference-providers)**
136+
- **[Pricing & Billing Guide](https://huggingface.co/docs/inference-providers/pricing)**
137+
- **[Hub Integration Details](https://huggingface.co/docs/inference-providers/hub-integration)**
138+
139+
### What was the HF-Inference API?
140+
141+
HF-Inference API is one of the providers available through Inference Providers. It was previously called "Inference API (serverless)" and is powered by [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index) under the hood.
142+
143+
For more details about the HF-Inference provider specifically, check out its [dedicated page](https://huggingface.co/docs/inference-providers/providers/hf-inference).

0 commit comments

Comments
 (0)