Skip to content

Commit 27d99b3

Browse files
committed
blogpost: featherless as a provider
1 parent 77fe327 commit 27d99b3

File tree

1 file changed

+129
-0
lines changed

1 file changed

+129
-0
lines changed
Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: "Featherless on Hugging Face Inference Providers 🔥"
3+
thumbnail: /blog/assets/inference-providers-featherless/thumbnail.png
4+
authors:
5+
- user: wxgeorge
6+
guest: true
7+
org: featherless-ai
8+
- user: pohnean-recursal
9+
guest: true
10+
org: featherless-ai
11+
- user: picocreator
12+
guest: true
13+
org: featherless-ai
14+
- user: celinah
15+
- user: sbrandeis
16+
---
17+
18+
19+
![banner image](https://place-hold.it/1680x900)
20+
<!-- ![banner image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/featherless-banner.png) -->
21+
<!-- TODO: add a banner -->
22+
23+
# Featherless on Hugging Face Inference Providers 🔥
24+
25+
We're thrilled to share that **Featherless** is now a supported Inference Provider on HF Hub!
26+
Featherless joins our growing ecosystem, enhancing the breadth and capabilities of serverless inference directly on the Hub’s model pages. Inference Providers are also seamlessly integrated into our client SDKs (for both JS and Python), making it super easy to use a wide variety of models with your preferred providers.
27+
28+
[Featherless](https://featherless.ai) supports a wide variety of text and conversational model, including the latest open-source models from DeepSeek, Meta, Google, Qwen, and much more.
29+
30+
Check out supported models here: [supported models list](https://featherless.ai/models).
31+
32+
We're quite excited to see what you'll build with this new provider!
33+
34+
Read more about Inference Providers in our [documentation](https://huggingface.co/docs/inference-providers).
35+
36+
## How it works
37+
38+
### In the website UI
39+
40+
41+
1. In your user account settings, you are able to:
42+
- Set your own API keys for the providers you’ve signed up with. If no custom key is set, your requests will be routed through HF.
43+
- Order providers by preference. This applies to the widget and code snippets in the model pages.
44+
45+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/user-settings-updated.png" alt="Inference Providers"/>
46+
47+
48+
2. As mentioned, there are two modes when calling Inference Providers:
49+
- Custom key (calls go directly to the inference provider, using your own API key of the corresponding inference provider)
50+
- Routed by HF (in that case, you don't need a token from the provider, and the charges are applied directly to your HF account rather than the provider's account)
51+
52+
53+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/explainer.png" alt="Inference Providers"/>
54+
55+
56+
3. Model pages showcase third-party inference providers (the ones that are compatible with the current model, sorted by user preference)
57+
58+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/model-widget-updated.png" alt="Inference Providers"/>
59+
60+
61+
### From the client SDKs
62+
63+
#### from Python, using huggingface_hub
64+
65+
The following example shows how to use DeepSeek-R1 using Hyperbolic as the inference provider. You can use a [Hugging Face token](https://huggingface.co/settings/tokens) for automatic routing through Hugging Face, or your own Hyperbolic API key if you have one.
66+
67+
Install `huggingface_hub` from source (see [instructions](https://huggingface.co/docs/huggingface_hub/installation#install-from-source)). Official support will be released soon in version v0.29.0.
68+
69+
```python
70+
from huggingface_hub import InferenceClient
71+
72+
client = InferenceClient(
73+
provider="featherless-ai",
74+
api_key="xxxxxxxxxxxxxxxxxxxxxxxx"
75+
)
76+
77+
messages = [
78+
{
79+
"role": "user",
80+
"content": "What is the capital of France?"
81+
}
82+
]
83+
84+
completion = client.chat.completions.create(
85+
model="deepseek-ai/DeepSeek-R1-0528",
86+
messages=messages,
87+
max_tokens=500
88+
)
89+
90+
print(completion.choices[0].message)
91+
```
92+
93+
#### from JS using @huggingface/inference
94+
95+
```js
96+
import { HfInference } from "@huggingface/inference";
97+
98+
const client = new HfInference("xxxxxxxxxxxxxxxxxxxxxxxx");
99+
100+
const chatCompletion = await client.chatCompletion({
101+
model: "deepseek-ai/DeepSeek-R1-0528",
102+
messages: [
103+
{
104+
role: "user",
105+
content: "What is the capital of France?"
106+
}
107+
],
108+
provider: "featherless-ai",
109+
max_tokens: 500
110+
});
111+
112+
console.log(chatCompletion.choices[0].message);
113+
```
114+
115+
## Billing
116+
117+
For direct requests, i.e. when you use the key from an inference provider, you are billed by the corresponding provider. For instance, if you use a Featherless AI API key you're billed on your Featherless AI account.
118+
119+
For routed requests, i.e. when you authenticate via the Hugging Face Hub, you'll only pay the standard provider API rates. There's no additional markup from us, we just pass through the provider costs directly. (In the future, we may establish revenue-sharing agreements with our provider partners.)
120+
121+
**Important Note** ‼️ PRO users get $2 worth of Inference credits every month. You can use them across providers. 🔥
122+
123+
Subscribe to the [Hugging Face PRO plan](https://hf.co/subscribe/pro) to get access to Inference credits, ZeroGPU, Spaces Dev Mode, 20x higher limits, and more.
124+
125+
We also provide free inference with a small quota for our signed-in free users, but please upgrade to PRO if you can!
126+
127+
## Feedback and next steps
128+
129+
We would love to get your feedback! Here’s a Hub discussion you can use: https://huggingface.co/spaces/huggingface/HuggingDiscussions/discussions/49

0 commit comments

Comments
 (0)