-
Notifications
You must be signed in to change notification settings - Fork 926
[Inference Providers] Featherless release blogpost #2883
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 15 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
dd4d876
blogpost: featherless as a provider
SBrandeis c848754
Update inference-providers-feeatherless.md
SBrandeis 9fef7b7
Update inference-providers-feeatherless.md
SBrandeis f87080d
Update inference-providers-feeatherless.md
SBrandeis 6e8fa38
Changes from code review
SBrandeis 6947c4b
add Lucain as author
SBrandeis 2dba9bb
add to _blog.yml
SBrandeis 36d6361
add inference tag
SBrandeis 30d5612
banner / thumbnail
SBrandeis 23697cb
fix: _blog + spelling
SBrandeis a876fab
change thumbnail
SBrandeis 832e0db
vb' wording
SBrandeis 41cef7e
Apply suggestions from code review
Vaibhavs10 f10a152
suggestions from wauplin
SBrandeis 98f52d6
Apply suggestions from code review
SBrandeis 21ad222
New inference-providers tag
SBrandeis 47dfea8
Update inference-providers-featherless.md
Vaibhavs10 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| --- | ||
| title: "Featherless AI on Hugging Face Inference Providers 🔥" | ||
| thumbnail: /blog/assets/inference-providers/welcome-featherless.jpg | ||
| authors: | ||
| - user: wxgeorge | ||
| guest: true | ||
| org: featherless-ai | ||
| - user: pohnean-recursal | ||
| guest: true | ||
| org: featherless-ai | ||
| - user: picocreator | ||
| guest: true | ||
| org: featherless-ai | ||
| - user: celinah | ||
| - user: Wauplin | ||
| - user: sbrandeis | ||
| --- | ||
|
|
||
|  | ||
|
|
||
| # Featherless AI on Hugging Face Inference Providers 🔥 | ||
|
|
||
| We're thrilled to share that **Featherless AI** is now a supported Inference Provider on the Hugging Face Hub! | ||
| Featherless AI joins our growing ecosystem, enhancing the breadth and capabilities of serverless inference directly on the Hub’s model pages. Inference Providers are also seamlessly integrated into our client SDKs (for both JS and Python), making it super easy to use a wide variety of models with your preferred providers. | ||
|
|
||
| [Featherless AI](https://featherless.ai) supports a wide variety of text and conversational models, including the latest open-source models from DeepSeek, Meta, Google, Qwen, and much more. | ||
|
|
||
| Featherless AI is a serverless AI inference provider with unique model loading and GPU orchestration abilities that makes an exceptionally large catalog of models available for users. Providers often offer either a low cost of access to a limited set of models, or an unlimited range of models with users managing servers and the associated costs of operation. Featherless provides the best of both worlds offering unmatched model range and variety but with serverless pricing. Find the full list of supported models on the [models page](https://huggingface.co/models?inference_provider=featherless-ai&sort=trending). | ||
|
|
||
| We're super excited to see what you'll build with this new provider! | ||
|
|
||
| Read more about how to use Featherless as an Inference Provider in its dedicated [documentation page](https://huggingface.co/docs/inference-providers/providers/featherless-ai). | ||
|
|
||
| ## How it works | ||
|
|
||
| ### In the website UI | ||
|
|
||
|
|
||
| 1. In your user account settings, you are able to: | ||
| - Set your own API keys for the providers you’ve signed up with. If no custom key is set, your requests will be routed through HF. Learn more about request types in the [docs](https://huggingface.co/docs/inference-providers/en/pricing#routed-requests-vs-direct-calls). | ||
| - Order providers by preference. This applies to the widget and code snippets in the model pages. | ||
|
|
||
| <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/user-settings-updated.png" alt="Inference Providers"/> | ||
|
|
||
|
|
||
| 2. As [mentioned](https://huggingface.co/docs/inference-providers/en/pricing), there are two modes when calling Inference Providers: | ||
| - Custom key (calls go directly to the inference provider, using your own API key of the corresponding inference provider) | ||
| - Routed by HF (in that case, you don't need a token from the provider, and the charges are applied directly to your HF account rather than the provider's account) | ||
|
|
||
|
|
||
| <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/explainer.png" alt="Inference Providers"/> | ||
|
|
||
|
|
||
| 3. Model pages showcase third-party inference providers (the ones that are compatible with the current model, sorted by user preference) | ||
|
|
||
| <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/model-widget-updated.png" alt="Inference Providers"/> | ||
|
|
||
|
|
||
| ### From the client SDKs | ||
|
|
||
| #### from Python, using huggingface_hub | ||
|
|
||
| The following example shows how to use DeepSeek-R1 using Featherless AI as the inference provider. You can use a [Hugging Face token](https://huggingface.co/settings/tokens) for automatic routing through Hugging Face, or your own Featherless AI API key if you have one. | ||
|
|
||
| Install `huggingface_hub` from source (see [instructions](https://huggingface.co/docs/huggingface_hub/installation#install-from-source)). Official support will be released soon in version v0.33.0. | ||
Vaibhavs10 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```python | ||
| import os | ||
| from huggingface_hub import InferenceClient | ||
|
|
||
| client = InferenceClient( | ||
| provider="featherless-ai", | ||
| api_key=os.environ["HF_TOKEN"] | ||
| ) | ||
|
|
||
| messages = [ | ||
| { | ||
| "role": "user", | ||
| "content": "What is the capital of France?" | ||
| } | ||
| ] | ||
|
|
||
| completion = client.chat.completions.create( | ||
| model="deepseek-ai/DeepSeek-R1-0528", | ||
| messages=messages, | ||
| ) | ||
|
|
||
| print(completion.choices[0].message) | ||
| ``` | ||
|
|
||
| #### from JS using @huggingface/inference | ||
|
|
||
| ```js | ||
| import { InferenceClient } from "@huggingface/inference"; | ||
|
|
||
| const client = new InferenceClient(process.env.HF_TOKEN); | ||
|
|
||
| const chatCompletion = await client.chatCompletion({ | ||
| model: "deepseek-ai/DeepSeek-R1-0528", | ||
| messages: [ | ||
| { | ||
| role: "user", | ||
| content: "What is the capital of France?" | ||
| } | ||
| ], | ||
| provider: "featherless-ai", | ||
| }); | ||
|
|
||
| console.log(chatCompletion.choices[0].message); | ||
| ``` | ||
|
|
||
| ## Billing | ||
|
|
||
| For direct requests, i.e. when you use the key from an inference provider, you are billed by the corresponding provider. For instance, if you use a Featherless AI API key you're billed on your Featherless AI account. | ||
|
|
||
| For routed requests, i.e. when you authenticate via the Hugging Face Hub, you'll only pay the standard provider API rates. There's no additional markup from us, we just pass through the provider costs directly. (In the future, we may establish revenue-sharing agreements with our provider partners.) | ||
|
|
||
| **Important Note** ‼️ PRO users get $2 worth of Inference credits every month. You can use them across providers. 🔥 | ||
|
|
||
| Subscribe to the [Hugging Face PRO plan](https://hf.co/subscribe/pro) to get access to Inference credits, ZeroGPU, Spaces Dev Mode, 20x higher limits, and more. | ||
|
|
||
| We also provide free inference with a small quota for our signed-in free users, but please upgrade to PRO if you can! | ||
|
|
||
| ## Feedback and next steps | ||
|
|
||
| We would love to get your feedback! Share your thoughts and/or comments here: https://huggingface.co/spaces/huggingface/HuggingDiscussions/discussions/49 | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(maybe) shorter