You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/inference-providers/register-as-a-provider.md
+12-7Lines changed: 12 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,13 @@ Want to be an Inference Provider on the Hub? Please reach out to us!
6
6
7
7
</Tip>
8
8
9
-
This guide details each of the following steps and provides implementation guidance.
9
+
This guide details the steps for registering as an inference provider on the Hub and provides implementation guidance.
10
10
11
-
1.**Implement standard task APIs** - Follow our task API schemas for compatibility (see [Prerequisites](#1-prerequisites))
12
-
2.**Submit a PR for JS client integration** - Add your provider to [huggingface.js](https://github.com/huggingface/huggingface.js/tree/main/packages/inference) (see [JS Client Integration](#2-js-client-integration))
13
-
3.**Register model mappings** - Use our Model Mapping API to connect your models to Hub models (see [Model Mapping API](#3-model-mapping-api))
14
-
4.**Implement a billing endpoint** - Provide an API for billing (see [Billing](#4-billing))
15
-
5.**Submit a PR for Python client integration** - Add your provider to [huggingface_hub](https://github.com/huggingface/huggingface_hub) (see [Python client integration](#5-python-client-integration))
11
+
1.**Implement standard task APIs** - Follow our task API schemas for compatibility (see [Prerequisites](#1-prerequisites)).
12
+
2.**Submit a PR for JS client integration** - Add your provider to [huggingface.js](https://github.com/huggingface/huggingface.js/tree/main/packages/inference) (see [JS Client Integration](#2-js-client-integration)).
13
+
3.**Register model mappings** - Use our Model Mapping API to link your models to Hub models (see [Model Mapping API](#3-model-mapping-api)).
14
+
4.**Implement a billing endpoint** - Provide an API for billing (see [Billing](#4-billing)).
15
+
5.**Submit a PR for Python client integration** - Add your provider to [huggingface_hub](https://github.com/huggingface/huggingface_hub) (see [Python client integration](#5-python-client-integration)).
16
16
6.**Provide an icon** - Submit an SVG icon for your provider.
17
17
7.**Create documentation** - Add documentation and do some communication on your side.
18
18
8.**Add a documentation page** - Add a provider-specific page in the Hub documentation.
@@ -162,10 +162,12 @@ In the future, we will add support for a new parameter (ping us if it's importan
162
162
}
163
163
```
164
164
#### Authentication
165
+
165
166
You need to be in the _provider_ Hub organization (e.g. https://huggingface.co/togethercomputer
166
167
for TogetherAI) with **Write** permissions to be able to access this endpoint.
167
168
168
169
#### Validation
170
+
169
171
The endpoint validates that:
170
172
-`hfModel` is indeed of `pipeline_tag == task` OR `task` is "conversational" and the model is
171
173
compatible (i.e. the `pipeline_tag` is either "text-generation" or "image-text-to-text" AND the model is tagged as "conversational").
@@ -200,10 +202,12 @@ With the following body (JSON-encoded):
200
202
```http
201
203
GET /api/partners/{provider}/models?status=staging|live
202
204
```
203
-
This gets all mapping items from the DB. For clarity/DX, the output is grouped by task.
205
+
This gets all mapping items from the DB. For clarity, the output is grouped by task.
204
206
205
207
<Tipwarning={true}>
208
+
206
209
This is publicly accessible. It's useful to be transparent by default and it helps debug client SDKs, etc.
210
+
207
211
</Tip>
208
212
209
213
Here is an example of response:
@@ -326,6 +330,7 @@ Before adding a new provider to the `huggingface_hub` Python library, make sure
326
330
</Tip>
327
331
328
332
### Implement the provider helper
333
+
329
334
Create a new file under `src/huggingface_hub/inference/_providers/{provider_name}.py` and copy-paste the following snippet.
330
335
331
336
Implement the methods that require custom handling. Check out the base implementation to check default behavior. If you don't need to override a method, just remove it. At least one of `_prepare_payload_as_dict` or `_prepare_payload_as_bytes` must be overwritten.
0 commit comments