Skip to content

Commit 5b90f97

Browse files
[inference providers] add some more info in the "register as provider" docs (#2026)
* add some more inofo in the register as provider docs * Update docs/inference-providers/register-as-a-provider.md Co-authored-by: vb <[email protected]> * Update docs/inference-providers/register-as-a-provider.md Co-authored-by: vb <[email protected]> --------- Co-authored-by: vb <[email protected]>
1 parent 1ed7bef commit 5b90f97

File tree

1 file changed

+36
-0
lines changed

1 file changed

+36
-0
lines changed

docs/inference-providers/register-as-a-provider.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ Congratulations! You now have a JS implementation to successfully make inference
124124
First step is to use the Model Mapping API to register which HF models are supported.
125125

126126
> [!TIP]
127+
> The completion of step 1. and 2. are pre-requisites for this step.
127128
> To proceed with this step, we have to enable your account server-side. Make sure you have an organization on the Hub for your company, and upgrade it to a [Team or Enterprise plan](https://huggingface.co/pricing).
128129
129130
### Register a mapping item
@@ -366,6 +367,8 @@ Content-Type: application/json
366367
}
367368
```
368369

370+
This API endpoint will be requested by our system every minute, in batches of up to 10,000 requests.
371+
369372
### Price Unit
370373

371374
We require the price to be a **non-negative integer** number of **nano-USDs** (10^-9 USD).
@@ -393,6 +396,39 @@ Inference-Id: unique-id-00131
393396
[response body]
394397
```
395398

399+
### Exposing pricing through OpenAI /models routes
400+
401+
If your API is OpenAI-compatible, we expect that you expose LLM pricing information and context length through the [`/v1/models` endpoint](https://platform.openai.com/docs/api-reference/models/list).
402+
403+
This powers our [provider comparison table](https://huggingface.co/inference/models) and other provider selection features like `:cheapest` (which selects the cheapest provider for a model).
404+
405+
406+
<div class="flex justify-center">
407+
<picture>
408+
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/provider-comparison-table.png">
409+
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers/provider-comparison-table-dark.png">
410+
</picture>
411+
</div>
412+
413+
The format we expect is as follows:
414+
415+
```json
416+
{
417+
{
418+
"id": "model-id-0",
419+
"object": "model",
420+
"created": 1686935002,
421+
"owned_by": "organization-owner",
422+
/// [...] other fields
423+
"pricing": {
424+
"input": 0.2, /// Price in US dollars per million input tokens
425+
"output": 2, /// Price in US dollars per million output tokens
426+
},
427+
"context_length": 200000, /// Supported context length in tokens
428+
},
429+
}
430+
```
431+
396432
## 5. Python client integration
397433

398434
> [!TIP]

0 commit comments

Comments
 (0)