Skip to content

Commit 8830dc8

Browse files
committed
add js integration section
1 parent 2782e6b commit 8830dc8

File tree

1 file changed

+55
-4
lines changed

1 file changed

+55
-4
lines changed

docs/api-inference/register-as-a-provider.md

Lines changed: 55 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,57 @@ For example, you can find the expected schema for Text to Speech here: [https://
5656

5757
## 2. JS Client Integration
5858

59-
Before proceeding with the next steps, ensure you've implemented the necessary code to integrate with the JS client and thoroughly tested your implementation.
59+
Before proceeding with the next steps, ensure you've implemented the necessary code to integrate with the JS client and thoroughly tested your implementation. Here are the steps to follow:
60+
61+
### 1. Implement the provider helper
62+
63+
Create a new file under packages/inference/src/providers/{provider_name}.ts and copy-paste the following snippet.
64+
```js
65+
66+
import { TaskProviderHelper } from "./providerHelper";
67+
68+
export class MyNewProviderTask extends TaskProviderHelper {
69+
70+
constructor() {
71+
super("your-provider-name", "your-api-base-url", "task-name");
72+
}
73+
74+
override prepareHeaders(params: HeaderParams, binary: boolean): Record<string, string> {
75+
// Override the headers to use for the request.
76+
return super.prepareHeaders(params, binary);
77+
}
78+
79+
makeRoute(params: UrlParams): string {
80+
// Return the route to use for the request. e.g. /v1/chat/completions route is commonly use for chat completion.
81+
raise NotImplementedError("Needs to be implemented")
82+
}
83+
84+
preparePayload(params: BodyParams): Record<string, unknown> {
85+
// Return the payload to use for the request, as a dict.
86+
raise NotImplementedError("Needs to be implemented")
87+
}
88+
89+
getResponse(response: TogetherBase64ImageGeneration, outputType?: "url" | "blob"): string | Promise<Blob> {
90+
// Return the response in the expected format.
91+
raise NotImplementedError("Needs to be implemented")
92+
}
93+
}
94+
```
95+
96+
Implement the methods that require custom handling. Check out the base implementation to check default behavior. If you don't need to override a method, just remove it. You have to define at least `makeRoute`, `preparePayload` and `getResponse`.
97+
98+
If the provider supports multiple tasks that require different implementations, create dedicated subclasses for each task, following the pattern used in the existing providers implementation, e.g. [Together AI provider implementation](https://github.com/huggingface/huggingface.js/blob/main/packages/inference/src/providers/together.ts).
99+
100+
For text-generation and conversational tasks, one can just inherit from BaseTextGenerationTask and BaseConversationalTask respectively (defined in [providerHelper.ts]((https://github.com/huggingface/huggingface.js/blob/main/packages/inference/src/providers/providerHelper.ts))) and override the methods if needed. Examples can be found in [Cerebras](https://github.com/huggingface/huggingface.js/blob/main/packages/inference/src/providers/cerebras.ts) or [Fireworks](https://github.com/huggingface/huggingface.js/blob/main/packages/inference/src/providers/fireworks.ts) provider implementations.
101+
102+
### 2. Register the provider
103+
104+
Go to [packages/inference/src/lib/getProviderHelper.ts](https://github.com/huggingface/huggingface.js//blob/main/packages/inference/src/lib/getProviderHelper.ts) and add your provider to `PROVIDERS`. Please try to respect alphabetical order.
105+
106+
### 3. Add tests
107+
108+
Go to [packages/inference/test/InferenceClient.spec.ts](https://github.com/huggingface/huggingface.js/blob/main/packages/inference/test/InferenceClient.spec.ts) and add new tests for each task supported by your provider.
60109

61-
TODO
62110

63111
## 3. Model Mapping API
64112

@@ -268,7 +316,7 @@ Before adding a new provider to the `huggingface_hub` Python library, make sure
268316
### 1. Implement the provider helper
269317
Create a new file under src/huggingface_hub/inference/_providers/{provider_name}.py and copy-paste the following snippet.
270318

271-
Implement the methods that require custom handling. Check out the base implementation to check default behavior. If you don't need to override a method, just remove it. At least one of _prepare_payload_as_dict or _prepare_payload_as_bytes must be overwritten.
319+
Implement the methods that require custom handling. Check out the base implementation to check default behavior. If you don't need to override a method, just remove it. At least one of `_prepare_payload_as_dict` or `_prepare_payload_as_bytes` must be overwritten.
272320

273321
If the provider supports multiple tasks that require different implementations, create dedicated subclasses for each task, following the pattern shown in fal_ai.py.
274322

@@ -337,6 +385,7 @@ class MyNewProviderTaskProviderHelper(TaskProviderHelper):
337385
- Go to [tests/test_inference_providers.py](https://github.com/huggingface/huggingface_hub/blob/main/tests/test_inference_providers.py) and add static tests for overridden methods.
338386
- Go to [tests/test_inference_client.py](https://github.com/huggingface/huggingface_hub/blob/main/tests/test_inference_client.py) and add VCR tests:
339387

388+
340389
a. Add an entry to `_RECOMMENDED_MODELS_FOR_VCR` at the top of the test module. This contains a mapping task <> test model. model-id must be the HF model id.
341390
```python
342391
_RECOMMENDED_MODELS_FOR_VCR = {
@@ -362,7 +411,7 @@ class MyNewProviderTaskProviderHelper(TaskProviderHelper):
362411
```bash
363412
pytest tests/test_inference_client.py -k <provider>
364413
```
365-
414+
366415
d. Commit the generated VCR cassettes with your PR.
367416

368417

@@ -371,3 +420,5 @@ class MyNewProviderTaskProviderHelper(TaskProviderHelper):
371420
**Question:** By default, in which order do we list providers in the settings page?
372421

373422
**Answer:** The default sort is by total number of requests routed by HF over the last 7 days. This order defines which provider will be used in priority by the widget on the model page (but the user's order takes precedence).
423+
424+
## Get in touch!

0 commit comments

Comments
 (0)