Add OVHcloud AI Endpoints as an Inference Provder #3541

eliasto · 2025-11-10T15:10:14Z

Hi team!

You will find this PR to add OVhcloud AI Endpoints as an Inference Provider, following the PR from HuggingFace.js.

Thank you for your review! 😄

HuggingFaceDocBuilderDev · 2025-11-12T11:27:34Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Wauplin

Hey @eliasto , thanks for the PR! I've made a first pass, please let us know when it's not in draft anymore or if you have any questions :)

src/huggingface_hub/inference/_providers/__init__.py

Wauplin · 2025-11-12T11:29:03Z

docs/source/en/guides/inference.md

+| --------------------------------------------------- | ----------------- | -------- | -------- | ------ | ------ | -------------- | ------------ | ---- | ------------ | ---------- | ---------------- | --------- | ------ | -------- | ---------- | --------- | --------- | --------- | -------- | --------- | ---- |
+| [`~InferenceClient.audio_classification`]           | ❌                 | ❌        | ❌        | ❌      | ❌      | ❌              | ❌            | ❌    | ✅            | ❌          | ❌                | ❌         | ❌         | ❌        | ❌         | ❌        | ❌      | ❌          | ❌         | ❌         | ❌   |
+| [`~InferenceClient.audio_to_audio`]                 | ❌                 | ❌        | ❌        | ❌      | ❌      | ❌              | ❌            | ❌    | ✅            | ❌          | ❌                | ❌         | ❌         | ❌        | ❌         | ❌        | ❌      | ❌          | ❌         | ❌         | ❌   |
+| [`~InferenceClient.automatic_speech_recognition`]   | ❌                 | ❌        | ❌        | ❌      | ✅      | ❌              | ❌            | ❌    | ✅            | ❌          | ❌                | ❌         | ❌         | ✅        | ❌         | ❌        | ❌      | ❌          | ❌         | ❌         | ❌   |


table to be updated accordingly to keep only conversational and text-generation

Wauplin · 2025-11-12T13:02:15Z

src/huggingface_hub/inference/_providers/ovhcloud.py

+    def _prepare_payload_as_dict(
+        self, messages: Any, parameters: dict, provider_mapping_info: InferenceProviderMapping
+    ) -> Optional[dict]:
+        return super()._prepare_payload_as_dict(messages, parameters, provider_mapping_info)


I think it's best to inherit from the base conversational task class instead of defining a base OVH cloud class. You can check an example for featherless-ai for instance which also defines a task provider for conversational and text-generation.

Wauplin · 2025-11-12T13:02:50Z

src/huggingface_hub/inference/_providers/ovhcloud.py

+    def __init__(self, task: str):
+        super().__init__(provider=_PROVIDER, base_url=_BASE_URL, task=task)
+
+    def _prepare_route(self, mapped_model: str, api_key: str) -> str:


better to have this method defined "per task" rather than in the base one using if statements

eliasto · 2025-11-12T15:03:12Z

Hi @Wauplin, thank you for your complete review! I just pushed the new changes, let me know if it's correct for you. 😄

Wauplin

Will have to test that properly later but sharing early feedback for you. Thanks for iterating, it almost looks good to me :)

Wauplin · 2025-11-12T16:12:11Z

src/huggingface_hub/inference/_providers/ovhcloud.py

+_BASE_URL = "https://oai.endpoints.kepler.ai.cloud.ovh.net"
+
+
+class OVHcloudAIEndpointsConversationalTask(BaseConversationalTask):


Suggested change

class OVHcloudAIEndpointsConversationalTask(BaseConversationalTask):

class OVHCloudConversationalTask(BaseConversationalTask):

(nit) could we simplify the name? ovhcloud will be the provider name that users will see

You right, I edited. Thank you for your help again! 😄

Wauplin · 2025-11-12T16:12:43Z

src/huggingface_hub/inference/_providers/ovhcloud.py

+    def _prepare_route(self, mapped_model: str, api_key: str) -> str:
+        return "/v1/chat/completions"


no need for that one (already the default route for conversational)

Suggested change

def _prepare_route(self, mapped_model: str, api_key: str) -> str:

return "/v1/chat/completions"

Wauplin · 2025-11-12T16:14:07Z

src/huggingface_hub/inference/_providers/ovhcloud.py

+    def _prepare_route(self, mapped_model: str, api_key: str) -> str:
+        return "/v1/chat/completions"


are you sure about that one? usually /v1/chat/completions is the route for conversational endpoint. text-generation is the task for raw text generation i.e. without server-side chat template rendering

Oh you right! I did a bad copy-paste. Thank you for noticing it! I edited. 😄

# Conflicts: # docs/source/en/guides/inference.md

eliasto · 2025-11-12T21:03:49Z

Thanks again for your wonderful help @Wauplin. Do you think I should make this PR ready when the billing integration is done on your side, or we do not need to wait for it? 😄

Wauplin · 2025-11-13T14:18:38Z

Do you think I should make this PR ready when the billing integration is done on your side, or we do not need to wait for it? 😄

Billing is an orthogonal problem so it's fine to move forward on this PR. You'll still need billing for a proper go-live on the platform but just for the Python SDK it's fine. For now only OVH and HF users will be able to test it.

Wauplin · 2025-11-13T14:19:19Z

I've given a try to the implementation with

from huggingface_hub import InferenceClient

completion = InferenceClient(provider="ovhcloud").chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(completion.choices[0].message)

and I can confirm it works as expected!

eliasto · 2025-11-13T14:21:09Z

@Wauplin Sounds great! Perfect! I changed the PR to ready. 😄

Wauplin · 2025-11-13T14:22:50Z

src/huggingface_hub/inference/_providers/__init__.py

    },
+    "ovhcloud": {
+        "conversational": OVHcloudConversationalTask(),
+        "text-generation": OVHcloudTextGenerationTask(),


In practice, do you have an example of text-generation-only model that is deployed on OVH-side?

Asking because most (if not all) users are interested in conversational models, i.e. on the Chat Completion and Responses APIs. The text-generation task is only useful for models generating text but not using a chat template e.g. gpt2-like models. It is not possible to register a model both as "conversational" and as "text-generation".

Looking at https://huggingface.co/api/partners/ovhcloud/models, it looks like only conversational models have been registered. If that's the plan, I would suggest to get rid of the OVHcloudTextGenerationTask class entirely. Makes everything cleaner and more aligned with most other providers.

You right, we only have conversational models. I removed the text generation capability! 😄

Wauplin

All good! Thanks for the iterations on this :)

Wauplin · 2025-11-14T10:22:42Z

@bot /style

github-actions · 2025-11-14T10:23:00Z

Style bot fixed some files and pushed the changes.

Wauplin · 2025-11-14T11:29:38Z

Failing CI is unrelated. Let's merge!

* Add OVHcloud AI Endpoints provider * Only add text-generation and conversational task from feedback * Edit name of class and text-generation * Remove text_generation capability * Apply style fixes --------- Co-authored-by: Lucain <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

eliasto marked this pull request as draft November 10, 2025 15:10

eliasto mentioned this pull request Nov 10, 2025

Add OVHcloud AI Endpoints documentation huggingface/hub-docs#2044

Merged

Wauplin reviewed Nov 12, 2025

View reviewed changes

eliasto force-pushed the add-ovhcloud-ai-endpoints-provider branch from 249da97 to 01c7439 Compare November 12, 2025 15:04

Wauplin reviewed Nov 12, 2025

View reviewed changes

eliasto added 3 commits November 12, 2025 16:02

Add OVHcloud AI Endpoints provider

3490a10

# Conflicts: # docs/source/en/guides/inference.md

Only add text-generation and conversational task from feedback

b9c7c08

Edit name of class and text-generation

9ff4047

eliasto force-pushed the add-ovhcloud-ai-endpoints-provider branch from 61eb19b to 9ff4047 Compare November 12, 2025 21:03

eliasto marked this pull request as ready for review November 13, 2025 14:20

Wauplin reviewed Nov 13, 2025

View reviewed changes

eliasto and others added 2 commits November 13, 2025 12:24

Remove text_generation capability

f0d2faf

Merge branch 'main' into add-ovhcloud-ai-endpoints-provider

2067f01

Wauplin approved these changes Nov 14, 2025

View reviewed changes

Apply style fixes

4777b1d

Wauplin merged commit 45e1474 into huggingface:main Nov 14, 2025
14 of 18 checks passed

		_BASE_URL = "https://oai.endpoints.kepler.ai.cloud.ovh.net"


		class OVHcloudAIEndpointsConversationalTask(BaseConversationalTask):

	class OVHcloudAIEndpointsConversationalTask(BaseConversationalTask):
	class OVHCloudConversationalTask(BaseConversationalTask):

		def _prepare_route(self, mapped_model: str, api_key: str) -> str:
		return "/v1/chat/completions"

Add OVHcloud AI Endpoints as an Inference Provder #3541

Add OVHcloud AI Endpoints as an Inference Provder #3541

Uh oh!

Conversation

eliasto commented Nov 10, 2025

Uh oh!

HuggingFaceDocBuilderDev commented Nov 12, 2025

Uh oh!

Wauplin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eliasto commented Nov 12, 2025

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eliasto commented Nov 12, 2025

Uh oh!

Wauplin commented Nov 13, 2025

Uh oh!

Wauplin commented Nov 13, 2025

Uh oh!

eliasto commented Nov 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Wauplin commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Wauplin commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Wauplin left a comment •

edited

Loading

github-actions bot commented Nov 14, 2025 •

edited

Loading